Timeline and episode-structured clinical data: pre-processing for Data Mining and analytics

Abstract

Data Mining has been used in the healthcare domain for diagnosis and treatment analysis, resource management and fraud detection. It brings a set of tools and techniques that can be applied to large-scale patient data to discover underlying patterns and provide healthcare professionals an additional source of knowledge for making decisions. The Southampton Breast Cancer Data System (SBCDS) containing some 16,000 timeline-structured records is a visually rich and highly intuitive system for the manual and automated transfer of demographic, pathology and treatment data into an episode-based structure. While expansion of the data mining capability in SBCDS is one of the objectives of our research, real-world patient data is generally incomplete, inconsistent and containing errors. This case study will focus on the data pre-processing stage in order to clean the raw data and prepare the final dataset for use in data mining and analytics. Some initial results are given for sequential patterns mining and classification which highlight the advantages of the approach.