Enhancing data integrity in internet of things-based healthcare applications: A visualization approach for duplicate detection

Emran, Nurul Akmar and Md Isa, Siti NoorBasirah and Harum, Norharyati and Machap, Logenthiran and Nordin, Azlin (2025) Enhancing data integrity in internet of things-based healthcare applications: A visualization approach for duplicate detection. Bulletin of Electrical Engineering and Informatics, 14 (5). pp. 3704-3715. ISSN 2089-3191

Text 0028229102025129152382.pdf Available under License Creative Commons Attribution Share Alike. Download (594kB)

Abstract

This study addresses the critical issue of data duplication in healthcare-related internet of things (IoT) datasets, which can compromise the reliability of analyses and patient outcomes. A Python-based visualization framework using Pandas and Matplotlib was developed to detect and represent duplicate records. The methodology was applied to six cancer-related datasets sourced from Kaggle, ranging from 300 to 55,000 records, encompassing numerical, textual, and categorical data types. The visualization technique provided clear insights into duplication patterns, identifying specific counts such as 7 duplicates in the wearable device dataset, 19 in the thyroid recurrence dataset, and 534 in the synthetic healthcare electronic health record (EHR) dataset. Compared to traditional detection methods, the visualization tool facilitated faster and more intuitive initial data assessment, demonstrating its effectiveness for rapid quality checks in healthcare datasets. However, scalability limitations were observed in larger datasets, where visual clarity declined. These findings highlight the value of visualization as a preliminary data quality assessment tool and suggest future integration with advanced detection algorithms to enhance robustness and scalability.

Item Type:	Article
Uncontrolled Keywords:	Data duplication, Duplicates detection, Healthcare, Internet of things data, Visualization
Divisions:	Faculty of Information and Communication Technology
Depositing User:	Norfaradilla Idayu Ab. Ghafar
Date Deposited:	11 Dec 2025 02:21
Last Modified:	11 Dec 2025 02:21
URI:	http://eprints.utem.edu.my/id/eprint/29164
Statistic Details:	View Download Statistic

Actions (login required)

View Item