An Ensembled Tabnet-Based Model Approach for Diabetes Disease Classification

Abstract

Despite the advancements in machine learning (ML) for classification tasks, accurately classifying diseases on limited-feature medical datasets remains challenging. Traditional ML models struggle with interpretability, necessitating an exploration of novel technique. This research developed and evaluated a novel TabNet-based ensemble model for diabetes classification, rating its performance against Extreme Gradient Boosting (XGBoost), Random Forest and base TabNet models. The study utilized the PIMA Indian Diabetes dataset from a public ML Repository, which contains 768 tuples (8 features and 1 outcome variable). A TabNet-based ensemble model was developed using a weighted averaging strategy. For comparative analysis, baseline models, including XGBoost, Random Forest, and a standalone TabNet model were also implemented and optimized. Model performance was assessed using key metrics: balanced accuracy, precision and recall (class 1), F1 score, and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC). The ensembled TabNet-based model consistently achieved the highest performance metrics: balanced accuracy of 83%, precision of 84%, recall of 89%, F1 score of 84%, and ROC-AUC of 90.4%  compared to XGBoost (accuracy 81% , precision 79% , recall 86%, F1 score 81%, ROC-AUC 88.6%) , Random Forest (accuracy 81%,  precision 78%, recall 87%, F1 score 81%, ROC-AUC 91.6%)  and base TabNet (accuracy 81%,  precision 80%, recall 82%, F1 score 81%, ROC-AUC 86.7%). The study recommends healthcare institutions to adopt the validated ensemble TabNet-based architecture as a standardized framework for clinical decision support systems across multiple diseases. Further, researchers should establish this methodology as the preferred approach for limited-feature medical datasets, extending beyond diabetes to include cardiovascular, hypertension, and cancer screening applications.