Imbalance-Aware AutoML for Financial Risk Prediction
Halim, Abdul
Abstract
Class imbalance is a persistent challenge in financial and business risk prediction because the minority class often represents the cases of greatest practical interest, such as fraud, default, or customer attrition. This study investigates where imbalance should be handled inside an automated machine-learning (AutoML) pipeline. Seven strategies are compared: an untreated baseline, SMOTE oversampling, an average-precision training objective, balanced class weighting, cost-sensitive decision-threshold tuning, a combined objective and-threshold strategy, and a full combination of SMOTE, objective optimization, and threshold tuning. The supplied experiment evaluates these strategies on eight datasets spanning approximately 0.17% to 44.5% minority prevalence, across ten random seeds and two engines, XGBoost-GPU and FLAML, for a total of 1,120 runs. The experimental design uses stratified 60/20/20 train-validation-test splits, leakage aware preprocessing, probability-based evaluation, and an asymmetric cost function in which a false negative costs five times a false positive. Nine evaluation measures are computed, with PR-AUC, recall, F1, G-mean, MCC, and total cost emphasized. For statistical comparison, the study ranks strategies within dataset-seed blocks and applies the Friedman test, with a critical-difference analysis. The results show that imbalance handling primarily improves the decision process rather than the ranking of observations. On XGBoost-GPU, the threshold strategy achieves the best average rank, followed by class weighting and the combined objective-plus-threshold strategy. On FLAML, the combined objective-plus-threshold strategy ranks first, followed by threshold tuning and class weighting. Across both engines, these three strategies form a consistent top group, whereas the untreated baseline and objective-only strategy are the weakest. The study therefore supports decision-level cost-sensitive thresholding as a particularly effective and robust component of imbalance-aware AutoML, while also showing that the preferred exact strategy can depend on the AutoML engine.
Permanent link — send this address to others:https://lib.su.edu.bd/thesis/123456789/3075