DeepShield: An Explainable AI Framework for Multimodal Deepfake Detection
Mia, Md. Melon
Abstract
Deepfakes increasingly combine manipulated facial appearance, synthetic or replaced speech, temporal inconsistencies, audio-visual synchronization artifacts, and context-level anomalies. A detector that relies on only one cue can therefore miss manipulations that preserve that cue while altering another. This thesis presents DeepShield, an explainable, multimodal, reliability-aware deepfake forensic framework that integrates Face, Voice, Temporal, Lip-Sync, and Background evidence through mask-aware learned attention fusion. The Face branch combines Swin Transformer, official GenConViT-ED, GenConViT-VAE, and frequency-domain evidence; the Voice branch uses MFCC and Log-Mel features with a CNN-LSTM; the Temporal branch models ordered face sequences using BiLSTM and temporal attention; the Lip-Sync branch analyzes aligned mouth-audio evidence; and the Background branch uses CLIP and DINOv2 representations over original and face-suppressed frames. Validation-only temperature scaling and a locked decision threshold are used before a single frozen internal TEST evaluation. The project-wide eligible primary registry contained 254,228 media items. Earlier leakage-safe and broad visual stages used 177,584 selected samples and 180,206 visual sequences respectively, while the final tightly aligned multimodal canonical experiment used 21,544 A/V cases split into 17,234 TRAIN, 2,159 validation, and 2,151 frozen TEST cases. The final frozen internal TEST achieved ROC-AUC 0.998277, accuracy 0.992562, precision 1.000000, recall 0.992385, F1-score 0.996178, expected calibration error 0.007261, and Brier score 0.005121. The results demonstrate strong performance on the locked aligned evaluation cohort while the thesis deliberately limits claims to this internal experimental setting. DeepShield is positioned as an automated forensic screening framework rather than legal proof, with explicit modality masks, uncertainty-aware decisions, and explainability outputs designed to support human review.
Permanent link — send this address to others:https://lib.su.edu.bd/thesis/123456789/3067