Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence

Ibrahim, MohdZamri and Saealal, Muhammad Salihin and J.Mulvaney, David. and Shapiai, Mohd Ibrahim and Fadilah, Norasyikin (2022) Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence. PLoS ONE, 17 (12). pp. 1-23. ISSN 1932-6203

Text 0235514082024141210.pdf Available under License Creative Commons Attribution. Download (3MB)

Abstract

Deeplearning is notably successful in data analysis, computer vision, and human control. Nevertheless, this approach has inevitably allowed the development of DeepFake video sequences and images that could be altered so that the changes are not easily or explicitly detectable. Such alterations have been recently used to spread false news or disinformation. This study aims to identify Deepfaked videos and images and alert viewers to the possible falsity of the information. The current work presented a novel means of revealing fake face videos by cascading the convolution network with recurrent neural networks and fully connected network (FCN) models. The system detection approach utilizes the eye-blinking state in temporal video frames. Notwithstanding, it is deemed challenging to precisely depict (i) artificiality in fake videos and (ii) spatial information within the individual frame through this physiological signal. Spatial features were extracted using the VGG16 network and trained with the ImageNet dataset. The temporal features were then extracted in every 20 sequences through the LSTMnetwork. Onanother note, the pre-processed eye-blinking state served as a probability to generate a novel BPD dataset. This newly-acquired dataset wasfedtothree models for training purposes with each entailing four, three, and six hidden layers, respectively. Every model constitutes a unique architecture and specific dropout value. Resultantly, the model optimally and accurately identified tampered videos within the dataset. The study model was assessed using the current BPD dataset based on one of the most complex datasets (FaceForensic++) with 90.8% accuracy. Such precision was successfully maintained in datasets that were not used in the training process. The training process wasalso accelerated by lowering the computation prerequisites.

Item Type:	Article
Divisions:	Faculty of Electrical and Electronic Engineering Technology
Depositing User:	Norfaradilla Idayu Ab. Ghafar
Date Deposited:	06 Jan 2025 10:55
Last Modified:	06 Jan 2025 10:55
URI:	http://eprints.utem.edu.my/id/eprint/28189
Statistic Details:	View Download Statistic

Actions (login required)

View Item