An ensemble architecture for forgery detection and localization in digital images

This thesis presents a unified ensemble approach for forgery detection and localization in digital images. The focus of the research is on two of the most common but effective forgery techniques: copy-move and splicing. The ensemble architecture combines a set of forgery detection and localization methods in order to achieve improved performance. The main contributions of this work are listed in the following. First, an extensive review of the current state of the art in forgery detection, with a focus on deep learning-based approaches is presented in Chapter 1. An important insight that is derived is the following: these approaches, although promising, cannot be easily compared in terms of performance because they are typically evaluated on custom datasets, not always publicly available, due to the lack of precisely annotated data. We then designed a keypoint-based copy-move detection algorithm, which is described in chapter 2. Compared to previous existing keypoints-based approaches, we added a density-based clustering step to filter out noisy keypoints matches. This method has been demonstrated to perform well on two benchmark datasets and outperforms one of the most cited state-of-the-art methods. In Chapter 3 a novel architecture is proposed to predict the 3D light direction of the light in a given image. This approach leverages the idea of combining, in a data-driven method, a physical illumination model that allows for improved regression performance. In order to fill in the gap of data scarcity for training highly-parameterized deep learning architectures, especially for the task of intrinsic image decomposition, we developed two data generation algorithms that were used to produce two datasets - one synthetic and one of real images - to train and evaluate our approach. The proposed light direction estimation model has then been employed to design a novel splicing detection approach, discussed in Chapter 4, in which light direction inconsistencies between different regions in the image are used to highlight potential splicing attacks. The ensemble scheme is described in the last chapter. It includes a "FusionForgery" module that optimally combines the outputs of the here proposed "base" methods and assigns a binary label (forged vs. pristine) to the input image. In the case of forgery prediction, our method also tries to further specialize the decision between splicing and copy-move attacks. If the image is predicted as copy-moved, an attempt to reconstruct the source regions used in the copy-move attack is also performed. The performance of the proposed approach has been assessed by training and testing it on a synthetic dataset, generated by us, comprising both copy-move and splicing attacks. The ensemble approach outperforms all of the individual "base" methods, demonstrating the validity of the proposed strategy.


  • Primary: Nicola Adami


Short Bio

In 2018 I graduated in Computer Engineering, with honors, at the University of Brescia, with a thesis on the numerical resolution of models for fluid dynamics (Navier-Stokes). Later I worked for a few months in an IT consulting company in Milan, familiarizing myself with technologies such as Java EE, Facelet, REST / SOAP services, JQuery. I then worked as a software developer in the technical office of a company that designs and builds machines for quality control through optical inspection, developing computer vision, machine learning and A.I. algorithms. In addition to technical skills, thanks to this experience I had the opportunity to work in teams, interact with other departments (production, electrical, testing) and evaluate critical issues and software requirements with customers and suppliers. Since 2019 I am following a PhD course in Information Engineering at the University of Brescia, supervised by prof. Nicola Adami. My research project consists in the development and analysis of algorithms to identify possible manipulations in images or videos. During my PhD I had the opportunity to use technologies and frameworks for the development of machine/deep learning solutions, such as Tensorflow, Keras, PyTorch. During this time I also carried out teaching support activities, proposing project tracks for the final exam of the Digital Image Processing course and I also held seminars on image processing topics. My research interests concern image/video processing, A.I, machine learning, computer vision, computer graphics.