^{1}

^{2}

^{2}

^{3}

^{2}

^{4}

^{1}

^{2}

^{3}

^{4}

Copy-move forgery is the most predominant forgery technique in the field of digital image forgery. Block-based and interest-based are currently the two mainstream categories for copy-move forgery detection methods. However, block-based algorithm lacks the ability to resist affine transformation attacks, and interest point-based algorithm is limited to accurately locate the tampered region. To tackle these challenges, a coarse-to-fine model (CFM) is proposed. By extracting features, affine transformation matrix and detecting forgery regions, the localization of tampered areas from sparse to precise is realized. Specifically, in order to further exactly extract the forged regions and improve performance of the model, a two-level local search algorithm is designed in the refinement stage. In the first level, the image blocks are used as search units for feature matching, and the second level is to refine the edge of the region at pixel level. The method maintains a good balance between the complexity and effectiveness of forgery detection, and the experimental results show that it has a better detection effect than the traditional interest-based copy and move forgery detection method. In addition, CFM method has high robustness on postprocessing operations, such as scaling, rotation, noise, and JPEG compression.

With the rapid development of technology worldwide, there are many ways to obtain and process images [

Two examples of the copy-move forgery. Left to right: original images, forged images through copy-move operations, and copy-move regions.

As one of the most common means of image tampering, copy-move forgeries may be accompanied by certain postprocessing, including JPEG compression, noise addition, and blurring, to change the image content and confuse the information recipient [

Our main contributions can be summarized as follows:

This paper proposes a coarse-to-fine model for detecting forged regions by the affine transformation matrix (CFM). The localization of the forged regions from sparse to accurate is achieved.

To further extract the forgery region accurately, a two-stage local search algorithm is designed in the refinement stage to better maintain the balance between complexity and effectiveness of forgery detection.

The method has better detection results and higher robustness to postprocessing operations such as scaling, rotation, noise, and JPEG compression.

Numerous methods for copy-move forgery detection (CMFD) have been proposed in the last decade, which are traditionally categorized into two classes: block-based and interest point-based methods.

In 2003, Fridrich [

Different from block-based algorithms, interest point-based CMFD algorithms are more robust against affine transformations. Unlike dividing an image, this method extracts interest points on the image, and image features are then extracted around the interest points. He et al. [

The block-based CMFD algorithm and interest point-based CMFD algorithm each have a similar framework as depicted in Figure

Preprocessing: its main purpose is to eliminate irrelevant information in the image and restore useful real information; the most common approach is to convert the image from an RGB version to a grayscale image

Feature extraction: local image information is extracted from an image block or interest point represented by a feature descriptor

Matching: similar pairs of image blocks or points are determined during the matching process

Common copy-move forgery detection framework-based CMFD.

Most existing algorithms based on image blocks suffer from some attacks, such as scaling, rotation, and noise addition, and interest point-based methods cannot locate the tampered region precisely. To solve these problems, a hybrid two-level method combining image blocks and interest points is proposed in this paper. We chose the SIFT as the feature descriptor to represent the interest point. Then, the adaptive oversegmentation method is used to improve the matching process and calculate the affine transformation matrix. Finally, the proposed local search algorithm is applied to image block level and pixel level, respectively, to locate the tampered region accurately.

In this paper, an accurate CMFD method based on interest point and local search algorithm is proposed. The process is illustrated in Figure

Framework of the proposed copy-move forgery detection method.

The main flow of the proposed algorithm is as follows: (1) feature extraction: interest points are detected in the input image represented by a feature descriptor, after which accurate interest point matches are obtained via a matching process; (2) affine transformation calculation: utilize a random verification algorithm to calculate the affine transformation matrix; (3) forgery region extraction: local search algorithm is applied to the image block level and the pixel level. The image block level realizes the location of the tampering region, and the pixel level is used to refine the tampering region boundary.

The image-level detection and pixel-level detection of the proposed model on the testing dataset show promising results. Our main contributions are as follows:

A method combining image blocks and pixels is proposed. Based on the block, the forged region can be located, and the pixel points are used to make the area boundary more refined. This method can make up for the poor performance of only extracting tampered areas with points of interest, thereby improving detection performance.

Considering the balance between algorithm complexity and performance, design a two-level local search algorithm. In the first stage, the image is divided into small blocks by rectangular blocks. If the image block contains the point of interest, it is marked as a forgery unit and calculated by affine transformation. The search algorithm matches the result to get the forgery region. In the second stage, the boundary of the forged area is extracted at the pixel level, and a secondary search algorithm is used for improvement to further improve the accuracy of model detection.

Four different postprocessing operations were performed on the test dataset, and the experimental results show that our model still exhibits high robustness.

In the rest of this section, we present the process of this detection algorithm as illustrated in Figure

The first phase of the proposed algorithm involves interest point detection and feature extraction based on SIFT features, referring to local features of an image. SIFT remains invariant to rotation, scaling, and light intensity and maintains stable robustness to changes in the viewing angle, affine transformation, and noise. The interest points and their corresponding descriptors are obtained. Based on these results, the proposed algorithm performs a matching operation to identify similar local regions.

To obtain good performance in matching and calculation of the affine transformation matrix, the adaptive oversegmentation method is adopted [

Next, a segmented image builds the image blocks set ^{th} image block are stored in _{i}

Framework of the proposed copy-move forgery detection method.

The 2NN algorithm utilizes the ratio of the distance between the nearest neighbor and the second nearest neighbor. If image blocks _{i} and _{j} must match, for any feature point, where is the ^{th} point in block _{i}, the calculation is as follows:_{1} is the closest neighbor, and _{2} is the second closest neighbor. The distance _{m} is calculated as_{m} denotes the distance between point ^{th} point in _{j}, and

In our experiment,

We iterate the 2NN process in different image blocks in our experiment until all blocks have been traversed, resulting in a dataset: _{i} and _{j} are stored in

Matching operations between image blocks can avoid failed matching due to the proximity of points to coordinates. To further prevent match failure, assuming that _{i} and _{j} are considered a failure and must be deleted. As such,_{p} is set to 3 to filter the failed pairs. Thus, most missed matches are filtered.

To better display the tampered region, affine transformation matrix _{y} denote translations and _{1}, _{2}, _{3}, and _{4} are associated with scaling and rotation. _{matrix} can obtain the affine transformation matrix

To verify the accuracy of matrix

We verify the matrix accuracy based on the distance between _{d} is the similarity threshold of the matrix (_{d} = 1.5 in our experiment). Then, we obtain the number of right point pairs count in

When rate is greater than 0.5, the matrix

In most cases, the source region and replication region may be covered by many image blocks. Many affine transformation matrices can be obtained through

Next, we will display the tampered region in the search algorithm.

Extracting the tampered region using only the interest point results in poor performance. By considering the balance between algorithm complexity and performance to more accurately extract the forgery region, we propose a local search algorithm that can be applied at the image block level and pixel level. The role of the local search algorithm is described in Figure

Role of local search algorithm. The region outlined in red is the forged region, and the blue small block is the forged unit.

The detection unit can find a corresponding unit via the affine transformation matrix, which is key to the local search algorithm. The detected unit can find corresponding unit through the matrix. Before executing the search algorithm, the forged units must be collated and added to the forgery region set (TR). Then, the local search algorithm is executed; steps are shown in Algorithm

Input: forgery region set (_{cnt}) (block or pixel), affine transformation matrix (_{end})

Output: forgery region set (_{cnt + 1})

Detection unit _{cnt} and obtain the neighborhood _{nei}; elements that have been detected in _{nei} are deleted. _{nei} is added to the set _{nei}.

Nondetection unit _{i} is removed from _{nei} and obtain detection unit _{i} and _{i} and _{cnt + 1}, and the neighborhood of _{i} is added to _{nei}_{nei} is empty.

Iterate steps 1 and 2 until all elements in _{cnt} have been detected.

_{cnt} is the result of the current detection, _{nei} is the set of neighborhood _{nei} may be the detected element; therefore, the detected elements in _{nei} must be deleted. Then, the corresponding unit _{i} is calculated by matrix

The successfully matched unit pairs are added to _{cnt + 1}. This operation is iterated until all elements in _{cnt} have been detected. Finally, the test result _{cnt + 1} is combined with the original result _{cnt + 1}_{cnt + 1} = _{cnt + 1} U _{cnt}. To understand the algorithm flow and prove the validity of the local search algorithm, a flow chart is used for descriptive purposes (Figure

Framework of the local search algorithm. The blue blocks are forged units, green tags stand for detecting units, red blocks are nonforged units, and white blocks are nondetected units.

Figure

To balance the complexity and accuracy of the algorithm, the two-stage local search algorithm is proposed: the image block level. And, the second stage is at the pixel level to refine the edge of the tampered region. The framework of the algorithm is displayed in Figure

Framework of two-stage tampered region localization algorithm.

In our method, interest points in the _{right}. First, a small, nonoverlapping rectangular block is used to cover the host image, and all image blocks are scanned. If the image block contains interest points in _{right}, the block is marked as a forged unit. Then, the image blocks as a detection unit are added to _{0}, and the search algorithm is employed on the image block level. Corresponding image blocks are calculated by the affine transformation _{i} calculates corresponding image block _{i}; in this case, image block _{i} cannot reach the center of another block _{i}) and needs to extract the true matching image block _{i}, so feature comparison must be executed between _{i} and _{i}. Then, the ZNCC (zero-based normalized cross-correlation) should be calculated between _{i} and _{i} as follows:_{j} and _{RD}) is set up to obtain similar image block pairs:

In our work, _{RD} is set to 0.55 once formula (_{i} and _{j})

are similar, and the results of the search algorithm are stored in _{1}

A filtering algorithm is used to render the test results more accurate. For each forged unit in _{1}_{nei} = {_{0}, _{1}, _{2}, _{3}, _{4}, _{5}, _{6}, _{7}}. In our experiment, if the number of forged units in _{nei} is less than 2, the detection element

It is challenging to extract the forgery region at the image block level, and the algorithm does not have good performance at the edge of the tampered region. Thus, the edge of _{1} is extracted, and we obtain an edge region _{0} and a center region _{1} on the image block level, where _{0} is considered inaccurate and _{1} is accurate. In matrix _{1} must be calculated. For the obtained pixel pairs, the ZNCC algorithm is used to measure similarities, and the threshold _{1}, from which, forgery region _{2} is obtained by combining the center region _{1} and the matching result _{1}. To improve the edge of the forged region, the edge of _{2} is extracted at pixel level in _{2}, and _{2} is used to execute local search algorithm. Assume that we get (_{i}_{I'} are the color features of _{i} and _{I'} conform to formula (_{3}.

The tampered region _{2} is obtained by combining _{3} and center region _{2}. After the filtering step, the morphological close operation is applied to _{3} to eliminate small gaps, after which the tampered region _{end} is generated. The algorithm is evaluated in the following section to demonstrate its effectiveness.

In this section, a series of experiments are conducted to evaluate the performance of the proposed CMFD method. Section

In the following experiments, a benchmark database [

To quantitatively evaluate the detection performance, we adopted two metrics: precision and recall. Precision is the fraction of pixels identified as forgery that are truly forgery, defined as the ratio of the number of correctly detected forged pixels to the total number of detected forged pixels. Recall refers to the fraction of forged pixels that are correctly classified, defined as the ratio of the number of correctly detected forged pixels to the number of forged pixels in the ground truth forgery image. Precision and Recall are calculated using (^{′} denotes the forged regions of the ground-truth of forged images. We provide the _{i} score as a measure that combines precision and recall in a single value.

Using these metrics, we show how precisely the CMFD algorithms identified tampered regions. To reduce the effects of random samples, the average precision and recall were computed for all images in the dataset.

Plain copy-move forgery is a kind of one-to-one copy-move method that does not involve other transformation operations. It is to cut the local area of the target image and then paste it into the target image again through rotation, scaling, and other operations to generate a new tampered image. We experimented on 48 plain copy-move forgery images in total. Figure

Copy-move forgery detection results under plain copy-move forgery. Top to bottom: test images from dataset, ground-truth forged regions, and forged regions detected by CFModel.

In addition to one-to-one copy-move forgery, we also experimented on the various attacks to verify the effectiveness of the proposed algorithm.

Scale: the tampered region is rescaled to between 91% and 109% of their original size with 2% step length.

In total, 48 × 10 = 480 images are experimented. Figure

Rotation: the tampered region is rotated at a rotation angle varying from 2° to 10° with a step length of 2°. In total, 48 × 5 = 240 images are experimented. Figure

Gaussian noise: the image intensities of the tampered region is normalized between 0 and 1 with added zero-mean Gaussian noise with standard deviations of 0.02 to 0.10 and a step length of 0.02. In total, 48 × 5 = 240 images are experimented. Figure

JPEG compression: the forged image is JPEG compressed with quality factors varying between 100 and 20 and a step length of 10. In total, 48 × 9 = 432 images are experimented. Figure

Copy-move forgery detection results for the scale. Top to bottom: test images from dataset, ground-truth forged regions, and forged regions detected by the proposed algorithm. Left to right represents the scale of 91%, 93%, 95%, 97%, 103%, 105%, 107%, and 109%.

Copy-move forgery detection results for the rotation. Top to bottom: test images from dataset, ground-truth forged regions, and forged regions detected by the proposed algorithm. Left to right, the angle of rotation is 2°, 4°, 4°, 6°, 6°, 8°, 8°, and 10°.

Copy-move forgery detection results for noise. Top to bottom: test images from dataset, ground-truth forged regions, and forged regions detected by the proposed algorithm. Left to right: the standard deviation is 0.02, 0.04, 0.04, 0.06, 0.06, 0.08, 0.08, and 0.1.

Copy-move forgery detection results for the JPEG. Top to bottom: test images from dataset, ground-truth forged regions, and forged regions detected by the proposed algorithm. Left to right, the quality factor is 20, 30, 40, 50, 60, 70, 80, and 90.

This section presents the comparison results between CFModel and the existing methods, and experiments on the dataset proposed in [

We first evaluated our algorithm under plain copy-move forgery attack. We experimented on 48 original images and 48 forged images, which are tampered by one-to-one copy-move forgery. Tables

Detection results of plain copy-move forgery at image level.

Image level | Precision (%) | Recall (%) | _{1} (%) |
---|---|---|---|

SIFT [ | 88.37 | 79.17 | 83.52 |

SURF [ | 91.47 | 89.58 | 90.52 |

Iterative [ | 67.14 | 97.91 | 79.66 |

Hybrid [ | 78.33 | 97.92 | 87.04 |

Detection results of plain copy-move forgery at pixel level.

Pixel level | Precision (%) | Recall (%) | Fi (%) |
---|---|---|---|

SIFT [ | 60.80 | 71.48 | 65.71 |

SURF [ | 68.13 | 76.43 | 72.04 |

Iterative [ | 72.23 | 96.46 | 82.61 |

Hybrid [ | 90.27 | 78.61 | 84.04 |

Bi [ | — | — | 92.87 |

Chen [ | — | — | 93.92 |

“/” denotes that corresponding results are not provided in the literature.

As noted in Table

In order to obtain a more detailed assessment of the discriminative properties of the method, the detailed data of copy-move forgery detection results, experimented on 1392 tampered images under various attacks in total, are shown in Figure

Detection results under various attacks. Top to bottom: scale, rotation, Gaussian noise addition, and JPEG compression. Left to right: precision, recall, and _{1}.

As shown in the figure, the precision rate and recall rate of our scheme reached a higher level than other methods, the

With the development of digital technology, digital images can be easily forged using image processing software. Forged images must be identified given the potential legal and other implications. In this paper, we propose a copy-move forgery detection algorithm using SIFT as the interest point and feature extraction method. The affine transformation matrix was then calculated, followed by a local search algorithm to locate the forged region. Experimental results show that the proposed scheme performs much better than state-of-the-art copy-move forgery detection algorithms and demonstrates good performance under various attacks. However, performance was poor when images contained noise; we will focus on this image type in later work.

Future research is mainly as follows:

To address the problem that the method cannot adapt to noisy operations, future plans are to incorporate richer texture feature information to achieve better robustness

In future work, we will focus on detection tasks with multiple copy-move tampered regions at the same image to realize practical applications of the detection algorithm

The data used to support the findings of this study are available from the corresponding author upon request.

The authors declare that they have no conflicts of interest.

This research was supported by the National Key Research and Development Program of China (2018YFB080402 and 2018YFB0804203), Regional Joint Fund of NSFC (U19A2057), the National Natural Science Foundation of China (61672259 and 61876070), Jilin Province Science and Technology Development Plan Project (20190303134SF and 20180201064SF), CERNET Innovation Project (NGII20190802), and Undergraduate Innovation and Entrepreneurship Training Program of Jilin University (202010183389).