This paper presents a copyright identification scheme for color images that takes advantage of the complementary nature of watermarking and fingerprinting. It utilizes an authentication logo and the extracted features of the host image to generate a fingerprint, which is then stored in a database and also embedded in the host image to produce a watermarked image. When a dispute over the copyright of a suspect image occurs, the image is first processed by watermarking. If the watermark can be retrieved from the suspect image, the copyright can then be confirmed; otherwise, the watermark then serves as the fingerprint and is processed by fingerprinting. If a match in the fingerprint database is found, then the suspect image will be considered a duplicated one. Because the proposed scheme utilizes both watermarking and fingerprinting, it is more robust than those that only adopt watermarking, and it can also obtain the preliminary result more quickly than those that only utilize fingerprinting. The experimental results show that when the watermarked image suffers slight attacks, watermarking alone is enough to identify the copyright. The results also show that when the watermarked image suffers heavy attacks that render watermarking incompetent, fingerprinting can successfully identify the copyright, hence demonstrating the effectiveness of the proposed scheme.
1. Introduction
Many researchers [1–17] have been engaged in finding the solution to protecting copyrights of digital images, which may be duplicated and distributed over the Internet without the authors’ permission. Generally speaking, there are two approaches to discovering image copyright infringement. One is watermarking [1–11] and the other is fingerprinting [12–17]. The main idea of watermarking is to embed a piece of information (i.e., watermark) in the host image. If a similar watermark can be retrieved from a suspect image, it is then considered a duplicated one. On the other hand, the principle of fingerprinting is to extract unique features (i.e., fingerprints) from both the host image and the suspect one for comparison. If their fingerprints are similar, the ownership of the image can then be confirmed.
There are some general considerations on the two techniques, including the processing time and robustness. In terms of processing time, watermarking is more efficient because fingerprinting needs extra time to compare the image’s fingerprint with those stored in the database. If the database is large, it will be very time consuming. On the other hand, fingerprinting is generally more robust [18, 19] because when a watermarked image suffers some image processing operations that modify the content of the image, the embedded watermark will usually be damaged or even destroyed. On the contrary, since normal image processing does not destroy the features of an image, the fingerprint of the image can therefore be preserved. In summary, fingerprinting is more robust whereas watermarking is more efficient. If the complementary natures of two approaches can be utilized properly, a robust and efficient scheme can then be developed to identify copyrights.
This paper proposes a novel scheme that combines the two techniques to identify copyrights for color images. The proposed scheme generates from the image a fingerprint, which also serves as the watermark. The watermark is then embedded in the host image to produce a watermarked image. When there is a dispute over the copyright of a suspect image, the suspect image will first be processed by watermarking, which tries to retrieve the watermark from the suspect image. If the watermark is identified, the copyright is confirmed at this stage; otherwise, the image will then be processed by fingerprinting, which utilizes the retrieved watermark as the fingerprint and compares it with those stored in the database. If a match is found, then the suspect image will be considered a duplicated one.
2. Related Background
The proposed scheme utilizes a special technique called image secret sharing (ISS), whose details can be found in the paper [20] we published in 2008. The following briefly describes the main idea of the technique utilized by the proposed scheme.
The ISS generates a share image from two images. In the proposed scheme, the two images are the logo image and the feature image (as depicted in Figure 1). The logo image can be any identifiable image. The feature image is generated from the input image as follows. First, the input image is split into nonoverlapping 8 × 8 blocks. Then, the 2D DWT is applied to each block to generate four subbands, LL2, LH2, HL2, and HH2. An example of 2D DWT is shown in Figure 2. Finally, the coefficients in the LL2 subband of each DWT block are used to generate the feature image. The ISS then generates a share image from the logo image and feature image. The share image will be used as the fingerprint of the input image by the scheme. The share image also serves as the watermark to be embedded in the host image. The benefit of the ISS scheme is that performing the XOR operation on the feature image and the share image will restore the logo image, which can then be used to identify the copyright.
The process of the ISS.
2D DWT.
3. The Proposed Copyright Identification Scheme
The proposed scheme contains two phases: the fingerprint and watermarked image generation phase and the authentication logo detection phase. The former phase extracts features from the host image, which, along with a logo image, is used to generate the fingerprint. The fingerprint also serves as the watermark, and the phase embeds it in the host image to produce a watermarked image. On the other hand, the latter phase extracts features and retrieves the watermark from the suspect image. The extracted features and the retrieved watermark are utilized to restore the logo image, which is used to identify the copyright. If it fails, the retrieved watermark then serves as the fingerprint and is compared with those in the database to determine if the suspect image is a duplicated one.
The fingerprint and watermarked image generation phase (shown in Figure 3) works as follows. In the beginning, feature extraction extracts the features of the host image and then logo scrambling disarranges the authentication logo to a scrambled logo image. After that, fingerprint generation takes as input the extracted features and the scrambled logo to generate the fingerprint. Finally, the fingerprint serves as a watermark and is embedded in the host image, which becomes a watermarked image. The fingerprint is also stored in a database for later use in the next phase.
The phases of the proposed scheme.
The authentication logo detection phase (shown in Figure 3) checks the watermark first and, if necessary, the fingerprint next. In the beginning, watermark retrieval regains the watermark from the suspect image. Next, the features of the suspect image are extracted by feature extraction. After that, logo restoration takes as input the retrieved watermark (the expected fingerprint of the suspect image) and the extracted features to recover and rearrange the scrambled logo to restore the authentication logo. The phase ends if the accuracy rate of the restored logo determined by logo comparison is high enough; otherwise, the process proceeds to retrieve the next available fingerprint from the database and then returns to logo restoration, which takes as input the retrieved fingerprint instead of the extracted watermark. The phase restores the logo from the retrieved fingerprint as well as the extracted features and proceeds to logo comparison. The looping process continues until the authentication logo is discovered or no fingerprint is available.
3.1. Fingerprint and Watermarked Image Generation Phase
The following paragraphs detail the stages in the fingerprint and watermarked image generation phase, including feature extraction, logo scrambling, fingerprint generation, and watermark embedding.
3.1.1. Feature Extraction
The feature extraction stage takes a color image as input and then extracts its features. The stage has two substages, sampling and feature generation. During sampling, the stage first transforms the RGB image to the YCbCr color space [21, 22]. Then, it partitions each of the three channels into several nonoverlapping blocks of size 8 × 8 (hence, a color image of size N×N will result in 3×N/8×N/8 nonoverlapping blocks). After partition, for each row of the corresponding blocks in the three channels, the stage takes the first four samples from the Y channel, the next two from the Cb, and the last two from the Cr (as depicted in Figure 4) to generate new packed blocks, each of size 8 × 8.
Illustration of sampling.
After sampling, the stage enters its second substage, feature generation. For each of the packed blocks, the stage applies 2D DWT to the block, resulting in four coefficients in the LL2 subband. Next, the stage computes the average (denoted by A) of the four coefficients and then obtains a feature type T according to the relationship of the four coefficients and the average A as expressed in (1). Consider
(1)T={1,ifonlyonecoefficientissmallerthanA.2,ifonlytwocoefficientsaresmallerthanA.3,ifonlyonecoefficientisgreaterthanA.4,ifallofthefourcoefficientsarethesameandhenceallequalA.
According to T, A, and the mapping table shown in Table 1, a feature share (called FT-share) of size 2 × 2 is determined for each block. The FT-shares represent the features of the input color image. They are assembled to form the feature image.
The mapping table.
Feature type T
Mean value position
White logo pixel
FT-share XOR FP-share
Black logo pixel
FT-share XOR FP-share
FT-share
FP-share
FT-share
FP-share
1
a<A<b,c,d
b<A<a,c,d
c<A<a,b,d
d<A<a,b,c
2
a,b≤A<c,d
c,d≤A<a,b
a,d≤A<b,c
b,c≤A<a,d
a,c≤A<b,d
b,d≤A<a,c
3
b,c,d≤A<a
a,c,d≤A<b
a,b,d≤A<c
a,b,c≤A<d
4
A=a=b=c=d
The coefficients of the LL2 subband (a is at the top left position, b the top right, c the bottom left, and d the bottom right)
The steps of the feature extraction stage are listed in Algorithm 1.
Algorithm 1: Feature extraction.
Input:A color image H (N × N).
Output:A feature image FT (N/8 × N/8).
Convert H to the YCbCr color space
Partition each of the Y, Cb, and Cr channels into N/8 × N/8 non-overlapping blocks of size 8 × 8
For each corresponding block of the three channels
Take the first 4 samples from the Y channel, the next 2 from the Cb, and last 2 from the
Cr as depicted in Figure 4 to form a packed block of size 8 × 8
End For
Apply 2D-DWT to each packed block to obtain N/8 × N/8 LL2 blocks of size 2 × 2
For each of the LL2 blocks
Compute the average A of the four coefficients
Obtain the feature type T according to (1)
Determine the FT-share according to T, A, and Table 1
End For
Assemble the FT-shares to form the feature image FT
3.1.2. Logo Scrambling
In order to disperse the intensity of attacks, the proposed scheme adopts Torus automorphism [23] to scramble the authentication logo. The stage uses a predetermined key, k, and the following equation to scramble the logo. Consider
(2)(xtyt)=(11kk+1)(xt-1yt-1)modT,
where (xt, yt) are the coordinates in state t and T is the coordinate size of the given image. Figure 5 shows an example of the authentication logo scrambled four times with k=1, T=128.
Example of scrambling.
3.1.3. Fingerprint Generation
The proposed scheme uses ISS (mentioned in Section 2) to generate the fingerprint. It determines a FP-share for each FT-share (generated in feature extraction) according to the color of the corresponding pixel in the scrambled logo by looking Table 1 up. For example, if the feature type is 2 and the FT-share is the same as the one in Table 2, then the FP-share will be either one of the two FP-shares in Table 2 according to the color of the corresponding pixel in the scrambled logo. After every FP-share is determined, the stage gathers all the FP-shares to form the fingerprint image. The fingerprint also serves as the watermark in the next stage.
Partial table of Table 1.
Feature type T
Mean value position
White logo pixel
FT-share XOR FP-share
Black logo pixel
FT-share XOR FP-share
FT-share
FP-share
FT-share
FP-share
⋮
2
b,c≤A<a,d
⋮
3.1.4. Watermark Embedding
The watermark embedding stage uses the resulting fingerprint image as a watermark and then embeds it in the host image. Generally speaking, for the RGB color space, the human visual system is more sensitive to the G channel than to the other two [22–28]. Therefore, the proposed scheme embeds the watermark into the less sensitive R and B channels of the host image. To be more specific, there are four areas: two in the R channel (I1 and I2) and two in the B channel (I3 and I4), used for watermark embedding (see Figure 6). The stage applies the 2D DWT to the R and B channels of the host image and next applies the 1D DWT to the resulting HL2 and LH2 to obtain the four blocks, I1, I2, I3, and I4.
The I1, I2, I3, and I4 blocks used for watermark embedding.
The proposed scheme embeds the watermark into the image by adjusting the coefficients in the I1 to I4 blocks according to a predefined M. The value of M affects the robustness and the quality. The larger the M is, the more robust the embedded watermark is, but the worse the quality of the watermarked image is. Figure 7 illustrates the adjustment of the coefficients. If the watermark bit is 1 and the current coefficient is between M×(i-1) and M×i, then the coefficient will be adjusted to be M×(i-1)-M/4 or M×i-M/4, whichever is closer to the current coefficient. On the other hand, the adjusted coefficient will be M×(i-1)+M/4 or M×i+M/4 if the watermark bit is 0.
Coefficient adjustment.
The embedding process is described as follows. First, the stage uses the current coefficient c and the predefined M to calculate S, sign, and (r0,r1) according to (3), (4), and (5), respectively. In the equations, sign indicates that the c is positive or negative; r0 is the remainder for watermark bit value 0 and, similarly, r1 for watermark bit value 1. Then, the stage determines the C_Low and C_High according to the value of the watermark bit. If the value is 0, (6) will be used, otherwise, (7). Finally, the stage adjusts the coefficient c to c′ according to (8). Consider
(3)S=|c|M,(4)sign={1,ifc≥0,-1,otherwise,(5)(r0,r1)={(M4,3M4),ifc≥0,(3M4,M4),otherwise,(6)C_Low=sign×(S×M+r0),C_High=sign×((S+sign)×M+r0),(7)C_Low=sign×((S-sign)×M+r1),C_High=sign×(S×M+r1),(8)c′={C_Low,if|C_Low-c|≤|C_High-c|C_High,otherwise.
The steps of the watermark embedding stage are listed in Algorithm 2.
Algorithm 2: Watermark embedding.
Input:A color image H (N×N) and a fingerprint image FP (N/4 × N/4).
Output:A watermarked image (N×N).
Apply DWT to the R and B channels of H to obtain the four embedding blocks I1,…,I4
For each coefficient c of the I1–I4 blocks and the corresponding pixel of FP
Compute the sign, S, and (r0, r1) by (3), (4), and (5), respectively
Compute the C_Low and C_High by (6) if watermark bit is 0, or (7) if otherwise
Adjust the coefficient to c′ by (8)
End For
Perform inverse DWT to produce the watermarked image
3.2. Authentication Logo Detection Phase
The phase is activated when a dispute over the copyright of a suspect image occurs. The following details the stages in the phase, including watermark retrieval and logo restoration.
3.2.1. Watermark Retrieval
The stage regains the watermark from the suspect image. First, the stage applies the DWT to the R and B planes of the suspect image in the same way as that in watermark embedding to obtain the embedding blocks, I1 to I4 (refer to Figure 6). Then, for each coefficient c of the I1–I4blocks, it obtains the watermark bit w according to
(9)w={1,if(c≥0,cmodM≥M2),or(c<0,|c|modM<M2),0,otherwise,
where M is the same as that in (3).
Finally, the watermark (which is supposed to be the fingerprint of the image) is restored by assembling every w for each coefficient c in the I1–I4 blocks.
Algorithm 3 lists the steps of the watermark retrieval stage.
Apply DWT to the R and B channels of S and obtain the embedding blocks I1–I4
For each coefficient c of the I1–I4 blocks
Obtain w according to (9)
End For
Assemble every watermark bit w to restore the fingerprint image FP′
3.2.2. Logo Restoration
The stage restores the authentication logo. As shown in Figure 8, it has four substages: scrambled logo recovery, logo unscrambling, logo enhancement, and logo resizing.
The four substages of logo restoration.
Scrambled Logo Recovery. After retrieving the fingerprint image (i.e., watermark) by watermark retrieval and extracting the feature image by feature extraction, the substage performs XOR operation on each pixel of the fingerprint image and the corresponding pixel of the feature image to retrieve the scrambled logo. Because both of the two images are black-and-white, each pixel is either 0 (black) or 1 (white). Therefore, the substage simply performs bitwise XOR operation on the two images and obtains a scrambled logo.
Logo Unscrambling. As mentioned in Section 3.1.2, the logo was scrambled before it is used to generate fingerprint in the fingerprint and watermarked image generation phase. The substage adopts (10), which is the inverse equation of (2), to rearrange the scrambled logo and restore the logo. Consider
(10)(xtyt)=(11kk+1)-1(xt-1yt-1)modT.
Logo Enhancement. The substage enhances the restored logo by erosion and dilation. Erosion removes pixels on object boundaries in an image and therefore can be used to remove smaller islands in the image; dilation, on the other hand, adds pixels to the boundaries of objects in an image and hence can be used to remove bright areas from the image. The substage performs erosion followed by dilation once, which is illustrated in Figure 9.
An example of erosion and dilation.
Erosion is performed by resetting each of the pixels according to (11). That is, if one of the neighbors is black, then the current pixel will be set to black; otherwise, it will be set to white. Let L(x, y) be the current pixel value to be determined and L(x+1,y), L(x+1,y+1), and L(x,y+1) its neighbors; then
(11)L(x,y)={0(black),ifL(x+1,y)orL(x+1,y+1)orL(x,y+1)isblack1(white),otherwise.
Dilation, on the contrary, resets the neighbors of each pixel rather than the pixel itself. If L(x, y) is black; the substage sets all of its neighbors, L(x+1,y), L(x+1,y+1), and L(x,y+1), to black; otherwise, the neighbors remain unchanged.
Logo Resizing. The proposed scheme adopts ISS to retrieve the authentication logo, which causes pixel expansion because one logo pixel is mapped to a share of four pixels (mentioned in Section 3.1.3). As a result, the retrieved logo from the previous stage will be larger than the original one. To resize the logo to its original size, the substage partitions the enhanced logo into several blocks of size 2 × 2, each of which is then reduced into one pixel with value L(x,y) according to the following rule:
(12)L(x,y)={0(black),thenumberoftheblackpixelsineachblock≥3,1(white),otherwise.
4. Experimental Results
Two kinds of experiments were conducted to prove the effectiveness of the proposed scheme. The first experiment shows the robustness of our scheme and the other demonstrates the capability of unique identification. In the experiments, the authentication logo used to generate the watermark (fingerprint) is shown in Figure 10.
The authentication logo (64 × 64).
Two common measurements used to estimate the robustness of our scheme are described as below.
(1) Peak Signal to Noise Ratio (PSNR). The measurement to estimate the color image quality after image processing is a variant version of normal PSNR [29]. The variant PSNR listed below does not consider the influence of the green channel because the channel is not modified by our scheme. Consider
(13)PSNR=10log102552(MSE(R)+MSE(B))/2dB,
where MSE is the mean square error between the original image and the modified image, which is defined as follows:
(14)MSE=∑i=1N∑j=1N(xij-xij′)2N2,
where xij represents the original pixel value and xij′ denotes the modified pixel value.
According to the definition of PSNR, the higher the value is, the better the quality of the modified image is. Generally, if the PSNR is greater than 30 dB, the quality of the modified image is acceptable.
(2) Accuracy Rate (AR). The measurement shown below is used to evaluate the correctness of the logo after it has been restored. Consider
(15)AR(%)=CPNP×100,
where NP is the number of pixels in the original logo and CP is the number of correct pixels obtained by comparing the pixels of the original logo with the corresponding ones of the restored logo. Figure 11 shows the restored logos with different AR values. As can been seen, restored logos with AR higher than 81% (the ones in the upper row) still can be visually recognized whereas the restored logo with AR equal to 76% is hard to identify. However, according to the description in Section 4.2, if AR is higher than 75%, the scheme still can identify the copyrights of the image.
The restored logo with their AR values (%).
4.1. Robustness Experiments
The experiments proved our scheme is robust to different kinds of attacks. The test images used in the experiment, including “Lena,” “Mandrill,” “Sailboat,” and “Peppers,” are shown in Figure 12. The commercial image processing software “Adobe Image Photoshop CS” was used to simulate several kinds of image attacks, some of which for “Lena” are shown in Figure 13.
The test images used in the experiments: (a) Lena, (b) Mandrill, (c) Sailboat, and (d) Peppers.
Some examples of "Lena" after different image attacks (PSNRs are listed in the last rows).
The experimental results of the test images are shown in Table 3. As mentioned above, if AR is less than 75%, the scheme cannot identify the copyrights of the image. Table 3 shows that our scheme failed to verify the copyrights for the images suffering the heavier attacks of JPEG, contrast, Gauss blurring, and scaling in watermark verification. Nevertheless, the duplications of those images can all be determined in fingerprint verification. In summary, our scheme can identify the copyrights of the suspect images under moderate attacks in watermark verification and determine the duplications of those suffering the heavy attacks in fingerprint verification.
The resulting AR values of the test images.
Attacks
AR (%) (Watermarking/fingerprinting)
Lena
Mandrill
Sailboat
Peppers
Average
None
99.93/99.93
99.76/99.76
99.98/99.98
99.63/99.63
99.83/99.83
JPEG
(90)
88.31/99.71
86.84/99.44
88.04/99.88
88.16/99.49
87.84/99.63
(80)
79.76/99.73
76.17/99.02
77.29/99.58
77.91/99.12
77.78/99.36
(70)
74.58/99.46
72.12/98.73
72.61/99.54
73.05/99.17
73.09/99.23
(60)
71.58/99.63
70.14/98.1
70.19/99.44
69.95/98.83
70.47/99.00
(50)
71.48/99.41
70.09/97.68
70.43/99.24
69.8/98.88
70.45/98.80
Contrast
(10)
92.6/99.85
80.71/99.78
88.62/99.95
90.31/99.27
88.06/99.71
(20)
82.86/99.76
73.12/99.56
79.27/99.78
81.81/98.68
79.27/99.45
(30)
75.85/99.46
70.61/99.07
75.22/99.63
74.63/98.07
74.08/99.06
(40)
72.61/99.27
70.83/97.92
72.31/99.19
72.51/98.14
72.07/98.63
(50)
71.7/98.93
70.41/96.19
71.19/98.75
71.09/96.58
71.10/97.61
(60)
70.39/97.68
70.65/92.7
70.95/98.1
70.53/95.8
70.63/96.07
(70)
70.8/94.65
70.65/90.31
71.12/97.71
71.36/95
70.98/94.42
Gauss blurring
(0.5)
83.64/99.41
73.44/97.29
78.17/99.17
80.54/99.05
78.95/98.73
(1.0)
76.1/98.88
70.92/95.78
74.17/98.44
73.88/98.34
73.77/97.86
(1.5)
73.63/98.17
70.95/94.34
72.44/97.29
72.34/97.73
72.34/96.88
Scaling
w: 256, h: 256
71.66/97.92
70.75/92.53
71.22/96.83
70.73/95.8
71.09/95.77
w: 128, h: 128
72.83/96.02
73.41/86.04
73.49/94.04
73.41/92.99
73.29/92.27
Brightness
(10)
96.66/96.78
94.41/94.92
97.85/97.85
94.46/94.46
95.85/96.00
(20)
90.65/91.48
89.06/90.7
95.9/95.9
88.33/88.33
90.99/91.60
(30)
85.72/87.62
83.98/86.43
93.55/93.58
83.69/83.64
86.74/87.82
Cropping
w: 480, h: 480
89.62/92.92
88.99/92.46
91.48/95.04
88.77/91.87
89.72/93.07
w: 448, h: 448
82.71/86.82
81.71/85.67
83.79/88.89
82.03/85.82
82.56/86.80
w: 432, h: 432
79.88/83.08
79.25/83.01
81.13/86.23
79/83.3
79.82/83.91
w: 424, h: 432
78.88/81.76
78.49/81.96
80.52/85.86
78.59/83.06
79.12/83.16
w: 424, h: 424
78.22/81.2
78/81.49
80.25/85.33
78.1/82.42
78.64/82.61
Moreover, there were 100 attacks in total and 53 of them resulted in an AR value higher than 75% in watermarking verification. That is to say, the copyrights of the 53% of the attacked images can be successfully identified in watermark verification, and hence only 47% of them need fingerprint verification.
4.2. Uniqueness Experiments
The experiment showed that our scheme has the capability of unique identification and is able to distinguish a copyrighted image from different ones. The four copyrighted images (with embedded watermarks) in Figure 12 along with seven unwatermarked images (Figures 14(a)–14(g)) were processed by our scheme to identify the copyright. The stored fingerprints of the watermarked images were used in fingerprint verification to restore the logos for all of the images.
The other seven color images used for testing uniqueness.
The results shown in Table 4 demonstrated the extraordinary unique identification capability of our scheme. It can be clearly seen that all the restored logos except the ones of the copyrighted images (those on the rightmost side) are unrecognizable, which proves that our scheme is actually able to distinguish a copyrighted image from different ones. It will not mistakenly identify the copyrights of an unwatermarked image.
Experimental results of the uniqueness experiment.
Lena
Mandrill
Sailboat
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Lena
Watermark
AR (%)
73.07
73.69
73.07
69.80
70.39
70.09
71.34
70.26
69.78
70.26
99.93
Fingerprint
AR (%)
59.06
56.03
49.24
62.72
59.62
44.34
62.35
60.40
45.65
63.21
99.93
Mandrill
Lena
Sailboat
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Mandrill
Watermark
AR (%)
73.74
73.69
73.07
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.76
Fingerprint
AR (%)
58.2
57.81
62.08
57.84
59.45
60.03
59.64
54.35
59.01
60.38
99.76
Sailboat
Lena
Mandrill
Peppers
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Sailboat
Watermark
AR (%)
73.74
73.07
73.07
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.98
Fingerprint
AR (%)
54.32
59.06
53.44
57.91
55.1
52.37
56.47
55.37
53.78
59.4
99.98
Peppers
Lena
Mandrill
Sailboat
F-16
Opera
Watch
New York
Tiffany
Terraux
Fourviere
Peppers
Watermark
AR (%)
73.74
73.07
73.69
72.73
73.52
73.79
72.64
74.35
73.82
72.2
99.63
Fingerprint
AR (%)
47.02
60.13
46.08
55.59
58.69
49.29
53.26
52.8
54.66
55.77
99.63
Moreover, the resulting AR values of all the unwatermarked images are all lower than 75%. Hence, it is reasonable for our scheme to confirm the copyright when AR is higher than 75%.
4.3. Discussion and Comparison
The fingerprint extracted from an image is more robust than the watermark embedded in the image. This can be seen from Figure 15, which shows the average AR values of the images suffering different attacks for watermarking and fingerprinting. The AR values for fingerprinting are all higher than those for watermarking. When the image undergoes image processing operations that heavily damage the embedded watermark, the extracted fingerprint still can survive the attacks. However, because linear comparison is a computationally intensive process, fingerprinting is costly in time if there are many fingerprints in the database.
The average AR values of test images: (a) JPEG quality, (b) contrast, (c) Gauss blurring, (d) scaling, (e) brightness, and (f) cropping.
Table 5 shows the processing time for watermark extraction and logo restoration and comparison in our scheme. The experiments were carried out on a computer equipped with the following hardware and software:
CPU: 3.16 GHz Intel(R) Xeon(R) CPU E3120,
RAM: 4 GB,
OS: Windows 7,
Computing language and environment: MATLAB.
The processing time (ms) for watermark extraction and logo restoration and comparison.
Stage
Lena
Mandrill
Sailboat
Peppers
Average
Watermark extraction
218
209
200
213
210
Logo restoration and comparison
20
29
39
28
29
Watermark extraction + logo restoration and comparison
238
238
239
241
239
As Table 5 shows, it takes about 0.21 seconds to extract the watermark from the input image and 0.029 seconds to restore and compare the logo. Therefore, the total processing time Tp (in seconds) of our scheme can be expressed by
(16)Tp=0.24+0.029×n,
where n represents the number of the fingerprints retrieved from the database. If the copyright can be identified in watermark verification, n is 0. That is, our scheme only needs 0.24 seconds in the best case. Otherwise, our scheme needs additional 0.029 seconds for each retrieved fingerprint in the database.
Because our scheme combines watermarking and fingerprinting techniques, it can be as efficient as pure watermarking schemes and also as robust as pure watermarking schemes. A pure watermarking scheme is very efficient because it only needs to make one watermark comparison to verify the copyright. However, it may not be as robust as a pure fingerprinting scheme when dealing with images that have suffered heavy attacks. Table 6 shows the comparison of our combined scheme and the other two pure schemes. If the copyright of the input image can be identified by watermarking verification, our scheme can be as efficient as pure watermarking schemes. If watermarking verification fails to identify the copyright, our scheme is still able to determine duplication in fingerprinting verification, which makes our scheme as robust as pure fingerprinting schemes.
Comparison of our combined scheme and the other two pure schemes.
Pure watermarking scheme
Pure fingerprinting scheme
Our combined scheme
Robustness
△
◯
◯
Efficiency
◯
△
◯
*◯: good; △: fair.
5. Conclusion
This paper presented a copyright identification scheme that takes advantage of the complementary nature of digital watermarking and fingerprinting. The experimental results showed that when the watermarked image suffers moderate attacks, watermarking verification alone is enough to identify the copyright, and there is no need for fingerprinting verification. In other words, the proposed scheme can identify the copyright efficiently in this situation. On the other hand, the experimental results also showed that when the watermarked image suffers heavy attacks that render watermarking verification incompetent, fingerprinting verification, although more time consuming, can successfully determine the duplication, hence demonstrating the robustness of the proposed scheme.
One distinguishing characteristic of the proposed scheme is that it does not need a separate watermark for watermarking verification and a separate fingerprint for fingerprinting verification. The proposed scheme extracts features from the input image to generate the fingerprint, which also serves as the watermark. Hence, only one piece of information is needed for both watermarking and fingerprinting verifications.
To further improve the scheme, retrieving the stored fingerprint image from the database to restore the correct authentication logo more quickly is worth studying. When there are more than one fingerprint image in the database, the original fingerprint of the host image must be correctly retrieved; otherwise, the correct authentication logo cannot be restored. Retrieving every stored fingerprint image to restore an authentication logo for comparison is very time consuming. The scheme should provide a more efficient way that is able to find the proper one in less time, which is the future work of the research.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
Financial support of this study by Tatung University, Taipei, Taiwan, under Grant B100-I07-036 is gratefully acknowledged.
CoatrieuxG.HuangH.ShuH.LuoL.A watermarking-based medical image integrity control system and an image moment signature for tampering characterization201317610571067LiuK.-C.Colour image watermarking for tamper proofing and pattern-based recovery2012654454542-s2.0-8486655457710.1049/iet-ipr.2011.0574ChenY.-L.HsuC.-T.Detecting recompression of JPEG images via periodicity analysis of compression artifacts for tampering detection2011623964062-s2.0-7995747014710.1109/TIFS.2011.2106121SuP. -C.ChangY. -C.WuC. -Y.Geometrically resilient digital image watermarking by using interest point extraction and extended pilot signals201381218971908PiperA.Safavi-NainiR.Scalable fragile watermarking for image authentication201374300311PawlakM.XinY.Robust image watermarking: an invariant domain approach2Proceedings of the IEEE Canadian Conference on Electrical and Computer EngineeringMay 20028858882-s2.0-0036069242LinW.-H.HorngS.-J.KaoT.-W.FanP.LeeC.-L.PanY.An efficient watermarking method based on significant difference of wavelet coefficient quantization20081057467572-s2.0-4764909949810.1109/TMM.2008.922795KongI.-K.PunC.-M.Digital image watermarking with blind detection for copyright verification1Proceedings of the 1st International Congress on Image and Signal Processing (CISP '08)May 20085045082-s2.0-5184910607110.1109/CISP.2008.546ElrefaeyL. A.AllamM. E.KaderH. A.SelimM.Robust blind image-adaptive watermarkingProceedings of the 25th National Radio Science Conference (NRSC '08)March 20082-s2.0-5064911755510.1109/NRSC.2008.4542331LeeM.-H.KwonO.-J.Color image watermarking based on DS-CDMA using Hadamard kernelProceedings of the 10th International Conference on Advanced Communication TechnologyFebruary 2008159215972-s2.0-4414908741910.1109/ICACT.2008.4494085LeeN.-Y.WangC.-C.Yet another wavelet watermarking scheme for copyright protectionProceedings of the 9th IEEE International Conference on E-Commerce TechnologyJuly 20074214242-s2.0-4994908498010.1109/CEC-EEE.2007.108SunH.-M.HongC.-J.ChenC.-H.A new approach to feature-based copyright protection of imagesProceedings of the 3rd International Conference on Information Technology: Research and Education (ITRE '05)June 20052332372-s2.0-3374569799810.1109/ITRE.2005.1503111ChangC.-C.ChuangJ.-C.An image intellectual property protection scheme for gray-level images using visual secret sharing strategy20022389319412-s2.0-003660501910.1016/S0167-8655(02)00023-5LuC.-S.HsutC.-Y.SunS.-W.ChangP.-C.Robust mesh-based hashing for copy detection and tracing of images1Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '04)June 20047317342-s2.0-11244351627LeeJ.-S.YoonK.-S.The system integration of DRM and fingerprinting3Proceedings of the 8th International Conference Advanced Communication Technology (ICACT '06)February 2006218021832-s2.0-33750931291YangS.-H.ChenC.-F.Robust image hashing based on SPIHTProceedings of the 3rd International Conference on Information Technology: Research and Education (ITRE '05)June 20051101142-s2.0-3374571206210.1109/ITRE.2005.1503078AhmedF.SiyalM. Y.A secure and robust DCT-based hashing scheme for image authenticationProceedings of the 10th IEEE Singapore International Conference on Communication Systems (ICCS '06)October 2006162-s2.0-46949096813SeoJ. S.HaitsmaJ.KalkerT.YooC. D.A robust image fingerprinting system using the Radon transform20041943253392-s2.0-154252986510.1016/j.image.2003.12.001VilaP. C.2007Barcelona, SpainUniversity Pompeu FabraHsiehS.-L.TsaiI.-J.HuangB.-Y.JianJ.-J.Protecting copyrights of color images using a watermarking scheme based on secret sharing and wavelet transform20083442492-s2.0-77951123800NabiyevV.GünayA.Towards a biometric purpose image filter according to skin detectionProceedings of the Second International Conference on Problems of Cybernetics and Informatics2008PoyntonC.Frequently asked questions about color2014, http://www.poynton.com/ColorFAQ.htmlVoyatzisG.PitasI.Applications of torus automorphisms in image watermarking3Proceedings of International Conference on Image Processing1996237240VoyatzisG.PitasI.Chaotic mixing of digital images and applications to watermarking2Proceeding of European Conference on Multimedia Applications, Services and Techniques (ECMAST '96)May 1996GonzalezR. C.WoodsR. E.20022ndNew York, NY, USAPrentice HallStollnitzE. J.DeRoseT. D.SalestinD. H.Wavelets for computer graphics: a primer, part 1199515376842-s2.0-002930798810.1109/38.376616KeyLawkColors: the human eye is most sensitive to GREEN2014, http://keylawk.blogspot.com/2007/10/colors-human-eye-is-most-sensitive-to.htmlZhaoS.Wavelength of maximum human visual sensitivity2014, http://hypertextbook.com/facts/2007/SusanZhao.shtmlNasirI.YingW.JianminJ.Novel multiple spatial watermarking technique in color imagesInternational Conference on Information Technology: New Generations (ITNG '08)April 20087777822-s2.0-4404909029510.1109/ITNG.2008.66