PROMISE: A QR Code PROjection Matrix Based Framework for Information Hiding Using Image SEgmentation

Abstract

As data sharing increases explosively, such information encoded in QR code is completely public as private messages are not securely protected. This paper proposes a new 'PROMISE' framework for hiding information based on the QR code projection matrix by using image segmentation without modifying the essential QR code characteristics. Projection matrix mapping, matrix scrambling, fusion image segmentation and steganography with SEL(secret embedding logic) are part of the PROMISE framework. The QR code could be mapped to determine the segmentation site of the fusion image as a binary information matrix. To further protect the site information, matrix scrambling could be adopted after the mapping phase. Image segmentation is then performed on the fusion image and the SEL module is applied to embed the secret message into the fusion image. Matrix transformation and SEL parameters should be uploaded to the server as the secret key for authorized users to decode the private message. And it was possible to further obtain the private message hidden by the framework we proposed. Experimental findings show that when compared to some traditional information hiding methods, better anti-detection performance, greater secret key space and lower complexity could be obtained in our work.

1.Introduction

The sharing of information is expanding with the increasing consumption rate of people in the world today. Because of the main features of QR code, including fast recognition and low cost, QR code is gradually becoming the link between offline and online content as the media for data sharing and collaboration. The traditional barcode only has a small storage capacity for information, so its development is limited. But two-dimensional codes in our time such as the DataMatrix[1], PDF417[2] code and QR code[3] are widely used in our daily life in the field of information storage[4], commodity identification, e-voting authentication[5], digital watermarking[6] and etc.. QR code, as the most common-use two-dimensional matrix code, was first designed in 1994 by the Japanese Denso Wave Corporation and has been adopted as a universal standard specification[7] published by ISO since 2000[8].

Manoj S. Rewatkar and Shital A. Raut[9] introduced some application methods which combine traditional information hiding techniques with QR codes, and made a comparison between these methods. Lin et al. first proposed in [10] that the secret information is hidden in the cover of QR code by using the feature of error correction of QR code. First using a shared key to encode the secret information, and then embedding it in the QR code. The ordinary browsers can only read the QR code content from the QR code, and only authorized users can encrypt the QR code and obtain secret message from it. However, once the encoded secret message is altered, the true secret message cannot be recovered. A method that can be used for authentication is presented in [11]. The QR code in this method is divided into two layers: public layer and private layer. The common layer is the same as the standard QR code storage layer which can be read by anyone. The private layer is created by replacing the black module with a specific texture mode. This texture mode is considered a black module for the standard QR code reader, and can only be read in a specific way. Therefore, not only the capacity of the QR code is increased, but also the sensitivity of the print-and-scan (P&S) process can be used for identity authentication. A two-layer two-dimensional code was also proposed in [12]. It can encode two independent information and read different information from the QR code by changing the scan angle. But its angle must be accurately calculated in advance to correctly scan the information. This method is relatively inapplicable in daily life. An improved application of visual secret-sharing scheme in QR code was proposed in [13], where extending the previous (n, n) sharing mechanism to (k, n) makes the QR code more secure when hiding information. You can also distribute information about secret messages into n shares. Each share is composed of a valid cover[14] of QR code that can be scanned and read by the QR code reader. And the secret message can be obtained by combining the information in all shares. A copy-proof authentication scheme was proposed by Changsheng Chen et al in[15], using the features of the QR code channel model to extract two sets of features. The extracted features are compared with the essential features of the QR code image in terms of frequency and spatial domains. And then a two-stage QR code authentication framework is formed by cascading. Based on the error correction function of the QR code, the secret message is hidden in the modified QR code module [16]. These modified modules act as keys that can only be used by authorized recipients. When the QR code is scanned by an ordinary code scanner, it can be scanned and read normally by using its error correction feature. And it is fantastic that later recovery of the wrong modules happens in the process of extracting secret information, which can effectively reduce the attention of the attacker.

The above studies focus primarily on modifying the image texture of the QR code and using the QR code error correction feature to embed the secret message. The visual image of the QR code will be changed by these methods. In this paper, the PROMISE framework proposed avoids the above deficiencies. While the carrier is a QR code, the suspicion of the attacker is not easy to arouse, and the experimental results show that our work could achieve better anti-detection performance, greater secret key space and lower complexity when compared to some traditional methods of information hiding.

As follows, the paper is organized. In section 2, the QR code's secret information hiding scheme and the secret message extraction process are introduced in detail. In section 3, the experimental results verify the feasibility and effectiveness of the method proposed, and assess and analyze the system's performance. Finally, an overall summary of the full text is made up of the fourth section.

2. The Proposed Method

2.1 Hidden Process of Secret Message

In this paper, we propose a new 'PROMISE' framework to hide information based on the QR code projection matrix in the QR code fusion image, as shown in Fig. 1. First, the public message is encoded into a QR code. Since the information matrix mapped by the QR code consists essentially of (0, 1), it could be converted after logistic scrambling into a scrambled 'new' matrix. According to its features, the matrix is projected onto an image and then the image, which is a grayscale and pixel fusion of the original QR code image of general information and the bottom background image, is divided into segments.

The secret information logic is embedded in the center segment after segment separation. To embed the encoded secret message into other image blocks, the STC algorithm is used. Finally, an art-like image of the QR code is formed. Without changing the characteristics of the QR code itself, our method could make full use of the essential features of the QR code to hide information. The image is more aesthetic overall by a pixel fusion of QR code image and background image. In addition, the hiding capacity of information expands greatly in that there is a more complex texture in the image after pixel fusion. The implementation consists of five sections:

Fig. 1. Hidden process of secret message

2.1.1 QR Code Generation

The public message is encoded into a QR code image in this section. This information is publicly available, that is, anybody can access the original information from the QR code through such a common scanner. The QR code is presented in the form of a matrix, with the binary '1' representing black blocks and the binary '0' representing white blocks at the corresponding matrix element positions. The QR code could, in other words, be seen as an information matrix.

Fig. 2. QR code information matrix mapping

To generate a QR code image including ordinary information, use the QR code open source development library ZXing. Then convert it to a (0,1) matrix that could be regarded as an information matrix.

2.1.2 Matrix Scrambling

It is conducive to the implementation of the next segmentation to obtain some details from the matrix. In order to obtain a scrambled QR code information matrix, we use the Logistic scrambling algorithm to scramble the original QR code information matrix. The scrambling parameters are uploaded as the key to the server after they are encrypted during the matrix generation process.

Fig. 3. Matrix scrambling

Hypothesizing the information matrix of two-dimensional code be Q and its size be n × n (n is the number of rows and columns of two-dimensional code information matrix), x, y represents the horizontal and vertical coordinates of the two-dimensional code information matrix respectively. P _xy is the value of the information matrix at (x, y) .

Logistic chaotic mapping iteration is:

X_k+1 = µX_k × (1 - X_k)       (1)

Within this equation: X_k ∈ (0,1);n = 0,1,⋅⋅⋅, n; µ ∈ (0, 4) , µ is a system control parameter. When µ in the range of [3.569, 4] , the system begins to enter a chaotic state, then take a value from this range as the key for scrambling the QR code information matrix and upload it onto the key server.

Given initial value X₀ = m₀, μ = µ₀ and the number of iterations N , Pseudo-random sequences A = {m₀, m₁,⋅⋅⋅, m_n - 1} can be generated. Then make a 0,1-judgment based on \(\begin{aligned}\left(y_{n}\right)=\left\{\begin{array}{l}0, x_{n}<0.5 \\ 1, x_{n} \geq 0.5\end{array}\right.\end{aligned}\) . Get the binarized sequence B of A , B = {v₀, v₁,⋅⋅⋅, v_n -1}. Hypothesizing two-dimensional code information matrix coordinates \(\begin{aligned}x=\frac{k}{n}\end{aligned}\), y = k mod n, F_xy = B_k, The original two-dimensional code information matrix 𝑄 through the operation:

P_{x, y} = F_{x, y} ⊕ Q_{x, y}       (2)

Then get the scrambled matrix P .

2.1.3 Image Fusion

QR Code is a typical binary image. The hidden data embedding ability would be limited if it were used directly as a carrier, and the detection resistance would be low. We enhance the texture of the QR code image before embedding the hidden message in order to increase the information embedding ability and boost the detection resistance of the QR code image.

Fig. 4. Fusion Image Generation

In this module, a pixel fusion of background image with rich texture and the ordinary QR code image are generated originally[17, 18]. The QR code image after fusion can still be scanned and identified correctly.

Assuming that the pixel value at the (I, j) position of the QR code image is I(I, j) and the pixel value at the (I, j) position of the natural image is j(I, j) , then the pixel value S(I, j) at the (I, j) position of the texture enhanced QR image is calculated as follows.

S(i, j) = α⋅(i, j) + β⋅(i, j)       (3)

In equation (3), the parameter α value is 0.6, and the parameter β value is 0.4. After calculation, a fused grayscale image is obtained.

2.1.4 Image Segmentation

The information matrix is transformed to obtain the information matrix of the scrambled QR code. According to the characteristics of the scrambled QR code information matrix, the image could be partitioned by using the partitioning algorithms[19, 20]. At present in this paper, PCA algorithm[21] is used to calculate the information matrix of the scrambled QR code. PCA is a principal component analysis which is often used to reduce data dimensions while still preserving important features of the original data set[22]. The implementation steps of PCA algorithm are as follows:

The QR code information matrix X after scrambling, size of n × n (n rows and columns of the matrix X ) as an original data set.

Centralization of all original data sets (zero averaging):

\(\begin{aligned}x_{j}^{(i)} \leftarrow x_{j}^{(i)}-\frac{1}{m} \sum_{i=1}^{m} x_{j}^{(i)}\end{aligned}\)       (4)

Calculate the covariance matrix of the original data:

\(\begin{aligned}C=\frac{1}{m} X X^{T}\end{aligned}\)       (5)

Find the eigenvalue of the covariance matrix 𝐶 and its corresponding eigenvector;

The eigenvectors are arranged into matrices according to the corresponding eigenvalues in descending order, and the first k rows are taken to form the matrix P .

Fig. 5. Image Segmentation using PCA​​​​​​​

We use the PCA algorithm in this paper to measure and select the two columns with the highest contribution value in the longitudinal direction of the matrix as the two cutting points. The theory of acquisition of the image's transverse cutting point is the same as that of the longitudinal cutting point. We just need to transpose and do another calculation of the scrambled QR code information matrix. The matrix can be divided into nine parts using four cutting sites as the dividing points of the cutting matrix. Then, after pixel fusion, we project this matrix's block position to the QR code image and guide the split QR code image into blocks. (The method used in this paper is merely a method of calculation and has no physical significance.) QR code images, color images, and fusion images may be the images described in this section. We use the gray image in this paper after the fusion of the pixels.

2.1.5 Secret Information Embedding

We first specify a set of SEL before embedding secret information into each small block of image, and embed the center block obtained by partitioning in SEL. After that, we perform BCH error correction coding on the secret information that needs to be hidden and then use STC[23] hiding algorithm to hide the secret information in every other image block according to SEL embedding logic.

Fig. 6. Secret Information Embedding Process​​​​​​​

STC algorithm is a steganographic model that minimizes embedded distortion. The principle is to find a path where the secret information is embedded with the least distortion through the Viterbi algorithm. Then the secret information is hidden in the carrier along this path to obtain the carrier containing the secret information. This algorithm does not directly embed secret information into the carrier, instead, it calculates the distortion of the original carrier after embedding the secret information to find an optimal hidden path before hiding the secret information. As a result, it can minimize the overall embedding distortion in the process of embedding secret information. In the method proposed in this paper, the STC algorithm is used to embed secret information in the lowest bit plane of a grayscale image. Its approach is as follows:

Introduce two binary vectors X = (x₁,⋅⋅⋅, x_n), Y = (y₁,⋅⋅⋅, y_n) ∈ (0,1)ⁿ, X represents the carrier image, Y stands for a carrier image containing secret message.

Find a target vector Y such that Hy^T= m , and satisfy D(x, y) minimum. Where m is the secret information to be hidden. D(x, y) is the distortion caused by information hiding in the carrier, that is, the cost, defined as:

\(\begin{aligned}D(x, y)=\sum_{i=1}^{n} \rho_{i}\left|x_{i}-y_{i}\right|\end{aligned}\)       (6)

In order to ensure use the number of changes in the sample value to reflect the embedding cost D(x, y) , the cost required to change x_i to embed a message ρ_i is always 1.

Find the sequence y that minimizes D through the Viterbi algorithm. Along the best path, the secret information is embedded in the lowest bit plane of the carrier image, that is, the secret message carrier image Y after the secret message m is embedded.

After the above steps, a fusion QR code image with secret message is finally formed. With a common scanner, this QR code image can be scanned to get public information, and the actual secret message is concealed in the image.

2.2 Extraction Process of Secret Message

In the extraction process, the user extracts the information by using the obtained secret key or uses a dedicated device to process the encrypted fusion QR code image to extract the encrypted information. The matrix transformation module and the image segmentation module in this process have the same processing methods as the secret information hiding process.

Fig. 7. Extraction Process of Secret Message​​​​​​​

2.2.1 Extract QR Code from The Fusion Image

First, the user obtains a fusion QR code image containing secret information and scans it with a common scanner to get the content of the QR code. Then the user uses the original ZXing library to generate a QR code image which is the same as the originally standard QR code image.

2.2.2 Generate The Secret Key

After obtaining the original QR code image, convert it into a QR code information matrix, before authorizing the user to obtain the key through the server. That is, the logistic algorithm private key µ, X₀ in the matrix transformation process, the hidden parameter h of the STC algorithm and the length L of the secret information needed in the secret information extraction process matrix. In order to carry out the correct matrix transformation and secret information extraction.

2.2.3 Matrix Scrambling

The authorized user obtains the logistic algorithm private key µ, X₀ in the key, and performs the same scrambling on the two-dimensional code (0,1) information matrix as in the secret information hiding process to obtain a transformed two-dimensional code information matrix.

2.2.4 Image Segmentation

The authorized user uses the matrix block algorithm (PCA algorithm, etc.) to execute the transformed QR code information matrix. According to the position calculated by the PCA algorithm, the scrambled QR code information matrix can be divided into segments to obtain a block matrix. By doing so, the authorized user can calculate the block information of the matrix, thus he can get the blocked method of the image.

2.2.5 Secret Message Extraction

The authorized user processes the fusion QR code image containing secret information (Art-like QR code image embedded with secret message) after specifying the blocked method. First the central block should be read to get SEL parameter. Then the secret information in each image block can be extracted according to the SEL embedding logic traversal. Finally all of these are combined into a complete secret message.

3. Experimental Results

3.1 Image Quality

The proposed method is simulated using Matlab R2018b. Operating environment is CPU: Intel i7 7500U, running memory is 16GB. For experiment, the objective analysis PSNR, SSIM, NPCR, MSE is first used to evaluate the performance of the proposed method. Then we compared our proposed method with Hugo, S-UNIWORD, LSB algorithms in the same indicators with a various payload.

NPCR measures the percentage of different pixels in two images. If the number measured by NPCR is lower, the two images are more similar[24, 26].

Suppose two images x y, whose i , j are the horizontal and vertical indices of pixels in both images. Define an Array D_{i, j} = 0, if X_{i, j} = Y_{i, j}, otherwise D_{i, j} = 1. The NPCR is defined as[25, 27]:

\(\begin{aligned}N P C R=\frac{1}{H \times W} \sum_{i, j} D_{i, j} \times 100 \%\end{aligned}\)       (7)

where H, W are the dimensions (size) of the image.

Then, using peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), we compare the proposed system, STC-LSB, Hugo, S-UNIWORD and LSB.

PSNR is the ratio of the maximum value of an image to the difference between the image and its steganographic version. The larger the ratio is, the less distortion of the stego-image and the better the quality we can obtain[28, 29].

\(\begin{aligned}P S N R=10 \times \log _{10} \frac{\left(2^{n}-1\right)^{2}}{M S E}\end{aligned}\)       (8)

where is the bit depth, and the grayscale image used in this paper has a value of 8. MSE is the mean square error, and the quality after image processing is calculated. The most direct idea is to compare the difference profile between the processed image and the real image, that is, the visibility of errors, and to evaluate the image quality through visual errors. MSE is an indicator based on this simple and direct idea.

MSE represents the mean square error (Mean Square Error) of the current image X and the reference image Y. H and W are the height and width of the image. The calculation formula is as follows：

\(\begin{aligned}M S E=\frac{1}{H \times W} \sum_{i=1}^{W} \sum_{j=1}^{H}\left(X_{i, j}-Y_{i, j}\right)^{2}\end{aligned}\)       (9)

The structural similarity index (SSIM) is another tool that can be used to measure the similarities between the original image and the stego image. It ranges from 0 to 1, and 1 means that both images are identical. It is expressed as followed, shown in[24, 30].

\(\begin{aligned}\operatorname{SSIM}=\frac{\left(2 m_{o} m_{s}+c_{1}\right)\left(2 \sigma_{o s}+c_{2}\right)}{\left(m_{o}^{2} m_{s}^{2}+c_{1}\right)\left(\sigma_{o}^{2}+\sigma_{s}^{2}+c_{2}\right)}\end{aligned}\)       (10)

Where m, m² and σ are the mean, variance and the standard deviation of the image. The subscripts o, s, represent the original and QR code fusion image respectively. σ_os is the covariance between both images. The constants c₁ c₂, equal to c₁= k₁L and c₂= k₂L receptively where k₁ = 0.01, k₂ = 0.03 and L = 255 is the maximum value of the grayscale image. The embedding capacity is the maximum size of the payload that can be successfully embedded in the container. When the embedding rate is 0.1, the results are shown in the table:

Table 1. PSNR, SSIM, NPCR and running time comparison​​​​​​​

It can be seen that the algorithm proposed in this paper has the highest PSNR and the lowest NCPR compared to several airspace steganography algorithms commonly used in the table. At the same time, the running time of the algorithm framework proposed in this paper is also the shortest and has lower complexity.

3.2 Anti-steganography Analysis

To test the anti-steganography analysis of our proposed system, In this part of experiments, we use ZXing to generate 6000 QR code images and use 6000 natural images of BOSSbase1.01[31] to produce 6000 texture-enhanced QR code images and embed random secret bits to these texture-enhanced images from 0.1 to 0.5bpac. Then we use spam686[32] to extract the 686-dimensional features of these images and use Linear classifier[33] to make a classification. The security of steganographic algorithm is evaluated by Minimum average misclassification rate P_E .

\(\begin{aligned}P_{E}=\min _{P_{F A}} \frac{P_{F A}+P_{M D}\left(P_{F A}\right)}{2}\end{aligned}\)       (11)

P_FA is represents probability of false-alarm, P_MD is represents probability of missed detection. The larger the P_E , the higher the probability of error detection by steganographic analysis, that is, the stronger the ability of the steganographic algorithm to resist steganographic analysis. It is generally believed that if P_E is greater than 0.4, the detection resistance of the method meets the requirements. The results are shown in the figure.

Table 2. Probability of missed detection comparison​​​​​​​

As it can be seen from the results in the table. Under the condition that the embedding rate is 0.1, the method we proposed has a high anti-detection performance and can effectively resist spam686 steganography detection. Under the condition that the embedding rate is 0.1 bpac, the actual number of embedded secret message bits is 16000 bits, which can fully meet the needs of hidden communication. And by using our framework, the anti-steganography detection ability improves comparing with using STC-LSB alone, and small effect achieves on the operating speed of the system.

3.3 System Gain Evaluation

In this part, we analysis the system gains by using the method we proposed, especially in key-space. Key space is the value range of the key. Alvarez et al.[34] suggest that the key space of the encryption system is at least 2¹⁰⁰ to be safe enough.

3.3.1 Matrix Scrambling

Logistic scrambling key space: The encryption algorithm in this article actually uses the initial key from the initial value of the Logistic project x₀ and the parameter value µ . Since these values are both double-precision real numbers, the maximum accuracy is desirable to be 10⁻¹⁵ . And because of the parameter value of the Logistic project µ ∈ (3.6, 4). Therefore, the key space of logistic encryption algorithm in this paper is 0.5 × 10¹⁵× 10¹⁵.

Key sensitivity: The matrix scrambling algorithm used in this article is very sensitive to key changes. When the amount of key change is only ∇ = 10⁻¹⁵ , the correct QR code information matrix is still not available. It can be seen that even if the key changes slightly, the resulting information matrix is still very different from the original information matrix, and the resulting image cutting site cannot be calculated. It shows that the algorithm has good key sensitivity.

3.3.2 Image Segmentation

After obtaining the split location, SEL embedding logic is added to the system, which makes the embedding scheme of secret information very flexible. Users can choose to embed secret message in any number and order in blocks 1, 2, 3, 4, 6, 7, 8, 9. Key space can be increased by A₈¹× A₈²× A₈³× A₈⁴× A₈⁵× A₈⁶× A₈⁷× A₈⁸ when segmenting is added. In summary, the system's overall key space can reach 10⁵⁵ , and far greater than 2¹⁰⁰ , enough to withstand violent attacks such as exhaustive attacks. In addition, the H-matrix height parameter in the STC hiding algorithm is also part of the key, and calculations are not included here.

4. Conclusion

This paper proposes a novel QR code information hiding framework: PROMISE, image segment information hiding based on the projection matrix of QR code. Experimental results show that the method proposed in this paper has the characteristics of high information hiding capacity, small image distortion, and strong anti-steganographic detection, compared to other QR code image information hiding methods. Furthermore, the key space expands greatly due to the addition of scrambling and segmenting. Even if the carrier is intercepted by the opponent, the secret information cannot be obtained without getting the private key entirely, which ensures the security of the secret information.

Any code scanner can obtain the information content of the QR code, but only authorized users with keys can access hidden secret messages successfully. Meanwhile, with others, the PROMISE framework is NOT isolated so that it could be combined with other algorithms of steganography and cryptography. The framework pioneers the way to use the QR code features to guide image segmentation for information hiding, which greatly enhances the flexibility of embedding secret information and maintains the QR code's readability. In matrix transformation & image segmentation, secret information embedding logic and secure key sharing strategy, future work will be improved and perfected. On the hardware, the whole framework would be applied as the system functions are implemented step by step.

However, there are still some limitations with this method proposed in our paper where SEL that is adopted uses some kinds of simple logistic algorithms. And the secret key uploaded to the server is essential for message extraction. Further work has been carried on about secret information embedding logistic and secret key-free scheme.