Automatic Cross-calibration of Multispectral Imagery with Airborne Hyperspectral Imagery Using Spectral Mixture Analysis

Abstract

The analysis of remote sensing data depends on sensor specifications that provide accurate and consistent measurements. However, it is not easy to establish confidence and consistency in data that are analyzed by different sensors using various radiometric scales. For this reason, the cross-calibration method is used to calibrate remote sensing data with reference image data. In this study, we used an airborne hyperspectral image in order to calibrate a multispectral image. We presented an automatic cross-calibration method to calibrate a multispectral image using hyperspectral data and spectral mixture analysis. The spectral characteristics of the multispectral image were adjusted by linear regression analysis. Optimal endmember sets between two images were estimated by spectral mixture analysis for the linear regression analysis, and bands of hyperspectral image were aggregated based on the spectral response function of the two images. The results were evaluated by comparing the Root Mean Square Error (RMSE), the Spectral Angle Mapper (SAM), and average percentage differences. The results of this study showed that the proposed method corrected the spectral information in the multispectral data by using hyperspectral data, and its performance was similar to the manual cross-calibration. The proposed method demonstrated the possibility of automatic cross-calibration based on spectral mixture analysis.

1. Introduction

As the number of Earth observation satellites has increased, an increasing amount of remote sensing data has been extended to applications of remote sensing in different fields of study. Remote sensing data acquired from multiple sensors at various acquisition times have been used for continuous data collection within a given time to increase the accuracy of analysis. Because result of remote sensing analysis depends on accurate data and consistent measurement over a given period, remote sensing data gathered from various imaging sensors and under different atmospheric conditions must be on a consistent radiometric scale. Radiometric calibration for the consistent radiometric scale can be conducted through the ground prior to launch, onboard the spacecraft post-launch, and cross-calibration based on reference images of the Earth. Among these methods, cross-calibration is the feasible solution to place both similar and different sensors on a common radiometric scale without the need of field data or sensor information. Hence, cross-calibration could play an important role in interoperability and data fusion (Chander et al., 2013).

Several previous studies performed cross–calibration techniques on multispectral images, such as Landsat TM and ETM. Song (2004) proposed a simple approach to cross-sensor calibration for NDVI indices. In their study, IKONOS and Landsat ETM+ were divided into vegetation and non-vegetation areas by the segmentation and classification of an IKONOS image. The NDVI values in each area were calibrated by histogram matching. Röder et al. (2005) developed the radiometric inter-calibration of Landsat TM and MSS, which was used to perform the normalization of radiometrically uncorrected images using the available knowledge of parameters for the radiometrically corrected image. Teillet et al. (2007) investigated Spectral Band Difference Effects (SBDE), which are significant in cross-calibration between multiple satellite sensors. They also demonstrated cross-calibration requires that the spectral dependencies of the sensor responses and scene illumination, atmosphere, and surface were taken into account. On the other hand, Brook and Dor (2011) used vicarious calibration targets to correct sensor radiance within a short period. Chander et al. (2013) introduced the Spectral Band Adjustment Factor (SBAF), which determines the spectral profile of the target and relative spectral responses between Landsat ETM+ and MODIS images. This study used additional Hyperion data to derive the spectral signature of the target for SBAF computation.

As Chander et al. (2013) presented on their study, hyperspectral images with narrow spectral band range is a good reference data for radiometric calibration of remote sensing data collected by multi-sensors, when the hyperspectral images are radiometrically corrected. For the effective analysis of precise spectral information in hyperspectral images, spectral unmixing or spectral mixture analysis, has been developed for its use in various applications to hyperspectral images (Heinz and Chang, 2001; Franke et al., 2009; Raksuntorn and Du, 2010). Spectral unmixing is the procedure by which the measured spectrum of a mixed pixel is decomposed into a collection of constituent spectra, or endmembers, and a set of corresponding fractions, or abundances, that indicate the proportions of each endmember present in the pixel (Keshava, 2003). Many previous studies on spectral unmixing applied nonnegative matrix factorization (NMF) for remote sensing analysis. Since Paatero and Tapper (1994) and Lee and Seung (1999) introduced the NMF, it has become well known as effective in finding reduced rank nonnegative factors to approximate a given nonnegative data matrix (Berry et al., 2007). Based on NMF, Yokoya et al. (2013) introduced Coupled NMF (CNMF) to fuse the hyperspectral and multispectral images collected from the same sensor. This technique was used to optimize the abundance maps of multispectral images and the endmembers of hyperspectral images by unmixing NMF iteratively so that the fused images resulted intermediate spectral information on similarities and differences between the multispectral and hyperspectral images.

Using hyperspectral data and spectral mixture analysis technique, we present an automatic cross-calibration method to calibrate the multispectral image used in this study. The spectral characteristics of the multispectral image were adjusted using linear regression analysis based on the endmember sets from two images, which were automatically extracted using spectral unmixing techniques. The bands in the hyperspectral image were aggregated based on the spectral response function to reduce the difference in relative spectral responses between the spectral bands of two images.

 

2. Methodology

The method proposed for the cross-calibration of hyperspectral images is divided into three parts. The first section includes the conversion to Top of Atmosphere (TOA) and histogram matching to reference the data and determine the target data. These data need to be calibrated before performing spectral unmixing between the reference data and the target data. The second section is the endmember set estimation. The endmember set estimation between the hyperspectral and multispectral images is performed using NMF based on the spectral unmixing approach. In spectral unmixing, the linear model is generally recognized as acceptable in many real-world scenarios even though it is not always true, such as in collected conditions with strong non-linearity (Keshava, 2003). Both hyperspectral images, which are reference image (REF) and multispectral images, which are target data (TAR), can be unmixed into endmember and abundance as follows:

In Eq. (1), REF is a hyperspectral image, υ represents noise, and emREF and abunREF are endmember spectrum collection and their abundance map of REF, respectively. In Eq. (2), TAR is a multispectral image that needs to be calibrated, emTAR and abunTAR are endmember sets and their abundance map of TAR, respectively. To estimate gain and offset through linear regression analysis, emREF and emTAR require the spectral information about identical materials on the scene. Because it is difficult to estimate emREF and emTAR in the same material directly from Eqs. (1) and (2), we adopted CNMF to calculate the optimized emREF and emTAR between REF and TAR (Yokoya et al., 2012). The origianl CNMF is focused on the image fusion of images collected simultaneously from same sensor system and estimates endmember sets and abundance maps assuming that the images have less atmospheric and radiometric differences (Yokoya et al., 2012). Because the concept of cross-calibration is to calibrate images radiometrically and atmospherically by using a radiometric- and atmospheric-corrected image, we focused to estimate endmember sets between images although the images have high atmospheric and radiometric differences.

Fig. 1 shows the entire process of the endmember set estimation that we proposed. Each NMF unmixing updates the endmember sets or abundance maps of REF and TAR, which was done using multiplicative update rules as follows:

Fig. 1.Process of the endmember set estimation

In Eqs. (3) and (4), em is either emREF or emTAR, and abum is either abunREF or abunTAR. IMG represents REF or TAR in each step. abunT and emT denote the transposition of a matrix abun and em, respectively. emi and abuni are the em and abun in ith iteration step and can be updated to emi+1 and abuni+1 using Eqs. (3) and (4) (Berry et al., 2007).

The initial endmember set of REF for the fi rst NMF unmixing was estimated using vertex component analysis (VCA), which is an endmember extraction algorithm known for its fast processing in estimating the endmember spectrum in hyperspectral images (Nascimento and Bioucas-Dias, 2005). The initial ratio in each pixel of abunREF was 1 divided by the total number of endmembers. Eq. (4) was used with REF as IMG and abunREF as abun to estimate abunREF first. Then emREF and the estimated abunREF were updated by Eqs. (3) and (4) where REF is IMG. The updated emREF was converted to emTAR by multiplying the spectral response function matrix between REF and TAR (BlackBridge, 2012). The updated abunREF was also converted to abunTAR by applying the point spread function by considering the spatial resolution of REF and TAR (Yokoya et al., 2012; Chi, 2013). The second NMF unmixing was performed using the converted emTAR and abunTAR. At this time, emTAR was updated by Eq. (3), where TAR was IMG and abunTAR is abun first. Then emTAR and the estimated abunTAR were updated by Eqs. (3) and (4), where REF is IMG. After the first and second NMF unmixing, the SAM value was estimated to measure the similarity of the updated emREF and emTAR. Because the SAM is insensitive to variations in illumination, it is effective to estimate the similarity of the updated emREF and emTAR from images collected with different radiometric and atmospheric conditions. It uses the vector direction rather than the vector length. SAM becomes closer to zero as two images become spectrally similar. It can be calculated by Eqs. (5), where Vresult1 denotes the generic pixel vector element of the previous result and Vresult2 denotes the generic pixel vector element of the updated result. (Stathaki, 2008).

The second iteration process was performed by re-starting the first NMF unmixing with the updated emREF, not the initial emREF estimated by VCA. The SAM value between the updated emREF and emTAR determined from the second iteration process. This was compared with the SAM value estimated in the first iteration process. If the current SAM value was smaller than the previous SAM value or the iteration loop reached a maximum number of iterations, the iteration loop was ended.

After the optimal endmember set between REF and TAR was estimated, band aggregation was performed based on the spectral response function of REF and TAR. Using emREF after the band aggregation and emTAR from the optimal endmember set, we performed linear regression analysis and calibrated the TAR by applying estimated gain and offset to get calibrated result, TARcalibrated, as follow:

 

3. Study Site and Data

The hyperspectral images collected by the airborne CASI-1500 sensor were used to calibrate RapidEye multispectral images. The CASI-1500 sensor was developed by ITRES Research Ltd. of Canada. This sensor is a pushbroom imaging spectrometer with a spectrum ranging from 380 nm to 1050 nm. The CASI images used in this study were taken in the Sejong-bo area in Sejong-Ri Yeongi-Myun Sejong-Si, Korea on 2 May 2014. Radiometric calibration, geometric correction, and optional environmental calibration were applied to collected stripes of CASI images to obtain orthocorrected images. RapidEye is a German geospatial information provider that operates fi ve observation satellites. It is the fi rst commercial sensor to detect the Red Edge band and provides data with a spatial resolution of 5 m GSD at nadir. The RapidEye scene for this study was collected at Sejong-si, including Sejong-bo, on 1 May 2014. The product level was 1B, which is radiometric- and sensor-corrected and provides imagery as seen from the spacecraft without correction for any geometric distortions inherent in the imaging process. The specifications of the CASI and RapidEye images and the site scene are presented in Table 1, Fig. 2, and Fig. 3.

Table 1.Specifications of CASI and RapidEye images

Fig. 2.Study site 1 displayed in 858.5 nm, 643.3 nm, and 557.2 nm as RGB channel

Fig. 3.Study site 2 displayed in 858.5 nm, 643.3 nm, and 557.2 nm as RGB channel

Registration between the CASI and RapidEye images was performed with manually collected GCPs. Study sites 1 and 2 were selected from areas that included water, vegetation, paddy fields, and urban structures.

In this study, the number of endmembers was set at 30. The maximum number of iterations of the entire loop was five because high numbers of the iteration of the entire loop caused abnormal abundance fractions because of the radiometric difference between the CASI and RapidEye images. The maximum number of iterations in individual NMF unmixing was set at 300 for optimal updating and appropriate time expenses. The threshold value to exit the individual NMF unmixing iteration by using Frobenius norm was set at 0.0001 (Yokoya et al., 2012).

The results of the proposed cross-calibration method were compared with the results of empirical line calibration (ELC) utility in an ENVI program, which performed the cross-calibration with manually collected regions of interest (ROIs); 30 ROIs from the scene were collected manually from invariant features of urban structure, water, forest, paddy fields, and more to process the empirical line calibration. Both results were statically evaluated using the root mean square error (RMSE) and SAM to compare the pixel difference and spectral difference in the calibrated results with the reference data. The RMSE and SAM values were calculated using Eqs. (7) and (5), respectively. In Eq. (7), n is the total number of pixel, and reference (i) and result (i) are the reflectance value of ith pixel in reference and result images, respectively. In Eq. (5), result 1 is reference and result 2 is the calibration result image. The average percent difference was also calculated to compare differences before and after cross-calibration. The average percent difference indicates the degree in percentage of the differences between the pixels in two images. It was calculated by Eq. (8), where reference is the matrix with reflectance values of reference and result is the matrix with reflectance values of calibration result (Chander et al., 2013).

 

4. Result and Discussion

Table 2 presents the results of the statistical evaluation of the cross-calibration of the RapidEye image using the CASI image. The RMSE and SAM indices of the results of ELC and the proposed method were calculated using CASI data and RapidEye-ATCOR data as references. The similarity of the spectral information in the cross-calibration results was evaluated by comparing with the CASI data. RapidEye-ATCOR is an atmospherically corrected version of RapidEye data in reflectance by using ATCOR. It was used as a reference to evaluate the effectiveness of the proposed method as an atmospheric correction tool. The RMSE and SAM indices in Table 3 show the spectral differences in the cross-calibration results compared to CASI and RapidEye-ATCOR. The results of the ELC showed higher RMSE and SAM values than the results of the proposed method did, comparing to the CASI image and RapieEye-ATCOR. The RMSE and SAM indices demonstrate that the results of the proposed method showed fewer spectral variations and fewer differences of reflectance values in the CASI image. The RMSE of the proposed method was increased in study site 2 while that of ELC was decreased, comparing to the results of study site 1. It could be caused by lower variety of extracted endmembers for proposed method because study site 2 had urban structures with high spectral diversity and abnormal reflectance values comparing to study site 1.

Table 2.Comparison of RMSE and SAM indices for cross-calibration results

Table 3.Average % difference before and after cross-calibration

Table 3 presents the average percentage differences between the results and the CASI image before and after cross-calibration, which were used to evaluate the differences in the pixel values before and after calibration. Before the cross-calibration, the average percentage difference between the CASI and original RapidEye images of study site 1 was −0.883%. After the cross-calibration, the average percentage difference between the CASI image and the ELC result was reduced to −0.478%. The average percentage difference between the CASI image and the results of the proposed method was greatly reduced to 0.045%. In study site 2, the average percentage difference was −0.833% before the calibration, but it increased to −0.053% and 0.069% after cross-calibration, using the ELC and the proposed method, respectively. As the SAM values on site 1, the higher value of the average percentage difference in the result of the proposed method for study site 2 could be caused by the spectral diversity and abnormality of urban structures on study site 2. The high spectral diversity and abnormal reflectance values could interrupt to extract endmembers of main components on study site 1, although the components that mainly cover the study site needed to be extracted as endmembers for the efficient cross-calibration. Since the difference of the lower SAM values and the higher average percentage difference value was not relatively high, the cross-calibration results of the proposed method and ELC had similar quality on study site 2.

Fig. 4 displays the spectral graphs of the pixels in three different materials in the study sites. The proposed result of water pixel displayed in Fig. 4 (a) has higher spectral similarity than the ELC result. The forest pixel in study site 2 demonstrated that the proposed calibration method corrected the spectral information in the RapidEye data closer to CASI data than the ELC did. At the shading net in ginseng fields, both the spectral information in ELC and the proposed method were close to the CASI data.

Fig. 4.Comparison of spectral graphs (a) at lake on study site 1, (b) at forest on study site 2, and (c) at shading net in ginseng fields on study site 2

 

5. Conclusion

In this study, we introduced the cross-calibration of a multispectral image using a hyperspectral image. Based on the spectral unmixing technique, the proposed method selected endmember sets from multispectral and hyperspectral images to estimate the optimized endmember sets automatically for identical materials on land. The results showed that the proposed method corrected the spectral variations in the multispectral image based on the hyperspectral image as much as the manual calibration of ELC did. Some results showed that the radiometric corrections performed by the proposed method were better than those performed by the ELC. The proposed method demonstrated the possibility of automatic cross-calibration between simultaneous remote sensing images. Automatic cross-calibration could save time without the need for manual input, such as in the selection of ROIs and the collection of ground data. The proposed method allows for the simple and easy radiometric calibration of remote sensing data if a reference image is available. Furthermore, this method could be developed for the radiometric correction of remote sensing data collection for time series analysis.