Parallel Computing on Intensity Offset Tracking Using Synthetic Aperture Radar for Retrieval of Glacier Velocity

Abstract

Synthetic Aperture Radar (SAR) observations are powerful tools to monitor surface's displacement very accurately, induced by earthquake, volcano, ground subsidence, glacier movement, etc. Especially, radar interferometry (InSAR) which utilizes phase information related to distance from sensor to target, can generate displacement map in line-of-sight direction with accuracy of a few cm or mm. Due to decorrelation effect, however, degradation of coherence in the InSAR application often prohibit from construction of differential interferogram. Offset tracking method is an alternative approach to make a two-dimensional displacement map using intensity information instead of the phase. However, there is limitation in that the offset tracking requires very intensive computation power and time. In this paper, efficiency of parallel computing has been investigated using high performance computer for estimation of glacier velocity. Two TanDEM-X SAR observations which were acquired on September 15, 2013 and September 26, 2013 over the Narsap Sermia in Southwestern Greenland were collected. Atotal of 56 of 2.4 GHz Intel Xeon processors(28 physical processors with hyperthreading) by operating with linux environment were utilized. The Gamma software was used for application of offset tracking by adjustment of the number of processors for the OpenMP parallel computing. The processing times of the offset tracking at the 256 by 256 pixels of window patch size at single and 56 cores are; 26,344 sec and 2,055 sec, respectively. It is impressive that the processing time could be reduced significantly about thirteen times (12.81) at the 56 cores usage. However, the parallel computing using all the processors prevent other background operations or functions. Except the offset tracking processing, optimum number of processors need to be evaluated for computing efficiency.

1. Introduction

Both intensity and phase information from Synthetic Aperture Radar (SAR) observations are very useful resources to monitor Earth’s displacement such as flood, earthquake, volcano, ground subsidence, glacier movement and waterlevel change, etc (Amelung et al., 2000; Hanssen, 2001; Hong et al., 2010a; Massonnet and Feigl, 1998; Rignot et al., 1995; Wdowinski et al., 2008; Zebker and Villasenor, 1992). It is wellknown that differentialradarinterferometry (DInSAR) applications which utilizes phase observations of SAR images provide surface’s displacement ranging from cm to mm accuracy according to wavelength of radar signal (Hanssen, 2001; Massonnet and Feigl, 1998). Development of advanced displacement technique such as Persistent ScattererInterferometry (PSI), Small Baseline Subset (SBAS), and SqueeSAR, enables us to retrieve a time-series monitoring of surface’s displacement very accurately (Berardino et al., 2002; Ferretti et al., 2011; Ferretti et al., 2001; Lanari et al., 2004). Because, however, interferometric coherence is often severely degraded due to decorrelation effect like geometric, temporal, volume, and Doppler decorrelation (Hanssen, 2001; Hong and Wdowinski, 2014; Hong et al., 2010b; Zebker andVillasenor, 1992), interferometric applications might be often limited. For example, the temporal and volume decorrelation on glacier and wetland environment, which are varying much faster than other solid earth surfaces, has been a critical factor to apply DInSAR technique to monitor these surfaces’movement. 

Offset tracking method is an alternative approach to estimate movement of ground features between two images (Baek et al., 2018; Nagler et al., 2015; Strozzi et al., 2002). Different from the radar interferometry, the offset tracking with intensity values does not rely on the phase information.Thus, it can be applied when the interferometric coherence was severely degraded by decorrelation effect. It also has advantage to apply straightforwardly withoutrelatively complicated interferometric processing such as phase unwrapping, baseline refinement, etc. Despite its usefulness, there are mainly two limitations: 1) estimation accuracy and 2) computing time. The accuracy of the offset tracking (about 12 ~ 15 cm)is degraded significantly compared with radarinterferometry (from a few mm to a few cm) because, it totally depends on window patch size and spatial resolution in range and azimuth direction (Nagler et al., 2015; Strozzi et al., 2002).Thanksto the development of SAR payload, it can collect surface’s information over wider area with better spatial resolution. Thus, the image size is greatly increased ranging from a few GB to two hundred of GB in each SARobservation (Pasco corporation, 2019; Moreira et al., 2013; Ouchi, 2013). Because the offset tracking calculates the cross correlation with moving window patch in whole images, the computing time should be also greatly increased.

Parallel data processing based on High Performance Computing (HPC) has been developed to solve complicated problems more simultaneously and efficiently saving computing time with multiple computing resources on a network. With the rapid development of computer technology industry, singlecore processor has disappeared even on personal computers, creating a computing environment with multi-core processors. Thus, it is useful to apply the parallel processing to improve computing efficiency.In this study, we propose that a parallel data processing should improve computing efficiency in the offset tracking application with a huge size of SAR observation. We investigate to find out the optimum parallel computing environment in a workstation with 56 core processors in the case of the offset tracking application for glacier ice velocity estimation. In addition, we also examine whether the parallel computing can affect from the window patch size for the offset tracking.

2. Parallel computation

The software to solve mathematical or practical problem has been written for serial computation traditionally. It means that a single processor (or core) in a single computer is used to execute the software. The software can be divided into a series of problems orinstructions asshown in Fig. 1(a).Those instructions are implemented one after anotherin serial computation (Barney, 2010). On the other hand, parallel computation enables to use simultaneously multiple processors (or cores) in a single or multiple computer network environments, which when connected with either physically or by network to solve a more complicated computational problem, the problemcan be broken into a series of instructions to use multiple processors in parallel computing. Thus, each instructions can be executed at the same time on different processors (Barney, 2010) as shown in Fig. 1(b).

Fig. 1. Schematic diagram of (a) serial computation versus (b) parallel processing (modified from (Barney, 2010)).

The parallel computing is based on parallelismwhich is several activities happening simultaneously (Hyde, 1995). The parallel computing has been established by parallel programming using a few languages which can provide relevant library, e.g., FORTRAN, C, or Matlab.The parallel processing has been considered as one of the highest computing method to solve very complex and huge numerical problemssuch as climate, chemicalreactions, human genome, geological activity, etc (Barney, 2010; Hyde, 1995). Since the multiple instructionsin a problem can be divided into small part of instructions, the computing time by the parallel processing can be reduced extremely.There are several parallel programming modelssuch asshared memory, shared threads models(OpenMP), distributes memory (MPI), data parallel, hybrid, single program multiple data (SPMD),multiple programmultiple data (MPMD) according to hardware and memory architectures.

The OpenMP is one of the shared threads models, and is an Application Program Interface (API) which defined by a major computer hardware and software vendors (Barney, 2010; Basumallik et al., 2007; Dagum and Menon, 1998). The OpenMP is based on compiler directive and the OpenMP Fortran API and C/C++ API were released on Oct, 1997 and 1998, respectively. It can be used very easily and simply to use shared-memory parallelism at the multi-platform environmentsuch as Unix andWindows.In this paper, we utilize Gamma software which has been developed based on the OpenMPAPIto investigate the computing performance of the parallel processing in estimation of glacier ice movement with SAR observations(Werner et al., 2000).

3. Dataset and Data processing

We collected two TanDEM-X SAR observations over the Narssap sermia which is a tidal outlet glacier located at the southwestern part of Greenland. The acquisition dates of observations are September 15 and September 26, 2013 with 11 days of time span. The pixel spacing in range and azimuth direction is 0.91 m and 1.94m, and number ofsamples and lines are 12606 and 29194, respectively. The total image size amounts for about 1.5 GB. The image acquisition mode is stripmap mode and polarization of the image is HHpol. Characteristics of the collected SAR images are summarized in Table 1. The amplitude images of both SAR observations are shown in Fig. 2.

Table 1. TanDEM-X SAR data characteristics used in this study

Fig. 2. Amplitude images of two TanDEM-XSARobservations acquired on September 15 and September 26, 2013. The glaciers shown in top and middle parts of the images flow from right to left direction.

In orderto evaluate computing performance with the OpenMPparallel processing,theGamma interferometric software package was used (Werner et al., 2000). The number ofthreads can be set by changing environment variable. We utilized a workstation equipped with 2.4 GHz dual physical CPU processors(total of 28 cores). In thisstudy, we turned on hyper-threading option ofthe CPU, hence 56 cores could be utilized for experiment. The installed memory is 64 GB and the operating system isCentOS 7.3 linux environment at the 200 GB of SSD hard disk composed with RAID level 1. The processing time was estimated by writing python codes.

The main processing step of the offset tracking can be divided into three steps: 1) convert the original SAR image formattoGamma’s own format, 2) co-registration the slave image with respect to the master image, and 3) calculation of offset over whole images with subwindow patch. The computing time was estimated during the above three steps by setting the number of threads of the workstation (e.g. 1, 2, 4, 8, 16, 32, and 56 which is the maximum number of processors). Because, the computing time of the intensity offset tracking would be increasing dramatically by setting up increased window patch size, we evaluated the processing time at the case of 32, 64, 128, and 256 window patch size. Moreover,small window patch size might not be sufficient to estimate offset between two SAR observations due to very fast ice movement or large surface displacement as shown in Fig. 3. Large window patch size often degraded the spatialresolution of the result which related to the accuracy of the ice velocity. Thus, suitable window patch size should be selected before application of the intensity offset tracking.The offset_pwr_tracking2 programin Gamma softwarewas used to calculate intensity cross-correlation offset tracking. The parameter of SLC oversampling factor was 2, and step in range and azimuth pixels was set to 32.

Fig. 3. Retrieval of ice velocity maps by variation of window patch size. The ice velocity which flows slowly can be detected with small window patch size, but it is not possible to calculate the offset between two SAR observations at the fast moving glacier area.

4. Result

1) Elapsed computing time

At first, we evaluated the elapsed computing time in each intensity offset tracking. In the case of generation of SLC image from the original TanDEM-X satellite format, it took about 61.9~64.2 seconds according to the number of processors(Fig. 4(a)).Asthe number of up to 8 processorsisincreased, the processing time was reduced. However, with the case of more than 16 cores, the processing time was slightly increased. In the conversion step, we noticed that the parallel processing does not improve the processing time much. If a number of SAR observations should be processed faster, the number of processors for the OpenMP processing could be set up to 8. In the co-registration step, the processing time could be significantly reduced from 263.1 seconds to 67.5 seconds when window patch size was set to 256 pixels in range and azimuth directions. Regardless of window patch size, similar processing time was estimated (Fig. 4(b)). The effect of the parallel processing was maximized when we applied the 16 processors. However, similar elapsed time was estimated above the 16 processors. The Fig. 4 indicates that the processing time of the offset tracking to calculate the cross-correlation between two SARobservationsismuch longerthan other processing steps. In the offset tracking step with 256 of window patch size, the processing times are 26,344.1, 17,696.0, 9,725.3, 5,311.6, 2,982.8, 2,478.4, and 2,055.3 seconds in each case of 1, 2, 4, 8, 16, 32, and 56 processors.The dramatic reduction of the processing time shows the advantage ofthe parallel processing evidently asshown in Fig. 4(c). Even though the processing time was reduced as the number of processors was increased, it looksthat 16 cores wassufficient to save the processing time in the test data.

Fig. 4. Elapsed computing time (sec) in each intensity offset tracking processing: (a) generation of SLC image from the original TanDEM-X satellite format, (b) coregistration of slave image with respect to master image, and (c) offset tracking using intensity crosscorrelation with different window patch size of 32 (black line), 64 (green line), 128 (red line), and 256 (blue line) pixel in both range and azimuth direction.

2) Difference of elapsed computing time

We calculated the difference of elapsed computing time at the multi-core uses with respect to a single core processing to evaluate the performance of the parallel processing (Fig. 5). The processing time of SLC generation step can be reduced from 0.2 to 1.8 seconds when the parallel processing was applied. However, 0.5 second ofthe processing time wasrequired more at the 56 cores environment (Fig. 5(a)). This result might be because the system resources were fully occupied with the parallel processing. Thus, it should be considered to find optimum number of processors to be used in practical usage.At the co-registration step, about 195.6 seconds of processing time can be saved with 16 core processors using 256 pixel of window patch size. The difference of time was calculated ranging from 91.7 to 195.6 seconds (Fig. 5(b)). As shown in Fig. 5(c), dramatic improvement in time was calculated at the offset tracking step. When full cores were applied for the calculation of the offset with 256 pixels of window patch size, about 24,288.6 seconds could be saved. However, the derivative of the difference in time was not large when more than 16 cores were applied.Thus, the 16 cores could be optimum number of processors for the offset tracking with intensity information in our system configuration.

Fig. 5. Difference of elapsedcomputingtime (sec) compared with single core processor in each intensity offset tracking processing: (a) generation of SLC image from the original TanDEM-X satellite format, (b) coregistration of slave image with respect to master image, and (c) offset tracking using intensity crosscorrelation with different window patch size of 32 (black line), 64 (green line), 128 (red line), and 256 (blue line) pixel in both range and azimuth direction.

3) Ratio of elapsed computing time

Similarly, we calculated the ratio of the elapsed computing time at the multi-core uses with respect to a single core processing (Fig. 6). Almost same ratio between single core use and the parallel processing was found in the SLCgeneration step in the Fig. 6(a).Thus, the parallel processing does not affect the processing time much in this step. As for co-registration step, the processing time with the OpenMP parallel processing can be faster about 3.9 times than a single core usage (Fig. 6(b)). In the offset tracking step, we can reduce the processing time about 12.8 times as shown in Fig. 6(c)).Thus, the parallel processing providesthe greatest performance in the tough computation such as calculation of cross-correlation coefficients.

Fig. 6. Ratio of elapsed computing time compared with single core processor in each intensity offset tracking processing: (a) generation of SLC image from the original TanDEM-X satellite format, (b) coregistration of slave image with respect to master image, and (c) offset tracking using intensity crosscorrelation with different window patch size of 32 (black line), 64 (green line), 128 (red line), and 256 (blue line) pixel in both range and azimuth direction.​​​​​​​

5. Conclusion and Discussion

The offset tracking method is attractive tool to generate two-dimensional displacement map when the interferometric coherence was significantly degraded from de-correlation effect. Since, however, the offset trackingmethod calculatesthe offset comparing intensity information of two input images, it requires intensive computing resourcesin time.In this paper, the efficiency of parallel computing has been evaluated at the aspect of the processing time using high performance computer. We utilized twoTanDEM-X SARobservations overthe Narsap Sermia in SouthwesternGreenland.Atotal 56 of 2.4GHz IntelXeon nodeswere utilized and theGamma software was used for offset tracking method by adjustment of the number of processors for parallel computing.The processing times of offsettracking atthe 256 by 256 pixel window patch size when single and 56 cores were used are 26,344.1 sec and 2,055.3 sec, respectively. It is impressive that the processing time could be reduced dramatically about thirteen times (12.81) at the 56 cores usage.

Although more processors can calculate with less processing time in the offset tracking step, the processing time is almost same at the specific number of processors in other processing step like SLC generation and co-registration. In the SLC generation step from the original satellite format, the parallel processing does not affect the computing time. On the other hand, the processing time can be reduced when we increased up to 16 cores in the co-registration step. However significant improvement could not be achieved with more processors than 16 cores. Due to limited number of processors in our computing environment, we tested with only 56 processors. Although the processing time ofthe offset tracking kept on reducing, the efficiency of the parallel processing with more than 16 cores does not show impressive result. Thus, if the computing resources are not sufficient to utilize full number of processors, the use of 16 cores could be the optimum number of the OpenMP parallel processing environment in the intensity offset tracking. Even though the result using only a workstation was presented,similar experiments were conducted with workstations that have different number of cores and CPU speed. The experiments showed that the most efficient performance could be achieved when 16 cores, which are similarresult ofthis study, were applied.