Object-oriented Classification of Urban Areas Using Lidar and Aerial Images

Abstract

In this paper, object-based classification of urban areas based on a combination of information from lidar and aerial images is introduced. High resolution images are frequently used in automatic classification, making use of the spectral characteristics of the features under study. However, in urban areas, pixel-based classification can be difficult since building colors differ and the shadows of buildings can obscure building segmentation. Therefore, if the boundaries of buildings can be extracted from lidar, this information could improve the accuracy of urban area classifications. In the data processing stage, lidar data and the aerial image are co-registered into the same coordinate system, and a local maxima filter is used for the building segmentation of lidar data, which are then converted into an image containing only building information. Then, multiresolution segmentation is achieved using a scale parameter, and a color and shape factor; a compactness factor and a layer weight are implemented for the classification using a class hierarchy. Results indicate that lidar can provide useful additional data when combined with high resolution images in the object-oriented hierarchical classification of urban areas.

1. Introduction

Remote sensing imagery from a large variety of airborne and spaceborne platforms provides valuable data for mapping, environmental monitoring, and other applications (Aplin and Smith, 2008; Csatho et al., 1998; Serpico and Roli, 1995). However, the majority of applications rely on pixel-based image processes and show unsatisfactory results for high-resolution images (Rego and Koch, 2003).

High-resolution images include rich contextual information, such as spatial and spectral information, but pixel-based classifications have faced substantial difficulties in utilizing this contextual information (Haala and Walter, 1999). Thus, object-oriented classification processing, which combines various information, is implemented to overcome these difficulties based on the classification of objects of various sizes, and with varying spatial and spectral features (Fraser et al., 2005; Gupta and Bhadauria, 2014). For each image object, a meaningful statistic is calculated in an increased uncorrelated feature space using observed shapes and features (Onishi, 2004).

Object-oriented hierarchical classification has been utilized in an efficient classification of forest cover types (McRoberts and Tomppo, 2007). In addition, the classification of tree species from IKONOS images using object-oriented classification has been achieved (Hájek, 2005). Principal component analysis (PCA) is used to preprocess data to reduce abundant data.

Object-oriented classification has been applied in the automated detection of urban changes (Doxani et al., 2012; Walter, 2004), in object recognition (Csatho et al., 1999), and in the classification of shaded urban areas (Zhou et al., 2009). Results have been compared with the geographical information system (GIS), and the possibility of revising the GIS database has been raised. Another GIS revision approach using remote sensing data with eCognition can be found in Benz et al. (2004), in which the authors made an appropriate link between remote sensing data and GIS based on multiresolution and object-oriented fuzzy analyses. A hierarchical image object network can provide a strong link to real-world objects.

This paper presents that the lidar data can provide supplementary information to the aerial image about building boundaries in the object oriented classification. A local maxima filter was used to extract lidar points on buildings to combine with aerial imagery. Fuzzy logic membership values were adopted for the accuracy assessment of the classification.

 

2. Study Area and Dataset

Ocean City, located in the northern part of the coastal area of Maryland, U.S., was selected as the test area since it contains a combination of buildings, roads, coast, and sea. This varied terrain is suitable for classification combining lidar and aerial imagery.

The dataset consists of stereo images and lidar data. An RC20 camera was used to acquire panchromatic aerial images; the camera model is UA9II 3043 and its calibrated focal length is 152.74 mm. The Airborne Topographic Mapper (ATM) laser system was used to acquire digital elevation data. The vertical accuracy of better than 10cm has been achieved by the ATM laser system. Fig. 1 shows an aerial image of the study area and indicates lidar data on the aerial image.

Fig. 1.Aerial image and lidar (red) for the study area

 

3. Object-oriented Classification

Classification is the process of assigning classes to pixels from remote sensing data. In an agricultural scene, for example, each pixel could be assigned the class wheat, rye, barley, or fallow. This is important for remote sensing and GIS work, since the classified image is much easier to incorporate than the original unclassified image. In this study, the fuzzy classification method is used. Fuzzy logic is a mathematical approach to quantifying uncertain statements.

The basic idea is to replace the two strictly logical statements, “yes” and “no,” by a continuous range of values between 0 and 1, where 0 means “exactly no” and 1 means “exactly yes.” All values between 0 and 1 represent a more or less certain state of “yes” and “no.” Thus, fuzzy logic is able to emulate human thinking and can even take into account linguistic rules. Fuzzy classification systems are well suited to handling the ambiguity involved in extracting information from remote sensing data. Parameter and model uncertainties are considered using fuzzy sets defined by membership functions. Instead of the binary “true” and “false” multivalued fuzzy logic allows transitions between “true” and “false.” Additionally, there are more or less strict realizations of the logical operations “and” or “or.” The membership degree for each land cover or land use class is given for each object. This enables a detailed performance analysis and gives an insight into the class mixture for each image object. This is a major advantage of fuzzy classification. In this study, a fuzzy nearest neighbor classifier is implemented. This is called object-oriented classification, and involves the grouping of neighboring pixels into meaningful elements.

The proposed method is based on integrating information from lidar and aerial images for object-oriented classification of urban areas. MATLAB was used for local maxima filtering and for the conversion from point data to images. ERDAS IMAGINE was used for the coordinate transformation and multiresolution segmentation to implement object-oriented classification. The method follows five major steps. First, lidar points on the building are extracted using the local maxima filter. Second, the building segmentation data from the lidar points are converted into images. Third, the coordinate systems are unified. Fourth, multiresolution segmentation and classification are implemented. The final step is accuracy assessment.

To provide information on the building location, lidar points on the building should be extracted from original lidar data. The basic concept is that the height of lidar points on the building are higher than neighboring ground points. The local maxima filter extracts points over a threshold given by a minimum height plus a threshold value. In this paper, the threshold value was 6 m, which is roughly equivalent to a two-story building or house. The size of the filter was 30 m by 30 m, since this exceeds the largest building and is an appropriate size for detecting only points on the building. The large filter size causes inaccuracies in the extraction as the real ground may have a tilt and may not be at sea level. This means that the large filter could extract a point on the hills as a building point and, in low-lying areas, neighboring ground points could be detected as building points.

Fig. 2 illustrates the basic concept of the local maxima filter.

Fig. 2.Local maxima filter

Fig. 3 shows an aerial image and lidar points on a building, and a large-scale aerial image and lidar points for the buildings that represent the commercial area of Ocean City.

Fig. 3.Aerial images and lidar points (red) on buildings

The building segmentation data are point data, which cannot be processed by eCognition. Therefore, the point data are converted into images. The image has the same resolution and dimensions as the aerial image for ease of use.

The size of a single pixel on the ground is around 0.09 m. Therefore, when lidar points on the buildings are converted to image format, an area of 11 by 11 pixels around one lidar point (around 1 m) is classed as a building area. Since lidar points are not dense enough to represent the whole building as a polygon, this assumption is required to provide the building boundary information.

Fig. 4 shows the image of building-segmented lidar data in the commercial area.

Fig. 4.Aerial image and building-segmented lidar data

 

4. Results and Discussion

4.1 Classification of commercial and residential areas

In the test commercial area, there are two buildings, roads, a parking lot, a white impervious surface, a grey impervious surface, shadow, sand, and trees. Thus, the upper level of the class hierarchy (i.e., the parent class) consists of impervious surfaces and non-impervious surfaces. The detailed class hierarchy is shown below in Fig. 5. The class hierarchy of the residential area is somewhat different to that of the commercial area since the white and grey impervious surfaces do not exist in the residential area. In addition, impervious color surfaces can be categorized into white- and grey-roofed houses, as shown in Fig. 5. The class hierarchy is the framework of object-oriented classification that creates the knowledge base for a given classification. It reduces the complexity and redundancy of classification. Fig. 6 shows the classification of an aerial image and building segmentation data from lidar.

Fig. 5.Class hierarchies for commercial and residential areas

Fig. 6.Classifi cation of an aerial image and building segmentation data from lidar in the commercial area

The building and the building facade have lidar points, but the pixel values from the aerial image are different when compared with the lidar data. Therefore, the building and the building facade were classified as other classes. In addition, the “building facade” and “tree” classes have similar characteristics since they both involve lidar points and similar pixel values in the aerial image.

Fig. 7 shows the classification of an aerial image and building segmentation data from lidar in the residential area.

Fig. 7.Classification of an aerial image and building segmentation data from lidar in the residential area

4.2 Accuracy assessment

Classified image objects are not just assigned to a single class; a detailed list with the membership values of each of the classes is contained in the class hierarchy. An image object is assigned to the class with the highest membership value, as long as this highest membership value is equal to or greater than the minimum membership value. An important issue is the detailed analysis of the accuracy after classification.

When using fuzzy classification methods, objects can belong to several classes with different degrees of membership in the case when class descriptions overlap. Thus, to evaluate the reliability or stability of classes, it is necessary to survey the different degrees of membership of the classified objects.

The statistics in Table 1, 2, 3 and 4 show the mean, standard deviation, and minimum and maximum fuzzy logic membership values for the commercial and residential areas. The fuzzy membership values are between 0 (maximum ambiguity) and 1 (maximum certainty). Membership function of each class is provided by eCognition software.

Table 1.Statistics for all objects in the classification using image only in the commercial area

Table 2.Statistics for all objects in the object-oriented classification in the commercial area

Table 3.Statistics for all objects in the classification using image only in the residential area

Table 4.Statistics for all objects in the object-oriented classification of the residential area

The overall accuracies reach 96.37% and 97.54% in the commercial and residential areas, respectively as described in Table 2 and Table 4. The overall accuracies of the pixel-based classification using only aerial imagery are 76.47% and 71.15% in the commercial and residential areas, respectively as described in Table 1 and Table 3. Overall accuracy has been improved by combining aerial image with lidar. Lidar data provides supplementary data to the aerial image for the information about building areas.

 

5. Conclusion

From the experiment, it can be seen that the object-oriented classification accuracy has been improved by combining information from aerial images and lidar. Object-oriented classification can implement automated classification using specialized membership functions. A local maxima filter was used to extract lidar points on buildings and these point data were converted to an image for processing in eCognition. The image of the building-segmented lidar data has the same resolution and dimensions as the aerial image. This image can provide supplementary data to the aerial image to provide information on building boundaries. Finally, fuzzy logic membership values were adopted for the accuracy assessment of the classification. Lidar offers accurate position and height information, but includes less information on the features’ geometrical shapes. Thus, the extraction of building points was used to provide supplementary data on the features’ geometrical shapes in this study, unlike other object-oriented classification studies, which usually use lidar DEM data.