• IEMEK Journal of Embedded Systems and Applications
• /
• v.16 no.3
• /
• pp.107-111
• /
• 2021

A 3D point cloud map is an essential elements in various fields, including precise autonomous navigation system. However, generating a 3D point cloud map using a single sensor has limitations due to the price of expensive sensor. In order to solve this problem, we propose a precise 3D mapping system using low-cost sensor fusion. Generating a point cloud map requires the process of estimating the current position and attitude, and describing the surrounding environment. In this paper, we utilized a commercial visual-inertial odometry sensor to estimate the current position and attitude states. Based on the state value, the 2D LiDAR measurement values describe the surrounding environment to create a point cloud map. To analyze the performance of the proposed algorithm, we compared the performance of the proposed algorithm and the 3D LiDAR-based SLAM (simultaneous localization and mapping) algorithm. As a result, it was confirmed that a precise 3D point cloud map can be generated with the low-cost sensor fusion system proposed in this paper.

• KIPS Transactions on Software and Data Engineering
• /
• v.12 no.12
• /
• pp.505-518
• /
• 2023

3D point cloud semantic segmentation is a computer vision task that involves dividing the point cloud into different objects and regions by predicting the class label of each point. Existing 3D semantic segmentation models have some limitations in performing sufficient fusion of multi-modal features while ensuring both characteristics of 2D visual features extracted from RGB images and 3D geometric features extracted from point cloud. Therefore, in this paper, we propose MMCA-Net, a novel 3D semantic segmentation model using 2D-3D multi-modal features. The proposed model effectively fuses two heterogeneous 2D visual features and 3D geometric features by using an intermediate fusion strategy and a multi-modal cross attention-based fusion operation. Also, the proposed model extracts context-rich 3D geometric features from input point cloud consisting of irregularly distributed points by adopting PTv2 as 3D geometric encoder. In this paper, we conducted both quantitative and qualitative experiments with the benchmark dataset, ScanNetv2 in order to analyze the performance of the proposed model. In terms of the metric mIoU, the proposed model showed a 9.2% performance improvement over the PTv2 model using only 3D geometric features, and a 12.12% performance improvement over the MVPNet model using 2D-3D multi-modal features. As a result, we proved the effectiveness and usefulness of the proposed model.

• Journal of the Korean Association of Geographic Information Studies
• /
• v.19 no.4
• /
• pp.169-185
• /
• 2016

Recently mean stand height has been added as new attribute to forest type maps, but it is often too costly and time consuming to manually measure 9,100,000 points from countrywide stereo aerial photos. In addition, tree heights are frequently measured around tombs and forest edges, which are poor representations of the interior tree stand. This work proposes an estimation of mean stand height using an image-based point cloud, which was extracted from stereo aerial photo with FUSION S/W. Then, a digital terrain model was created by filtering the DSM point cloud and subtracting the DTM from DSM, resulting in nDSM, which represents object heights (buildings, trees, etc.). The RMSE was calculated to compare differences in tree heights between those observed and extracted from the nDSM. The resulting RMSE of average total plot height was 0.96 m. Individual tree heights of the whole study site area were extracted using the USDA Forest Service's FUSION S/W. Finally, mean stand height was produced by averaging individual tree heights in a stand polygon of the forest type map. In order to automate the mean stand height extraction using photogrammetric methods, a module was developed as an ArcGIS add-in toolbox.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.10a
• /
• pp.422-424
• /
• 2021

In this paper, it leads the approach of fusing multiple RGB cameras for visual objects recognition based on deep learning with convolution neural network and 3D Light Detection and Ranging (LiDAR) to observe the environment and match into a 3D world in estimating the distance and position in a form of point cloud map. The goal of perception in multiple cameras are to extract the crucial static and dynamic objects around the autonomous vehicle, especially the blind spot which assists the AV to navigate according to the goal. Numerous cameras with object detection might tend slow-going the computer process in real-time. The computer vision convolution neural network algorithm to use for eradicating this problem use must suitable also to the capacity of the hardware. The localization of classified detected objects comes from the bases of a 3D point cloud environment. But first, the LiDAR point cloud data undergo parsing, and the used algorithm is based on the 3D Euclidean clustering method which gives an accurate on localizing the objects. We evaluated the method using our dataset that comes from VLP-16 and multiple cameras and the results show the completion of the method and multi-sensor fusion strategy.

• Journal of Auto-vehicle Safety Association
• /
• v.13 no.4
• /
• pp.7-13
• /
• 2021

This paper presents an vulnerable road user (VRU) classification and tracking algorithm using vision and LiDAR sensor fusion method for urban autonomous driving. The classification and tracking for vulnerable road users such as pedestrian, bicycle, and motorcycle are essential for autonomous driving in complex urban environments. In this paper, a real-time object image detection algorithm called Yolo and object tracking algorithm from LiDAR point cloud are fused in the high level. The proposed algorithm consists of four parts. First, the object bounding boxes on the pixel coordinate, which is obtained from YOLO, are transformed into the local coordinate of subject vehicle using the homography matrix. Second, a LiDAR point cloud is clustered based on Euclidean distance and the clusters are associated using GNN. In addition, the states of clusters including position, heading angle, velocity and acceleration information are estimated using geometric model free approach (GMFA) in real-time. Finally, the each LiDAR track is matched with a vision track using angle information of transformed vision track and assigned a classification id. The proposed fusion algorithm is evaluated via real vehicle test in the urban environment.

• Korean Journal of Remote Sensing
• /
• v.35 no.5_1
• /
• pp.727-736
• /
• 2019

In this paper, we present how to create a point cloud for a target area using multiple unmanned aerial vehicle images and to remove the gaps and overlapping points between datasets. For this purpose, first, IBA (Incremental Bundle Adjustment) technique was applied to correct the position and attitude of UAV platform. We generate a point cloud by using MDR (Multi-Dimensional Relaxation) matching technique. Next, we register point clouds based on observation vectors between stereo images by doing this we remove gaps between point clouds which are generated from different stereo pairs. Finally, we applied an occupancy grids based integration algorithm to remove duplicated points to create an integrated point cloud. The experiments were performed using UAV images, and our experiments show that it is possible to remove gaps and duplicate points between point clouds generated from different stereo pairs.

• The Journal of the Korea Contents Association
• /
• v.12 no.9
• /
• pp.40-47
• /
• 2012

Existing art, media art, accompanied by a new path of awareness and perception instrumentalized by the human body, creating a new way to watch the interaction is proposed. Western art way to create visual images of the point cloud that represented a form that is similar to the Pointage. This traditional painting techniques using digital technology means reconfiguration. In this paper, a new appreciation of fusion of aesthetic elements and digital technology, making the point cloud in the form of video. And this holographic film projection of the spectator, and gestures to interact with the video content is presented. A Process of making contents is intent planning, content creation, content production point cloud in the form of image, 3D gestures for interaction design process, go through the process of holographic film projection. Visual and experiential content of memory recall process takes place in the consciousness of the people expressed. Complete the process of memory recall, uncertain memories, memories materialized, recalled. Uncertain remember the vague shapes of the point cloud in the form of an image represented by the image. As embodied memories through the act of interaction to manipulate images recall is complete.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2021.06a
• /
• pp.223-225
• /
• 2021

실감 미디어에 대한 수요가 증가함에 따라 고품질의 실감 미디어에 대한 중요성이 증가하고 있다. 이러한 실감미디어를 제작하기 위해 사용되는 일반적인 기법 중 하나인 Multi View Stereo 는 깊이 영상 추정 및 해당 깊이 영상을 이용하여 3 차원에 point cloud 를 생성하는 fusion 과정을 거치게 된다. 본 논문에서는 다중 시점 영상의 깊이 영상을 정합하는 fusion 과정을 개선하기 위한 방법을 제안한다. 제안하는 방법에서는 깊이 영상, 색상정보를 이용하여 기준 시점의 depth map 을 이용한 fusion 과정을 거친다. 실험을 통하여 제안한 알고리즘을 이용한 결과가 기존보다 개선됨을 보인다.

• Korean Journal of Remote Sensing
• /
• v.24 no.4
• /
• pp.333-340
• /
• 2008

For the indoor spatial modeling by terrestrial LiDAR and the analyzing its positional accuracy result, two terrestrial LiDARs which have different specification each other were used at test site. This paper shows disparity of accuracy between (1) the structural coordinate transformation by point cloud unit using control points and (2) the relative registration among all point cloud units then structural coordinate transformation in bulk, under condition of limited number of control points. As results, the latter had smaller size and distribution of errors than the former although different specifications and acquistion methods are used.

• Journal of Urban Science
• /
• v.7 no.2
• /
• pp.53-60
• /
• 2018

Digital Twin is a technology that creates a photocopy of real-world objects on a computer and analyzes the past and present operational status by fusing the structure, context, and operation of various physical systems with property information, and predicts the future society's countermeasures. In particular, 3D rendering technology (UAS, LiDAR, GNSS, etc.) is a core technology in digital twin. so, the research and application are actively performed in the industry in recent years. However, UAS (Unmanned Aerial System) and LiDAR (Light Detection And Ranging) have to be solved by compensating blind spot which is not reconstructed according to the object shape. In addition, the terrestrial LiDAR can acquire the point cloud of the object more precisely and quickly at a short distance, but a blind spot is generated at the upper part of the object, thereby imposing restrictions on the forward digital twin modeling. The UAS is capable of modeling a specific range of objects with high accuracy by using high resolution images at low altitudes, and has the advantage of generating a high density point group based on SfM (Structure-from-Motion) image analysis technology. However, It is relatively far from the target LiDAR than the terrestrial LiDAR, and it takes time to analyze the image. In particular, it is necessary to reduce the accuracy of the side part and compensate the blind spot. By re-optimizing it after fusion with UAS and Terrestrial LiDAR, the residual error of each modeling method was compensated and the mutual correction result was obtained. The accuracy of fusion-based 3D model is less than 1cm and it is expected to be useful for digital twin construction.