• Title/Summary/Keyword: robotic vision

Search Result 126, Processing Time 0.02 seconds

Ensemble-based deep learning for autonomous bridge component and damage segmentation leveraging Nested Reg-UNet

  • Abhishek Subedi;Wen Tang;Tarutal Ghosh Mondal;Rih-Teng Wu;Mohammad R. Jahanshahi
    • Smart Structures and Systems
    • /
    • v.31 no.4
    • /
    • pp.335-349
    • /
    • 2023
  • Bridges constantly undergo deterioration and damage, the most common ones being concrete damage and exposed rebar. Periodic inspection of bridges to identify damages can aid in their quick remediation. Likewise, identifying components can provide context for damage assessment and help gauge a bridge's state of interaction with its surroundings. Current inspection techniques rely on manual site visits, which can be time-consuming and costly. More recently, robotic inspection assisted by autonomous data analytics based on Computer Vision (CV) and Artificial Intelligence (AI) has been viewed as a suitable alternative to manual inspection because of its efficiency and accuracy. To aid research in this avenue, this study performs a comparative assessment of different architectures, loss functions, and ensembling strategies for the autonomous segmentation of bridge components and damages. The experiments lead to several interesting discoveries. Nested Reg-UNet architecture is found to outperform five other state-of-the-art architectures in both damage and component segmentation tasks. The architecture is built by combining a Nested UNet style dense configuration with a pretrained RegNet encoder. In terms of the mean Intersection over Union (mIoU) metric, the Nested Reg-UNet architecture provides an improvement of 2.86% on the damage segmentation task and 1.66% on the component segmentation task compared to the state-of-the-art UNet architecture. Furthermore, it is demonstrated that incorporating the Lovasz-Softmax loss function to counter class imbalance can boost performance by 3.44% in the component segmentation task over the most employed alternative, weighted Cross Entropy (wCE). Finally, weighted softmax ensembling is found to be quite effective when used synchronously with the Nested Reg-UNet architecture by providing mIoU improvement of 0.74% in the component segmentation task and 1.14% in the damage segmentation task over a single-architecture baseline. Overall, the best mIoU of 92.50% for the component segmentation task and 84.19% for the damage segmentation task validate the feasibility of these techniques for autonomous bridge component and damage segmentation using RGB images.

Predicting Unseen Object Pose with an Adaptive Depth Estimator (적응형 깊이 추정기를 이용한 미지 물체의 자세 예측)

  • Sungho, Song;Incheol, Kim
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.11 no.12
    • /
    • pp.509-516
    • /
    • 2022
  • Accurate pose prediction of objects in 3D space is an important visual recognition technique widely used in many applications such as scene understanding in both indoor and outdoor environments, robotic object manipulation, autonomous driving, and augmented reality. Most previous works for object pose estimation have the limitation that they require an exact 3D CAD model for each object. Unlike such previous works, this paper proposes a novel neural network model that can predict the poses of unknown objects based on only their RGB color images without the corresponding 3D CAD models. The proposed model can obtain depth maps required for unknown object pose prediction by using an adaptive depth estimator, AdaBins,. In this paper, we evaluate the usefulness and the performance of the proposed model through experiments using benchmark datasets.

Effects of Cultivation Method on the Growth and Yield of a Cucumber for Development of a Robotic Harvester (오이수확용 로봇개발을 위한 재배방식이 생육 및 수량에 미치는 영향)

  • Lee, Dae-Won;Min, Byung-Ro;Kim, Hyun-Tae;Im, Ki-Taek;Kim, Woong;Kwon, Young-Sam;Nam, Yooun-Il;Choi, Jae-Woong;Sung, Si-Hong
    • Journal of Bio-Environment Control
    • /
    • v.7 no.3
    • /
    • pp.226-236
    • /
    • 1998
  • If the lowest leaves of the cucumber were removed or training cultivable method was changed, a computer vision system could divide well the cucumber fruit from the others, and also an end-effector could reach and grip cucumber fruit and cut well its fruit stalk. Therefore, this study investigated whether removal leaves and training cultivable method of a cucumber could affect its growth and yield. They can help to be designed the vision system and the end-effector. A cucumber fruit grew by 6-l5cm long for 2 days regardless of removing leaves. Removal leaves didn't affect growth of cucumber fruit. Number of cucumber fruit was produced within 10% different values by three methods (A, B, C) of removal leaves. The first grade rate (best quality) of 4 B and C was 56.7%, 53.1%, 56.3% respectively. Consequently, proper removal leaves were better than traditional way, which does not remove a leaf, because they make cucumber plant ventilate more freely and absorb more light.

  • PDF

Reliability Evaluation of ACP Component under a Radiation Environment (방사선환경에서 ACP 주요부품의 신뢰도 평가)

  • Lee, Hyo-Jik;Yoon, Kwang-Ho;Lim, Kwang-Mook;Park, Byung-Suk;Yoon, Ji-Sup
    • Journal of Nuclear Fuel Cycle and Waste Technology(JNFCWT)
    • /
    • v.5 no.4
    • /
    • pp.309-322
    • /
    • 2007
  • This study deals with the irradiation effects on some selected components which are being used in an Advanced Spent Fuel Conditioning Process (ACP). Irradiation test components have a higher priority from the aspect of their reliability because their degradation or failure is able to critically affect the performance of an ACP equipment. Components that we chose for the irradiation tests were the AC servo motor, potentiometer, thermocouples, accelerometer and CCD camera. ACP facility has a number of AC servo motors to move the joints of a manipulator and to operate process equipment. Potentiometers are used for a measurement of several joint angles in a manipulator. Thermocouples are used for a temperature measurement in an electrolytic reduction reactor, a vol-oxidation reactor and a molten salt transfer line. An accelerometer is installed in a slitting machine to forecast an incipient failure during a slitting process. A small CCD camera is used for an in-situ vision monitoring between ACP campaigns. We made use of a gamma-irradiation facility with cobalt-60 source for an irradiation test on the above components because gamma rays from among various radioactive rays are the most significant for electric, electronic and robotic components. Irradiation tests were carried out for enough long time for total doses to be over expected threshold values. Other components except the CCD camera showed a very high radiation hardening characteristic. Characteristic changes at different total doses were investigated and threshold values to warrant at least their performance without a deterioration were evaluated as a result of the irradiation tests.

  • PDF

Detection of Zebra-crossing Areas Based on Deep Learning with Combination of SegNet and ResNet (SegNet과 ResNet을 조합한 딥러닝에 기반한 횡단보도 영역 검출)

  • Liang, Han;Seo, Suyoung
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.39 no.3
    • /
    • pp.141-148
    • /
    • 2021
  • This paper presents a method to detect zebra-crossing using deep learning which combines SegNet and ResNet. For the blind, a safe crossing system is important to know exactly where the zebra-crossings are. Zebra-crossing detection by deep learning can be a good solution to this problem and robotic vision-based assistive technologies sprung up over the past few years, which focused on specific scene objects using monocular detectors. These traditional methods have achieved significant results with relatively long processing times, and enhanced the zebra-crossing perception to a large extent. However, running all detectors jointly incurs a long latency and becomes computationally prohibitive on wearable embedded systems. In this paper, we propose a model for fast and stable segmentation of zebra-crossing from captured images. The model is improved based on a combination of SegNet and ResNet and consists of three steps. First, the input image is subsampled to extract image features and the convolutional neural network of ResNet is modified to make it the new encoder. Second, through the SegNet original up-sampling network, the abstract features are restored to the original image size. Finally, the method classifies all pixels and calculates the accuracy of each pixel. The experimental results prove the efficiency of the modified semantic segmentation algorithm with a relatively high computing speed.

Development of Robotic Inspection System over Bridge Superstructure (교량 상판 하부 안전점검 로봇개발)

  • Nam Soon-Sung;Jang Jung-Whan;Yang Kyung-Taek
    • Proceedings of the Korean Institute Of Construction Engineering and Management
    • /
    • autumn
    • /
    • pp.180-185
    • /
    • 2003
  • The increase of traffic over a bridge has been emerged as one of the most severe problems in view of bridge maintenance, since the load effect caused by the vehicle passage over the bridge has brought out a long-term damage to bridge structure, and it is nearly impossible to maintain operational serviceability of bridge to user's satisfactory level without any concern on bridge maintenance at the phase of completion. Moreover, bridge maintenance operation should be performed by regular inspection over the bridge to prevent structural malfunction or unexpected accidents front breaking out by monitoring on cracks or deformations during service. Therefore, technical breakthrough related to this uninterested field of bridge maintenance leading the public to the turning point of recognition is desperately needed. This study has the aim of development on automated inspection system to lower surface of bridge superstructures to replace the conventional system of bridge inspection with the naked eye, where the monitoring staff is directly on board to refractive or other type of maintenance .vehicles, with which it is expected that we can solve the problems essentially where the results of inspection are varied to change with subjective manlier from monitoring staff, increase stabilities in safety during the inspection, and make contribution to construct data base by providing objective and quantitative data and materials through image processing method over data captured by cameras. By this system it is also expected that objective estimation over the right time of maintenance and reinforcement work will lead enormous decrease in maintenance cost.

  • PDF