• 전자통신동향분석
• /
• 제38권4호
• /
• pp.104-115
• /
• 2023

Communication for multiagent reinforcement learning (MARL) has emerged to promote understanding of an entire environment. Through communication for MARL, agents can cooperate by choosing the best action considering not only their surrounding environment but also the entire environment and other agents. Hence, MARL with communication may outperform conventional MARL. Many communication algorithms have been proposed to support MARL, but current analyses remain insufficient. This paper presents existing communication algorithms for MARL according to various criteria such as communication methods, contents, and restrictions. In addition, we consider several experimental environments that are primarily used to demonstrate the MARL performance enhanced by communication.

• International Journal of Internet, Broadcasting and Communication
• /
• 제16권3호
• /
• pp.205-211
• /
• 2024

Effective warehouse management requires advanced resource planning to optimize profits and space. Robots offer a promising solution, but their effectiveness relies on embedded artificial intelligence. Multi-agent reinforcement learning (MARL) enhances robot intelligence in these environments. This study explores various MARL algorithms using the Multi-Robot Warehouse Environment (RWARE) to determine their suitability for warehouse resource planning. Our findings show that cooperative MARL is essential for effective warehouse management. IA2C outperforms MAA2C and VDA2C on smaller maps, while VDA2C excels on larger maps. IA2C's decentralized approach, focusing on cooperation over collaboration, allows for higher reward collection in smaller environments. However, as map size increases, reward collection decreases due to the need for extensive exploration. This study highlights the importance of selecting the appropriate MARL algorithm based on the specific warehouse environment's requirements and scale.

• 전자통신동향분석
• /
• 제35권6호
• /
• pp.137-149
• /
• 2020

Several multi-agent reinforcement learning (MARL) algorithms have achieved overwhelming results in recent years. They have demonstrated their potential in solving complex problems in the field of real-time strategy online games, robotics, and autonomous vehicles. However these algorithms face many challenges when dealing with massive problem spaces in sparse reward environments. Based on the centralized training and decentralized execution (CTDE) architecture, the MARL algorithms discussed in the literature aim to solve the current challenges by formulating novel concepts of inter-agent modeling, credit assignment, multiagent communication, and the exploration-exploitation dilemma. The fundamental objective of this paper is to deliver a comprehensive survey of existing MARL algorithms based on the problem statements rather than on the technologies. We also discuss several experimental frameworks to provide insight into the use of these algorithms and to motivate some promising directions for future research.

• 한국군사과학기술학회지
• /
• 제26권4호
• /
• pp.361-370
• /
• 2023

Swarm has recently become a critical component of offensive and defensive systems. Multi-agent reinforcement learning(MARL) empowers swarm systems to handle a wide range of scenarios. However, the main challenge lies in MARL's scalability issue - as the number of agents increases, the performance of the learning decreases. In this study, transfer learning is applied to advanced MARL algorithm to resolve the scalability issue. Validation results show that the training efficiency has significantly improved, reducing computational time by 31 %.

• Membrane and Water Treatment
• /
• 제11권4호
• /
• pp.275-282
• /
• 2020

Hranfa's marl, a local natural mineral, is selected for the decontamination by adsorption of aqueous effluents in textile industry. Its physicochemical characterization is first performed. It is composed mainly of Calcite, Quartz, Ankerite and Muscovite. Its specific surface area is 40 ㎡ g^-1. Its adsorption performance is then tested in batch conditions using an industrial organic dye, Bemacid Red E-TL, as a model pollutant. The measured adsorption capacity of Hranfa's marl is 16 mg g^-1 which is comparable to that of other types of natural adsorbents. A hybrid process is tested coupling adsorption of the dye on marl in suspension and microfiltration. An adsorption reactor is inserted into the circulation loop of a microfiltration pilot using ceramic membranes. This makes possible a continuous extraction of the treated water provided that a periodic replacement of the saturated adsorbent is done. The breakthrough curve obtained by analyzing the dye concentration in the permeate is close to the ideal one considering that no dye will cross the membrane as long as the adsorbent load is not saturated. These first experimental data provide proof of concept for such a hybrid process.

• 자원환경지질
• /
• 제6권3호
• /
• pp.171-182
• /
• 1973

동남(東南) Spain의 지중해(地中海) 연안(沿岸)에 위치(位置)한 Albudeite 지형적(地形的)으로 잘 표현(表現)된 Miocene의 marl 및 석회암층(石灰岩層)과 이를 덮는 Pliocene 및 Ouaternary의 다양(多樣)한 퇴적층(堆積層)이 잘 노출(露出)되어 복잡(複雜)한 분포(分布)를 보이고 있다. 즉(卽) 본지역(本地域)의 중앙부(中央部)를 서(西)에서 동(東)으로 흐르는 Mula강(江)을 사이에 두고 북부(北部)에는 NE 및 NW계(系)의 단층운동(斷層運動)에 의(依)해 이루어진 경근지괴(傾勤地塊)로서의 Upper Miocene의 marl과 석회암(石灰岩) 협층(挾層)으로 된 tableland가 연(軟)한 marl층(層)을 덮는 굳은 아어란상(亞魚卵狀)~아쇄설성(亞碎屑性) 석회암층(石灰岩層)의 capping effect에 의(依)하여 이루어져 있고 지역남부(地域南部)에는 이 tableland의 지표면(地表面)(석회암(石灰岩) cap)과 거의 동사적(同斜的)인 bevelled cuesta가 연(軟)한 marl과 굳은 석회암(石灰岩)(아쇄설성(亞碎屑性) 합(合) Ostracod) 또는 사질암(砂質岩) 협층(挾層)의 차별침식(差別侵飾)에 의(依)하여 이루어져 있어 이들을 지형적(地形的)으로 돌출(突出)된 양(兩) 석회암층(石灰岩層)을 Key bed로 하여 Upper Miocene을 다시 하부(下部), 중부(中部) 및 상부(上部)를 삼분(三分)하였다. 남부(南部) Cuesta의 scarp slope에 이루어진 격심(激甚)한 양곡측면(兩谷側面)과 북부(北部) tablelanl의 남측(南側) footslope에 급재(級在)하는 퇴화구준(退化丘俊)의 침식측면등(侵蝕側面等)에 노출(露出)되는 marl과 이를 덮는 colluvium, alluvium 또는 capped gravel과의 복잡(複雜)한 boundary tracing에 있어서와 또한 Mula강(江) 유역(流域)에 상봉(相逢)한 고도(高度)로 분포(分布)되는 일련(一連)의 단구상(段丘狀) 충적층등(沖積層等)에 대(對)한 층서(層序) 수립(樹立)과 mapping에 있어서는 이들에 대(對)한 지형(地形) 발달사적(發達史的) 이해(理解)와 항공사진해석의 적용(適用)이 매우 효과적이어서 임존지질천(臨存地質踐)에는 Mula강(江)의 channel에 따르는 협장(挾長)한 사기층(四紀層)을 제외(除外)하고는 Upper Miocene (M3) 일색(一色)으로 되어 있었으나 이번 시도(試圖)에 의(依)하여 총(總)9개(個)의 층서적(層序的) 단위(單位)로 세분(細分)하여 mapping할 수가 있었다.

• 한국지반환경공학회 논문집
• /
• 제17권2호
• /
• pp.41-49
• /
• 2016

본 논문에서는 HDPE 표면처리(textured) 지오멤브레인의 경계면 전단거동을 파악하고자 하였다. 표면처리 지오멤브레인과 marl, 그리고 직포(woven geotextile)와의 경계면에서 발생하는 경계면 전단강도를 측정하였으며, 표면처리의 영향을 파악하기 위해 매끈한(smooth) 지오멤브레인과 직포와의 경계면 전단강도를 측정하여 비교 분석하였다. 경계면 전단강도는 대형직접전단 시험기를 이용하여 측정하였으며, 다양한 조건에 대해 거동 변화를 알아보기 위해 수침조건과 수직응력을 변화시켰다. 시험에 사용된 수직응력은 총 6단계로 저압(12, 24, 45kPa)과 고압(100, 500, 1,000kPa)으로 구분하여 적용하였다. 시험결과 수침에 의한 경계면 전단강도의 감소는 유의미한 수준으로 나타났으며, 수직응력의 영향은 불확실했다. 표면처리 여부에 따라 경계면 전단강도는 큰 차이를 보여주었는데 매끈한 지오멤브레인의 경계면 전단강도는 표면처리 지오멤브레인에 비해 절반까지 감소하는 것으로 나타났다.

• Geomechanics and Engineering
• /
• 제19권5호
• /
• pp.393-405
• /
• 2019

This paper evaluates the geotechnical and geo-engineering properties of the South Pars Zone (SPZ) marls in Assalouyeh, Iran. These marly beds mostly belong to the Aghajari and Mishan formations which entail the gray, cream, black, green, dark red and pink types. Marls can be observed as rock (soft rock) or soil. Marlstone outcrops show a relatively rapid change to soils in the presence of weathering. To geotechnically characterise the marls, field and laboratory experiments such as particle-size distribution, hydrometer, Atterberg limits, uniaxial compression, laboratory direct-shear, durability and carbonate content tests have been performed on soil and rock samples to investigate the physico-mechanical properties and behaviour of the SPZ marls in order to establish empirical relations between the geo-engineering features of the marls. Based on the experiments conducted on marly soils, the USCS classes of the marls is CL to CH which has a LL ranging from 32 to 57% and PL ranging from 18 to 27%. Mineralogical analyses of the samples revealed that the major clay minerals of the marls belong to the smectite or illite groups with low to moderate swelling activities. The geomechanical investigations revealed that the SPZ marls are classified as argillaceous lime, calcareous marl and marlstone (based on the carbonate content) which show variations in the geomechanical properties (i.e., with a cohesion ranging from 97 to 320 kPa and a friction angle ranging from 16 to 35 degrees). The results of the durability tests revealed that the degradation potential showed a wide variation from none to fully disintegrated. According to the results of the experiments, the studied marls have been classified as calcareous marl, marlstone and argillaceous lime due to the variations in the carbonate and clay contents. The results have shown that an increase in the carbonate content leads to a decrease in the degradation potential and an increase in the density and strength parameters such as durability and compressive strength. A comparison of the empirical relationships obtained from the regression analyses with similar studies revealed that the results obtained herein are reasonably reliable.

• 정보처리학회논문지:컴퓨터 및 통신 시스템
• /
• 제11권9호
• /
• pp.281-288
• /
• 2022

산업 응용 분야에서 AGV는 공장이나 창고와 같은 대규모 산업 시설의 무거운 자재를 운송하기 위해 자주 사용된다. 특히, 주문처리 센터에서는 자동화가 가능하여 유용성이 극대화된다. 이러한 주문처리 센터와 같은 창고에서 생산성을 높이기 위해서는 AGV들의 정교한 운반 경로 제어가 요구된다. 본 논문에서는 대중적인 협력 MARL 알고리즘인 QMIX에 적용될 수 있는 구조를 제안한다. 성능은 두 종류의 주문처리 센터 레이아웃에서 세 가지의 메트릭으로 측정하였으며, 결과는 기존 QMIX의 성능과 비교하여 제시된다. 추가적으로, AGV들의 행동 패턴에 대한 가시적인 분석을 위해 훈련된 AGV들의 운반 경로를 시각화한 히트맵을 제공한다.

• Geomechanics and Engineering
• /
• 제18권3호
• /
• pp.267-275
• /
• 2019

Tabriz is a large Iranian city and the capital of the East Azerbaijan province. The bed rock of this city is mainly consisted of marl layers. Marl layers have some outcrops in the northern and eastern parts of city that mainly belong to the Baghmisheh formation. Based on their colors, these marls are classified into three types: yellow, green, and gray marls. The city is developing toward its eastern side wherein various civil projects are under construction including tunnels, underground excavation, and high-rise building. In this regard, the swelling behavior assessment of these marls is of critical importance. Also, in lightweight structures with foundation pressure less than swelling pressure, several problems such as walls cracking and jamming of door and windows may occur. In the present study, physical properties and swelling behavior of Baghmisheh marls are investigated. According to the X-ray diffractometer (XRD) results, the marls are mainly composed of Illite, Kaolinite, Montmorillonite, and Chloride minerals. Type and content of clay minerals and initial void ratio have a decisive role in swelling behavior of these marls. The swelling potential of these marls was investigated using one-dimensional odometer apparatus under stress level up to 10 kPa. The results showed that yellow marls have high swelling potential and expansibility compared to the other marls. In addition, green and gray marls showed intermediate and low swelling potential and swelling pressure, respectively.