• Electronics and Telecommunications Trends
• /
• v.38 no.4
• /
• pp.104-115
• /
• 2023

Communication for multiagent reinforcement learning (MARL) has emerged to promote understanding of an entire environment. Through communication for MARL, agents can cooperate by choosing the best action considering not only their surrounding environment but also the entire environment and other agents. Hence, MARL with communication may outperform conventional MARL. Many communication algorithms have been proposed to support MARL, but current analyses remain insufficient. This paper presents existing communication algorithms for MARL according to various criteria such as communication methods, contents, and restrictions. In addition, we consider several experimental environments that are primarily used to demonstrate the MARL performance enhanced by communication.

• International Journal of Internet, Broadcasting and Communication
• /
• v.16 no.3
• /
• pp.205-211
• /
• 2024

Effective warehouse management requires advanced resource planning to optimize profits and space. Robots offer a promising solution, but their effectiveness relies on embedded artificial intelligence. Multi-agent reinforcement learning (MARL) enhances robot intelligence in these environments. This study explores various MARL algorithms using the Multi-Robot Warehouse Environment (RWARE) to determine their suitability for warehouse resource planning. Our findings show that cooperative MARL is essential for effective warehouse management. IA2C outperforms MAA2C and VDA2C on smaller maps, while VDA2C excels on larger maps. IA2C's decentralized approach, focusing on cooperation over collaboration, allows for higher reward collection in smaller environments. However, as map size increases, reward collection decreases due to the need for extensive exploration. This study highlights the importance of selecting the appropriate MARL algorithm based on the specific warehouse environment's requirements and scale.

• Electronics and Telecommunications Trends
• /
• v.35 no.6
• /
• pp.137-149
• /
• 2020

Several multi-agent reinforcement learning (MARL) algorithms have achieved overwhelming results in recent years. They have demonstrated their potential in solving complex problems in the field of real-time strategy online games, robotics, and autonomous vehicles. However these algorithms face many challenges when dealing with massive problem spaces in sparse reward environments. Based on the centralized training and decentralized execution (CTDE) architecture, the MARL algorithms discussed in the literature aim to solve the current challenges by formulating novel concepts of inter-agent modeling, credit assignment, multiagent communication, and the exploration-exploitation dilemma. The fundamental objective of this paper is to deliver a comprehensive survey of existing MARL algorithms based on the problem statements rather than on the technologies. We also discuss several experimental frameworks to provide insight into the use of these algorithms and to motivate some promising directions for future research.

• Journal of the Korea Institute of Military Science and Technology
• /
• v.26 no.4
• /
• pp.361-370
• /
• 2023

Swarm has recently become a critical component of offensive and defensive systems. Multi-agent reinforcement learning(MARL) empowers swarm systems to handle a wide range of scenarios. However, the main challenge lies in MARL's scalability issue - as the number of agents increases, the performance of the learning decreases. In this study, transfer learning is applied to advanced MARL algorithm to resolve the scalability issue. Validation results show that the training efficiency has significantly improved, reducing computational time by 31 %.

• Membrane and Water Treatment
• /
• v.11 no.4
• /
• pp.275-282
• /
• 2020

Hranfa's marl, a local natural mineral, is selected for the decontamination by adsorption of aqueous effluents in textile industry. Its physicochemical characterization is first performed. It is composed mainly of Calcite, Quartz, Ankerite and Muscovite. Its specific surface area is 40 ㎡ g^-1. Its adsorption performance is then tested in batch conditions using an industrial organic dye, Bemacid Red E-TL, as a model pollutant. The measured adsorption capacity of Hranfa's marl is 16 mg g^-1 which is comparable to that of other types of natural adsorbents. A hybrid process is tested coupling adsorption of the dye on marl in suspension and microfiltration. An adsorption reactor is inserted into the circulation loop of a microfiltration pilot using ceramic membranes. This makes possible a continuous extraction of the treated water provided that a periodic replacement of the saturated adsorbent is done. The breakthrough curve obtained by analyzing the dye concentration in the permeate is close to the ideal one considering that no dye will cross the membrane as long as the adsorbent load is not saturated. These first experimental data provide proof of concept for such a hybrid process.

• Economic and Environmental Geology
• /
• v.6 no.3
• /
• pp.171-182
• /
• 1973

Gemorphological and photogeological techniqes are applied to the problem of geological mapping of a semi-arid area, Albudeite, Southeastern Spain. As a result of this, a geological and surface materials map is made which shows the upper Miocene formation, which mainly consists of marl, limestone and sandstone, is further subdivided into three members, i. e. lower, middle and upper, and the post-Miocene deposits were differentiated into seven stratigraphic units, and mapped. The relationships between geology, landforms and land comlexes previously reognized have been reviewed. The methods adopted have proved to be valuable in interpreting and mapping a compex relationship in which highly variable bedrock outcrops and shallow surface materiales produced under sub-aerial conditios.

• Journal of the Korean GEO-environmental Society
• /
• v.17 no.2
• /
• pp.41-49
• /
• 2016

This paper evaluates the interface shear strength of HDPE textured geomembrane. The interface shear strength between textured geomembrane and marl, and textured geomembrane and woven geotextile were measured; and the smooth geomembrane was used to evaluate the effect of "texture" on the interface shear strength. The interface shear strength was measured using a large direct shear testing device under several conditions including the presence of water, and the normal stresses that were 12, 24, 45, 100, 500, and 1,000 kPa. From testing results, it was found that there was meaningful reduction in the interface shear strength in the presence of water, but the effect of normal stress was not clear. The interface shear strength was measured to be significantly different for smooth geomembrane, whose strength was measured to be as small as half that of the textured geomembrane.

• Geomechanics and Engineering
• /
• v.19 no.5
• /
• pp.393-405
• /
• 2019

This paper evaluates the geotechnical and geo-engineering properties of the South Pars Zone (SPZ) marls in Assalouyeh, Iran. These marly beds mostly belong to the Aghajari and Mishan formations which entail the gray, cream, black, green, dark red and pink types. Marls can be observed as rock (soft rock) or soil. Marlstone outcrops show a relatively rapid change to soils in the presence of weathering. To geotechnically characterise the marls, field and laboratory experiments such as particle-size distribution, hydrometer, Atterberg limits, uniaxial compression, laboratory direct-shear, durability and carbonate content tests have been performed on soil and rock samples to investigate the physico-mechanical properties and behaviour of the SPZ marls in order to establish empirical relations between the geo-engineering features of the marls. Based on the experiments conducted on marly soils, the USCS classes of the marls is CL to CH which has a LL ranging from 32 to 57% and PL ranging from 18 to 27%. Mineralogical analyses of the samples revealed that the major clay minerals of the marls belong to the smectite or illite groups with low to moderate swelling activities. The geomechanical investigations revealed that the SPZ marls are classified as argillaceous lime, calcareous marl and marlstone (based on the carbonate content) which show variations in the geomechanical properties (i.e., with a cohesion ranging from 97 to 320 kPa and a friction angle ranging from 16 to 35 degrees). The results of the durability tests revealed that the degradation potential showed a wide variation from none to fully disintegrated. According to the results of the experiments, the studied marls have been classified as calcareous marl, marlstone and argillaceous lime due to the variations in the carbonate and clay contents. The results have shown that an increase in the carbonate content leads to a decrease in the degradation potential and an increase in the density and strength parameters such as durability and compressive strength. A comparison of the empirical relationships obtained from the regression analyses with similar studies revealed that the results obtained herein are reasonably reliable.

• KIPS Transactions on Computer and Communication Systems
• /
• v.11 no.9
• /
• pp.281-288
• /
• 2022

AGVs are often used in industrial applications to transport heavy materials around a large industrial building, such as factories or warehouses. In particular, in fulfillment centers their usefulness is maximized for automation. To increase productivity in warehouses such as fulfillment centers, sophisticated path planning of AGVs is required. We propose a scheme that can be applied to QMIX, a popular cooperative MARL algorithm. The performance was measured with three metrics in several fulfillment center layouts, and the results are presented through comparison with the performance of the existing QMIX. Additionally, we visualize the transport paths of trained AGVs for a visible analysis of the behavior patterns of the AGVs as heat maps.

• Geomechanics and Engineering
• /
• v.18 no.3
• /
• pp.267-275
• /
• 2019

Tabriz is a large Iranian city and the capital of the East Azerbaijan province. The bed rock of this city is mainly consisted of marl layers. Marl layers have some outcrops in the northern and eastern parts of city that mainly belong to the Baghmisheh formation. Based on their colors, these marls are classified into three types: yellow, green, and gray marls. The city is developing toward its eastern side wherein various civil projects are under construction including tunnels, underground excavation, and high-rise building. In this regard, the swelling behavior assessment of these marls is of critical importance. Also, in lightweight structures with foundation pressure less than swelling pressure, several problems such as walls cracking and jamming of door and windows may occur. In the present study, physical properties and swelling behavior of Baghmisheh marls are investigated. According to the X-ray diffractometer (XRD) results, the marls are mainly composed of Illite, Kaolinite, Montmorillonite, and Chloride minerals. Type and content of clay minerals and initial void ratio have a decisive role in swelling behavior of these marls. The swelling potential of these marls was investigated using one-dimensional odometer apparatus under stress level up to 10 kPa. The results showed that yellow marls have high swelling potential and expansibility compared to the other marls. In addition, green and gray marls showed intermediate and low swelling potential and swelling pressure, respectively.