• Title/Summary/Keyword: Macro Actions

Search Result 21, Processing Time 0.024 seconds

A Method for Learning Macro-Actions for Virtual Characters Using Programming by Demonstration and Reinforcement Learning

  • Sung, Yun-Sick;Cho, Kyun-Geun
    • Journal of Information Processing Systems
    • /
    • v.8 no.3
    • /
    • pp.409-420
    • /
    • 2012
  • The decision-making by agents in games is commonly based on reinforcement learning. To improve the quality of agents, it is necessary to solve the problems of the time and state space that are required for learning. Such problems can be solved by Macro-Actions, which are defined and executed by a sequence of primitive actions. In this line of research, the learning time is reduced by cutting down the number of policy decisions by agents. Macro-Actions were originally defined as combinations of the same primitive actions. Based on studies that showed the generation of Macro-Actions by learning, Macro-Actions are now thought to consist of diverse kinds of primitive actions. However an enormous amount of learning time and state space are required to generate Macro-Actions. To resolve these issues, we can apply insights from studies on the learning of tasks through Programming by Demonstration (PbD) to generate Macro-Actions that reduce the learning time and state space. In this paper, we propose a method to define and execute Macro-Actions. Macro-Actions are learned from a human subject via PbD and a policy is learned by reinforcement learning. In an experiment, the proposed method was applied to a car simulation to verify the scalability of the proposed method. Data was collected from the driving control of a human subject, and then the Macro-Actions that are required for running a car were generated. Furthermore, the policy that is necessary for driving on a track was learned. The acquisition of Macro-Actions by PbD reduced the driving time by about 16% compared to the case in which Macro-Actions were directly defined by a human subject. In addition, the learning time was also reduced by a faster convergence of the optimum policies.

A Performance Improvement Technique for Nash Q-learning using Macro-Actions (매크로 행동을 이용한 내시 Q-학습의 성능 향상 기법)

  • Sung, Yun-Sik;Cho, Kyun-Geun;Um, Ky-Hyun
    • Journal of Korea Multimedia Society
    • /
    • v.11 no.3
    • /
    • pp.353-363
    • /
    • 2008
  • A multi-agent system has a longer learning period and larger state-spaces than a sin91e agent system. In this paper, we suggest a new method to reduce the learning time of Nash Q-learning in a multi-agent environment. We apply Macro-actions to Nash Q-learning to improve the teaming speed. In the Nash Q-teaming scheme, when agents select actions, rewards are accumulated like Macro-actions. In the experiments, we compare Nash Q-learning using Macro-actions with general Nash Q-learning. First, we observed how many times the agents achieve their goals. The results of this experiment show that agents using Nash Q-learning and 4 Macro-actions have 9.46% better performance than Nash Q-learning using only 4 primitive actions. Second, when agents use Macro-actions, Q-values are accumulated 2.6 times more. Finally, agents using Macro-actions select less actions about 44%. As a result, agents select fewer actions and Macro-actions improve the Q-value's update. It the agents' learning speeds improve.

  • PDF

Modeling of unreinforced brick walls under in-plane shear & compression loading

  • Kalali, Arsalan;Kabir, Mohammad Zaman
    • Structural Engineering and Mechanics
    • /
    • v.36 no.3
    • /
    • pp.247-278
    • /
    • 2010
  • The study of the seismic vulnerability of masonry buildings requires structural properties of walls such as stiffness, ultimate load capacity, etc. In this article, a method is suggested for modeling the masonry walls under in-plane loading. At the outset, a set of analytical equations was established for determining the elastic properties of an equivalent homogeneous material of masonry. The results for homogenized unreinforced brick walls through detailed modeling were compared in different manners such as solid and perforated walls, in-plane and out-of-plane loading, etc, and it was found that this method provides suitable accuracy in estimation of the wall linear properties. Furthermore, comparison of the results of proposed modeling with experimental out coming indicated that this model considers the non linear properties of the wall such as failure pattern, performance curve and ultimate strength, and would be appropriate to establish a parametric study on those prone factors. The proposed model is complicated; therefore, efforts need to be made in order to overcome the convergency problems which will be included in this study. The nonlinear model is basically semi-macro but through a series of actions, it can be simplified to a macro model.

Failure Analysis of Circulating Water Pump Shaft in Power Plant (발전 계획에서 순환 물 펌프 고장 분석)

  • Lee, Jaehong;Jung, Nam-gun
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.7 no.1
    • /
    • pp.125-128
    • /
    • 2021
  • This paper presents the root cause failure analysis of the circulating water pump in the 560 MW thermal power plant. A fractured austenitic stainless-steel shaft operated for 24 years was examined. Fracture morphology was investigated by micro and macro-fractographic analysis. The metallurgical analyses including chemical analysis, metallography and hardness testing were performed. The analysis reveals that the pump shaft was fractured due to the reverse bending load with combination of rotating bending load. Corrective actions for plant operator was recommended based on the analysis.

A Study on the Effects of Job Crafting and Innovation Behavior on Organizational Commitment -Focusing on the moderating effect of knowledge sharing- (잡크래프팅과 혁신행동이 조직헌신에 미치는 영향에 관한 연구 -지식공유의 조절효과를 중심으로)

  • Kim, Mi-yeon;Seo, Young-Wook
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.22 no.6
    • /
    • pp.441-452
    • /
    • 2021
  • This work verifies the effect of Job Crafting on organizational commitment through innovative actions, and analyzes the regulatory effect, by setting up knowledge sharing among members as factors that promote Job Crafting and Innovative Action. Depending on the level of knowledge sharing, the indirect effects of Job Crafting from organizational commitment of the members through innovative actions are expected to vary, and the controlled mediated effects are analyzed. Totally, 463 individuals were surveyed, including start-ups, small and medium-sized enterprises, public institutions, and large enterprise workers. The hypothesis was subsequently verified through analysis by applying SPSS 18.0 and SPSS Process Macro. Our results reveal that Job Crafting has a positive impact on the innovation behavior of the members. Secondly, a higher level of knowledge sharing had a greater positive outcome in the member's innovation behavior. Third, all innovative actions positively impacted the organizational commitment of the members. The fourth notable point was that innovation behavior imparted only a moderate impact on the association between Job Crafting and Organizational Commitment. Finally, it was determined that there was no significance in the controlled mediated effect of knowledge sharing on the indirect effects of Job Crafting affecting organizational commitment through innovative actions. Based on the data verifying the hypothesis, we present here the theoretical and practical implications of this study.

The Dynamics of Korean Stock Market in Response to Fiscal and Monetary Shocks Around Foreign Currency Crisis and Stock Market Opening (재정정책과 통화정책의 충격에 대한 한국 주식시장의 동태적 반응에 관한 연구 - 외환위기와 주식시장 개방을 전후하여 -)

  • Jeong, Jinho
    • KDI Journal of Economic Policy
    • /
    • v.27 no.2
    • /
    • pp.239-251
    • /
    • 2005
  • This paper investigates the effectiveness of economic policy on the stock market in Korea around foreign currency crisis and stock market opening. For this purpose, the paper applied SUR technique to a set of monthly data over the period 1982.01 to 2004.12. The study finds the following results. First, for the entire sample period, Korean stock market appears to have effectively incorporated all of the past information about fiscal policy moves. However, the paper finds an evidence that some of the past monetary actions have significant impacts upon current stock returns implying that the information about past monetary moves has been overlooked. Second, there is an evidence to suggest that, after foreign currency crisis, the macro economic policy actions may influence stock market in a different way. In particular, after foreign currency crisis, monetary policy influences stock market in a more delayed pattern while past fiscal policy moves are well incorporated into current stock returns. Third, before stock market opening to foreign investors, some of the past economic policy actions have significant effects on current stock returns. On the contrary, after stock market opening, none of the past macro economic information has significant impact upon current stock returns. The results imply that stock market opening may contribute to the active utilization of economic information for market participants in Korea.

  • PDF

Failure Criterion Including Brittle Damage (취성재의 결함을 고려한 파괴기준에 대한 연구)

  • Yoh, Eun-Gu;Lee, Yong-Shin
    • Proceedings of the KSME Conference
    • /
    • 2001.06a
    • /
    • pp.507-510
    • /
    • 2001
  • Brittle failure mechanism has been well known as growth of initial micro-damage, that causes macro crack and failure in the end. Several precise criteria are suggested recently, based on experiments values in a whole load range. Among them, Mohr-Coulomb's criterion is used widely these days, but it has a big error compared with the real failure behavior since it does not show reciprocal actions of stresses. In this study, a new brittle failure criterion is proposed, which includes the effects of brittle damage evolution by taking a brittle damage parameter specifically. Comparisons between the proposed model and the previous ones are also given.

  • PDF

Progressive collapse analysis of steel building considering effects of infill panels

  • Zoghi, Mohammad Abbasi;Mirtaheria, Masoud
    • Structural Engineering and Mechanics
    • /
    • v.59 no.1
    • /
    • pp.59-82
    • /
    • 2016
  • Simplifier assumptions which are used in numerical studies of progressive collapse phenomenon in structures indicate inconsistency between the numerical and experimental full-scale results. Neglecting the effects of infill panels and two-dimensional simulation are some of these assumptions. In this study, an existing seismically code-designed steel building is analyzed with alternate path method (AP) to assess its resistance against progressive collapse. In the AP method, the critical columns be removed immediately and stability of the remaining structure is investigated. Analytical macro-model based on the equivalent strut approach is used to simulate the effective infill panels. The 3-dimentional nonlinear dynamic analysis results show that modeling the slabs and infill panels can increase catenary actions and stability of the structure to resist progressive collapse even if more than one column removed. Finally, a formula is proposed to determine potential of collapse of the structure based on the quantity and quality of the produced plastic hinges in the connections.

The Opportunities and Challenges of Implementing BEAM Plus in Hong Kong from the Perspectives of Government and Developers

  • Lau, Ka-ho;Fu, Man-man;Yim, Yik-fung;Zayed, Tarek;Sun, Yi
    • International conference on construction engineering and project management
    • /
    • 2020.12a
    • /
    • pp.333-342
    • /
    • 2020
  • Due to the enhancing environmental concerns worldwide with the need of increasing demand for sustainability of building design, maintenance and operation, key stakeholders including the government and developers in many countries strike for the benefits in implementing the green design and building concepts in constructing, infrastructure as well as the buildings. Different countries have their standards or certifications for green buildings while the adoption rate of BEAM-Plus in HK is relatively less compared with other developed countries such as Europe, USA and Japan. Therefore, in the present research, BEAM-Plus, the beginning assessment method of green standard implemented in HK, will be mainly discussed. Current situation of BEAM-Plus implementation in HK will be reviewed and then adopt a systematic approach via literature review and research paper, questionnaire with Analytic Hierarchy Process (AHP) method to depict the opportunities and challenges from the perspective of government and developers regarding implementing BEAM-Plus in HK and thus investigate the implementation gaps. It is found that for both the macro level of opportunity and challenge, the most important criterion is political, in which the weighting value are 0.3114 and 0.2321 respectively. It is obvious that government plays a critical and significant role in affecting the development of BEAM plus. Technological difficulty is also an important factor that challenging and hindering the implementation of BEAM plus, the weighting value is 0.2194 under challenge hierarchy. More experts and professionals should be imported to Hong Kong to enhance the technique is building green buildings. At the end of this paper, solutions and actions will also be suggested and concluded in alleviating the challenges. Finally, solutions and actions are suggested and concluded in alleviating the challenges. Findings from this research can guide developers to consider adopting green elements, government and Green Building Council in HK to review green buildings' policy.

  • PDF

A Historical Approach to the Development of Democracy and the Archival Society in Korea (한국 기록관리와 '거버넌스'에 대한 역사적 접근)

  • OH, Hang-Nyeong
    • The Korean Journal of Archival Studies
    • /
    • no.11
    • /
    • pp.15-40
    • /
    • 2005
  • 'Governance' is a subject that is widely discussed these days in the government and the academic world. I think that the new concept, 'governance', is a strategy to develop the democracy of the society in its institutional and functional aspects. When we are going to discuss about governance, public records and their relationships, without understanding the meaning of 'public' in Korea, we cannot expect to discuss the matter precisely. It is said that Korean public service sectors are awash with authoritarianism and unreasonableness, and that they are at the center of seething corruption and bribe scandals. It is the legacy that the regimes adopted in the aftermath of the Japanese colonial rule for 35 years. The colonial legacy included not simply the practice of the Japanese colony, but also people who had collaborated the Japanese. The American military government and Rhee, Sung-Man regime also appointed the same officials to government posts. As was the same case in other areas including economy, press, education, politics, law, etc. In this point of view, "Righting historical wrongs", a controversial issue now in Korea, aims at establishing the right relationships between an individual and the public, and eventually laying the foundation of democracy for future generation, a procedure of achieving good governance. Apparently, Korea has made progress in developing democracy, as well as in reforming the government policy and organization. Many independent commissions are performing the projects instead of the government institutions that mandated to do the job, but has not played their roles. The e-government projects in Korea was launched in 2001 by the former administration. However, the confusion of records-management after the promulgation of the act is the result of the lack of strategy and the inconsistency of the vision. Good record-keeping supports effective, transparent and accountable government. Accountability is a key element of good governance. It is a recognized fact that without information, there is no guidance for decision-making, and accountability. Thus without records, there is no accountability for the decisions of actions. Transparency means that the decisions taken and their enforcement are carried out based on led-out rules and regulations. When both accountability and transparency are non-existent, good governance is bound to fail. Archival institutions have to give an attention to inner-governance because of the new trend of archival practice, namely 'macro-appraisal'. This 'macro-appraisal' is a kind of a functions-based approach. However, macro-appraisal focuses not just on function, but on the three-way interaction of function, structure and citizen, which combined reflect the functioning of the state within civil society, that is to say, its governance. In conclusion, the public and democracy are major challenges in the Korean society. The so-called good governance requires good record management. In this respect, records managers are in the front line of instituting good governance, and achieving better public and democracy for future generation, a procedure of achieving good governance.