A Case Study: Design and Develop e-Learning Content for Korean Local Government Officials in the Pandemic

Abstract

e-Learning content can be defined as digital content to achieve educational goals. Since it is an educational material that can be distributed in offline, online, and mobile environments, it is important to create content that meets the learner's education environment and educational goals. In particular, if the learner is a public official, the vision, philosophy, and characteristics of each local government must reflect. As non-face-to-face online education expands further due to the COVID-19 pandemic, local governments that have relied on onsite education in the past urgently require developing strong basic competency education and special task competency content that reflect regional characteristics. Such e-learning content, however, hardly exists and the ability to independently develop them is also insufficient. In this circumstance, this case study describes the process of self-production of e-learning content suitable for Busan's characteristics by the Human Resource Development (HRD) Institute of Busan City, a local government. The field of instructional design and instructional technology is always evolving and growing by blending technological innovation into instructional platform design and adapting to the changes in society. Busan HRD Institute (BHI), therefore, tried to implement blended learning by developing content that reflected the recent trend of micro-learning in e-learning through a detailed analysis. For this, an e-learning content developer with certain requirements was selected and contracted, and the process of developing content through a collaboration between the client and developer was described in this study according to the ADDIE model of Instructional Systems Development (ISD).

1. Introduction

The spread of the COVID-19 pandemic has brought radical changes throughout the social system. New methods are being tried in almost all social structures and management, such as society, culture, economy, and industry. These changes do not end in a short period of time, but are promoting major changes in society as a whole. In the field of human resources administration of government institutions, the conversion to online non- face-to-face methods is accelerating beyond 2020 in selection tests, service systems, and work methods. Above all, the rapid non-face-to-face cultural phenomenon intertwined with the digital revolution is an opportunity for the rapid development of online learning and a change of perception in the field of education.

However, the Republic of Korea is finding the best solution while quickly responding to various environmental crises and digital revolutions. As [1] predicted earlier, the central government of Korea provides the ubiquitous access function of the cloud platform, and the local government selects necessary courses and connects them to their own platforms and utilizes them as needed. In the field of civil servants HRD, National Human Resource Development Institute (NHI) has been developing various online educational content and distributing them to government and public institutions. In particular, NHI, which operates the Government e- Learning Platform (GLP), is seeking institutional and structural changes in education and training, such as building an 'Intelligent Open Platform' in preparation for the post Corona era [2][3]. GLP's e-learning content is jointly using by 110 ministries in the education sector and 90 pan-governmental organizations (central & local administrative organizations, or educational institutions).

Institutions are contemplating the systematic management of e-learning for civil servants from various perspectives. It includes strategic and institutional aspects of education and training for civil servants, education demand of institutions, operating staffs, and learners. However, many local HRD institutions only share and utilize the e-learning content provided by GLP, but lack in-house development capabilities in various aspects such as manpower, time, and budget. Therefore, the need for preemptive response by developing e-learning content suitable for the characteristics of local governments has grown.

This is a case study of a pilot project (Project #1) for the Busan Metropolitan City Government's HRD Institute (BHI) to develop and operate its own content needed, as the past onsite offline courses were canceled or rapidly switched to online classes during the pandemic. The process of developing the new content that can be directly developed and managed by Busan City, the local government, was described and analyzed in accordance with the representative instructional design ADDIE model. The ultimate purpose of this case study is to provide reference samples so that local governments can analyze the curriculum for e-Learning content that is not provided by GLP and self-produce content suitable for the characteristics of local governments.

2. Background

2.1. Learning Environment and Status

BHI is annually conducting on-site face-to-face classes or e-learning education in three parts: ‘basic competency’, ‘task competency’, and ‘future growth competency’. The institute had planned a total of 376 courses in 2020, of which 87 courses (15.4%) were annually for face-to-face learning and 479 courses (84.6%) for e-learning. However, BHI completely suspended offline courses except for e-learning since the first outbreak of COVID-19 in Korea on January 20, 2020. Table 1 shows the changes since then.

Table 1. Changes in the Number of Courses and Enrollment (Busan HRD Institute)

^1. It was established based on the analysis, which was continued during 2020 under COVID-19.

^2. Many originally offline courses still has been conducting as synchronous online class using ZOOM.

^3. This is organized as asynchronous Anytime Learning (ATL) and is a self-directed online learning designed to give students the flexibility to learn at their own pace (Over 1,500 private rental micro-learning content not included).

The curriculum was revised six times in 2020 until the end of the year due to the changing circumstances under COVID-19. Of the 77 courses planned face-to-face for 2020, 41(48.2%) were replaced with online classes using ZOOM. As a result, by the end of 2020, 10 courses reduced and learners decreased by 4, 031(52.2%). Contrary, in the case of e-Learning courses, 138(28.8%) courses were added and the learners increased by 9, 697 (13.7%). In the end, the response to the pandemic in 2020 shows that it was rushed to increase only the number of learners for limited online courses and content. As the pandemic continues, the 2021 plan added only 58(9.4%) online courses, but learners increased even more by 42, 937(53.5%) as shown in Table 1.

In the summer of 2020, BHI quickly launched an e-learning content development plan that could replace some face-to-face courses. To develop content to be utilized in connection with the establishment of the e learning management platform, we secured KRW 50 million of the project cost and the outsourcing development process started. The scope of the specific project is the development of one course ‘Understanding Busan, the Maritime Capital’, and it is an e-learning content consisting of 9 lessons of 15 to 18 minute. When development is complete, it is ported to a new learning management system (LMS) and designed to be operated from 2021. The purpose of this project is to preemptively respond to the demand for e-learning content development due to the expansion of online education in the post-corona era, and then to strengthen the competency to develop content that meets the latest trends on its own.

2.2. Methodology and Trend

As shown in Table 2, the specific development phase started in September and proceeded in line with the ADDIE timeline [4], with the goal of completing the development process by December 2020. Clients and developers selected as partners worked closely together to select experts to have content gathering and lectures, course design and development since October. As emphasized by [5], it is very important to systematically extract the opinions of experts. Therefore, the representativeness of the participants, their expertise, the total number of participants, and the authenticity of participation and responses were carefully considered throughout the entire project using the Delphi technique [6][7].

In the era of the new normal, local governments must also quickly reflect national strategies and policies, such as the Korean New Deal policy. Busan has already established and implemented a Busan version of New Deal policy in accordance with the government's New Deal at a time of rapid change with a new culture or digital transformation. BHI needed e-content for public officials that could quickly respond to changes to a platform intelligent city, a people-centered technology city, and a digital-based administrative city that Busan was aiming for [8].

Table 2. Timeline for the Pilot Project (‘Understanding Busan, the Maritime Capital’)

The coursework ‘Understanding Busan, the Maritime Capital’ BHI have chosen to develop for e-learning had been one of the 87 onsite courses until COVID-19. However, in order to overcome the pandemic environment, macroscopic curriculum analysis and course re-planning were conducted first (See 3.1). Also, microscopic analysis was performed concurrently to derive course-specific goals and targets, learning strategies, development types, flowcharts, and directions. This is because it was necessary to introduce ‘blended learning’, which urgently needed the convergence of online education and on-site education. BHI wanted to explore alternative ways to better achieve learning effectiveness by using it.

Education companies have already rapidly converted onsite lectures to non-face-to-face by grafting major technologies of the 4th Industrial Revolution, and started to develop and utilize AI-based personalized learning management platform in connection with e-classes [9]. Of course, local government educational institutions are following these changes eagerly. In the case of Seoul and Gyeonggi Province, they started to develop and manage their own e-learning content. BHI also started to strengthen its competency to develop its own specialized e-learning content in preparation for the prolonged non-face-to-face education system based on such environmental change analysis and benchmarking.

ATD (Association for Talent Development), the world's largest HRD conference, presented digital learning, HRD, flipped learning, micro learning, AR/VR learning, design thinking, and gamification as recent notable trends. Among them, micro-learning as an educational method is a technology that delivers key educational content and messages with short content of five to ten minutes rather than the traditional one-hour e-learning content. It is more attractive to busy office workers and millennials who accustomed to short videos rather than long content. This reflects the fact that long content such as existing online education content has low concentration and minimal learning effect for the generation already familiar with short videos [10]-[11].

The effectiveness of these formats and methods reveals in the implementation phase, the time learners spend sitting in front of a computer to take an online course. This is the most well-known phase in the ISD process that most non-designers consider. Evaluation and feedback should be conducted for the next ten percent improvement of the project [4]. Waiting for that last ten percent might take longer than the total time allocated for the project, or one more year. Therefore, BHI empirically verified all processes using Donald Kirkpatrick’s four-step evaluation model [12], but this was not specifically mentioned in this paper. While this project was in progress, next the other started, so the evaluation and feedback reflected in the entire process of the next project as well as this one (See table 6).

3. The Case Study

3.1. Curriculum Investigation & Needs Analysis

The analysis of the overall implementation and curriculum investigation for the establishment of the 2021 plan were carried out in parallel with the Analysis-Design-Development process of this project. Along with the changes in the educational environment such as digital transformation revealed in the latest HRD trend analysis in [8]-[11], BHI conducted needs analysis of public officials in Busan from July 13 to 17, 2020. The target audience was Special Matter Experts(SMEs) and public officials in Busan city, local county, and public institutions. The survey methods were web-survey, Focus Group Interview (FGI), and extant data analysis. In the case of the learner survey, 4, 358 out of total 4, 490 targets for each course responded (response rate 97.1%). Specifically, 2, 225(51.1%) city officials, 1, 925(44.2%) county officials, and 208 employees in public institutions (4.8%) responded to the survey. FGI was conducted with 15 SMEs (2 city councilors, 9 city sub- professionals, 4 academic experts).

The number of enrolled online course learners increased steadily from 10, 984 in 2018, to 29, 502 in 2019, to 80, 257 in 2020, an increase of about 631 percent in three years. Consequently, BHI established the ‘e- Learning Development Plan’ including four strategic goals based on the results of needs analysis and collecting opinions from 15 SMEs who have diversity in the digital age. This plan includes ten tasks such as diversification of e-content or offline learning, promotion of customized self-directed learning, and establishment of active e learning infrastructure. As such, in accordance with the demand for e-learning courses increases, the structure of outdated content has been precisely analyzed to further develop a total of 2, 077(72.2%) courses over the past three years, and strived to keep them up to date.

Table 3 shows the changes due to the curriculum shift in a pandemic situation. In the meantime, BHI shared most of the content provided by NHI or rented private content for a fee. BHI had set 289 GLP joint utilization courses (200 e-learning, 89 MOOC) and 328 private e-learning courses in 2020 and 75, 530 learners completed them. However, in 2021, the institute reduced 107 public content and added 162 private rental e classes instead based on in-depth analysis results. In addition, BHI offered 1, 500 additional rental micro learning contents not included in the statistics. The city also has been operating the '4U Danbi(a good rain)' course in order to strengthen responsiveness to future changes from June 2020 in connection with the city's administrative portal. The courses were all composed of the latest micro-learning trend content and offered six weeks of content covering issues and trends in a variety of fields, including management, leadership, humanities, history, and innovation.

Table 3. Changes in the e-Learning Statistics in 2020-2021 (Busan HRD Institute)

This online learning for public officials in Busan is operated in the following three categories. First, it is the BHI cyber education system (http://busan.nhi.go.kr), which jointly utilizes NHI's GLP (http://e- learning.nhi.go.kr) content. BHI selects from among the content of the NHI and uses it in connection with links, mainly providing common content related to public officials and administrative work. The reason is that local governments or individual faculties lack the time, resources, or technical expertise to develop quality asynchronous video lectures [13]. Second, BHI use the self-development e-learning link system (http://bshrd.runningplus.net). A private company is selected every year under the 'Busan Metropolitan City Public Official e-Learning Content Use Agreement', and e-learning content is provided by the server of the company. It is mainly related to language learning or obtaining a certificate. Lastly, the BHI's Integrated Education Management System (http://bs.lms.go.kr). Now, the development of a cyber education platform is in progress as a second-phase development project. It is developed by following GLP's content development guidelines, and e-learning content developed by BHI or other institutes can be loaded and used from 2021.

HRD for public officials is characterized by the importance of strengthening ‘basic competency’ education for internalization of government tasks, core policies, and public values and character. In the list of private rental content, however, there is absolutely nothing about the basic and task competencies for public officials. The NHI e-learning platform also has been only providing the central government's core tasks or policies, and there are no e-learning content dealing with each local government’s core tasks or policies. As a result, Busan has always relied on face-to-face education for special 'task competency' fields such as maritime, fishery, and port logistics, which are characteristic of Busan. As such, as the needs of learners and sub-institutions cannot be met under this pandemic, competency education of local governments through e-learning is inevitably imbalanced. Consequently, BHI decided to establish a learning strategy based on the results of analysis of the demands of public officials and the extant data, and to develop blended content in stages.

3.2. Designing the Course

BHI has set priorities for content development and started to develop 'Understanding Busan, the Maritime Capital' in the 'task capability' area as the first pilot project in 2020. And then, the content development of 'Understanding 2030 World Expo and Joining' and 'Busan City Vision and Key Initiatives', which are urgently needed micro-learning content, led to the 2nd and 3rd projects in 2021. As shown in Table 3, the implementation and evaluation phases of these projects has been in progress since the end of 2021. In consonance with the result of these project, it was decided to sequentially develop two more courses: ‘Understanding Shipping and Port Logistics’, ‘Blockchain and Public Administration’. This study is about the first pilot project, and introduces the process of designing and developing blended e-learning content in which micro-learning was transformed and reconstructed.

3.2.1. Transformation of e-Learning into Micro-learning

Since the ATD 2016 ICE (International Conference & EXPO), the keyword ‘micro-learning’ appears frequently, and its effectiveness and application method are attracting attention. ‘Micro-learning’ is a type of informal learning, in which people frequently access and learn anytime, anywhere using short units of content [14]. Micro-learning, also called bite-size learning, is to enable “properly convey one concept and apply it to business” instead of “squeezing a large amount of content”. It can be used for informal training with a focus on performance gain or to teach large, complex material broken down into manageable pieces [15].

The article [14] outlined seven attributes that define micro-learning units and content, including: time, content size, curriculum type, process, modularity, and learning type. Apart from being short, micro-units should be compatible with various types of media, including computers and mobiles. In addition, while micro- learning, mobile learning, and e-learning are closely related, they are different philosophically and technologically. Therefore, the design and development of e-learning and micro-learning content are tailored toward the use of computers and mobiles (smartphones, tablets) respectively, while micro-learning is standardized to utilize both media [16].

During formal process of education so far, learners go to the HRD center in the morning, and spend all day for studying many subjects, and instructors have lectures many subjects for a continuous duration of 45 minutes or sometimes more. So, this traditional face-to-face method of continuous speaking for long time creates boredom, and learning effectiveness is bound to decrease due to these traditional type of the learning process of learning [17]. Summarizing this, local governments' HRD institutions must design and operate a curriculum to enable blended learning by developing e-learning content that fit their own characteristics in the future.

3.2.2. The Roles and Responsibilities

As a client, BHI held a bid notice for 5 days (Sep.16-22) to select a developer company to develop content. As of the announcement date, we selected companies that have developed e-learning content (submission of performance certificate) of KRW 50 million or more in a single case within the last 5 years as stated by the laws and regulations. In principle, content development is carried out by the contractor directly, and only instructional designers are allowed to participate as freelancers. Table 4 shows the roles and duties of each group participating in this project.

Table 4. Team Roles and Duties

Talent Management Innovation Center led the project and took charge of the instructional design. The Administration Support Team and the Smart Education Team supported administrative, contract, security, or other system matters. Meanwhile, invited experts from the Busan Development Institute (BDI) who will help developers review the content of the content as SMEs, and experts who will write manuscripts and give lectures as a focus group. In addition, several experts who will advise on design concept planning and template development as well as content technology and quality in educational engineering participated in the project. In particular, after thoroughly analyzing the task instruction, the developer established a strategy according to the following development direction and started the specific development process by reflecting the client's requirements.

Four directions and strategies for this development project:

1. Improving teaching ability through close collaboration in teaching design

2. Improving concentration through high-quality graphics

3. Improving information delivery by adding a story

4. Enhancing interest with various shooting techniques

Content Development Requirements (DER) and the number of requests for each are System Function Requirement(8), Performance(2), Equipment Composition(1), System Interface(2), Test (2), Security(1), Quality(1), Constraint(2), Project Management(6), and Project Support(3). When development requirements or content is changed, the client and developer mutually consult and decide mentioned by the task change procedure.

3.2.3. Critical Path of Development

The most important factor in content creation is the content creation purpose and educational method. Ultimately, as highlighted in Figure 1, the client should review and confirm the course contents (manuscripts) and curriculum while the frequently collaborating project manager, evaluation analyst, instructional designer with developer from the design to development stage so that the developer reflects the educational purpose and method well. Initially, the developer thoroughly analyzed the BHI's duty instructions to help design mentioned by the course goals, learning strategies, and development types, and also made detailed planning for each page of content through direct storyboard design. Subsequently, through prototype design meetings and workshops, the review and confirmation steps were repeated several times so that the content development proceeds in the direction desired by the client, BHI. This meticulous procedure is well illustrated in Figure 1 below.

Figure 1. Critical Path of Development

3.2.4. The Types of Content Production

First, in terms of lecture contents, the project manager held a kick-off meeting, prepared lecture plans and manuscripts through lecturer orientation (via online, handouts, etc.), and selected filming sites for each session and content. At this time, we reflected the relevance of each class, the current policy direction and issues. Above all, we recruited experts and collected opinions through a focus group or SMEs review meeting. The expert group was invited by relevant organizations such as the Maritime Policy Division and the Busan Development Institute (BDI). In particular, the course was structured with content that does not overlap with the self development e-learning of GLP, centered on public office values or administrative work. Table 5 shows the detailed lecture contents for each field and types of production.

Table 5. Course Details and Types of Production

The development type consisted of two studio types, 6 field visit types, and finally one Virtual Reality (VR) type. Studio type is a method of adding 2D and 3D graphics based on Chroma key shooting video. The field visit type applies subtitles to the filmed on-site visit video to generate interest like entertainment. The VR type quickly and clearly shows infographics or news graphics of content synthesized and expressed by combining VR images. Considering that the trend is rapidly changing due to the 4th industrial revolution, the key point of the storyboard design is not a PT-based lecture format, but CG-based content designed to meet the trends. In addition, it was designed in HTML5 (Hyper Text Markup Language 5) format for smooth use on mobile instead of FLASH and compatible with GLP platform.

3.3. Development Phase

3.3.1. Working on Content Production

The ISD manual [4] emphasizes that valuations during each of the other four ADDIE elements after analysis provide the quality control mechanism that ensures an honest and meaningful snapshot of both process and product. As shown in Table 1, In addition to manuscript review and course design, BHI also frequently performed review and feedback through analysis and evaluation in the entire development process, including storyboard design, creating prototypes, video shooting and editing, and porting. In the entire content development process, the organic relationship between the client (Admin., PM and ID) and the developer can be seen in Figure 1.

On October 2020, instructional designer, storyboard producers, and instructors reviewed storyboard samples in agreement with the development direction and strategy at a meeting of instructional designers. After actual content production began, content was prototyped for each production type, and a review meeting was held again. Next, we evaluated the content or storyboard and revised in both evaluating at the design and development phase, and reflected in the next stage. When the process reached 50 percent of the total, late November as in Table 2, we hold an interim report, demonstrated prototype, and conducted pilot test.

3.3.2. Pilot Testing and Ready to Deliver

Pilot testing is a chance to evaluate a project before it goes into full implementation and is a key component of the development stage [4]. Before loading into the LMS, BHI (innovation center) not only supplemented the image, but also checked the content of each step, transplanted it to the system, and performed an e-moderating test for the entire process. After checking all the functions necessary and the delivery material for course operation such as website links, PM report on completion and finally pay to the developer. This means that even if the designer has made the final modifications and is ready to implement, this is not to be the last change in the improvement. As stated in [4], about 90 percent of the development process had completed when it was installed in a local government’s integrated LMS and normal operation monitoring performed once again before starting the process in 2021. For the remaining 10 percent improvement of the entire project, the warranty period of this content had set one year.

4. Implementation and Evaluation

BHI has been offering this new course ‘Understanding Busan, the Maritime Capital’ every month since May 2021. Roughly 100 learners registered every month, and 647 completed the course during 8 months of 2021 (completion rate 90 percent). The target learners of public officials from the city and district county of Busan access to PC or mobile to learn and complete the course with a progress rate of 95 percent or higher.

Figure 2. BHI’s LMS and Implementation Screenshot

Since the implementation and evaluation phase takes a long time after completion of the development, it is easy for clients to forget the last role of the developer, the contractor, while the clients go through these phases. In this phase, if the designers run the course directly he or she designed, they might be able to improvise a fix for missing or faulty design elements on the spot as e-moderators. However, they never actually teaches or operate the course in most cases, so making alterations on the fly is a real challenge. Therefore, BHI empirically verified all processes using Kirkpatrick's four-step evaluation model [12]: (L1) reacting, (L2) learning, (L3) behavior, and (L4) results.

At the final stage of the 'Evaluation' of ADDIE, the analyst included two essential components of evaluation of the 'execution' process: evaluation of learners' impressions of education (Level 1) and validation of goals achieved by learners (Level 2). However, measuring the remaining levels 3 and 4 of Kirkpatrick is quite difficult and extensive, so this study focused more on design and development. Nevertheless, while these levels are linear, the analyst used the levels in a specific order to achieve the evaluation objectives. Each of these stage has specific qualities and fits distinctive needs, so the evaluation of this project requires further study in conjunction with other subsequent development projects in Table 6.

Table 6. Process of Subsequent e-Learning Development Projects

5. Conclusions

The pandemic aftermath that began in 2020 is unlikely to end in a short-term crisis. However, the sudden expansion of non-face-to-face online education left us with many tasks because the education and training method of civil servants has depended on the past collective education. Before checking the effectiveness of learning, it is urgent to switch to offline environment due to increase in demand, but it is an opportunity for recognizing the problem and making new attempts. Given these points, this case study is meaningful in that it deals with the development of content in the field of strengthening basic competency for internalization of government tasks or core policies. Moreover, it presents a new perspective on the e-learning course of special task competency that fits the characteristics of local government in addition to public service values and personality.

In the case of local governments, most of them lack the ability to develop e-learning content, so they rely on the content provided by the central government when design their curriculum. This study tried to show the specific procedure for pilot development of online content dealing with the marine sectors, fisheries, and port logistics areas, which are the biggest characteristics of Busan. It took a lot of time to collect and analyze the demands of public officials in Busan and existing data. However, this study tried to explore the development potential of local government or public institution's e-learning by showing the process of developing content in the form of blended learning that combines e-learning and micro-learning.

To sum up, the success of e-learning requires a broad understanding of the content development process. And although the process of developing effective e-content can be performed by various guidelines, this case was presented in keeping with the ADDIE model. Of course, professional knowledge, experience, and insight are required to conduct a case study, but it is difficult to generalize the research results because it is a study on a specific case. Nor does it provide specific prescriptions for each phase of ISD. And even the implementation and evaluation process of the ADDIE model is still in progress. Nevertheless, this study is meaningful in that it showed the need for a macroscopic design and shows the process of close collaboration well to meet the needs of learners and sub-organizations in the pandemic. As it is expected that there are still limitations of online learning such as learners' immersion in class and feedback on educational content, further studies are required.