International Journal of Contents

The Korea Contents Association (한국콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

University Students' Perceptual Lecture Evaluation of Online Lectures During the COVID-19 Situation

  • Nam, Sangzo (Division of Aviation, Hotel, Tourism Management, Mokwon University) ;
  • Cho, Soohyun (Division of Aviation, Hotel, Tourism Management, Mokwon University)
  • Received : 2021.10.28
  • Accepted : 2022.03.22
  • Published : 2022.03.28
Download PDF
⟨ Previous Next ⟩

Abstract

Students' perceptions of generosity and fairness in lecture evaluation and grades, communication with professors, and self-fidelity and satisfaction during the COVID-19 situation were statistically analyzed by surveying students at M university in Daejeon. These data were analyzed in the context of parameters that might impact online class lecture evaluations, namely gender and school year. Descriptive analysis shows students' perceptions of online lectures are significantly high. As for differences by gender and school year, the t-test results indicate female students generally have better perceptions of online classes than male students. However, there is no statistical difference between male and female students regarding the generosity of lecture evaluation. Also, ANOVA test results show that as the school year increases, the general perceptions for online classes become negative. However, there is no statistical difference by school year regarding the generosity of lecture evaluation. Regression analysis shows that the "perceptual generosity of grades" most significantly influenced the "perceptual generosity of lecture evaluation."

Keywords

1. Introduction

Various social phenomena have occurred due to the continuation of COVID-19, and the expansion of e learning has been remarkable in the education field. According to the 2020 Survey of the Korean e-learning Industry [1], regular education institutions collected 26.96 billion won in 2020, up 21.1% from 2019. The adoption rate of e-learning by regular educational institutions in 2020 was 96.4%, up 7.4% from 89.0% in 2019. Looking at the introduction of e-learning, junior colleges and four-year universities have opened e-learning classes to 71.6% and 73.3%, respectively, of courses in the “regular curriculum.” Lou et. al. claimed that small group learning using computer technology had significantly more positive effects than individual learning using computer technology [2]. As such, people looked forward to normalizing education through in-person face-to- face classes. However, in 2021, universities were unable to implement face-to-face classes except for limited practical or experimental classes. Therefore, college students in Korea have been forced to take online lectures since 2020 and through 2021. Aside from the initial confusion, online lectures are now operating normally as an inevitable alternative. However, Hong argues that the sudden conversion to online lectures due to COVID- 19 serves only as a transfer of knowledge through unilateral lectures and that there can be fundamental limitations in increasing learning through knowledge exchange [3].

Many studies have addressed e-learning tasks and experiences under the circumstances of COVID-19 pandemic [4]. Studies on e-learning class experience mainly cover subject development experience and influencing factors for full time online classes.

However, it is difficult to find previous studies that investigated students' perceptions of lecture evaluations in online classes under the COVID-19 situation. Due to the social interest in university education and increased demand for improved lecture quality, the importance of lecture evaluation has also increased [5]. In addition, the proportion of lecture evaluations is increasing with the proportion of lecture quality improvement in institutional evaluation indicators, such as university structural reform and university characterization projects [6]. According to research on the evaluation status of lectures at Korean universities, most universities conduct lecture evaluations to improve the quality of their classes; however, most universities only secure evaluation scores without taking further analysis [7]. Therefore, it is necessary to investigate the evaluation of online lectures during the COVID-19 pandemic situation.

The purpose of this study is to investigate the three major arbitrary issues that have presented conflicting conclusions in previous lecture evaluation study: gender difference, school year difference and the impact of grade in online lectures together with other factors included in previous studies such as perception of communication with professors, self-fidelity, and satisfaction during the COVID-19 situation.

We surveyed 297 replies of students (after preprocessing) who experienced online classes over three semesters at M university in Daejeon.

2. Literature Review

2.1. Studies on Online Class in COVID-19

During the COVID-19 pandemic era from 2020, there were many studies on full time e-learning classes. Research on subject development published during COVID-19 pandemic period were mainly conducted in Korean writing classes, English classes, music classes, art or design classes, physical education and dance classes, nursing classes, engineering or science classes, and so on [8-17]. Also, a number of studies have been published on development experience of on-demand content classes or online real-time classes [18-21]. Nam published a study comparing students' perceptions of on-demand content lectures and online real-time lectures [22]. There were studies on professors’ perceptions of full time online lectures [23, 24]. Do identified constraints in converting face-to-face lecture into online lecture [25]. Lee and Suh proposed that satisfaction level of the remote video class was significantly higher than face-to-face class, and that the combination of remote video classes and face-to-face classes was more satisfactory than other cases [26]. Lee and Kim found that in-depth project based learning research in a non-face-to-face learning environment improved students’ learning strategy and satisfaction [27]. Han and Nam proposed faculty competency factors to improve online classes [28]. There were studies on students’ perceptions and preference of full time online lectures [29-31]. Jung identified students’ difficulty and needs for support for full time online lectures [32]. Han proposed modeling architecture for competency of students [33]. Kim et al. claimed the importance of interaction among teacher, students and peers for learning achievement [34]. Yu identified the correlation between the self-efficacy in online classes and self-directed learning [35]. Um suggested that learners' attitudes toward e-learning were positively influenced by perceived e-learning usefulness, self-management of learning and self-efficacy [36]. Na and Park investigated the satisfaction and success factors of college students for online lectures during the COVID-19 situation, and suggested that learning accessibility and teaching methods were the most influential factors [37]. Jo and Bae raised the importance of the grade evaluation method in order to increase the students’ satisfaction of online lectures during the COVID-19 situation [38]. Han reported that face-to-face education could be more interactive than synchronous online education, and that synchronous online education could have better interactive between instructor and learner, and among learners than asynchronous online education [39]. Choi proposed the importance of communication with instructors to prevent the decline in the learning effect of college freshmen in non-face-to-face classes due to COVID-19 [40]. Through previous studies, we found that as elements for successful online lectures, competencies in the professor sector such as communication with students and competencies in the student sector such as self-fidelity are also needed.

2.2. Studies on Lecture Evaluation

Studies of lecture evaluation are mainly conducted on the development of lecture evaluation scales and on the factors influencing lecture evaluation [41-45]. Recenty, language analysis studies on descriptive lecture evaluation have emerged [46, 47]. dApollonia and Abrami reported that student ratings were moderately valid; however, administration of instructor, and course characteristics influenced student ratings of instruction [48]. Seol researched to find factors affecting student’s rating on lectures at university, and he suggested that there was no statistical difference between gender, school year, class size and student’s grade affecting the student’s rating [49]. However, Choi and Kim suggested that male students evaluate lectures more generously than female students, and that higher school year students evaluate lectures more generously than lower school year students[50]. Song proposed that class size and lesson difficulty influenced the teaching evaluation [51]. Lyu and Lee reported that grades were an important factor in the participation in lecture evaluation, and that lecture evaluation of high-performing students was more likely to produce homogeneous evaluation results than that of relatively low-performing students [52]. The study of Greenwald and Gillmore identified that students' evaluative ratings of instruction correlated positively with expected course grades [53]. Ryan performed a survey of faculty members concerning the students’ evaluation, and reported faculty members’ definite reduction in morale and job satisfaction, and changes in various instructional practices (mainly reduced course work demands on students) [54]. However, Marsh and Roche debunked popular myths that student evaluations of teaching were substantially biased by low workload and grading leniency. They proposed that relation of workload and evaluation was positive, and relation between grade and evaluation was nonlinear [55]. Feistauer and Richter claimed that students' individual perceptions of teaching and the fit of these perceptions with the particular teacher greatly influence their evaluations [56]. Nam presented a test survey study on the possibility of distortion of online class lecture evaluation [57].

Previous studies showed that expected grades, gender, school year, students’ performance level (e.g., self fidelity and satisfaction), and instructor administration (e.g., communication with students), class size, and course workload are the main factors influencing lecture evaluations. However, the results of previous studies on the relationship between lecture evaluation and factors such as gender, school year, and grade influence were found to be contradictory. Therefore, we conducted not only research on gender difference and school year difference in lecture evaluation, but also research on whether cognitive factors such as perceptions of expected grade, fairness of grading, satisfaction, self-fidelity, and communication with professors affect lecture evaluation under the CODID-19 situation.

3. Research Methodology

3.1. Research Data

Between July and August 2021, we conducted an online survey of approximately 3, 000 students at M University in Daejeon using a Korean Social Science Data Center online survey tool. The survey received 321 responses, of which 297 were collected after excluding unfaithful responses.

As we can see from Table 1, the portion of male is 49.49% and that of female is 50.51%.

Table 1. Frequency of repliers by gender

E1CTBR_2022_v18n1_85_t0001.png 이미지

Table 2 shows that the proportion of freshman, sophomore, junior and senior is 29.97%, 24.24%, 25.59% and 20.20% respectively.

Table 2. Frequency of repliers by school year

E1CTBR_2022_v18n1_85_t0002.png 이미지

3.2 Data Processing

Descriptive statistics were performed. As for the differences such as perceptions of “generosity of online lecture evaluation, ” “fairness of online lecture evaluation, ” “generosity of online lecture grades, ” “fairness of online lecture grades, ” “online lecture communication with professors, ” “self-fidelity for online lecture” and “satisfaction for online lecture” between male and female, independent t-test was performed. As for such difference among three more group variables such as school year, ANOVA test was performed using E-STAT 3.0. Also, regression analysis for the influence of perceived generosity of online lecture grades, fairness of online lecture grades, fairness of online lecture evaluation, online lecture communication with professors, self fidelity for online lecture and satisfaction for online lecture to the perception of generosity of online lecture evaluation was performed by SPSS 20.

3.3 Research Hypothesis

In order to investigate three arbitrary points in lecture evaluation such as gender difference, school year difference and grade influence, we classified hypothesis into three categories matching the purpose of study such as 1. Verification of gender differences in perceptions of online lecture evaluation, grades, communication with professors, self-fidelity and satisfaction. 2. Verification of school year differences in perceptions of online lecture evaluation, grades, communication with professors, self-fidelity and satisfaction. 3. Verification of grade influence on lecture evaluation.

. Verification of gender differences

H1. The perceived generosity of online lecture evaluation would not be different by gender.

H2. The perceived fairness of online lecture evaluation would not be different by gender.

H3. The perceived generosity of online lecture grades would not be different by gender.

H4. The perceived fairness of online lecture grades would not be different by gender.

H5. The perceived online lecture communication with professors would not be different by gender.

H6. The self-fidelity for online lecture would not be different by gender.

H7. The satisfaction for online lecture would not be different by gender.

. Verification of school year differences

H8. The perceived generosity of online lecture evaluation would not be different by school year.

H9. The perceived fairness of online lecture evaluation would not be different by school year.

H10. The perceived generosity of online lecture grades would not be different by school year.

H11. The perceived fairness of online lecture grades would not be different by school year.

H12. The perceived online lecture communication with professors would not be different by school year.

H13. The self-fidelity for online lecture would not be different by school year.

H14. The satisfaction for online lecture would not be different by school year. . Verification of grade influence on lecture evaluation

H15. The perceived generosity of online lecture evaluation would not be influenced by perceived generosity of online lecture grades, perceived fairness of online lecture grades, perceived fairness of online lecture evaluation, perceived online lecture communication with professors, self-fidelity for online lecture and satisfaction for online lecture.

4. Experimental Results

4.1. Descriptive Statistics

Before conducting the hypothesis test, in order to check students’ perception of candidate factors, we performed descriptive analysis.

As shown in Table 3, the average point of 3.609 on a 5-point Likert scale for perception of generosity means that students have tendency to give generous lecture evaluation for online.

Table 3. Perception of generosity of online lecture evaluation

E1CTBR_2022_v18n1_85_t0003.png 이미지

As shown in Table 4, the average point of 4.064 on a 5-point Likert scale for fairness of online lecture evaluation means that students think they are very fair in online lecture evaluation.

Table 4. Perception of fairness of online lecture evaluation

E1CTBR_2022_v18n1_85_t0004.png 이미지

As shown in Table 5, the average point of 3.687 on the 5-point Likert scale for perception of generosity of online lecture grades means that students perceive online lecture grades as generous.

Table 5. Perception of generosity of online lecture grades

E1CTBR_2022_v18n1_85_t0005.png 이미지

As shown in Table 6, the average point of 3.892 on the 5-point Likert scale for perception of fairness of online lecture grades means that students think the result of grades is very fair.

Table 6. Perception of fairness of online lecture grades

E1CTBR_2022_v18n1_85_t0006.png 이미지

As shown in Table 7, students perceive communication with professor positively.

Table 7. Perception of online lecture communication with professors

E1CTBR_2022_v18n1_85_t0007.png 이미지

As shown in Table 8, the self-fidelity for online lecture is very positive.

Table 8. Self-fidelity for online lecture

E1CTBR_2022_v18n1_85_t0008.png 이미지

As shown in Table 9, the satisfaction for online lecture is also very positive.

Table 9. Satisfaction for online lecture

E1CTBR_2022_v18n1_85_t0009.png 이미지

4.2. Hypothesis Test

We performed hypothesis tests in order to verify gender difference and school year difference, and finally, conducted the last hypothesis test on factors affecting online lecture evaluation.

H1. The perceived generosity of online lecture evaluation would not be different by gender: The perceived generosity of online lecture evaluation by gender was not statistically different at 95% confidence level as shown in Table 10. According to Levine's test, assuming equal variance, females’ perceived generosity was slightly higher than that of male.

Table 10. Independent t-test for perceived generosity difference of online lecture evaluation by gender

E1CTBR_2022_v18n1_85_t0022.png 이미지

H2. The perceived fairness of online lecture evaluation would not be different by gender: The perceived fairness of online lecture evaluation by gender was statistically different as shown in Table 11. According to Levine's test, assuming equal variance, females’ perceived fairness was significantly higher than that of male at 95% confidence level.

Table 11. Independent t-test for perceived fairness difference of online lecture evaluation by gender

E1CTBR_2022_v18n1_85_t0010.png 이미지

H3. The perceived generosity of online lecture grades would not be different by gender: The perceived generosity of online lecture grades by gender was statistically different as shown in Table 12. According to Levine's test, assuming equal variance, females’ perceived generosity was significantly higher than that of male at 95% confidence level.

Table 12. Independent t-test for perceived generosity difference of online lecture grades by gender

E1CTBR_2022_v18n1_85_t0011.png 이미지

H4. The perceived fairness of online lecture grades would not be different by gender: The perceived fairness of online lecture grades by gender was not statistically different at 95% confidence level as shown in Table 13. According to Levine's test, assuming not equal variance, females’ perceived fairness was slightly higher than that of male.

Table 13. Independent t-test for perceived fairness difference of online lecture grades by gender

E1CTBR_2022_v18n1_85_t0023.png 이미지

H5. The perceived online lecture communication with professors would not be different by gender: The perceived online lecture communication with professors by gender was not statistically different at 95% confidence level as shown in Table 14. According to Levine's test, assuming equal variance, females’ perceived online lecture communication with professors was slightly higher than that of male.

Table 14. Independent t-test for perceived online lecture communication with professors difference by gender

E1CTBR_2022_v18n1_85_t0012.png 이미지

H6. The self-fidelity for online lecture would not be different by gender: The self-fidelity for online lecture by gender was statistically different as shown in Table 15. According to Levine's test, assuming equal variance, females’ self-fidelity was significantly higher than that of male at 95% confidence level.

Table 15. Independent t-test for self-fidelity for online lecture difference by gender

E1CTBR_2022_v18n1_85_t0013.png 이미지

H7. The satisfaction for online lecture would not be different by gender: The satisfaction for online lecture by gender was statistically different as shown in Table 16. According to Levine's test, assuming not equal variance, females’ satisfaction for online lecture was significantly higher than that of male at 95% confidence level.

Table 16. Independent t-test for satisfaction for online lecture difference by gender

E1CTBR_2022_v18n1_85_t0014.png 이미지

H8. The perceived generosity of online lecture evaluation would not be different by school year: Even though senior student’s perceived generosity was the highest, the perceived generosity of online lecture evaluation was not statistically different as shown in Table 17.

Table 17. ANOVA-test for difference of perceived generosity of online lecture evaluation by school year

E1CTBR_2022_v18n1_85_t0015.png 이미지

H9. The perceived fairness of online lecture evaluation would not be different by school year: The perceived fairness of online lecture evaluation was not statistically different as shown in Table 18. Freshmen’s perceived fairness of online lecture evaluation was the highest followed by that of sophomore students.

Table 18. ANOVA-test for difference of perceived fairness of online lecture evaluation by school year

E1CTBR_2022_v18n1_85_t0016.png 이미지

H10. The perceived generosity of online lecture grades would not be different by school year: The perceived generosity of online lecture grades was not statistically different as shown in Table 19. Senior students’ perceived generosity of lecture grades was the highest followed by that of junior students.

Table 19. ANOVA-test for difference of perceived generosity of online lecture grades by school year

E1CTBR_2022_v18n1_85_t0024.png 이미지

H11. The perceived fairness of online lecture grades would not be different by school year: The perceived fairness of online lecture grades was not statistically different as shown in Table 20. Freshmen’s perceived fairness of online lecture grades was the highest followed by that of sophomore students.

Table 20. ANOVA-test for difference of perceived fairness of online lecture grades by school year

E1CTBR_2022_v18n1_85_t0017.png 이미지

H12. The perceived online lecture communication with professors would not be different by school year: The perceived online lecture communication with professors was statistically different at 95% confidence level as shown in Table 21. Freshmen’s perception of online lecture communication with professors was quite positive. But the higher the school year, the lower the awareness of the online lecture communication with professors.

Table 21. ANOVA-test for difference of perceived online lecture communication with professors by school year

E1CTBR_2022_v18n1_85_t0018.png 이미지

H13. The self-fidelity for online lecture would not be different by school year: The self-fidelity for online lecture was statistically different at 95% confidence level as shown in Table 22. The self-fidelity was generally quite high, but that of senior students was the lowest.

Table 22. ANOVA-test for difference of self-fidelity for online lecture by school year

E1CTBR_2022_v18n1_85_t0019.png 이미지

H14. The satisfaction for online lecture would not be different by school year: The satisfaction for online lecture was statistically different at 95% confidence level as shown in Table 23. Freshmen’s satisfaction was quite high, but satisfaction of junior and senior students fell considerably.

Table 23. ANOVA-test for difference of satisfaction for online lecture by school year

E1CTBR_2022_v18n1_85_t0020.png 이미지

H15. The perceived generosity of online lecture evaluation would not be influenced by perceived generosity of online lecture grades, perceived fairness of online lecture grades, perceived fairness of online lecture evaluation, perceived online lecture communication with professors, self-fidelity for online lecture and satisfaction for online lecture: The research model is as shown in Figure 1.

E1CTBR_2022_v18n1_85_f0001.png 이미지

Figure 1. Research model for the influence of various factors to the perceptual generosity of online lecture evaluation

Table 24. Regression analysis for the influence of various factors to the perceptual generosity of online lecture evaluation

E1CTBR_2022_v18n1_85_t0021.png 이미지

*p<.05, **p<.01 ***p<.000

As shown in Table 24, the perceptions of “fairness of online lecture grades, ” “communication with professors, ” “self-fidelity for online lecture” and “satisfaction for online lecture” did not significantly influence the “perceptual generosity of online lecture evaluation” statistically. However, the perceptions of “generosity of online lecture grades” and “fairness of online lecture evaluation” significantly influence the “perceptual generosity of online lecture evaluation” statistically. The “perceptual generosity of online lecture grades” was the most significant factor influencing “perceptual generosity of online lecture evaluation” statistically, and the statistical effect was very serious.

5. Conclusion

As for the perception of generosity and fairness of online lecture evaluation, average points of 3.609 and 4.064 respectively on the 5-point Likert scale mean that students have tendency to give generous lecture evaluation for online lectures and think they are very fair in online lecture evaluation. The average point of 3.687 and 3.892 respectively on the 5-point Likert scale for perception of generosity and fairness of online lecture grades mean that students perceive online lecture grades as generous but think the result of grades is very fair. This is due to the school's policy of making the grades of online lectures more generous than before through the transition from relative evaluation to absolute evaluation or changes in the relative evaluation ratio during the COVID-19 situation. However, average points of 4.212 and 3.949 respectively on the 5-point Likert scale for the perception of self-fidelity and satisfaction show very positive educational effects of online lectures. Also, students perceive communication with professor positively by showing 3.579 average points on the 5- point Likert scale. These results can be inferred that the students who participated in the survey are relatively sincere students.

Even though female students’ perceived generosity of online lecture evaluation was slightly higher than that of male students, the perceived generosity of online lecture evaluation by gender was not statistically different at 95% confidence level. Our result was contrary to the study of Choi and Kim [50], who claimed that male students were conducting generous lecture evaluations, but showed the same conclusion as Seol's study [49], which suggested there was no statistical difference between evaluations from male and female students despite female students having better perceptions of the online lectures. Female students’ perceived generosity of online lecture grades was significantly higher than that of male at 95% confidence level. However, despite female students’ perceived fairness was slightly higher than that of male students, the perceived fairness of online lecture grades by gender was not statistically different. Also, despite female students’ perceived online lecture communication with professors was slightly higher than that of male students, the perceived online lecture communication with professors by gender was not statistically different. However, female students’ self fidelity and satisfaction were significantly higher than those of male students at 95% confidence level. These results indicate that the educational effect of online lectures is better for female students than male students.

Even though senior students’ perceived generosity of online lecture evaluation was slightly high, the perceived generosity of online lecture evaluation was not statistically different at 95% confidence level. This result was contrary to the study of Choi and Kim [50], who claimed that senior students were conducting generous lecture evaluations, but showed the same conclusion as Seol's study [49], which suggested there was no statistical school year difference. Also, the perceived fairness of online lecture evaluation was not statistically different. However, we found that freshmen’s perceived fairness of online lecture evaluation was the highest followed by that of sophomore students. The perceived generosity of online lecture grades was not statistically different. We found that senior students’ perceived generosity of lecture grades was the highest followed by that of junior students. Also, the perceived fairness of online lecture grades was not statistically different. Ironically, freshmen’s perceived fairness of online lecture grades was the highest followed by that of sophomore students. These results mean that the higher school year, students feel generous grade and not fair in grade. The perceived online class communication with professors was statistically different at 95% confidence level. Freshmen’s perception of online class communication with professors was quite positive. But the higher the school year, the lower the awareness of the online class communication with professors. The self-fidelity and satisfaction for online class were both statistically different at 95% confidence level. The self-fidelity was generally quite high, but self-fidelity of the senior students was the lowest. Freshmen’s satisfaction was quite high, but satisfaction of the junior and senior students fell considerably. These results mean that as the school years goes up, the educational effect of online lectures is negative.

As for the influence of perceptions of “fairness of online lecture evaluation, ” “generosity of online lecture grades, ” “fairness of online lecture grades, ” “communication with professors, ” ”self-fidelity for online lectures, ” and “satisfaction for online lecture” on the “perceptual generosity of online lecture evaluation, ” only the perceptions of “generosity of online lecture grades” and “fairness of online lecture evaluation” significantly influenced the “perceptual generosity of online lecture evaluation” statistically. Furthermore, the statistical effect of perceptual generosity of online lecture grades on perceptual generosity of online lecture evaluation was very significant. This result was contrary to the Seol’s study [49] and Marsh and Roche’s study [55] which claimed there was no relation between grade and evaluation, but showed the same conclusion as Lyu and Lee’s study [52] and Greenwald and Gillmore’s study [53]. This means that lecture evaluations for online courses can be distorted by generous grades.

The limitation of this research is that it is an analysis of students belonging to one university. Therefore, follow-up studies may include students from many schools.

References

  1. National IT Industry Promotion Agency, 2020 Survey of Korean e-learning industry, Seoul, May 2021.
  2. Y. Lou, P. C. Abrami, and S. d'Apollonia, "Small group and individual learning with technology: A meta-analysis," Review of Educational Research, vol. 71, no. 3, pp. 449-521, 2001, doi: http://dx.doi.org/10.3102/00346543071003449.
  3. J. Hong, "Online lectures after COVID-19," Korea Public Administration Forum, vol. 172, pp. 36-41, 2021.
  4. J. Lee, E. Sung, J. Lee, K. Y. Lim, and S. Han, "Challenges and tasks facing online classes during the COVID-19 pandemic," Journal of Educational Technology, vol. 36, no. Special, pp. 671-692, 2020. https://doi.org/10.17232/KSET.36.3.671
  5. J. Kim, "Developing a student evaluation instrument for college teaching," Journal of the Korea Academia-Industrial Cooperation Society, vol. 18, no. 6 pp. 187-196, 2017, doi: http://dx.doi.org/10.5762/KAIS.2017.18.6.187.
  6. H. Jeong, "A study on students awareness of course evaluation: focusing on integrity of responsive attitudes and satisfaction," The Journal of Educational Methodology, vol. 28, no. 3, pp. 443-470, 2016, doi: http://dx.doi.org/10.17927/TKJEMS.2016.28.3.443.
  7. S. Han, H. Kim, and J. Lee, "A comprehensive study of Korean students' evaluation of university teaching," The Journal of Educational Administration, vol. 23, no. 3, pp. 379-403, 2005.
  8. C. Jung, J. Yoon, "Online real-time lecture operation examples and training effects: Focusing on the case of
  9. B. K. Lee, "A study on learners' response to online college English class as general education due to the COVID-19 pandemic," Korean Journal of General Education, vol. 14, no. 4, pp. 97-112, 2020. https://doi.org/10.46392/kjge.2020.14.4.97
  10. E. Lim, S. Kwon, "A study on the status of non-face-to-face music online remote classes," The Korean Journal of Arts Education, vol. 18, no. 3, pp. 165-184, 2020.
  11. E. Hong, J. Ryu, "Qualitative research of art therapy-based online class for university students," Journal of Clinical Art Psychotherapy, vol. 10, no. 2, pp. 1-23, 2020.
  12. S. Y. Park, "A Study on the Efficient Design Education Program through Non-face-to-face Online Practical Classes," M.D. theses, Graduate School of Education, Pukyung National Univ., Busan, Korea, 2020.
  13. J. Kim, H. Shin, "A study on teaching experiences in online creative dance classes," The Southeast Korea Dance Society, vol. 8, no. 2, pp. 83-101, 2020.
  14. E. Choi, J. Yun, and S. Park, "Factors influencing the other behaviors taken by nursing student during online lectures," Journal of the Korea Convergence Society, vol. 11, no. 9, pp. 433-441, 2020, doi: http://dx.doi.org/10.15207/JKCS.2020.11.9.433.
  15. W. Jang, M. Choi, and H. Hong, "A case study on the operation of non-face-to-face experimental class at university with COVID-19 pandemic," Journal of Learner-Centered Curriculum and Instruction, vol. 20, no. 17, pp. 937-966, 2020.
  16. H. K. Sohn, "A study on development of a convergent teaching method to enhance the efficiency of science liberal arts education in university: Focusing on the use of YouTube contents based on platform media," The Korean Society of Science & Art, vol. 38, no. 3, pp. 117-128, 2020, doi: http://dx.doi.org/ 10.17548/ksaf.2020.06.30.117.
  17. S. Son, K. Moon, "Effects of integrated simulation education among nursing students during the COVID-19 pandemic in Korea," International Journal of Contents, vol. 17, no. 3, pp. 38-47, 2021, doi: http://dx.doi.org/10.5392/IJOC.2021.17.3.038.
  18. H. Lee, M. Kim, "Case study of digital contents for online education in college-focused on virtual museum," Korean Journal of General Education, vol. 14, no. 4, pp. 81-96, 2020. https://doi.org/10.46392/kjge.2020.14.4.81
  19. Y. Huh, "The effects of contents quality factors on academic persistence in online education responding to COVID-19: Focused on the mediating learning flow and satisfaction," Journal of Knowledge Information Technology and Systems, vol. 15, no. 5, pp. 759-770, 2020, doi: http://dx.doi.org/ 10.34163/JKITS.2020.15.5.019.
  20. H. Kim, "An exploration on non-face-to-face Korean class eExperience of foreign undergraduate students: Focusing on real-time online class utilizing Zoom," The Journal of Humanities and Social Science, vol. 11, no. 4, pp. 1679-1692, 2020.
  21. S. Han, G. Lee, "Comparative analysis of instructors' perception of synchronous online classes: A case study of a university," Culture and Convergence, vol. 42, no. 7, pp. 395-418, 2020.
  22. S. Nam, "Comparative analysis of online real-time lecture and on-demand contents lecture under the COVID-19 Situation in Korea," Journal of Advanced Information Technology and Convergence, vol. 10, no. 2, pp. 177-198, 2020. https://doi.org/10.14801/JAITC.2020.10.2.177
  23. J. W. Yoon, "A study on the C University professor's perception of the online class," The Journal of Humanities and Social science, vol. 11, no. 5, pp. 2413-2426, 2020.
  24. H. Yoon, J. Kwon, "Study on the instructor perceptions of online classes: Focusing on the case of H University," The Journal of Humanities and Social Science, vol. 11, no. 5, pp. 1281-1296, 2020.
  25. J. Do, "An investigation of design constraints in the process of converting face-to-face course into online course," Journal of Education & Culture, vol. 26, no. 2, pp. 153-173, 2020, doi: http://dx.doi.org/10.24159/JOEC.2020.26.2.153.
  26. H. Lee, E. Suh, "A comparative study on the class satisfaction between remote video class and face-to-face class," The Journal of the Korea Contents Association, vol. 21, no. 7, pp. 440-447, 2021, doi: http://dx.doi.org/10.5392/JKCA.2021.21.07.440.
  27. Y. Lee, E. Kim, "An analysis of learning outcomes and learning satisfaction of project-based learning in non-faceto- face learning environment," The Journal of the Korea Contents Association, vol. 21, no. 6, pp. 814-825, 2021, doi: http://dx.doi.org/10.5392/JKCA.2021.21.06.814.
  28. S. Han, Y. O. Nam, "Faculty competency factor needs analysis to improve the quality of online classes for higher education," Journal of Learner-centered Curriculum and Instruction, vol. 20, no. 13, pp. 1129-1149, 2020.
  29. E. K. Park, "Perception of learner anxiety towards online college English classes during COVID-19," Multimedia-Assisted Language Learning, vol. 23, no. 3, pp. 320-338, 2020. https://doi.org/10.15702/MALL.2020.23.3.320
  30. H. Chung, A. Kim, and H. Joo, "A study on the types of perceptions on online classes of college students," Journal of Learner-Centered Curriculum and Instruction, vol. 20, no. 18, pp. 1359-1381, 2020.
  31. H. Park, "A case study on the perceptions and preferences of graduate students of interpretation and translation for online learning," Journal of interpretation & translation institute, vol. 24, no. 3, pp. 43-66, 2020.
  32. Y. M. Jung, "The difficulty and needs for support for online class experience of college students," The Journal of Humanities and Social science, vol. 11, no. 5, pp. 685-698, 2020.
  33. S. U. Han, "Modeling of competency of excellent learners in online classes at teachers' colleges," The Journal of Educational Research, vol. 18, no. 3, pp. 19-41, 2020.
  34. J. Kim, K. Sohn, E. Lee, J. Jeoung, H. Jang, and W. Lee, "The effects of interaction between instructor-student and student-student on learning achievement in synchronous e-learning for major classes for university students: The mediating role of learning flow," Journal of Agricultural Education and Human Resource Development, vol. 52, no. 3, pp. 25-48, 2020.
  35. K. A. Yu, "A study of university students' academic self-efficacy, problem solving & self-directed learning abilities in an online general education class," The Journal of General Education, vol. 13, pp. 33-58, 2020, doi: http://dx.doi.org/10.24173/jge.2020.10.13.2.
  36. N. Um, "Learners' attitude toward e-learning: the effects of perceived system quality and e-learning usefulness, selfmanagement of learning, and self-efficacy," International Journal of Contents, vol. 17, no. 2, pp. 41-47, 2021, doi: http://dx.doi.org/10.5392/IJOC.2021.17.2.041.
  37. S. Na, D. Park, "Analysis of the importance and satisfaction of online lecture service quality according to the COVID-19 situation," The Korean Journal of Growth and Development, vol. 29, no. 3, pp. 439~447, 2021. https://doi.org/10.34284/KJGD.2021.08.29.3.439
  38. J. Jo, J. Bae, "The effect of COVID-19 on academic satisfaction with online lecture types and contents -Perspectives of the domestic and foreign university students," Journal of the Korea Academia-Industrial Cooperation Society, vol. 22, no. 3, pp. 643-650, 2021, doi: http://dx.doi.org/ 10.5762/KAIS.2021.22.3.643.
  39. H. Han, "An exploration of interaction factors and analysis on interaction-level of synchronous online education in university," The Journal of the Korea Contents Association, vol. 21, no. 4, pp. 14-25, 2021, doi: http://dx.doi.org/10.5392/JKCA.2021.21.04.014.
  40. H. Choi, "A study on the non-face-to-face teaching experience of college freshmen due to Covid-19," Korean Journal of General Education, vol. 15, no. 1, pp. 273-286, 2021. https://doi.org/10.46392/kjge.2021.15.1.273
  41. M. Seo, E. Chi, and C. Hwang, "Developing and validating the measurement instrument of higher education learning outcomes," Journal of Educational Evaluation, vol. 26, no. 2, pp. 275-296, 2013.
  42. S. Oh, "A theoretical and empirical study on factors and construct validity of student course evaluation instrument: Focused on the student course evaluation case of S' University," The Korea Educational Review, vol. 26, no. 1, pp. 25-49, 2020. https://doi.org/10.29318/KER.26.3.2
  43. Y. Choi, R. Yun, "A study on developing and validating the lecture evaluation tool-Focusing on the H University case," Journal of Korean Association for Educational Information and Media, vol. 26, no. 1, pp. 25-49, 2020. https://doi.org/10.15833/KAFEIAM.26.1.025
  44. K. Lee, "A study on validity and reliability of students' evaluation," Journal of the Korean Data and Information Science Society, vol. 21, no. 1, pp. 87-98, 2010.
  45. E. Yoon, "A study on the variables affecting the evaluation of lectures," Korean Journal of General Education, vol. 12, no. 5, pp. 185-210, 2018.
  46. J. Kim, J. Cheong, and H. Jeong, "University narrative lecture evaluation status and network analysis: A case study of S University," Journal of Learner-Centered Curriculum and Instruction, vol. 21, no. 15, pp. 149-164, 2021.
  47. J. Shin, J. Choi, "Text mining analysis of college students' descriptive course evaluation," Journal of Learner-Centered Curriculum and Instruction, vol. 19, no. 16, pp. 77-99, 2019.
  48. S. dApollonia, P. Abrami, "Navigating student ratings of instruction," The American Psychologist, vol. 55, no. 11, pp. 1198-1208, 1997.
  49. H. Seol, "A Study on the variables affecting the student's evaluation of teaching at a university," Journal of Holistic Education, vol. 23, no.4, pp. 53-63, 2019.
  50. B. Choi, J. Kim, "Multilevel analysis of the effects of student and course characteristics on student evaluations of university teaching," The Journal of Yeolin Education, vol. 21, no. 1, pp. 77-100, 2013.
  51. Y. Song, "Analysis of factors affecting satisfaction of teaching evaluation in the university," Journal of Competency Development & Learning, vol. 13, no. 2, pp. 141-163, 2018.
  52. C. Lyu, J. Lee, "A study on student factors associated with the student evaluation of teaching at universities," Korean Management Review, vol. 32, no. 3, pp. 789-807, 2003.
  53. A. Greenwald, G. Gillmore, "Grading leniency is a removable contaminant of student ratings," The American psychologist, vol. 52, no. 11, pp. 1209-1217, 1997, doi: http://dx.doi.org/ 10.1037/0003-066X.52.11.1209.
  54. J. Ryan, "Student evaluation: the faculty responds," Research in Higher Education, vol. 12, no. 4, pp. 317-333, 1980, doi: http://dx.doi.org/ 10.1007/BF00976185.
  55. H. Marsh, L. Roche, "Effects of grading leniency and low workload on students' evaluations of teaching: Popular myth, bias, validity, or innocent bystanders?," Journal of Educational Psychology, vol. 92, no. 1, pp. 202-228, 2000, doi: http://dx.doi.org/ 10.1037/0022-0663.92.1.202.
  56. D. Feistauer, T. Richter, "How reliable are students' evaluations of teaching quality? A variance components approach," Assessment and Evaluation in Higher Education, vol. 42, no. 8, pp. 1263-1279, 2017, doi: http://dx.doi.org/ 10.1080/02602938.2016.1261083.
  57. S. Nam, "A pilot survey on online lecture evaluation," in Proc. 2021 Spring Conference of the Korea Contents Association, Daejeon, Korea, Aug. 27-28, 2021, pp. 199-200.