DOI QR코드

DOI QR Code

Integrated Media Platform-based Virtual Office Hours Implementation for Online Teaching in Post-COVID-19 Pandemic Era

  • Chen, Mingzi (College of Telecommunication and Information Engineering, Nanjing University of Posts and Telecommunications) ;
  • Wei, Xin (College of Telecommunication and Information Engineering, Nanjing University of Posts and Telecommunications) ;
  • Zhou, Liang (College of Telecommunication and Information Engineering, Nanjing University of Posts and Telecommunications)
  • Received : 2021.04.22
  • Accepted : 2021.07.01
  • Published : 2021.08.31

Abstract

In post-COVID-19 pandemic era, students' learning effects and experience may sharply decrease when teaching is transferred from offline to online. Several tools suitable for online teaching have been developed to guarantee and promote students' learning effects. However, they cannot fully consider teacher-student interaction in online teaching. To figure out this issue, this paper proposes integrated media platform-based virtual office hours implementation for online teaching. Specifically, an integrated media platform (IMP) is first constructed. Then, virtual office hours (VOH) is implemented based on the IMP, aiming at increasing student-teacher interactions. For evaluating the effectiveness of this scheme, 140 undergraduate students using IMP are divided into one control group and three experimental groups that respectively contain text, voice and video modes. The experiment results indicate that applying VOH in the IMP can improve students' online presence and test scores. Furthermore, students' participating modes during VOH implementation can largely affect their degree of presence, which can be well classified by using principal component analysis. The implication of this work is that IMP-based VOH is an effective and sustainable tool to be continuously implemented even when the COVID-19 pandemic period ends.

Keywords

1. Introduction

During the prevention and critical control period of COVID-19, in most countries, the original learning mode of students has been changed. From the survey by UNESCO in April 2020, the unprecedented disruption in offline teaching has affected more than 1.5 billion students [1]. How to complete teaching tasks has become a vital concern for universities. Under this circumstance, more and more universities are obligated to perform online teaching in post-COVID-19 pandemic era.

Based on the information immediacy principle, online teaching can be divided into three main modes: synchronous online teaching, asynchronous online teaching, and blended online teaching. Among them, synchronous online teaching requires platforms for live broadcasting like ZOOM, Tencent Meeting, etc. Asynchronous online teaching refers to websites like MOOCs. In this mode, teachers release materials in advance so that students can control the pace of learning by themselves. Blended online teaching requires students to finish self directed learning according to teachers’ instructions firstly. Then, teachers explain the important concept in the materials and answer the questions from students during class time.

As the above online teaching modes are carried out under the separation of space through the Internet, finding a multimedia platform to implement these modes online becomes a dominant issue for universities. Massive open online courses (MOOCs), one of the asynchronous multimedia platforms, provides convenience but limits the interactions between teacher and students.

Besides a multimedia platform to support online teaching in post-COVID-19 pandemic era, it is also crucial to consider about the teacher-student interaction. Several existing interactive multimedia platforms devise computer-supported collaborative learning functions and highlight the importance of designing virtual world environments [2]-[4]. Among them, virtual office hours (VOH) is a key interactive method [5]. However, there are some limitations in current VOH researches. Firstly, VOH has not been effectively implemented in multimedia platform to achieve teacher-student interaction in online teaching yet. Secondly, whether VOH with teacher-student interaction would affect students' learning effects or their online presence still need to be discussed. Thirdly, there are different modes of VOH with teacher-student interaction. Whether different VOH with teacher-student interaction would lead to different learning effects also needs to be studied.

Based on the above analysis, in this paper, we propose an integrated media platform-based virtual office hours implementation for online teaching. Our contributions can be summarized as follows:

1) We design an integrated media platform (IMP) in post-COVID-19 pandemic era and add a teacher-student interaction VOH to this platform.

2) We find that VOH with teacher-student interaction has great influence on students’ learning effects. The CoI survey instrument and final exam illustrate that VOH in online teaching is correlated to students’ presence and test scores, and positively affects these two factors. The t-test of CoI survey instrument shows that cognitive presence is significantly affected by VOH in post-COVID-19 pandemic era.

3) We prove that different modes of VOH with teacher-student interaction are associate with students’ learning outcomes. The classification of principal components reveals that students’ unique VOH implementation modes can be interpreted.

The sections below are structured as follows. Section 2 discusses the related work of existing teaching modes and multimedia platforms. Section 3 shows the concept and design of our IMP. Section 4 covers the application of VOH with teacher-student interactions for IMP, including the details of participants, measurements and experiment procedure settings. Section 5 presents the results of t-test, variance analysis and factor analysis. Section 6 gives conclusions as well as expectations for future work.

2. Literature Review

2.1 Existing Teaching Modes

The existing teaching models mainly include traditional classroom teaching, large-scale open online course (MOOCs) teaching, flipped classrooms [6-9]. Their respective characteristics and advantages and disadvantages are shown in Table 1.

Table 1. Three different kinds of teaching modes

E1KOBZ_2021_v15n8_2732_t0001.png 이미지

The learning content of most basic theory courses in engineering college is relatively abstract and boring. It makes students generally find it difficult to learn and understand, lacking interest in learning. Thus, it is necessary to explore new teaching models to better motivate students' learning enthusiasm.

2.2 Multimedia Platform

Integrating graphics, text, audio, video, and animation, as modern educational technology, multimedia platform definitely make classroom teaching in schools full of interest. It spreads multi-dimensional and diversified information [10]. For some abstract and difficult content in classroom teaching, multimedia platform helps to display them vividly. Then, it can effectively stimulate students’ multimodal perceptions and improve students' attention and enthusiasm for learning. In this way, students obtain better learning effects.

Despite of these advantages, [11] found that some teacher uses their courseware in multimedia platform but ignores the time spend on the knowledge formation. When teacher flashes their courseware, students cannot catch up with the courseware. This phenomenon makes it difficult for students to comprehend the knowledge taught in class. Therefore, it is essential for teachers to pay attention to the feedback of students when using multimedia platforms. In others words, teacher-student interaction should not be ignored during online teaching.

3. Integrated Media Platform

Integrated media platform does not simply transplant course lecture videos, tests, Q&A, etc. to another platform. This platform enables teachers to fully grasp students’ learning situation immediately under the post-COVID-19 pandemic era [12]. Its design goal is to stimulate and motivate their interest in course learning, deepen their understanding of class teaching knowledge. The design concept includes multimedia presentation, multi-resource aggregation, and multi-mechanism guarantee. Based on this principle, we construct an integrated media platform (IMP), shown in Fig. 1. The platform is functioned with three main modules, including self-directed learning module, platform management module, and teacher-student interaction module.

E1KOBZ_2021_v15n8_2732_f0001.png 이미지

Fig. 1. Structure of the constructed integrated media platform

3.1 Self-directed Learning

This module includes image-text learning and audio-visual learning, which are overlapping. On the one hand, image and text of this module are convenient for students to read and consolidate the prerequisite knowledge. Furthermore, it protects teachers from spending too much class time on basic content. On the other hand, audio-visual animations help demonstrate the course content clearly, motivating students’ interest, expanding students' knowledge, stimulating their cognition level of their major.

3.2 Platform Management

This module mainly conducts students’ supervision and communication. The teacher as administrator supervises students’ learning status through their behaviors. Students’ behavior including their marks gets in individual quiz and battle quiz. The purpose of regular quiz is to enable students to make full use of the scattered time outside of class. Without being restricted by time and place, students deepen their understanding of the course concepts and promote communication with their classmates. Our IMP gives their self-report to students and teachers in ranked marks so as to make them informed of their behaviors.

3.3 Teacher-student Interaction

The interaction module consists of three modes, discussion in text mode, voice mode and video mode. Text mode is manifested by discussion board, where teacher gives a post and students leave their message. Voice mode and video mode are carried by teacher’s special post ask whether someone needs a synchronous meeting on the discussion board. If teacher gets replies, he or she arrange a feasible schedule for students to meet online through a live link. Students click the link if they need an interview with the teacher.

4. Methods

In this section, VOH with teacher-student interaction is applied to IMP. In order to evaluate the effect of this online teaching scheme, students’ online presence and final test scores are used to explain their subjective experience and learning outcomes.

4.1 Virtual Office Hours Interactions

Office hours is defined as out-of-class consultations when the professor and teaching assistant arrange time outside the classroom with the students [13]. It is an important extension of classroom where content presented in the class or students’ personal concerns can be discussed [14].

However, there exist defects in traditional office hours. Firstly, it does not scale very well with limited teachers who need to meet large amounts of students. Moreover, it is inconvenient because students may spend a lot of time to university for attending the office hours meeting. In order to solve the above problems, virtual office hours (VOH) was put forward to perform office hours under the online environment [5]. Researchers in Harvard university introduced VOH to introductory computer science course CS 50 using a third-party software, receiving positive responses [15].

Without physical interaction in the post-pandemic period, holding the VOH is highly necessary. For the purpose of observing how VOH effects students’ performance, we carry out experiments by using the interaction module of IMP. Concretely, VOH is implemented in teacher-student interaction module. When the post to ask whether students need to attend the synchronous meeting is uploaded, students who need to attend reply with “1”. After checking office hours, teacher provides the meeting link at the bottom of that post. By clicking the link, students can see the information about time schedule to attend the synchronous meeting. The meeting includes two modes, the voice meeting and video meeting. The difference of these two modes is whether participants open the camera. The operation is explained in 4.3 Procedure in detail.

4.2 Measurements

Here, a class of 140 sophomores who major in Electronic and Communication Engineering from Nanjing University of Posts and Telecommunications, China were selected. These students had already passed the prerequisites such as ‘Advanced Mathematics’ and ‘Circuit Analysis’. Here, we selected a basic and representative course ‘Signals and Systems’ for evaluating online teaching effects.

4.2.1 Online Presence

Presence is explained as a perception of being in the learning community from the perspective of space and emotion [16,17]. It means students’ subjective experience in different physical environment [18,19].

It is always related to teachers’ behavior, gesture and emotion, students’ study environment and cognitive scale. In order to employ qualitative modes of enquiry, community of inquiry (CoI) survey instrument is put forward [20,21].

Specifically, in Garrison’s CoI theory shown in Fig. 2, presence can be divided into three elements: teaching presence, social presence and cognitive presence. Among them, teaching presence is often referred to as teachers’ instruction of designing, facilitating, and directing to realize educationally worthwhile [22]. Social presence is defined as the level of emotional expression or noticing others while getting along with course participants [23]. Cognitive presence is described as students learning process of knowledge. It includes triggering event, exploration, construction and resolution [16, 24, 25]. Changes to different elements of presence may lead to different learning effects.

E1KOBZ_2021_v15n8_2732_f0002.png 이미지

Fig. 2. Garrison’s CoI framework

Students’ online presence was measured by CoI survey instrument contained 34 questions, where teaching presence (TP) is measured by question TP1-TP13, social presence (SP) is measured by question SP1-SP9, cognitive presence (CP) is measured by question CP1-CP12 [21, 26, 27]. The CoI survey instrument was utilized on a 5-point Likert scale. When students felt some questions hard to respond, an addition option “It is difficult for me to answer” was also permitted.

4.2.2 Final Test

The test was designed by the teacher in the study. All of the items were derived from the learning materials presented in the ‘Signal and Systems’ course.

4.3 Procedure

Here, 140 students were divided into four groups in our experiment. Students were alternative to choose if they would like to participate in VOH. To further explore the effect of three different modes, we first randomly assigned the students participated in VOH to three different groups, 11 in text group, 10 in voice group, and 10 in video group. Then, the remaining students were taken as control group. Considering that most of the students have not taken part in office hours before, meeting alone may make them uncomfortable. Therefore, three experimental groups were designed to share the collaborative meeting space. Specifically, each student in these three groups firstly prepared and reviewed for the final test. They needed to submit their queries to the discussion board in IMP during review process. Subsequently, students in these three experimental groups attended their VOH, respectively. Table 2 showed the details of VOH arrangement. Immediately after VOH, all of students filled out the CoI questionnaire, reflecting their presence. Finally, they completed a final test for this course. Fig. 3 depicts the course VOH using IMP. Students in text group directly get the reply documents in IMP. Students in voice group can share screen and voice. That is, they can hear teacher’s voice as the material shown in screen. For video group, students not only share screen, voice and image but also observe teacher’s instructive expressions. We hypothesize that students seeing the image of teacher are more likely to achieve better learning results.

Table 2. The details of VOH arrangement

E1KOBZ_2021_v15n8_2732_t0002.png 이미지

E1KOBZ_2021_v15n8_2732_f0003.png 이미지

Fig. 3. VOH in IMP

5. Results

Questionnaires were sent to 140 students and a total of 102 valid responses were received, 71 from control group, 11 from text group, 10 from voice group, and 10 from video group. In this section, effects of VOH on the above four groups on presence and test scores were evaluated by t-test and one-way ANOVA. After that, unique participating modes of students were classified by principal component analysis (PCA).

5.1 Effect to Learning Presence

T-test is an efficient way to show the difference between two variables. To explore whether VOH participating students have higher online presence, we applied t-test to the students’ performances on the CoI survey instrument. From the results of t-test, only CP10 and CP12 items had significant impacts. The condition with or without the assumption of equal variances for CP10 and CP12 were considered in Table 3, respectively. When Sig. of Levene’s Test for Equality of Variances greater than 0.05, the condition for equal variances was checked, otherwise not. Thus, CP10 (t(100) = -2.022, p = 0.046) and CP12 (t(100) = -2.599, p = 0.011) should be considered. The heading of CP10 is “I had many ways of testing and applying the knowledge derived from this course”. The CP12 is “I utilized strategies learned from this course to solve class exercises or concrete issues in real life,” [28]. The results partly supported students taking part in VOH would show higher cognitive presence. Specifically, CP10 and CP12 represent the knowledge applying and transfer in cognitive learning.

Table 3. Result of independent sample test

E1KOBZ_2021_v15n8_2732_t0003.png 이미지

Equal var means the condition for equal variances can be checked, vice versa

* means p less than 0.05

5.2 Effect to Test Scores

Here, means and standard deviations of the students’ final grades were presented in Fig. 4. It showed that three experimental groups obtained higher final grades than the control group. Moreover, video group and voice group displayed higher stability. One-way ANOVA is an effective method to determine whether different implementation modes have an effect on test scores. The Welch and Brown-Forsythe in one-way ANOVA both revealed a main effect of the VOH condition. In Table 4, Post hoc Games-Howell test showed that experimental groups outperformed control group in final test scores significantly. The results demonstrated students belonging to three experimental groups indeed had good learning effects online under teacher’s guidance in VOH.

E1KOBZ_2021_v15n8_2732_f0004.png 이미지

Fig. 4. Means and standard deviations of the students’ final scores

Table 4. Post hot Games-Howell test

E1KOBZ_2021_v15n8_2732_t0004.png 이미지

* means p less than 0.05

5.3 Participating Modes Classification

In the above two subsections, we found that adding VOH can promote students’ online presence and test scores. Here, we tried to further concern the influence of VOH implementation modes on students’ online presence with CoI survey instrument.

Reliability and validity analysis is needed before factor analysis. The reliability coefficient of the questionnaires is 0.961. According to [29,30], the Cronbach coefficient reflects the reliability of the questionnaire. If its value is greater than 0.7, the questionnaire can be considered reliable and there also exists degree of difference among question items. When the reliability coefficient is greater than 0.9, it reveals that the respondents have good consistency in their responses to the questionnaire. This result illustrates that the data reliability quality is high. What is more, the KMO value is 0.804, greater than 0.6 [31]. The significance is less than 0.05, indicating a high degree of correlation among factors. Therefore, the collected data can be used for further analysis.

The purpose of factor analysis is to find intrinsic factors that can reflect the internal relationships among variables. The key role of these factors in the original data cannot be ignored. Through factor analysis, we can not only retain the characteristics of original data but also acquire the internal relationship between VOH implementation modes and students’ presence. Compared with other dimension reduction methods like LDA, LLE, Laplacian Eigenmaps, etc. principal component analysis (PCA) is applied in our experiment [32-34]. It is also one of the most widely used data dimension reduction schemes. Specifically, the main idea of this scheme is to map N-dimension original data to K-dimension feature, which should be as dispersed as possible in the internal relationship space. That is to say, we need to find the largest variance when all the original data is converted to implicit subspace.

Here, variance is expressed as \(\operatorname{Var}(x)=\frac{1}{m} \sum_{i=1}^{m}\left(x_{i}-\mu\right)^{2}\). But if only picking the direction with the greatest variance, the other directions will be close to that direction. In other words, a linear correlation between any two directions is not expected because we want to represent the original information as much as possible. The correlation is expressed as the covariance of two fields, which is \(\operatorname{Cov}(x, y)=\frac{1}{m} \sum_{i=1}^{m} x_{i} y_{i}\). Only when the covariance is zero, the features represented are orthogonal. Therefore, diagonalization of covariance matrix is required by reducing the elements other than the diagonal to zero. In this way, prior conditions for PCA are satisfied.

From the data, 34 questions in CoI represent the presence were extracted [35]. The communalities of rescaled extraction among these 34 items were ranged from 0.576 to 0.880. As the CoI questions were from the same magnitude, the covariance matrix was applied for the PCA dimension reduction. As shown in Fig. 5, when the component number increase, the curve becomes flat after 4 components. From Table 5, the eigenvalues in column of total, the top four principal components were all larger than 1. While starting from the fifth, the eigenvalues were less than 1. Therefore, only four greatest principal components were kept in default. In other words, the top four principal components were capable to explain 75.887 percent of the total variation. Moreover, when we set the component number as 3 or 5, it is hard to figure out the regularity and interpretability in our experiment.

E1KOBZ_2021_v15n8_2732_f0005.png 이미지

Fig. 5. Scree plot of PCA

Table 5. Total variance explained (only top 5 components are listed)

E1KOBZ_2021_v15n8_2732_t0005.png 이미지

We carried out PCA for the 34×34 CoI covariance matrix for 102 valid response data. Firstly, a 34×4 component matrix was computed. Since not all coefficients of questions in the CoI survey instrument were high, we listed the high ranked features correspond to significant PC entries in positive or negative value. After that, we speculated a possible explanation to the PCs in Table 6. This table showed the feature weights for PC1, PC2, PC3 and PC4.

Table 6. Component matrix for PC1, PC2, PC3 and PC4. Significant entries are colored red (positive values) and blue (negative values)

E1KOBZ_2021_v15n8_2732_t0006.png 이미지

From this table, features providing the largest positive contribution to PC1 were CP1, CP2, CP4, CP8, SP5 and SP6. Students with a high projection on PC1 had a strong sense of cognitive and social presence. Students’ learning activities used to solve problems created high cognitive presence. They felt comfortable when they provided queries or discussed in online scenarios. These students can be classified as those participating in text VOH. They provided queries by handling in texts, got replies back also from texts. Students showed the highest satisfaction with simple VOH design among experimental groups [36]. For illustration in Fig. 6, consider students in this PC, they achieved higher average value in these positive features listed above.

E1KOBZ_2021_v15n8_2732_f0006.png 이미지

Fig. 6. Students’ CoI value in CP1, CP2, CP4, CP8 and SP5, SP6

Similarly, the top largest positive entries correspond to PC2 were TP2, TP3 and TP6 and the top largest negative entries correspond to PC2 were SP5, SP6, SP8 and SP9. Since the factor loadings of SP8 and SP9 showed higher significance in PC3 and it would be discussed in the following section of PC3. Students with PC2 projection had high teaching presence but low social presence. They confirmed the helpful effect of instructions. Yet limited contribution was found in online discussion. These students could be classified in video mode VOH as teacher communicated with them face to face. Thus, teacher was capable to clarify the teaching goal and instructions, indicating a high teaching presence in Fig. 7. However, they listened carefully without discussing with others during the whole meeting, demonstrating low social presence.

E1KOBZ_2021_v15n8_2732_f0007.png 이미지

Fig. 7. Students’ CoI value in TP2, TP3, TP6 and SP5, SP6

In Table 6, the features that have the greatest positive impact on PC3 were CP7, CP11, CP12. The features that have the greatest negative impact on PC3 were SP4, SP8 and SP9. We could obviously see that students with PC3 projection had high cognitive presence and low social presence. These students did not feel good when they conversed through Tencent Meeting thus had little communication with other course participants. Without communication, they could hardly develop a sense of collaboration or feel acknowledged. What is more, combing new information and developing solutions to course problems alone were common to them. For illustration, even plot of all significant contributions (CP7, CP11, CP12, SP4, SP8, SP9) were overlapping, the negative contribution questions (SP4, SP8, SP9) could show clear mode feature as well. In Fig. 8, students manifested low social presence. Students in high PC3 projection can be classified in voice mode VOH. When they were in the meeting, students muted themselves which mainly prohibit chance of immediate communication. Therefore, practice and activities were finished independently after class.

E1KOBZ_2021_v15n8_2732_f0008.png 이미지

Fig. 8. Students’ CoI value in SP4, SP8, SP9

As is illustrated in Table 6, all factor loadings in PC4 were less than 0.35 in absolute value. With high projection on PC4, students were low in all three kinds of presence. This group of students did not show clear features compared to other groups. Some participants in this course could hardly leave a distinct impression on their classmates. Meanwhile, because they did not know each other, it is difficult to form a sense of course belonging. Since the rest three principal components of students were in obvious mode feature, this group was classified as control group. Control group did not take part in VOH, losing the chance to know more about their classmates.

6. Conclusion

This paper designs an IMP to improve the interactions and even learning outcomes in post- COVID-19 pandemic era. We also study the influence on students’ presence in the online course ‘Signals and Systems’ when VOH is added in IMP. What stands out in our present work is that VOH with teacher-student interaction would lead to higher online presence and grades in post-COVID-19 pandemic period. In addition, cognitive presence is significantly affected by VOH. Furthermore, students’ participating modes can be projected to different groups and can be effectively separated by PCA. When the unique students’ trajectories were found, early intervention or prior instructions may be utilized to those students in weak behaving groups. The findings have practical implications for educational practices to improve students’ learning outcomes. Because IMP is also not constrained by time and space, it encourages universities to launch platforms like IMP in post-COVID-19 pandemic era.

In our future work, the following issues will be studied. First, the performance of students involved in experiment will still be deserved to observe in other disciplines. Second, the study enrolled are Chinese sophomores. Future researches could explore whether the VOH implementation has the same effects on learners from different cultures considering presence in online teaching.

Acknowledgement

This work is partly supported by the National Natural Science Foundation of China (Grant No. 62071254), the Postgraduate Research and Practice Innovation Program of Jiangsu Province (Grant No. KYCX20_0733), Teaching Reform Research Project of NJUPT (Grant No. JG00220JX02), the open research fund of Key Lab of Broadband Wireless Communication and Sensor Network Technology (Nanjing University of Posts and Telecommunications), Ministry of Education (Grant No. JZNY202111), and the Priority Academic Program Development of Jiangsu Higher Education Institutions.

References

  1. UNESCO, "Startling disparities in digital learning emerge as covid-19 spreads: Un education agency," Tech. rep., UN agency's Institute for Statistics and the International Telecommunication Union, Apr. 21, 2020. [Online]. Available: https://news.un.org/en/story/2020/04/1062232
  2. D.R. Garrison, E-learning in the 21st century, New York, NY, USA: RoutledgeFalmer, 2003
  3. Y. Masanori, M. Hideya, G. Yoshiko, Y. Seisuke, and H. Kojiro, "A computer-supported collaborative learning design for quality interaction," IEEE Multimedia, vol. 23, no. 1, pp. 48-59, 2016. https://doi.org/10.1109/MMUL.2015.95
  4. S.T. Bulu, "Place presence, social presence, co-presence, and satisfaction in virtual worlds," Computers & Education, vol. 58, no. 1, pp. 154-161, 2012. https://doi.org/10.1016/j.compedu.2011.08.024
  5. C.A. Merlic and M.J. Walker, "World Conference of the Web Society San Francisco," in Proc. of WebNet 96, California, USA, October 15-19, 1996.
  6. E. Murphy, MA Rodriguez-Manzanares, and M. Barbour, "Asynchronous and synchronous online teaching: Perspectives of Canadian high school distance education teachers," British Journal of Educational Technology, vol. 42, no. 4, pp.583-591, 2011. https://doi.org/10.1111/j.1467-8535.2010.01112.x
  7. A. M. Kaplan, M. Haenlein, "Higher education and the digital revolution: About MOOCs, SPOCs, social media, and the Cookie Monster," Business Horizons, vol. 59, no. 4, pp. 441-450, 2016. https://doi.org/10.1016/j.bushor.2016.03.008
  8. Y. Dong, J. Ang, and Z. Sun, "Designing Path of SPOC blended teaching and learning mode in post-MOOC Era," in Proc. of ICEIT 2021, Chengdu, China, pp. 24-28, Jan. 18-20, 2021.
  9. J. Singh, S. Nagpal, S. Inglis, and J. Jacob-John, "International students' experiences in a flipped classroom environment: An Australian perspective," International Journal of Educational Management, vol. 33, no. 6, pp. 1303-1315, 2019.
  10. K. Kounlaxay, Y. Shim, S. Kang, H. Kwak, and S. K. Kim, "Learning Media on Mathematical Education based on Augmented Reality," KSII Transactions on Internet and Information Systems, vol. 15, no. 3, pp. 1015-1029, 2021.
  11. S. Cairncross and M. Mannion, "How multimedia functions in engineering education," Engineering Science & Education Journal, vol. 8, no. 3, pp. 100-106, 2017. https://doi.org/10.1049/esej:19990302
  12. J.S. Fu and M. Shumate, "News media, social media, and hyperlink networks: An examination of integrated media effects," The Information Society, vol. 33, no. 2, pp. 53-63, 2017. https://doi.org/10.1080/01972243.2016.1271379
  13. J.C. Gallego, "A survey of native and nonnative TAs' office hours: Importance, attendance, and content," ERIC, pp. 18, 1995.
  14. B. Gibbons-Kunka, "Synchronous office hours in an asynchronous course: Making the connection," International journal of information and communication technology education: an official publication of the Information Resources Management Association, vol. 13, no. 4, pp. 98-110, 2017. https://doi.org/10.4018/ijicte.2017100108
  15. D. J. Malan, "Virtualizing Office Hours in CS50," in Proc. of 14th Annual ACM Conference on Innovation and Technology in Computer Science Education, Paris, France, July, 2009.
  16. Y.J. Joo, K.Y. Lim, and E.K. Kim, "Online university students' satisfaction and persistence: Examining perceived level of presence, usefulness and ease of use as predictors in a structural model," Computers & Education, vol. 57, no. 2, pp. 1654-1664, 2011. https://doi.org/10.1016/j.compedu.2011.02.008
  17. K. Kozan and J.C. Richardson, "Interrelationships between and among social, teaching, and cognitive presence," The Internet and Higher Education, vol. 21, no. 2, pp. 68-73, 2014. https://doi.org/10.1016/j.iheduc.2013.10.007
  18. J. Steuer, "Defining virtual reality: Dimensions determining telepresence," Journal of Communication, vol. 42, no. 4, pp. 73-93, 1992. https://doi.org/10.1111/j.1460-2466.1992.tb00812.x
  19. B.G. Witmer and M.J. Singer, "Measuring presence in virtual environments: A presence questionnaire," Presence: Teleoperators and Virtual Environments, vol. 7, no. 7, pp. 225-240, 1998. https://doi.org/10.1162/105474698565686
  20. D.R. Garrison, M. Clevelandinnes, and T.S. Fung, "Exploring causal relationships among teaching, cognitive and social presence: Student perceptions of the community of inquiry framework," Internet & Higher Education, vol. 13, no. 1, pp. 31-36, 2010. https://doi.org/10.1016/j.iheduc.2009.10.002
  21. K.P. Swan, J.C. Richardson, P. Ice, D.R. Garrison, and J.B. Arbaugh, "Validating a measurement tool of presence in online communities of inquiry," E-Mentor, vol. 2, 2008.
  22. T. Anderson, L. Rourke, D.R. Garrison, and W. Archer, "Assessing teaching presence in a computer conferencing context," JALN, vol. 5, no. 2, pp. 1-17, 2001.
  23. C.H. Tu and M. McIsaac, "The relationship of social presence and interaction in online classes," American Journal of Distance Education, vol. 16, no. 3, pp. 131-150, 2002. https://doi.org/10.1207/S15389286AJDE1603_2
  24. D.R. Garrison, "Online community of inquiry review: Social, cognitive, and teaching presence issues," Online Learning, vol. 11, no. 1, 2007.
  25. S. Kilis and Z. Yildirim, "Posting patterns of students' social presence, cognitive presence, and teaching presence in online learning," Online Learning, vol. 23, no. 3, pp. 179-195, 2019.
  26. J.B. Arbaugh, M. Cleveland-Innes, S.R. Diaz, D.R. Garrison, P. Ice, J.C. Richardson, and K.P. Swan, "Developing a community of inquiry instrument: Testing a measure of the community of inquiry framework using a multi-institutional sample," Internet & Higher Education, vol. 11, no. 3-4, pp. 133-136, 2012. https://doi.org/10.1016/j.iheduc.2008.06.003
  27. P. Shea and T. Bidjerano, "Measures of quality in online education: An investigation of the community of inquiry model and the net generation," Journal of Educational Computing Research, vol. 39, no. 4, pp. 339-361, 2008. https://doi.org/10.2190/EC.39.4.b
  28. V. Kovanovic, S. Joksimovic, O. Poquet, T. Hennis, I. Cukic, P. De Vries, M. Hatala, S. Dawson, G. Siemens, and D. Gasevic, "Exploring communities of inquiry in massive open online courses," Computers & Education, vol. 119, no. 4, pp. 44-58, 2018. https://doi.org/10.1016/j.compedu.2017.11.010
  29. J. Kwon and J. Kim, "A Study on the Design and Effect of Computational Thinking and Software Education," KSII Transactions on Internet and Information Systems, vol. 12, no. 8, pp. 4057-4071, 2018. https://doi.org/10.3837/tiis.2018.08.028
  30. C. Zhao and C. Ma, "A Research on the Relationship between Teaching Presence and Online Learners' Basic Psychological Needs" in Proc. of the 2018 International Conference, Beijing, China, May. 26-28, 2018.
  31. H.F. Kaiser, "An index of factorial simplicity," Psychometrika, vol. 39, no. 1, pp. 31-36, 1974. https://doi.org/10.1007/BF02291575
  32. W.T. Anderson, "An introduction to multivariate statistical analysis," Biometrics, vol. 41, no. 3, pp. 815, 1985. https://doi.org/10.2307/2531310
  33. I.T. Jolliffe, "Principal component analysis," Journal of Marketing Research, vol. 25, no. 4, p. 410, 1988.
  34. H. Hershcovits, D.Vilenchik, and K.Gal, "Modeling engagement in self-directed learning systems using principal component analysis," IEEE Transactions on Learning Technologies, vol. 13, no. 1, pp. 164-171, 2020. https://doi.org/10.1109/tlt.2019.2922902
  35. P. Shea and T. Bidjerano, "Learning presence as a moderator in the community of inquiry model," Computers & Education, vol. 59, no. 2, pp. 316-326, 2012. https://doi.org/10.1016/j.compedu.2012.01.011
  36. E. Murphy, M.A. Rodriguez-Manzanares, and M. Barbour, "Asynchronous and synchronous online teaching: Perspectives of canadian high school distance education teachers," British Journal of Educational Technology, vol. 42, no. 4, pp. 583-591, 2011. https://doi.org/10.1111/j.1467-8535.2010.01112.x