Integrated Media Platform-based Virtual Office Hours Implementation for Online Teaching in Post-COVID-19 Pandemic Era

Abstract

In post-COVID-19 pandemic era, students' learning effects and experience may sharply decrease when teaching is transferred from offline to online. Several tools suitable for online teaching have been developed to guarantee and promote students' learning effects. However, they cannot fully consider teacher-student interaction in online teaching. To figure out this issue, this paper proposes integrated media platform-based virtual office hours implementation for online teaching. Specifically, an integrated media platform (IMP) is first constructed. Then, virtual office hours (VOH) is implemented based on the IMP, aiming at increasing student-teacher interactions. For evaluating the effectiveness of this scheme, 140 undergraduate students using IMP are divided into one control group and three experimental groups that respectively contain text, voice and video modes. The experiment results indicate that applying VOH in the IMP can improve students' online presence and test scores. Furthermore, students' participating modes during VOH implementation can largely affect their degree of presence, which can be well classified by using principal component analysis. The implication of this work is that IMP-based VOH is an effective and sustainable tool to be continuously implemented even when the COVID-19 pandemic period ends.

1. Introduction

During the prevention and critical control period of COVID-19, in most countries, the original learning mode of students has been changed. From the survey by UNESCO in April 2020, the unprecedented disruption in offline teaching has affected more than 1.5 billion students [1]. How to complete teaching tasks has become a vital concern for universities. Under this circumstance, more and more universities are obligated to perform online teaching in post-COVID-19 pandemic era.

Based on the information immediacy principle, online teaching can be divided into three main modes: synchronous online teaching, asynchronous online teaching, and blended online teaching. Among them, synchronous online teaching requires platforms for live broadcasting like ZOOM, Tencent Meeting, etc. Asynchronous online teaching refers to websites like MOOCs. In this mode, teachers release materials in advance so that students can control the pace of learning by themselves. Blended online teaching requires students to finish self directed learning according to teachers’ instructions firstly. Then, teachers explain the important concept in the materials and answer the questions from students during class time.

As the above online teaching modes are carried out under the separation of space through the Internet, finding a multimedia platform to implement these modes online becomes a dominant issue for universities. Massive open online courses (MOOCs), one of the asynchronous multimedia platforms, provides convenience but limits the interactions between teacher and students.

Besides a multimedia platform to support online teaching in post-COVID-19 pandemic era, it is also crucial to consider about the teacher-student interaction. Several existing interactive multimedia platforms devise computer-supported collaborative learning functions and highlight the importance of designing virtual world environments [2]-[4]. Among them, virtual office hours (VOH) is a key interactive method [5]. However, there are some limitations in current VOH researches. Firstly, VOH has not been effectively implemented in multimedia platform to achieve teacher-student interaction in online teaching yet. Secondly, whether VOH with teacher-student interaction would affect students' learning effects or their online presence still need to be discussed. Thirdly, there are different modes of VOH with teacher-student interaction. Whether different VOH with teacher-student interaction would lead to different learning effects also needs to be studied.

Based on the above analysis, in this paper, we propose an integrated media platform-based virtual office hours implementation for online teaching. Our contributions can be summarized as follows:

1) We design an integrated media platform (IMP) in post-COVID-19 pandemic era and add a teacher-student interaction VOH to this platform.

2) We find that VOH with teacher-student interaction has great influence on students’ learning effects. The CoI survey instrument and final exam illustrate that VOH in online teaching is correlated to students’ presence and test scores, and positively affects these two factors. The t-test of CoI survey instrument shows that cognitive presence is significantly affected by VOH in post-COVID-19 pandemic era.

3) We prove that different modes of VOH with teacher-student interaction are associate with students’ learning outcomes. The classification of principal components reveals that students’ unique VOH implementation modes can be interpreted.

The sections below are structured as follows. Section 2 discusses the related work of existing teaching modes and multimedia platforms. Section 3 shows the concept and design of our IMP. Section 4 covers the application of VOH with teacher-student interactions for IMP, including the details of participants, measurements and experiment procedure settings. Section 5 presents the results of t-test, variance analysis and factor analysis. Section 6 gives conclusions as well as expectations for future work.

2. Literature Review

2.1 Existing Teaching Modes

The existing teaching models mainly include traditional classroom teaching, large-scale open online course (MOOCs) teaching, flipped classrooms [6-9]. Their respective characteristics and advantages and disadvantages are shown in Table 1.

Table 1. Three different kinds of teaching modes

The learning content of most basic theory courses in engineering college is relatively abstract and boring. It makes students generally find it difficult to learn and understand, lacking interest in learning. Thus, it is necessary to explore new teaching models to better motivate students' learning enthusiasm.

2.2 Multimedia Platform

Integrating graphics, text, audio, video, and animation, as modern educational technology, multimedia platform definitely make classroom teaching in schools full of interest. It spreads multi-dimensional and diversified information [10]. For some abstract and difficult content in classroom teaching, multimedia platform helps to display them vividly. Then, it can effectively stimulate students’ multimodal perceptions and improve students' attention and enthusiasm for learning. In this way, students obtain better learning effects.

Despite of these advantages, [11] found that some teacher uses their courseware in multimedia platform but ignores the time spend on the knowledge formation. When teacher flashes their courseware, students cannot catch up with the courseware. This phenomenon makes it difficult for students to comprehend the knowledge taught in class. Therefore, it is essential for teachers to pay attention to the feedback of students when using multimedia platforms. In others words, teacher-student interaction should not be ignored during online teaching.

3. Integrated Media Platform

Integrated media platform does not simply transplant course lecture videos, tests, Q&A, etc. to another platform. This platform enables teachers to fully grasp students’ learning situation immediately under the post-COVID-19 pandemic era [12]. Its design goal is to stimulate and motivate their interest in course learning, deepen their understanding of class teaching knowledge. The design concept includes multimedia presentation, multi-resource aggregation, and multi-mechanism guarantee. Based on this principle, we construct an integrated media platform (IMP), shown in Fig. 1. The platform is functioned with three main modules, including self-directed learning module, platform management module, and teacher-student interaction module.

Fig. 1. Structure of the constructed integrated media platform

3.1 Self-directed Learning

This module includes image-text learning and audio-visual learning, which are overlapping. On the one hand, image and text of this module are convenient for students to read and consolidate the prerequisite knowledge. Furthermore, it protects teachers from spending too much class time on basic content. On the other hand, audio-visual animations help demonstrate the course content clearly, motivating students’ interest, expanding students' knowledge, stimulating their cognition level of their major.

3.2 Platform Management

This module mainly conducts students’ supervision and communication. The teacher as administrator supervises students’ learning status through their behaviors. Students’ behavior including their marks gets in individual quiz and battle quiz. The purpose of regular quiz is to enable students to make full use of the scattered time outside of class. Without being restricted by time and place, students deepen their understanding of the course concepts and promote communication with their classmates. Our IMP gives their self-report to students and teachers in ranked marks so as to make them informed of their behaviors.

3.3 Teacher-student Interaction

The interaction module consists of three modes, discussion in text mode, voice mode and video mode. Text mode is manifested by discussion board, where teacher gives a post and students leave their message. Voice mode and video mode are carried by teacher’s special post ask whether someone needs a synchronous meeting on the discussion board. If teacher gets replies, he or she arrange a feasible schedule for students to meet online through a live link. Students click the link if they need an interview with the teacher.

4. Methods

In this section, VOH with teacher-student interaction is applied to IMP. In order to evaluate the effect of this online teaching scheme, students’ online presence and final test scores are used to explain their subjective experience and learning outcomes.

4.1 Virtual Office Hours Interactions

Office hours is defined as out-of-class consultations when the professor and teaching assistant arrange time outside the classroom with the students [13]. It is an important extension of classroom where content presented in the class or students’ personal concerns can be discussed [14].

However, there exist defects in traditional office hours. Firstly, it does not scale very well with limited teachers who need to meet large amounts of students. Moreover, it is inconvenient because students may spend a lot of time to university for attending the office hours meeting. In order to solve the above problems, virtual office hours (VOH) was put forward to perform office hours under the online environment [5]. Researchers in Harvard university introduced VOH to introductory computer science course CS 50 using a third-party software, receiving positive responses [15].

Without physical interaction in the post-pandemic period, holding the VOH is highly necessary. For the purpose of observing how VOH effects students’ performance, we carry out experiments by using the interaction module of IMP. Concretely, VOH is implemented in teacher-student interaction module. When the post to ask whether students need to attend the synchronous meeting is uploaded, students who need to attend reply with “1”. After checking office hours, teacher provides the meeting link at the bottom of that post. By clicking the link, students can see the information about time schedule to attend the synchronous meeting. The meeting includes two modes, the voice meeting and video meeting. The difference of these two modes is whether participants open the camera. The operation is explained in 4.3 Procedure in detail.

4.2 Measurements

Here, a class of 140 sophomores who major in Electronic and Communication Engineering from Nanjing University of Posts and Telecommunications, China were selected. These students had already passed the prerequisites such as ‘Advanced Mathematics’ and ‘Circuit Analysis’. Here, we selected a basic and representative course ‘Signals and Systems’ for evaluating online teaching effects.

4.2.1 Online Presence

Presence is explained as a perception of being in the learning community from the perspective of space and emotion [16,17]. It means students’ subjective experience in different physical environment [18,19].

It is always related to teachers’ behavior, gesture and emotion, students’ study environment and cognitive scale. In order to employ qualitative modes of enquiry, community of inquiry (CoI) survey instrument is put forward [20,21].

Specifically, in Garrison’s CoI theory shown in Fig. 2, presence can be divided into three elements: teaching presence, social presence and cognitive presence. Among them, teaching presence is often referred to as teachers’ instruction of designing, facilitating, and directing to realize educationally worthwhile [22]. Social presence is defined as the level of emotional expression or noticing others while getting along with course participants [23]. Cognitive presence is described as students learning process of knowledge. It includes triggering event, exploration, construction and resolution [16, 24, 25]. Changes to different elements of presence may lead to different learning effects.

Fig. 2. Garrison’s CoI framework

Students’ online presence was measured by CoI survey instrument contained 34 questions, where teaching presence (TP) is measured by question TP1-TP13, social presence (SP) is measured by question SP1-SP9, cognitive presence (CP) is measured by question CP1-CP12 [21, 26, 27]. The CoI survey instrument was utilized on a 5-point Likert scale. When students felt some questions hard to respond, an addition option “It is difficult for me to answer” was also permitted.

4.2.2 Final Test

The test was designed by the teacher in the study. All of the items were derived from the learning materials presented in the ‘Signal and Systems’ course.

4.3 Procedure

Here, 140 students were divided into four groups in our experiment. Students were alternative to choose if they would like to participate in VOH. To further explore the effect of three different modes, we first randomly assigned the students participated in VOH to three different groups, 11 in text group, 10 in voice group, and 10 in video group. Then, the remaining students were taken as control group. Considering that most of the students have not taken part in office hours before, meeting alone may make them uncomfortable. Therefore, three experimental groups were designed to share the collaborative meeting space. Specifically, each student in these three groups firstly prepared and reviewed for the final test. They needed to submit their queries to the discussion board in IMP during review process. Subsequently, students in these three experimental groups attended their VOH, respectively. Table 2 showed the details of VOH arrangement. Immediately after VOH, all of students filled out the CoI questionnaire, reflecting their presence. Finally, they completed a final test for this course. Fig. 3 depicts the course VOH using IMP. Students in text group directly get the reply documents in IMP. Students in voice group can share screen and voice. That is, they can hear teacher’s voice as the material shown in screen. For video group, students not only share screen, voice and image but also observe teacher’s instructive expressions. We hypothesize that students seeing the image of teacher are more likely to achieve better learning results.

Table 2. The details of VOH arrangement

Fig. 3. VOH in IMP

5. Results

Questionnaires were sent to 140 students and a total of 102 valid responses were received, 71 from control group, 11 from text group, 10 from voice group, and 10 from video group. In this section, effects of VOH on the above four groups on presence and test scores were evaluated by t-test and one-way ANOVA. After that, unique participating modes of students were classified by principal component analysis (PCA).

5.1 Effect to Learning Presence

T-test is an efficient way to show the difference between two variables. To explore whether VOH participating students have higher online presence, we applied t-test to the students’ performances on the CoI survey instrument. From the results of t-test, only CP10 and CP12 items had significant impacts. The condition with or without the assumption of equal variances for CP10 and CP12 were considered in Table 3, respectively. When Sig. of Levene’s Test for Equality of Variances greater than 0.05, the condition for equal variances was checked, otherwise not. Thus, CP10 (t(100) = -2.022, p = 0.046) and CP12 (t(100) = -2.599, p = 0.011) should be considered. The heading of CP10 is “I had many ways of testing and applying the knowledge derived from this course”. The CP12 is “I utilized strategies learned from this course to solve class exercises or concrete issues in real life,” [28]. The results partly supported students taking part in VOH would show higher cognitive presence. Specifically, CP10 and CP12 represent the knowledge applying and transfer in cognitive learning.

Table 3. Result of independent sample test

Equal var means the condition for equal variances can be checked, vice versa

* means p less than 0.05

5.2 Effect to Test Scores

Here, means and standard deviations of the students’ final grades were presented in Fig. 4. It showed that three experimental groups obtained higher final grades than the control group. Moreover, video group and voice group displayed higher stability. One-way ANOVA is an effective method to determine whether different implementation modes have an effect on test scores. The Welch and Brown-Forsythe in one-way ANOVA both revealed a main effect of the VOH condition. In Table 4, Post hoc Games-Howell test showed that experimental groups outperformed control group in final test scores significantly. The results demonstrated students belonging to three experimental groups indeed had good learning effects online under teacher’s guidance in VOH.

Fig. 4. Means and standard deviations of the students’ final scores

Table 4. Post hot Games-Howell test

* means p less than 0.05

5.3 Participating Modes Classification

In the above two subsections, we found that adding VOH can promote students’ online presence and test scores. Here, we tried to further concern the influence of VOH implementation modes on students’ online presence with CoI survey instrument.

Reliability and validity analysis is needed before factor analysis. The reliability coefficient of the questionnaires is 0.961. According to [29,30], the Cronbach coefficient reflects the reliability of the questionnaire. If its value is greater than 0.7, the questionnaire can be considered reliable and there also exists degree of difference among question items. When the reliability coefficient is greater than 0.9, it reveals that the respondents have good consistency in their responses to the questionnaire. This result illustrates that the data reliability quality is high. What is more, the KMO value is 0.804, greater than 0.6 [31]. The significance is less than 0.05, indicating a high degree of correlation among factors. Therefore, the collected data can be used for further analysis.

The purpose of factor analysis is to find intrinsic factors that can reflect the internal relationships among variables. The key role of these factors in the original data cannot be ignored. Through factor analysis, we can not only retain the characteristics of original data but also acquire the internal relationship between VOH implementation modes and students’ presence. Compared with other dimension reduction methods like LDA, LLE, Laplacian Eigenmaps, etc. principal component analysis (PCA) is applied in our experiment [32-34]. It is also one of the most widely used data dimension reduction schemes. Specifically, the main idea of this scheme is to map N-dimension original data to K-dimension feature, which should be as dispersed as possible in the internal relationship space. That is to say, we need to find the largest variance when all the original data is converted to implicit subspace.

Here, variance is expressed as \(\operatorname{Var}(x)=\frac{1}{m} \sum_{i=1}^{m}\left(x_{i}-\mu\right)^{2}\). But if only picking the direction with the greatest variance, the other directions will be close to that direction. In other words, a linear correlation between any two directions is not expected because we want to represent the original information as much as possible. The correlation is expressed as the covariance of two fields, which is \(\operatorname{Cov}(x, y)=\frac{1}{m} \sum_{i=1}^{m} x_{i} y_{i}\). Only when the covariance is zero, the features represented are orthogonal. Therefore, diagonalization of covariance matrix is required by reducing the elements other than the diagonal to zero. In this way, prior conditions for PCA are satisfied.

From the data, 34 questions in CoI represent the presence were extracted [35]. The communalities of rescaled extraction among these 34 items were ranged from 0.576 to 0.880. As the CoI questions were from the same magnitude, the covariance matrix was applied for the PCA dimension reduction. As shown in Fig. 5, when the component number increase, the curve becomes flat after 4 components. From Table 5, the eigenvalues in column of total, the top four principal components were all larger than 1. While starting from the fifth, the eigenvalues were less than 1. Therefore, only four greatest principal components were kept in default. In other words, the top four principal components were capable to explain 75.887 percent of the total variation. Moreover, when we set the component number as 3 or 5, it is hard to figure out the regularity and interpretability in our experiment.

Fig. 5. Scree plot of PCA

Table 5. Total variance explained (only top 5 components are listed)

We carried out PCA for the 34×34 CoI covariance matrix for 102 valid response data. Firstly, a 34×4 component matrix was computed. Since not all coefficients of questions in the CoI survey instrument were high, we listed the high ranked features correspond to significant PC entries in positive or negative value. After that, we speculated a possible explanation to the PCs in Table 6. This table showed the feature weights for PC1, PC2, PC3 and PC4.

Table 6. Component matrix for PC1, PC2, PC3 and PC4. Significant entries are colored red (positive values) and blue (negative values)

From this table, features providing the largest positive contribution to PC1 were CP1, CP2, CP4, CP8, SP5 and SP6. Students with a high projection on PC1 had a strong sense of cognitive and social presence. Students’ learning activities used to solve problems created high cognitive presence. They felt comfortable when they provided queries or discussed in online scenarios. These students can be classified as those participating in text VOH. They provided queries by handling in texts, got replies back also from texts. Students showed the highest satisfaction with simple VOH design among experimental groups [36]. For illustration in Fig. 6, consider students in this PC, they achieved higher average value in these positive features listed above.

Fig. 6. Students’ CoI value in CP1, CP2, CP4, CP8 and SP5, SP6

Similarly, the top largest positive entries correspond to PC2 were TP2, TP3 and TP6 and the top largest negative entries correspond to PC2 were SP5, SP6, SP8 and SP9. Since the factor loadings of SP8 and SP9 showed higher significance in PC3 and it would be discussed in the following section of PC3. Students with PC2 projection had high teaching presence but low social presence. They confirmed the helpful effect of instructions. Yet limited contribution was found in online discussion. These students could be classified in video mode VOH as teacher communicated with them face to face. Thus, teacher was capable to clarify the teaching goal and instructions, indicating a high teaching presence in Fig. 7. However, they listened carefully without discussing with others during the whole meeting, demonstrating low social presence.

Fig. 7. Students’ CoI value in TP2, TP3, TP6 and SP5, SP6

In Table 6, the features that have the greatest positive impact on PC3 were CP7, CP11, CP12. The features that have the greatest negative impact on PC3 were SP4, SP8 and SP9. We could obviously see that students with PC3 projection had high cognitive presence and low social presence. These students did not feel good when they conversed through Tencent Meeting thus had little communication with other course participants. Without communication, they could hardly develop a sense of collaboration or feel acknowledged. What is more, combing new information and developing solutions to course problems alone were common to them. For illustration, even plot of all significant contributions (CP7, CP11, CP12, SP4, SP8, SP9) were overlapping, the negative contribution questions (SP4, SP8, SP9) could show clear mode feature as well. In Fig. 8, students manifested low social presence. Students in high PC3 projection can be classified in voice mode VOH. When they were in the meeting, students muted themselves which mainly prohibit chance of immediate communication. Therefore, practice and activities were finished independently after class.

Fig. 8. Students’ CoI value in SP4, SP8, SP9

As is illustrated in Table 6, all factor loadings in PC4 were less than 0.35 in absolute value. With high projection on PC4, students were low in all three kinds of presence. This group of students did not show clear features compared to other groups. Some participants in this course could hardly leave a distinct impression on their classmates. Meanwhile, because they did not know each other, it is difficult to form a sense of course belonging. Since the rest three principal components of students were in obvious mode feature, this group was classified as control group. Control group did not take part in VOH, losing the chance to know more about their classmates.

6. Conclusion

This paper designs an IMP to improve the interactions and even learning outcomes in post- COVID-19 pandemic era. We also study the influence on students’ presence in the online course ‘Signals and Systems’ when VOH is added in IMP. What stands out in our present work is that VOH with teacher-student interaction would lead to higher online presence and grades in post-COVID-19 pandemic period. In addition, cognitive presence is significantly affected by VOH. Furthermore, students’ participating modes can be projected to different groups and can be effectively separated by PCA. When the unique students’ trajectories were found, early intervention or prior instructions may be utilized to those students in weak behaving groups. The findings have practical implications for educational practices to improve students’ learning outcomes. Because IMP is also not constrained by time and space, it encourages universities to launch platforms like IMP in post-COVID-19 pandemic era.

In our future work, the following issues will be studied. First, the performance of students involved in experiment will still be deserved to observe in other disciplines. Second, the study enrolled are Chinese sophomores. Future researches could explore whether the VOH implementation has the same effects on learners from different cultures considering presence in online teaching.