Analysis of 'Better Class' Characteristics and Patterns from College Lecture Evaluation by Longitudinal Big Data

Abstract

The purpose of this study was to analyze characteristics and patterns of 'better class' by using the longitudinal text mining big data analysis technique from subjective lecture evaluation comments. First, this study classified upper 30% classes to deduce certain characteristics and patterns from every five-year subjective text data for 10 years. A total of 47,177courses (100%) from spring semester 2005 to fall semester 2014 were analyzed from a university at a metropolitan city in the mid area of South Korea. This study extracted meaningful words such as good, course, professor, appreciation, lecture, interesting, useful, know, easy, improvement, progress, teaching material, passion, and concern from the order of frequency 2005-2009. The other set of words were class, appreciation, professor, good, course, interesting, understanding, useful, help, student, effort, thinking, not difficult, explanation, lecture, hard, pleasant, easy, study, examination, like, various, fun, and knowledge 2010-2014. This study suggests that the characteristics and patterns of 'better class' at college, should be analyzed according to different academic code such as liberal arts, fine arts, social science, engineering, math and science, and etc.

1. INTRODUCTION

College course evaluation by students has a useful meaning to the instructor who is in charge, and the college administration to maintain a course quality control. Usually the purpose of the evaluation is to find out whether course objectives are achieved, and course is delivered effectively according to the lesson plan [1]. 

The attention of college course evaluation has been increasing in terms of providing information for faculty appointment and promotion as well as the demand for the course improvement each year. Currently the results of students’ course evaluation are a decisive formal quality control system at college to improve teaching ability for course instructors [2], [3]. The results of course evaluation are usually reported to individual faculty in the form of quantitative score base, not including the form of qualitative information in terms of explanations on what should be revised and corrected for better class management. This means that the subjective expression of students about the course is not delivered to the faculty with exactly what students want, and needs for the class. Therefore, the evaluation results cannot be utilized as useful feedback data to the instructor to make them realize what is better class for getting students interests and providing practical support to students, and how the instructor should revise the lesson plan [4].

At most college, course evaluation results have been automatically accumulated by quantitative big data base because most of students should fill out the course evaluation in the form of questionnaire and essay type questions right after the semester ends. However, since the quantitative data is utilized as an important administrative purpose first, the essay type of students comments are usually used as not important reference data both to the instructor and to the administration. It is needed to provide proper guideline for ‘better class’ not only from the pattern analysis of quantitative big data source, but also from the interpretation of qualitative data source by textmining.

This study used text mining technique to analyze unstructured data of course evaluation response questions. Textmining extracts highly frequent words by statistical analysis from natural language type sentences. The technique is able to figure out overall framework of the task from big data abalysis of highly frequent words, and this study selected this data analysis technique considering of unstructured words to be used in this study.

Recently, educational research widely used text mining techniques both to reason meaningful concepts and characteristics of certain tasks, and to analyze final results and products of the tasks to match the purposes of the research [5], [6], [7]. This study analyzed many years of bag data on lecture response questions to find out the characteristics of ‘better class’ based on the students’ perception. 

The purpose of this study was to analyze text mining approach from course evaluation qualitative big data source to suggest what were the characteristics of ‘better class’ at college level. This study categorized upper 30% of course evaluation scores for 10 years at every semester, and analyzed students answer texts from qualitative evaluation items. This study used techniques such as frequency sampling, words ordering, and any pattern change for two sets of 5 year net data to identify the pattern and characteristic of ‘better class’ from the longitudinal text mining analyses.

 

2. RESEARCH METHODS

2.1 Sampling

The data in this study was collected from the results of course evaluation between the year of 2005 and 2014 at university located in the middle area of South Korea. The starting year 2005 means especially to provide the strong administrative demand utilizing instructional strategies such as application of ICT (Information Communication Technology), constructivism oriented lesson design, and SDL (Self-Directed Learning) [8], [9] to most of courses. Also. 10 years were needed to analyze the accumulated longitudinal data to standardize pattern and characteristics of text mining analysis. The reason of grouping and analyzing from 2010 to 2014 was to reflect from diffusing of government financial support to colleges and universities to reinforce teaching and learning competencies. Considering of 10 years accumulated big data, this study divided two 5 years cycle to apply innovative changes of college and university education. The total number of the courses were 47,177. 

The statistical data of this study extracted from course name, open semester & year, and evaluation score & comments by excel format. Fig. 1. shows this results.

 

Fig. 1. Course evaluation data sampling example

 

2.2 Methodology 

To analyze students’ comments on course evaluation, text mining techniques such as an Elastic Search system, EunJeon Korean language morpheme analysis technique, and Mysql, Apache-PHP for search engine, were used. These techniques were applied to extract evaluation comments words and to analyze frequencies for those extracted words. Fig. 2. explained the framework of text mining system developed in this study [5]. 

 

Fig. 2. Framework of course evaluation analysis system

 

Elastic Search is an open source search engine developed by Apache Lucene base with supporting functions of expansion, high fusibility, multi tenancy, text oriented, and strong professional search. Words extraction and frequency were analyzed by Korean language morpheme analyzer and category (or part of speech) tagging. Morpheme analysis means of figuring out language attributes such as the root of a word, prefix, suffix, and POS (part-of-speech). POS tagging is to make markup words considering of morpheme meanings and contexts of words. For example, Korean sentence

가방에 들어가신다 → 가방/NNG + 에/JKM + 들어가/VV + 시/EPH + ㄴ다/EFN [10].

The Korean language representative open source Elastic Search ‘EunJeon’ has features in proper speed and qualitative morpheme analysis, and free license. Fig. 3. shows the process of EunJeon Korean language analysis with the formation of comments DB → Eunjeaon Elastic Search → morpheme analysis DB [5].

From Fig. 3, the total number of comments data for 10 years was 265,939 cases, and 6.726,965 morphemes separated from ‘EunJeon’ Elastic Search. Fig. 4. also shows the accumulated output data from the process of searching condition input → data extraction → morphemic frequencies → output [5]. 

 

Fig. 3. Morpheme analyzing system for course evaluation comments analysis

 

Fig. 4. Output process of the data in course evaluation analysis system

 

Fig. 5. shows the screen of Elastic Search system and ‘Eunjeaon’ morpheme system to be used in this study to analyze course evaluation comments by text mining technique [10].

 

Fig. 5. The developed screen of course evaluation comments text analysis system

 

The screen from fig. 5. was designed for data analysis and extraction morphemes for the indication of searching conditions such as semester-year, lecture evaluation score levels (upper 30%, mid 40%, lower 30%), lecture size (small: 1-35 students, standard: 36-50 students, large: 51+ students), course requirements (general education courses, major education courses), student year (freshmen, sophomore, junior, senior), major areas (liberal arts, social science, teacher education, science and engineering, fine arts), and lecture type (lecture only, lecture+practical training, lecture+lab, practical training). 

Words and frequencies by morpheme analysis were generated from the unstructured students evaluation comments, for example, ‘submit’ button with the input of upper score 30% and a certain semester-year specification, provides the most referred words orderly in a certain specified lecture.

 

3. RESULT

3.1 Analysis results of ‘better class’ characteristics from year 2005 to year 2009 by lecture evaluations

This study categorized 10 years accumulated lectures by three level searching conditions such as upper 30%, mid 40%, and lover 30% from the evaluation scores by students. Then, students’ unstructured lecture evaluation comments for 10 years, were analyzed to extract the most referred words by students and their frequencies. The total number of analyzed morphemes used in this study was 6,726,965, and those words were reclassified by lecture traits and quality to be used as useful morphemes. Meaningless auxiliary words and some other unidentified words were excluded.

This study analyzed 5 year periods traits and change transition of ‘better class’ from 10 year accumulated longitudinal data to examine evaluation word pattern recognitions and sustainable characteristics of ‘better class’ reflected by social demands toward college education.

 

Table 1. Text mining results of Upper 30% lectures for first 5 years

 

Table 1 explains the most referred words with more than 1,000 frequencies out of upper 30% lectures by text mining results from year 2005 to year 2009. From table 1, the most referred words were good, class, professor, appreciation, lecture, fun, useful, know, easy, help, understand, student, work hard, not difficult, new, passion, study, need, interest, knowledge, presentation, improvement, progress, text, passion, and opportunity in order of frequency.

Except the neutral words such as good and professor, table 1 indicated that the proper words for a ‘better class’ were fun, useful, help, understand, not difficult, new, passion, need. Interest, knowledge, presentation, improvement, text, passion, concern, theory, and confidence. These words were underlined on the table 1, and extracted from the students’ essay type evaluation answers as followings.

 

‘The lecture was fun and very helpful because the profession explained very well. (student 1)’

‘The lecture was very useful and the professor explained the content very easily and it was helpful to understand the class. (student 2)’

‘Even though the project was very difficult, the professor explained the content very easily and it was helpful. (student 3)’

‘The professor explained easily with extra materials for students like me to make better understanding. (student 4)’

‘The professor used easy examples to make me understand. (student 5)’

‘The professor selected not difficult text. (student 6)’

‘The lecture was not difficult. (student 7)’

‘The lecture included new knowledge and very useful (student 8)’ \

‘The professor was very passionate, but it was a little hard. (student 9)’ \

‘The professor taught selected important materials, and it was very well understandable . (student 10)’

‘The professor provide real world new information and it was interesting. (student 11)’

‘I could improve my academic knowledge level. (student 12)’

‘The lecture was fun and memorable, and text was helpful. (student 13)’

 

3.2 Analysis results of ‘better class’ characteristics from year 2010 to year 2014 by lecture evaluations

Another set of analysis was conducted with the same method as on first 5 year analyses. Following table 2 explains another set of the most referred words from upper 30% courses between year 2010 and year 2014.

 

Table 2. Text mining results of Upper 30% lectures for second 5 years 

 

From table 2, except the neutral words such as good and professor, proper words for a ‘better class’ were fun, useful, help, thinking, not difficult, explanation, eagerly, pleasant, easy, examination, variety, passion, interest, and knowledge, These words were underlined on the table 2, and extracted from the students’ essay type evaluation answers as followings.

 

‘The lecture was fun and  interesting. (student 1)’

‘The lecture was very well understandable with many example cases and video materials. (student 2)’

‘It was very help and learned a lot of knowledge. (student 3)’

‘It was useful and helpful to me. (student 4)’

‘The lecture was helpful to pass a license test and to master the techniques. (student 5)’

‘The professor made students to think and share with other students’ various thought . (student 6)’

‘The professor explained very easily. (student 7)’

‘The lecture was very useful because the professor delivered with passion. (student 8)’

‘The examination discriminated students. (student 9)’

‘The professor taught us with various teaching skills and with many different materials. (student 10)’

‘The professor prepared the class very eagerly and passionately. (student 11)’

‘It was interesting comparing to other class. (student 12)’

 

3.3 Word cloud results of ‘better class’ characteristics from year 2005 to year 2014 by lecture evaluations 

Fig. 6 and 7 show the text mining results categorized by 5 year terms from 2005 to 2014 by Claude form. The bold words are the core words from word Claude visualization except vague neutral words. They are fun, understand, useful, easy, help, not difficult, passion, interest, and etc.

 

Fig. 6. Upper 30% courses text mining results categorized by 5 year terms from year 2005 to year 2009 by Word cloud form 

 

Fig. 7. Upper 30% courses text mining results categorized by 5 year terms from year 2010 to year 2014 by Word cloud form

 

4. CONCLUSION

This study analyzed the characteristics and patterns of ‘better class’ by text mining analysis to find what are the most referred evaluation comments words and frequencies of those words out of upper 30% evaluation scores by college students based on 10 year longitudinal data.

First, this study found the meaningful and significant key words such as fun, easy, understand, useful, passion, and etc. from the analyses. These words can be explained that students perceive a ‘better class’ means learning materials by delivering easily, fun, useful, and passionate. Therefore, students are able to inspire interests and passion from the class accordingly. These words were similar results from TV show about 100 famous college lectures core elements at 2012 [11]. It emphasized ‘point out the main concepts with easy, repeatedly, passionate, and with care’. These words are the decisive characteristics of ‘better class (lecture)’. What are the’better lecture’? This study finalized the ‘better lecture’ should be easily understandable. fun to learn, and delivered with professors’ passion . This result supports the fidings from theprevisous research such as [5], [11].

Secondly, students perceived ‘better class’ as easy, understandable, and useful class during first 5 years from year 2005 to year 2009, while for second 5 years from year 2010 to year 2014, there were other key words such as fun, pleasant, and interest added up to. The words, Knowledge and text from year 2005 to year 2009, were not mentioned with more than 1,000 frequencies during second 5 years. This result can be assumed that students activities have been changed from text oriented lecture type to newly recommended students’ active participation in the class during that time because college education emphasized more student oriented teaching skills and learning activities to college faculty. In other perspective, college evaluation system has set most of college lectures toward criterion reference test system. This means that students are more concerned on test difficulty, discrimination, and fairness. 

Thirdly, this study also indicated that a ‘better class’ at college is able to be controlled by faculty members in all times. At most of colleges, instructors are domain experts, not teaching expert. Although teaching and learning centers at college provides various teaching strategies and student oriented teaching skills, faculty members passively participate, and are not interested. Therefore, reflective feedback from students should be delivered to faculty members and provide proper class consulting to improve their classes.

Finally, this study suggests that the characteristics and patterns of ‘better class’ at college, should be analyzed according to different academic code such as liberal arts, finearts, social science, engineering, math and science, and etc. The ‘better class’ can be interpreted in different way based on the academic areas, and provide different delusion to the faculty when they have student evaluation comments on the specific lecture. However, this research can provide very important implications to college education what and how it moves to provide better qualified education to students. Also, when considering Korean college environment in terms of future population cliff, college education should focus on the fundamental teaching and learning basic strategies, and therefore increasing student satisfaction.