SAVE THE DATE Tapia 2018 Orlando, FL September 19-22, 2018

2017 Tapia Conference

Towards Personalized Performance Feedback by Mining the Dynamics of Facial Keypoint Data


Presenter: Christian E. Lopez (The Pennsylvania State University)


Proper feedback has the potential improve students’ performance in a wide variety of tasks. Studies suggest that with systematic feedback individuals’ can enhance their problem-solving skills. Moreover, studies have shown a strong correlation between students’ affective state and their learning performance. Understanding students’ affective state allows instructors to provide personalized assistance that can enhance students’ learning experience. Unfortunately, this personalized assistance might be difficult to achieve in online learning environments, where in-person interactions are challenging. This has sparked an increasing interest in the development of automated systems capable of providing feedback based on students’ affective state. These systems can infer a student’s affective state, and use this information to provide proper feedback during the learning process, with the goal of enhancing their performance. However, current methods do not consider the student’s unique characteristics. Hence, individual differences in facial expression can deteriorate the effectiveness of these systems in providing appropriate feedback to all the students. In light of the limitations of current methods, this work presents a machine learning method for predicting a student’s performance by using his/her unique facial keypoint data, thereby bypassing the need to infer their affective states. A case study involving 31 students is used to validate the proposed method. The support vector machine model employed in this work yielded an accuracy of 91.3% for an individual-task specific model. In contrast, a general model yielded an accuracy of only 86.9%, thereby supporting the authors’ argument that individual-task specific models are more suited for advancing personalized feedback.