Multi-Sensor modeling of teacher instructional segments in live classrooms
Abstract
We investigate multi-sensor modeling of teachers' instructional segments (e.g., lecture, group work) from audio recordings collected in 56 classes from eight teachers across five middle schools. Our approach fuses two sensors: A unidirectional microphone for teacher audio and a pressure zone microphone for general classroom audio. We segment and analyze the audio streams with respect to discourse timing, linguistic, and paralinguistic features. We train supervised classifiers to identify the five instructional segments that collectively comprised a majority of the data, achieving teacher-independent F1 scores ranging from 0.49 to 0.60. With respect to individual segments, the individual sensor models and the fused model were on par for Question & Answer and Procedures & Directions segments. For Supervised Seatwork, Small Group Work, and Lecture segments, the classroom model outperformed both the teacher and fusion models. Across all segments, a multi-sensor approach led to an average 8% improvement over the state of the art approach that only analyzed teacher audio. We discuss implications of our findings for the emerging field of multimodal learning analytics.
Publication Title
ICMI 2016 - Proceedings of the 18th ACM International Conference on Multimodal Interaction
Recommended Citation
Donnelly, P., Blanchard, N., Samei, B., Olney, A., Sun, X., Ward, B., Kelly, S., & Nystrand, M. (2016). Multi-Sensor modeling of teacher instructional segments in live classrooms. ICMI 2016 - Proceedings of the 18th ACM International Conference on Multimodal Interaction, 177-184. https://doi.org/10.1145/2993148.2993158
