Understanding when students are active-in-thinking through modeling-in-context
Abstract
Learning-in-action depends on interactions with learning content, peers and real world problems. However, effective learning-in-action also depends on the extent to which students are active-in-thinking, making meaning of their learning experience. A critical component of any technology to support active thinking is the ability to ascertain whether (or to what extent) students have succeeded in internalizing the disciplinary strategies, norms of thinking, discourse practices and habits of mind that characterize deep understanding in a domain. This presents what we call a dilemma of modeling-in-context: teachers routinely analyze this kind of thinking for small numbers of students in activities they create or customize for the needs of their students; however, doing so at scale and in real-time requires some automated processes for modeling student work. Current techniques for developing models that reflect specific pedagogical activities and learning objectives that a teacher might create require either more expertise or more time than teachers have. In this paper, we examine a theoretical approach to addressing the problem of modeling active thinking in its pedagogical context that uses teacher-created rubrics to generate models of student work. The results of this examination show how appropriately constructed learning technologies can enable teachers to develop custom automated rubrics for modeling active thinking and meaning-making from the records of students' dialogic work. Practitioner Notes What is already known about this topic Many immersive educational technologies, such as digital games and simulations, enable students to take consequential action in a realistic context and to interact with peers, mentors and pedagogical agents. Such technologies help students to be active-in-thinking: engaging deeply with, reflecting on and otherwise making meaning of their learning experience. There are now many immersive educational technologies with integrated authoring tools that enable teachers to customize the learning experience with relative ease, reducing barriers to adoption and improving student learning. Educational technologies that support learning-in-action typically contain student models that operate in real-time to control the behavior of pedagogical agents, deliver just-in-time interventions, select an appropriate content or otherwise measure and promote active thinking, but these student models may not work appropriately if teachers customize the learning experience. Much as there are authoring tools that allow teachers to customize the curriculum of a given learning technology, there is a need for authoring tools that allow teachers to customize the associated student models as well. What this paper adds This paper presents a novel, rubric-based approach to develop automated student models for new activities that teachers develop in digital learning environments that promote active thinking. Our approach combines machine learning techniques with teacher expertise, allowing teachers to participate in the design of automated student models of active thinking that with further development could be scaled by leveraging their skills in rubric development. Our results show that a rubric-based approach can outperform a machine learning approach in this context. More importantly, in some cases, the rubric-based approach can produce reliable automated models based on the information that a teacher can easily provide. Implications for practice and/or policy If integrated into authoring tools, the rubric-based approach could allow teachers to participate in the design of automated models for educational technologies customized to their instructional needs. Through this design process, teachers could develop a better understanding of how the automated modeling system works, which in turn could increase the adoption of educational technologies that promote active thinking. Because the rubric-based approach enables teachers to identify key connections among concepts relevant to the pedagogical context, rather than general concepts or linguistic features, it is more likely to facilitate targeted feedback to help promote the development of active thinking.
Publication Title
British Journal of Educational Technology
Recommended Citation
Swiecki, Z., Ruis, A., Gautam, D., Rus, V., & Williamson Shaffer, D. (2019). Understanding when students are active-in-thinking through modeling-in-context. British Journal of Educational Technology, 50 (5), 2346-2364. https://doi.org/10.1111/bjet.12869
