Preliminary Experiments with Transformer based Approaches To Automatically Inferring Domain Models from Textbooks
Abstract
Domain modeling is a central component in education technologies as it represents the target domain students are supposed to train on and eventually master. Automatically generating domain models can lead to substantial cost and scalability benefits. Automatically extracting key concepts or knowledge components from, for instance, textbooks can enable the development of automatic or semi-automatic processes for creating domain models. We explore in this work the use of transformer based pre-trained models for the task of keyphrase extraction. Specifically, we investigate and evaluate four different variants of BERT, a pre-trained transformer based architecture, that vary in terms of training data, training objective, or training strategy to extract knowledge components from textbooks for the domain of intro-to-programming. We report results obtained using the following BERT-based models: BERT, CodeBERT, SciBERT and RoBERTa.
Publication Title
Proceedings of the 15th International Conference on Educational Data Mining, EDM 2022
Recommended Citation
Banjade, R., Oli, P., Tamang, L., & Rus, V. (2022). Preliminary Experiments with Transformer based Approaches To Automatically Inferring Domain Models from Textbooks. Proceedings of the 15th International Conference on Educational Data Mining, EDM 2022 https://doi.org/10.5281/zenodo.6853051
