Electronic Theses and Dissertations

Date

2024

Document Type

Dissertation

Degree Name

Doctor of Philosophy

Department

Engineering

Committee Chair

Mohammed Yeasin

Committee Member

Ebenezer Olusegun George

Committee Member

Mohammed Yeasin

Committee Member

Mohmmadreza Davoodi

Committee Member

Vida Abedi

Abstract

While Intelligent systems such as Intelligent tutoring system (ITS), assistive technology solutions, and video gaming has the potential to significantly reduce cognitive load by automating tasks and providing timely information, it can also increase demand on a user's working memory (WM) capacity, if not designed and implemented thoughtfully. WM is responsible for temporarily storing and processing information to execute cognitive tasks. However, due to its limited capacity and duration, excessive use of WM can lead to high cognitive load (CL), adversely affecting performance on various mental tasks (e.g., learning, comprehension, driving, and problem-solving). Cognitive load refers to the amount of WM resources used for encoding, maintaining, or recalling information. A reliable CL prediction is essential for designing real-world systems such as education and instructional design, brain-computer interface (BCI), and human-computer interaction (HCI). Electroencephalography (EEG) is one of the most widely utilized methods for brain signal recording due to its excellent temporal resolution, non-invasive and non-intrusive nature, and cost- effectiveness. Nevertheless, robust and reliable CL prediction using machine learning from EEG data remains challenging due to various factors: limited sample sizes, noisy recordings, ineffective data representations, and the absence of robust predictive models. This dissertation introduces various signal processing and deep learning techniques to address these challenges. First, to mitigate the high noise levels inherent in EEG recordings, we developed a novel eigenspace-based bootstrap sampling method capable of generating low-noise Event-Related Potential (ERP) from single-trial EEG signals. Second, to overcome the "small sample" issue for training deep models, we propose two variants of generative adversarial networks (GANs), DCGAN and EGAN, to generate synthetic spatial-spectral representations of EEG and multichannel EEG data. Third, we develop two different representations of EEG recordings: (1) spatial-spectral representation (i.e., topomap) to maintain both spatial (electrodes location) and spectral (frequency) information and (2) spatial-spectral-temporal representation ("EEG movie") consisting of sequences of topomap images capturing dynamic spatial-spectral features over time. Lastly, we developed optimized and interpretable convolutional neural network (CNN) and transformer-based models for CL prediction and analysis using topomap images and EEG movies, respectively. The analysis in this dissertation is based on EEG data collected during a modified Sternberg visual and auditory WM task, in which participants were presented with four levels of CL, defined by encoding 2, 4, 6, or 8 English characters in the memory. Empirical evaluations show that our CNN models achieve high prediction accuracy (over 94%) when trained on topomap images. Additionally, experimental results indicate that our noise reduction approach and data augmentation with generative models (DCGAN and EGAN) significantly improve CL prediction performance. Moreover, by applying optimized spatial-temporal models (3D CNN, EEG-ViT) to EEG movies, we achieved the highest CL prediction performance (over 98 %) by leveraging spatial-spectral and temporal patterns in the data. Furthermore, we applied Gradient-weighted Class Activation Mapping (Grad-CAM) to the CNN models to identify the contribution of distinct brain regions (e.g., the prefrontal cortex) to predicting CL levels at different frequency bands. The proposed methodologies offer novel, data-driven solutions to the primary challenges in applying deep learning-based models for predicting cognitive events in developing practical applications (e.g., WM assessment, BCI design, and neural disorder diagnosis).

Comments

Data is provided by the student.”

Library Comment

Dissertation or thesis originally submitted to ProQuest.

Notes

Open Access

Download

Share

COinS