UNIVERSITY OF HERTFORDSHIRE COMPUTER SCIENCE RESEARCH COLLOQUIUM presents "Multimodal Deep Learning for Cognitive Reflection Score Prediction: Fusing Audio and Text Representations" Dr. Mohammad Tayaraninajaran (University of Hertfordshire) 26 November 2025 13:00 -14:00 Room LB216 Everyone is Welcome to Attend Refreshments will be provided Abstract: Cognitive reflection, the ability to suppress an intuitive and spontaneous (“system 1”) wrong answer in favour of a reflective and deliberative (“system 2”) right answer, is a key concept in cognitive and behavioural research. This paper, for the first time, presents a machine learning algorithm to predict Cognitive Reflection Test (CRT) scores from audio and text signals. A dataset of 161 participants was collected, in which the participants were asked to recall and talk about a negative and a positive personal memory. The participants were then asked to complete a standard CRT questionnaire. To convert the problem into a binary classification, the participants were divided into two groups: above and below the median CRT scores. To extract the features, Google’s BERT (text feature extraction) and Facebook’s Wav2Vec 2.0 (audio feature extraction) are used in this paper. A deep learning model is then developed to combine the audio and text features and transform them into a new feature space in which the features are more discriminative and the data records belonging to the same class are clustered together. This makes the classification task easier for the machine learning algorithms. Several experiments are conducted in this paper, and the results indicate that the proposed transformation algorithm enhances the performance of the classification algorithms. --------------------------------------------------- Hertfordshire Computer Science Research Colloquium http://cs-colloq.cs.herts.ac.uk