UNIVERSITY OF HERTFORDSHIRE
             COMPUTER SCIENCE RESEARCH COLLOQUIUM

                            presents

     
	"Context-Aware Language Understanding in Human-Robot Dialogue with LLMs"
	

                   Dr. Svetlana Stoyanchev,
              (Cambridge Research Laboratory, Toshiba Europe)

                        18 February 2025
                         13:00 -14:00 
                          Room C408

                  Everyone is Welcome to Attend
		  Refreshments will be provided
				  
				                               

Abstract: 

In this work, we explore the use of large language models (LLMs) as interpreters of user utterances within a human-robot language interface. A user interacting with a robot that operates in a physical environment should be able to issue commands that interrupt the robot’s actions, for example, corrections or refinements of the task. This study addresses the context-aware interpretation of user utterances, including those issued while the robot is actively engaged in task execution, exploring whether LLMs, without fine-tuning, can translate a
user’s directive utterances into corresponding sequences of robot actions.  Using an interactive multimodal interface—combining text and video—for a virtual robot operating in a simulated home environment, we collect a dataset of user utterances that guide the robot through various household tasks, simultaneously capturing manual interpretation when the automatic one fails. The collected dataset is used to compare the interpretive performance of the proprietary gpt-5-mini model with an open-source Qwen3 model. 

Biography:

Dr. Svetlana Stoyanchev is a Senior Research Engineer and Embodied AI Team Lead at the Cambridge Research Laboratory, Toshiba Europe. Her research interests include multimodal communication, human-robot interaction, task-oriented and argumentation dialogue. She has held research positions at The Open University, Columbia University, AT&T Labs Research, and Interactions Corporation.  Dr. Stoyanchev serves on the advisory  board of the Special Interest Group on Discourse and Dialogue (SIGDIAL) and was Program Co-Chair of SIGDIAL 2023 conference. She has authored more than 50 peer-reviewed publications and has supervised PhD and MPhil students in collaboration with the academic partners. 

---------------------------------------------------
Hertfordshire Computer Science Research Colloquium
http://cs-colloq.cs.herts.ac.uk