Individual and Social Processes in Image Description Generation (via Zoom)

Abstract: Most language use is driven by specific communicative goals in interactive setups, where often visual perception goes hand in hand with language processing. I will discuss some recent projects by my research group related to modelling language generation in socially and visually grounded contexts, arguing that such models can help us to better understand the cognitive processes underpinning these abilities in humans. I will first focus on individual cognitive processes and present a model of image description generation that exploits information from human gaze patterns recorded during language production. I will then move on to two-person dialogue setups and describe our recent work on generating referring descriptions that are grounded in the conversational and visual context. I’ll end the talk by discussing key open questions and challenges in relations to these topics.  

Bio: Raquel Fernández is an Associate Professor at the Institute for Logic, Language & Computation (ILLC), University of Amsterdam, where she leads the Dialogue Modelling Group. Her work and interests revolve around language use, encompassing topics that range from computational semantics and pragmatics to the dynamics of dialogue interaction, visually-grounded language processing, and child language acquisition, among others.

After studying Cognitive Science and Language in Barcelona, she received her PhD in Computational Linguistics from King's College London. She has held research positions at the Linguistics Department of the University of Potsdam and at the Center for the Study of Language and Information (CSLI), Stanford University. Over her career, she has been awarded several personal fellowships by the Dutch Research Council (NWO) and she is a recent [2019] recipient of an ERC Consolidator Grant. Other distinctions include having been a member of the Editorial Board of the Computational Linguistics journal and the Dialogue & Discourse journal, co-president of the SemDial Workshhop Series for ten years, and member of the scientific advisory board of SIGdial in several occasions.