Veranstaltungsort: Frankfurt Institute for Advanced Studies, Ruth-Moufang-Str. 1, 60438 Frankfurt am Main, Lecture Hall

Am 19.05.2022 um 14 Uhr hält Prof. Dr. Gemma Roig, Professorin für Data Analytics and Modeling an der Goethe-Universität, einen Vortrag zu DNN Modellen.

Title:

Task-specific Vision DNN Models and Their Relation for Explaining Different Areas of the Visual Cortex    

Abstract:

Deep Neural Networks (DNNs) are state-of-the-art models for many vision tasks. We propose an approach to assess the relationship between visual tasks and their task-specific models. Our method uses Representation Similarity Analysis (RSA), which is commonly used to find a correlation between neuronal responses from brain data and models. With RSA we obtain a similarity score among tasks by computing correlations between models trained on different tasks. We demonstrate the effectiveness and efficiency of our method to generating task taxonomy on the Taskonomy dataset. Also, results on PASCAL VOC suggest that initializing the models trained on tasks with higher similarity score show higher transfer learning performance. Finally, we explore the power of DNNs trained on 2D, 3D, and semantic visual tasks as a tool to gain insights into functions of visual brain areas (early visual cortex (EVC), OPA and PPA). We find that EVC representation is more similar to early layers of all DNNs and deeper layers of 2D-task DNNs. OPA representation is more similar to deeper layers of 3D DNNs, whereas PPA representation to deeper layers of semantic DNNs. We extend our study to performing searchlight analysis using such task specific DNN representations to generate task-specificity maps of the visual cortex. Our findings suggest that DNNs trained on a diverse set of visual tasks can be used to gain insights into functions of the visual cortex. This method has the potential to be applied beyond visual brain areas.

Roig Group