Modeling Individual Differences in High-Level Visual Cognition Using DNNs

Abstract

Deep neural networks (DNN) can be a useful tool in the wider computational modeling toolkit to provide mechanistic explanations of individual differences in high-level visual cognition. DNNs are well-suited to model sources of individual differences caused by representational variability. DNNs can generate representations from images that are informed by their experience and neural architecture, which can be used with cognitive models to simulate responses and response times given the images. This allows us to create end-to-end image-computable models of high-level visual cognition tasks that can directly act on the images that human participants would see. However, there remain foundational questions to be answered. In the first project, we asked which measure of representational variability was the best and leveraged the best measure to quantify the variability caused by model differences in model randomization, dataset distribution, and architecture. In the second project, we asked what factors should be manipulated to generate reliable representational variability across models by analyzing a large collection of pretrained DNNs and tested our findings by training our own set of models. In the final project, using DNNs in conjunction with a cognitive model, we tested whether we could replicate a classic pattern of results in categorization and then cause individual differences in performance based on manipulating DNN training. We have evidence towards how representational variability should be measured, what factors should be manipulated in DNNs to cause representational variability, and how to leverage representational variability to cause individual differences in cognitive models of high-level visual cognition tasks. Taken together, we completed the groundwork towards leveraging differences in DNNs to model individual differences in high-level visual cognition, opening the way for new model pursuits to understand the mechanisms underlying individual differences.