This project investigates human perception, combining psychophysical experiments and computational modeling. Currently, we have five research foci:
First, we develop methods to gain more information from psychophysical data, linking traditional methods with machine learning approaches [ ]. We characterized serial dependencies in behavioral responses and introduced methods to correct for them. Our software package to perform Bayesian inference for the psychometric function for non-stationary data is freely available and widely used [ ]. In addition, we investigate reliable supra-threshold psychophysical paradigms which are more intuitive and require less training and are thus more likely to yield reliable crowd-sourcing data [ ].
Our second focus is the development of an image-based model of spatial vision. We integrated the large psychophysical literature on detection and discrimination experiments and proposed a model based on maximum-likelihood decoding of a population of model neurons predicting the most important spatial vision data sets simultaneously, using a single set of parameters [ ]. Remarkably, the model generalises well to data it was not trained upon (natural images), produces a highly sparse code, and is useful in applications (in perceptual visual quality metrics). In the future, we plan to assay whether the model could be useful as a preprocessing step for deep neural networks (DNNs).
Third, we are exploring similarities and differences of DNNs and the human visual system. One line of work uses DNNs as generative models for texture synthesis, casting doubt on the popular notion that the peripheral visual system's internal representations are texture-like [ ]. In a second line of work, we compare the robustness of humans and current convolutional DNNs on object recognition under various types of image degradation, finding the human visual system (still) to be much more robust [ ]. These differences cannot be overcome by training on distorted images (i.e., data augmentation): While DNNs cope well with the exact distortion they were trained on, they still show a strong generalisation failure towards unseen distortions.
Fourth, we investigate perception of causality, focusing on the arrow-of-time that we previously studied theoretically [ ]. Preliminary data suggest that human observers can discriminite forward and backward played movies of autoregressive (AR) motion with non-Gaussian additive independent noise, i.e., they appear sensitive to the subtle temporal dependencies of the residuals of the AR-motion. Currently we are testing how long the motion sequences have to be for successful arrow-of-time discrimination and how well observers can generalise from one type of non-Gaussian noise to another.
Another project pertaining to the wider field of psychology is described in the project report on social networks, dealing with the optimization of spaced repetition algorithms for learning [ ].