Tutorials

(1) an interdisciplinary perspective on conducting evidence-based human behaviour research from the viewpoints of visual perception, environmental psychology, and spatial cognition.

(2) AI methods for the semantic interpretation of embodied multimodal interactions (e.g., rooted in behavioural data), and the (empirically-driven) synthesis of interactive embodied cognitive experiences in real-world settings relevant to both everyday life as well to professional creative-technical spatial thinking

(3) the relevance and impact of research in cognitive human-factors in spatial cognition for the design and implementation of human-centred AI technologies

The main technical focus of the tutorial is to provide a high-level demonstration of general AI-based computational methods and tools that can be used for multimodal human behavioural studies. Of special focus are visuospatial, visuo-locomotive, and visuo-auditory cognitive experiences in the context of application areas such as architecture and built environment design, narrative media design, product design, cognitive media studies, and autonomous cognitive systems (e.g., robotics, autonomous vehicles). Presented methods are rooted in foundational research in artificial intelligence, spatial cognition and computation, spatial informatics, human-computer interaction, and design science.

The tutorial utilises case-studies to demonstrate the application of the foundational practical methods and tools. This will also involve practical examples from large-scale experiments in domains such as evidence-based architecture design, communication and media studies, and cognitive film studies.

SCOPE AND AUDIENCE

SCOPE. Interdisciplinary scientific agenda targeting an audience with an interest or curiosity in visual and spatial cognition, visual perception, and artificial intelligence (emphasis on knowledge representation and reasoning, and high-level event perception). Particular focus will be utilising case-studies to demonstrate the state of the art in artificial intelligence, cognitive vision, and applied perception with respect to their impact on eye-tracking in particular, and multi-modal human behavioural research in general.

AUDIENCE. (1) Interdisciplinary audience interested to learn about how research in Cognition, AI, Interaction, and Design comes together; (2) Young researchers (e.g., masters and early stage doctoral candidates) desirous of exploring open research questions and avenues for visual perception research, and how it could influence the design of AI technologies; (3) Learn about applications of spatial cognition research in application domains such as (building) architecture design, (visuo-auditory) narrative media design, human-robot interaction, autonomous vehicles; (4) Design practitioners from areas such as architecture, animation, visual art, digital media, interaction design seeking to get insights from existing case-studies involving eye-tracking based visual perception in their respective domains of application; (5) Research generally curious to lean about what AI methods have to offer to behavioural research in cognitive science and psychology.

Further details about the tutorial may be consulted at: https://codesign-lab.org/space-cognition-ai/

The expected learning outcomes of this tutorial include:
Understanding the basic principles of the SONDA method: continuous psychophysics, eye-tracking signal processing, fundamentals of neuro-ophthalmology.
Familiarization with the different components of the SONDA method, including stimuli design, spatio-temporal analysis based on cross-correlograms, saccadic perimetry, and saccadic dynamic properties.
Gaining insights into the strengths and limitations of the SONDA method and how to choose the appropriate analysis techniques for a given research question.
Understanding how to interpret and communicate the results of SONDA analyses to other researchers, clinicians, and the broader scientific community.
This tutorial is targeted towards researchers and students with a background in visual neuroscience, psychophysics, neurology, optometry, or ophthalmology who are interested in learning more functional vision assessment through eye movement data. Participants should have a basic understanding of statistical methods and programming skills.

The importance of this tutorial lies in the increasing relevance and availability of eye-tracking tools. Eye movements provide a valuable source of information about how the visual system processes information and generates behavior that can be used to investigate a wide range of research questions, from basic visual neuroscience to clinical applications. The SONDA method offers a powerful framework for analyzing eye movement data by integrating multiple types of analyses into a single experimental pipeline and providing a unified approach for investigating different aspects of visual perception in a simple and fast manner.

By providing a comprehensive overview of the SONDA method, compatible experimental setups and applications, this tutorial will enable participants to stay at the forefront of eye movement research in visual perception and contribute to developing new knowledge and applications in this area. Participants will be able to work with real eye movement data from different clinical populations and develop practical skills in applying the SONDA method to their research questions.

High-Dynamic-Range(HDR) Imaging is the most dramatic example. Human optics cannot transmit the full range of a million:1 transparency test targets to their retinal images. One such HDR target contained 40 Gray squares segments placed on a max-luminance surround subtending 16°x19° The squares’ luminance range was [250,000:1]; retinal luminance range[33;1]. A second HDR target with these 40 Gray squares on a min-luminance surround had luminance range [250,000:1]; retinal luminance range [5000:1]. Observers reported that whites (with equal luminances) appeared the same white in both experiments. Remarkably, blacks appeared the same black in both experiments despite the change in retinal luminances range from 33:1 to 5000:1. There is no single Response Function (Luminance-in vs. Appearance-out) that fits all complex HDR scenes.

Our new Python (open-platform) program calculates glare on each pixel (representing a receptor) as the sum of the individual contributions from every other scene segment. Glare responds to the content of the entire scene. Glare is a scene-dependent optical transformation.
Quantitative measurements, and pseudocolor renderings are needed to appreciate the magnitude, and spatial patterns of glare. Glare’s gradients are nearly invisible, or invisible when you inspect them.

Neural processing performs vision’s second scene-dependent spatial transformation. Neural processing generates appearances that do not correlate with the quanta catch of receptors, such as Simultaneous Contrast, and Assimilation Illusions. As well, Edwin Land’s Black and White Mondrian, and Ted Adelson’s Checkershadow make the same argument for visual appearance’s dependence on the spatial content of complex scenes. Illusions are generated by neural processing of the “rest-of-the-scene”. The neural network input is the simultaneous array of all receptors’ responses after glare.

Recent glare studies of Lightness illusions used scene luminances of only 200:1 The absolute amounts of glare are much smaller. However, the principles are the same. White scene segments have only very subtle modifications of uniformity near boundaries; Gray segments have larger changes in uniformity and average value; Black segments have considerable disruptions of uniformities and substantial variability of average values. Both Contrast and Assimilation have two unusual properties. First, they both restrict the “rest-of-the-scene” to only Whites and Blacks segments. Whites are glare’s maximum donor; Blacks are most affected by glare. The second property is relative angular size, angular separation of donor and receiving pixel, and enclosure. Contrast has larger Grays and larger surrounds. That separates Contrast’s donor and most-affected receiving pixels, and places individual pixels at lower intensities on the Glare Spread Function. Michael White’s Assimilation has long-narrow Grays with long-narrow surrounds. That brings Assimilation donor and most-affected receiving pixels closer together , and places individual pixels at higher intensities on the Glare Spread Function. Both of these properties accentuate the effects of glare.

In summary, the tutorial describes how optical glare affects imaging in human vision, cameras and displays. As vision’s first spatial imaging transformation it modifies the pattern of scene luminances, and makes a different pattern of light on retinal receptors. The complete array of all receptors is the simultaneous input to the second spatial transformation, neural image processing. The tutorial does not discuss theoretical mechanisms and neural models, it pays attention to observations of appearances of scene segment. Vos and van den Berg’s 1999 CIE Glare Spread function is the basis of understanding Glare. The tutorial describes our new Python code that calculate the pattern of light on receptors. These scene transformation are a morass of gradients that are hard to see. The tutorial explains the need for numerical analysis (histograms and plots of calculated retinal luminance) as well as Pseudocolor renditions to visualize retinal patterns.
The tutorial explains glare’s role in vision, and in technology covering many topics including HDR, LDR illusions, visibility of gradients, Pseudocolor pattern visualizations, and other topics related to glare’s spatial transformations of scene luminances, and their appearances.