University of Hertfordshire
School of Computer Science

Homepage Daniel Polani


NIPS 2008

Mini-Symposium (Vancouver, 11. Dec. 2008)
Workshop(Whistler, 12. Dec. 2008)

Principled Theoretical Frameworks for the Perception-Action Cycle

Daniel Polani, University of Hertfordshire, UK Naftali Tishby, The Hebrew University, Israel

Description

A significant emphasis in trying to achieve adaptation and learning in the perception-action cycle of agents lies in the development of suitable algorithms. While partly these algorithms result from mathematical constructions, in modern research much attention is given to methods that mimic biological processes.

However, mimicking the apparent features of what appears to be a biologically relevant mechanism makes it difficult to separate the essentials of adaptation and learning from accidents of evolution. This is a challenge both for the understanding of biological systems as well as for the design of artificial ones.

Therefore, recent work is increasingly concentrating on identifying general principles rather than individual mechanisms for biologically relevant information processing. One advantage is that a small selection of principles can give rise to a variety of - effectively equivalent - mechanisms. The ultimate goal is to attain a more transparent and unified view on the phenomena in question.

Possible candidates for such principles governing the dynamics of the perception-action cycle include but are not limited to information theory, Bayesian models, energy-based concepts or group-theoretical principles. The workshop aims at bringing together various principle-based directions for the investigation of various aspects of the perception-action cycle and at identifying promising directions of work.

Schedule

Mini-SymposiumWorkshop (Room Change: Glacier Room/Westin)
11. Dec. 200812. Dec. 2008
07:30 Information Theory and the Perception-Action Loop
Naftali Tishby, Hebrew University
08:45 Discussion/Coffee
09:15 Information Bottleneck Optimization with Spiking Neurons with Application to Predictive Coding
Lars Büsing, University of Graz
10:00 Bayesian Modelling of a Sensorimotor Loop: Application to Handwriting
Estelle Gilet, CNRS - INRIA Rhône Alpes
10:30 Noon Break
13:30 Introduction
Naftali Tishby, Hebrew University and Daniel Polani, University of Hertfordshire
13:50 Encoding the Location of Objects with a Scanning Sensorimotor System
David Kleinfeld, UC San Diego
14:40 Break
14:50 Perception and Action in Robotics15:30 Information Theory and the Perception-Action Loop
Sebastian Thrun, Stanford University Daniel Polani, University of Hertfordshire
15:40 Dealing with risk in the perception-action cycle16:30 Coffee
Yael Niv, Princeton University
16:30 End17:00 Are power-laws in human behaviour caused by critical adaptive control?
Felix Patzelt, University of Bremen
17:30 Fundamental Dynamic Properties of Coupled Systems
Stefan Winter, Oliver Wyman Consulting
18:00 Empowerment: The External Channel Capacity of a Sensorimotor Loop and a Method for its Estimation
Tobias Jung, University of Texas at Austin
18:30 End of Workshop

Minisymposium

Introduction

Naftali Tishby, Hebrew University and Daniel Polani, University of Hertfordshire


Encoding the Location of Objects with a Scanning Sensorimotor System

David Kleinfeld, UC San Diego

Sensory perception in natural environments involves the dual challenge to encode external stimuli and manage the influence of changes in body position that alter the sensory field. We examined the mechanisms used to integrate sensory signals elicited by both external stimuli and motor activity through the use of behavioral, electrophysiological, and computation tools in conjunction with the vibrissa system of rat. We show that the location of objects is encoded in an ""region-of- interest centered"", as opposed to ""body-centered"", coordinate system. The underlying circuit for this computation is consistent with gating by a shunt, a common motif in cortical circuitry.

Perception and Action in Robotics

Sebastian Thrun, Stanford University

This overview talk investigates the perception action cycle from the robotics perspective. The speaker will discuss existing robotic implementation, discuss alternative architectures, and provide insights from the field of robotics. The speaker led the winning DARPA Grand Challenge team and heads Stanford's autonomous car research group. He will provide ample examples from real-time control of self-driving cars in complex traffic situations.

Dealing with risk in the perception-action cycle

Yael Niv, Princeton University

Risk (or outcome variance) is omnipresent in the natural environment, posing a challenge to animal decision-making as well as to artificial agents. Indeed, much empirical research in psychology, economics and ethology has shown that humans and animals are sensitive to risk in their decision-making, often preferring a small but certain outcome to a probabilistic outcome with a higher expected payoff. In light of this it may be surprising that optimal control methods such as reinforcement learning do not explicitly take risk into account. In this talk, I will review some of the literature on behavioral risk sensitivity in humans and in animals. I will then discuss a number of recent studies in which the neural basis of risk sensitivity has been investigated. While it is not surprising that the brain represents risk, the role of risk in decision making is still far from clear. I will argue that risk might play a more integrated role in learning and action selection than was previously postulated, specifically through the mechanism of reinforcement learning. However, this still leaves open the question of what we should learn from this neural solution: are there principled (normative) reasons for the prevalence of risk sensitivity? Does a general solution to optimal action selection have to take risk into account?


Workshop

Information Theory and the Perception-Action Loop

Naftali Tishby, Hebrew University


Information Bottleneck Optimization with Spiking Neurons with Application to Predictive Coding

Lars Büsing, University of Graz

We apply the online learning algorithm for IB optimization to the predictive coding task outlined in (Bialek et al., 2007) on the closed loop.

Bayesian Modelling of a Sensorimotor Loop: Application to Handwriting

Estelle Gilet, CNRS - INRIA Rhône Alpes

This paper concerns the Bayesian modelling of a sensorimotor loop. We present a preliminary model of handwriting, that provides both production of letters and their recognition. It is structured around an abstract internal representation of letters, which acts as a pivot between motor and sensors models. The representation of letters is independent of the effector usually used to perform the movement. We show how our model allows to solve a variety of tasks, like letter reading, recognizing the writer, and letter writing (with different effectors). We show how the joint modelling of the sensory and motor systems allows to solve reading tasks in the case of noisy inputs by internal simulation of movements.

Information Theory and the Perception-Action Loop

Daniel Polani, University of Hertfordshire

We outline recent approaches to characterize the sensorimotor loop of agents from an informational perspective. This view allows the principled characterization of agent behaviours in an environment and of possible paths towards minimalistic AI.

Are power-laws in human behaviour caused by critical adaptive control?

Felix Patzelt, University of Bremen

The perception-action cycle can be seen as closed control loop. A classical control paradigm is to stabilize an inverse pendulum. When humans solve this task while balancing a stick or during upright standing, their behaviour exhibits non-Gaussian fluctuations with long-tailed distributions contrasting technical controllers. The origin of these fluctuations is not known, but their statistics suggests a fine tuning of the underlying system to a critical point. We investigated whether this self-tuning may be caused by the annihilation of local information due to success of control. We found that it can lead to critical noise amplification, a fundamental principle, which produces complex dynamics even in very low-dimensional state estimation tasks. It generally emerges when an unstable dynamics becomes stabilized by an adaptive controller that has a finite memory. Starting from this theory, we developed a realistic model of adaptive closed loop control by including constraints on memory and delays. To test this model, we performed psychophysical experiments where humans balanced an unstable target on a screen. It turned out, that the model reproduces the long tails of the distributions together with other characteristics of the human control dynamics. Fine-tuning the model to match the experimental dynamics identifies parameters characterizing a subject's control system which can be independently tested. Our results suggest, that the nervous system involved in closed loop motor control nearly optimally estimates system parameters on-line from very short epochs of past observations.

Fundamental Dynamic Properties of Coupled Systems

Stefan Winter, Oliver Wyman Consulting


Empowerment: The External Channel Capacity of a Sensorimotor Loop and a Method for its Estimation

Tobias Jung, University of Texas at Austin

Empowerment, the external channel capacity of a sensorimotor loop has been introduced recently as a quantity, similar to predictive information to characterize the sensorimotor efficiency and evaluate the compatibility of the niche of an with its sensorimotor loop, and its quality. While computable in simple scenarios, its evaluation in continuous or higher-dimensional situations is still difficult. Here, we present a computational approach to that purpose and demonstrate an instructive application.


Last changed at Sun Dec 7 00:23:42 2008 by D. Polani