We use several research platforms for our studies in computational and biological vision, neural computation, and robotics: an anthropomorphic robot head, a simulation environment for virtual human characters (the "Virtual Livingroom"), and a number of digital cameras, and two stereo rigs. For some of our psychophysics work we are fortunate to be able to use Virtual Reality equipment at the University of Rochester. (Click on any image below to enlarge it.)

Our most prominent piece of equipment is a human-size anthropomorphic robot head. It was built to mimic the major degrees of freedom of the human neck/eye system. For us, it is not only a tool for doing research into active computer vision and autonomous learning, but also a platform for testing computational models of the primate visual system. All-in-all the head has nine degrees of freedom. The neck can pan left and right and tilt up and down. Both eyes can do individual pan and tilt movements. The jaw can open and close. The mouth can express smiling and frowning and the eyebrows can move up and down. All motors are standard off-the-shelf servo motors controlled over a serial interface. The original design of the head was done in collaboration with YFX Studio, a special effects company from Sacramento, California. Since then, we have made a number of in-house improvements to the robot head. The robot's eyes are a pair of miniature CCD cameras with a digital IEEE 1394 interface giving 640 by 480 pixel image pairs at 30 Hz. A second robot head, which was designed by a group of mechanical engineering students at UCSD in the context of a senior design project, is currently awaiting its completion.

The "Virtual Livingroom" is a simulation environment for studying cognitive architectures, behavior organization, active vision, and autonomous learning. It brings together a number of software packages for computer graphics and computer vision. The main components are OpenGL, SGI Performer, BDI DI-Guy, and Open CV. The combination of these tools, allows us to control and animate human characters with a complex kinematic structure in real time, simulate aspects of their visual information processing, and use the result to generate behavior while the characters learn about objects and other social agents in their environment. In particular, we are using this platform for modeling the emergence of gaze following in late infancy as part of the MESA project. However, this platform has countless application domains for embodied cognitive modeling without incurring some of the drawbacks of real robotics research. We see this simulation approach as complementary to real robotics. There are pros and cons on both sides and the proper choice of modeling platform depends on the precise question being asked.

In our visual psychophysics work we mostly address topics in attention and visual working memory. Much of this work is done directly in our lab, where we have a number of graphics workstations to render visual stimuli and record subjects' responses. Eye tracking equipment is available in the department, too. Beyond that, we are fortunate to be able to use research resources in other labs. Most importantly, much of our research happens in collaboration with the Virtual Reality Lab at the University of Rochester, which is headed by Dana Ballard and Mary Hayhoe. They have state-of-the-art Virtual Reality (VR) equipment for simulating a range of environments with different interaction paradigms including haptic force feedback for virtual grasping of objects, a virtual driving simulator with 6DoF motion platform, and a virtual lab environment allowing free walking. Of particular interest is their ability to accurately measure eye movements of a subject wearing a VR helmet. Beyond all this, their lab offers all the support and software necessary to do exciting experiments.

Back to Top