Zhang wins NSF CAREER Award for robot reflection, imagination research

Hao Zhang, assistant professor of computer science at Colorado School of Mines, has won a National Science Foundation CAREER Award to develop methods for robot reflection and imagination.

Why do robots need to reflect and imagine?

Ask Zhang and he’ll tell you that it’s not so much about make-believe but rather robots’ ability to continually adapt to new scenarios without having to physically experience them first.

Zhang will receive $400,000 in funding over five years for his project, “Robot Reflection in Lifelong Adaptation.” Here, he answers a few more questions about the research and why it’s important for robots to be able to adapt to new and changing situations and yes, sometimes even imagine.

Q: What is your latest research focused on?

A: I am developing robot reflection and imagination capabilities to enable lifelong robot adaptation to change and novelty in not only the environment but also the robot itself. The long-term vision is to create lifelong collaborative robots that adapt at a near-human level in unstructured and ever-evolving worlds over their lifetimes.

Lifelong adaptation is vital for enabling robots to operate over long periods of time. Without effective adaptation, lifelong robots will inevitably fail, jeopardizing both their missions and surrounding environments. Change and novelty are two of the greatest challenges that prevent robots from achieving lifelong adaptation. The goal of this award is to develop and gain a fundamental understanding of an innovative paradigm called “robot reflection.” Inspired by theories of cognitive reflection for human adaptation, the robot reflection paradigm will allow a robot to reason about itself and adjust its own adaptation process without requiring human supervision.

To enable robot reflection for lifelong adaptation, the technical aims of this award are divided into three areas. The first thrust enables self-aware robot adaptation in context, through a method that offsets a robot perception and decision-making model according to combinations of changes in the environment and the robot itself. The second thrust addresses adapting the robot’s adaptation according to the degree of novelty in the robot’s experiences, through a novelty estimation approach that compares the current experiences against past episodes in the robot’s memory to estimate novelty, and a machine forgetting method to update the robot’s fixed-capacity memory by adaptively selecting past episodes to forget while avoiding catastrophic forgetting. The third thrust focuses on robot adaptation by imagination, without requiring a robot to physically experience a new scenario, through a novel approach that formulates robot imagination as sequential decision-making and explores interpretable methods for modeling the underlying relationships between experienced and new scenarios for data-efficient imagination.

Q: What do you find most exciting about your research?

A: I am excited about the potential of this research to continuously improve performance of robots over their lifetimes, in a way like humans. Consider a robot performing search and rescue – it would need to operate for days or months, working in dramatically varying environments and with different human teammates. Environmental changes may occur because of shifts in the weather, time of day, visual obstruction and terrain changes. Robots themselves may also change due to damage, natural wear and tear and varying configurations. In addition, in human-robot teaming, robots usually need to collaborate with different types of first responders, who perform different activities with different operation styles, personalities and preferences. So being capable of adaptation over their lifetimes is an exciting and necessary ability for robots to continue operating effectively and autonomously.

I am also excited about this research because it elegantly integrates theories of human cognitive reflection and design of algorithms with rigorous mathematical foundations. Clearly, there still exists a significant gap in adaptation abilities between robots and humans, especially when dealing with change and novelty. While using human reflection as an inspiration and a source of theories, this research develops principled algorithms of robot reflection and imagination that are based on rigorous mathematical foundations and will be evaluated using physical robots in real-world applications.

Q: What is the potential impact of this work?

A: This work stands to provide significant benefit to society by enabling robots that reflect to improve performance over their lifetimes. By introducing the fresh perspective of robot reflection and imagination, this research opens new doors in lifelong adaptation for robots to address change and novelty in unstructured and ever-evolving worlds, thus advancing knowledge in the robot intelligence community.

This research also provides a vital solution to improve safety, productivity and cost efficiency of using robots to address environmental, social and economic issues. This work has broader impacts across all applications of lifelong autonomy, such as robot-assisted inspection, subterranean and space exploration and autonomous driving, in which robots must operate over long periods of time without human supervision.

Q: How does this research agenda inform your teaching and outreach activities?

A: The highlight of this work is to integrate research with teaching by using reflection as a pedagogical tool to improve student learning. It is well recognized that reflection plays a critical role in education and it should be included as an integral part of student learning. However, reflection has yet not been widely used in teaching strategies in university education. Accordingly, the education component of this work will focus on developing a reflection module as a pedagogical tool and evaluate it in robotics and artificial intelligence courses. This research will also be integrated into my K-12 outreach program named PROGRESS (Program for Robotics Outreach on Gender and Racial Equity in School and Society) to broaden participation of K-12 students and teachers in robotics and computer science.