Human–robot teams struggle with ambiguous and low-bandwidth communication methods such as gestures, voice commands, and kinesthetic teaching, all of which suffer from occlusion, noise, processing delays, and high training burdens. Existing planners typically rely on static models and cannot incorporate live human preferences, forcing disruptive manual replanning. These limitations hinder smooth, dynamic, and scalable coordination between humans and autonomous robots, particularly in complex and time-sensitive environments.
The invention uses an augmented reality interface to display robots’ intended actions and trajectories as aligned virtual overlays, allowing users to impose constraints by interacting with objects in the scene. These interactions are converted into Answer Set Programming constraints through an Action Restrictor, which the ASP planner integrates to generate updated symbolic action sequences for multi-robot teams. A multi-threaded runtime executes and monitors plans while continuously updating and replanning based on AR-derived feedback, supporting real-time negotiation between humans and robots.
• Enables precise visualization of robot intent for low-ambiguity communication
• Delivers real-time, high-bandwidth interaction through AR overlays
• Reduces training burden by relying on intuitive AR gestures
• Supports dynamic replanning by injecting constraints directly into an ASP planner
• Coordinates multiple robots simultaneously through parallel plan execution
• Operates across common AR platforms without proprietary hardware requirements
• Improves safety by previewing potentially hazardous robot actions in AR
• Increases operational efficiency through reduced task delays and waiting time
• United States 11,958,183 Utility Filed 09/18/2020 Issue date 04/16/2024 Status Patented
Prototype
This technology is available for licensing.
Strong potential for robotics companies, industrial automation providers, and AR platform developers seeking intuitive, real-time collaboration systems that enhance efficiency, safety, and scalability in human-robot teaming.
Information available upon request.