Researchers Develop a Robot That Mimics User's Movements in VR

Cornell and Brown University researchers have unveiled VRoxy, an advanced telepresence robot that responds automatically and in real-time to a user's movements and gestures in virtual reality (VR).

This technology facilitates seamless collaboration between a remote user in a confined space, such as an office, and teammates in a larger environment through virtual reality.

VRoxy is the latest achievement in remote robotic embodiment developed by researchers in the Cornell Ann S. Bowers College of Computing and Information Science.

All About VRoxy

Mose Sakashita, a doctoral student in the field of information science, emphasized the advantages of virtual reality in enabling various locomotion techniques commonly used in VR games.

This functionality empowers remote users to occupy a limited physical space while actively participating in activities in a larger remote environment.

Sakashita is the lead author of the study titled "VRoxy: Wide-Area Collaboration From an Office Using a VR-Driven Robotic Proxy," which will be presented at the ACM Symposium on User Interface Software and Technology.

The research team said that the real-time responsiveness of VRoxy is crucial for both local and remote teammates. With VRoxy acting as a proxy, a remote teammate in a confined office can engage in group activities taking place in a significantly larger space, such as a collaborative design scenario.

The robot mimics the user's body position and other nonverbal cues, offering a more immersive and realistic experience compared to traditional telepresence robots and video conferencing platforms.

VRoxy builds on the foundation of a previous Cornell robot called ReMotion, which required both local and remote users to have identical hardware and workspaces.

However, according to the researchers, VRoxy transcends these limitations by mapping subtle movements from the remote user in VR to more significant actions in the physical space.

Immersive Viewing Modes

With a 360-degree camera, a monitor displaying facial expressions, a robotic pointer finger, and omnidirectional wheels, VRoxy provides users with immersive viewing modes, including Live mode for real-time interaction and navigational mode for efficient mobility.

The automatic nature of VRoxy allows remote teammates to focus entirely on collaboration rather than manual robot control. In future iterations, researchers aim to enhance VRoxy with robotic arms for physical object interaction and develop real-time mapping capabilities, potentially expanding its applications in educational settings.

"The great benefit of virtual reality is we can leverage all kinds of locomotion techniques that people use in virtual reality games, like instantly moving from one position to another," Sakashita said in a statement.

"This functionality enables remote users to physically occupy a very limited amount of space but collaborate with teammates in a much larger remote environment," he added.