Project Lead Unreal VR
PC
3
~6 Months
VR cooking
A long-term VR research project focused on studying human grasp behavior in virtual reality to improve robotic grasping systems for cooking assistance scenarios. The project explored how humans interact with kitchen objects in VR and how those interactions could be translated into robotic learning systems.
Overview
The Virtual Cooking project aimed to analyze and replicate human grasp behavior for robotic-assisted cooking applications.
The research focused on:
- Capturing realistic human hand interactions in VR
- Analyzing grasp types and force application
- Comparing virtual grasp behavior with real-world grasping
- Collecting motion and interaction data for robotics research
The project combined VR interaction systems, motion capture technology, and experimental data collection inside a synchronized real-world and virtual kitchen environment.
Technical Break Down
Contributions
Contributed in a technical leadership role throughout the project, supporting both development workflow and team coordination.
Responsibilities included:
- Managing GitLab repositories and maintaining project version control workflows
- Delegating tasks and coordinating development priorities across a small team of two developers
- Supporting team members with Unreal Engine setup and game-engine-related workflows
- Assisting with HTC Vive hardware setup, configuration, and troubleshooting
- Providing technical guidance and onboarding support throughout development
Worked extensively with the CyberGlove system to capture detailed finger and hand movement data.
Responsibilities included:
- Setting up and calibrating the CyberGlove hardware
- Configuring finger joint tracking and sensor alignment
- Troubleshooting calibration inconsistencies
- Synchronizing glove input with VR interaction systems
The glove system was used to capture precise grasping movements for research analysis.
Configured and calibrated the OptiTrack tracking system for full spatial motion tracking inside the lab environment.
This included:
- Setting up tracking cameras and calibration volumes
- Configuring rigid bodies for hand and object tracking
- Synchronizing OptiTrack data with the CyberGlove system
- Ensuring accurate spatial alignment between physical and virtual environments
Special focus was placed on maintaining reliable positional tracking during interaction-heavy experiments.
Integrated HTC Vive hardware into the research pipeline to provide immersive VR interaction and spatial synchronization.
Responsibilities included:
- Setting up Vive tracking systems and VR calibration
- Synchronizing Vive positional data with CyberGlove input
- Aligning tracking spaces across multiple hardware systems
- Maintaining stable VR runtime environments during experiments
Implemented custom VR interaction systems inside Unreal Engine to support grasp analysis and data collection.
Key implementations included:
- Sticky grasp mechanics using Unreal collision systems
- Object interaction logic for grasp detection
- Recording world-space hand positions and rotations during interactions
- Saving raw CyberGlove finger rotation data for evaluation pipelines
These systems formed the core interaction layer used during participant testing sessions.
Explored replacing the CyberGlove system with Leap Motion hand tracking due to hardware degradation in several glove joints. This involved: testing alternative hand-tracking pipelines evaluating tracking precision and reliability researching improved data acquisition methods comparing optical hand tracking against glove-based tracking systems The goal was to improve overall grasp accuracy and reduce hardware limitations.
Worked on aligning the virtual kitchen environment with the real-world lab setup to support spatial consistency and haptic feedback.
Responsibilities included:
- Setting up the virtual kitchen environment in Unreal Engine
- Calibrating virtual hand materials and tracking alignment
- Matching physical object placement with virtual representations
- Configuring the OptiTrack lab space to mirror the VR environment
This synchronization enabled participants to physically interact with real objects while immersed in VR.
Participated in running research experiments and collecting participant interaction data. This included: preparing and calibrating all hardware before sessions assisting participants during VR experiments monitoring tracking stability and interaction recording collecting and organizing motion/grasp datasets for evaluation The collected data was later used for analysis of grasp behavior and robotic learning research.
Key REsults
The project investigated how humans naturally grasp kitchen objects in immersive environments and how that information could help improve robotic grasping systems.
Areas of focus included:
- Grasp positioning
- Finger articulation and joint rotation
- Interaction force behavior
- Comparison of virtual and real-world grasping
- Human-to-robot interaction modeling
Tech Stack
- Unreal Engine
- HTC Vive
- CyberGlove
- OptiTrack Motion Capture System
- Leap Motion
- Photogrammetry (Meshroom)
- C++ / Blueprint Systems
- VR Interaction Systems
- Motion Capture & Tracking Pipelines
- Research Data Collection Systems