The COVID-19 pandemic has introduced new norms into our lives, bringing new business opportunities for industries through innovative technological experiences in AR, VR, and various mixed reality applications. In this project, we collaborated with ITRI and III to combine MR (mixed reality) technology with a physical claw machine, launching the “MR Claw Machine—Happy Farm.” By wearing MR glasses, users can enter a world where virtual objects and physical objects coexist, offering interactive experiences and challenge tasks.
In the “MR Claw Machine—Happy Farm,” players can choose from three different levels of difficulty, with higher scores awarded for faster task completion. First, physical markers are placed inside the claw machine, and six-degree-of-freedom spatial positioning is used to align virtual toys with these physical markers, enhancing the sense of real immersion. After donning the MR glasses, players will see the Happy Farm’s scenery, including grass, trees, cottages, and flowing water. Through 3D virtual and physical overlay positioning, players use glove sensors and finger-bending detection to control the movement and opening/closing of the virtual claw machine’s claw. Vibration feedback simulates the sensation of grabbing objects, achieving a realism of over 90%. For the auditory experience, the relative position and distance between sound sources and cameras are used to simulate or synthesize 3D spatial sounds, creating a more realistic challenge effect.
In the realm of virtual reality applications, Taiwan’s software sector predominantly focuses on games, film and animation, and integrated marketing. However, there is significant potential for growth in key areas such as interactive content creation tools, 3D virtual-physical overlay positioning, virtual-physical tactile feedback technology, and virtual-physical sound technology. This project attempts to integrate multiple technologies, aiming to construct a more diverse five-sensory experience, thereby promoting the development of a more comprehensive industry chain.
Client｜Institute for Information Industry (III), Industrial Technology Research Institute (ITRI)
Creative Team｜Yuri Studio
Executive Director｜Allan Yang
Art Director｜Allan Yang
3D Art Design｜Yun-Chung Tsai, Yi-Rou Jiao
UIUX Designer｜Allan Yang
Programming Design｜Aslan Wu, Yu-Ren Chang
Sensing Device｜Allan Yang, Aslan Wu