Today the US Patent & Trademark Office published a patent application from Apple relating to relates to a VR glove capable of measuring the movement of individual finger and thumb bones. The VR gloves, made with smart fabric technology could be used in many applications including games, education and military training.
Virtual reality (VR) technology can be used for many applications such as military training, educational learning, and video games. VR technology can use one or more electronic devices to simulate a virtual environment and the user's physical presence in that virtual environment. One type of VR technology is augmented reality (AR) technology, where the user's real environment can be supplemented with computer-generated objects or content. Another type of VR technology is mixed reality (MR) technology, where the user's real environment and the virtual environment can be blended together.
VR/AR/MR technology can be simulated using one or more electronic devices. One electronic device can be a VR headset, where the user can use the VR headset to see the simulated virtual environment. As the user moves their head to look around, a display included in the headset can update to reflect the user's head movement. Another electronic device can include one or more cameras. The one or more cameras can be used to capture the user's real environment in AR technology and/or can be used for positional tracking. Yet another electronic device can include VR gloves. VR gloves can be worn over the user's hands and can allow the user to touch, feel, and hold virtual objects in real-time. VR gloves capable of accurately detecting the positions and motions of the user's hands and fingers without occlusion may be desired.
Apple's invention relates to a VR glove capable of measuring the movement of individual finger and thumb bones. The VR glove can include a plurality of inertial measurement units (IMUs) to track the movement of one or more finger and/or hand sections.
The IMUs can include one or more motion sensors, such as a gyroscope and an accelerometer, for measuring the orientation, position, and velocity of objects (e.g., finger bones) that the IMU can be attached. An IMU can be located proximate to a finger (or thumb) bone and can measure the inertial motion of the corresponding bone.
In some examples, the VR glove may include magnetometers to determine the direction of the geo-magnetic field. The VR glove can also include one or more other electronic components, such as a plurality of electrodes for sensing the heading, enabling capacitive touch, and/or contact sensing between finger tips.
The VR glove can also include force sensors, actuators for haptic feedback, temperature sensors, and heaters. The VR glove can further include logic such as an on-board controller, a connector, a transceiver, a battery, and the like.
One or more buses can be used to carry signals between the electronic components to, e.g., the on-hand controller. The VR glove can be a fabric glove where one or more (e.g., all) electronics components can be knitted or woven into the glove.
In some examples, the fabric can be stitched together using conductive threads. Examples of the disclosure can further include detecting movement of the glove without using magnetometers by comparing the motion detected by the IMUs with local frame(s). Heading initialization can include rotation optimizers, and heading correction can include detecting irregularities (e.g., collision, rotations, etc.) of the nodes associated with the IMUs.
Apple's patent FIG. 1 below illustrates an exemplary model of a human hand; FIG. 2 illustrates a back view of an exemplary VR glove.
Apple further notes that the VR glove #230 can be capable of fine-level motion capture. That is, the VR glove can be capable of discerning between the movement of an entire finger (e.g., the user waiving their index finger) and the movement of a finger joint (e.g., the user bending the index finger), for example.
Apple's patent FIG. 4B below illustrates an exemplary hand reference; FIG. 4C illustrates exemplary local frames for a plurality of fingers and a thumb included in a hand reference.
Apple's patent FIG. 6A below illustrates an exemplary collision of two or more fingers; FIG. 6B illustrates exemplary rotations of two or more nodes.
Apple further notes that the VR gloves can be worn over the hands of the user and can track the motion (e.g., positions and velocities) of the user's hands and fingers. In some instances, the VR gloves can allow the user to touch, feel, and hold virtual objects in real-time.
In some examples, one or more functions and features of the VR gloves can allow the user to interact with the simulated virtual environment in the absence of cameras and with reduced occlusion.
The VR gloves can also communicate with one or more other components (e.g., the host device) in the system. For example, the host device can store one or more properties of control objects (e.g., a coffee mug) and can transmit information (e.g., the shape, location, feel, etc.) about the control object to the VR gloves.
VR Gloves using Smart Fabric
Apple notes that the VR glove can be a knitted or woven glove where one or more (e.g., all) electronics components can be integrated into the fabric of the glove.
In some examples, glove can include a fabric with multiple sections having a different number of layers.
For example, sections can comprise a single layer where an electronic component may not exist. Where an electronic component (e.g., an IMU) exists, the fabric can be a two-layer fabric with a pocket that secures the electronic component between the layers. Electronic components can be respectively positioned.
For example, IMUs can be located proximate to the center of a finger bone. Electronic components that can be woven into the fabric of the glove can include, but are not limited to, the IMUs, the electrodes, and the controller. In some examples, the fabric can be stitched together using conductive threads.
Apple's patent application that was published today by the U.S. Patent Office was originally filed back in Q3 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
Three of the eight inventors include Zebina Mass, Product Development that came to Apple via NASA; Andy Sinesio, Engineering Manager, Siri Music; and Jean Lu, Touch & Sensing Mechanical Prototyping.