Researchers at Tohoku University have proposed a novel approach to recapture 3-D motion data from a flexible magnetic flux sensor array, using deep learning and a structure-aware temporal bilateral filter. This approach has made tracking activities more efficient. It can track various movements, including fingers manipulating objects, beetles moving inside a vivarium with leaves and soil, and opaque fluid flow.
The challenges associated with 3-D tracking
Computing the 3D configuration of markers from flux sensor data is challenging. This is because the existing numerical methods experience system noise, dead angles, the need for initialization, and constraints in the sensor array’s layout.
Using Optical cameras is the leading method to track movements at present. But, optical cameras suffer from accuracy and authenticity problems. For instance, the camera fails to detect the motion if a finger or object veils the view. The SOTA performance Magnetic tracking technology used to detect sharp motion also faces limitations. The classic tracking technique creates bias, and magnetic sources have a dead-angle intricacy.
The novel approach
Therefore, the researchers have employed deep neural networks that learn the regression from the simulation flux values to the LC coils’ 3D configuration. This can be applied to the actual LC coils at any position and orientation within the capture capacity. To address system noise and the dead-angle limitation caused by the hardware and sensing principle’s characteristics, the researchers have proposed a structure-aware temporal bilateral filter for reconstructing motion sequences.
The marker is oriented vertically to the flux sensor plan, and the captured motion patterns are computed with the novel approach and numerical methods. The traditional way has a particular bias. But the proposed deep-learning system solves the difficulty, thus achieving increased accuracy. The filters are used to reconstruct the data when the motion contains occasional dead-angle data, giving continuous and reliable motion. The lightweight wireless markers do not require any power supply. Thus this method can track movements for a long time.
The new integrated system can track multiple LC coils at 100Hz speed at millimeter level accuracy. As the system learns itself, this helps to reconstruct the tracking loss due to the dead-angle.
The proposed structure-aware temporal bilateral filter leverages the sensors’ raw measurement. Therefore their performance surpasses other conventional filters. The flux sensor design’s flexibility allows users to reconfigure it based on their applications, making this novel approach suitable for various virtual reality applications.
Paper: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9226608
Tanushree Shenwai is a consulting intern at MarktechPost. She is currently pursuing her B.Tech from the Indian Institute of Technology(IIT), Bhubaneswar. She is a Data Science enthusiast and has a keen interest in the scope of application of artificial intelligence in various fields. She is passionate about exploring the new advancements in technologies and their real-life application.
