OSTNet: Calibration Method for Optical See-Through Head-Mounted Displays via Non-Parametric Distortion Map Generation

(IEEE ISMAR 2020 Poster)

Project Page

https://www.ar.c.titech.ac.jp/projects/ostnet-2019

Abstract

We propose a spatial calibration method for Optical See-Through Head-Mounted Displays (OST-HMDs) having complex optical distortion such as wide field-of-view (FoV) designs. When designing OST-HMDs, a trade-off exists between the complexity of the optics and image quality. For example, extending FoV often requires OST-HMDs to employ complex optics such as a curved beam combiner, which inevitably induce stronger optical distortion. Such non-linear, viewpoint-dependent optical distortion makes existing spatial calibration methods either impossible to handle or difficult to compensate without intensive computation. To overcome this issue, we propose OSTNet, a non-parametric calibration method that creates a generative 2D distortion model for given six-degree-of-freedom (DoF) viewpoint pose. Specifically, we train a variational autoencoder from a sparse dataset of a distortion observation of different viewpoints, and map the optical distortion into a latent space, then learn a regression from a 6-DoF pose to a distortion map. In the experiment, we applied our method on an OST-HMD with 90 degree FoV using 125 training positions, and showed that the calibration error in the viewing angular was on average about 9.68 arcmin, which is comparable to a conventional linear interpolation method. We then further provide discussion on how we improve the approach in a more practical setup. We think our method allows people to design OST-HMDs with more flexible optics while compensating complex distortion in software at the rendering in real time.

Publications

• Kiyosato Someya, Yuichi Hiroi, Makoto Yamada, Yuta Itoh, “OSTNet: Calibration Method for Optical See-Through Head-Mounted Displays via Non-Parametric Distortion Map Generation,” IEEE ISMAR Adjunct 2019, Beijing, China, pp 259-260, Oct. 14-18, 2019