HySAR
HySAR: Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality Projection
(IEEE TVCG 2018, presented in IEEE VR 2018)
Project Page
https://www.ar.c.titech.ac.jp/projects/hysar
Abstract
Spatial augmented reality (SAR) pursues realism in rendering materials and objects. To advance this goal, we propose a hybrid SAR (HySAR) that combines a projector with optical see-through head-mounted displays (OST-HMD). In an ordinary SAR scenario with co-located viewers, the viewers perceive the same virtual material on physical surfaces. In general, the material consists of two components: a view-independent (VI) component such as diffuse reflection, and a view-dependent (VD) component such as specular reflection. The VI component is static over viewpoints, whereas the VD should change for each viewpoint even if a projector can simulate only one viewpoint at one time. In HySAR, a projector only renders the static VI components. In addition, the OST-HMD renders the dynamic VD components according to the viewer's current viewpoint. Unlike conventional SAR, the HySAR concept theoretically allows an unlimited number of co-located viewers to see the correct material over different viewpoints. Furthermore, the combination enhances the total dynamic range, the maximum intensity, and the resolution of perceived materials. With proof-of-concept systems, we demonstrate HySAR both qualitatively and quantitatively with real objects. First, we demonstrate HySAR by rendering synthetic material properties on a real object from different viewpoints. Our quantitative evaluation shows that our system increases the dynamic range by 2.24 times and the maximum intensity by 2.12 times compared to an ordinary SAR system. Second, we replicate the material properties of a real object by SAR and HySAR, and show that HySAR outperforms SAR in rendering VD specular components. 空間型拡張現実感(Spatial Augmented Reality, SAR)と透過型の拡張現実感ディスプレイ(例: Microsoft HoloLens)を組み合わせた、高品位な質感再現レンダリング手法を提案した。
日本語(Japanese)
HySAR: プロジェクションマッピングと透過型ARを組み合わせた質感再現
SARはプロジェクションマッピングとも呼ばれ、現実空間に所望の質感をもつ3次元物体を再現する試みに活用されている。しかし、SARで描画される模様は見る方向が変わっても同じであるため、見る方向に応じてキラッと光る（＝鏡面反射の位置が変わる）ような光沢感や、構造色のような複雑な質感を、複数の観察者がいる環境で正確に表現することができなかった。
本研究では、「 HySAR ( Hybrid Spatial Augmented Reality)」と呼ばれる、従来のSARにメガネ型の透過型ディスプレイを組み合わせたレンダリングフレームワークを提案する。HySARでは、光沢のような見る方向に依存して変化する情報はメガネ型ディスプレイに、見る方向が変わっても変化しない情報はプロジェクションマッピングにより提示する。HySARにより視点依存の質感表示を実現するだけでなく、質感表示において本質的に重要な、色域（色の表現範囲）の向上、ダイナミックレンジ（明るさの表現範囲）の向上、を実現した。
（本研究は、大阪大学・岩井大輔准教授との共同研究です）
Publications
- Takumi Hamasaki, Yuta Itoh, Yuichi Hiroi, Daisuke Iwai, and Maki Sugimoto, "HySAR: Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality Projection," IEEE Transactions on Visualization and Computer Graphics (Proceedings of IEEE Conference on Virtual Reality and 3D User Interfaces), Vol. 24, No. 4, pp. 1457-1466, 2018. [pdf]
- Yuichi Hiroi, Yuta Itoh, Takumi Hamasaki, Daisuke Iwai, and Maki Sugimoto, "HySAR: Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality Projection," In Proceedings of IEEE Virtual Reality, pp. 209-210, 2017.