Marc Pollefeys
Computer Vision and Geometry lab (CVG)
Institute of Visual Computing
Dept. of Computer Science, ETH Zurich and Microsoft MR & AI lab Zurich
Zurich, Switzerland

Preserving Image-Based Localization

Image-based localization is a core component of many augmented/mixed reality (AR/MR) and autonomous robotic systems. Current localization systems rely on the persistent storage of 3D point clouds of the scene to enable camera pose estimation, but such data reveals potentially sensitive scene information. This gives rise to significant privacy risks, especially as for many applications 3D mapping is a background process that the user might not be fully aware of. We pose the following question: How can we avoid disclosing confidential information about the captured 3D scene, and yet allow reliable camera pose estimation? We proposed the first solution to what we call privacy preserving image-based localization. The key idea of our approach is to transform the map representation from a 3D point cloud to a 3D line cloud. This novel representation obfuscates the underlying scene geometry while providing sufficient geometric constraints to enable robust and accurate 6-DOF camera pose estimation. Inversely, the same concept can be used to transform 2D feature points in the image to 2D lines similarly allowing to preserve privacy and render image inversion attacks ineffective. Extensive experiments on several datasets and localization scenarios underline the high practical relevance of our proposed approach.