Mixed Reality: How Computer Vision Makes It Possible

Prof. Dr. Marc Pollefeys, Institute of Visual Computing, ETH Zurich & Microsoft MR & AI lab Zurich

Mixed Reality combines real and virtual elements in interactive experiences. This will change how we solve complicated real-world tasks and how we learn or have fun in the future. For these experiences to be seamless and intuitive, the Mixed Reality devices need to understand the environment and the user. Specifically, to create the illusion of a static hologram in the world, the device needs to keep track of how the user moves through the world to apply the same motion to the virtual camera when rendering the holograms. The 3D geometry of the scene can be used to enable holograms to interact with the world, and tracking the user’s hands and eyes enables the intuitive manipulation of the hologram. All these capabilities rely heavily on advanced real-time computer vision algorithms. This presentation will briefly introduce some of the algorithms running on devices such as Microsoft HoloLens and present some applications of Mixed Reality.

Talk in English