Stylized 3D Scene Representation from Scanned Environments

Abstract

The goal of 3D indoor scene stylization is to transform the digital representation of real-world scenes into a stylized version. This process is a complex interplay of artistic aesthetics and technical design. While there have been significant advancements in both 3D style transfer and scene understanding, the potential of integrating scene understanding into the process of 3D indoor scene stylization remains largely untapped. This thesis explores the integration of scene understanding into 3D indoor scene stylization, focusing on the creation of immersive Virtual Reality (VR) applications.

We present a pipeline beginning with the 3D scanning of real-world spaces, followed by scene understanding, and the use of Blender and its Python API for stylization, culminating in an interactive VR experience. Our approach demonstrates how scene understanding can enrich the stylization process, and how this integration can be leveraged to create immersive VR applications.

We conducted a user study to evaluate the quality of applying different stylization techniques for 3D indoor scenes in VR. The main focus of this study was to investigate the impact of taking scene understanding into account for the stylization. The findings indicate that the quality of materials and textures in the scene plays a substantial role in enhancing users’ sense of immersion in the virtual environment.