About Cascais - Street Explore

Building Capture and Reconstruction

To capture the building, I began by filming three loops around its perimeter at different heights, keeping the camera focused on the facade throughout the process. I then imported the raw footage into Spirula Studio, where frames were automatically extracted and used to generate a 3D point cloud and train the primary exterior Gaussian splat.

To reconstruct the interior, I imported the exterior splat into Blender using Kiri Engine's 3DGS addon as a spatial reference. After aligning the virtual walls, doors, and windows with the real-world scan, I rendered the interior as a 360-degree panorama and processed it through Spirula Studio to generate the interior splat.

Since the building was scanned from the ground without the use of a drone, viewing it from above revealed noticeable gaps in the reconstruction. My initial approach was to fill these gaps using 3D mesh data from OpenStreetMap, but the difference in visual style between the mesh and the splats was too pronounced. To maintain a consistent appearance, I instead generated a Gaussian splat from street-view panoramas, allowing the building to blend naturally into its surrounding neighborhood.

Choosing a Web Renderer

Bringing the project to the web required finding the right balance between performance and flexibility. My first implementation using Spark in Three.js proved too demanding, as it loaded the entire splat file at once, making it unsuitable for a smooth, interactive walkthrough.

I also explored alternatives such as SuperSplat and PlayCanvas, but ultimately settled on GaussianSplatLite because of its efficient sorting and stochastic rendering mode. This allowed me to maintain smooth performance in the browser while retaining full control over custom camera movements and interaction logic through Three.js.

I also made an initial attempt to build the project using PlayCanvas. However, since I was less familiar with its workflow, manipulating the splats and controlling camera movements became a more time-consuming process.

Each of the available renderers has its own advantages and disadvantages. Solutions such as SuperSplat and PlayCanvas provide comprehensive toolsets and their own rendering environments, but I ultimately prioritized the flexibility and direct control offered by a custom Three.js implementation.

Matching the Interior and Exterior

One of the main challenges was ensuring visual consistency between the interior design and the exterior scan. The street-view reconstruction contained a mixture of open and closed windows, which meant that the 3D-modeled windows had to match their original counterparts in terms of position, color, materials, and transparency.

This required carefully aligning the modeled window frames and glass properties in Blender with the corresponding windows captured in the exterior splat, so that the transition between the two environments remained as seamless as possible.

Managing Splat Bleeding

Another major technical challenge was the nature of Gaussian splats and their tendency to bleed through surfaces. As of September 2026, the available tools do not provide a reliable way to strictly constrain the spatial placement of individual splats. Consequently, when multiple splats are combined, their points can overlap and become visible within each other's environments, including through interior walls.

Although some splats can be manually cleaned, removing too many points can compromise textures, reflections, and the illusion of flat surfaces. This makes aggressive cleanup impractical without negatively affecting the overall visual quality.

For this project, I addressed the issue by designing the interior layout with deliberate offsets from the exterior shell, particularly along the two main sides of the building. This allowed the exterior splats to bleed slightly into the interior around the outer edges, while keeping the main rooms relatively free from visual clutter.

Custom Collision and Camera Movement

Implementing custom collision detection was another significant challenge, particularly when trying to align the collision geometry with the visual representation of the Gaussian splats. Since the splats do not provide conventional, clearly defined surfaces, matching the collision boundaries to the perceived geometry required careful adjustment.

This became especially noticeable when interacting with the stairs. While moving down them, the player would occasionally shake due to inconsistencies between the collision system and the visual geometry.

I resolved this by introducing vertical constraints at specific locations along the stairs, preventing unwanted vertical movement and stabilizing the player's position. I also implemented camera interpolation and smoothing to make the movement more gradual and eliminate abrupt changes in camera position.