Rejected: Native Gaussian Splatting Pre-visualization in Godot Engine

Context

V-Sekai’s AI-generated VR content requires high-fidelity PreViz that preserves details from Flux/TRELLIS outputs. Current deep learning mesh-based workflows lose visual quality during generation. Key metrics:

  • 50k+ splats @ 90 FPS in Godot viewport
  • T-pose compliance for rigging compatibility

Problem Statement

  1. Quality Loss: Mesh conversion destroys AI-generated surface details
  2. Performance Limits: Existing Godot renderer lacks 3DGS optimization
  3. Workflow Friction: Manual retopology adds 3-5 hours/asset

Proposed Solution

Native 3D Gaussian Splatting (3DGS) renderer for Godot 4.3+ with:

  • Vulkan-optimized atomic splat sorting
  • T-posed characters
  • Direct Flux/TRELLIS data ingestion

Implementation Steps:

  1. Develop SPIR-V compute pipeline for depth sorting
  2. Create Godot Editor tools for splat timeline scrubbing
  3. Implement BC7-compressed covariance textures

Implementation Plan

  1. Phase 1: Core Renderer (4 Weeks)

    • Vulkan splat shader with spherical harmonics processing
    • GPU radix sort via compute shaders
    • Flux skeleton alignment system

  2. Phase 2: PreViz Tooling (2 Weeks)

    • SH coefficient debug overlay
    • Performance profiling HUD
    • Splat LOD distance culling

  3. Validation:

    • Success: 50k splats @ 90 FPS (RTX 3060), <0.5mm T-pose drift
    • Failure: >16ms/frame spikes, >2mm bone position deviation

Benefits

  • 98% PreViz quality retention vs 78% mesh workflow
  • Zero manual retopology required
  • 1:1 AI-to-engine visual parity

Risks and Limitations

  • Vulkan 1.3+ required (excludes legacy GPUs)
  • 18% VRAM overhead vs mesh rendering
  • Requires NVIDIA GPUs for optimal performance

Alternatives Considered

Option Pros Cons
Mesh PreViz Full compatibility Large quality loss
External Render Higher fidelity 300ms latency penalty

When to Avoid This Solution

  • WebGL-targeted projects
  • Teams without Vulkan shader expertise
  • Low-end mobile VR deployments

Organizational Alignment

Supports V-Sekai’s 2025 priorities:

  1. AI-to-engine content pipeline
  2. Photorealistic real-time PreViz
  3. Godot Vulkan performance leadership

Proposal Status

Status: Rejected

Decision Makers:

  • V-Sekai Pipeline Lead

Tags

  • Rendering
  • AI/ML
  • High Priority

References

  1. Reference Implementation (Inria)
    https://github.com/graphdeco-inria/gaussian-splatting
  2. Playcanvas Engine Demo
    https://github.com/playcanvas/engine
  3. Cesium Experimental Branch
    https://github.com/CesiumGS/cesium/tree/splat-spz-concept
  4. Unity Implementation (Aras-P)
    https://github.com/aras-p/UnityGaussianSplatting
  5. Godot Implementation (2Retr0)
    https://github.com/2Retr0/GodotGaussianSplatting
  6. Godot Implementation (haztro)
    https://github.com/haztro/godot-gaussian-splatting
  7. Blender Omniview Addon (Kiri Engine)
    https://github.com/Kiri-Innovation/3dgs-render-blender-addon
  8. Trellis https://github.com/microsoft/TRELLIS
  9. Flux