Proposed: Camera matching with fSpy.

Metadata

  • Status: Proposed
  • Deciders: V-Sekai
  • Tags: V-Sekai

The Backdrop

V-Sekai’s mission to be gateways to 3D environments needs camera matching, which is currently a bottleneck in our workflow.

The Challenge

Our artists and developers need to align concept screenshots within the 3D spaces due to inconsistent camera parameter estimation.

The Strategy

We will adopt the fSpy tool that uses vanishing points to estimate camera parameters from images, facilitating a more precise camera match-moving process.

Guide

  1. Find parallel lines and use the longest lines you can.
  2. You can move the axes to recenter.

The Upside

The integration of fSpy into our workflow promises:

  1. A considerable reduction in time for setting up camera parameters.
  2. Enhanced accuracy and realism in matching 3D models with background images.
  3. Streamlined asset creation processes, thereby supporting quick iterations.

The Downside

Some challenges associated with this strategy include:

  • An initial learning phase required for effective use of the fSpy tool.
  • Potential limitations in images that lack well-defined perspective lines or geometric structure.
  • Reliance on an external tool which may necessitate ongoing maintenance efforts.

The Road Not Taken

Rather than using fSpy, we could continue manual estimation techniques or even develop a custom solution tailored to our specific requirements—both are valid but likely less efficient approaches.

The Infrequent Use Case

For cases where fSpy struggles due to images with no clear vanishing points, alternative methods of camera estimation must be devised, potentially falling back on manual techniques or other specialized tools.

In Core and Done by Us?

fSpy integration should become central to our asset creation pipeline. We’ll need to invest in training and potentially develop complementary in-house tools to bridge the gap between fSpy outputs and our existing systems.

Further Reading

  1. V-Sekai GitHub Repository - Explore how V-Sekai leverages social VR functions within Godot Engine.
  2. V-Sekai Game Project - Delve into the collaborative effort of creating social VR/VRSNS/metaverse elements, all open-source and accessible.

AI assistant Aria assisted with this proposal.