Superseded by <20250427-backpack.md>: Phoenix-Based Recommender System Integration

Context

Godot Engine’s V-Sekai project requires a scalable recommendation system (200k+ user-items) with modern metadata handling. The current implementation does not exist. The decision to use standard Elixir Phoenix is driven by the fact that our backend (uro) is already implemented in Elixir, ensuring seamless integration and leveraging existing expertise.

┌────────────────────────────────────────────────────────────────────────────────┐
│                V-Sekai Backpack (General Scene Processing)                     │
├───────────────────────────┬───────────────────────────┬────────────────────────┤
│      Scene Ingestion      │   Knowledge Integration   │    Model Serving       │
│                           │                           │     (IREE/ONNX)        │
│  [Oban.Job, Membrane]     │  [librecommender/Rustler] │  [object-detection]    │
│                           │                           │  [keypoint-detection]  │
│                           │                           │  [semantic             |
│                           │                           │    segmentation]       |
│                           │                           │  [image-to-image]      │
│                           │                           │  [image-to-text]       │
│                           │                           │  [text-to-speech]      │
│                           │                           │  [speech-to-speech]    │
│                           │                           │  [speech-to-text]      │
│                           │                           │  [chain-of-thought]    │
├───────────────────────────┼───────────────────────────┼────────────────────────┤
│ - Godot Scene Upload      │ - JSON-LD Context Binding │ - Object Detection     │
│ - Multi-Camera Processing │ - Object Ontology         │ - Keypoint Analysis    │
│ - Oban Batch Insert       │ - librecommender ML       │ - Membrane Pipeline    │
│ - Scene includes XMP      │                           │                        │
│   JSON-LD descriptions    │                           │                        │
├───────────────────────────┴───────────────────────────┴────────────────────────┤
│                          General Analysis Pipeline                             │
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
│ │ Godot Client → [Oban.Worker] → Membrane Pipeline → [Rustler NIF] → CRDB    │ │
│ │   │               │               │                   │                    │ │
│ │   ▼               ▼               ▼                   ▼                    │ │
│ │ Raw Scene    View Params     Keypoint Vectors        Object Detection      │ │
│ │              (Membrane       (YOLO-NAS-POSE)          (RF-DETR +           │ │
│ │               Filter)                                   librecommender)    │ │
│ └─────────────────────────────────┬─────────────────────┬────────────────────┘ │
├───────────────────────────────────┼─────────────────────┼──────────────────────┤
│          General Storage          │                     │                      │
│ ┌───────────────────────┐ ┌───────┴───────┐ ┌───────────┴───────────┐          │
│ │ Scene Metadata        │ │ Detection     │ │ Keypoint Graph        │          │
│ │ ∘ Camera Calibration  │ │ Results       │ │ ∘ Vector Embeddings   │          │
│ │ ∘ Oban Job States     │ │ ∘ JSON-LD     │ │ ∘ Kinematic Chains    │          │
│ └───────┬───────────────┘ └───────┬───────┘ └───────────┬───────────┘          │
│         │                         │                     │                      │
│         └───────┬─────────────────┴─────────┬───────────┘                      │
├─────────────────┼───────────────────────────┼──────────────────────────────────┤
│    Analytics    │       Query Services      │         Integration              │
│ ┌───────────────▼──────────────┐ ┌──────────▼───────────┐ ┌───────────────────┐│
│ │ Detection Metrics            │ │ Vector Similarity    │ │ Avatar Control    ││
│ │ ∘ Oban Job Stats             │ │ ∘ librecommender     │ │ ∘ Membrane Stream ││
│ │ ∘ Membrane Throughput        │ │   KNN Search         │ │ ∘ Rustler NIF     ││
│ │ ∘ Oban web                   │ │                      │ │                   ││
│ │ ∘ Oban metrics               │ │                      │ │                   ││
├────────────────────────────────────────────────────────────────────────────────┤
│ Manages user authentication, authorization, and access control policies        │
│ to ensure secure system interactions and data privacy compliance               │
├────────────────────────────────────────────────────────────────────────────────┤
│ Dynamically adjusts processing parameters based on environmental inputs        │
│ and contextual awareness for optimal performance across variable conditions    │
├────────────────────────────────────────────────────────────────────────────────┤
│ Orchestrates allocation and prioritization of computational resources          │
│ across distributed components to maintain quality-of-service objectives        │
├────────────────────────────────────────────────────────────────────────────────┤
│ Synchronizes data flows and task execution between pipeline stages             │
│ through distributed scheduling and workflow management                         │
├────────────────────────────────────────────────────────────────────────────────┤
│ Implements self-contained processing modules with nested hierarchies           │
│ for recursive task decomposition and scalable architecture                     │
└────────────────────────────────────────────────────────────────────────────────┘

Problem Statement

  1. Scalability bottlenecks in Python-based recommendation serving
  2. Inefficient user/item bag management
  3. Missing JSON-LD attribute compatibility

Proposed Solution

Implement standard Elixir Phoenix framework with Rustler-based NIF integration for librecommender, featuring:

Implementation Steps:

  1. Create Phoenix context and schemas for recommendation entities
  2. Implement NIFs for bag operations using Rustler
  3. Add JSON-LD context annotation layer
  4. Support both item-item and user-item recommendation algorithms

Implementation Plan

  1. Phase 1: Phoenix core setup

    • Port uro entity relationships
    • Rustler integration tests

  2. Phase 2: Recommender bridge

    • Implement protocol for librecommender
    • JSON-LD attribute mapping

  3. Validation:

    • Success: <500ms p99 inference latency
    • Failure: >5% data serialization errors

Benefits

  • 10x concurrent recommendation capacity
  • Native Ecto/Phoenix metadata queries
  • Future-proofed for Fediverse integration

Risks

  1. Rust-Elixir type conversion overhead
  2. NIF complexity
  3. JSON-LD namespace collisions

Alternatives Considered

Option Pros Cons
Pure Python Faster MVP Limits scale
Pure Elixir Simpler stack Lacks optim algos

When to Avoid

Projects requiring simple recommendation needs (<1k items) or without Rust ops support.

Organizational Alignment

Supports V-Sekai goals for UGC and fediverse interoperability.

Tags

  • ML
  • Backend
  • High Priority

Proposal Status

Status: Superseded by <20250427-backpack.md>

References

  1. Phoenix Framework
  2. Rustler 0.30 NIF guide
  3. V-Sekai Roadmap 2025
  4. Librecommender Serving Guide
  5. Deep Interest Network for Click-Through Rate Prediction