Superceded by <20250427-recommender-system.md>: Phoenix-Based Recommender System Integration (Using Pythonx)

Context

Godot Engine’s V-Sekai project requires a scalable recommendation system (200k+ user-items) with modern metadata handling. The current implementation does not exist. The decision to use standard Elixir Phoenix is driven by the fact that our backend (uro) is already implemented in Elixir, ensuring seamless integration and leveraging existing expertise. Integration with Python-based ML libraries like librecommender will be handled using pythonx.

┌────────────────────────────────────────────────────────────────────────────────┐
│                V-Sekai Backpack (General Scene Processing)                     │
├───────────────────────────┬───────────────────────────┬────────────────────────┤
│      Scene Ingestion      │   Knowledge Integration   │    Model Serving       │
│                           │                           │     (Pythonx/ONNX)     │
│  [Oban.Job, Membrane]     │  [pythonx->librecommender]│  [object-detection]    │
│                           │                           │  [keypoint-detection]  │
│                           │                           │  [semantic             │
│                           │                           │    segmentation]       │
│                           │                           │  [image-to-image]      │
│                           │                           │  [image-to-text]       │
│                           │                           │  [text-to-speech]      │
│                           │                           │  [speech-to-speech]    │
│                           │                           │  [speech-to-text]      │
│                           │                           │  [chain-of-thought]    │
├───────────────────────────┼───────────────────────────┼────────────────────────┤
│ - Godot Scene Upload      │ - JSON-LD Context Binding │ - Object Detection     │
│ - Multi-Camera Processing │ - Object Ontology         │ - Keypoint Analysis    │
│ - Oban Batch Insert       │ - librecommender ML       │ - Membrane Pipeline    │
│ - Scene includes XMP      │   (via pythonx)           │                        │
│   JSON-LD descriptions    │                           │                        │
├───────────────────────────┴───────────────────────────┴────────────────────────┤
│                          General Analysis Pipeline                             │
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
│ │ Godot Client → [Oban.Worker] → Membrane Pipeline → [pythonx call] → CRDB   │ │
│ │      │               │               │                   │                 │ │
│ │      ▼               ▼               ▼                   ▼                 │ │
│ │  Raw Scene     View Params     Keypoint Vectors     Object Detection       │ │
│ │              (Membrane       (YOLO-NAS-POSE)      (RF-DETR +               │ │
│ │               Filter)                               librecommender)        │ │
│ └─────────────────────────────────┬─────────────────────┬────────────────────┘ │
├───────────────────────────────────┼─────────────────────┼──────────────────────┤
│          General Storage          │                     │                      │
│ ┌───────────────────────┐ ┌───────┴───────┐ ┌───────────┴───────────┐          │
│ │ Scene Metadata        │ │ Detection     │ │ Keypoint Graph        │          │
│ │ ∘ Camera Calibration  │ │ Results       │ │ ∘ Vector Embeddings   │          │
│ │ ∘ Oban Job States     │ │ ∘ JSON-LD     │ │ ∘ Kinematic Chains    │          │
│ └──────────┬────────────┘ └───────┬───────┘ └───────────┬───────────┘          │
│            │                      │                     │                      │
│            └───────────┬──────────┴─────────┬───────────┘                      │
├────────────────────────┼────────────────────┼──────────────────────────────────┤
│       Analytics        │   Query Services   │         Integration              │
│ ┌──────────────────────▼───────────┐ ┌──────▼─────────────┐ ┌────────────────┐ │
│ │ Detection Metrics                │ │ Vector Similarity  │ │ Avatar         | |
| |                                  | |                    | |   Control      │ │
│ │ ∘ Oban Job Stats                 │ │ ∘ librecommender   │ │ ∘ Membrane     | |
| | ∘ Membrane Throughput            | |   KNN Search       | |     Stream     │ │
│ │ ∘ Oban web                       │ │     (via pythonx)  │ │ ∘ pythonx      | |
| | ∘ Oban metrics                   | |                    | |     call       │ │
│ └──────────────────────────────────┘ └────────────────────┘ └────────────────┘ │
├────────────────────────────────────────────────────────────────────────────────┤
│ Manages user authentication, authorization, and access control policies        │
│ to ensure secure system interactions and data privacy compliance               │
├────────────────────────────────────────────────────────────────────────────────┤
│ Dynamically adjusts processing parameters based on environmental inputs        │
│ and contextual awareness for optimal performance across variable conditions    │
├────────────────────────────────────────────────────────────────────────────────┤
│ Orchestrates allocation and prioritization of computational resources          │
│ across distributed components to maintain quality-of-service objectives        │
├────────────────────────────────────────────────────────────────────────────────┤
│ Synchronizes data flows and task execution between pipeline stages             │
│ through distributed scheduling and workflow management                         │
├────────────────────────────────────────────────────────────────────────────────┤
│ Implements self-contained processing modules with nested hierarchies           │
│ for recursive task decomposition and scalable architecture                     │
└────────────────────────────────────────────────────────────────────────────────┘

Problem Statement

  1. Scalability bottlenecks in Python-based recommendation serving when run standalone.
  2. Inefficient user/item bag management if handled purely in Elixir without optimized libraries.
  3. Missing JSON-LD attribute compatibility layer for recommendation metadata.

Proposed Solution

Implement standard Elixir Phoenix framework integrated with Python’s librecommender via the pythonx library, featuring:
Implementation Steps:

  1. Create Phoenix context and schemas for recommendation entities.
  2. Implement Elixir functions that use pythonx to call librecommender’s bag operations and recommendation algorithms.
  3. Add JSON-LD context annotation layer within Phoenix/Ecto schemas.
  4. Support both item-item and user-item recommendation algorithms by calling the appropriate Python functions.

Implementation Plan

  1. Phase 1: Phoenix Core & Pythonx Setup
    • Port uro entity relationships to Phoenix/Ecto schemas.
    • Set up pythonx and configure the Python environment.
    • Develop basic pythonx integration tests calling simple Python functions.
    • Stage 1: Implement Oban workers that directly call librecommender functions via pythonx for initial testing and simple workflows.
  2. Phase 2: Recommender Bridge & Membrane Integration
    • Implement the Elixir protocol/interface for calling librecommender’s core recommendation logic via pythonx.
    • Map JSON-LD attributes within Ecto schemas or dedicated context modules.
    • Stage 2: Integrate pythonx calls within Membrane pipelines, potentially triggered by Oban jobs, for more complex, stream-based processing (e.g., analyzing scene data).
  3. Validation:
    • Success: <500ms p99 inference latency for recommendation calls via pythonx.
    • Failure: >5% data serialization errors between Elixir and Python OR significant performance degradation due to pythonx overhead compared to benchmarks.

Benefits

  • Leverages Elixir’s concurrency for handling recommendation requests.
  • Utilizes mature Python ML ecosystem (librecommender) without rewriting algorithms.
  • Native Ecto/Phoenix metadata queries.
  • Future-proofed for Fediverse integration.

Risks

  1. Elixir-Python Interop Overhead: Performance impact of data serialization/deserialization between BEAM and Python using pythonx.
  2. Python’s GIL: Potential concurrency limitations within the Python process called by pythonx, although Elixir’s concurrency can manage multiple external calls.
  3. Deployment Complexity: Managing both Elixir and Python environments/dependencies in deployment.
  4. Error Handling: Robustly handling errors originating from the Python side within Elixir.
  5. JSON-LD Namespace Collisions: Ensuring consistent JSON-LD context handling.

Alternatives Considered

Option Pros Cons
Pure Python Faster MVP, unified ML stack Limits backend scalability, concurrency
Pure Elixir Simpler stack, BEAM concurrency Lacks optimized ML algos, requires rewrite
Elixir + Rustler Potentially highest performance NIF Rust dev complexity, NIF maintenance

When to Avoid

Projects requiring simple recommendation needs (<1k items), without complex backend orchestration needs, or where minimizing deployment complexity (avoiding Python runtime) is paramount.

Organizational Alignment

Supports V-Sekai goals for UGC and fediverse interoperability by integrating advanced recommendations into the Elixir backend.

Tags

  • ML
  • Backend
  • High Priority
  • Python Integration

References

  1. Phoenix Framework
  2. Pythonx Documentation
  3. V-Sekai Roadmap 2025
  4. Librecommender Serving Guide (Note: We are using Python via pythonx, not the Rust serving directly)
  5. Deep Interest Network for Click-Through Rate Prediction
  6. Oban
  7. Membrane Framework