Aximmetry's Technical Architecture and Engineering Practices in Real-Time Virtual Production and Live Streaming

In today's field of Virtual Production and broadcast-grade live streaming, real-time performance, visual consistency, and system stability are the three core challenges. Aximmetry (especially the Aximmetry DE dual-engine version), through deep integration with Unreal Engine, has built a mature real-time rendering pipeline and has become the preferred solution for many professional teams. This article will delve into the core implementation principles and engineering applications of Aximmetry in virtual production and live streaming from a technical architecture perspective.

I. Dual-Engine Architecture: The Collaborative Mechanism of Unreal Engine + Aximmetry

Aximmetry DE adopts a uniqueDual-Engine Parallel Architecture：

• Unreal Engine Responsible for high-fidelity scene construction, PBR materials, lighting (Lumen), particle systems (Niagara), and complex geometry rendering.
• Aximmetry Engine Responsible for video I/O, Chroma Keying, multi-camera tracking synchronization, node logic control, and final compositing output.

The two exchange data through Shared Texture and low-latency IPC (Inter-Process Communication). The Render Target output from the UE side is transferred to Aximmetry with extremely low latency (typically < 1 frame), where it completes final keying, color correction, and multi-channel output. This architecture preserves UE's cinematic visual quality while compensating for its shortcomings in broadcast-grade video signal processing and real-time stability.

In practical engineering, it is recommended to use Unreal Engine 5.1 or later paired with the Aximmetry 2024 series, and enable nDisplay 或 is correctly enabled. to achieve multi-machine cluster rendering.

II. Advanced Chroma Keyer Technology

Aximmetry's keying module is one of its most competitive technologies. Its core algorithm is based onMulti-sample layered analysis + intelligent edge repair, supporting the following key features:

• Multi-Sample Keying: Allows separate sampling for different brightness areas, significantly reducing Spill Suppression and edge artifacts.
• Fine Detail Retention: Built-in Hair Mask and Detail Enhancement algorithms effectively preserve alpha information for hair strands, gauzy clothing, and semi-transparent objects.
• Realistic Shadow Catcher: Through Separate Shadow Pass technology, real actor shadows can be independently extracted and projected onto the UE virtual scene's plane, achieving physically correct light and shadow integration.

Engineering Tuning Recommendations:

• Adjust the Core Range、Edge Softness 和 Spill Correction parameters in the keying node in conjunction with the Waveform monitor and Vectorscope for fine-tuning.
• For complex lighting environments, it is recommended to use 4K SDI input + 12-bit color depth, and enable the Degrain noise reduction node to reduce the impact of noise on keying accuracy.

III. Camera Tracking and Synchronization System

The core of real-time virtual production lies inframe synchronization between the real camera and the virtual cameraAximmetry supports the following tracking protocols:

• High-End Optical Tracking: Mo-Sys, stYpe, Ncam, receiving 6DoF (Position + Rotation) data via FreeD or proprietary TCP/UDP protocols, with latency controllable within 1-2 frames.
• Inertial + Optical Hybrid Tracking: Vicon, OptiTrack, HTC Vive Tracker.
• Low-Cost Solutions: Intel RealSense, Orbbec, phone IMU combined with ARKit/ARCore.

The Tracking Mapper node within Aximmetry enables multi-point calibration (Lens Distortion Calibration + nodal point offset), and achieves frame-accurate synchronization with external video devices via Genlock 和 Timecode(LTC / PTP). In XR projects, it is crucial to configure the Frustum Rendering parameters to ensure the LED wall display content perfectly matches the real camera's field of view, avoiding perspective distortion.

IV. Node-Based Programming and Real-Time Control Logic

Aximmetry uses avisual Node Graph for logic construction, similar to the workflow of Houdini or TouchDesigner. Its advantages include:

• Modular Design: All functions (Video In, Keyer, Tracker, Render, Output, OSC/MIDI/DMX control) exist as nodes, supporting custom Compounds.
• Real-Time Performance: Node calculations are efficiently scheduled between GPU/CPU, supporting Hot Modification during live broadcasts without interrupting output.
• Data Interfaces: Built-in powerful data binding capabilities, allowing connection to external data sources like Excel, SQL, OSC, MQTT, WebSocket, for dynamic graphics, real-time scores, or audience interaction.

Typical Live Workflow Node Chain Example: SDI Input → Tracker Sync → Chroma Keyer → UE Render Target → Color Correction → Multi-View Output (Program + Preview) → NDI/SRT Encoder

V. XR and LED Wall Virtual Production Technology Implementation

In LED wall XR projects, Aximmetry's core technologies areReal-time Frustum Rendering 和 and the Color Matching Pipeline。

• It uses camera tracking data to calculate the current field of view's projection area on the LED wall in real time, rendering only necessary content to reduce GPU load.
• The built-in ICVFX (In-Camera VFX) toolset enables quick LED wall color calibration, lens distortion correction, and Black Level Compensation.
• Supports Multi-Machine Setup, achieving distributed rendering via Aximmetry Remote to meet the demands of large XR studios.

Key Performance Optimization Points:

• Color Matching Tool When using Nanite + Lumen , it is recommended to keep the virtual scene's Poly Count within a reasonable range and enable Temporal Upsampling。
• For GPU load monitoring, NVIDIA Nsight or Aximmetry's built-in Performance Monitor is recommended.

Summary and Technology Trends

With its dual-engine architecture, high-precision keying algorithm, flexible node system, and comprehensive tracking and I/O support, Aximmetry has become a mature technology platform bridging Unreal Engine and broadcast-grade live production pipelines. It transforms traditional post-production-driven virtual production workflows into a modern engineering system centered on real-time, pre-production, and controllability.

In the future, with AI-assisted keying, Neural Rendering, and support for higher frame rates (120fps+), Aximmetry is expected to further lower the technical barrier for virtual production, enabling more teams to achieve cinematic real-time visual effects in an engineering manner.

For engineers planning virtual studios or XR projects, it is recommended to start with the official tutorials from Aximmetry Academy , prioritizing mastery of the Tracking Calibration 和 and Advanced Keying modules, as these form the foundation for stable system operation.