Zero-Copy Cross-Boundary Bridging: Bidirectional Shared Memory and D3D12 Resource Direct Pipeline between Aximmetry and Unreal Engine 5

In the technical architecture of modern Mixed Reality (MR) and In-Camera VFX (ICVFX),“Multi-Engine Collaboration”is the inevitable choice for achieving极致画质 and industrial-grade broadcast safety.

Typically,Unreal Engine 5 (UE5)with its Nanite (ultra-high-precision geometry) and Lumen (real-time global illumination) technologies, plays the role of an unparalleled “3D Dream Weaver”; while Aximmetry with its rigid broadcast-grade I/O routing, ultra-low latency chroma keying, and PTP/Genlock clocking, serves as the “Hardcore Physical Compositing Base” for the entire broadcast system.

However, when technical teams attempt to combine these two giants, they encounter a graphics-level“Inter-Process Pixel Transport Catastrophe (Inter-Process Bottleneck)”：

1. The PCIe Bus Throughput Black Hole: In a typical compositing pipeline, Aximmetry needs to send the captured 4K 10-bit HDR live video (input media) into UE5 for virtual occlusion or environmental refraction calculations; simultaneously, the 3D scene and Alpha depth map rendered by UE5 must be transmitted back to Aximmetry in real-time for final high-precision color compositing and broadcast distribution. If conventional inter-process communication (such as traditional Windows sockets, NDI, or even CPU memory-based shared regions) is used, image data must be repeatedly and frequently transferred between GPU VRAM → CPU System Memory (RAM) → GPU VRAM The data volume for a single 4K 60fps 32-bit floating-point uncompressed texture is as high as 16 GbpsThis high-frequency “VRAM-Memory” copy instantly saturates the PCIe bus bandwidth, causing severe GPU render thread stalls and frame drops.
2. Timing Tearing and Sub-Frame Desynchronization: Since Aximmetry's compositing main thread and UE5's render thread run in different process spaces, without underlying hardware-level synchronization, severeRace Conditionsoccur when both read and write to the same shared pixel region. This manifests as jagged timing tearing at the edges of the frame, and “sub-frame latency desynchronization” between the virtual background and the live actor during fast motion.

To achieve instantaneous 4K HDR pixel stream travel between the two engines without consuming any PCIe bandwidth or introducing any CPU overhead,Aximmetry a highly proprietary“Bidirectional Zero-Copy Direct Pipeline Based on DirectX 12 Shared Heaps and Hardware-Level GPU Fences”。

I. VRAM Boundary Dissolution: Zero-Copy Cross-Process Intercommunication via D3D12 Shared Heaps and NT Handles

To enable pixel flow with “0 microsecond” cross-boundary migration, Aximmetry completely eliminates the CPU memory as a relay station. At the DirectX 12 driver level, it virtually connects the GPU VRAM spaces of the Aximmetry process and the UE5 process:

D3D12 Shared Heap Allocation

During system initialization, the Aximmetry VRAM manager directly calls the underlying API of the Windows Display Driver Model (WDDM) to allocate a cross-process shareable D3D12 Heap (D3D12_HEAP_FLAG_SHARED)in physical VRAM. The physical address of this heap is locked firmly in the GPU's high-speed VRAM, disallowing any page-out to CPU system memory.

NT Handle Cross-Process Secure Bridging

Aximmetry encapsulates the GPU physical pointer of this shared heap into a Windows kernel-level Shared NT HandleThrough Aximmetry's dedicated UE5 plugin (Aximmetry Camera Adapter), Unreal Engine 5's rendering device (D3D12 Device) directly imports this NT handle upon startup and performs “Resource In-place Reconstruction”。

on the shared VRAM region within its virtual memory space.

• Bidirectional Copy-Free Texture Binding (In-VRAM Texture Binding) Aximmetry → UE5:
• After being captured, unpacked, and converted to ACEScg FP16 format by Aximmetry, the live video and high-precision alpha mask are written directly into this shared VRAM block. UE5's Material System (Material Graph) directly binds it as a Texture Sampler via the shared handle, requiring no data transfer. UE5 → Aximmetry:

The background frame and depth map (G-Buffer) rendered by UE5 similarly expose the VRAM address directly to Aximmetry's 3D compositor via the shared handle.In this exceptionally clean pipeline,pixels never leave the core area of the VRAM chip

The data transfer overhead between engines is completely eliminated, and the PCIe bus is perfectly freed by 100%.

II. Fence Hard Lock: Cross-Engine Asynchronous Rendering Timing Coordination Based on GPU FencesAfter solving the physical channel of "VRAM sharing," the most stringentRead-Write Timing Synchronization

must be addressed. If Aximmetry is writing a video texture while UE5 simultaneously reads it, "dirty data" and screen tearing occur.“Instead of using OS semaphores that block the CPU thread, Aximmetry introduces a GPU-level”"Hardware-Level Shared GPU Fences"

mechanism:

Hardware-Level Fence Object Creation (ID3D12Fence)Aximmetry creates a global hardware-level synchronizer queryable by both engines inside the GPU—ID3D12Fence

This Fence has an incrementing 64-bit counter value and runs directly in the GPU's Command Queue Scheduler.

• Asynchronous Command Queue "Traffic Light" Control Write Notification: When Aximmetry's GPU completes the despill and format conversion of a live video frame and finishes writing to the shared VRAM, it appends a Signal(Fence, CurrentValue)
• command to the GPU render queue. Read Wait: Before Unreal Engine 5's D3D12 command list begins rendering the 3D scene and sampling this live texture, its GPU queue first executes a non-blocking Wait(Fence, CurrentValue)
• instruction. Hardware-Level Seamless Suspension:

If Aximmetry's write is not yet complete, the GPU's hardware scheduler suspends UE5's rendering instruction for that item, immediately redirecting the GPU's Compute Units to execute other computational tasks in UE5's scene (e.g., particle updates), preventing the entire GPU thread from idling (stalling).

Sub-Pixel Temporal Alignment Once Aximmetry's Signal is issued, the GPU scheduler instantly wakes UE5's read and rasterization rendering. This "traffic light" control, established directly at the GPU hardware kernel level 1 millimeters, tightly constrains the timing jitter between the two independent engines to

, completely eliminating any temporal misalignment or screen tearing caused by inter-process communication latency jitter.

III. High-Bandwidth Pixel Decoupling: Cross-Engine G-Buffer Depth Interception and Real-Time Physical Occlusion“In Extended Reality (XR) and green screen compositing, the biggest headache for technical directors is”"Dynamic Occlusion"

For example, when a live actor walks in front of a virtual pillar and then moves behind a virtual table. To achieve this dynamic interleaving, traditional methods involve slicing the background inside UE5, but this introduces multiple layers of material mask rendering and edge artifacts. Aximmetry leverages this DirectX 12 shared VRAM heap to implement highly efficient：

G-Buffer Depth Interception and Real-Time Compositing Algorithm

Intercept and Extract UE5's 32-bit Depth Buffer The moment UE5 completes its 3D rendering, the Aximmetry plugin, via the D3D12 shared handle, intercepts and losslessly extracts the current frame's。

32-bit high-precision depth map (Z-Buffer)

directly from Unreal Engine's swap chain.

• Real-Time 3D Spatial Depth Comparison
• This depth map is instantly pushed within VRAM to Aximmetry's GPU compositing node. At this point, Aximmetry's proprietary compositor performs an extremely lightweight pixel-level depth test inside the GPU:
• The system compares the tracked depth of the live camera (or the depth of a preset 3D garbage mask) in real-time with the virtual depth value of the corresponding pixel in UE5's Z-Buffer.

If the live actor's depth is less than the virtual object's depth, the system retains the live pixel (actor in front of the object);

Otherwise, it overwrites the live actor with the virtual background pixel (actor behind the object).Zero-Artifact Depth FeatheringBecause this comparison is performed in parallel within Aximmetry's 32-bit floating-point VRAM environment, the compositing engine can apply adaptive

Sub-pixel Depth Softening

at the comparison boundary. It eliminates the common “black edges” and “hard jagged cuts” at traditional depth occlusion boundaries, allowing the live actor and the complex virtual 3D scene to interweave perfectly, presenting a high-order occlusion relationship akin to physical compositing.

Conclusion: A Unified Digital Spacetime

Aximmetry In today's relentless push towards high resolution, high dynamic range, and low latency in virtual production, the "data silos" and "bus barriers" between multiple software platforms are the primary culprits in bottlenecking top-tier server hardware performance. Without eliminating data copy overhead in multi-engine collaboration at the underlying level, any claimed leap in image quality is merely a castle in the sky built at the cost of stability.“Unreal Engine 5 is like a genius painter, splashing light and shadow in the 3D world; while Aximmetry acts like a stern judge of time and physics, rigorously coordinating data in the real world.”。

The greatness of