Light and Shadow Coexistence: Aximmetry Dynamic Shadow Reconstruction

In compositing for green screen or blue screen virtual studios, the gold standard for judging whether a live-action actor truly “integrates” into a virtual 3D world is not how clean the keying is, butthe shadow cast by the actor on the virtual ground (Real-time Virtual Shadow Casting)。

In the physical world, light passing through a body leaves a physical shadow on the ground and walls, characterized by a gradient penumbra and extremely high contact occlusion. It is this shadow that visually establishes the spatial contact relationship between the body and the ground.

However, when a Technical Director (TD) attempts todeploy Genlock in native Unreal Engine 5 (UE5)When rendering the projection of a live-action character, a severe geometric reconstruction crisis is encountered:

• “Flat Card” Projection Distortion: In UE5, a live-action actor is typically carried on a 2D Billboard facing the camera. If a virtual light source (e.g., Direct Light) is used directly to illuminate this billboard to generate a projection, when the virtual light source angle is tilted, the shadow cast on the ground exhibits a highly unnatural, flat, and distorted “cardboard” silhouette. Once the camera moves, the shadow can even suffer from severe perspective misalignment.
• Foot “Floating” and Loss of Contact Shadow: When the engine's Virtual Shadow Map (VSM) or Cascaded Shadow Map (CSM) processes a 2D patch, due to the lack of real 3D mesh topology, it cannot produce fine, high-frequency dark areas at the junction between the actor's shoe sole and the virtual ground —Contact ShadowsThis causes the actor to appear in the frame like a sticker floating in mid-air.

Aximmetry Aximmetry, with its proprietary“Dynamic Light Projection Matrix & Micro-Occlusion Pipeline”, reconstructs the generation and physical attenuation of shadows at the GPU memory level, completely eliminating the “floating sensation” of live-action characters in the digital space.

I. Perspective Reset: 3D Projection Matrix Based on Virtual Light Source Space Transformation

To make a 2D live-action image produce a realistic projection with 3D depth perspective, the 2D pixels must be re-projected into the “Light Frustum” of the light source when calculating the shadow.

Aximmetry deploys a set of“Light-Space Reprojection”"Penumbra Feathering Based on Motion Vectors"

Capture Virtual Light Source Vector

Aximmetry extracts the 3D spatial coordinates, orientation (Vector), and light type (Directional/Point/Spot) of the main light source (e.g., sunlight or stage spotlight) in the UE5 scene in real-time.

Construct Light-Space Projection Matrix

The system uses the virtual light source as the viewpoint to construct a real-time projection matrix for the actor's Alpha channel. When rendering the shadow frame, Aximmetry's shader takes the keyed 2D silhouette of the actor and forcibly re-projects it onto the surface of the virtual ground geometry along the physical direction of the light source. This means that no matter how the camera moves, the shadow on the ground is dynamically stretched and perspective-shifted 1:1 according to the physical angle of the light source, completely eliminating the stiffness of the “flat card” projection.

II. Foot Anchoring: Contact Occlusion Based on Screen Space and Distance Field

In the physical world, when a shoe sole presses against the ground, light can barely enter the tiny crevices, forming an extremely deep and sharp dark edge. This is the key to establishing the visual anchor of “feet firmly on the ground.”

Aximmetry introduces“Sub-pixel Foot Contact Shadow Injection”technology:

Extract Physical Foot Contact Point Coordinates

Aximmetry precisely locks theZero-Height Contact Line。

Generate Micro-Occlusion Gradient

Around the contact line, Aximmetry does not rely on the standard shadow calculations of the UE scene. It invokes a dedicated GPU 2D shader, using the contact line as the center, to radiate outward and generate a non-linear, high-saturationAmbient Occlusion (AO) Attenuation Gradient。

Hardware-Level G-Buffer Injection

This high-frequency dark area data is directly injected into the G-Buffer (Depth and Ambient Occlusion channels) of the Unreal Engine rendering pipeline. When the rendering engine performs the final pixel shading, this delicate physical contact shadow firmly “locks” the actor's soles onto the virtual floor, instantly eliminating any sense of floating or suspension in space.

III. Physical Penumbra Dissolution: Adaptive Dynamic Blur Filter Based on Distance Attenuation

Real shadows have the physical property of being “sharp near, soft far” — the projection close to the feet is very sharp, while the shadow cast by the head onto a distant surface becomes very soft and blurred (penumbra) due to light wave diffraction and the physical size of the light source.

Aximmetry deploys a“Distance-Based Variable Blur”pipeline:

Depth Field Mapping

Aximmetry calculates theSpatial DistanceThe closer to the sole, the smaller the distance value; the farther from the head, the larger the distance value.

Variable Kernel Blur

When rendering the Shadow Map on the GPU, Aximmetry uses this distance value in real-time as a multiplier for the Gaussian Blur Kernel Radius.

• At the sole (distance approaches 0), the blur kernel shrinks to sub-pixel level, and the shadow edge is sharp as a knife;
• In the head projection area (larger distance), the blur kernel linearly expands to tens of pixels, and the shadow edge exhibits an extremely natural optical diffusion.

Radiance Attenuation

Simultaneously, the depth (opacity) of the shadow undergoes non-linear physical attenuation as distance increases, perfectly simulating the secondary filling of distant shadows by ambient light in the physical world.

Conclusion: Letting the Digital Illusion Take Root

In the art of real-time virtual production, the shadow is the most subtle yet powerful “gravitational bond” connecting the virtual 3D space and the real carbon-based actor. Without this projection obeying physical laws, the actor and the scene would remain perpetually disconnected.

Unreal Engine 5, with its VSM and Lumen, reconstructs the light and shadow order of the digital world, but when faced with live-action video lacking 3D geometric topology, it inherently has algorithmic blind spots regarding projection perspective and micro-contact occlusion.

Aximmetry Leveraging its profound insight into physical optical properties, it acts deep within the memory as the“Physical Stitcher of Gravity and Light and Shadow”。

It endows shadows with true 3D perspective through light-space reprojection, firmly anchors the actor's feet to the ground through sub-pixel micro-occlusion technology, and restores the optical aesthetic of “sharp near, soft far” through adaptive penumbra blur. It is precisely because of Aximmetry's hardcore dynamic shadow reconstruction pipeline that the virtual ground truly bears the “physical weight” of the real actor, allowing the blended virtual-real image to take root in the dimension where light and shadow intertwine.