Dr. Elliot
  1. Home
    2.  » 
  2. Posts
    3.  » 

Scene Proxies in Unreal Engine

#Programming
#CPP
#C++
#UnrealEngine

One of the largest performance overheads in Unreal is the need for actors and primitive components to draw something in a world. Today I want to show you a way to bypass these abstractions entirely and manually draw primitives to the world with minimal overhead.

Before we dive into the implementation, let’s talk about how the engine actually manages what’s being rendered in a UWorld.

Understanding the Scene Architecture

Inside the UWorld class, there’s a property called Scene. If you’ve worked with other graphical applications, this concept should be familiar, it’s the common way to refer to a collection of primitives being sent to a renderer for drawing. Unreal manages these using what are called scene proxies, which act as the bridge between the game thread and the render thread. You can do this with any primitive type, however, the more complex the primitive, the more difficult it becomes.

Today we’ll be working with several core classes:

FStaticMeshSceneProxyDesc
FStaticMeshSceneProxy
FPrimitiveSceneDesc
FPrimitiveSceneInfoData
FCustomPrimitiveData

Let’s break down each of these, because understanding their relationships is crucial to using them effectively.

FStaticMeshSceneProxyDesc

This struct inherits from FPrimitiveSceneProxyDesc and serves as a descriptor, essentially a data packet containing everything needed to create a scene proxy. It’s a wrapper around the data you’d typically expect your UPrimitiveComponent to have, tracking properties like whether the primitive casts shadows, whether it should render to the depth pass, the main pass, velocity pass, and so on. Think of it as a snapshot of all the rendering-relevant state at a specific moment in time.

FStaticMeshSceneProxy

As the comment in PrimitiveSceneProxy.h suggests, this proxy is a mirrored representation of a UStaticMeshComponent. Here’s where things get interesting: Unreal tracks this resource on the render thread. It’s the render thread’s window into what the game thread wants to draw. This separation is fundamental to Unreal’s architecture, the game thread prepares data, and the render thread consumes it asynchronously.

The proxy contains all the mesh data, material references, LOD information, and rendering state needed to actually submit draw calls. It’s designed to be read frequently by the render thread while being modified infrequently by the game thread.

FPrimitiveSceneDesc

This is the descriptor used when adding or updating a primitive in the scene. It contains the transformation matrix, bounding volumes, the owner object reference, and a pointer to the FPrimitiveSceneInfoData. Essentially, it’s the parameter struct for telling the scene “here’s a new thing to render” or “this thing moved.”

FPrimitiveSceneInfoData

This structure is the scene’s internal representation of a primitive. It contains the actual FPrimitiveSceneProxy pointer, along with a unique PrimitiveSceneId that the scene uses to track this primitive in its internal data structures. This is what gets stored in the scene’s arrays and spatial acceleration structures.

FCustomPrimitiveData

This allows you to pass arbitrary float data to your materials. It’s an array of floats (typically 4-byte aligned for GPU efficiency) that gets uploaded to the GPU and is accessible in materials through the Custom Primitive Data node. Very useful for per-instance variation without creating unique material instances.

The Performance Problem

One of the largest bottlenecks in the engine happens at the end of each frame, specifically in UWorld::SendAllEndOfFrameUpdates(). As the name suggests, at the end of the frame, the engine gathers any and all primitives that were marked dirty during that frame. “Dirty” can mean a lot of things: transform updates, material changes, mesh swaps, visibility changes, or any of a dozen other state modifications. This gets especially complex with sophisticated primitive components like Niagara particle systems.

The main issue? This architecture completely prevents you from parallelizing transform updates on primitives. No matter how many threads you use to calculate new positions, they all get funneled through this single-threaded end-of-frame update on the game thread.

Once the game thread finishes collecting these updates, they’re sent to the FScene to be organized, sorted, and integrated into the scene’s internal data structures. This happens in FScene::UpdateAllPrimitiveSceneInfos(), which then calls into the truly massive FScene::Update() function (lines 5175-6480 in RenderScene.cpp as of UE 5.6—yes, over 1,300 lines).

This function handles:

  • Updating primitive transforms and bounds
  • Rebuilding spatial acceleration structures (octrees, etc.)
  • Invalidating cached lighting and shadow data
  • Updating Lumen scene representation
  • Recalculating visibility states
  • Updating GPU Scene data structures
  • And much, much more…

It’s a critical function, but it’s also a significant cost when you have hundreds or thousands of moving objects. So you may be asking: “Elliot, why can’t I just add the proxy to the scene myself and update its transform without the UWorld’s involvement?” Well, you’re in luck. That’s exactly what I’m going to show you.

Setting Up the Data Structure

First, we need to create a structure to hold all our proxy-related data:

struct FTestProxyData
{
	FStaticMeshSceneProxyDesc ProxyDesc;
	FPrimitiveSceneDesc SceneDesc;
	FPrimitiveSceneInfoData SceneInfoData;
	FCustomPrimitiveData PrimitiveData;
};

You’ll need to add RenderCore to your module’s .Build.cs file for this:

PublicDependencyModuleNames.AddRange(new string[] { 
    "Core", 
    "CoreUObject", 
    "Engine",
    "RenderCore"  // ... 
});

Adding the Proxy to the Scene

Now, let’s add this to the scene. For this example, we’ll place an actor in our world with the necessary properties:

void AYourActor::BeginPlay()
{
	Super::BeginPlay();

	TArray<TObjectPtr<UMaterialInterface>, TInlineAllocator<1>> Materials;
	Materials.Add(Material);
        
	Data = MakeUnique<FTestProxyData>();

	Data->PrimitiveData.Data = { 255.0f, 0.0f, 0.0f };
	
	Data->ProxyDesc.StaticMesh = Mesh;  // Your UStaticMesh reference
	Data->ProxyDesc.CustomPrimitiveData = &Data->PrimitiveData;
	Data->ProxyDesc.NaniteResources = Mesh->GetRenderData()->NaniteResourcesPtr.Get();
	Data->ProxyDesc.Scene = GetWorld()->Scene;
	Data->ProxyDesc.Owner = this;  // The actor owns this proxy
	Data->ProxyDesc.ComponentId = Data->SceneInfoData.PrimitiveSceneId;
	Data->ProxyDesc.OverrideMaterials = MakeArrayView(Materials);
	

	Proxy = ::new FStaticMeshSceneProxy(Data->ProxyDesc, false);
	Data->SceneInfoData.SceneProxy = Proxy;

	Data->SceneDesc.World = GetWorld();
	Data->SceneDesc.PrimitiveUObject = this;
	Data->SceneDesc.PrimitiveSceneData = &Data->SceneInfoData;
	Data->SceneDesc.LocalBounds = Mesh->GetBounds();

	Data->SceneDesc.RenderMatrix = GetActorTransform().ToMatrixWithScale();
	Data->SceneDesc.AttachmentRootPosition = GetActorTransform().GetLocation();
	Data->SceneDesc.Bounds = Mesh->GetBounds().TransformBy(GetActorTransform());
    
	GetWorld()->Scene->AddPrimitive(&Data->SceneDesc);
	GetWorld()->Scene->UpdatePrimitiveTransform(&Data->SceneDesc);
}

Let’s break down what’s happening here:

  1. Material Setup: We create an inline array to avoid heap allocation for the common single-material case
  2. Custom Primitive Data: We can pass arbitrary data to shaders—in this case, RGB values
  3. Proxy Descriptor: We’re filling in all the metadata the proxy needs to know how to render
  4. Scene Descriptor: This is what the scene uses to track and update our primitive
  5. Adding to Scene: AddPrimitive() registers it with the scene, and UpdatePrimitiveTransform() ensures the transform is properly initialized

Proper Cleanup

This is critical—we used render thread resources, so we need to clean them up on the render thread to avoid use-after-free bugs:

void AYourActor::EndPlay(const EEndPlayReason::Type EndPlayReason)
{
	Super::EndPlay(EndPlayReason);

	GetWorld()->Scene->RemovePrimitive(&Data->SceneDesc);

	TUniquePtr<FTestProxyData> DataToDelete = MoveTemp(Data);	
	ENQUEUE_RENDER_COMMAND(CleanupSceneProxy)([DataToDelete = MoveTemp(DataToDelete)](FRHICommandListImmediate&) mutable
	{
		DataToDelete.Reset();
	});
}

Why this pattern? The scene proxy might still be referenced by in-flight render commands. By deferring deletion to the render thread, we guarantee all references are done before we delete.

Parallelizing Transform Updates

Now for the payoff, let’s parallelize those transform updates:

void AYourActor::Tick(float DeltaSeconds)
{
	Super::Tick(DeltaSeconds);

	// We gotta tell the scene we're about to update multiple primitives
	// This allows it to batch internal processing
	GetWorld()->Scene->StartUpdatePrimitiveTransform(Datas.Num());

	ParallelFor(Datas.Num(), [&](int32 Index)
	{
		auto& Data = Datas[Index];

		// Update the transform matrix
		Data->SceneDesc.RenderMatrix = CalculateTransformForIndex(Index).ToMatrixWithScale();
		
		// Update the bounds
		Data->SceneDesc.Bounds = Mesh->GetBounds().TransformBy(Data->SceneDesc.RenderMatrix);
		
		// Push the update to the scene
		// This is thread-safe because each thread updates a different primitive
		GetWorld()->Scene->UpdatePrimitiveTransform(&Data->SceneDesc);
	});
}

This completely bypasses UWorld::SendAllEndOfFrameUpdates(). Each thread updates its own subset of primitives, and the scene handles the updates in a thread-safe manner. The StartUpdatePrimitiveTransform() call pre-allocates any necessary internal structures to avoid race conditions.

Important Considerations

This example is simplified for clarity. In production code, you’d want to consider:

  1. Memory Layout: TArray<TUniquePtr<>> isn’t cache-friendly. Consider using a single allocation with offsets, or TChunkedArray for better iteration performance.

  2. Batch Size: ParallelFor has scheduling overhead. Don’t parallelize unless you have enough work (typically 100+ items minimum).

  3. Bounds Updates: We update bounds every frame here, but if your mesh doesn’t deform, you can cache transformed bounds and only recalculate when the transform actually changes.

  4. Dirty Tracking: Implement your own dirty flags to avoid updating unchanged primitives.

  5. LOD Selection: The scene proxy handles LOD selection, but you might want to manually control this for very distant objects.

IPrimitiveComponent Interface

There’s another piece to this puzzle: the IPrimitiveComponent interface. By implementing this interface, you can:

  • Precache PSOs: Tell the engine which Pipeline State Objects you’ll need, reducing hitching
  • Handle visibility queries: Control culling behavior without a full component
  • Respond to scene changes: Get callbacks when the lighting rebuilds or world is reloaded

Here’s a minimal implementation:

class FCustomPrimitiveComponent : public IPrimitiveComponent
{
public:

	// Called when the primitive is added to the scene
	virtual void OnRegister() override
	{
		// ...
	}
	
	// Called when the primitive is removed from the scene
	virtual void OnUnregister() override
	{
		// ...
	}
	
	virtual FPrimitiveSceneProxy* GetSceneProxy() const override
	{
		return SceneProxy;
	}
	
	// Can be used to precache PSOs
	virtual void CollectPSOPrecacheData(const FPSOPrecacheParams& Params, FComponentPSOPrecacheParamsList& OutParams) override
	{
		// Could add your material/vertex factory combinations here
	}

private:
	FStaticMeshSceneProxy* SceneProxy = nullptr;
};

This gives you the benefits of the component system (PSO precaching, editor integration, etc.) without the overhead of the full UPrimitiveComponent machinery.

Manual scene proxy management is a powerful technique for squeezing maximum performance out of Unreal’s rendering system. By bypassing the component abstraction layer and working directly with the scene, you gain fine-grained control over when and how updates happen.

The biggest benefit is the ability to parallelize transform updates and eliminate the end-of-frame synchronization bottleneck. For systems with large numbers of simple, moving objects, this can translate to dramatic frame rate improvements.

You should note, you’re operating outside the safety rails of the component system, so careful attention to threading, memory management, and resource lifetimes is essential.

If you have questions, optimizations, or interesting use cases you’ve discovered, please drop a comment below or reach out on Discord. I’d love to hear how you’re using this technique in your projects!

Thanks for reading!

Code examples tested with Unreal Engine 5.6. Some APIs may change in future versions.