Hi,
I am trying to figure out a proper way to manage following situation. We have a sublayer stack in a persistent stage that gets rendered using USDImagingGLEngine using Vulkan backend (USD version is 24.03). Between frames the stack composition can change: sublayers might get removed or added etc.
What I’m seeing is that for example if a new layer is added to the stack or existing layer removed and new added in its place, all existing prims are removed from cache and then added back during repopulation, which causes for example shader recompilation because material prims are also removed. Here’s a log snippet from USDIMAGING_CHANGES messages from a very minimal stage for reproduction:
[Repopulate] Populating </> on stage rootLayer
[Repopulate] Root path: </>
[Add HdPrim Info] </mysphere> adapter=UsdImagingSphereAdapter
[Add dependency] </mysphere> -> </mysphere>
[Add HdPrim Info] </Camera/Camera> adapter=UsdImagingCameraAdapter
[Add dependency] </Camera/Camera> -> </Camera/Camera>
[Repopulate] 2 variability tasks in worker
---- RENDERED FRAME 1 ------
[Objects Changed] Notice recieved from stage with root layer @anon:000002957CCAE990:rootLayer@
- Resync queued: /
[usdPathsToResync] Updating cache map for /
[Resync Prim]: </>
[_GatherDependencies] Found entry in flattened cache for / with 2 paths
- affected child prim: </Camera/Camera>
- affected child prim: </mysphere>
[Remove Rprim] </mysphere>
[Remove Sprim] </Camera/Camera>
[Remove PrimInfo] </Camera/Camera>
[Remove PrimInfo] </mysphere>
[Remove dependency] </Camera/Camera> -> </Camera/Camera>
[Remove dependency] </mysphere> -> </mysphere>
[Repopulate] Populating </> on stage rootLayer
[Repopulate] Root path: </Camera/Camera>
[Repopulate] Root path: </mysphere>
[Add HdPrim Info] </Camera/Camera> adapter=UsdImagingCameraAdapter
[Add dependency] </Camera/Camera> -> </Camera/Camera>
[Add HdPrim Info] </mysphere> adapter=UsdImagingSphereAdapter
[Add dependency] </mysphere> -> </mysphere>
[Repopulate] 2 variability tasks in worker
---- RENDERED FRAME 2 ------
Before rendering frame 2 a new layer was added to the stack that added an override for the XForm of /mysphere.
Minimal code that can reproduce it (actual renderer setup and invocation details are omitted):
auto rootLayer = pxr::SdfLayer::CreateAnonymous("rootLayer");
auto rootStage = pxr::UsdStage::Open(rootLayer);
// Sphere
auto sphereLayer = pxr::SdfLayer::CreateAnonymous("SphereLayer");
auto sphereStage = pxr::UsdStage::Open(sphereLayer);
auto sphere = pxr::UsdGeomSphere::Define(sphereStage, pxr::SdfPath{"/mysphere"});
// Camera and its XForm
auto cameraLayer = pxr::SdfLayer::CreateAnonymous("CameraLayer");
auto cameraStage = pxr::UsdStage::Open(cameraLayer);
auto cameraXFormPrim = cameraStage->DefinePrim(pxr::SdfPath{"/Camera"}, pxr::UsdGeomTokens->Xform);
auto camera = pxr::UsdGeomCamera::Define(cameraStage, pxr::SdfPath{"/Camera/Camera"});
auto camXform = pxr::UsdGeomXformable{cameraXFormPrim};
pxr::UsdGeomXformCommonAPI(camXform).SetScale(pxr::GfVec3f{1.0f});
pxr::UsdGeomXformCommonAPI(camXform).SetRotate(pxr::GfVec3f{0.0f});
pxr::UsdGeomXformCommonAPI(camXform).SetTranslate(pxr::GfVec3f{0.0f, 0.0f, 10.0f});
// Sphere XForm override layer
auto transformLayer = pxr::SdfLayer::CreateAnonymous("TransformLayer");
auto transformStage = pxr::UsdStage::Open(transformLayer);
auto opNameToken = pxr::TfToken{"MyTransform"};
auto xFormPrim = transformStage->OverridePrim(pxr::SdfPath("/mysphere"));
auto refXform = pxr::UsdGeomXformable{xFormPrim};
pxr::UsdGeomXformOp scaleXform = refXform.AddXformOp(pxr::UsdGeomXformOp::TypeScale, pxr::UsdGeomXformOp::PrecisionFloat, opNameToken);
scaleXform.Set(pxr::GfVec3f{0.2f}, pxr::UsdTimeCode::Default());
rootLayer->InsertSubLayerPath(sphereLayer->GetIdentifier());
rootLayer->InsertSubLayerPath(cameraLayer->GetIdentifier());
// Render the stage
renderEngine->render(rootStage);
// Another layer added to sublayers stack
rootLayer->InsertSubLayerPath(transformLayer->GetIdentifier());
// Render the stage again, all prims are removed and then repopulated
renderEngine->render(rootStage);
It seems I’m missing something very obvious because adding a layer to subLayers stack probably shouldn’t cause total prim cache repopulation? Or if it is by design, what would be a better approach in this kind of situation where we want to modify the subLayers stack of a stage but not struggle with the slowness it causes? Most problematic for us the shader recompilation that happens due to material prims getting recreated. What is most strange is that even prims that have no relation whatsoever to the changes new layer introduces are also removed.
Only workaround we have found at the moment is to keep a set of persistent layers in the subLayers stack and transfer the content from incoming layers stack into it (using either TransferContent or SdfCopySpec), but it makes it suboptimal due to constant copying and gets messy when layer count changes.