Is there a way to reload a Sdf layer robustly?

Community USD
1

Howdy!

I have a test.usda file that sublayers in some other USD files layer1.usda, layer2.usda, layer3.usda, etc. These layerx.usda files are always exported by the USD Python API, and all the contents are correct. When I do Usd.Stage.Open("test.usda") to load the stage, all the layers are correctly composed into a single stage and I can access every prim from the stage. After that, some of these layer files are updated by other applications, and I wanted to use layer.Reload() to reload only the layers that have been changed instead of reloading the entire stage, unfortunately that doesn’t seem to work well, it works for some layers, but not for others. For example, I have a layer file exported from Houdini which always throws the exception below when I reload it, I can’t even use Sdf.Layer.FindOrOpen() or Usd.Stage.Open() to load it separately, but Usd.Stage.Open("test.usda") does pull in every single prim from it, which is weird.

pxr.Tf.ErrorException: 
	Error in 'textFileFormatYyerror' at line 3116 in file pxr/usd/sdf/textFileFormat.yy : 'syntax error at '' in </somePrim.primvars:st:indices> on line 268 in file /path/to/layer3.usda
'

There’s nothing particularly wrong on line 268, but it always fails on a line that starts with a primvar like primvars:st or primvars:st:indices. Also, for some layers I was getting a MemoryError from Python, not sure what happened there.

It seems to me that the text parser is quite restrictive and Reload() only works for very simple USD files like a light rig, so I have to use a different approach. Since the composition engine seems to be able to compose these layers without issue, I tried to make a copy of the layer’s USD file (to avoid conflicting layer identifiers), loaded it as a brand new Sdf layer, and then compose it on top of the layer stack:

shutil.copy("layer3.usda", "layer3_copy.usda")
newLayer = Sdf.Layer.FindOrOpen("layer3_copy.usda")
stage.GetSessionLayer().subLayerPaths.insert(0, newLayer.identifier)

But again, this doesn’t work, I was getting the same errors as before. And for layers that do work well with this approach, I’ve noticed that USD cannot send change notices properly after that so the renderer was not updating to reflect the changes, but that might be a bug in the rendering code which I haven’t investigated much.

As a last resort, I’ve given up on reloading a single layer so I decided to just reload the entire stage by calling stage.Reload(), and that immediately gives me a SIGSEV because the layer files have been changed externally.

I apologize for not having a simple reproducible case, the layer USD files are really huge so I can’t really post them here. Just as a general question though, what is the recommended way to reload an arbitrary layer without crashing the program? I don’t want to clear the entire stage from memory and load it again as if it was opened for the first time.

2

As far as I’m aware Sdf.Layer.Reload() is intended to be able to reload a layer correct and notify other areas, like the stage, to recompose accordingly.

Depending on your case you might need to Sdf.Layer.Reload(force=True)?

Could you provide your OS, USD build and Python version. I assume you’re testing this code standalone with solely the Python API? Or are you triggering the reload in an application like Maya - which itself might also be doing something with the stage/layers via notifications.

It’d be great if you could provide a single USD file that has e.g. erroneous primvars:st:indices and primvars:st just so we also have something to look at to investigate.

1 Like
3

Are you sure you’re not reloading it WHILE it’s being written?

One way to avoid that issue is to write a temp file and then copy it over the destination file once it’s completely written to disk.

4

SdfLayer.Reload() is quite robust and general, however you must be quite sure that another application is not writing or saving to the layer’s file when you reload, and that can require synchronization.

Also, don’t be afraid to simply call stage.Reload(). SdfLayer::Reload() will only actually reload the layer if it’s backing store has changed:It uses the ArResolver interface to compare each layer’s modification time to that of when it was opened or last saved/reloaded by the process. Since UsdStage::Reload just calls Reload on all its layers, it will sparsely recompose the minimal amount based on what has changed.

3 Likes
5

Thanks everyone for the info! It’s good to know that Sdf.Layer.Reload() is actually quite robust. As you said, it might be something else in my application that is interfering with the stage with notifications.

I did some testing and found that my code does work when running in a fresh Python context, but the same code does not work when running natively inside my application, this is quite complicated so I might need more time to figure out why. The fault is likely on my side, if I can find a simple reproduce I will post updates here.