Hi,
it seems that 24.05 UsdUtils resolve UDIM paths to individual files. This hits the servers with thousands of file calls.
here are two runs on one of our production USD files in two versions of USD (houdini 20.0 vs 20.5)
this is a single call to UsdUtils.ExtractExternalReferences(path)
usd version: (0, 22, 8)
number of paths: 42
INFO:fin.core.debug:TIMEIT. func:'resolve' in '__main__' took: 0.0500 sec
usd version: (0, 24, 5)
number of paths: 332
INFO:fin.core.debug:TIMEIT. func:'resolve' in '__main__' took: 27.0686 sec
this is just a single isolated call. when the render farm hits the servers with this call our pipeline simply stops as the servers cannot cope with the number of file queries.
other functionality of UsdUtils like UsdUtils.ModifyAssetPaths is similarly affected
is there any way to disable it? an environment variable to get to the old functionality? I think udim resolution should be up to the client code to resolve if needed.
I have a feeling you are right about this, but it probably needs a bit of discussion.
Unless there is a way to disable it, it might warrant a programmatic switch to opt-in or opt-out if it’s not there already… If there is no existing option it could warrant opening an issue at the repo.
This code is relied upon for packaging up assets into USDZ packages, so it needs to be able to find all of the UDIM tiles, as they are all dependencies. There is a “userProcessingFunc” configuration point, but it gets called after we have already located the UDIM tiles (at, as you note, considerable expense, since tiles could exist anywhere on your “searchpath”).
Something I think we probably should be doing that we are not currently is opening a ArResolverScopedCache inside the top-level dependency API. If you were to create one yourself (I believe as a context object in Python) before calling the dependency code, that may help some, but to really make a difference you’d likely need to derive your own resolver from ArDefaultResolver that implements a “directory caching ArResolverScopedCache” implementation, which is something we do internally with our own searchpath-based resolver.
We’ll use your GitHub Issue to discuss how we can make the dependency code more configurable in this respect… resolving UDIMs yourself is tricky because of “anchored paths” (i.e. those that are relative to the layer that contains the reference) which is why we take on the burden of resolving the paths before we call user-code.