USD Asset Resolver - Practical Example/Reference Implementations

Thanks for the tip @spiff , we need to check with SideFX how they handle resolve scoped caches.

Here is a video of the v0.7.0 release with:

  • export (linux)/set(windows) TF_DEBUG=CACHEDRESOLVER_RESOLVER_CONTEXT
  • Python logging enabled in the log_function_args function in PythonExpose.py

Compared to the v0.6.6 release, this fixed the double context initialization. The problem that still remains is what you are describing: Each reference node with a new path is triggering a new context creation, because it passes in the reference path as the context path (in the Resolver::_CreateDefaultContextForAsset method). The cached resolvers caching (as indicated by "Reusing context on different stage
" shows that the re-using mechanism as coded here: VFX-UsdAssetResolver/src/CachedResolver/resolver.cpp at b6714ee45e15ca7804176b404fd4911c157e01e6 · LucaScheller/VFX-UsdAssetResolver · GitHub is working. I think context re-using should be done by the host application though.)

I am not sure why SideFX decided to implement it that way, I’ll write a ticket. If you create it via a sublayer node or directly via python, you can see it works as expected: The default context is re-used and only initialized once.

I’m going to paste the sidefx RFE I created and their response in its entirety as it’s quite enlightening:

We are testing a custom asset resolver and are getting very different behaviour between using the pxr.Ar library to resolve identifiers and letting Solaris use the same resolver via Reference nodes in LOPs.

To test, I have removed all our custom tooling besides the asset resolver and the core db library needed to query for resolutions.
For the sake of experimentation, I am caching all valid identifier:path pairs (some 15,000) in the context during the creation of the ResolverContext object. This takes about 7 seconds.

As I understand the Ar interface, this ResolverContext should get initialized just once, unless otherwise manually refreshed. This matches the behaviour I see if I use the Resolver via pxr.Ar.GetResolver(). I can resolve as many paths as I like and the ResolverContext object is re-used for every Resolve, giving very fast performance.

However if I create a Reference LOPs node in Solaris, every time I change the file pattern field, I can see via logging that a new ResolverContext is created and initialized.

This obviously renders the caching I’m trying to do useless, as the same ResolverContext object (where the cache lives) is not being re-used for each Resolve.

One thing I have noticed is that if I switch back to a file pattern value that has been used before in Solaris then the Resolved path is produced immediately without a new ResolverContext being created. The same does not apply if a path has been resolved via pxr.Ar.GetResolver() and is then used again in Solaris - in that case a new ResolverContext is created. This perhaps suggests that Solaris is caching the values used on its nodes and re-using the ResolverContexts per path? I can only guess at that though.

Can you give us some guidance here? Is this expected behaviour in Solaris? Why is a new ResolverContext object created for each Resolve? Are we misunderstanding something about the Ar Interface? Are we using it differently to other users who have custom Asset Resolvers?

I have checked with the performance monitor and we are not seeing any unexpected cooking of the node. The node cooks once per edit of the file pattern field, whether we are providing a new or previously used file pattern. There are no callbacks.

Response:

Resolver contexts are created per-stage in Houdini. This is a feature of USD that we expose so that you can have LOP networks containing different USD stages, each using different resolver contexts. This means that Houdini has to create a resolver context every time it creates a brand new stage. If you have a LOP network with one node in it, that node creates a USD stage when it cooks. If you change a parm on that node, it throws away the stage it created last time it cooked, and creates a new stage for its next cook. I presume this is why you are seeing a new resolver context being created every time you change the file pattern on the reference LOP.

If you have two LOP nodes in a chain, the first node in the chain creates the stage, and the second node (generally) modifies that same stage. So changing parms on the second node won’t create a new stage, so it won’t require creating a new resolver context. Even if this first node is a NULL, followed by a Reference node, changes to the reference node shouldn’t create a new resolver context because it is reusing the stage created by the Null LOP (which doesn’t have to recook).

Reusing resolver contexts when you return to a previously used file pattern I don’t really have any insight into. Solaris isn’t doing anything to cache these resolver contexts. Perhaps the USD library is doing something like this. You don’t say how you’re configuring your resolver context (using the referenced file path as the “resolver context asset path” I’m guessing?), but Solaris does nothing to interfere with the normal operation of the USD/Ar libraries or file resolving mechanism. We just have to expose the controls in a unique way because we allow for so many simultaneous (completely independent) stages to exist in a single Houdini session.

I have updated our build to 0.7.0 and although I’m no longer seeing the ResolverContext.Initialize() running multiple times per-resolve as I was before, I’m still seeing it run each time I enter an as yet unused new path (but whic has already cached in the ResolverContext).

If I copy & paste a reference node I get as you get - the ResolverContext is reused, but if I set a new asset path, a new context is created.

I’m running on Windows - any chance you or someone else is able to validate your changes on your end on Windows? I am seeing logging of

Something I don’t exactly follow from their explanation, is that if I (via pxr.Usd) create a multiple stages in memory, it doesn’t re-initialize the Asset Resolver per-stage, so I’m not sure that their response gives the full picture.


Edit as I can’t add another reply:

Here’s some logging that might help explain what’s going on - perhaps I’m missing something in my integration?

This is everything that gets logged after changing the reference identifier, until the ResolverContext.Initialize() gets called. I’m noting that the at the point it’s creating the new context, it’s already has the resolved path, and is then calling CreateDefaultContextForAsset. Is there a Context created per unique identifier?

If you’d prefer to split this off into a new thread to keep this thread more on track I don’t mind doing that. I appreciate any help!

Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_CreateDefaultContextForAsset('<resolved_asset_path_v001.usd>')
Resolver::_IsContextDependentPath()
Resolver::_CreateDefaultContextForAsset('<resolved_asset_path_v001.usd>') - Constructing new context
ResolverContext::ResolverContext('<resolved_asset_path_v001.usd>') - Creating new context
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
ResolverContext::Initialize()
2024-05-15 19:14:36,371 - INFO - PythonExpose(99) - ::: ResolverContext.Initialize