Python logging enabled in the log_function_args function in PythonExpose.py
Compared to the v0.6.6 release, this fixed the double context initialization. The problem that still remains is what you are describing: Each reference node with a new path is triggering a new context creation, because it passes in the reference path as the context path (in the Resolver::_CreateDefaultContextForAsset method). The cached resolvers caching (as indicated by "Reusing context on different stage
" shows that the re-using mechanism as coded here: VFX-UsdAssetResolver/src/CachedResolver/resolver.cpp at b6714ee45e15ca7804176b404fd4911c157e01e6 ¡ LucaScheller/VFX-UsdAssetResolver ¡ GitHub is working. I think context re-using should be done by the host application though.)
I am not sure why SideFX decided to implement it that way, Iâll write a ticket. If you create it via a sublayer node or directly via python, you can see it works as expected: The default context is re-used and only initialized once.
Iâm going to paste the sidefx RFE I created and their response in its entirety as itâs quite enlightening:
We are testing a custom asset resolver and are getting very different behaviour between using the pxr.Ar library to resolve identifiers and letting Solaris use the same resolver via Reference nodes in LOPs.
To test, I have removed all our custom tooling besides the asset resolver and the core db library needed to query for resolutions.
For the sake of experimentation, I am caching all valid identifier:path pairs (some 15,000) in the context during the creation of the ResolverContext object. This takes about 7 seconds.
As I understand the Ar interface, this ResolverContext should get initialized just once, unless otherwise manually refreshed. This matches the behaviour I see if I use the Resolver via pxr.Ar.GetResolver(). I can resolve as many paths as I like and the ResolverContext object is re-used for every Resolve, giving very fast performance.
However if I create a Reference LOPs node in Solaris, every time I change the file pattern field, I can see via logging that a new ResolverContext is created and initialized.
This obviously renders the caching Iâm trying to do useless, as the same ResolverContext object (where the cache lives) is not being re-used for each Resolve.
One thing I have noticed is that if I switch back to a file pattern value that has been used before in Solaris then the Resolved path is produced immediately without a new ResolverContext being created. The same does not apply if a path has been resolved via pxr.Ar.GetResolver() and is then used again in Solaris - in that case a new ResolverContext is created. This perhaps suggests that Solaris is caching the values used on its nodes and re-using the ResolverContexts per path? I can only guess at that though.
Can you give us some guidance here? Is this expected behaviour in Solaris? Why is a new ResolverContext object created for each Resolve? Are we misunderstanding something about the Ar Interface? Are we using it differently to other users who have custom Asset Resolvers?
I have checked with the performance monitor and we are not seeing any unexpected cooking of the node. The node cooks once per edit of the file pattern field, whether we are providing a new or previously used file pattern. There are no callbacks.
Response:
Resolver contexts are created per-stage in Houdini. This is a feature of USD that we expose so that you can have LOP networks containing different USD stages, each using different resolver contexts. This means that Houdini has to create a resolver context every time it creates a brand new stage. If you have a LOP network with one node in it, that node creates a USD stage when it cooks. If you change a parm on that node, it throws away the stage it created last time it cooked, and creates a new stage for its next cook. I presume this is why you are seeing a new resolver context being created every time you change the file pattern on the reference LOP.
If you have two LOP nodes in a chain, the first node in the chain creates the stage, and the second node (generally) modifies that same stage. So changing parms on the second node wonât create a new stage, so it wonât require creating a new resolver context. Even if this first node is a NULL, followed by a Reference node, changes to the reference node shouldnât create a new resolver context because it is reusing the stage created by the Null LOP (which doesnât have to recook).
Reusing resolver contexts when you return to a previously used file pattern I donât really have any insight into. Solaris isnât doing anything to cache these resolver contexts. Perhaps the USD library is doing something like this. You donât say how youâre configuring your resolver context (using the referenced file path as the âresolver context asset pathâ Iâm guessing?), but Solaris does nothing to interfere with the normal operation of the USD/Ar libraries or file resolving mechanism. We just have to expose the controls in a unique way because we allow for so many simultaneous (completely independent) stages to exist in a single Houdini session.
I have updated our build to 0.7.0 and although Iâm no longer seeing the ResolverContext.Initialize() running multiple times per-resolve as I was before, Iâm still seeing it run each time I enter an as yet unused new path (but whic has already cached in the ResolverContext).
If I copy & paste a reference node I get as you get - the ResolverContext is reused, but if I set a new asset path, a new context is created.
Iâm running on Windows - any chance you or someone else is able to validate your changes on your end on Windows? I am seeing logging of
Something I donât exactly follow from their explanation, is that if I (via pxr.Usd) create a multiple stages in memory, it doesnât re-initialize the Asset Resolver per-stage, so Iâm not sure that their response gives the full picture.
Edit as I canât add another reply:
Hereâs some logging that might help explain whatâs going on - perhaps Iâm missing something in my integration?
This is everything that gets logged after changing the reference identifier, until the ResolverContext.Initialize() gets called. Iâm noting that the at the point itâs creating the new context, itâs already has the resolved path, and is then calling CreateDefaultContextForAsset. Is there a Context created per unique identifier?
If youâd prefer to split this off into a new thread to keep this thread more on track I donât mind doing that. I appreciate any help!
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
Resolver::_CreateDefaultContextForAsset('<resolved_asset_path_v001.usd>')
Resolver::_IsContextDependentPath()
Resolver::_CreateDefaultContextForAsset('<resolved_asset_path_v001.usd>') - Constructing new context
ResolverContext::ResolverContext('<resolved_asset_path_v001.usd>') - Creating new context
Resolver::_IsContextDependentPath()
Resolver::_IsContextDependentPath()
ResolverContext::Initialize()
2024-05-15 19:14:36,371 - INFO - PythonExpose(99) - ::: ResolverContext.Initialize
I asked SideFX the following (Q&A are shortened/summarized):
Questions:
Does Houdini use scoped caches per node or stage?
When opening a file via the LOPs reference node, it seems to create a new context (by calling Resolver::_CreateDefaultContextForAsset). It seems to do this every time the node is cooked, we were wondering if there is a reason for doing this? When loading a reference/payload/sublayer directly via python, it doesnât do this (also not when using a sublayer node).
Does Houdini manage contexts? So when multiple stages point to the same context, does it re-use/bind the context or does it re-init per stage?
Answers:
No, because LOP node cooking can involve multiple stages, which may have different contexts, and there is no guarantee a series of LOP node cooks would be working with the same stage or the same resolver context. Also: Since scoped resolver caches need to have a well defined lifetime, their usage and life time management in a node based system is hard to define.
The âAsset Referenceâ LOP gets created by default with the âreference primitiveâ set to âautomaticâ. This mode automatically chooses a reference prim for you from the specified file. To do this, it has to create a stage using that file as the root layer. It is when creating this temporary stage (which only exists long enough to see if there is a defaultPrim or pick one of the root prims) that you are likely seeing the resolver context created with the toy.usd as the âcontext assetâ). This stage is almost immediately discarded, and the actual LOP stage is then created.
No, Houdini does not âmanageâ contexts, other than by creating the context attached to each new stage it is instructed to create. While context sharing (across different node networks) would be possible to add, it is something the resolver should take care off. With no additional caching layer for contexts, Houdini isnât introducing any additional (Houdini-specific) overhead or user-dependent management of this cache.
So everything in Houdini (and the CachedResolver) should be working as expected (I think). The answer to 2. explains why we are seeing the âCreateDefaultContextForAssetâ calls. This also explains why it doesnât call that when referencing it in via Python. (Side note: The reason we are seeing the â_IsContextDependentPathâ in between the CreateDefaultContextForAsset calls is that it first resolves the context path and then passes that resolved path to the CreateDefaultContextForAsset method. (So the context path itself is resolved.) )