Sorry for the delay in replying.
Thanks Luca, was lovely to see everyone at the meeting
What is already possible now is doing what ColinE suggested in the above post: When your pipeline already knows most paths upfront it can pre-cache these and store them (in its asset resolver context).
Yeah - this was something many people have said they have had to implement. As you and @sunya mentioned - regardless of any Pcp-level batching possibilities, sounds like you’d still end up with a myriad of different requests for each nested layer anyway. So as you say - it may not save that much vs a single up-front lookup. I did wonder though if it may still result in several orders of magnitude fewer requests if asset structure is amenable, but maybe it’s not valuable enough all told.
I didn’t get around to it in the last weeks, but I’ll add another resolver (a Hybrid/Cached Resolver) soon to my UsdAssetResolver repo here…
Your repo is excellent - many thanks for all the work there! Be great to team up on this if you have any bandwidth. People have expressed a lot of interest in having common implementations of all this. The TSC has setup the OpenAssetIO roadmap to add the many of these things to the core API to save each site having to implement the same mechanisms… (multi-language, re-usable/distributable read-through cache). Be a shame to be working on the same thing in parallel.
One idea I had was to have an ArResolver implementation where a call to Resolve would record the asset path and then start a timer and block. Additional calls to Resolve from other threads would also record those paths and block…
Quite a few studios have told us they’re doing this to make things practically scalable, it’s certainly something we’re looking at having available as a mix-in in the core OpenAssetIO library too.
This sounds kind of similar to what @LucaScheller and ColinE mentioned – taking advantage of external knowledge to guide a caching scheme.
We’d be really interested in learning what re-usable parts we can make to help simplify getting this working against pipeline-specific logic.
Futures/promises is something I’d like to see in USD resolution in the future. I’m happy to put up a formal proposal.
@dhruvgovil would love to chat to you more on this outside of the USD context.
Thanks again for all the input, very much appreciated.