SDF_FORMAT_ARGS and Custom AR_Resolvers

Context

Hi, I’m currently working on a Usd_Resolver for (AYON) [Resolver](GitHub - input/ayon-usd-resolver: Home of AYON USD resolver plugin) and we recently implemented a system by @BigRoyNL so that we can nicely write out usd files into the pipeline system.

But this system writes out the SDF_FORMAT_ARGS with the URI we construct

Uri looks like this for example:

ayon://Usd_Base//TestD/asset?product=usdAsset_model&version=latest&representation=usd:SDF_FORMAT_ARGS:layer_id=model&order=100

aparently they are important for the saving system to know what type of file it is in case of .usd [houdini_post_about_SDF_FORMAT](SDF_FORMAT_ARGS for binary vs. ascii usds | Forums | SideFX)

and there is a function to “just” split the data [Sdf_SplitIdentifier](OpenUSD/pxr/usd/sdf/assetPathResolver.cpp at 2864f3d04f396432f22ec5d6928fc37d34bb4c90 · PixarAnimationStudios/OpenUSD · GitHub) into the path and the args.

also i did not see a resolver that used SplitIdentifier in order to remove the args.

[LucaScheller](VFX-UsdAssetResolver/src/FileResolver/resolver.cpp at f5e1ba325e1847654c8bed241e94da49803f9352 · LucaScheller/VFX-UsdAssetResolver · GitHub)
[WD-S3](usd-s3-resolver/S3Resolver/resolver.cpp at 7730cacc07e0c7088e02c02292feef9125b0a154 · westerndigitalcorporation/usd-s3-resolver · GitHub)

Question

It would be interesting to know if it is reasonable to simply “remove” the data from the assetPath or if I should handle it differently or even take it into account.

Any input would be interesting because most of the time, there is a reason that data is given to the resolver, so I assume there’s something I should do with it here.

Manny, thanks for reading.
If you read it on creation day, have a great weekend. If not, just have a great Day

Hi @Root42 , no ArResolver should need to worry about SDF_FORMAT_ARGS . In the normal course of operation of all shipping OpenUsd asset-processing code, which includes stage composition/population, and UsdUtils utilities for dependency analysis and asset localization and packaging, the client of Ar strips the Sdf arguments from the string before handing it off to the ArResolver to resolve.

Any script or code that wants to perform “raw” processing of a layer’s assetPaths using Sdf API’s is on the hook to do so as well. It feels like we should document this strongly somewhere, though I’m unsure exactly where - open to suggestions!

Hi @spiff thanks for the answers.

open to suggestions!

i guess the first place i would look is DevGuide and then Spec (Maybe a section that describes how loading and writing works in a general ?) i assume the later is intended to be a bit like the c++ ISO standard description what everything should be and how it should work. (im guessing because that not what its currently doing) and then i would maybe even link it into AR just for good measure.

but take it with a grain of salt as i encountered it the first time while working with @BigRoyNL sooo maybe other people would look elsewhere.

also you mentioned that it should not reach the resolver if i understand you right

no ArResolver should need to worry about SDF_FORMAT_ARGS

but when i run the resolver within SideFs Houdini 20.0.710 using a tool developed by @BigRoyNL and/or the reference node (its a bit of a cheap test i know) then im greeted with the following

ps: getAsset() is a custom function that caches server communication just to avoid confusion.

Resolver::_Resolve('ayon://Usd_Base//SDF_Format_Args_Test/char_sph?product=usdAsset_model&version=latest&representation=usd:SDF_FORMAT_ARGS:layer_id=model&order=100') 
Resolver::_IsAyonPath (ayon://Usd_Base//SDF_Format_Args_Test/char_sph?product=usdAsset_model&version=latest&representation=usd:SDF_FORMAT_ARGS:layer_id=model&order=100) 
Resolver::_IsAyonPath true 
resolverContextCache::getAsset: (ayon://Usd_Base//SDF_Format_Args_Test/char_sph?product=usdAsset_model&version=latest&representation=usd:SDF_FORMAT_ARGS:layer_id=model&order=100)

ps: i just did a test using “Raw” usd23.05 and got this (the test is just a stage that loads a payload and the payload then has the uri in it, same as in Houdini)

Resolver::_Resolve('ayon://Usd_Base//SDF_Format_Args_Test/char_sph?product=usdAsset_model&version=latest&representation=usd:SDF_FORMAT_ARGS:layer_id=model&order=100') 
Resolver::_IsAyonPath (ayon://Usd_Base//SDF_Format_Args_Test/char_sph?product=usdAsset_model&version=latest&representation=usd:SDF_FORMAT_ARGS:layer_id=model&order=100) 
Resolver::_IsAyonPath true 
resolverContextCache::getAsset: (ayon://Usd_Base//SDF_Format_Args_Test/char_sph?product=usdAsset_model&version=latest&representation=usd:SDF_FORMAT_ARGS:layer_id=model&order=100)

if i understand you right the SDF_FORMAT_ARGS should not reach the _Resolve() method or did you mean something else by ArResolver should need to worry and im just miss interpreting ?

To make sure Houdini isn’t doing anything wrong here, could I get more information about this tool you’re referring to?

Houdini uses :SDF_FORMAT_ARGS: extensively when importing data from SOPs and BGEO files into USD, and I haven’t heard any reports of these args making it into resolvers, so I’m hoping that this is a bug in this “tool” you’re referring to…

Thanks,
Mark

hello mtucker.
the upper example is both the normal reference node the “tool” is just a wrapper around the same node (just pices together the Uri)

also: the second example the one with the exact same output is just “raw” usd

im gonna append an Zip with the specific file used for the example. this will also have the python script used for Raw usd testing (the path in it is a “fake” path i used an absolute path but in order to put it all into the same zip i did it as a relative path not sure if the relative path is correct tho),

i can ship the resolver if you want but as it needs the AYON-Environment it will probably not work for you.
if you have a resolver that has the print outs in the functions would be cool if you could just see what happens for you.

Hello there again.
i realized i forgot to append the zip in the other message so im gonna try to build a very small reproducible example here.

Testing notes

so my test is constructed from 3 parts.
A python script that just runs Usd.Stage.Open on the test_entry.usd file.
the test_entry.usd file is just the file that uses the payload.usd
and the payload.usd finnaly then sublayers the “paths” or Uris

here are some things i tested inside the payload.usd:

  1. test.usd:SDF_FORMAT_ARGS:order=100 - dosnt strips off SDF_FORMAT_ARGS
  2. ./test.usd:SDF_FORMAT_ARGS:order=100 - strips off SDF_FORMAT_ARGS
  3. /path/to/test.usd:SDF_FORMAT_ARGS:order=100 - strips off SDF_FORMAT_ARGS
  4. C:/path/to/test.usd:SDF_FORMAT_ARGS:order=100 - dosnt strips off SDF_FORMAT_ARGS (under Linux did not test with windows, i assume under windows this is an absolute path)
  5. ayon://path/to/test.usd:SDF_FORMAT_ARGS:order=100 - dosnt strips off SDF_FORMAT_ARGS

usds.zip (934 Bytes)

here are some observations i made

every time a “wrong” relative path is created. The SDF_FORMAT_ARGS are not removed eg ./file.usd will have them removed but file.usd will not have them removed.
im not sure if that’s wanted or not.

i also want to mention that the example paths i showed atop dont work if i just put them into Usd.Stage.Open(), (i belie i got it to not remove the ARGS once but i have no record of this and cant reproduce it so i will chuck it up to probably just ran the wrong test)

also i just want to mention. the resolver we have is not registered as a URI resolver normally. i however did a test where i ran the test with the resolver resisted with "uriSchemes": ["ayon"] enabled and came to the same output

thanks for reading and especially your time with this issue.

Contens of the files in the .zip

payload.usd

#usda 1.0
(
    defaultPrim = "char_sph"
    subLayers = [
        @ayon://path/to/test.usd:SDF_FORMAT_ARGS:order=100@
    ]
)

test_entry.usd

#usda 1.0
(
    defaultPrim = "char_sph"
)
def Xform "char_sph" (
    prepend payload = @./payload.usd@
)
{
}