Reference and Payload customData

I’ve been wanting to store some custom metadata with references and payloads. The Sdf.Reference has Custom Data but Sdf.Payload does not.

Is there a particular reason why this does not exist on payloads?

Why? (Background details)

Some more details. For our automated “asset” publishes we’re working with some sort of contribution system where when someone publishes e.g. a model for an asset they do something like:

  • Add it to the “model” layer of the Asset.
  • (Optional) add it to a variant set in variant, e.g. /asset{model=main}
  • Add it at a specified strength order, so they can force a contribution to be stronger or weaker. E.g. order=100

Then the reference to that model output gets referenced into that prim spec at e.g. /asset{model=main} with that specified order. If other references are found in that prim spec with order metadata then it places it at the correct “index” depending on the other orders. Additionally if the metadata detects that it’s just a new version of the same contribution it removes the older contribution, so you don’t have the main model contribution v001 and v002 in there - but just the one.

As such, I’ve been storing customData with the Sdf.Reference with that identifier and order.

Any design tips to better approach tracking these ‘automated’ builds would be very nice! :slight_smile:

Currently the contributions themselves are all references (even though the individual department layers like model and look, etc. are behind a ./payload.usd that is payloaded in) I was just wondering what the reasoning was that Payloads wouldn’t have custom data applicable to them.

Hi @BigRoyNL ,
This is an inconsistency that we allowed to happen because customData on Sdf.Reference was a feature added in the “original” design of Presto years before USD that we never made use of. Since it adds cost and complexity, when we later designed the (not an original feature) Sdf.Payload structure, and again when we made it list editable circa 2018, we considered adding customData but decided not to.

In terms of alternate suggestions, the only thing I can currently think of is motivated by the one “weakness” of your order approach in that it’s kind of throwing a dart with respect to what other orders may already be in use… so instead, could you formulate ordering in terms of partial-order? So that when someone wants to add a model to an asset, they specify the other model(s) that they definitely want it to be stronger than? Then you can operate on the existing published order and just the model/assetInfo of the referenced/payloaded layers.

Interesting background!

Technically I could - although it’d complicate things a lot. Because to be able to do so I’d need to identify what’s already being contributed at the time the artist starts specifying it for their new publishes. As such, it basically means I need to load the existing layers just when they are manipulating the data.

The reason I liked being able to store the metadata directly with the reference was the fact that if someone were to duplicate a Prim in whatever software they are using, they’d basically be bringing along the custom data and thus our “automated builder” would detect that new entry in the build system as well. If I were to store that data elsewhere, e.g. custom data on the root layer, or sidecar files or whatever I’d need any application manipulating the file to also author this additional metadata. I was just hoping that I could make it so that our ‘build file’ could be authored from any DCC with very little knowledge of the pipeline details except for injecting the metadata at the time of adding new entries - and after that it’s just manipulating prims like in any DCC.

Anyway, I think I’ll need to rethink the strategy I want for this. It might be best to abandon the idea that the USD file itself will be the ground truth for future builds, and may be manipulated in any way and instead just write to the database previous contributions to the USD Asset build process instead of allowing it to be authored by anything anywhere.

If your goal is to “keep the metadata with the Prim/PrimSpec”, you should still be able to achieve that, albeit with a little redundancy… you can use the prim’s customData (which you can author using either UsdPrim or SdfPrimSpec API’s) to record your order metadata, such that each string key in the customData dictionary is the value of the reference asset-path concatenated with the target prim-path (if any), and the mapped value is the order. You might organize the dictionary into prepend, append, delete, and explicit sub-dictionaries since any given asset path could technically show up in more than one of those.