Import usdc/usdz from memory

[mwestphal]

/Hi,

Im a programmer working on a reader of usd files, I’m able to read from disk without issues but I want to be able to read from memory.

I’m able to read .usda file from memory alrightm using sdfLayer::ImportFromString, but usdc/usdz proves more challenging.

There is dedicated ReadFromString methods in usdc and usdz fileformats but they do not seem to be usable out of the box ?

Here is how it looks so far:

std::istream stream(streambuf.get());std::string content(std::istreambuf_iterator(stream), {});
auto layer = pxr::SdfLayer::CreateAnonymous(".usda");    // OK
// auto layer = pxr::SdfLayer::CreateAnonymous(".usdc"); // KO
// auto layer = pxr::SdfLayer::CreateAnonymous(".usdc"); // KO
layer->ImportFromString(content);
this->Stage = pxr::UsdStage::Open(layer);

With “.usdc" I get:

parse error matching PXR_INTERNAL_NS_pegtl::ascii::one<‘#’> at ‘PXR-USDC’ in </>

With ".usdz” I get:

Cannot create anonymous layer: creating package usdz layer is not allowed through this API.

Am I using the right API ? Is there another API to read such file from memory ?

[mwestphal]

(edited as I make progress)

[mwestphal]

I’ve found ArInMemoryAsset but I’m unable to create a file usdc/usdz layer and stage from it, it doesn’t seem be the right level of API.

[dhruvgovil]

I don’t believe there’s a way to instantiate a package completely in memory, without a file format plugin that is.

[mwestphal]

Thanks for the feedback!

instantiate a package

You are refering to .usdz right ? What about usdc ?

a file format plugin that is.

I’ve looked around and found plugin for other formats, but nothing for reading from memory.
I suppose you mean implementing my own file format plugin ?

If that’s the answer then I’m a bit confused by the presence of this method in usd:
SdfUsdzFileFormat::ReadFromString

[dhruvgovil]

Looking at that method, the usdz version of it assumes the format is a usda. I think it might be an oversight but I don’t see any methods for reading from an in memory buffer built into the file formats.

There is however an ArInMemoryAsset that could be used to create an in memory buffer and then read as a layer but I don’t see a straightforward way to do it via the currently exposed api.

[mwestphal]

@dhruvgovil

but I don’t see any methods for reading from an in memory buffer built into the file formats.

Yeah, that seems to be my understanding as well

ArInMemoryAsset that could be used to create an in memory buffer and then read as a layer but I don’t see a straightforward way to do it via the currently exposed api.

I tried it too and I’m able to create an “asset” from memory but I coulnd’t find a way to create a layer from that asset indeed.

TBH thats is quite surprising to me as it feels like reading a file from memory would a very important usecase for USD, but maybe I’m wrong.

[dhruvgovil]

You can create a layer in memory, which covers most people’s in memory needs. Loading from a buffer is surprisingly rare as a need. Though I agree it would be nice to have

Maybe after the holidays, the Pixar folks can chime in and see if I’m missing anything though.

[spiff (F Sebastian Grassia)]

Like @dhruvgovil said, in our (and most external users, it seems?) workflows, the ability to create, write-to, and “Export” an in-memory (called “anonymous”) layer has covered most bases since the creation of Presto twenty+ years ago.

Firstly, what you need to do if you want to pursue this (and then why) : The missing piece is a custom “URI” ArResolver which, given an identifier (which it gets from SdfLayer::CreateNew()), will create the ArInMemoryAsset you need and attach it to a concrete Layer (more specifically, the underlying instance of an SdfAbstractData container associated with the SdfFIleFormat you have indicated in the extension of the identifier). So, if you choose a URI class/protocol like “inmem”, then you could create layers like SdfLayer::CreateNew("inmem:aBufferLayer.usdc")

OK, that’s a little misleading, because that’s the simpler "Creating a new empty layer that you could then use the normal SdfLayer API’s to populate, and the resolver would actually be creating an ArWritableAsset. But your use case, I believe, is to create a Layer for reading, from a sequential buffer you already have in memory. For that to work, you need to encode in the identifier (or supply via SdfFileFormatArguments in your calls to SdfLayer::FindOrOpen()) enough information so that your resolver can find the buffer and use it as the source for the ArInMemoryAsset it creates. The simple, non-robust way to do that would be to encode a pointer as text. Blech. Better would be to add public API to your resolver that is available to your application where it can register buffers (maybe as unique_ptr’s?) associated with identifiers, so that when SdfLayer comes calling, it can find the appropriate buffer, and this also gives the resolver an opportunity to manage those buffers.

And now we get to why we’re putting you through all that rigamarole… SdfLayer actually has a number of facilities and public API that allow reasoning (timestamps, updating identifiers, determining if backing asset has changed and/or needs to be reloaded, etc) that fall into the category of “asset management”. All of that is mediated through the ArResolver abstraction, which is why we force you to go through that abstraction if you want to introduce any new kind of datasource - unless you’re willing to create a complete new FileFormat, which I think goes counter to your needs.

Dhruv mentioned InMemory/Anonymous layers as an existing mechanism that is “close”, and that does not require its own ArResolver. My answer is: it should!. But anonymous layers were deployed twenty years ago long before USD was a twinkle, and if you look at sdf/layer.cpp, you’ll see there a special cases for them polluting the code in numerous places. I think if we had the new “Ar 2.0” interface (designed and deployed around seven years ago to address community needs like URI resolvers and varied asset datasources) in place, then we would have worked hard to deploy anonymous layers as an “anon:” URI resolver, with maybe just the lightest sugar remaining for convenience, like SdfLayer::CreateAnonymous that simply prepends the URI class to the provided identifier.

I may have fudged some details in there, corrections welcome, and let us know if you have any further questions.

[mwestphal]

I wont pretend I understand half of this. I will need some time to take it all in and try to experiment.
In any case that looks like a much more advanced usage of OpenUSD compared to what we have been doing, which is basically reading a usd file from disk and converting it to our own memory representation of the data.

Thanks for all the feedback and for confirming that this is not supported out of the box and require more work.

[spiff (F Sebastian Grassia)]

Acknowledged, @mwestphal ; though, could you help me understand the need for streaming, here?

@mwestphal

what we have been doing, which is basically reading a usd file from disk and converting it to our own memory representation of the data.

This is pretty much what everybody is doing, typically by using the provided API’s to traverse prims on a stage, extracting and converting data as fits your system/encoding. Where exactly does the need to stream the USD layer contents come in?

Btw, I had an idea over the weekend that possibly could accommodate your need with only a small extension of the existing architecture. Need to talk it over with the team, and will post back here if it bears any fruit.

[mwestphal]

@spiff

This is pretty much what everybody is doing, typically by using the provided API’s to traverse prims on a stage, extracting and converting data as fits your system/encoding. Where exactly does the need to stream the USD layer contents come in?

What I do is a bit different, I do not construct a stage from my own system, I just read files from disk (or stream) and show them: https://f3d.app

Hence we are converting from a open usd file to our own representation of the data to display it, we do not use OpenUSD API to construct the data, we just use it to open the data.

The need for streaming is because some users do not have a filesystem at all (Android, sandboxed system, whatever) so they cant read from disk but only from an istream or a memory dump containing a usda/c/z file.

[spiff (F Sebastian Grassia)]

Thanks! I’ll just reiterate that the problem of “Accessing data via SdfLayer (or other formalizations like HioImage for textures) API’s from varied and disparate, non-file-backed sources” is exactly the problem ArResolver was designed to address, though granted it adds overhead, and there might be a path to creating (read only, possibly) anonymous layers from ArAssets.

[pmolodo (Paul Molodowitch)]

@spiff

But your use case, I believe, is to create a Layer for reading, from a sequential buffer you already have in memory. For that to work, you need to encode in the identifier (or supply via SdfFileFormatArguments in your calls to SdfLayer::FindOrOpen() ) enough information so that your resolver can find the buffer and use it as the source for the ArInMemoryAsset it creates.

You may also have to enable an experimental env setting, USDC_USE_ASSET (thanks @jomiller-nvidia for recalling the setting), that in theory should allow working with the .usdc format without having it on disk, but at last check, it was buggy / unstable. That was admittedly at least a year ago, though, so perhaps it’s improved since then.

Would be awesome to get non-filesystem-backed .usdc reading working properly, though…

[spiff (F Sebastian Grassia)]

@pmolodo:

1. Usdc’s use of ArAsset should be stable now
2. Turning on that setting is now just a pessimization (disabling direct mmapping when it otherwise could be applied) - usdc should now use ArAssets when supplied one.

[pmolodo (Paul Molodowitch)]

Great, we’ll have to give that a spin again!

[spiff (F Sebastian Grassia)]

We discussed the possibilities and pitfalls of being able to directly instantiate an Anonymous SdfLayer from an ArAsset. We believe we could make this work for most FileFormats, but not for usdz assets, for reasons (and more) I alluded to in the GitHub Issue you filed.

But although you may not be running into it at the moment, this pattern seems vulnerable the moment you want to put such a layer on a stage. Under the assumption that under this constraint (no filesystem access), all of the layers in a composition would need to be handled this way, you’d be faced with the prospect of needing to modify each layer’s embedded asset paths to identify the anonymous layers you created for the other layers, and of course be tracking all of those dependencies and mappings yourself. THis is because as the composition engine comes across a reference to another asset, it will resolve it through ArResolver to find the layer to open and add to the composition.

That’s a key reason why the “natural” solution to this problem is to deploy an ArResolver that can be populated with data streams and identifier mappings to those streams. Then you can put any such layer on a Stage without modification, and the resolver keeps track of all the layer mappings, and all the downstream clients of the system “just work”. And it would enable usdz packages to work also, I think (there might be some work needed to the UsdzPackageResolver which we’d be happy to look into if so.)