Hey everyone, as an artist creating content for Vision Pro, I’m looking for a reliable USD workflow, so that I can confidently update geometry and materials in a DCC, knowing how they will render on the Vision Pro.
Based on my understanding of USD, this is a fair use case. However, I need help understanding the correct workflow here.
I have had limited success with the following: Maya/Blender(geo) → USD → Reality Converter → USDZ → Reality Composer Pro(Materials) → XCode
When doing more than a simple texture swap in RCP, I must create a new Material to access the shader tree. But then we are locked into RCP for future edits.
Layout in RCP could be better too. Doing layout in a DCC allows us to take advantage of light and occlusion baking, and more mature tools for simply positioning multiple assets.
I’d like to understand this workflow better, where I should do my layout, material editing, and how to structure the USD files. Any guidance from this group would be greatly appreciated!
Hi Campbell,
This might be better answered on the Apple dev forums, but I’ll try and help here as well for the workflow parts. I would recommend filing feedback with for Reality Composer Pro at http://feedbackassistant.apple.com
Both Blender and Maya can export directly to USDZ so you can cut out the Reality Converter part for conversion. You just need to provide a .usdz extension when saving files from them. I recommend being on the latest version of Blender (4.0) or the latest Maya USD plugin because we contribute to both to help compatibility. That would change your workflow to just DCC (Maya/Blender) → Reality Composer Pro → Xcode
Reality Composer Pro works best with .usd , .usda and .usdc files because it can author data back to the USD file in question as needed. While USDZ files will also work, any changes need to be authored as an overriding layer on top of that as USDZ files are read-only.
If you stick with non-usdz (edit: corrected to non-usdz) files, then you can use the same files within your DCCs (if the DCC supports using USD layers) and any changes would work bidirectionally.
Thanks @dhruvgovil, this is super helpful. I will work through some of this, moving to non-usdz files, and report back. I did a quick test with a .usda, but I find that RCP is not importing the materials (though you can see them in the bottom right preview), let alone assigning them. I have attached an image for reference, but I will dig deeper into this, and compare my files with their examples.
Could you clarify your third point here? Did you mean “stick with non-usdz files”?
I see that you’re referencing the file in. Blender has an unfortunate default for root prim in the exporter that breaks material assignments when referencing blender generated usd files in any USD based application.
In the blender export options, make sure you provide a root prim like /root and that will make your blender generated files better.
I’ve provided a fix for this that will be in blender 4.1, but unfortunately wasn’t done in time to be accepted for Blender 4.0
For background, what’s happening is that USD only allows referencing one prim hierarchy from a file. Without that root prim defined, blender puts the materials and meshes in separate prim hierarchies. So when it gets referenced into a USD stage, only one survives.
Another issue you may hit is that your textures don’t come in if your export resolves them to paths outside your rkassets directory. In that case, my recommendation is to copy your textures into your rkassets directory and then use those ones within blender, such that when you export your usda, all the files will be accessible to Reality Composer Pro.
Adding /root did the trick, and I can see in tests how the materials are falling outside of that hierarchy. Thank you!
The textures are working for me, but this I guess brings up my next question. Does the rkassets folder essentially become my working directory? With using Blender to work on the assets, I assume I should be exporting USDs and Textures into there. If this is the case, is there a tool in Blender that gives a similar functionality to Maya’s “USD Layer Editor”?
Simply using the Exporter in Blender, I can make this work by Importing USD from rkassets → making edits → selecting the Prim? (not the root) → Exporting “selected” with /root set again to overwrite the original. It would be great to have the assets simply open in Blender, make the edits, then save updates to the original files.
I have been playing with this a lot this morning. A few things have helped a lot:
Ensuring my USD assets have a proper root
Using USDA/C instead of USDZ
Working directly in the rkassets folder (for edits)
Next steps for me to test out are:
how well Blender can edit the materials (I noticed my Ambient Occlusion textures were dropped)
how to layout a scene in Blender (I opened the RCP scene.usda file and it appeared to come in well, but exporting it again from Blender didn’t work as expected)
Since you’re doing all this - I wonder if some it could be worthwhile if you’d be able to do one of these “simple test runs” as you screen record and maybe talk over it as to what you’re doing, why you’re doing it and what types of questions pop up in your head.
Those types of “videos” can help for:
Newcomers: Just seeing how other people approach things can help them understand where to start.
UX Designers/Developers: Seeing what you have questions about or think about as you go through certain operations can help identify potential design shortcomings towards end-users. Maybe certain labels are wrong? or certain buttons aren’t where you’d expect them to be?
You: It could allow someone to see you doing something in particular and have them say “enable that checkbox and you’re good to go” or “here’s a shortcut”.
Would you be interested in doing some recordings like that?
To your directory , the rkassets doesn’t have to be your working directory, but all your assets must resolve to within it.
So in theory you could work outside of it, as long as you either have paths in your files that resolve there (e.g relative paths) or are putting another layer on top of your content to fix up the pathing.
Having the layer above can be quite handy if you’re working with RealityKit specific data that won’t survive a round trip via Blender. It’s one of the powers of USD. As long as your hierarchy doesn’t change, you can keep updating your mesh.usdc while preserving the other overrides in some other layer.
To the other question about Blender workflows with USD, Blender currently doesn’t have a USD layer editing workflow. It’s another reason why I encourage splitting up the responsibilities of the layers. As long as your hierarchy is consistent, you can isolate and focus each portion of the USD layer stack to each DCC or workflows strengths.
Hey @BigRoyNL, I’d be happy to do that. I’ll DM you with some questions and thoughts.
@dhruvgovil this is really interesting. I’m still coming to terms with the terminology here, but by “layer” are you speaking about another layer of USD information? Something that handles things like transforms, animations, material assignments, etc? I’m going to go back an reread the USD specs
Currently, I’m a one-man-band with a limited technical understanding (I can inspect/edit usda files, and operate a handful of shell scripts, but that’s about my limit). How would you recommend I build and manage a USD hierarchy for use in RealityKit? Let me know if I should start a new thread on this.
Each file that makes up a USD stage is called a layer. The stage being the result of all the layers combining together via different types of composition like layer stacks, referencing etc…
The nice thing with USD is that you can then separate out your work as makes sense for your individual workflow.
For example, as a single creator you could work all in one layer or you could decide that you want to split out things like geometry into their own layer separate from materials.
In the case of Reality Composer Pro, I like to work with a layer (or set of layers) that define my data coming out of my DCCs. Then I assign my RealityKit specific data in a new layer for things like RealityKit specific shaders that wouldn’t go back into my DCC well at this time.
That way, as long as I don’t move something that I’ve overridden in the hierarchy, I can move them around in space or change their mesh or do other modifications in my DCC while preserving all my RealityKit specific edits.
If you’re familiar with Maya, it’s like referencing in a scene and your new scene just records the things you override. Or in Photoshop, it’s like referencing in a file on disk as a smart layer.
I think I’m getting it now, and I can spot the errors more easily. I certainly didn’t understand the role of the root, and I was able to fix up my Maya examples by adding an form between the stage and the geo, and then making user the materials were also in there
I got thrown by RCP “looking” like it was importing files correctly, but then they were breaking when used in a scene.
Thanks again for all your detailed support here @dhruvgovil! I’ll be trying to build out a scene over this next week, and I’ll no doubt have more questions around layout. But I now have a better base understanding of how these parts fit together.
I’d like to use a blender build that already uses your fix.
Would that be 4.1.0 beta or 4.0.3 RC as well?
Also, thank you for this helpful explanation and the #b3d contributions.
I tried the 4.1.0 beta but it introduced too many regressions for my (mainly animated) asset workflow for AR Quick Look. My exports are unusable and also result in 80-fold file sizes. What is going on?
Gone back to the 4.0.0 alpha of the USD branch, with your advice in mind to manually set Root Prim to /root.
