Linking AO texture map to secondary UV channel (st1)

Community USD
,
1

Hello fellow 3D people,

I have been waiting for .usdz and AR QuickLook on iOS to support linking the Ambient Occlusion (AO) texture map to a secondary UV set for a long time.

This would allow 3D content creators like me to utilize a non-unwrapped/overlapping (tiling) UV channel (st0) for base PBR channels such as Diffuse Color, Roughness, and Metallic, while still being able to use a separate unwrapped/non-overlapping UV channel (st1) for AO.

With a workflow like this, tiling surface textures can be combined with geometry-wide global AO, thus increasing surface detail (texel density) when looking at 3D assets up-close while at the same time maintaining beautiful AO and minimizing file size. This is an essential feature to enable low-bandwidth, high-quality 3D visuals on AR QuickLook and on upcoming devices like the Vision Pro.

Reverse-engineering the .usdz file for the Vision Pro on Apple.com (https://www.apple.com/105/media/us/apple-vision-pro/2023/7e268c13-eb22-493d-a860-f0637bacb569/ar/apple_vision_pro_ios16.usdz), I noticed that this .usdz file is actually using a secondary UV set for AO.

So, it seems .usdz and AR QuickLook can now support this feature, but there does not seem to be any documentation available on how to prepare and export .usdz assets with these characteristics.

Can anyone here please enlighten me (and the rest of the community) on tools/workflow for creating this kind of .usdz file?

2

Iā€™m typing from memory, so forgive any errors below. Maya generally exports this correctly, so Iā€™d use that as a reference if you can.

Anyway, in general (and again this might have mistakes because I rarely do this manually, or things I forget):

  1. Your Mesh needs two st maps (what Pixar generally use as uv-maps) both with texCoord2f[] types like below. While USD allows for arbitrary channel names, I recommend calling them st and st1 for maximum compatibility across multiple applications.
// uvSet 1
texCoord2f[] primvars:st = [(0, 0), (1, 0), (1, 1), (0, 1)] (
    interpolation = "vertex"
)
int[] primvars:st:indices = [0, 1, 3, 2]

// uvSet 2
texCoord2f[] primvars:st1 = [(1, 1), (0, 1), (0, 0), (1,0)] (
    interpolation = "vertex"
)
int[] primvars:st1:indices = [0, 1, 3, 2]
  1. You need respective shaders that read the tex coords from the mesh itself
# Read st
def Shader "TexCoordReader_st"
{
    uniform token info:id = "UsdPrimvarReader_float2"
    token inputs:varname = "st"
    float2 outputs:result
}

# Read st1

def Shader "TexCoordReader_st1"
{
    uniform token info:id = "UsdPrimvarReader_float2"
    token inputs:varname = "st1"
    float2 outputs:result
}
  1. Your UsdUVTexture shader prim then needs to point to the respective texture reader
float2 inputs:st.connect = <.../MyMaterial/TexCoordReader_st1:result>

Hope the above helps. In theory, any DCC can author the above fairly easily but Maya is the usual reference point (using the latest Maya USD plugin)

3

Thanks for the quick reply Dhruv, and nice to see that there is indeed support in AR QuickLook for this.

Our pipeline is primarily using Blender, and as far as I can see thereĀ“s no support for exporting AO maps to USD materials with the built-in exporter: Universal Scene Description ā€” Blender Manual

I will move the workflow to Maya and see how it works for now, but AO support in the Blender USD exporter would be very welcome.

Since Blender has no baked AO channel in their Principled BDSF shader model, this could be done in a similar way as the .gltf exporter is doing it with a custom node group with a specific name that the exporter will pipe into the AO channel of the USD file at export.

A possible workaround could also be to export the .USD file from Blender in text format, and create a python script that adds the AO channel and assigns it to st1, then pack it all up as a .usdz as a post-processing step.

4

Iā€™ll bring this up with the Blender devs and see what they think. Thanks for the suggestion and corresponding information

5

Thanks for bringing this up with the Blender team, Dhruv. Considering the wide industry support Blender is gaining, I think having a full-featured USDZ exporter would be great for USD/USDZ adoption in general.

Let me know if you want help with testing, feedback, or input, IĀ“m happy to assist if needed.

6

Oh one thing I should mention, but thanks to Michael Kowalski at NVIDIA, the next release of Blender will have support for letting Python Add-ons extend the USD exporter.

So if youā€™re handy with Python or any add on developers are interested , itā€™ll make it much easier to add new export features without waiting for full integration into Blender.

1 Like
7

IĀ“ve been doing some export tests in Maya 2024 with the latest Maya USD integration, and I am having problems with the exported USDZ files.

When I export a USDZ with the AO map linked to UV2 (st1), the file displays as intended in ā€œPreview.appā€ on Mac OS Sonoma 14.0 Beta (23A5328b), but the file preview in the Finder shows the file incorrectly, with the AO map in UV1 (st). When viewing the same USDZ file on my iPad running the iOS 17 Public Beta 5, the AO map is also incorrectly linked to UV0 (st0). Please refer to attached screenshots:

SCR-20230830-dml-2
SCR-20230830-dml-21920Ɨ2414 398 KB

So, it seems thereĀ“s some unexpected behavior in how MacOS and iOS display the files I export with AO linked to UV2 (st1) from Maya, or I am doing something wrong in how I set up the Maya geometry/shaders.

HereĀ“s the Maya test project: USDZUVTest.zip - Google Drive

Next, IĀ“ll try to convert the exported .usdz from Maya to ASCII .usd and look into if there are any differences in how the UV sets are defined compared to the Apple Vision Pro .usdz file (linked to in my initial post) with AO in UV2 (st1) that displays correctly across devices.

8

@psqv Could you file feedback for that on https://feedbackassistant.apple.com please? Thatā€™ll help bring it up with the RealityKit teams. Thank you!

If youā€™re handy with Python, you might be able to post process the file to bake the connections to the varname, and set a default value on the node.

Something like this might work, and seems to fix the file on my end at least. But even if this works, please file a feedback ticket as well

for prim in stage.TraverseAll():
    if prim.IsA(UsdShade.Shader):
        shader = UsdShade.Shader(prim)
        inputs = shader.GetInputs()
        for input in inputs:
            name = input.GetBaseName()
            if "varname" not in name:
                continue

            connection, sourceName, sourceType = UsdShade.ConnectableAPI.GetConnectedSource(
                input
            ) or (None, None, None)
            if connection is None:
                continue

            target_prim = connection.GetPrim()
            material = UsdShade.Material(target_prim)
            var = material.GetInput(sourceName)
            input.Set(var.Get())
9

Thanks for the post processing script @dhruvgovil, will test it on some more complex files to validate.

IĀ“ll file a bug report with Feedback Assistant and follow up with my contacts on the RealityKit team.

10

That was reported as FB12425311 on 8/23/23 and it even affects ARQL in the VisionPro simulator

11

I also filed a bug for this issue 8/8/23 with MaterialX shader usd files on VisionOS, confirmed both in the simulator and running on Vision Pro hardware.

FB12892939 (Texture maps connected to a secondary UV map channel, does not display correctly on VisionOS)

12

Apologies for the delays , but I also put in a PR to Maya here Write non-connected uvSet names by dgovil Ā· Pull Request #3339 Ā· Autodesk/maya-usd Ā· GitHub that does the same thing as my script above, but in-line with the export.

That should resolve the compatibility issues with RealityKit versions on older systems and also with other DCCs that weā€™ve encountered that donā€™t support reading the connection. Of course, long term, the goal is that everyone should support reading the connection as well but in the meantime I feel like this is a pragmatic compromise.