Safe production workflows in games

Community USD
1

We are building a game engine that uses USD for all authoring data. Both DCC data to allow for good interop with other tool, as well as all game play properties etc.

Games generally lean much more on prefabs than scene composition / layering.

Games are generally split into multiple completely different scenes. With many prefab instances in it and some bespoke prims fully defined in the scene. Each of these scenes mostly has only one final composition that is used in practice. We can’t just change the scene completely in a different shot in a game Everything has to be one consistent world.

Many games also don’t follow a strict pipeline flow from modeling => texturing => animation => lighting => post processing. Rather it’s a very iterative process. So leaning on layers as overrides is much less commonly used in games.

I can see two purposes for layers:

  1. You want to avoid version control conflicts and split your scene based on how you work. For most game productions I would probably use an approach of enforcing that you can only segment parts of a scene (Only the layer who introduces a prim can change it’s attributes), as opposed to arbitrarily override prims/attributes in any layer in the composition tool. For the purposes of keeping things simple.
  2. You have a live game and want to author some events where part of your game changes. By using layers for the event you can easily enable / disable all of those changes. For this the full functionality of layers is very useful.

There are two things that games lean on very heavily to keep content under control. In games we call it prefabs, but it’s basically the same as references.

Overridable Prefabs (references, instanceable = false)

The entire hierarchy shows up in the scene and you can override anything in the scene. It’s very powerful and gives full control.

It quickly leads to a lot of breakages as prefabs get changed and scenes happened to have direct overrides to some part of the scene.

Generally it is something to avoid, since it very quickly leads to breakages when prefabs get refactored and overrides in scenes using it no longer work.

Exposed attribute Prefabs. (references, instanceable = true)

You generally only get to edit the root of the prefab instance. However a common workflow is that technical artists expose specific attributes anywhere in the prefab hierarchy and connect them up to attribute in the prefabs root prim. The exposed attributes are explicitly designed to be tweakable in the composition scene. The composition scene can position the object and change the exposed attributes. Intentionally limiting what you can edit in the scene.

Why?

  1. This both helps with mental load, given that prefabs can contain a prims / attributes, that a level designer may not fully understand. Being able to quickly see the 10 important tweakable properties on the instance is very useful.
  2. The ability to guarantee that scenes don’t break when I change my prefab (as long as i keep the same exposed attributes).
  3. Enforce safe workflows by default where you don’t accidentally do changes in the scene as overrides to a prefab instance that was not meant to for overriding in the scene.

I exposed this in our tooling as usd connections. Where anywhere in the hierarchy of a prefab you can expose a property, which goes into the root of the prefab as exposed:someParameter with the property having a connection to it. This way any instancable = true, references prim can overrides those exposed parameters.

So far I am pretty happy with this approach. We integrated this into our tool & baking workflow. Basically when getting values we always follow the connections to arrive at the final attribute that is actually used.

Unfortunately doing this means we are not creating composition scenes that other tools understand anymore. It is possible to just generate a layer and turn those exposed properties into overrides to allow for interop with other tools that don’t natively understand our use of connections to drive exposed attributes. But of course that’s going to get brittle when other tools start editing those exposed attributes as overrides.

To me exposed properties on instances, seem like quite essential functionality for any safe production workflows.

So my question is, is this useful and worth exploring for the wider USD ecosystem? Have others built similar / same system? Is there something I am missing that exists to solve the problem of safe production workflows for prefab instances.

2

Hi @Joachim_Ante. The setup you describe is exactly what UsdShade prescribes for Materials and the shading networks they encapsulate (the “interface” prefix on the Material prim is inputs rather than exposed), and indeed we do promote instancing Materials. It “works” for materials because the utilities in UsdShade for extracting and processing Materials are aware of the connections from shading properties to Material interface properties, and that they imply “forwarding” of those values to the shaders at runtime.

The good forward-looking news is that, when OpenExec gets further along, it will provide general purpose dataflow semantics for “computing the value of properties” that will make the lookup you’re doing yourself into a core behavior, and then any app that is evaluating the scene will just get the right value. Note that any app that is trying to ingest/convert the USD representation into its own native repr will still need to interpret the connections itself, though. I’ll also note that getting execution to “do the right thing” for instanced prims, especially in a setup like this where unique values are being provided per-instance-root is a special challenge we’ve identified and look forward to digging into.

We use this “connection forwarding” behavior in Presto more broadly for non-instanced use-cases, also, where we’ll have a character with a complex rigging setup of many prims, but separately (but still contained inside the model), an “Animation hierarchy” of prims that contain only the controls that animators want to see, in the organization that makes sense to them. Then the properties on the rig prims connect to those properties in the Animation interface, and the rigging prims become a “hidden detail” of the model that are sometimes “locked” (a feature we hope to propose/provide for USD soon) so that they cannot be easily directly overridden.

3

As I am thinking more about this, I do want to note that the “special challenge” in allowing connected, per-instance values to prototypes via execution dataflow is more than just how to get the execution engine to do the right thing… it’s acknowledging that if we enable this behavior, we’re providing an easy and convenient way to decimate the performance benefits you get from instancing. It’s true that the Stage itself will still reap the rewards from native instancing in that it will need to populate many fewer prims. But, if there’s even one such connection in an instance prototype, then Exec and Imaging will no longer be able to share a set of prototype prims for each instance. That connection could cause an interior transform to change, or points to pose differently, and those demand different prototypes.

The “protection from upstream changes” case you make for this pattern is absolutely compelling, but we’ll need to think about the best ways to facilitate it, and making it easy to “defeat” instancing optimizations may or may not be that.

4

Is the idea longer term with OpenExec that you can bind an attribute to pull data via OpenExec?

So that one could make a property either be an explicit attribute or a value pulled via OpenExec?

So that the application layer / rendering layer doesn’t have to have a wrapper / if for every attribute. I would often expect that i start out with a specific authored attribute and then as rigs get more complex, someone might decide that this value is better to be driven.

For your optimization challenge, for us it’s not a problem since we bake / deduplicate everything anyway. But I appreciate the concern for applications that render directly from the usd representation.

5

Yes, with OpenExec, instead of always going directly to UsdAttribute::Get(), whenever you need to know the “effective value” of an attribute, you will instead use new API’s that will compute the value of the attribute, which might be determined by a simple dataflow connection (as in your case), or complex rigging that affects the attribute. If there are no computations/rigging affecting an attribute, the effective value falls back to UsdAttriubte::Get() . If you’d like to know more, you can read the white paper on OpenExec.