Animation Splines and SkelAnimation question

Community USD
1

I was pointed at the Spline animation work from the ASF slack channel ( https://github.com/PixarAnimationStudios/OpenUSD-proposals/tree/2bad4b86aa02185b5a54affed2e31c9f7792396e/proposals/spline-animation#readme ) but I had a question about if/how it was going to support SkelAnimations.

In GLB, in skeleton animation you have an array a tracks. Each track identifies what property it animates, and specifies keyframes at time points. The time points do not have to align between tracks, and you only have to animate the properties you want. E.g. you may rotate all the joints, but only specify position for the root bone (and not do scaling at all).

SkelAnimation in USD seems different (unless I am missing something). You specify the array of joints, then you have to specify for each joint the rotation, position, and scale (you cannot skip any due to the encoding scheme). Also, the time codes are shared across all joints, so if you specify rotation at time T in one rotation vector, you have to specify it for all the rotation vectors at that time code - you cannot skip values from what I can see.

If the above is correct, how will splines work with Skel Animation? Splines are based on sparse vectors with non-aligned time codes right?

Thanks!

1 Like
2

Thought I had answered this, @alankent … maybe I did in ASWF. In any case, for the record, you are absolutely correct, and we have identified some OM work we need to do to allow a “second mode” of Skeleton/Animation definition for UsdSkel that will need to exist before we can encode spline animations. Basically, we will need to allow a SkelSkeleton to be defined as a hierarchy of SkelJoint prims, which are Xformable, and therefore will be able to have scalar valued attributes that can have splines (a sub-task will be providing scalar translate and scale XformOps).

3

We are excited to hear about a hierarchy of joints! This has been a hurdle for us for fully representing our characters in USD, and one of my co-workers has implemented a new joint schema, and a lot of other work, so we could have a hierarchy of joints along with other prims.

Will your new SkelSkeleton allow other prims in the hierarchy besides joints?

4

There’s nothing today preventing other prims from riding along in the hierarchy, but I think what you’re asking is whether we will be further enhancing the object model to allow for more core behaviors, and I think the answer will have to be… Joints and splines are the priority right now, and other things will likely be post-OpenExec, as that will provide us a solid foundation upon which to build other rigging/constraint behaviors.

1 Like
5

To understand this better in case I am missing something, in many other game engines where the bone hierarchy is part of the saved model (not encoded as an array of strings in a property, to be instantiated into a hierarchy at runtime) you can do things like parent an object to a bone. Want a character to carry a ball? Drop the ball in as a child of the hand. It then automatically follows the hand. You cannot do this in USD, right? There is no prim for the hand except at runtime, so you cannot put it into a USD file. You have to write code to wire it up. Or am I missing something?

6

The sort of thing you’re talking about will be facilitated by the “second mode” spiff talks about above.

To do what you are talking about today, you’d need to maintain your own logic to compose an animation and perform constraining like holding a ball, in your engine. You can still use a USD file to carry the skin bindings and weights, but you’ll need to do that composition yourself.

Encoding a change of scene hierarchy at a particular time, ie. in the USD file, a character catches a ball at 5 seconds, and the ball is therefore reparented, is not something you can do. This would be accomplished by animating a constraint weight from zero to one, but USD doesn’t currently have this concept; constraints will be enabled by OpenExec. This pattern is not the general pattern you need in a game in any case, although it has uses of course.

As an example game to illustrate why, a ball player has two hands. the game has eight balls, and twelve players. It’s not practical or efficient at runtime to have all the possible constraints encoded in the scene. In this case, everything would be tied up in game state logic ~ balls are either in free flight, or held in a particular hand by a particular character; and your engine would be responsible per frame to either evolve the ball’s position by physics, or to teleport it to the player’s hands.

You could also parent and unparent the ball to the hand, given a joint hierarchy, of course. But that is bending the use case of UsdSkel to the breaking point.

There are two concerns at work here, which are superficially related to the degree that it seems like UsdSkel solves for it.

One concern is encoding the data required by linear-blend skinning. The second concern is a hierarchy of joints meant to pose the character. Although these are commonly the same in games, they are equally commonly not the same. For example, a skin-bind skeleton may have a great many bones that are unrelated to posing. For example, there may be a computed secondary bone whose purpose is to prevent a shoulder shape from collapsing. This bone is necessary for LBS, but it is not something an animator will pose. Similarly, there may be a ragdoll hierarchy for physics. This may have twenty crucial bones for posing, and not the hundred bones in the character. At runtime, the game engine constrains the LBS skeleton to the ragdoll skeleton, and also, the game engine computes in some manner the locations of the 80 other bones not in the ragdoll.

All of this is to say, that when we get to a joint based skeleton, we should also consider the need of constraining an animator’s posing rig skeleton (which likely has additional non skin-bound joints) to an LBS skeleton, or constraining a sparse ragdoll to an LBS skeleton, presumably via OpenExec. At that point USD will have ways to describe game-like functionality.

7

Thanks Nick. That is what I thought - I just wanted to make sure there was not some other unknow OpenUSD corner behind “nothing preventing other prims from riding along in the hierarchy”.

I have another personal hope with a “new” skeletal approach - encoding efficiency. Currently glTF (for example) you encode an animation vector with keyframe time keys per property you animate. The head does not move much? Only generate keyframe data for when it does move. Reduces the data volume to transfer.

In OpenUSD, you have to specify values for all animated properties at each time point. That means a lot more data needs to be encoded. Fine when the files are on disk, but painful if distributing OpenUSD assets over a network. So I am looking forward to making OpenUSD more efficient for data transfers. It may never be as good as glTF, but sending OpenUSD to say a phone can still be useful in some scenarios without having to convert it to glTF etc.

8

Yeah, efficiency is a tricky thing, depending very much on your use-case. In USD, the cost of querying 200 or so individual attributes (times 20,000 animated crowds characters) is significantly more than querying three vectorized attributes, and in an important use-case for VFX, we’re only pulling data for a single frame at a time. The batched encoding also allows us to “instance” all the Skels, which the new encoding will not, leading to many, many more prims in a scene at crowds-scale. And UsdSkel’s primary use case, initially, was crowds.

So we’re now acknowledging that we have another important use-case that will come with a different set of tradeoffs and definition of “Efficient” :slight_smile: