GameEngine as a SceneIndex plugin

Community Hydra
1

Hey,

In my spare time (…) I’m experimenting with “behaviours” all described in a usd file, and then handled by a “game engine” which is actually a SceneIndex plugin.

Initially I thought about writing all in a custom app, which then changes the status of various usd-prims and then just use hydra for rendering, but then I thought that the whole logic has to be written in the custom app, and OpenUSD and Hydra were going to be just a storage and a renderer.

In this case, if the logic (the actual engine) is a SceneIndex plugin, you could actually “play” a scene in usdview or any other host that supports Hydra-2.0 basically.

I’ve got some basic things going so far, like, I can go through doors that open when the “character” (camera or a specific prim) gets close to them, or lights changing intensity/exposure based on distance to the character or shooting new assets in the scene.
This works in my simple usdviewer and in the default usdview (the “shooting” is triggered via api, setting a primvar on a root “/game” prim, and everything else is like based on current-camera update.

I was wondering pros and cons of something like this, and one thing that got to my mind is that there is no way currently to serialize the Hydra scene to a file on disk.

The usd file describes just the “initial state” of the game, and all that happens is hydra-modifications, but not having the ability to serialize the status of the hydra scene on disk, feels an important limitation.

Will there ever be a way to serialize Hydra to disk ?
( This would actually deserve its own topic, as it could be pretty useful for offline-renders too ).

One first question I have is about editing transforms: is there a better way than using a generic DataSource editor for xforms ? I’ve seen a MaterialNetwork, but couldn’t find a xform one.

I’m currently doing this, inspired by one of the Overlay examples, and it seems to be working as expected:

matrixContainer =
    HdContainerDataSourceEditor()
        .Set(L("matrix"), HdRetainedTypedSampledDataSource<GfMatrix4d>::New(newMatrix))
        .Finish();

prim.dataSource = 
    HdContainerDataSourceEditor(prim.dataSource)
        .Overlay(L("xform"), matrixContainer)
        .Finish();

Cheers!

1 Like
2

Hi Paolo. Our team plans to contribute a PR for a text hydra delegate.

3

Thanks @nvmkuruc nice.
that’s just output from Hydra right ? a delegate dumping hdprims in text, am I understanding that correctly ?
I was wondering if a hydra serialize/deserialize would be useful, in a way to also skip usd-composition entirely, and just “render” a scene.

In my case, I’m writing out a json file to read it back it as the “state of the game”, but I guess my original question was really generic, and I think I will start a new thread about that :slight_smile:

4

Update on this: I’ve added ReactPhysics3d and I wanted to have a centralized place to update the “game”, so I’ve made an Xform prim and added a “frame” attribute, which I thought I could update on each frame and trigger a _PrimsDirtied call… but apparently that’s not happening.
Only if I update one of the schema’s attributes it then gets dirtied.
So I’ve added a timeSampled transform on this empty Xform.

I assume this is due to having enabled the new hydra api which is schema based only, right ?
This reminds me that we use extra parameters on Cameras and other prims, currently in a hydra-1.0 pipeline, and moving forward those need to be added to a schema to continue to trigger updates…

Anyway, here is a video of a realtime rigid body sim handled by the gamemanager (sceneindex plugin) then updating elements in the hydra scene

1 Like
5

I feel this is becoming a blog… sorry about that, I will actually do a blog about this gameengine at some point, but I wanted to share an update.
Thanks to @nporcino and his easy-to-use LabSound library, I’ve added a new behaviour to my SceneIndex GameEngine, where I can tag a usd prim as an “audiosource” with spatial support.

Here the vide (better with sound/headphones) where you can “hear” 3 different audiosources, from the kettle, the kitchenaid and kids playing outside (a cube as source).

And I’ve added a “door” behaviour to the fridge door ( I had to reorganize the structure of the fridge to make it easier to rotate with the right pivot ).

6

And as expected, running it as a SceneIndex plugin makes it available in usdview and Solaris and in all usd+hydra compatible hosts basically.

1 Like
7

With a SceneIndex plugin, performing dynamic LOD-switching is also pretty straight forward

2 Likes
8

Hey @paoloemilioselva, this came up in the ASWF USD games working group, and it looks like a really cool exploration. Have you or are you planning to share this as open source at some point? I’d love to see how you’re doing things, particularly because it’s Hydra 2.0, something I’ve not worked with yet.

It’s tough to say that my studio would want to dive in this deep yet, but we’ve been chewing on how to handle things like LOD-switching in applications so your last update is very, very interesting.

9

Hi Levi,

I wanted to be in that meeting, just too early for me, I hope I can see the recording soon.
And yes, I’d like to clean up a bit the code and then release it opensourced as it is really an exploration of what’s possible as a SceneIndex plugin (specifically, a SingleInputFilteringSceneIndex plugin) in Hydra-2.0 and it could work as example for something much better.

Cheers!

1 Like
10

usdview as container for a videogame ? :sweat_smile:

If you are wondering if you can use a controller in a sceneindex plugin the answer is obviously yes :slight_smile:

What I find fascinating about this is that, with all its limitations, I can potentially deliver an entire videogame setup as a python script or a usda file and you can play it in usdview (yeah, you need a custom scene index plugin, but no host-specific modifications).

This scene is currently this in usda form (and it runs in usdview and Solaris)
I’ve been a bit lazy, I should have schematize this, but for now, just a bunch of primvars to drive the behaviour of each prim.

#usda 1.0
(
    framesPerSecond = 120
    metersPerUnit = 1
    timeCodesPerSecond = 120
    startTimeCode = 1
    endTimeCode = 100
)

def Xform "sphere1" ( kind = "component" )
{
    matrix4d xformOp:transform:xform = ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 50, 0, 1) )
    uniform token[] xformOpOrder = ["xformOp:transform:xform"]

    def Mesh "mesh_0"
    {
        ...geometry primvars...

        custom bool primvars:behaviour:rigidbody = 1
        custom bool primvars:behaviour:rigidbody:dynamic = 1
        custom bool primvars:behaviour:rigidbody:kinematic = 0
        custom string primvars:behaviour:rigidbody:shape = "sphere"
    }
}

def Xform "sphere2" ( kind = "component" )
{
    matrix4d xformOp:transform:xform = ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (5, 80, 0, 1) )
    uniform token[] xformOpOrder = ["xformOp:transform:xform"]

    def Mesh "mesh_0"
    {
        ...geometry primvars...

        custom bool primvars:behaviour:rigidbody = 1
        custom bool primvars:behaviour:rigidbody:dynamic = 1
        custom bool primvars:behaviour:rigidbody:kinematic = 0
        custom string primvars:behaviour:rigidbody:shape = "sphere"
    }
}

def Xform "ground" ( kind = "component" )
{
    matrix4d xformOp:transform:xform = ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 0, 0, 1) )
    uniform token[] xformOpOrder = ["xformOp:transform:xform"]

    def Mesh "mesh_0"
    {
        ...geometry primvars...

        custom bool primvars:behaviour:joystick:driven = 1
        custom bool primvars:behaviour:rigidbody = 1
        custom bool primvars:behaviour:rigidbody:dynamic = 0
        custom bool primvars:behaviour:rigidbody:kinematic = 1
    }
}

def Xform "_gamemanager_"
{
    custom bool primvars:behaviour:gamemanager:status = 1
    custom float primvars:behaviour:gamemanager:frame ( interpolation = "constant" )
    float primvars:behaviour:gamemanager:frame.timeSamples = {
        0: 0,
        100: 100,
    }
}
1 Like
11

So neat ! Thank you for the work in progress updates.

I would be definitely interested to embed it into my ImGuiHydraEditor as an extra out of the box Hydra Example :slight_smile:

1 Like
12

I’ve realised I haven’t provided an update on the LODs here, and instead of having a rubber toy transformed into a sphere… I thought I’d make a more realistic scenario, with a buildings-generator script I’ve made a while back and individual units/cells switching LODs with an overall control for LOD-switching to make more or less aggressive.

1 Like
13

Wow, your experiments are incredible, thanks for sharing Paolo!

1 Like