Framebuffer inter operation of external swap chain and hydra

Community Hydra
1

Hi guys,

I am currently looking at frame buffer inter operation from a C++ QT application with hydra. There doesn’t seem to be much documentation on this and also on frame buffer interoperation with external swap chains in general. I already looked at some unit tests and the source of USD view but I find it hard to grasp how the mechanism exactly works. It seems to me that there are only examples / tests with openGL contexts, made by the host programs, which are using hydra to render into a frame buffer in that context. I know that you can specify AOVs. But how does the underlying renderer know, that it should render into an external OpenGL frame buffer (for example QT’s OpenGLWidget). Is there some implicit logic I did miss? How can I make hydra render into an existing frame buffer? Is it possible to make hydra render into an existing Metal or Vulkan framebuffer or do I need an OpenGL context for inter operation? Maybe you can point me on some documentation, simple examples or code pieces I can start to look at.

cheers,
Robert

2

Hi Robert,

I used a QOpenGLWindow to render with Hydra Storm in an OpenGL context using a draw target and saving it to the disk. Not sure this is super helpful for what you want to do though…
The magic with Hydra Storm is that if you have defined an OpenGL context and set it as the current context, it will find it and render in it.
So in theory you shouldn’t need the drawTarget I used, only setting the QT openGL context should be enough.

The pseudo code was this :

I was storing in the class :

QOpenGLContext *m_context       = nullptr;
pxr::GlfGLContextSharedPtr   mpOpenGLContext;
pxr::GlfDrawTargetRefPtr     mDrawTarget;

Then in the cpp file :

void HydraOpenGLWindow::initializeGL()
{
    QOpenGLWindow::initializeGL();
    mGLInitialized = true;
    InitializeHydra();
}

void HydraOpenGLWindow::InitializeHydra()
{
    // prep draw target
    if (nullptr == mDrawTarget){
        GlfGLContextScopeHolder contextHolder(mpOpenGLContext);

        mDrawTarget = GlfDrawTarget::New(GfVec2i(width(), height()));
        mDrawTarget->Bind();
        mDrawTarget->AddAttachment("color", GL_RGBA, GL_FLOAT, GL_RGBA);
        mDrawTarget->AddAttachment("depth", GL_DEPTH_COMPONENT, GL_FLOAT, GL_DEPTH_COMPONENT);
        mDrawTarget->Unbind();
    }
}

void HydraOpenGLWindow::renderNow()
{
    if (!isExposed())
        return;
    
    bool needsInitialize = false;

    if (!m_context) {
        m_context = new QOpenGLContext(this);
        QSurfaceFormat s = requestedFormat();
        DebugSurfaceFormat(s);
        m_context->setFormat(s);
        m_context->create();

        needsInitialize = true;
    }

    //GlfGLContextScopeHolder contextHolder(mpOpenGLContext);
    m_context->makeCurrent(this);

    if (needsInitialize) {
        initializeOpenGLFunctions();
        initialize();
        
        auto logger = new QOpenGLDebugLogger(m_context);

		logger->connect(logger, &QOpenGLDebugLogger::messageLogged,
			[](const QOpenGLDebugMessage& message) { qInfo() << message; });

		if (logger->initialize())
		{
			qInfo() << "OpenGL logger initialized";
			logger->startLogging(QOpenGLDebugLogger::SynchronousLogging);
			logger->enableMessages();
			// Disable NVidia performance spam "APISource", 131185,
			// 		"Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_ARB, usage hint is "
			// 		"GL_STATIC_DRAW) will use VIDEO memory as the source for buffer object operations.",
			// 		"NotificationSeverity", "OtherType"

			//logger->disableMessages(QVector<GLuint>{ 131185 });
            
		}
		else
		{
			qInfo() << "!!!OpenGL logger not initialized!!!";
		}
    }

    render();

    m_context->swapBuffers(this);
}

void HydraOpenGLWindow::render()
{
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
    glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
    const int pixelRatio = devicePixelRatio();
    m_device->setSize(size() * pixelRatio);
    m_device->setDevicePixelRatio(pixelRatio);
    glViewport(0, 0, width() * pixelRatio, height() * pixelRatio);
    
    DrawTexture();
    DrawTriangle();

    renderHydra();
    ++m_frame;
}

void HydraOpenGLWindow::renderHydra()
{
    //GlfGLContextScopeHolder contextHolder(mpOpenGLContext);

    const int w    = width();
    const int h    = height();

    mDrawTarget->Bind();
    mDrawTarget->SetSize(GfVec2i(w, h));

    int viewportWidth   = 0;
    int viewportHeight  = 0;
    mpHydraViewportData->GetViewportSize(viewportWidth, viewportHeight);
    DbgAssert(w == viewportWidth && h == viewportHeight);
    mpHydraViewportData->HydraRender();
    
    mDrawTarget->Unbind();
    mDrawTarget->Resolve();
    mDrawTarget->WriteToFile("color", "d:\\colorHydraOpenGL.png");

    /*
I am not sure if this was working or not.
// Blit the resulting color buffer to the window
    GLint restoreReadBuffer;
    glGetIntegerv(GL_READ_BUFFER, &restoreReadBuffer);//Save values
    GLint restoreDrawBuffer;
    glGetIntegerv(GL_DRAW_BUFFER, &restoreDrawBuffer);

    glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);
    glBindFramebuffer(GL_READ_FRAMEBUFFER, mDrawTarget->GetFramebufferId());

    glBlitFramebuffer(0, 0, w, h,
                      0, 0, w, h,
                      GL_COLOR_BUFFER_BIT,
                      GL_NEAREST);

    //Restore values
    glBindFramebuffer(GL_DRAW_FRAMEBUFFER, restoreDrawBuffer);
    glBindFramebuffer(GL_READ_FRAMEBUFFER, restoreReadBuffer);
    */
}
3

Thanks for your example and explanations! So HdStorm just renders to the current OpenGL context and the bound FBO. But what if I am using Metal or Vulkan as HGI for HdStorm? Will it use HgiInterop internally to copy the metal/vulkan fb into the bound OpenGL fb or do I need to do that depending on which HGI I have chosen? Second, I saw in the HgiInterop code, that only OpenGL is supported as target. Is there a way to make hydra render directly to a Vulkan/Metal render target instead of transforming the buffers to OpenGL fbo via HgiInterop? I am thinking to use the new QRhiWidget (since QT 6.7), which could also maintain other APIs than OpenGL.

4

If you’re using TaskController, the presentation to the GL context is done by the TaskController’s PresentTask and you can control this using APIs like TaskController::SetEnablePresentation and TaskController::SetViewportRenderOutput. I think these get set automatically if there’s a current GL context. If you have a CPU renderer which just creates AOVs, you can make your own GL texture from the AOV render buffer and draw that to your context. We use both methods.

5

Thank you Jerry for pointing me to TaskController and PresentTask. These were my missing links to get a general understanding of it. So my conclusion is:

  1. The normal framebuffer interop procedure is to have an openGL context by the app (e.g. USDView) and an openGL framebuffer.
  2. The framebuffer needs to be bound and the context needs to be current.
  3. To get the colour and/or depth AOV into the applications frame buffer the PresentTask has to be used, which in turn uses HgiInterop to convert and render the AOV from a render buffer (which can be OpenGL, Metal or Vulkan depending on the chosen HGI) to the bound openGL frame buffer.
  4. It is currently not possible to use the PresentTask/HgiInterop to render into a metal or Vulkan application frame buffer.
  5. The Framebuffer, where the presentTask renders into can be set by the application to a specific value by HdxTaskController::SetPresentationOutput.
  6. Other than that the app has to access the AOV RenderBuffers, get the API resources from it and render/convert them by itself to a specific Render API framebuffer.
  7. To make hydra render directly into an application framebuffer it is necessary to setup a custom HdxTask or Pass, which is setting up the Bprims for the AOVs with application managed Textures (this point I am not so sure of)

Please correct me if I am wrong.

Thanks and cheers,
Robert

6

SetPresentationOutput sets which AOV will be rendered to the framebuffer, not which framebuffer.

7

Maybe we are having two different functions in mind. I think you are referring to HdxTaskController::SetViewportRenderOutput(TfToken const& name). Which seems indeed to set the AOV, which will be rendered into the frame buffer?
I was referring to HdxTaskController::SetPresentationOutput(
TfToken const &api,
VtValue const &framebuffer), which really seems to set the fb, but it isn’t used by usdview or any test. That is if I am not totally off the track.

8

Sorry, yes. I was thinking of the wrong API.