How to readback AOVs from glslfx shader?

Hi,

i am working on a custom outline glslfx shader for hdstorm. My idea was to use the primid and instanceid AOVs for edge detection. I wrote a custom HdxTask, which uses an HdSt_ImageShaderRenderPass. To supply the AOVs to my shader i use HdStRenderPassShader::UpdateAovInputTextures().
I get HD_HAS_primIdReadback and HD_HAS_instanceIdReadback in my shader and the samplers are there. I can read the textures with texelFetch but i get some weird data. I get -1 in the red channel correctly for uninitialized primIds but for all other pixels, where various prim/instance ids are stored i only get 0. Maybe it is because the readback textures are using all a floating point sampler, which is sampling from an integer texture? I didn’t find the code where codegen is inserting the readback sampler into the shader. In nvidia nsight it shows, that the shader is reading from a non multisampled version of primID AOV, which is blitted over from the multisampled version at an earlier time in the render cycle. The textures are containing both the correct data. Only the shader doesn’t seem to fetch the correct data as described. Maybe i am missing something important or maybe there is a better approach to achieve what i am trying to do?
I looked at other HdxTasks like OitResolveTask, which also uses HdSt_ImageShaderRenderPass but does no readback. It uses ssbo buffers for oit data instead. This would be my second idea to supply my data in a similar way. However, i would need to have custom render tasks for this, which would write my ssbo data via a shader snippin while doing scene rendering, like oitrendertask is doing.

This is my test shader code: I get something in the red channel (primId.x/instanceId.x) but only for the -1 data. Everything else is 0.

– glslfx version 0.1

– configuration
{
  “techniques”: {
    “default”: {
      “fragmentShader”: {
          “source”: \[ “ColoredOutline.Image” \]
      }
    }
  }
}

– configuration
{
  “techniques”: {
    “default”: {
      “fragmentShader”: {
      “source”: \[ “ColoredOutline.Image” \]
      }
    }
  }
}

---

– glsl ColoredOutline.Image

vec4 imageShader(vec2 uv)
{
#if defined(HD_HAS_primIdReadback) && defined(HD_HAS_instanceIdReadback)
  // Sample the id-Buffer at the frag coordinate.
  vec4 primId = texelFetch(HdGetSampler_primIdReadback(), ivec2(gl_FragCoord.xy), /\* lod = */ 0);
  vec4 instanceId = texelFetch(HdGetSampler_instanceIdReadback(), ivec2(gl_FragCoord.xy), /* lod = \*/ 0);
  return vec4((primId.x + instanceId.x) != 0, (primId.y + instanceId.y) != 0, (primId.z + instanceId.z) != 0, 1.0);
#else
  return vec4(0);
#endif
}

Here is my HdxTask code:

void ColoredOutlineTask::_Sync(HdSceneDelegate\*, HdTaskContext\*, HdDirtyBits\* dirtyBits)
{
HD_TRACE_FUNCTION();
HF_MALLOC_TAG_FUNCTION();

if (!renderPassState)
{
    // We do not use renderDelegate->CreateRenderPassState because
    // ImageShaders always use HdSt
    renderPassState = std::make_shared<HdStRenderPassState>();
    renderPassState->SetEnableDepthTest(false);
    renderPassState->SetEnableDepthMask(false);
    renderPassState->SetAlphaThreshold(0.0f);
    renderPassState->SetAlphaToCoverageEnabled(false);
    renderPassState->SetColorMasks({HdRenderPassState::ColorMaskRGBA});
    renderPassState->SetBlendEnabled(true);

    renderPassState->SetBlend(HdBlendOp::HdBlendOpAdd,
                              HdBlendFactor::HdBlendFactorOne,
                              HdBlendFactor::HdBlendFactorOneMinusSrcAlpha,
                              HdBlendOp::HdBlendOpAdd,
                              HdBlendFactor::HdBlendFactorOne,
                              HdBlendFactor::HdBlendFactorOneMinusSrcAlpha);

    std::string shaderPath = "[xxx]/Hydra/Shaders/ColoredOutline.glslfx";

    renderPassShader = std::make_shared<HdStRenderPassShader>(TfToken(shaderPath));
    renderPassState->SetRenderPassShader(renderPassShader);
}

if ((*dirtyBits) & HdChangeTracker::DirtyParams)
{
    hasAovInputBindings = false;
}

*dirtyBits = HdChangeTracker::Clean;

}

void ColoredOutlineTask::Prepare(HdTaskContext\* ctx, HdRenderIndex\* renderIndex)
{
HD_TRACE_FUNCTION();
HF_MALLOC_TAG_FUNCTION();

if (!renderPass)
{
    HdRprimCollection collection;
    HdRenderDelegate* renderDelegate = renderIndex->GetRenderDelegate();

    if (!TF_VERIFY(dynamic_cast<HdStRenderDelegate*>(renderDelegate), "Colored Outline Task only works with HdSt"))
    {
        return;
    }

    renderPass = std::make_shared<HdSt_ImageShaderRenderPass>(renderIndex, collection);
    renderPass->SetupFullscreenTriangleDrawItem();
}

if (!hasAovInputBindings)
{
    const HdRenderPassAovBindingVector& ctxAovBindings = GetAovBindings(ctx);

    aovInputBindings.clear();
    aovBindings.clear();

    for (auto& aovBinding : ctxAovBindings)
    {
        if ((aovBinding.aovName == HdAovTokens->primId) or (aovBinding.aovName == HdAovTokens->instanceId))
        {
            aovInputBindings.emplace_back(aovBinding);
        }
        else if (aovBinding.aovName == HdAovTokens->color)
        {
            aovBindings.emplace_back(aovBinding);
        }
    }
    hasAovInputBindings = true;
}

renderPassShader->UpdateAovInputTextures(aovInputBindings, renderIndex);

}

void ColoredOutlineTask::Execute(HdTaskContext\* ctx)
{
HD_TRACE_FUNCTION();
HF_MALLOC_TAG_FUNCTION();

GLF_GROUP_FUNCTION();

if (!TF_VERIFY(renderPassState))
{
    return;
}
if (!TF_VERIFY(renderPassShader))
{
    return;
}

renderPassState->SetAovBindings(aovBindings);
UpdateCameraFraming(ctx);

renderPass->Execute(renderPassState, GetRenderTags());

}

Meanwhile i was cloning HdxRenderTask and the related shaders and modified them, to setup additional SSBOs, which i write to in the shaders to store id data and some other int data. However, the problem is, that i need depth testing for this and with early fragment testing on, there are synchronization issues with simultaneously rendered fragments with different depths, because the depthtest of fragment A might be executed before fragment B writes to the depthbuffer.

So i switched back to the id AOV readback approach to prevent an extra early z pass. However i recognized, that HdStorm is not supporting MRT currently. There is no call to glDrawBuffers to enable the renderPassShader to write to the ID target and the color target.

Are there plans to support mrt for HdStorm? For now i think i need to clone the HdStRenderPass and HdStRenderPassState and modify it to support MRT for the set color AOVs.

Meanwhile i did a fork of OpenUSD and added support for mrt into the AOV attachment initialization code in HdStRenderPassState. This works and i can write the primIDs in an int32 rendertarget simultaneously when the color target is rendered.

However the second problem, which is probably also the problem i had as i initially posted this thread, is that HdSt doesn’t seem to support integer textures as shader input. I guess it doesn’t happen very frequently, that somebody is using HdStorm directly via hydra without the USD scene index and needs some special shader setup and render graph :D.

So i will check if there is an easy way to extend HdSt to make codegen emit integer samplers for integer textures. Because otherwise if you try to add an integer AOV as readback texture like PrimID AOV, it will produce undefined behaviour (at least in OpenGL HGI). Currently the codegen is always emitting a normal floating point sampler, also for integer textures.

Are there plans to add support for integer shader input textures to HdSt?

Cheers,

Robert

Just as final information: I did a workaround for the integer sampler codegen setup problem. I basically used the Hgi directly in my rendertask. This is also done that way by the visualize AOV task actually. Probably due to the same issues with codegen. So the Hgi is generating integer samplers correctly. It maybe also the case, that codegen is only emitting float samplers for int textures for OpenGL Hgi’s. I recognized multiple code paths, which probably stem from the pure OpenGL history of HdSt.