How to readback AOVs from glslfx shader?

Community Hydra
,
1

Hi,

i am working on a custom outline glslfx shader for hdstorm. My idea was to use the primid and instanceid AOVs for edge detection. I wrote a custom HdxTask, which uses an HdSt_ImageShaderRenderPass. To supply the AOVs to my shader i use HdStRenderPassShader::UpdateAovInputTextures().
I get HD_HAS_primIdReadback and HD_HAS_instanceIdReadback in my shader and the samplers are there. I can read the textures with texelFetch but i get some weird data. I get -1 in the red channel correctly for uninitialized primIds but for all other pixels, where various prim/instance ids are stored i only get 0. Maybe it is because the readback textures are using all a floating point sampler, which is sampling from an integer texture? I didn’t find the code where codegen is inserting the readback sampler into the shader. In nvidia nsight it shows, that the shader is reading from a non multisampled version of primID AOV, which is blitted over from the multisampled version at an earlier time in the render cycle. The textures are containing both the correct data. Only the shader doesn’t seem to fetch the correct data as described. Maybe i am missing something important or maybe there is a better approach to achieve what i am trying to do?
I looked at other HdxTasks like OitResolveTask, which also uses HdSt_ImageShaderRenderPass but does no readback. It uses ssbo buffers for oit data instead. This would be my second idea to supply my data in a similar way. However, i would need to have custom render tasks for this, which would write my ssbo data via a shader snippin while doing scene rendering, like oitrendertask is doing.

This is my test shader code: I get something in the red channel (primId.x/instanceId.x) but only for the -1 data. Everything else is 0.

– glslfx version 0.1

– configuration
{
  “techniques”: {
    “default”: {
      “fragmentShader”: {
          “source”: \[ “ColoredOutline.Image” \]
      }
    }
  }
}

– configuration
{
  “techniques”: {
    “default”: {
      “fragmentShader”: {
      “source”: \[ “ColoredOutline.Image” \]
      }
    }
  }
}

---

– glsl ColoredOutline.Image

vec4 imageShader(vec2 uv)
{
#if defined(HD_HAS_primIdReadback) && defined(HD_HAS_instanceIdReadback)
  // Sample the id-Buffer at the frag coordinate.
  vec4 primId = texelFetch(HdGetSampler_primIdReadback(), ivec2(gl_FragCoord.xy), /\* lod = */ 0);
  vec4 instanceId = texelFetch(HdGetSampler_instanceIdReadback(), ivec2(gl_FragCoord.xy), /* lod = \*/ 0);
  return vec4((primId.x + instanceId.x) != 0, (primId.y + instanceId.y) != 0, (primId.z + instanceId.z) != 0, 1.0);
#else
  return vec4(0);
#endif
}

Here is my HdxTask code:

void ColoredOutlineTask::_Sync(HdSceneDelegate\*, HdTaskContext\*, HdDirtyBits\* dirtyBits)
{
HD_TRACE_FUNCTION();
HF_MALLOC_TAG_FUNCTION();

if (!renderPassState)
{
    // We do not use renderDelegate->CreateRenderPassState because
    // ImageShaders always use HdSt
    renderPassState = std::make_shared<HdStRenderPassState>();
    renderPassState->SetEnableDepthTest(false);
    renderPassState->SetEnableDepthMask(false);
    renderPassState->SetAlphaThreshold(0.0f);
    renderPassState->SetAlphaToCoverageEnabled(false);
    renderPassState->SetColorMasks({HdRenderPassState::ColorMaskRGBA});
    renderPassState->SetBlendEnabled(true);

    renderPassState->SetBlend(HdBlendOp::HdBlendOpAdd,
                              HdBlendFactor::HdBlendFactorOne,
                              HdBlendFactor::HdBlendFactorOneMinusSrcAlpha,
                              HdBlendOp::HdBlendOpAdd,
                              HdBlendFactor::HdBlendFactorOne,
                              HdBlendFactor::HdBlendFactorOneMinusSrcAlpha);

    std::string shaderPath = "[xxx]/Hydra/Shaders/ColoredOutline.glslfx";

    renderPassShader = std::make_shared<HdStRenderPassShader>(TfToken(shaderPath));
    renderPassState->SetRenderPassShader(renderPassShader);
}

if ((*dirtyBits) & HdChangeTracker::DirtyParams)
{
    hasAovInputBindings = false;
}

*dirtyBits = HdChangeTracker::Clean;

}

void ColoredOutlineTask::Prepare(HdTaskContext\* ctx, HdRenderIndex\* renderIndex)
{
HD_TRACE_FUNCTION();
HF_MALLOC_TAG_FUNCTION();

if (!renderPass)
{
    HdRprimCollection collection;
    HdRenderDelegate* renderDelegate = renderIndex->GetRenderDelegate();

    if (!TF_VERIFY(dynamic_cast<HdStRenderDelegate*>(renderDelegate), "Colored Outline Task only works with HdSt"))
    {
        return;
    }

    renderPass = std::make_shared<HdSt_ImageShaderRenderPass>(renderIndex, collection);
    renderPass->SetupFullscreenTriangleDrawItem();
}

if (!hasAovInputBindings)
{
    const HdRenderPassAovBindingVector& ctxAovBindings = GetAovBindings(ctx);

    aovInputBindings.clear();
    aovBindings.clear();

    for (auto& aovBinding : ctxAovBindings)
    {
        if ((aovBinding.aovName == HdAovTokens->primId) or (aovBinding.aovName == HdAovTokens->instanceId))
        {
            aovInputBindings.emplace_back(aovBinding);
        }
        else if (aovBinding.aovName == HdAovTokens->color)
        {
            aovBindings.emplace_back(aovBinding);
        }
    }
    hasAovInputBindings = true;
}

renderPassShader->UpdateAovInputTextures(aovInputBindings, renderIndex);

}

void ColoredOutlineTask::Execute(HdTaskContext\* ctx)
{
HD_TRACE_FUNCTION();
HF_MALLOC_TAG_FUNCTION();

GLF_GROUP_FUNCTION();

if (!TF_VERIFY(renderPassState))
{
    return;
}
if (!TF_VERIFY(renderPassShader))
{
    return;
}

renderPassState->SetAovBindings(aovBindings);
UpdateCameraFraming(ctx);

renderPass->Execute(renderPassState, GetRenderTags());

}