Hello,
I have been using UsdImagingGLEngine for a simple renderer in my c++ application and now I need to port it to macOS/Metal.
Where can I find simple example that renders a simple USD scene using Metal? I have been struggling a lot trying to implement Storm delegate, but cannot find an easy to follow example.
All I need is to render USD scene into MTLTexture, similar way I could do with UsdImagingGLEngine and its Render function.
Thanks in advance!
Hi Hateom,
I also did some research on it recently and found, that UsdImagingGLEngine is actually using the default Hgi for the OS you are on. If you are on MacOS it renders with metal under the hood, because it will chose metal as default Hgi. However, it converts the metal rendered framebuffer (or colour AOV) to a GL texture, which is then rendered in your bound GL framebuffer. This is done via HgiInterop. So if you need a metal texture, you could probably extract the AOV texture directly and cast it back to the metal texture type (GetAovTexture function). Hydra needs definitely more documentation and examples.
I almost always have to read and understand hydra code to learn things. Sometimes there are helpful comments in the code but I am also missing a central documentation on how things work related to hydra.
For what it’s worth - I found a working code sample (complete Xcode project) in Apple documentation:
1 Like
I use that one as a starting point. You can find all the details you need in there, and it boils down to the following.
make an engine. Be sure to throw out the old one if you’ve loaded a new stage (not shown).
// this will cause the creation of the Hydra hardware context.
_self->engine.reset(new UsdImagingGLEngine(_self->driver));
_self->engine->SetEnablePresentation(false);
_self->engine->SetRendererAov(HdAovTokens->color);
//_self->engine->SetRendererAov(HdAovTokens->depth);
_self->material.SetAmbient(GfVec4f(kAmbient, kAmbient, kAmbient, 1.0f));
_self->material.SetSpecular(GfVec4f(kSpecular, kSpecular, kSpecular, 1.0f));
_self->sceneAmbient = GfVec4f(kAmbient, kAmbient, kAmbient, 1.0f);
set up a viewport and frustum
GfFrustum frustum;
frustum.SetPositionAndRotationFromMatrix(modelViewMatrix);
frustum.SetProjectionType(pxr::GfFrustum::ProjectionType::Perspective);
double targetAspect = double(width) / double(height);
float filmbackWidthMM = cameraMode->GetCamera().sensor.aperture.y.mm;
double hFOVInRadians = 2.0 * atan(0.5 * filmbackWidthMM /
cameraMode->GetCamera().optics.focal_length.mm);
double fov = (180.0 * hFOVInRadians)/M_PI;
frustum.SetPerspective(fov, targetAspect, _self->znear, _self->zfar);
GfMatrix4d projMatrix = frustum.ComputeProjectionMatrix();
_self->engine->SetCameraState(modelViewMatrix, projMatrix);
// Viewport setup.
GfVec4d viewport(0, 0, width, height);
_self->engine->SetRenderViewport(viewport);
_self->engine->SetWindowPolicy(CameraUtilMatchVertically);
set up default lighting and material, details omitted
_self->engine->SetLightingState(lights,
_self->material, _self->sceneAmbient);
render and selection settings
// Set up some render parameters
auto& params = _self->params;
params.clearColor = GfVec4f(0.0f, 0.0f, 0.0f, 0.0f);
params.colorCorrectionMode = HdxColorCorrectionTokens->sRGB;
params.complexity = 1.f;
params.cullStyle = pxr::UsdImagingGLCullStyle::CULL_STYLE_BACK_UNLESS_DOUBLE_SIDED;
params.drawMode = pxr::UsdImagingGLDrawMode::DRAW_SHADED_SMOOTH;
//params.domeLightCameraVisibility = false;
params.enableLighting = _self->enableLighting;
params.enableIdRender = false;
params.enableSampleAlphaToCoverage = true;
params.enableSceneMaterials = _self->enableSceneMaterials;
params.enableSceneLights = _self->enableSceneLights;
params.flipFrontFacing = _self->flipFrontFacing;
params.forceRefresh = false;
params.frame = timeCode;
params.gammaCorrectColors = true;
params.highlight = true;
params.showGuides = _self->showGuides;
params.showProxy = _self->showProxy;
params.showRender = _self->showRender;
_self->engine->SetSelected(selection->GetSelectionPaths());
render
TfErrorMark mark;
_self->hgi->StartFrame();
_self->engine->Render(stage->GetPseudoRoot(), params);
_self->hgi->EndFrame();
TF_VERIFY(mark.IsClean(), "Errors occurred while rendering!");
get the result
id<MTLTexture> metalColorTexture = nil;
HgiTextureHandle colorTexture;
HdStRenderBuffer* colorRenderBuffer = nullptr;
//colorRenderBuffer = dynamic_cast<HdStRenderBuffer*>(_self->engine->GetAovRenderBuffer(HdAovTokens->color));
if (colorRenderBuffer) {
colorTexture = colorRenderBuffer->GetResource(/* multiSampled = */false).Get<HgiTextureHandle>();
}
else {
colorTexture = _self->engine->GetAovTexture(HdAovTokens->color);
}
if (colorTexture) {
HgiMetalTexture* hgiMetalTexture = static_cast<HgiMetalTexture*>(colorTexture.Get());
if (hgiMetalTexture) {
metalColorTexture = hgiMetalTexture->GetTextureId();
}
}
2 Likes
Now, I tried integrating the code above in my existing Metal application. I am doing pretty much exactly the same what I see in the sample project, and I am getting Expection EXC_BAD_ACCESS when trying to instantiate UsdImagingGLEngine.
The crash happens here:
HdStStagingBuffer::HdStStagingBuffer(HdStResourceRegistry *resourceRegistry)
: _resourceRegistry(resourceRegistry)
, _head(0)
, _capacity(0)
, _activeSlot(0)
{
_tripleBuffered = resourceRegistry->GetHgi()->GetCapabilities()->
IsSet(HgiDeviceCapabilitiesBitsUnifiedMemory);
}
Looks like “GetHgi()” returns nullptr here - for some reason there’s no renderer in the resource registry… even though it’s created the same way as in the sample from Apple:
m_model = UsdStage::Open("path");
SdfPathVector excludedPaths;
m_hgi = Hgi::CreatePlatformDefaultHgi();
HdDriver driver{HgiTokens->renderDriver, VtValue(m_hgi.get())};
m_engine.reset(new UsdImagingGLEngine(
m_model->GetPseudoRoot().GetPath(),
excludedPaths, SdfPathVector(),
SdfPath::AbsoluteRootPath(), driver));
And this is what I get in the logs:
Coding Error: in SetDrivers at line 245 of /Users/hateom/Workspace/USD_/pxr/imaging/hdSt/renderDelegate.cpp -- Failed verification: ' _hgi ' -- HdSt requires Hgi HdDriver
[1] 24777 segmentation fault
OK, looks like it’s a known issue described here:
As a sanity check, you created m_hgi. Instead of calling resourceRegistry->GetHgi(), does m_hgi->GetCapabilities()->
IsSet(HgiDeviceCapabilitiesBitsUnifiedMemory); work if you use it instead?
I also had trouble with the issue you pointed at. My workaround was to compile my program with symbol visibility NOT set to hidden. Otherwise it will generate symbol compatibility problems as in the linked issue and the HGI creation will crash. I had this issue on macOS with apple clang.
Yes, the same - arm based macOS app compiled with clang.
My app was built with symbols hidden by default.
Took me a lot of time to figure this out…
It would be helpful to have steps to reproduce. I double checked all of my arm based macOS apps compiled with clang, and for sure, I have symbols hidden by default, and I don’t see hgi creation crashing as described in this thread, using the code I posted above.
Sorry, not the HGI creation crashes, but the later access of it e.g. by RenderDelegate creation:
_hgi = pxr::Hgi::CreatePlatformDefaultHgi();
_hgiDriver = new pxr::HdDriver({pxr::HgiTokens->renderDriver, pxr::VtValue(_hgi.get())});
pxr::HdRendererPluginRegistry& registry = pxr::HdRendererPluginRegistry::GetInstance();
HdPluginRenderDelegateUniqueHandle renderDelegate =
registry.CreateRenderDelegate(registry.GetDefaultPluginId(true));
It only crashes if this code is executed in a module compiled with symbols hidden. If UsdImagingGLEngine is used, it is executing it in the OpenUSD modules and it won’t crash obviously.
The issue with the type_info mismatch is, that the hydra dynamic library doesn’t export a lot of symbols explicitly, including HGI for example. Given that also the type_info of it won’t be explicitly exported. That means if that dynlib is linked in by the main program with set(CMAKE_CXX_VISIBILITY_PRESET hidden) the symbol is not linked with the main programs version of e.g. HGI type_info and another symbol is generated, which makes the comparison fail.
Thanks for that clarification. I understand the scenario now!
I’d also point out that these symbol visibility issues seem to depend on randomness in link order and can come and go when you rebuild. We regularly find issues with dynamic_cast between two classes caused by the same problem, but a clean build might just move the problem to a different set of classes.
1 Like
m_engine.reset(new UsdImagingGLEngine(
m_model->GetPseudoRoot().GetPath(),
excludedPaths, SdfPathVector(),
SdfPath::AbsoluteRootPath(), driver));
They have removed engine.reset() in OpenUSD 24.08. What is the recommended way to do this with the current API?
.reset(...) is just a method on every shared_ptr. In my original example with _self->engine.reset(…) that was the case. m_engine in this thread, earlier, refers to @hateom’s application, where m_engine is also a shared_ptr.
Hope this helps
1 Like
I am now also implementing a direct AOV transfer from Metal Hgi to my app and there are some caveats: I am using QT with the Metal Rhi and rendering my scene using hydra with HdStorm and HdxTaskController. I can extract the AOV texture via _engine->GetTaskContextData(HdAovTokens->color, &aov) and access the underlying metal texture. My Problem is, that the AOV is accessed earlier by my app than it seems to be ready and I get an AOV with only the clear value when I read it after engine->execute(). I have to render it into the QT metal framebuffer via the QT CommandBufferQueue. I can initialise QT to use the hydra MetalDevice but QT and hydra will use two different CommandBufferQueues. The issue with this is, that the two queues are not synchronized and I don’t see an easy way to inject metal CommandBufferQueue synchronisation into the Metal Hgi backed render setup. So I think the only way left is making an intermediate AOV texture in Hydra, which doesn’t get cleared, like the colorcorrection task is doing with the intermediateTexture. Just without the color correction. Maybe you know of an easier way to safely access the rendered AOV from another Metal CommandBufferQueue on the same MtlDevice.
@Grimmigbeisser There’s work in progress to enable multi-context render synchronization. Until that’s ready, for myself, I’ve temporarily added a constructor to HgiMetal so I can supply a CommandQueue and therefore manage synchronization. i.e.:
HGIMETAL_API
HgiMetal(id<MTLDevice> device, id<MTLCommandQueue> queue);
This is not a recommended solution, and I’m not providing the (simple) under the hood details, I’m mentioning it here in case it gives you ideas in how to solve your own synchronization issue for now.
1 Like
Hey thanks a lot for the info!