Andaharoo
(Andrew)
November 20, 2024, 10:58pm
1
We ran asan (address sanitizer) on a simple application show below. With USD 24.08 and 24.11.
#include <fstream>
#include <iostream>
#include <string>
#include "UsdIncludeStart.hpp"
#include "pxr/base/plug/plugin.h"
#include "pxr/base/plug/registry.h"
#include "pxr/usd/usd/stage.h"
#include "UsdIncludeEnd.hpp"
#include "gtest/gtest.h"
void loadPlugins()
{
std::string usd_path_str = <our plugin path>;
pxr::PlugRegistry& registry = pxr::PlugRegistry::GetInstance();
registry.RegisterPlugins(usd_path_str + "/ar/resources");
registry.RegisterPlugins(usd_path_str + "/ndr/resources");
registry.RegisterPlugins(usd_path_str + "/sdf/resources");
registry.RegisterPlugins(usd_path_str + "/usd/resources");
registry.RegisterPlugins(usd_path_str + "/usdGeom/resources");
registry.RegisterPlugins(usd_path_str + "/usdLux/resources");
registry.RegisterPlugins(usd_path_str + "/usdPhysics/resources");
registry.RegisterPlugins(usd_path_str + "/usdProc/resources");
registry.RegisterPlugins(usd_path_str + "/usdShade/resources");
registry.RegisterPlugins(usd_path_str + "/usdSkel/resources");
registry.RegisterPlugins(usd_path_str + "/usdUI/resources");
}
TEST(RawUSDTest, TestLoad)
{
const char* scene_str = R"(#usda 1.0
def Xform "hello"
{
def Sphere "world"
{
}
}
)";
// Load all the plugins to registry
loadPlugins();
std::ofstream out;
out.open("usd_test.usda");
ASSERT_TRUE(out.is_open()) << "Failed to write usd string to file";
out << scene_str << std::endl;
out.close();
pxr::UsdStageRefPtr stage = pxr::UsdStage::Open("usd_test.usda");
ASSERT_NE(stage, nullptr);
}
This gives us:
=================================================================
==12==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 48 byte(s) in 2 object(s) allocated from:
#0 0x7dabfa4b4c74 in operator new(unsigned long) (<redacted>/libasan.so.8+0xb4c74)
#1 0x7dabf93d26e3 in pxrInternal_v0_24_11__pxrReserved__::Sdf_IdentityRegistry::Identify(pxrInternal_v0_24_11__pxrReserved__::SdfPath const&) (<redacted>/libusd_sdf.so+0x3d26e3)
SUMMARY: AddressSanitizer: 48 byte(s) leaked in 2 allocation(s).
================================================================================
Any idea as to why?
dhruvgovil
(Dhruv Govil)
November 21, 2024, 3:19am
2
Can you try clearing the stage cache? It sometimes shows up as a leak
Andaharoo
(Andrew)
November 21, 2024, 4:02am
3
Not super familiar with the cache here.
Reading some documentation I would need to set one up, then clear it afterwards?
I’m guessing context is just an object to set the cache as active for the lifetime of that object? Is there some default cache?
I gave the following a try, but didn’t work.
pxr::UsdStageCache cache;
pxr::UsdStageCacheContext context(cache);
pxr::UsdStageRefPtr stage = pxr::UsdStage::Open("my file");
stageCache.Clear();
Andaharoo
(Andrew)
November 21, 2024, 4:27am
4
Found related Issue:
opened 11:56AM - 16 May 24 UTC
### Description of Issue
We are seeing a strange memory leak report when runn… ing a binary compiled with Clang's address sanitizer linked with USD v23.11. That is, our code is compiled with address sanitizer enabled, while USD is compiled in release mode using its own script and gcc 11. The error from ASAN is:
```
==1591245==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 1544 byte(s) in 1 object(s) allocated from:
#0 0x39ff87 in operator new(unsigned long)
#1 0x7f54cf89fcd6 in std::vector<__gnu_cxx::_Hashtable_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::list<pxrInternal_v0_23__pxrReserved__::(anonymous namespace)::Tf_RegistryManagerImpl::_RegistrationValue, std::allocator<pxrInternal_v0_23__pxrReserved__::(anonymous namespace)::Tf_RegistryManagerImpl::_RegistrationValue> > > >*, std::allocator<__gnu_cxx::_Hashtable_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::list<pxrInternal_v0_23__pxrReserved__::(anonymous namespace)::Tf_RegistryManagerImpl::_RegistrationValue, std::allocator<pxrInternal_v0_23__pxrReserved__::(anonymous namespace)::Tf_RegistryManagerImpl::_RegistrationValue> > > >*> >::reserve(unsigned long) (.constprop.0) registryManager.cpp
```
From that I gather that the Tf_RegistryManagerImpl's _registrationFunctions member is not freed, which is strange, because the singleton instance itself is freed. The leak is always reproducible and doesn't require calling any USD functions beyond the DSO's initialisation.
### Steps to Reproduce
1. Run any binary compiled with Clang and address sanitizer, linked with USD v23.11 on Rocky 8.
### System Information (OS, Hardware)
Rocky Linux 8
### Package Versions
USD: v23.11
Compiler used for USD: gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
Compiler used for our code: clang version 14.0.6
### Build Flags
build_usd.py --build-variant=release --materialx --no-prman --no-tutorials --no-examples --no-docs --no-tools --no-python --use-cxx11-abi=1
Edit: fixed formatting
Someones fix:
PixarAnimationStudios:dev
← moshev:issue3088
opened 07:33AM - 20 Jun 24 UTC
### Description of Change(s)
Delete the dynamically allocated Tf_RegistryMana… gerImpl singleton instance when the static TfRegistryManager singleton instance which uses it is destroyed.
### Fixes Issue(s)
- 3088
<!--
Please follow the Contributing and Building guidelines to run tests against your
change. Place an X in the box if tests are run and are all tests passing.
-->
- [X] I have verified that all unit tests pass with the proposed changes
<!--
Place an X in the box if you have submitted a signed Contributor License Agreement.
A signed CLA must be received before pull requests can be merged.
For instructions, see: http://openusd.org/release/contributing_to_usd.html
-->
- [X] I have submitted a signed Contributor License Agreement
I guess the real problem is no one has figured out yet why this is being held onto. We shouldn’t need a clear?
dhruvgovil
(Dhruv Govil)
November 21, 2024, 2:57pm
5
Ah yeah good find. I forgot about that.
Yes, I think its unclear what is causing it to be held on to.
Andaharoo
(Andrew)
November 21, 2024, 7:55pm
6
Looks like removing the dead count check in pxr/usd/sdf/identity.cpp fixes it. We’re guessing the threshold is used for performance reasons. But believe this is a “safe” temporary fix.
}
TfAutoMallocTag2 tag("Sdf", "Sdf_IdentityRegistry::Identify");
rawId = new Sdf_Identity(this, path);
_ids[path] = rawId;
_deadThreshold = std::max(_MinDeadThreshold, _ids.size() / 8);
return Sdf_IdentityRefPtr(TfDelegatedCountIncrementTag, rawId);
}
void UnregisterOrDelete() {
if (++_deadCount >= _deadThreshold) {
// Clean house!
_deadCount = 0;
tbb::spin_mutex::scoped_lock lock(_idsMutex);
for (auto iter = _ids.begin(); iter != _ids.end();) {
if (iter->second->_refCount == 0) {
delete iter->second;
iter = _ids.erase(iter);
}
else {
spiff
(F Sebastian Grassia)
November 25, 2024, 1:09am
7
SdfIdentities are useful for interactive authoring workflows (like in Presto), but are a notable performance drain, and yes, the deadCount is an attempt to mitigate that in some of our workflows.
We have put a fari bit of effort into avoiding the creation of Identities in USD read-only codepaths… if you can identify where your example is creating them and report it, it’s something we’d be eager to look into, rather than short-circuiting the leaky optimization. Btw, OpenUSD creates a rather substantial number of global, long-term caches that will be considered leaks…