Remove SdfTextFileFormat Plugin

Community USD
1

Hi everyone,

I’m currently working on developing a FileFormat plugin for a proprietary, in-house file format that uses the “.sdf” extension. However, I’ve run into a bit of a snag because this extension conflicts with the existing SdfTextFileFormat that’s included in USD.

To work around this issue, I’ve considered simply deleting the ‘plugin.json’ file associated with the SdfTextFileFormat plugin. In my preliminary tests, removing this file doesn’t seem to cause any immediate problems, but I’m concerned about potential hidden dependencies or unforeseen consequences down the line.

Has anyone here dealt with a similar situation or have insights into the dependencies that might be tied to the SdfTextFileFormat plugin in USD? I want to ensure that this workaround won’t disrupt any other USD functionalities or cause issues in the larger ecosystem.

Thanks in advance for your guidance and suggestions!

Best,
Nick

2

Hi @Nick , sorry about that, and we think we should probably change that file extension, as it is meant to be an internal detail, and sdf is just pretty overloaded in CG… Would you file a GitHub Issue about it?

But until we can get to that… unfortunately we identified a problem with your approach: SdfLayer::CreateAnonymous(), a very commonly called method, when called with no tag argument that includes a file extension, defaults to the .sdf extension. When we deleted the entry from plugInfo.json locally, CreateAnonymous() failed. In your build, it will attempt to create an in-memory layer backed by your internal format, which, unless it is “complete”, feature-wise, for reading and writing, with the usd data model, is probably going to cause unexpected behavior in a variety of scenarios.

We think that you should be able to more robustly work around this by writing a URI resolver whose extension is (e.g.) .sdf_Nick, which means the paths you’d author in your USD assemblies would possess that extension. Your resolver would preserve that extension in the “resolved path” it produces (because that is what Sdf uses to look up the SdfFIleFormat plugin), but your resolver’s OpenAsset method would know to strip off the “_Nick” before actually attempting to open the file into an ArAsset.

Please let us know if that doesn’t make sense or you run into problems.

3

Hey @spiff

Thanks for the advice on the “.sdf” file extension issue and the idea about using a URI resolver. I get the problem with the current setup and why it’s not great to stick with “.sdf”. I’ll go ahead and report this issue on GitHub and start looking into how a URI resolver works to see if that can fix things. I appreciate your help and will let you know how it goes.

4

Hey @spiff ,

I’d like to run my approach by you to make sure I’m on the right track.

  1. Adding Reference: I plan to add a reference by appending _nick to the asset path. This seems straightforward, but how does say >>usdview my_file.sdf resolve this? Or would the user need to know to append _nick ?

  2. Custom SdfFileFormat Plugin: Next, I’ll register my custom SdfFileFormat plugin and use the sdf_nick extension.

  3. Custom SdfArResolver Plugin: Finally, I’m registering my custom SdfArResolver plugin, also with the sdf_nick extension. A couple of questions come to mind here:

    • Is it possible to register a URI resolver specifically by file extension? From what I’ve seen, registration seems more focused on URI schemes.

    • I believe my simple implementation would only require that I override the _OpenAsset method?

I appreciate any insights or suggestions you might have. Thanks in advance for your help! This is my first look into AR.

Nick

5
  1. usdview does resolve the path from the command line, though it reminds me that not all DCC’s necessarily do, so that could be an issue. Yes, the user would need to know to append _nick, but usdview was actually designed to be easily “wrapped”. Check out the Launcher class in usdviewq, and how the usdview binary (very simplisticly) uses it. At Pixar, we actually publish two other “usdview apps” called modelview and shotview that create Launchers and do some special pre-configuration of the asset resolver to allow users to type “just a model name” on the command line, and we’ll do some searching based on the currently configured show, etc, to find it. You could do something similar that would automatically append _nick. It’s a usdview-specific solution, though, and won’t work for usdcat, unfortunately.
  2. Right.
    1. Unfortunately, no. The only way to avoid additionally having to use a URI prefix (which I forgot to mention) would be for you to register a “main” resolver in your build. The ArDefaultResolver is designed to be derivable, so if that’s what you are using now, you should be able to create your own with the one extra behavior fairly easily.
    2. In addition, you will need to override Resolve() so that it checks to see if the asset with _nick stripped actually exists, rather than testing the full path. Otherwise none of your asset paths will resolve. There may be one or two similar pieces of API…
6

Hey @spiff ,

I am considering a new approach, how about if I simply modify the existing SdfTextFileFormat plugin, such that I run some checks to determine what type of sdf file I have and return a layer appropriately.

Nick

1 Like
7

Hah - that’s more than feasible, in fact that’s exactly what we do with the UsdFileFormat (because the usd extension can be used with either the usda or usdc file formats, so you even have a working example!

And that will be less (and more contained) code to rip out when we address the Issue you filed. NIce thinking, @Nick !

8

Sorry to open this topic back up, but I finally got around to implementing my custom SDF File Format plugin and I wanted to share my approach. I also am looking for any gotchas.

I built my File Format plugin like any other except I inherited from the SdfTextFileFormat class.

class UsdSdfFileFormat : public SdfTextFileFormat {
public:

    // SdfTextFileFormat overrides.
    virtual bool CanRead(const std::string &file) const override;
    virtual bool Read(SdfLayer* layer,
                      const std::string& resolvedPath,
                      bool metadataOnly) const override;

protected:
    SDF_FILE_FORMAT_FACTORY_ACCESS;

    virtual ~UsdSdfFileFormat();
    UsdSdfFileFormat();
    SDF_API
    explicit UsdSdfFileFormat(const TfToken& formatId,
                               const TfToken& versionString = TfToken(),
                               const TfToken& target = TfToken());

private:
    bool _ReadFromStream(SdfLayer *layer,
                         std::istream &input,
                         bool metadataOnly,
                         std::string *outErr) const;

    bool _IsSdfRcsFormat(const std::string& filepath) const;
};

I removed the plugInfo.json at \lib\usd\sdf\resources from my USD build. My current plugInfo.json is:

 {
    "Plugins": [
        {
            "Info": {
                "Types": {
                    "UsdSdfFileFormat": {
                        "bases": [
                            "SdfFileFormat"
                        ],
                        "displayName": "Modified SDF Text File Format Plugin",
                        "extensions": [
                            "sdf"
                        ],
                        "formatId": "sdf",
                        "primary": true,
                        "target": "usd"
                    }
                }
            },
            "LibraryPath": "@PLUG_INFO_LIBRARY_PATH@",
            "Name": "usdSdf",
            "ResourcePath": "@PLUG_INFO_RESOURCE_PATH@",
            "Root": "@PLUG_INFO_ROOT@",
            "Type": "library"
        }
    ]
}

Everything seems to be functioning correctly in my tests.

Do I need the SdfMetadata tag? The SdfFileFormat type flag?
From the default plugInfo.json:

{
    "Plugins": [
        {
            "Info": {
                "SdfMetadata": {
                    "payloadAssetDependencies": {
                        "appliesTo": "prims",
                        "displayGroup": "Pipeline",
                        "type": "asset[]"
                    }
                },
                "Types": {
                    "SdfFileFormat": {
                        "displayName": "Sdf file format base class",
                        "target": "sdf"
                    },
                    "SdfTextFileFormat": {
                        "bases": [
                            "SdfFileFormat"
                        ],
                        "displayName": "Sdf Text File Format",
                        "extensions": [
                            "sdf"
                        ],
                        "formatId": "sdf"
                    }
                }
            },
            "LibraryPath": "../../usd_sdf.dll",
            "Name": "sdf",
            "ResourcePath": "resources",
            "Root": "..",
            "Type": "library"
        }
    ]
}

Thanks,
Nick

9

Hi @Nick , I think you can/should leave the SdfMetadata and SdfFileFormat entries alone in pxr/sdf, and elide them from you new module’s plugInfo.

While I don’t think it will land in time for the next release, we are hoping to tackle the Issue you filed soon, obviating the need for this workaround. Stay tuned!

–spiff

10

Thanks @spiff , with some experimentation I found that moving SdfMetadata and SdfFileFormat tags to my new module’s pluginfo seems to work. If I left SdfFileFormat in the pxr/sdf pluginfo my tests fail.

In any case happy to hear that the issue is getting addressed.

Thanks,
Nick