How To Debug Your Plugin

Contents

1. Debugging Basics
2. Debugging Plugins With Visual Studio
3. Debugging Plugins With Xcode

Debugging Basics

Debugging plugin code requires a few additional steps that are usually not necessary when debugging an application directly. The main reason for this is that the plugin is never executed directly, but loaded at some point at runtime by a host process.

Thus a plugin is never debugged directly, but instead the debugger needs to be attached to the host process, which will then be able to apply any breakpoint settings and allow proper debugging.

This slight increase in debugging difficulty is typical and expected when debugging runtime modules (see also: Steps necessary to debug a VST in digital audio workstations).

The exact steps to debug a plugin with OBS Studio differ between platforms but boil down to this:

1. Prepare a local build of OBS Studio built in "Debug" configuration
2. Configure the plugin build system to use "Debug" configuration
3. Either setup the OBS Studio IDE project or local system to load plugins from custom locations
   • Alternatively the plugin can be copied manually into the plugin destination for the local platform
4. Launch OBS Studio with a debugger attached

Note

This guide does not contain any guidelines for debugging a plugin on Ubuntu. We encourage maintainers to add this information to the Wiki as well.

Debugging Plugins With Visual Studio

Developers have two choices as far as debugging their plugin in Visual Studio is concerned:

1. Launch OBS Studio compiled in "Debug" configuration by a separate Visual Studio project, or
2. Add the plugin project to the OBS Studio project and launch OBS Studio directly

Both variants have their own benefits and drawbacks:

1. Is the easiest to set up but will limit the developer to directly browse plugin source code only
2. Is more complex to set up but will allow browsing the plugin source code as well as OBS Studio source code in a single project

Warning

If the Visual Studio solutions are generated by CMake, be aware that re-configuring the project will also make CMake re-build the Visual Studio solution and thus the custom debugging setting might be overwritten with defaults.

Variant 1: Launch and debug plugin with OBS Studio host

This variant requires the developer to manually adjust their plugin solution in Visual Studio to set up a custom command that should be executed for debugging.

1. Open plugin solution in Visual Studio
2. Open the properties of the solution
3. Under "Common Properties", change the "Single Startup Project" under "Configure Startup Projects" to your actual plugin project (it will probably be set to CMake's "ALL_BUILD" meta project by default)

The name of the plugin project should now appear in bold letters in the "Solution Explorer"

4. Next, open the properties for the plugin project
5. Under "Debugging", change the following values:
   • For "Command" browse to the Debug rundir of the separate OBS Studio project. This should commonly be in a sub-directory build_x64\rundir\Debug of the separate OBS Studio checkout that was built in "Debug" configuration at least once. Select obs64.exe from this directory
   • For "Working Directory", chose the same rundir as above
   • For "Environment" set up the environment variables OBS_PLUGINS_PATH and OBS_PLUGINS_DATA_PATH to point to the plugin's Debug rundir. This should also be in a sub-directory build_x64\rundir\Debug of the plugin's checkout directory.
     • Use the pattern VARIABLE=VALUE, with each variable on its own line

6. Finally, just click "Local Windows Debugger" in the toolbar to run OBS Studio and start debugging the plugin:

This setup allows setting up breakpoints in any part of the plugin code and even allows following call stacks back into OBS Studio code. Because a locally compiled version of OBS Studio is used to launch the plugin, which allows Visual Studio to access the debug symbols and thus source code of both OBS Studio as well as the plugin.

Note

In some situations breakpoints set at a very early stage of a plugin's "lifetime" might not be triggered if the Visual Studio debugger takes too long to attach itself to OBS Studio while it's launching. This is an unfortunate side-effect of the way this process works in Visual Studio. If this happens to be the case, one can try variant 2 to fix this.

Variant 2: Add Plugin To OBS Studio Project

The second variant requires adding the plugin and plugin-support projects to an existing OBS Studio solution and setting up the environment variables to allow OBS Studio to load the plugin from its own rundir:

1. Open an existing OBS Studio solution
2. Add the plugin project and plugin-support project to the solution. The "Solution Explorer" should look similar to this:

3. Open the "Debugger" properties of the frontend\obs-studio project and change the "Environment" property
   • Set up the environment variables OBS_PLUGINS_PATH and OBS_PLUGINS_DATA_PATH to point to the plugin's Debug rundir. This should also be in a sub-directory build_x64\rundir\Debug of your plugin's checkout directory.
   • Use the pattern VARIABLE=VALUE, with each variable on its own line
4. Click "Local Windows Debugger" in the toolbar to run OBS Studio and start debugging the plugin.

This setup allows setting breakpoints in OBS Studio code as well as plugin code, as both code bases are present within the solution.

Note

If desired, the plugin project can be added as a dependency to the obs-studio project in the solution properties. This would make Visual Studio automatically re-compile the plugin if any source code changes were detected.

Debugging Plugins With Xcode

Xcode allows developers to either debug their plugin directly (by launching it via a pre-compiled OBS Studio from another local Xcode project) or to add the plugin project to an existing OBS Studio project which allows adding the project as a dependency to the obs-studio target.

Tip

Ensure that both projects have identical codesigning settings to ensure that macOS will not block loading the plugin if is not signed correctly.

Warning

There is an open Pull-Request for OBS Studio to fix the broken functionality of loading plugins from paths provided via environment variables. Until that bug is fixed, step #X below will have no effect and instead the generated plugin bundle needs to be copied from its rundir to ~/Library/Application Support/obs-studio/plugins before debugging.

Variant 1: Launch and debug plugin with OBS Studio host

1. Open the plugin's Xcode project
2. Select the current target from the status area at the top of the window and select "Edit Scheme..."

3. In the "Info" tab, change the "Executable" by selecting "Other.." and then choosing OBS.app from a local OBS Studio project.
   • The app bundle can be found in the build_macos/UI/Debug sub-directory of an OBS Studio checkout

4. In the "Arguments" tab, add the "Environment Variables" OBS_PLUGINS_PATH and OBS_PLUGINS_DATA_PATH and change their values to point to the plugin's Debug rundir. This should be in a sub-directory build_macos\rundir\Debug of the plugin's checkout directory.

5. Click the play button or press CMD+R to start debugging the plugin

As both the plugin and OBS Studio are built in "Debug" configuration on the same machine, the debugger has no issue to read the debug symbols contained in all binaries and associate them with the source code available on the local machine.

Variant 2: Add The Plugin As A Sub-Project

Xcode projects can be added to any other existing project and will retain their internal structure and hierarchy. Their schemes are also added to the current project, which gives the impression of a "merged" project that contains all elements of both original projects.

0. IMPORTANT: Ensure that neither the OBS Studio nor plugin Xcode projects are currently opened by Xcode
1. Open an existing OBS Studio Xcode project
2. Add files to the project via File -> Add Files to "obs-studio"
3. Browse to the plugin repository and select the associated Xcode project from the build_macos directory
4. Choose "Reference files in place" in the following dialog and click "Finish".
5. Edit the Scheme for obs-studio and add the environment variables OBS_PLUGINS_PATH and OBS_PLUGINS_DATA_PATH in the "Arguments" tab, setting their values to point to the plugin's Debug rundir. This should be in a sub-directory build_macos\rundir\Debug of the plugin's checkout directory.
6. Now the obs-studio target can be ran and debugged with full access to both project's source code and breakpoints can be freely set in both.

Note

To enable automatic re-compilation of the plugin if any of its source code has been changed, add the plugin target to the "Target Dependencies" of the obs-studio target. Refer to the Xcode documentation for more information on this.