Secrets management

[nferch]

Context

Several types of secrets:

• Tokens for third-party SaaS systems used for app, platform, and SDK development.
• Credentials for external APIs such as the agent hosting system.
• User/identity-specific secrets, such as Github Personal Access Tokens

Current state

• More sensitive secrets are read from environment variables at bundle, less sensitive secrets are embedded into DXOS Config.
• Saas and API credentials are baked into the CLI prior to publishing to NPM
• Saas and API credentials are bundled alongside the application code, and served over HTTP
• These secrets can be trivially recovered
• There is little provision for secret rotation other than republishing

Ideal state

• Secrets can be configured at runtime (i.e. without needing the full source code repository or needing to build the monorepo), by the app operator, who may be a different entity than the app developer
• Secrets can be rotated without having to rebuild or republish the application
• Secrets management is generic enough to handle other use cases for developers building on top of the SDK

Options (not mutually exclusive)

• Encrypt the secrets at build time and decrypt them using a key that is easier to change or loaded dynamically
• Proxy SaaS services, injecting the provider API token and authenticating requests through a separate mechanism. This proxy can also operate as a CORS proxy for SaaS services with limited CORS proxy configurability
• Build or leverage an existing secrets management system to load the secrets at runtime

[wittjosiah]

Worth noting that HALO itself is likely a solution for some of this. The "User/identity-specific secrets" end of the spectrum, for storing personal credentials.

[nferch]

Agree. Although that has its own set of challenges for managing its own secrets and recovery. Hopefully we can find some solution that composes well with existing solutions in a manner that is easy for users to use.