Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature - add HashiCorp Vault secret provider #165

Merged
merged 59 commits into from
Aug 31, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
e216bc3
Feature - add HashiCorp Vault secret provider
stijnmoreels Aug 20, 2020
295028e
pr-test: don't use multiple secret paths
stijnmoreels Aug 20, 2020
c138690
pr-fix: finishing touches on renaming to single secret path
stijnmoreels Aug 20, 2020
7b1a68d
pr-build: use 'powershell' io 'ps'
stijnmoreels Aug 20, 2020
4d165ec
pr-sug: use diff ps cmd's
stijnmoreels Aug 21, 2020
a0e33e7
pr-sug: update with more logging
stijnmoreels Aug 21, 2020
766b44e
pr-sug: use diff type of dir path
stijnmoreels Aug 21, 2020
1cc3313
pr-sug: use file in '-OutFile' arg
stijnmoreels Aug 21, 2020
9f45d53
pr-sug: don't use new folder
stijnmoreels Aug 21, 2020
88567ec
pr-fix: project var
stijnmoreels Aug 21, 2020
ef14c11
pr-sug: provide more logging during downloading HashiCorp Vault
stijnmoreels Aug 21, 2020
b0a888c
pr-fix: update with blank guards for mountpoint and secretpath
stijnmoreels Aug 21, 2020
8de8563
pr-fix: provide some tracing during downloading and unzipping
stijnmoreels Aug 21, 2020
db0064c
pr-sug: use more secure vault exe retrieval
stijnmoreels Aug 21, 2020
b7aa82d
pr-fix: correct config key retrieval
stijnmoreels Aug 21, 2020
f3fbfd7
pr-sug: update with working dir
stijnmoreels Aug 21, 2020
b370365
pr-sug: give execution permissions to .exe file
stijnmoreels Aug 21, 2020
29e5881
pr-sug: redirect errors to console output
stijnmoreels Aug 21, 2020
eb7867f
pr-sug: use linux vault download
stijnmoreels Aug 21, 2020
1184e70
Merge branch 'master' into feature/hashicorp-secret-provider
stijnmoreels Aug 21, 2020
1a8a472
pr-fix: change for linux version
stijnmoreels Aug 21, 2020
3cb987c
Merge branch 'feature/hashicorp-secret-provider' of https://github.co…
stijnmoreels Aug 21, 2020
5c9cdd2
pr-sug: remove error output
stijnmoreels Aug 21, 2020
e34a430
pr-sug: update w/ more logging and async redirection
stijnmoreels Aug 21, 2020
af3e569
pr-sug: move download func to template
stijnmoreels Aug 21, 2020
007c4f3
pr-sug: add download template to Nuget release pipeline
stijnmoreels Aug 21, 2020
85cba75
pr-doc: update feature docs with HashiCorp Vault secret provider
stijnmoreels Aug 21, 2020
939c13d
pr-sug: rename to 'keyValueMountPoint' in custom config extension
stijnmoreels Aug 21, 2020
5b7af01
pr-style: restyle switch and inline var
stijnmoreels Aug 21, 2020
0e70977
Update Arcus.Security.Tests.Integration.csproj
stijnmoreels Aug 21, 2020
5e1850a
pr-sug: add mount point tests and correct '...Vault' suffix in docs
stijnmoreels Aug 21, 2020
bacd3a4
Merge branch 'feature/hashicorp-secret-provider' of https://github.co…
stijnmoreels Aug 21, 2020
d07bc15
pr-sug: add integration test for UserPass KeyValue V1
stijnmoreels Aug 21, 2020
58d9d99
pr-sug: move HashiCorp Vault version to variable template
stijnmoreels Aug 21, 2020
dcf274e
pr-sug: move integration types to respective categories
stijnmoreels Aug 21, 2020
7b78a2c
pr-sug: moving mounting to server
stijnmoreels Aug 21, 2020
6c7c593
pr-sug: use windows VM image
stijnmoreels Aug 21, 2020
a2a94c7
pr-sug: remove batch runnable script
stijnmoreels Aug 21, 2020
50d7bee
pr-sug: use more stable disposing of HashiCorp Vault
stijnmoreels Aug 24, 2020
beca954
pr-sug: add close mainwindow on disposing test server
stijnmoreels Aug 24, 2020
9f1426a
pr-sug: test with only netcoreapp3.1
stijnmoreels Aug 24, 2020
c768047
pr-sug: update with more correct disposing
stijnmoreels Aug 24, 2020
72f2620
pr-sug: force vaults to close
stijnmoreels Aug 24, 2020
af7655e
pr-sug: use status for starting server
stijnmoreels Aug 24, 2020
feaff51
pr-sug: return server
stijnmoreels Aug 24, 2020
1039965
pr-sug: update with redirect lines w/o error
stijnmoreels Aug 24, 2020
0640461
pr-sug: update with correct process starting
stijnmoreels Aug 24, 2020
048a5f9
pr-temp: only netcoreapp3.1
stijnmoreels Aug 24, 2020
1a74b9f
pr-sug: skip v1 test
stijnmoreels Aug 24, 2020
92354a6
pr-sug: update with back to Linux
stijnmoreels Aug 25, 2020
ef6c14c
pr-test: unskip v1 keyvalue secret engine userpass test
stijnmoreels Aug 25, 2020
511b30f
Merge branch 'master' into feature/hashicorp-secret-provider
stijnmoreels Aug 26, 2020
84368d4
pr-sug: use parameter input variables directly io mapping to env vari…
stijnmoreels Aug 31, 2020
2f2d129
pr-sug: use 'new' parameter input value approach & rm bash validation
stijnmoreels Aug 31, 2020
b6dd83b
pr-sug: add HashiCorp Vault feature docs link to XML docs
stijnmoreels Aug 31, 2020
10b9644
pr-add: get secret guard against blank secret name
stijnmoreels Aug 31, 2020
672f5c0
Merge branch 'feature/hashicorp-secret-provider' of https://github.co…
stijnmoreels Aug 31, 2020
1f2c167
Merge branch 'master' into feature/hashicorp-secret-provider
stijnmoreels Aug 31, 2020
54484c2
pr-style: complete header line for custom HashiCorp Vault feature docs
stijnmoreels Aug 31, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
*.user
*.userosscache
*.sln.docstates
*.local.json

# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs
Expand Down
6 changes: 6 additions & 0 deletions build/ci-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ variables:
- group: 'Arcus - GitHub Package Registry'
- group: 'Build Configuration'
- template: ./variables/build.yml
- template: ./variables/test.yml

stages:
- stage: Build
Expand Down Expand Up @@ -108,6 +109,11 @@ stages:
inputs:
packageType: 'sdk'
version: '$(DotNet.Sdk.VersionBC)'
- template: 'templates/download-hashicorp-vault.yml'
parameters:
targetFolder: '$(Build.SourcesDirectory)'
version: $(HashiCorp.Vault.Version)
vaultBinVariableName: 'Arcus.HashiCorp.VaultBin'
- template: test/run-integration-tests.yml@templates
parameters:
dotnetSdkVersion: '$(DotNet.Sdk.Version)'
Expand Down
6 changes: 6 additions & 0 deletions build/nuget-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ variables:
- group: 'Arcus - GitHub Package Registry'
- group: 'Build Configuration'
- template: ./variables/build.yml
- template: ./variables/test.yml
- name: 'Package.Version'
value: ${{ parameters['Package.Version'] }}

Expand Down Expand Up @@ -94,6 +95,11 @@ stages:
inputs:
packageType: 'sdk'
version: '$(DotNet.Sdk.VersionBC)'
- template: 'templates/download-hashicorp-vault.yml'
parameters:
targetFolder: '$(Build.SourcesDirectory)'
version: $(HashiCorp.Vault.Version)
vaultBinVariableName: 'Arcus.HashiCorp.VaultBin'
- template: test/run-integration-tests.yml@templates
parameters:
dotnetSdkVersion: '$(DotNet.Sdk.Version)'
Expand Down
42 changes: 42 additions & 0 deletions build/templates/download-hashicorp-vault.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
parameters:
stijnmoreels marked this conversation as resolved.
Show resolved Hide resolved
- name: targetFolder
type: string
default: '$(Build.SourcesDirectory)'
- name: version
type: string
- name: vaultBinVariableName
type: string
default: 'Arcus.HashiCorp.VaultBin'

steps:
- powershell: |
$vault_zip = "vault_${{ parameters.version }}_linux_amd64.zip"
$vault_url = "https://releases.hashicorp.com/vault/${{ parameters.version }}/$vault_zip"
$destination = "${{ parameters.targetFolder }}/$vault_zip"
if (!(Test-Path $destination)) {
Write-Output "Downloading $vault_url to $destination"
[Net.ServicePointManager]::SecurityProtocol = 'Tls12'
Invoke-WebRequest -Uri $vault_url -OutFile $vault_zip
ls
if (Test-Path $vault_zip) {
Write-Output "Downloaded .zip file to $vault_zip"
} else {
Write-Error "Could not find downloaded .zip file $vault_zip"
}
}
Expand-Archive -LiteralPath $vault_zip -DestinationPath ${{ parameters.targetFolder }}
ls
$vault_bin = "${{ parameters.targetFolder }}/vault"
if (Test-Path $vault_bin) {
Write-Output "Extracted HashiCorp Vault to executable file"
} else {
Write-Error "Could not find extracted HashiCorp Vault executable file"
}
Write-Host "##vso[task.setvariable variable=${{ parameters.vaultBinVariableName }}]$vault_bin"
workingDirectory: ${{ parameters.targetFolder }}
displayName: 'Download HashiCorp Vault'
- bash: |
chmod +x $VAULT_BIN
env:
VAULT_BIN: '${{ parameters.targetFolder }}/vault'
displayName: 'Make HashiCorp Vault executable runnable'
2 changes: 2 additions & 0 deletions build/variables/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
variables:
HashiCorp.Vault.Version: 1.5.0
8 changes: 6 additions & 2 deletions docs/preview/features/secret-store/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,13 @@ Once register, you can fetch all secrets by using `ISecretProvider` which will g
## Built-in secret providers
Several built in secret providers available in the package.

* [Environment variables](./../../features/secret-store/provider/environment-variables)
* [Configuration](./../../features/secret-store/provider/configuration)
* [Azure key vault](./../../features/secret-store/provider/key-vault)
* [Environment variables](./../../features/secret-store/provider/environment-variables)

And several additional providers in seperate packages.

* [Azure Key Vault](./../../features/secret-store/provider/key-vault)
* [HashiCorp](./../../features/secret-store/hashicorp-vault)
* [User Secrets](./../../features/secret-store/provider/user-secrets)

If you require an additional secret providers that aren't available here, please [this document](./../../features/secret-store/create-new-secret-provider) that describes how you can create your own secret provider.
Expand Down
100 changes: 100 additions & 0 deletions docs/preview/features/secret-store/provider/hashicorp-vault.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
title: "HashiCorp Vault secret provider"
layout: default
---

# HashiCorp Vault secret provider
HashiCorp Vault secret provider brings secrets from the KeyValue secret engine to your application.

## Installation
Adding secrets from HashiCorp Vault into the secret store requires following package:

```shell
PM > Install-Package Arcus.Security.Providers.HashiCorp
```

## Configuration
After installing the package, the addtional extensions becomes available when building the secret store.

```csharp
public class Program
{
public static void Main(string[] args)
{
CreateHostBuilder(args).Build().Run();
}

public static IHostBuilder CreateHostBuilder(string[] args)
{
return Host.CreateDefaultBuilder(args)
.ConfigureSecretStore((context, config, builder) =>
{
// Adding the HashiCorp Vault secret provider with the built-in overloads.
// =======================================================================

// UserPass authentication built-in overload:
// ------------------------------------------
builder.AddHashiCorpVaultWithUserPass(
// URI where the HashiCorp Vault is running.
vaultServerUriWithPort: "https://uri.to.your.running.vault:5200",
// Username/Password combination to authenticate with the vault.
username: "admin",
password: "s3cr3t",
// Path where the secrets are stored in the KeyValue secret engine.
secretPath: "my-secrets"
);

// Following defaults can be overridden:

// Mount point of UserPass athentication (default: userpass).
builder.AddHashiCorpVaultWithUserPass(..., userPassMountPoint: "myuserpass");

// Version of the KeyValue secret engine (default: V2).
builder.AddHashiCorpVaultWithUserPass(..., keyValueVersion: VaultKeyValueSecretEngineVersion.V1);

// Mount point of KeyValue secret engine (default: kv-v2).
builder.AddHashiCorpVaultWithUserPass(..., keyValueMountPoint: "secret");

// Kubernetes authentication built-in overload:
// --------------------------------------------
builder.AddHashiCorpVaultWithKubernetes(
// URI where the HashiCorp Vault is running.
vaultServerUriWithPort: "https://uri.to.your.running.vault:5200",
// Role name of the Kubernetes service account.
roleName: "admin",
// JSON web token (JWT) of the Kubernetes service account,
jwt: "ey.xxx.xxx",
// Path where the secrets are stored in the KeyValue secret engine.
secretPath: "my-secrets"
);

// Mount point of Kubernetes authentication (default: kubernetes).
builder.AddHashiCorpVaultWithKubernetes(..., kubernetesMountPoint: "mykubernetes");

// Version of the KeyValue secret engine (default: V2).
builder.AddHashiCorpVaultWithKubernetes(..., keyValueVersion: VaultKeyValueSecretEngineVersion.V1);

// Mount point of KeyValue secret engine (default: kv-v2).
builder.AddHashiCorpVaultWithKubernetes(..., keyValueMountPoint: "secret");

// Custom settings overload for when using the [VaultSharp](https://github.com/rajanadar/VaultSharp) settings directly:
// --------------------------------------------------------------------------------------------------------------------
var tokenAuthentication = new TokenAuthMethodInfo("token");
var settings = VaultClientSettings("http://uri.to.your.running.vault.5200", tokenAuthentication);
builder.AddHashiCorpVault(
settings,
// Path where the secrets are stored in the KeyValue secret engine.
secretPath: "my-secrets");

// Version of the KeyValue secret engine (default: V2).
builder.AddHashiCorpVault(..., keyValueVersion: VaultKeyValueSecretEngineVersion.V1);

// Mount point of KeyValue secret engine (default: kv-v2).
builder.AddHashiCorpVault(..., keyValueMountPoint: "secret");
})
.ConfigureWebHostDefaults(webBuilder => webBuilder.UseStartup<Startup>());
}
}
```

[&larr; back](/)
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFrameworks>netstandard2.0</TargetFrameworks>
<Authors>Arcus</Authors>
<Description>Provides support for HashiCorp</Description>
<Copyright>Copyright (c) Arcus</Copyright>
<PackageLicenseUrl>https://github.com/arcus-azure/arcus.security/blob/master/LICENSE</PackageLicenseUrl>
<PackageProjectUrl>https://github.com/arcus-azure/arcus.security</PackageProjectUrl>
<PackageIconUrl>https://raw.githubusercontent.com/arcus-azure/arcus/master/media/arcus.png</PackageIconUrl>
<RepositoryUrl>https://github.com/arcus-azure/arcus.security</RepositoryUrl>
<RepositoryType>Git</RepositoryType>
<PackageTags>HashiCorp;OSS;Security</PackageTags>
<AssemblyName>Arcus.Security.Providers.HashiCorp</AssemblyName>
<RootNamespace>Arcus.Security.Providers.HashiCorp</RootNamespace>
<PackageId>Arcus.Security.Providers.HashiCorp</PackageId>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="VaultSharp" Version="1.4.0.5" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\Arcus.Security.Core\Arcus.Security.Core.csproj" />
</ItemGroup>

</Project>
Loading