-
Notifications
You must be signed in to change notification settings - Fork 150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Core key vault firewall should not be set to "Allow public access from all networks" #4260
base: main
Are you sure you want to change the base?
Core key vault firewall should not be set to "Allow public access from all networks" #4260
Conversation
Unit Test Results0 tests 0 ✅ 0s ⏱️ Results for commit e9833c4. ♻️ This comment has been updated with latest results. |
/test 8af920d |
🤖 pr-bot 🤖 🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12660338621 (with refid (in response to this comment from @jonnyry) |
/test-extended 8af920d |
🤖 pr-bot 🤖 🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/12661150197 (with refid (in response to this comment from @jonnyry) |
core/terraform/keyvault.tf
Outdated
# | ||
resource "null_resource" "add_deployment_tag" { | ||
triggers = { | ||
always_run = timestamp() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does this always need to run? Once it's added once, it shouldn't get removed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The intention was so if the tag is removed in Azure, it will always be readded.
However as discussed, have removed the use of tags altogether, so the provisioner has been removed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can the Storage Account rules in this script be handles the same way?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes certainly - planning to have a look at storage accounts after this.
/test-destroy-env |
Destroying PR test environment (RG: rg-tre26f9d939)... (run: https://github.com/microsoft/AzureTRE/actions/runs/12669260987) |
/test 2970a5d |
🤖 pr-bot 🤖 🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12669597448 (with refid (in response to this comment from @jonnyry) |
2970a5d
to
dcb0b8f
Compare
/test 272589f |
🤖 pr-bot 🤖 🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12670289419 (with refid (in response to this comment from @jonnyry) |
/test bf9fd32 |
🤖 pr-bot 🤖 🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12670349633 (with refid (in response to this comment from @jonnyry) |
/test-destroy-env |
Destroying PR test environment (RG: rg-tre26f9d939)... (run: https://github.com/microsoft/AzureTRE/actions/runs/12670413797) |
bf9fd32
to
dcb0b8f
Compare
PR test environment destroy complete (RG: rg-tre26f9d939) |
/test dcb0b8f |
🤖 pr-bot 🤖 🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12671159667 (with refid (in response to this comment from @jonnyry) |
/test dcb0b8f |
🤖 pr-bot 🤖 🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12671848713 (with refid (in response to this comment from @jonnyry) |
CHANGELOG.md
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have time to go over this the next few days and guess @marrobi is the same. Just wanted to point out we now have 2 vaults being used from the deployer point of view.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When CMK is enabled another vault is created in the mgmt resource group
Notes on test run starting with an empty environment: KV exception added here: https://github.com/microsoft/AzureTRE/actions/runs/12671848713/job/35314921879#step:3:432
KV exception removed here: https://github.com/microsoft/AzureTRE/actions/runs/12671848713/job/35314921879#step:3:8259
|
/test 135be76 |
🤖 pr-bot 🤖 🏃 Running tests: https://github.com/microsoft/AzureTRE/actions/runs/12674163834 (with refid (in response to this comment from @jonnyry) |
Notes on test run starting with an existing TRE: KV exception added here: https://github.com/microsoft/AzureTRE/actions/runs/12674163834/job/35322577601#step:3:456
KV exception removed here:
https://github.com/microsoft/AzureTRE/actions/runs/12674163834/job/35322577601#step:3:1181 |
CHANGELOG.md
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a pending ask to enable the deployer to access resources such as these keyvaults over private network only.
- Will this make this approach obsolete?
- If not, this means we will need to do all of this in a conditional way. Right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- That would mean all TRE deployers would need to switch to private self hosted runners right? If so, yes this would be obsolete.
- If we want to support both deployment patterns - deployment from GitHub hosted runners + deployment from private self hosted runners with KV set to private networking - then yes we'd need to do it conditionally (in order to prevent the KV from being fully public).
I guess it depends on whether implementing keyvaults private networking only means switching off the ability to use Github hosted runners - is that the plan?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The goal I'm referring to that resources will be accessed via private network only which means private runners.
The question for you, is weather this PR comes from a similar place but doesn't go as far yet and just limits which public IPs can access. Or, in a situation where private agents / network is done, will you still need this method of limiting public IPs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep when the codebase switches to deployment from private runners ONLY, then this change won't be needed anymore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To clarify, in your deployment/usecase you would also like to use private runners?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm just considering most orgs that are concerned with this might want to go all the way. This solution might not be enough in those cases.
Also considering the number of changes you had to do just for this resource... next up will be the ACR, storage account and yet another keyvault in the mgmt resource group. All in all it might not be worth the investment and complexities if the end goal is anyway to go private agents...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree private runners should be the objective, but when are they scheduled to be implemented? Is this a reasonable stop gap until then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need to ensure the solution can still be deployed without private runners. Most orgs start out testing out the solution etc, and don't have the infrastructure for private runners.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My point is that we might not want to offer this middle way of opening the deployer IP in all currently defined public resources due to the complexities of supporting all the resources I mentioned above.
It might be we offer public like today and secured via private runners. Testing out could be done via the current way and if you want to be secure then it would mean you need private agents.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Supporting all resources might be too much (e.g. the ACR) but the KV is one that stands out as worthy of tightening network access on; though I understand that private runners will supersede this once implemented.
Resolves #4250
What is being addressed
PUBLIC_DEPLOYMENT_IP_ADDRESS
variable if set) during deploymentHow is this addressed
devops/scripts/kv_add_network_exception.sh
devops/scripts/kv_remove_network_exception.sh
core/terraform/deploy.sh
core/terraform/scripts/letsencrypt.sh
devops/scripts/destroy_env_no_terraform.sh
core/terraform/destroy.sh
devops/scripts/key_vault_list.sh
devops/scripts/set_contributor_sp_secrets.sh
A bug in azurerm provider was encountered which required the use of a terraform provisioner:
azurerm_key_vault
was required to work around an azurerm provider bug which means if a key vault is being re-created (it was previously soft deleted), the network acls are not updated. This can be removed when the bug is fixed, or a different workaround found.Updates since inital commit (as discussed with @marrobi):