Add an orchestrator for az cleanroom CCF networks #286
+270
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Begins to address #265, which found that the az cleanroom extension doesn't automatically maintain the health of a CCF network.
The solution which starts with this PR involves a container image which will run, monitoring the health of the network and calling the extensions
update
function when nodes become unhealthyThe
update
function retires unhealthy nodes and provisions new nodes to maintain a desired number.Limitations
This implementation currently only works if the orchestrator container is run on the same host that deployed the CCF network initially, as it accesses "sibling" az cleanroom client containers from the host and mounts credentials from the host.
Eventually both of these limitations will be addressed, by properly handling credentials spinning up independent client containers