Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OCPBUGS-42810: actively move bootstrap member lead #1369

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

OCPBUGS-42810: actively move bootstrap member lead #1369

wants to merge 1 commit into from

Conversation

tjungblu
Copy link
Contributor

This PR will actively try to move the leadership away from the bootstrap member to another healthy member.

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 19, 2024
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Nov 19, 2024
@openshift-ci-robot
Copy link

@tjungblu: This pull request references Jira Issue OCPBUGS-42810, which is invalid:

  • expected the bug to target the "4.18.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This PR will actively try to move the leadership away from the bootstrap member to another healthy member.

/hold

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from dusk125 and Elbehery November 19, 2024 13:22
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Nov 19, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tjungblu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 19, 2024
@tjungblu tjungblu force-pushed the OCPBUGS-42810_moveleader branch 2 times, most recently from 5720e20 to a747960 Compare November 20, 2024 13:08
@tjungblu
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade 10

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Nov 20, 2024

@tjungblu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/ce14d7c0-a741-11ef-9f14-d84ba44f8a80-0

@tjungblu tjungblu force-pushed the OCPBUGS-42810_moveleader branch from a747960 to 0c031aa Compare November 20, 2024 13:25
@tjungblu
Copy link
Contributor Author

/retest

1 similar comment
@tjungblu
Copy link
Contributor Author

/retest

@tjungblu
OCPBUGS-42810: actively move bootstrap member lead
369b1cd 
This PR will actively try to move the leadership away from the bootstrap member to another healthy member.

Signed-off-by: Thomas Jungblut <[email protected]>
@tjungblu tjungblu force-pushed the OCPBUGS-42810_moveleader branch from 0c031aa to 369b1cd Compare November 28, 2024 13:06
@tjungblu
Copy link
Contributor Author

/cherry-pick release-4.18

@openshift-cherrypick-robot
Copy link

@tjungblu: once the present PR merges, I will cherry-pick it on top of release-4.18 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@tjungblu
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade 10

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Nov 28, 2024

@tjungblu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/01c7f930-ada0-11ef-8674-28fc71df2147-0

@tjungblu
Copy link
Contributor Author

/retest-required

@tjungblu
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade 10

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Nov 28, 2024

@tjungblu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/e6fa4930-adab-11ef-8910-6373b3a73b44-0

@tjungblu
Copy link
Contributor Author

tjungblu commented Dec 2, 2024

/payload-aggregate periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade 10

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 2, 2024

@tjungblu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/37cd8410-b0a6-11ef-8f86-550f1a2ac5ba-0

@tjungblu
Copy link
Contributor Author

tjungblu commented Dec 3, 2024

/retest

/payload-aggregate periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade 10

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 3, 2024

@tjungblu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.18-e2e-azure-ovn-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/63892fc0-b180-11ef-99d0-e245fa792cd0-0

lance5890
lance5890 reviewed Dec 6, 2024
View reviewed changes
pkg/operator/bootstrapteardown/bootstrap_teardown_controller.go
err = c.removeBootstrap(timeoutCtx, safeToRemoveBootstrap, hasBootstrap, bootstrapID)
if hasBootstrap {
if err := c.ensureBootstrapIsNotLeader(ctx, bootstrapMember); err != nil {
klog.Errorf("error while ensuring bootstrap is not leader: %v", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we return err here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not really, I wouldn't want the controller to degrade when that operation fails once. Mind you, this happens very very rarely and affects our CI pass rates in very few cases of hundreds.

@vrutkovs
Copy link
Member

vrutkovs commented Jan 8, 2025

/retest

@vrutkovs
Copy link
Member

vrutkovs commented Jan 8, 2025

/test e2e-metal-ovn-ha-cert-rotation-shutdown

@vrutkovs
Copy link
Member

/test e2e-metal-ovn-ha-cert-rotation-shutdown e2e-metal-ovn-sno-cert-rotation-shutdown

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 14, 2025

@tjungblu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-etcd-certrotation 369b1cd link false /test e2e-aws-etcd-certrotation
ci/prow/e2e-aws-etcd-recovery 369b1cd link false /test e2e-aws-etcd-recovery
ci/prow/e2e-aws-ovn-etcd-scaling 369b1cd link true /test e2e-aws-ovn-etcd-scaling
ci/prow/e2e-metal-ovn-sno-cert-rotation-shutdown 369b1cd link false /test e2e-metal-ovn-sno-cert-rotation-shutdown
ci/prow/e2e-metal-ovn-ha-cert-rotation-shutdown 369b1cd link false /test e2e-metal-ovn-ha-cert-rotation-shutdown

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lance5890 lance5890 lance5890 left review comments

@dusk125 dusk125 Awaiting requested review from dusk125

@Elbehery Elbehery Awaiting requested review from Elbehery

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@tjungblu @openshift-ci-robot @openshift-cherrypick-robot @vrutkovs @lance5890