Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run cinder services as cinder user #417

Merged

Conversation

eharney
Copy link
Contributor

@eharney eharney commented Jul 12, 2024

Run all cinder services as the cinder user instead of root.

This updates our kolla usage and apache configs to work in this configuration.

Jira: https://issues.redhat.com/browse/OSPRH-10083

@openshift-ci openshift-ci bot requested review from Akrog and dprince July 12, 2024 14:03
@Akrog
Copy link
Contributor

Akrog commented Jul 16, 2024

/retest

@ASBishop
Copy link
Contributor

/hold
we need to do some cross service testing when glance is using cinder for its backend, and they share an os-brick lock path.

@ASBishop
Copy link
Contributor

I tested a local, rebased version of this PR to determine whether changing the UID affects any operation or interoperability with other services that rely on file permissions. The good news is this is (nearly) a non issue.

The initial concern was the os-brick lock file directory that's shared between the cinder-volume, cinder-backup and glance-api control plane services. Consider the lock file named 'os-brick-connect_volume'. The file is created the first time any service acquires the lock, and the file will have the ownership and permissions of that service. After the lock is released, other services that need to acquire the lock must be able to open the file in write mode in order to take the lock, which in turn implies the other services need write access to the lock file. Prior to this PR the file could be created by a cinder service running as root, which should have caused problems for the glance-api service when running as the 'glance' user. Fortunately, tests show this doesn't happen, apparently as a positive side effect of k8s's UID and namespace remapping. Glance is able to acquire the lock even when the lock file's permissions suggest it doesn't have write permission. Further testing with this PR, with cinder services running as the 'cinder' user, there are no problems with any service acquiring the lock, regardless of who created the file (i.e. which service first acquired the lock).

The next concern pertains to NFS volumes, and whether volumes created when cinder was running as root would encounter a problem after switching to run as 'cinder'. This is a non-issue, because cinder's "remotefs" driver has provisions for managing the NFS volume's backing file's permissions. Switching from root to 'cinder' is seamless.

A final concern pertains to using an extraMount in order to perform image conversion operations on an NFS share provided by the extraMount. Tests show an issue does arise if the NFS share (on the server) doesn't allow non-root users to write to the share. This can be handled by documenting the issue. The NFS share's directory permission may need to be updated to grant write permission to the 'cinder' user (UID / GID 42407).

@rajathere
Copy link

I did a fair bit of testing myself and tldr; everything seems to work fine.

Basic Test: After applying changes, pods should run as cinder user

Some advance tests:

  1. Can cinder user create/acquire locks: SUCCESS
  2. If glance creates a lock, can cinder acquire it and vice versa: SUCCESS
  3. If glance acquires the lock, does cinder wait for lock to be released before proceeding with the operation: SUCCESS

So everything seems to work fine with this change.
One question i still have is how the different can access/create/delete locks in the HostPath directory on the host node even without the correct permissions but that should not block this PR.
LGTM.

@ASBishop
Copy link
Contributor

/unhold

Copy link
Contributor

@ASBishop ASBishop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@ASBishop
Copy link
Contributor

/retest

Copy link
Contributor

openshift-ci bot commented Nov 25, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ASBishop, eharney, rajathere

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

These all ran kolla_set_configs && kolla_start,
but this is unnecessary because kolla_set_configs is
run w/ sudo from kolla_start.

Just call kolla_start instead.

This is necessary to run cinder services as
unprivileged cinder users instead of root -- without
this change, containers fail to start because kolla_start
can't open "/run_command" for writing.
Run cinder-api and cinder-scheduler as the cinder user.

This reconfigures httpd with the necessary mount
permissions to run as the cinder user.
These services do not need to run as the root
user, run them as the cinder user instead.
@eharney eharney force-pushed the cinder-service-runas branch from 1599bf6 to f6aa4d5 Compare November 25, 2024 14:09
@openshift-ci openshift-ci bot removed the lgtm label Nov 25, 2024
@ASBishop
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Nov 25, 2024
@openshift-merge-bot openshift-merge-bot bot merged commit 8564a3c into openstack-k8s-operators:main Nov 25, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants