Error when trying to attach to breakpointHook: unable to move container mounts to new mountpoint: EOPNOTSUPP: Operation not supported on transport endpoint

[rien]

Hi, I'm trying out cntr to debug a nix package, but I get the following error when trying to connect:

# cntr attach -t command cntr-/nix/store/cv9dl6yr37a3c0iqqjmz76ba5bsda01n-notmuch-0.35
unable to move container mounts to new mountpoint: EOPNOTSUPP: Operation not supported on transport endpoint

My system information:

• OS: NixOS unstable (my current NixOS configuration)
• cntr version: 1.5.1
• Filesystem info: /nix and / are two separate zfs datasets, /tmp is tmpfs

I have tried the following things, without change of error message:

• reboot
• run cntr as root with sudo
• run cntras root with su
• run cntr as root with doas
• run nix-build as root with sudo

It seems like some kernel options/capabilities/modules may be missing, but I don't know which one that would be.

[Mic92]

What happens if you enable posix acls in zfs? https://blog.alt255.com/post/posix-acls/

[Mic92]

Looks like an issue with writing to FUSE, whats your kernel version?

[Mic92]

It would be a great help if you can reproduce the issue with nixos module and nixos-shell (https://github.com/Mic92/nixos-shell)

[Mic92]

Can you also provide me with the output of strace when you are attaching to a container?

[rien]

Kernel version is 5.15.29, I'll try to strace.

[Mic92]

This command should get you most information: strace -s512 -o /tmp/trace.log -f -yy cntr attach ...

[rien]

Here you go:
trace.log

[Mic92]

Ok. back to posix acls:

153017 getxattr("/proc/self/fd/5", "system.posix_acl_access", 0x7f2bb4000cb0, 4096) = -1 EOPNOTSUPP (Operation not supported)
153017 writev(4</dev/fuse<char 10:229>>, [{iov_base="\20\0\0\0\241\377\377\377\6\0\0\0\0\0\0\0", iov_len=16}], 1) = 16
153017 read(4</dev/fuse<char 10:229>>,  <unfinished ...>
153015 <... mount resumed>)             = -1 EOPNOTSUPP (Operation not supported)

Try enabling them on your zfs dataset: https://blog.alt255.com/post/posix-acls/

[rien]

Yes, it is fixed now:

[root@chaos:~]# cntr attach -t command cntr-/nix/store/rp72fd5ypa3kv1bgh6wrl2395zn5ndaa-notmuch-0.35

[nixbld@localhost:/var/lib/cntr]$ ls
bin  build  dev  etc  nix  proc  run  tmp  var

Thanks!

[Mic92]

Let's keep it open. The error message is horrible and the issue not documented.

[rien]

Alright, for reference, I've executed the following commands for each pool that might be involved:

   $ zfs set xattr=sa vpool/media               #  optional, but better perfomance
   $ zfs set acltype=posixacl vpool/media