Cannot create backup subflows to remote endpoints?

[asirir]

Hello,

I'm on kernel 6.5.0-41 (Ubuntu 22). I have a very simple host / network configuration like so:

HOST_1:A <-> HOST_2:P
HOST_1:B <-> HOST_2:Q

Where A,B,P,Q are IP addresses (so each host has 2 interfaces) and routing tables are setup so that traffic is pinned down between those IPs. My client is on HOST_1, server on HOST_2, the client program opens the initial connection to HOST_2:P (for some port).

I would like to have my primary connection between A <-> P and a backup subflow between B <-> Q.

This is what I tried:

# HOST_1:
ENDPOINT B => backup

# HOST_2:
ENDPOINT Q => signal backup

But mptcp monitor shows no backup flag in the established subflow between B <-> Q. And this seems to agree with what @pabeni said in #237:

Indeed subflows created in response to incoming ADD_ADDR will always be non backup. We could improve the scenario with an additional PM netlink setting, that will allow configuring the backup status for such subflows.

But then @matttbe also mentioned:

I don't think we need to modify the RFC: both the client and sender can add the backup in the MPJ. So when the ADD_ADDR is received, the client can add the backup flag if the endpoint it is using has this flag. Same on the server side, no?

But in that case I expect the B <-> Q subflow to bear the backup flag (as endpoint B has backup flag in my setup).

Note that I could make a backup subflow with:

# HOST_1:
ENDPOINT B => backup subflow

# HOST_2: (no endpoints)

But the problem with this setup is, the subflow is established between B <-> P - because A <-> P is where the primary connection is, the client is not aware of the address B on the server (without a signal endpoint). I also tried several other variations with similar results.

Put another way, what I'd like is for the server to advertise an endpoint, and then for the client to open a backup subflow to that endpoint. Is this not possible?

Many thanks!

[asirir]

Update:- it appears I can achieve what I desire with the following endpoint setup:

# HOST_1:
ENDPOINT B => fullmesh backup

# HOST_2:
ENDPOINT Q => signal

In this case the backup flag is observed on the B <-> Q. Reading the documentation for fullmesh:

If the peer did not announce any additional
addresses using the MPTCP ADD_ADDR sub-option, this will
behave the same as a plain subflow endpoint. When the peer
does announce addresses, each received ADD_ADDR sub-option
will trigger creation of an additional subflow to generate
a full mesh topology.

So I suppose as long as I can guarantee that my peer (in this case, my server program) always announce an endpoint, this setup should work for me? Specifically, I do not want the client to open regular subflows.

[asirir]

Interestingly, fullmesh backup isn't totally legal as per https://www.mptcp.dev/pm.html#in-kernel-path-manager

fullmesh: The MPTCP path manager will try to create an additional subflow for each known peer address, using this endpoint as the source IP address. It requires the subflow flag, and it is not compatible with the signal one.

[matttbe]

Put another way, what I'd like is for the server to advertise an endpoint, and then for the client to open a backup subflow to that endpoint. Is this not possible?

Yes it is possible. But:

• you endpoint Q with signal backup is currently not working, see PM: Backup flags is not set in MPJ SYN+ACK even with there is a 'signal' endpoint having the backup flag #507. That would be great if you can the patches I recently shared :) lore GitHub. Note that I also fixed the endpoint Q with only the backup flag if I'm not mistaken.
• Currently, upon the reception of an ADD_ADDR, the PM will not look at the endpoints list, it will pick the source IP by looking at the routing table (the result of ip route get <IP Q>), except if there are endpoints with the fullmesh flag (more details here). I guess that's why your workaround works.

Interestingly, fullmesh backup isn't totally legal as per https://www.mptcp.dev/pm.html#in-kernel-path-manager

fullmesh: The MPTCP path manager will try to create an additional subflow for each known peer address, using this endpoint as the source IP address. It requires the subflow flag, and it is not compatible with the signal one.

Yes, it is allowed: signal fullmesh is not, but you can have subflow fullmesh backup.

[matttbe]

@asirir

• you endpoint Q with signal backup is currently not working, see PM: Backup flags is not set in MPJ SYN+ACK even with there is a 'signal' endpoint having the backup flag #507

This should be fixed in our export branch. Patches are now in Linus tree, but not backported to stable yet. If you are able to check that these patches fix your issue, that would be great!

[asirir]

Thanks @matttbe - sadly I'm in an environment where I can only use stable kernel releases. Will upgrade as soon as a newer kernel lands. Will close this issue as it's been addressed. Cheers,

[matttbe]

@asirir: patches have been queued for the stable versions. They should be released soon, but I'm not sure when they will be available in your distribution.