Support for new groupsv2

[Aarontheissueguy]

Ive recently switched to UbPorts on my VollaPhone and used Axolotl for a couple of days. Everything seems to be working fine except groups. According to my research there are two group types in Signal: the old one which shares meta data with Signal servers and the new one which does not share any meta data with signal and supports modern features like tagging(@person). The latter one causes a row of weird behaviour like the following: messages arrive as private messages even though they were sent to a group, disappearing messages after 0 seconds, Axolotl users can not be added to groups.

I installed Axolotl directly through the openStore and do not have the technical know how to suggest a fix for this problem. It makes groups pretty much unusable.

[Flaburgan]

Hello @Aarontheissueguy and thank you for taking time to report an issue.
Indeed, Axolotl doesn't support yet the new group system of Signal. We're aware of that limitation and it is the priority number one for us. We hope to release a new version to fix that soon.

[nanu-c]

That needs a series of patches

• support for uuids Support uuids #294
• announce that we support new groups to the signal server
• add a migration path from old groups to new groups
• support all the new things that you can do with new groups

[Spraltik]

I would simply like to add that I'm experiencing the same issue with new groups. When people try to add me to the new groups they get the follow error: "This user can't be added to the group until they upgrade Signal.". Old groups are still working fine.

[raboof]

On https://t.me/axolotl_dev it was mentioned that the strategy to add support for new groups will likely be to use the https://github.com/signalapp/libsignal-client library instead of the current https://github.com/signal-golang/textsecure implementation. Apparently some work in this has already been done by @nanu-c and @Blackoverflow .

[Spraltik]

Are there any updates on this issue? I see it's planned for the v1.0.0 release but no one is assigned yet. It has made Axolotl unusable for me.

[nanu-c]

It's in the works and we know that it's urgent but it needs a lot of changes + a rust library, that is used in golang

[ejpcmac]

@nanu-c Is there a way we can help? I’ve never written go, but I can definitely read/write Rust without any issue, and also take my hands on Go if needed. Is your plan to use libsignal-client?

[nanu-c]

hey @ejpcmac development of groupsv2 is going on here https://github.com/signal-golang/textsecure/tree/groups_v2 and basically what has to be done is translate the group decryption to golang. https://github.com/signalapp/Signal-Android/blob/master/libsignal/service/src/main/java/org/whispersystems/signalservice/api/groupsv2/GroupsV2Operations.java#L356

As an example, the title decryption from https://github.com/signalapp/Signal-Android/blob/master/libsignal/service/src/main/java/org/whispersystems/signalservice/api/groupsv2/GroupsV2Operations.java#L707 is like https://github.com/signal-golang/textsecure/tree/groups_v2

So all the theoretical work is done, what it needs now is diligence work to add the 1000 lines of code for group handling.

[nanu-c]

Or basically more than 1000 lines

[ejpcmac]

@nanu-c I’ll get my hands on the project today and in the days to come, mainly reading code, getting a proper build environment & so on. Is there an Matrix or IRC channel to discuss about how I can help once I’m done with my first understanding of the code? Or is it only on Telegram?

Also, do you have some resources / hints about the Signal protocol? Or is it only reverse engineering? (No problem with reverse engineering on my side, but if you have hints on what to look at in priority I may be efficient more quickly). I plan to allocate 1 to 2 hours every two days* on the project for at least two weeks, renewable if needed :)

* Let’s see how it survives to my business days, but I was on a quite good energy last week on other personal projects so I’m quite confident.

[Flaburgan]

Hello @ejpcmac and thanks for join the effort. All communications are happening through Axolotl_dev Telegram channel at the moment, but I think you can join with a bridge from Matrix.

About the Signal protocol, you can read their repositories of course (https://github.com/signalapp) but also the blog: https://signal.org/blog/ there are interesting big picture articles there like https://signal.org/blog/signal-private-group-system/

[Be-ing]

Hi @ejpcmac I'm trying to organize people interested in working on a native adaptive Linux Signal client using the Signal Foundation's Rust libraries. I'm concerned that working on Axolotl is not a good solution for long term maintainability, especially looking at the work @nanu-c has started to integrate the zkgroup Rust library and seeing that branch is +1,451 −322 just to bind an existing library and IIUC there is still a lot left to do. I've started discussions on the Signal and Pine64 forums with my thoughts about various options for technical architectures.

[ejpcmac]

Hi @Be-ing, glad to hear about such project. I’d thought about writing a mobile GTK or integrating with Chatty too. Great minds think alike :) In the mean time, Axolotl is for now the most mature alternative Signal client out there, and I’m engaged to help on bringing groups v2 to it, so I’ll finish my commitment before joining any other effort. I’m already getting used to its code base and will learn about the signal protocol in the process.

Regarding your project, I think the best thing to to would be to build a Rust lib that has a complete protocol support, on top of existing official Rust libs. As things get added to the official libs, you can remove them from the unofficial one. Such lib could the be used to build clients with whatever interface people like to use, maybe even including Axolotl.

[Be-ing]

To be clear, I haven't started any code so there isn't really a project yet to speak about. I'm now leaning towards using Qt Quick & Kirigami because I'm familiar with Qt and have been wanting to learn QML. I have been discussing this in the Plasma Mobile Matrix room. They suggested making a QObject wrapper class around the Rust libraries using cxx, run that in its own thread to handle the networking, and use Qt signals & slots to communicate with QAbstractItemModels backing the QML. QML, Rust, and integrating Rust & C++ are skills I'll be able to reuse for Mixxx. We're planning to rewrite Mixxx's GUI with QML and integrate Rust in the future.

It seems that libsignal-client already implements enough to get started. The official Android, iOS, and Electron clients are using it now. I asked them about documentation and they said that the library is still under heavy development and incomplete, but I'm unclear what it can't do yet so I just asked for clarification about that. It seems it can at least register an account and send & receive text messages, which would satisfy most of my personal use cases. Personally I don't use Signal groups (at the moment), so getting groups working in Axolotol is not my most urgent priority. Most of my Signal usage is text messages and occasionally audio calls, but even a client that just handled text messages would be sufficient for me to use the PinePhone I just ordered as my main phone and leave Android behind.

[Be-ing]

I initialized a repository on KDE's GitLab server. If you're interested in helping, join the #plasmamobile:kde.org Matrix room.

[nanu-c]

There is already a signal client in rust, it's Whisperfish for sailfishOs https://gitlab.com/rubdos/whisperfish you can copy a lot of code there. It uses rust + qt. But it's also highly tied to Sailfish System API's (dbus), so for it to run on Ubuntu touch you need a QML frontend, because otherwise you don't have access to the fileshareing(content hub) for contact import and sending attachments or the system clipboard. Same goes for Sailfish.

What is missing in the official rust libraries is the storage and the part communicating with the signal server. The +1,451 −322 are mostly the part with communicating with the signal server so you have to write it, too. Groups v2 have a partly encrypted protobuf and every section has to be decrypted manually.

So feel free to start a new project, but keep in mind, that even the Whisperfish project, that sees commits everyday, is working since 3-4 months on the groupsv2 support.

[ejpcmac]

@nanu-c Thanks for sharing this other project. I think we should try to converge to some common building blocks in the end, using as much code in common as possible, so it’s easier to review and to follow the official client developments. Then, for the UI parts, they will inevitable be multiple projects, due to parts that are specific to different mobile OSes, but that’s not an issue IMO.

[nanu-c]

The interesting part is, how is the UI communicating with the rest of the signal Client?
Currently axolotl/textsecure is designed as backend service that provides a websocket to the UI, which means it's independent of the host OS. It runs on Windows/Linux/BSD/MacOs/UbuntuTouch/RaspberryPI/Mobian. Actually it is used as a 2fa gateway for nextclouds and we have a monitoring system build with it.

The UI is now either provided by qt webengineview or eletctron but should be also doable in a gtk webview.

Personally i am not a fan of c++, when we relay on dbus, we will break any non linux systems, so for axolotl a way to go would be a rust service, that provides a websocket for the UI.

So @Be-ing when you are going to rewrite an UI please write all the signal logic and the database connection and the connection to the signal server in rust and not in c++, because then it reusable for Axolotl and whisperfish.

[Be-ing]

I am aware of Whisperfish and I don't think its code will be much help to us. Jolla's proprietary QML libraries are a nonstarter and all the Sailfish OS specific stuff would just get in the way. I think starting a new Kirigami application from scratch will be easier. We may learn some lessons from Whisperfish's architecture, but I doubt that adapting their code will save us work.

What is missing in the official rust libraries is the storage and the part communicating with the signal server.

If I understand what you're saying, the upstream Rust libraries only handle processing the messages, but we'll need our own code around that for the networking I/O?

Personally i am not a fan of c++

I've been working in C++ for 6 years, I know. It's an awful language. Nobody should have to read pages of implementation details of the standard library when trying to compile a typo.

So yes, I am planning to do as much as practical in Rust and only use C++ as a small glue layer between the Rust library and the QAbstractItemModels backing the QML. The only part that I am thinking of making as a separate background daemon is a Linux daemon to reimplement Firebase Cloud Messaging as discussed in #14. Of course that will be a lot of work, so to get the project off the ground I think it would be a good idea to start with WebSockets like Axolotl and the official Android client without Google Play or microG. So I intend for the Rust backend to not have any platform-specific dependencies or APIs.

So what does the Rust part need to do? Networking I/O and managing an encrypted database? I'd be happy to collaborate with you and the Whisperfish developers on a shared Rust library. I'm sure your experience would be a big help.

[Be-ing]

I got in touch with the Whisperfish developers and they were already wanting to decouple their application logic from Sailfish OS, so I'll be working with them and KDE people to refactor Whisperfish into a dual Sailfish/Plasma Mobile application. We're trying to move as much as practical into independent Rust libraries. I hope we can collaborate on that library and have multiple frontend applications working on it.

[raboof]

I don't think it was mentioned in this issue yet that the work in axolotl to support groupsv2 is ongoing in https://github.com/nanu-c/axolotl/tree/groupsv2_2

[Id2ndR]

Branch groupsv2_2 was merged