1.0 Zephyr support

[beriberikix]

I was very pleasantly surprised to discover this repo today; as a long time user of U-Blox products I've always wanted to see official drivers (and they're open source to boot!) I was also excited to see a Zephyr port. Are there any plans to upstream as an official driver for Zephyr? Lot's of folks in the Zephyr community are interested in U-Blox modems but it would be great to get the broad support this project offers.

[RobMeades]

Hi there, and glad to pleasantly surprise someone; can't say it happens very often ;-).

We are quite invested in the use of Zephyr and are fully open for such a move; it would definitely be of benefit. That said, our short term road map is quite crowded (MQTT, Wifi sockets, positioning support, etc.) and we have not checked the viability of such an integration. If you have any views on the latter, from what you know of Zephyr that would be very beneficial.

[beriberikix]

Great to hear about interest in Zephyr! From what I know, I think such an integration is very viable. Here's some thoughts:

• Broadly speaking there are existing drivers that this library could be used with via a thin wrapper to match the Zephyr APIs
• There's a modem driver where the cellular modems live. There's a community-implemented driver for the Sara R4. I can envision this replacing that implementation and add the rest of the U-Blox cellular family.
• Separately, there's a generic GSM driver (based on PPP,) though implementing a proper driver per above would certainly provide a better implementation.
• Like modems, there's Wi-Fi drivers
• There's active discussion about defining a GNSS API and it would be awesome if U-Blox was driving the conversation

The community is mostly engaged on Slack or mailing list. Let me know if I can help connect or answer any questions.

[beriberikix]

Oh, two more things:

• There's a very robust Bluetooth subsystem
• Initial ESP32 support recently landed, so might be easier to add the Espressif modules.

[RobMeades]

Thanks for the well structured comments, you know how to spend your Saturday nights ;-) (and I my Sunday mornings :-)).

A motivation for us to head in this direction, aside from the potentially broader consumer base, would be to ease future support for Zephyr. The things likely holding us back are internal resource constraints (see road-map to fulfil above) and the fact that Zephyr seems to be changing very frequently at the moment, hence we might have some difficulty keeping up (it would be really easy for Zephyr to break us, as it already has once and we've only moved Zephyr version once!). We'd probably have to snap to a Zephyr version, though one that is closer to master/HEAD (or main/HEAD).

We're on the Slack channel and mailing list, though I must admit I don't follow the Slack channel as it is quite busy.

I will discuss with @chriskarls and others, but my uncorroborated thoughts are:

• we need at least 3 months to put in place the stuff we've already promised,
• during that time we could look at the Zephyr APIs and try to make some sort of plan as to what we might sync with and how/when,
• when we have the above we post it back here for comment.

Bearing in mind the above is not a promise, just my unvarnished thoughts, does it at least sound bearable?

On the specifics of the GNSS API, we had noticed that and I am about to take a look at the beginnings of our GNSS API. Now I am NOT a GNSS expert (I come from the cellular bit of u-blox) but I do have access to those experts and they will of course be reviewing our GNSS API. Our intention is to start small (i.e. not much more than something like uGnssLocationGet()) but to make those small bits compatible with where we might be in a year from now; the last thing anyone wants is an API in flux. Anyway, we are relatively unlikely to have the bandwidth to actually contribute to Zephyr, we will be fast followers, but that means we have an interest in the APIs remaining sympathetic and so I will watch that GNSS API thread.

[beriberikix]

That sounds bearable indeed! FWIW, Zephyr has a concept of LTS releases and the next LTS is slated for v2.7. Current schedule is 2.6 for 5/21 and 2.7 for 10/21. I'm not aware of any APIs are expected to change that would affect these drivers (but there certainly could be!) so worth starting to design sooner. And if there is a desire to make changes (or anything needed for GNSS) before LTS, it should probably land for 2.6 to be baked by 2.7

[warasilapm]

Hi folks,

It's been a year since this issue was last touched and I am finding myself in a situation where I'm strongly considering using ubxlib over the community sara-r4 driver for my zephyr application. What's the status with the vision to have Zephyr driver support?

[RobMeades]

@warasilapm: I've prodded our product management on this question, since it is an issue of priority; hopefully they will comment here soon.

[manemannen]

Hi there, we are currently focusing on the 1.0 release which is a few weeks out. We are just completing some final things. Once that is done and we start on a new batch of features, this will be one of the items on top of that list.

[detchison]

Any update to this.. As SARA-R4 Support is needed for a Zephyr project I am working on, I am interested if there is an upcoming update to allow use with Zephyr with the more recent Zephyr versions and DTs

[RobMeades]

Hi there. Funny you should mention more recent Zephyr versions as issue #74 has caused us to do more work on that since the requester needed NRFConnect 2.1.0, which is Zephry 3.x, SPI. The current master branch here should work with NRFConnect 2.1.0, we just haven't moved our testing there from NRFConnect 1.6.1. The preview_spi_rmea I know works with NRFConnect 2.1.0 because I tested it that way (the SPI additions needed to be made compatible with an Zephyr 3.x SPI structure change).

But to answer your question both succinctly and confusingly:

• the ubxlib core code on master here should work with NRFConnect 2.1.0/Zephyr 3.x but the .overlay files here will not, they are still "old Zephyr",
• we had determined that there was a minimum starting point for "upstream Zephyr" support that we would do, but we haven't actually done yet; I'll refresh my memory as to what that actually was and report back here,
• we will be doing a 1.2 release around March next year which will definitely have tested support for NRFConnect 2.x.y/Zephyr 3.x.

[detchison]

Thanks for the update. Not sure exactly if I am reading you correctly, but I too am using NrfConnect 2.1.0, You say that the overlay files themselves haven't been updated, but will the master branch currently work with the new structure, or do I need to structure and define my uart nodes in my overlay in the old way for it to work? I should also mention its a Nordic Nrf52840, since you reference specific concerns about versioning and Nordic MCUs

[RobMeades]

The master branch here should work with the new DTS file structure: we have updated our code so that it will support nRFConnect 2.1.0/Zephyr 3.0 in addition to nRFConnect 1.6.1/Zephyr 2 and we have tested it with nRFConnect 2.1.0/Zephyr 3.

The "word of caution" thing is that we are not yet ready to update our overlay files to the nRFConnect 2.1.0/Zephyr 3 format, so our regression testing remains on nRFConnect 1.6.1/Zephyr 2, hence there is a small possibility we could break that compatibility somehow or other. However, it is pretty darned obvious when you're getting involved with DTS stuff and I ran with it myself only last week so I believe it's good.

If you do have problems then please let us know and we'll fix it.

[detchison]

Ok I very much appreciate that.. In looking at the Zephyr getting started, it says to reference the ubxlib directory in added the ZEPHYR_EXTRA_MODULES to get it setup, which directory of the master branch should I be downloading and referencing in my project? That isn't exceedingly clear in the documentation. Is it just the base ubxlib master directory and referencing the ubxlib.h there?

[RobMeades]

If you take a look at the CMakeLists.txt we use in our runner build you'll see that we set ZEPHYR_EXTRA_MODULES to the ubxlib directory, so for instance /home/rob/ubxlib. Zephyr should then pick up the zephyr directory off there which has its module.yml file in it.

[detchison]

Right on.. I have that..

`

SPDX-License-Identifier: Apache-2.0

cmake_minimum_required(VERSION 3.20.0)

find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(mwpms_tank_mon)
list(APPEND ZEPHYR_EXTRA_MODULES src/drivers/ubxlib)

target_sources(app PRIVATE src/modules/cell_module.c)
target_sources(app PRIVATE src/modules/settings_module.c)
target_sources(app PRIVATE src/modules/clock_scheduling_module.c)
target_sources(app PRIVATE src/modules/pressure_sensor_module.c)
target_sources(app PRIVATE src/modules/temp_baro_module.c)
target_sources(app PRIVATE src/main.c)
` in my projects CMakeLists.txt

However still am getting this error on build
d:/Development/nRf/MWPMS/TankMon/FirmwareSource/MWPMS_TankMon/mwpms_tank_mon_firmware/prj.conf:66: warning: attempt to assign the value 'y' to the undefined symbol UBXLIB

I will add that the ubxlib folder as downloaded from the master branch is at /drivers/ubxlib in my project. So I am assuming it should be loading those config options, but I dont think it is..

My instinct is that those should be loaded from the KConfig in port/platform/zephyr correct?

[RobMeades]

Did you also do the other bit under https://github.com/u-blox/ubxlib/tree/master/port/platform/zephyr#integration, i.e.:

You must then also enable UBXLIB either via menuconfig or by adding the following line to your prj.conf:

CONFIG_UBXLIB=y

ubxlib also requires some Zephyr config to be enabled, but you currently need to check runner/prj.conf/runner_linux/prj.conf to get these correct.

If so, is ubxlib a visible option in your menuconfig?

[detchison]

Yes, that is the part throwing that error.. CONFIG_UBXLIB=y errors out because it is undefined, which leads me to believe that the module isn't getting loaded. If I leave it out, it will build, but it won't build with that line, which is what lead me to that conclusion, its like it isn't being loaded via the CMakeLists

[RobMeades]

Sounds like it. I assume the relative paths in your CMakeLIsts.txt are an OK thing? Just to be clear you say "ubxlib folder as downloaded from the master branch is at /drivers/ubxlib" and then in the CMakeLists.txt you have pasted in I see src/drivers/ubxlib, I guess those are the same places?

I'm no Zephyr expert but I guess that if the module has been loaded it would appear in your menuconfig? So if it is not we can safely assume it is, ummm, not. Hmph.

[RobMeades]

Could something have changed with the Zephyr module inclusion mechanism between Zephyr 2 and Zephyr 3? I would hope not, and since we use it as well and the builds worked for me last week that would imply there's no version problem.

[detchison]

correct... the drivers folder is alongside the my "modules" folder. Does it have to be an absolute path you think?

[RobMeades]

Worth a try as an experiment.

[detchison]

Tried it... nothing, same error

[RobMeades]

You could try the other approach we suggest.

The easiest way though is to just include the file ubxlib.cmake in this directory to the applications CMakeLists.txt. This is the only thing required to get full access to ubxlib.

Doing that will also by default implicitly setup suitable configuration variables for ubxlib. If that is not wanted then the CMake variable UBXLIB_NO_DEF_CONF must be defined before the inclusion.

[detchison]

What would that line look like in CMakelists.txt if I did it that way? Not sure I am following, its been a long day.

[RobMeades]

Something like:

include(${UBXLIB_BASE}/zephyr/ubxlib.cmake)

[detchison]

Correction, I was able to get both to to be set and build, now to test functionality.

[RobMeades]

FYI, you can find our Zephyr-specific examples here, in case it is of help:

https://github.com/u-blox/ubxlib_examples_xplr_iot

[detchison]

I looked at these examples.. Was a little puzzled, I see the part about setting the pins to -1 since they are described in the DeviceTree, however, where in these functions do i tell it the DT node to use... does it automatically look for an applicable device? Either way probably not the thread for that question.

[RobMeades]

Those examples try to hide the complexity of the device configuration, since they are targeted at a particular board. Our generic examples "let it all hang out" in that respect, so maybe see how this example populates the device configuration:

https://github.com/u-blox/ubxlib/blob/master/example/sockets/main.c#L122

...with the configuration advice here:

https://github.com/u-blox/ubxlib/tree/master/example/sockets#using-a-cellular-module

[dsconyers]

Hi @RobMeades,
I'm trying to integrate ubxlib into Nordic's NUS sample for NCS v2.1.0 and I'm getting the attached build error where it doesn't like the malloc() and free() redefinitions in u_port_clib.c . Attached is a copy of the build output and CMakeLists.txt file:

• ubxlib_NUS_integration_build_error.txt
• CMakeLists.txt

I have CONFIG_UBXLIB=y set in my prj.conf file.

Do you see any issue with my CMakeLists.txt file?

[dsconyers]

Replacing the following line in my CMakeLists.txt file:
list(APPEND ZEPHYR_EXTRA_MODULES ${UBXLIB_BASE})
with:
include(${UBXLIB_BASE}/zephyr/ubxlib.cmake)
Allowed it to build completely.

[RobMeades]

Hi there @dsconyers: apologies, yes, we have all of this sorted for NCS up to 2.2 and Zephyr 3 but the code is not quite ready for release yet (for other reasons), hence it is not pushed here. I'm not the most expert at the rather complicated Zephyr build environment: the change above might mean that you now have slightly different .conf file stuff (e.g. for instance using the Zephyr minimal C library, which I believe doesn't have malloc()/free() and hence we need to provide our own wrappers, versus newlib which does have malloc()/free()) but, provided it is building to completion and it looks like it has all of the port*.c, cell*.c, gnss*.c and short_range*.c stuff in it, for instance, then you likely have it right.

If you find that this doesn't run correctly, I can push a preview branch of what we have here to get you moving until we release the proper thing, which should be in just over a week.

[dsconyers]

@RobMeades,
Thank you! A preview branch would be awesome since I'm on a tight schedule with a demo coming up in early April and it sounds like it would help. I'm still on the steep learning curve of the Zephyr build environment. So, I'll take any help I can get. Thanks again!

[beriberikix]

@dsconyers glad you were able to get it to build properly! The Zephyr Project has a very active Discord server where the maintainers and community hang out and are available to help. Hope on over to https://chat.zephyrproject.org/ if you're not already on there!

[RobMeades]

Hokay, find a preview of the release branch here: https://github.com/u-blox/ubxlib/tree/preview_release_branch_rmea.

It should be all good for building Zephyr under NCS 1.6 to 2.2 and, if you like that kind of thing, you can now build it for Zephyr (and ESP-IDF/Arduino[ESP-IDF]) using the PlatformIO system/IDE, which gives you the fexlibility to use Zephyr outside NCS if you wish.

Note that your Zephyr .conf file must now include:

CONFIG_INIT_STACKS=y
CONFIG_THREAD_STACK_INFO=y
CONFIG_THREAD_NAME=y

[RobMeades]

One note about the PlatformIO/Zephyr thing: as far as we are aware PlatformIO currently only integrates Zephyr 2, not Zephyr 3; I believe that's right @beriberikix?

[beriberikix]

Correct, unfortunately: platformio/zephyr#15

[RobMeades]

Yes, I guess that they have other worries right now.

[dsconyers]

As expected, I was able to successfully build Nordic's NUS sample with the preview_release_branch_rmea integrated for NCS 2.1.0 & 2.2.0, but not 2.3.0. Thanks again for the preview!

[RobMeades]

There's a 2.3.0!? We've only just caught up with 2.2.0!

Anyway, thanks for the confirmation.

[RobMeades]

@dsconyers: the preview branch should now work for NCS 2.3.0/Zephyr 3.3 as well, if you need that.

[dsconyers]

Nice! I do actually. Thank you!
Anyone using the NORA-B12x modules (or any FEM) will need NCS v2.3.0 to get the maximum transmit power when using the MPSL due to this issue.

[RobMeades]

Very interesting, thanks for pointing that one out.

[JaanViru]

Thanks for the preview release! We will test it out with ncs v2.3 as well.

[allard-potma]

I'm trying the preview branch with NCS2.2.0 and newlib.

CONFIG_NEWLIB_LIBC=y
CONFIG_NEWLIB_LIBC_NANO=n

This does not build and generates the following error:

libc-hooks.c:25:1: note: 'malloc' is defined in header '<stdlib.h>'; did you forget to '#include <stdlib.h>'?

To resolve this I had to remove/rename malloc.h in port\platform\zephyr\src.

[RobMeades]

<sigh>, sorry about that, I really hate all this conditional compilation of something as basic as a C library.

Let me try to reproduce it, and maybe we have to add a test build that includes this permutation; we chose to test with the minimal Zephyr C lib 'cos we wanted to be sure that we could cope without some of the things that it doesn't provide and we absolutely need, but I'd guess that most people probably want newlib anyway so it is a more likely permutation.

[beriberikix]

Interested in more complications? picolibc was recently introduced and users may prefer it in the future because it brings more optimizations.

[RobMeades]

!!!!!!!!!

[beriberikix]

To be fair, it's very new and an optional module!

[RobMeades]

@allard-potma: now fixed for newlib I believe, thanks for reporting this.

The correct fix was to delete malloc.h: we used to need this to compile our code with the Zephyr minimal libc but it seems times have changed i.e. (a) it is no longer required in the minimal libc case and (b) either the order of includes has changed or libc-hooks.c got introduced, which picks up our empty malloc.h instead of the one that comes with GCC.

[RobMeades]

On its second anniversary, we believe we have finally put in place the required support for this issue:

1. ubxlib may be used as a Zephyr module; this has been there for some time.
2. ubxlib now supports, and is tested with, Zephyr 3.3 (nRFConnect 2.3.0); final relevant commit d338ebb.

We also support building for Zephyr under PlatformIO, though we note that PlatformIO does not yet support Zephyr 3.

Are you happy to close this issue @beriberikix?

[beriberikix]

Awesome! 🎉

The original intent of this issue was to get this module upstreamed into Zephyr as an upstream module. That's less of a personal interest, though others might still care about that distinction.

Instead, I'll update the issue title and close as this is now 1.0+.

[RobMeades]

Thanks @beriberikix!