OSM-internal pbf support

[Lokks]

Hi @RaczeQ !
First of all, I want to express my gratitude for creating such an awesome tool!

I was wondering if you have considered supporting "internal" PBF files (for example, from osm-internal.download.geofabrik.de). It would be great to have changeset_id or even more columns in the output.

I am considering this tool as a replacement for Osmium, and I believe it could also be beneficial to have something like osmium .config.json file for the quackosm[cli] as enrichment of OsmTagsFilter file you already have.
This might also address issue #140

Thank you once again for your work!

[RaczeQ]

Hello @Lokks 👋🏻
Sorry for the long overdue response. Thank you very much for your kind words 😄

Answering to your questions:

I'm not opposed to having the access to internal Geofabrik PBF files, however I have never used this type of files, so I don't know how the authentication works and if that would add some complexity to the files downloading logic. I have delegated this job to the pooch library, maybe there are settings for passing auth parameters.

---

Regarding the changeset_id column, I'm not sure if it would be possible for QuackOSM to read changeset files, because I'm using the ST_ReadOSM function from the DuckDB spatial extension and only selected columns / bytes are read from the file. Here is the documentation: https://github.com/duckdb/duckdb-spatial/blob/main/docs/functions.md#st_readosm

For this feature I'd encourage you to create an issue in the duckdb spatial respository to include these columns.

---

Speaking of the config file: I have never considered the possibility to pass a config file with multiple jobs, since I'm mainly the Python user and CLI was added on top of the Python API. I think that would be possible, but maybe as an additional command (like quackosm-bulk) to not disrupt the internal logic of the currently working system.

For #140, there are some changes to be made internally and I thought about having a string parameter with values like nwr, wr, w and other combinations to make it simple. It will work both in API and CLI of course.