Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GDAL Filesystem rework, updates, fixes, ST_MakeValid. #187

Merged
merged 10 commits into from
Nov 22, 2023
Merged

Conversation

Maxxen
Copy link
Member

@Maxxen Maxxen commented Nov 21, 2023

Changes

  • Update to DuckDB v0.9.2
  • Standardize PascalCase naming for spatial functions. Only important for future doc generation.
  • Add simple version of ST_MakeValid (no extra args)
  • Disable test for database csv export/import. This broke in v0.9.2 due to the csv sniffer rejecting blobs
  • Update GDAL to 3.8
    • New PMTiles and JSONFG drivers
    • Remove XLSX hacks now that there are proper open options for headers/field types
  • Rework GDAL<->DuckDB filesystem integration, hopefully fixing most of the issues that have occurred
    • We now provide a separate file handler for every client context, with a randomized unique prefix. If GDAL spawns its own threads it can still access the client context through the fs.
    • Should hopefully work with concurrent connections (as they get their own FS).
    • Improved error handling. Some error messages are a little wonky but we can work on that.
    • We now implement unlink, so gdal will delete tempfiles properly.
    • TODO: Look into cache/buffering for network file access. Some formats (xlsx) are a lot slower than they need to be, also upstream patch to remove file handlers to GDAL.

Fixes

Closes #184
Closes #182
Closes #181
Closes #180

@Maxxen Maxxen changed the title Dev GDAL Filesystem rework, updates, fixes, ST_MakeValid. Nov 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant