Open Data Cube 1.9.0

Open Data Cube 1.9.0 release

1.9.0 is first major release of the Open Data Cube in several years. The focus has been on retiring technical
debt and unused features and preparing for major architectural changes in subsequent releases.

As there are some backwards incompatibilities in this release, we recommend that you read the migration notes
at docs/installation/MIGRATION-1.8-to-1.9 before upgrading.

Major changes from recent 1.8.x releases include:

• A new index driver that uses PostGIS spatial indexes to provide faster and more accurate geospatial search
  and provide better support for storing data that covers regions where traditional lat/long seach is
  inadequate (i.e. covering polar regions or crossing the anti-meridian).

  The new postgis index driver implements a new API for working with dataset lineage, and only supports
  a single location per dataset. It is otherwise largely backwards compatible with the legacy postgres index driver.

  The legacy "postgres" index driver is still available, but will be removed in a future release.

• A new configuration layer that provides more predictable and consistent behaviour and will be easier to
  extend in future.

  Given the nature and scope of the changes, the new configuration layer is not fully backwards compatible
  with the old implementation, but the vast majority of use cases should only require minor adjustments
  to existing configuration and code at most.

• The old datacube.utils.geometry library is now deprecated. odc-geo is used internally throughout,
  and we recommend updating all your code to use odc-geo rather than the deprecated internal library.

• The long-deprecated executor and ingestion workflows have been removed.

Changes since 1.9.0-rc13

• API autodocs cleanup (#1688)
• Further metadata fix for new lineage API (#1690)
• Update release process ready for post-1.9.0 release (#1691)
• Removed all references to the postgis driver as "experimental" in tests and documentation (#1693)
• Update whats_new.rst etc. for 1.9.0 release (#1694)

With thanks to all who contributed to 1.9.0 development: @omad, @whatnick, @pjonsson, @alexgleith, @Kirill888, @robbibt, @snowman2, @caitlinadams, @Ariana-B, and @SpacemanPaul

Open Data Cube 1.9.0-rc13 pre-release

Open Data Cube 1.9.0-rc13

Pre-release of the lineage API bug-fix needed for Explorer pre-release.

This will most likely be the last release candidate before the 1.9.0 release.

Changes since 1.9.0-rc13

• Update and cross-reference 1.8 to 1.9 migration notes (#1686)
• Fix SimpleDocNav lineage handling with PostGIS index (#1687)
• Update whats_new.rst for 1.9.0-r13 pre-release (#1689)

Includes contributions from @SpacemanPaul and @Ariana-B

Open Data Cube 1.9.0-rc12 Pre-Release

We are getting very close to a formal 1.9.0 release.

If you do NOT want to upgrade immediately upon the 1.9.0 release, you should pin datacube<1.9 in your constraints and/or requirements files.

What's Changed

• Remove multiple location support from postgis driver and remove unnecessary logic copied from eodatasets3 (#1658)
• Documentation fixes (#1659)
• Don't use importlib_metadata (#1657)
• Pin upstream libraries to get CI tests running with numpy2 (#1661)
• Calculate spatial extent in epsg:4326 and reproject when no dedicated spatial extent for the requested CRS is available (#1662)
• Fix broken alembic migration (#1667)
• Fix broken documentation build (#1668)
• Rename DatasetType to Product in all the tests (#1671)
• Fix a bunch of typos and spelling mistakes (#1672)
• Documentation updates for 1.9 release (#1664, #1669)
• Fix metadata issues with new Lineage API. (#1679)
• Suppress annoying config warning when configuring by environment variable (#1680)
• Suppress internal deprecation warnings when running tests against deprecated code (#1681)
• Split index.abstract module into a package (#1682)
• Cherry picks from 1.8 and prepare for 1.9.0-rc12 release (#1684)

Includes contributions from @caitlinadams, @alexgleith, @pjonsson, @omad, @Ariana-B, @whatnick and @SpacemanPaul

Open Data Cube 1.8.20 Release

This will likely be the last 1.8 series release before the formal release of 1.9.0. Maintenance releases in the 1.8 series will continue for some time after the 1.9 release. If you do NOT want to upgrade immediately upon the 1.9.0 release, you should pin datacube<1.9 in your constraints and/or requirements files.

Includes bugfixes, cleanup, maintenance updates and changes to the docker build and CI infrastructure.

What's Changed

• Update docker image and CI (:pull:1624, :pull:1625, :pull:1626, :pull:1639, :pull:1641, :pull:1642)
• Update Slack links to Discord (:pull:1620)
• Automatic updates of upstream library/image versions (:pull:1629, :pull:1644, :pull:1653, :pull:1660)
• Don't error when adding a dataset whose product doesn't have an id value (:pull:1630)
• Update docs (:pull:1631,:pull:1651)
• Pin out bad versions of dask (:pull:1663)
• Ensure correct boundary scalar precision with numpy 2 (:pull:1673)
• Add more string representations of nan (:pull:1678)
• Update whats_new.rst for 1.8.20 release (:pull:1685)

Includes contributions from @caitlinadams, @alexgleith, @pjonsson, @omad, @Ariana-B, @whatnick and @SpacemanPaul

Open Data Cube 1.9.0-rc11 Pre-Release

Open Data Cube 1.9.0 Release Candidate 11

Key enhancements in this release candidate include:

• bug fixes to postgis driver
• improved handling of datasets crossing the anti-meridian
• enhancements to the index API to support use cases from Datacube Explorer
• psycopg2 is now an optional dependency for installation (but must be installed to connect to a postgres database)

(Note that rc10 was a bungled release and has been yanked from PyPI)

Changes since 1.9.0-rc9:

• Permissions management cleanup in postgis driver. (#1613)
• Add skip_broken_datasets and dc_load_limit config options. (#1616)
• Enable global environment variables with ODC_ALL naming convention (#1616)
• Remove workaround for an odc-geo bug that is now fixed. (#1622)
• Fix call to geopolygon search. (#1627)
• Use antimeridian package to "fix" extent polygons. (#1628)
• Record lineage when adding datasets with postgis index (#1632)
• Update schema logic (#1634)
• Drop valid-area check and anti-meridian fix 3857 extents (#1635)
• Remove problematic "common_timestamp" postgresql function from postgis driver. Some internal API changes
  required to accommodate and preserve all previous index-driver level behaviour. (#1623)
• Cherry picks from 1.8 (#1624-#1626, #1629-#1631) (#1637)
• Updates to index APIs - add order_by to dataset search, index name attribute, product most recent change method (#1643)
• Fix alembic migrations (#1645)
• Make psycopg2 an optional dependency (#1648)
• Cherry pick recent merged PRs from develop branch and update whats_new for rc10 release. (#1647)

Open Data Cube 1.9.0-rc10 Pre-Release (yanked)

Open Data Cube 1.9.0 Release Candidate 10

This pre-release was bungled and has been yanked from PyPI, please see Release Candidate 11.

Key enhancements in this release candidate include:

• bug fixes to postgis driver
• improved handling of datasets crossing the anti-meridian
• enhancements to the index API to support use cases from Datacube Explorer
• psycopg2 is now an optional dependency for installation (but must be installed to connect to a postgres database)

Changes since 1.9.0-rc9:

• Permissions management cleanup in postgis driver. (#1613)
• Add skip_broken_datasets and dc_load_limit config options. (#1616)
• Enable global environment variables with ODC_ALL naming convention (#1616)
• Remove workaround for an odc-geo bug that is now fixed. (#1622)
• Fix call to geopolygon search. (#1627)
• Use antimeridian package to "fix" extent polygons. (#1628)
• Record lineage when adding datasets with postgis index (#1632)
• Update schema logic (#1634)
• Drop valid-area check and anti-meridian fix 3857 extents (#1635)
• Remove problematic "common_timestamp" postgresql function from postgis driver. Some internal API changes
  required to accommodate and preserve all previous index-driver level behaviour. (#1623)
• Cherry picks from 1.8 (#1624-#1626, #1629-#1631) (#1637)
• Updates to index APIs - add order_by to dataset search, index name attribute, product most recent change method (#1643)
• Fix alembic migrations (#1645)
• Make psycopg2 an optional dependency (#1648)
• Cherry pick recent merged PRs from develop branch and update whats_new for rc10 release. (#1647)

Open Data Cube 1.9.0-rc9 Pre-Release

Open Data Cube 1.9.0 Release Candidate 9

This is the first release candidate to include access to the new hyperspectral loading API. If you install odc-stac and pass driver="rio" or driver="zarr" to dc.load() or dc.load_data(), you can now make use of the powerful new hyperspectral loading engine from odc-stac. Eventually this functionality will be split off from odc-stac into a separate Python package. (if you pass nothing to driver you get the old legacy ODC loader engine.) This hyperspectral loading work was performed by @Kirill888 and funded by the CSIRO Space and Astronomy unit.

This release candidate also addresses bugs in the configuration API and the index driver API. Some minor changes to the index driver spatial search API have been made to bring them into consistency with the dc.load() API. (The semantics of the dc.load() API spatial search API has been preserved, although the implementation has changed to allow simpler interaction with the index API.)

Changes since 1.9.0-rc8:

• Ensure config API works with a blank config/empty file. (#1604)
• Various minor maintenance fixes. (#1607)
• Misc cleanup, and add support for geospatial queries to count methods in postgis driver. (#1608)
• Add new driver based loader (via odc.loader module in the odc-stac package) (#1609)
• Fix 1.9 docker image in GHA (#1610)
• Consolidate spatial search argument handling in index layer and prepare for release. (#1611)

1.8.19

Open Datacube Release 1.8.19

Ensures floating point COGs generated by ODC code are written out in a form that both GDAL and ESRI ArcGIS will handle correctly.

What's Changed

• Update readthedocs stylesheet for dark theme styling by @Ariana-B in #1579
• Update docker image to GDAL 3.9/Python 3.12/Ubuntu 24.04 by @SpacemanPaul in #1587
• Add deprecation warning for config environment names that will not be supported in 1.9 by @SpacemanPaul in #1592
• Always write floating point bands to cogs with nodata=nan for ESRI and GDAL compatibility by @SpacemanPaul in #1602
• Update whats_new for 1.8.19 release by @SpacemanPaul in #1612

Datacube 1.9.0-rc8 pre-release

Datacube 1.9.0 Release Candidate 8

Bugfix pre-release.

The 1.9.0 branch will remain in pre-release until we have working 1.9-compatible versions of the following key ODC packages: datacube-explorer, datacube-ows, eodatasets, and odc-apps-dc-tools.

Work on migrating these packages is underway, and this pre-release addresses an issue identified in the course of that work.

Changes since 1.9.0-rc7

Due to an error in packaging, schema creation and maintenance via alembic for the postgis index driver has been broken for wheels based installations (including installing from PyPI via pip) since it was first introduced in 1.9.0-rc1. There was a failed attempt to fix this in the previous pre-release. This pre-release finally fixes it properly.

• Fix packaging so that alembic data files are correctly packaged in wheels and update whats_new.rst ready for 1.9.0-rc8 pre-release. (#1599)

Datacube 1.9.0-rc7

Datacube 1.9.0 release candidate 7

1.9.0-rc6 was released from the wrong branch and has been yanked from PyPI. This is the version that was meant to be rc6.

Bugfix pre-release.

The 1.9.0 branch will remain in pre-release until we have working 1.9-compatible versions of the following
key ODC packages: datacube-explorer, datacube-ows, eodatasets, and odc-apps-dc-tools.
Work on migrating these packages is underway, and this pre-release addresses issues identified in
the course of that work.

Changes since 1.9.0-rc5

• Update whats_new.rst, ready for 1.9.0-rc7 release. (:pull:1598)
• Fix multi-threading race condition in config API. (:pull:1596)
• Move alembic.ini to a location where it will get installed by pip (without -e). (:pull:1597)

This pre-release includes contributions from @SpacemanPaul