TileDB-SOMA 1.15.0rc4

This is a release candidate for 1.15.0 which we anticipate releasing in earlly December 2024. It contains several important developments.

• Spatial transcriptomics: alpha version of the feature.
• Performance improvements: particularly for tiledbsoma.io.to_anndata / tiledbsoma.io.to_h5ad
• Seurat I/O: incremental work for Seurat v5
• New shape feature: Python tutorial here; Academy docs to be shipped by the time of 1.15.0
• Documentation infrastructure: please see the pre-release docs (note latest in the URL)
• For developers only: We now build our C++ library using C++ 20. This doesn't affect non-developer installs, which will get artifacts from PyPI/r-universe/Conda.
• The TileDB-Py package is no longer a required dependency.

Upcoming deprecation notices

• The .used_shape accessor for SparseNDArray and DenseNDArray is scheduled for removal in 1.16: as of 1.15 the .shape accessor replaces it.

Spatial transcriptomics

• [python] Add export for MultiscaleImage to SpatialData by @jp-dark in #3355
• [python] Add PointCloudDataFrame to SpatialData points model by @jp-dark in #3357
• [python] Add MultiscaleImage level SpatialData exporter by @jp-dark in #3342
• [python] Spatial changes required for somacore 1.0.24 by @jp-dark in #3333
• [python] Fix and test SpatialData transformation exporter by @jp-dark in #3330
• [python] Rewrite Xarray support to use dask arrays by @jp-dark in #3325
• [python] Return identity for MultiscaleImage transform to/from level 0 by @jp-dark in #3323
• [python] Add MultiscaleImage methods to access level URI by @jp-dark in #3322
• [python] Add export for PointCloudDataFrame to SpatialData shapes by @jp-dark in #3298
• [python] Create xarray backend for DataArray types by @jp-dark in #3243
• [python] Update spatial API to match somacore 1.0.23 by @jp-dark in #3274
• [c++] Cast vertex list to WKB by @XanthosXanthopoulos in #3222
• [c++] Geometry Dataframe by @XanthosXanthopoulos in #3212

Performance improvements

• [python] Optimization of ExperimentAxisQuery to_anndata by @bkmartinjr in #3359
• [python] Fix incorrect type handling in ExperimentAxisQuery sparse reader by @bkmartinjr in #3375
• [python/C++] Release GIL in IntIndexer C++ code by @bkmartinjr in #3344
• [python] Partition sparse matrix reads in tiledbsoma.io.to_anndata by @bkmartinjr in #3328
• [python/c++] COO to CSX conversion optimization by @bkmartinjr in #3304
• [python] Performance improvement for experiment-level resize by @johnkerl in #3308
• [python] Add ManagedQuery bindings by @nguyenv in #3281
• [python] Ingest somacore classes by @ryan-williams in #3307
• [c++] Fix heap management in fastercsx by @johnkerl in #3365
• [c++] Add ManagedQuery constructor that takes SOMAArray by @nguyenv in #3279
• [c++] Add ManagedQuery::set_layout by @nguyenv in #3278
• [c++] Use schema from tiledb::Array instead of ManagedQuery by @nguyenv in #3277
• [c++] Move casting from SOMAArray to ManagedQuery by @nguyenv in #3275

Seurat I/O

• [r] Add initial support for ragged array writing for Seurat v5 by @mojaveazure in #2523

New shape

• [c++] Some use_current_domain unit-test/feature-flag teardown, part 4 of 4 by @johnkerl in #3372
• [c++] Some use_current_domain unit-test/feature-flag teardown, part 3 of 4 by @johnkerl in #3371
• [c++] Some use_current_domain unit-test/feature-flag teardown, part 2 of 4 by @johnkerl in #3370
• [python/r] Remove 2.27-related feature flag by @johnkerl in #3368
• [r/python] Revert #3300 by @johnkerl in #3358
• [python/r] Enforce dataframe domain lower bound == 0 by @johnkerl in #3300
• [c++] Use valid ASCII for wide-as-possible string current domain by @johnkerl in #3367
• [python] Use new shape in one more spot by @johnkerl in #3321
• [python] Fix bad merge of soma-shapes notebook by @johnkerl in #3309
• [python] Tutorial notebook for the new-shape feature by @johnkerl in #3294
• [r] Data refresh for new shapes by @johnkerl in #3303
• [python/r] Move beyond the new-shape feature flag by @johnkerl in #3301
• [python] Fix dense example data for core 2.27 by @johnkerl in #3296
• [python/docs] Replace experiment directories with .tgz files in source control by @johnkerl in #3295
• [python] Doc updates for new-shape feature by @johnkerl in #3285
• [python] Run upgrade-shapes on notebook example experiments by @johnkerl in #3289
• [python] Include tiledbsoma_upgrade_shape for DenseNDArray by @johnkerl in #3288
• [python] More auto-detect of function_name_for_messages by @johnkerl in #3286
• [r] More fixes for unit-test cases with dense + core 2.27 by @johnkerl in #3280
• [python] Fix last 2.27+Python+dense failing test case by @johnkerl in #3269
• [r] Fixes for dense+2.27 by @johnkerl in #3270
• [python] Fix 3D/4D cases with core 2.27 by @johnkerl in #3268
• [c++] Apply subarrays for dense reads and writes by @johnkerl in #3263
• [python] Fix some dense+2.27 failing test cases by @johnkerl in #3265
• [python] Centralize sparse/dense pybind11 shape methods by @johnkerl in #3261
• [python] Re-run, and update, example notebooks with new shape feature by @johnkerl in #3290

Documentation infastructure

• [docs] Fix Sphinx issue with linkless summaries by @johnkerl in #3284
• [docs] Add new things to .rst files by @johnkerl in #3283
• [python] Misc. formatting fix-ups by @johnkerl in #3287
• [python] _multiscale_image: add missing docstrings, typo fixes by @ryan-williams in #3312

C++20

• [ci] Change manylinux version for sdist&wheels CI (C++ 20) by @XanthosXanthopoulos in #3351
• [c++] Another C++ 20 doc-spot by @johnkerl in #3352
• [python] Update README.md to remove fmt reference by @nguyenv in #3347
• [c++] Use std::format in spdlog (C++ 20) by @XanthosXanthopoulos in #3343
• [c++/python] Switch to std::span by @XanthosXanthopoulos in #3340
• [c++/python/r] Migrate to C++20 by @XanthosXanthopoulos in #3331
• [python/docs] Use ubuntu-24.04 (C++ 20) for readthedocs by @ryan-williams in #3339

Packaging and CI

• [c++/python/r] Use core 2.27.0-rc3 by @johnkerl in #3366
• [packaging] More failure info in update-tiledb-versions.py by @johnkerl in #3363
• [python] Run pre-commit autoupdate by @johnkerl in #3326
• [r/python] Note on dev installs and spdlog failures by @johnkerl in #3324
• [r/ci] Remove tiledb-r specific install by @mojaveazure in #3319
• [r/ci] Try to unbreak r-valgrind CI by @johnkerl in #3318
• [r/ci] Avoid log truncation with "Last 13 lines of output" by @johnkerl in #3313
• [r] Run tests in closer to internal-dependency order by @johnkerl in #3311
• [python] Unbreak CI with anndata==0.11 by @johnkerl in #3305
• [python] Fix some warnings at anndata==0.11 by @johnkerl in #3306

Remove TileDB-Py as a required dependency

• [python] Update README.md to remove tiledb-py by @nguyenv in #3346

Miscellaneous internals

• [r] Use standard name zzz.R for init code by @johnkerl in #3332
• [r] Address two more compiler warnings by @johnkerl in #3320
• [c++] Fix a compiler warning by @johnkerl in #3310
• [python] Consolidate all _set_coords methods into single _util function by @nguyenv in #3292
• [python] Rename set_reader_coords to set_coords by @johnkerl in #3253