TileDB-SOMA 1.9.2

What's Changed

• [Backport release-1.9] [python] Add back deprecated function tiledbsoma_build_index by @github-actions in #2346
• [r] tiledbsoma-r 1.9.2 by @johnkerl in #2349

Full Changelog: 1.9.1...1.9.2

TileDB-SOMA 1.9.1

This release fixes a wheel-build issue with 1.9.0.

• [python/ci] Unbreak wheel builds by @johnkerl in #2339
• [r] tiledbsoma-r 1.9.1 by @johnkerl in #2342

Full Changelog: 1.9.0...1.9.1

1.9.0

Note: PyPI wheels are missing due to #2337 so there will be an immediate 1.9.1.

New features added

• [python] Let ingest accept additional_metadata: dict[str, Metadatum] by @ryan-williams in #2258
• [c++/python] Use a shared threadpool for the reindexer by @beroy in #2148
• [c++] Optimizing indexer for pandas and pyarrow by @beroy in #2159

Breaking changes

• [Backport release-1.9] [python/ci] typeguard==4.2.1, make requirements_dev.txt canonical by @github-actions in #2332
• [python] Drop support for Python 3.7 by @johnkerl in #2181

Bug fixes

• [python] Fix race in SOMADataFrame schema with multiple enumerated columns by @johnkerl in #2194
• [python] Fix context-type for tiledbsoma_build_index by @johnkerl in #2224
• [python] Fix an ingestion corner case with empty chunks by @johnkerl in #2283
• [python] Fix registration path with non-standard obs/var index names by @johnkerl in #2281
• [c++] Match delete[] with new[] by @johnkerl in #2259
• [c++] Fix display of Arrow schema for enum of bytes datatype by @johnkerl in #2305
• [c++] Use std::future::get() instead of ::wait() by @eddelbuettel in #2320

Minor changes

• Use core 2.21.1 by @johnkerl in #2301
• Use TileDB core 2.21.0, TileDB-Py 0.27.0, TileDB-R 0.25.0 by @johnkerl in #2294
• [python] Expose tiledbsoma.io.ExperimentAmbientLabelMapping by @johnkerl in #2175
• [c++] Address issues related to reindexer advanced features by @beroy in #2186
• [ci] Fix scripts/bld typo (aarch64 opts) by @ryan-williams in #2193
• [python] Fix a lint fail on main by @johnkerl in #2200
• [python] Fix and test setup.py flags to external shared objects by @jdblischak in #2221
• [python] Typo fixes, broken README link by @ryan-williams in #2235
• [python] Attempt to repro #1565 by @ryan-williams in #2236
• [python] Verify pa.{Chunked,Integer}Array args to SparseNDArray.read by @ryan-williams in #2234
• [python/ci] Restore pre-merge lint checks by @ryan-williams in #2247
• [python] Factor io/ingest.py, create io/{_common,outgest}.py by @ryan-williams in #2246
• [python] Bump somacore to 1.0.9 by @ryan-williams in #2251
• [ci/python] Use Python 3.11 for single version CI runs by @jp-dark in #2255
• [python] Remove deprecated Python 3.7 AST types by @jp-dark in #2271
• [python/ci] Complete 3.8 -> 3.11 work by @johnkerl in #2278
• [python] Neaten registration-mapping test code by @johnkerl in #2285
• [python] Extract fs wrapper to util file by @johnkerl in #2290
• [python] Fix many typeguard warnings by @johnkerl in #1960
• [python/ci] Factor pytest.fixtures, verify #1988 by @ryan-williams in #2299
• [python] Make _util.read_h5ad a @contextmanager by @ryan-williams in #2291
• [python] Make test_empty_categorical_query work with TileDB<2.21 by @ryan-williams in #2308
• [python/ci] Pin typeguard==4.1.5 by @ryan-williams in #2313
• [python] Implement IntIndexer as class that wraps around clib.IntIndexer by @jp-dark in #2310
• [Backport release-1.9] [python/ci] Typofix from #2305 by @github-actions in #2331
• [r] Call array schema evolution after loop setting up enums by @eddelbuettel in #2201
• [ci] Updated hard-coded core version in another file by @johnkerl in #2207
• [ci] Add MacOS job to test shared object copying by @jdblischak in #2220
• Append LD_LIBRARY_PATH for census by @johnkerl in #2218
• [ci] Only run build tests when build files are modified by @jdblischak in #2229
• Needed changes for census profiling by @beroy in #2074
• [r] Correct parenthesis in covr() call activate quiet=FALSE by @eddelbuettel in #2243
• [c++] Fix building TileDB if a CMake toolchain is not specified by @teo-tsirpanis in #2244
• [r/ci] Set options(pkgType = "binary") for MacOS by @mojaveazure in #2257
• [r/ci] Point at correct TileDB-R location by @johnkerl in #2275
• ingest.py cleanups, doc nits by @ryan-williams in #2256
• [r] Add tests for enum value filters in SOMAExperimentAxisQuery by @mojaveazure in #2316
• [r] R version for release-1.9 branch by @johnkerl in #2329

New Contributors

• @jdblischak made their first contribution in #2220

Full Changelog: 1.8.1...1.9.0

1.8.1

Bug fixes

• [Backport release-1.8] [python] Fix an ingestion corner case with empty chunks by @github-actions in #2284
• [Backport release-1.8] [python] Fix registration path with non-standard obs/var index names by @github-actions in #2288

Minor changes

• [Backport release-1.8] [python] Verify pa.{Chunked,Integer}Array args to SparseNDArray.read by @github-actions in #2263
• [Backport release-1.8] [python] Bump somacore to 1.0.9 by @github-actions in #2264
• [Backport release-1.8] [python] Factor io/ingest.py, create io/{_common,outgest}.py by @github-actions in #2266
• [Backport release-1.8] [python] Remove deprecated Python 3.7 AST types by @github-actions in #2272
• [Backport release-1.8] [python] Needed changes for census profiling by @github-actions in #2261
• [Backport release-1.8] [ci/python] Use Python 3.11 for single version CI runs by @github-actions in #2267
• [Backport release-1.8] [python/ci] Restore pre-merge lint checks by @johnkerl in #2268
• [Backport release-1.8] [python/ci] Complete 3.8 -> 3.11 work by @github-actions in #2280
• [Backport release-1.8] [python] Neaten registration-mapping test code by @github-actions in #2287
• [r] Prepare for tiledbsoma 1.8.1 by @johnkerl in #2289
• [Backport release-1.8] [r/ci] Point at correct TileDB-R location by @github-actions in #2276
• [Backport release-1.8] [r/ci] Set options(pkgType = "binary") for MacOS by @github-actions in #2265
• [Backport release-1.8] [r] Correct parenthesis in covr() call activate quiet=FALSE by @github-actions in #2262
• [Backport release-1.8] [c++] Match delete[] with new[] by @github-actions in #2260
• [Backport release-1.8] [c++] Fix building TileDB if a CMake toolchain is not specified by @johnkerl in #2274
• [Backport release-1.8] [ci] Only run build tests when build files are modified by @johnkerl in #2232
• [Backport release-1.8] [docs] Typo fixes, broken README link by @github-actions in #2273

Full Changelog: 1.8.0...1.8.1

TIleDB-SOMA 1.8.0

New features added

• [python] Add default thread pool in SOMATileDBContext by @ebezzi in #2001
• [Backport release-1.8] [python] Expose tiledbsoma.io.ExperimentAmbientLabelMapping by @github-actions in #2179

Breaking changes

• [python] Drop support for Python 3.7 by @johnkerl in #2181

Minor changes

• [Backport release-1.8] [python] Refactor pybind11 inheritance to reduce repetition by @github-actions in #2162
• [r/ci] Controlled downgrade for TileDB-R 0.23 by @johnkerl in #2157
• [release-1.8] Revert #2162 by @johnkerl in #2164
• Sync main to release-1.8 as much as possible, in prep for #2158 by @johnkerl in #2165
• [Backport release-1.8] Backport #2158 and #2166 to release-1.8 by @johnkerl in #2170
• [python] Refactor pybind11 inheritance to reduce repetition by @nguyenv in #2126
• [Backport release-1.8] [c++/ci] Update make format and CI format-checker to find more C++ files by @johnkerl in #2168
• [python] Use _internal_tiledb_config in native_context by @nguyenv in #2166
• [python] Do not copy libtiledbsoma.so into package_data if it already exists by @github-actions in #1937
  -- * Sync main to release-1.8 by @johnkerl in #2192
• [Backport release-1.8] fix scripts/bld typo (aarch64 opts) by @github-actions in #2196
• [Backport release-1.8] [python] Fix a lint fail on main by @github-actions in #2206
• [Backport release-1.8] [ci] Updated hard-coded core version in another file by @github-actions in #2213
  -- * [r] tiledbsoma-r 1.8.3 by @johnkerl in #2215
• [release-1.8] Backport #2220 and #2221 by @johnkerl in #2222

Bug fixes

• [Backport release-1.8] [python] Fix race in SOMADataFrame schema with multiple enumerated columns by @github-actions in #2199
• [Backport release-1.8] [r] Call array schema evolution after loop setting up enums by @github-actions in #2204
• [Backport release-1.8] [python] Fix inconsistency in DictFilterSpec docstring by @github-actions in #2151
• [release-1.8] Fix context-type for tiledbsoma_build_index by @johnkerl in #2227
• [release-1.8] Append LD_LIBRARY_PATH for census by @johnkerl in #2217

TileDB-SOMA 1.7.3

New features added

• [Backport release-1.7] [python] Add default thread pool in SOMATileDBContext by @github-actions in #2174
• [Backport release-1.7] [python] Expose tiledbsoma.io.ExperimentAmbientLabelMapping by @github-actions in #2178

Breaking changes

• [Backport release-1.7] [python] Drop support for Python 3.7 by @github-actions in #2183

Minor changes

• [Backport release-1.7] [python] Refactor pybind11 inheritance to reduce repetition by @github-actions in #2162
• [r/ci] Controlled downgrade for TileDB-R 0.23 by @johnkerl in #2157
• [release-1.7] Revert #2162 by @johnkerl in #2164
• Sync main to release-1.7 as much as possible, in prep for #2158 by @johnkerl in #2165
• [Backport release-1.7] Backport #2158 and #2166 to release-1.7 by @johnkerl in #2170
• [Backport release-1.7] [c++/ci] Update make format and CI format-checker to find more C++ files by @johnkerl in #2171
• Use core 2.19.2 for the release-1.7 branch by @johnkerl in #2177
• [Backport release-1.7] [python] Do not copy libtiledbsoma.so into package_data if it already exists by @github-actions in #2163
• Sync main to release-1.7 by @johnkerl in #2192
• [Backport release-1.7] fix scripts/bld typo (aarch64 opts) by @github-actions in #2195
• [Backport release-1.7] [python] Fix a lint fail on main by @github-actions in #2205
• [Backport release-1.7] [ci] Updated hard-coded core version in another file by @github-actions in #2212
• [r] tiledbsoma-r 1.7.3 by @johnkerl in #2215
• [release-1.7] Backport #2220 and #2221 by @johnkerl in #2223

Bug fixes

• [Backport release-1.7] [python] Fix race in SOMADataFrame schema with multiple enumerated columns by @github-actions in #2198
• [Backport release-1.7] [r] Call array schema evolution after loop setting up enums by @github-actions in #2203
• [Backport release-1.7] [python] Fix inconsistency in DictFilterSpec docstring by @github-actions in #2161
• [release-1.7] Fix context-type for tiledbsoma_build_index by @johnkerl in #2226
• [release-1.7] Append LD_LIBRARY_PATH for census by @johnkerl in #2216

Full Changelog: 1.7.2...1.7.3

TileDB-SOMA 1.7.2

What's Changed

• [Backport release-1.7] [r] Fix Seurat ingestion with missing command logs collection by @johnkerl in #2155
• [r] tiledbsoma 1.7.2 by @johnkerl in #2156

Full Changelog: 1.7.1...1.7.2

TileDB-SOMA 1.7.1

New features added

• [Backport release-1.7] [r] Support ingestion of SeuratCommands in #2106
• [Backport release-1.7] [r] Support outgestion of SeuratCommands in #2114
• [Backport release-1.7] [python] tiledbsoma.io.update_matrix in #2130
• [Backport release-1.7] [python] Outgest additional X layers to AnnData in #2144
• [Backport release-1.7] [python] Add type checks to obsm, obsp, varm and varp early in ingestion in #2138

Breaking changes

Minor changes

• [r] TileDB-SOMA-R 1.7.1 in #2145
• [r] Fix quoting for a particular Docker-build environment (release-1.7) in #2102
• [Backport release-1.7] [r] Let get_tarball.R interop with update-tiledb-version.py in #2104
• [Backport release-1.7] [python] Reorganize Reindexer bindings into separate file in #2111
• [Backport release-1.7] [python] Relax numba dependency in #2127
• [Backport release-1.7] [python] Support Python 3.12 in #2135
• [Backport release-1.7] [python/ci] Leave 3.12 support provisional in #2142
• [Backport release-1.7] [python] Lint fix in #2143

Bug fixes

• [Backport release-1.7] [python] Do not mutate adata.obs/adata.var on ingest in #2105
• [Backport release-1.7] [python] Specially handle cast for object and category dtypes in #2107
• [Backport release-1.7] [python] Accumulate obs/var IDs across batched reads in register_anndatas() in #2113

Full Changelog: 1.7.0...1.7.1

TileDB-SOMA 1.7.0

TileDB-SOMA uses Tile-DB Core 2.19, TileDB-Py 0.25, and Tile-R 0.23.

New features added

• [Backport release-1.7] [python] Using optimized reindexer in blockwise iterators by @johnkerl in #2093
• [Backport release-1.7] [python] Reindexer external python API by @johnkerl in #2092
• [Backport release-1.7] [python] Save off ingest-time obs/var index names for use at outgest by @github-actions in #2072
• [python] Allow specification of X names on ingest by @johnkerl in #1993
• [r] Arrays should remain open after write by @aaronwolen in #1947
• [r] Add support for registering arrays with write_soma() by @mojaveazure in #2013
• [r] Add write_soma.character() by @mojaveazure in #2021
• [r] Update documentation on g++ requirement for R package by @CharlesBordet in #2031
• [r] Support cross compiling on MacOS with R by @jeroen in #2034

Minor changes

• [Backport release-1.7] [c++] Handling edge cases for C++ re-indexer (#2098) (#2100)
• [Backport release-1.7] [python] Elaborate docstring re relativity for add-new by @github-actions in #2089
• [Backport release-1.7] [python] Rename utils.py to _index_util.py by @github-actions in #2058
• [Backport release-1.7] [python] Add common.h to MANIFEST.in by @github-actions in #2043
• [Backport release-1.7] [python] Update unit tests for pandas 2.2 and scipy 1.12 by @github-actions in #2044
• [Backport release-1.7] [r] Enable opt-in use of 'catchsegv', enable in CI on Linux by @github-actions in #2046
• [Backport release-1.7] [python] Silence a unit-test warning by @github-actions in #2070
• [Backport release-1.7] [c++/ci] Fix a pkgdown CI error about unnecessary semicolons by @github-actions in #2051
• Depend on TileDB core 2.19, TileDB-Py 0.25, TileDB-R 0.23 by @johnkerl in #2002
• Update to use somacore==1.0.7. by @thetorpedodog in #2000
• Bump jinja2 from 3.0.0 to 3.1.3 in /doc by @dependabot in #2007
• [python] Depend on somacore 1.0.6 by @johnkerl in #1940
• [python] Add unit tests for obsm, obsp, and to_anndata by @ebezzi in #1934
• [python] QueryCondition should error out if given empty set by @nguyenv in #1967
• [python] Dataframe read path by @nguyenv in #1793
• [r] tiledbsoma-r 1.7.0 by @johnkerl in #2041
• [r/ci] Show CPU info in R CI as well as Python CI by @johnkerl in #1956
• [r] Apply PR 1943 to main by @johnkerl in #1957
• [r] Tweak coverage data generation by @eddelbuettel in #1990
• [r] Bump apis/r/DESCRIPTION for main past current latest release by @johnkerl in #2005
• [r] Add a helper function to determine max value per int type by @eddelbuettel in #2022
• [c++] Modify ManagedQuery to perform async queries by @nguyenv in #1953
• [c++] Have tiledbsoma-cli link to tiledbsoma by @teo-tsirpanis in #1965
• [c++] Extend codecov to cover tiledbsoma library by @beroy in #1970
• [c++] Append TILEDBSOMA_COVERAGE to cmake flags by @nguyenv in #2019
• [c++] Reindexer overrides and fast COO/CSR by @beroy in #1728
• [Backport release-1.7] Use core 2.19.1 by @github-actions in #2085

Bug fixes

• [Backport release-1.7] [c++] Fix for error when the key array size is smaller than thread count by @github-actions in #2097
• [Backport release-1.7] [python] Fix some docstring typos by @johnkerl in #2095
• [Backport release-1.7] [python] Pass config options to DataFrame.read by @github-actions in #2059
• [Backport release-1.7] [c++] Fix zero-size lookup chunk size for the C++ re-indexer by @github-actions in #2062
• [Backport release-1.7] [python] Chunked writes for DataFrame; byte-caps for Arrow-table writes by @github-actions in #2064
• [python] Fix from_anndata() to respect var_id_name by @aaronwolen in #1963
• [python] Fix data loss for multiple writes with Enumeration extend by @nguyenv in #1978
• [python] Error if extending enum past index type limit by @nguyenv in #1986
• [r] Ensure factors are releveled before appending w/ expanded levels by @eddelbuettel in #1983
• [r] Ensure addition of factor levels does not overflow capacity of index type by @eddelbuettel in #2008

Ongoing progress

• [python] Add Windows support for the Python API by @teo-tsirpanis in #1811 (Note: this is still a WIP)
• [python] Revert "Add Windows support for the Python API" by @johnkerl in #1959
• [python] Add back Windows support for the Python API, but without CI by @teo-tsirpanis in #1961

New Contributors

• @teo-tsirpanis made their first contribution in #1811
• @dependabot made their first contribution in #2007
• @CharlesBordet made their first contribution in #2031
• @jeroen made their first contribution in #2034

Full Changelog: 1.6.2...1.7.0

TileDB-SOMA 1.7.0rc1

What's Changed

In addition to https://github.com/single-cell-data/TileDB-SOMA/releases/tag/1.7.0rc0:

• [Backport release-1.7] [c++/ci] Fix a pkgdown CI error about unnecessary semicolons by @github-actions in #2051
• [Backport release-1.7] [python] Rename utils.py to _index_util.py by @github-actions in #2058
• [Backport release-1.7] [python] Pass config options to DataFrame.read by @github-actions in #2059
• [Backport release-1.7] [c++] Fix zero-size lookup chunk size for the C++ re-indexer by @github-actions in #2062
• [Backport release-1.7] [python] Chunked writes for DataFrame; byte-caps for Arrow-table writes by @github-actions in #2064

Full Changelog: 1.7.0rc0...1.7.0rc1