TileDB-SOMA 1.2.7

This is a cherry-pick of a few Python-only bugfix items from main. The bulk of current development work at present is on the R and C++ fronts, most likely to be delivered in an upcoming 1.3.0 release.

Note: there is no 1.2.6.

• [python] Backport #1471 from main to release-1.2 branch by @johnkerl in #1483
• [python] Backport #1472 from main to release-1.2 branch by @johnkerl in #1484
• [python] Backport #1468 from main to release-1.2 branch by @johnkerl in #1485
• [python] Backport #1459 from main to release-1.2 branch by @johnkerl in #1486

Full Changelog: 1.2.5...1.2.6

TileDB-SOMA 1.2.5

This is a cherry-pick of a few items from main, branched from 1.2.4. The main goal is getting the lifecycle: maturing tags released into GitHub and PyPI.

What's Changed

• [python] 1.2.5 backports by @johnkerl in #1432

Full Changelog: 1.2.4...1.2.5

TileDB-SOMA 1.2.4

New features added

• [python] MacOS arm64 wheel-builds for PyPI by @johnkerl in #1385

Ongoing development toward 1.0 R release

• [r] SOMASparseNDArray: sparseMatrix zero-based shim to facilitate soma_joinid lookup by @mlin in #1313
• [r] Port notebooks to R vignettes by @aaronwolen in #1337
• [r] Put like with like in test-helpers by @johnkerl in #1349
• [r] Add some whitespace for improved readability by @johnkerl in #1350
• [r] Unit-test failure on Mac M1 by @johnkerl in #1357
• [r] Re-order methods in ephemeral.R by @johnkerl in #1359
• [r] Allow default index_column_names=c("soma_joinid") by @johnkerl in #1363
• [r] Refactor build by as-needed rebuilding libtiledbsoma by @eddelbuettel in #1361
• [r] Update MacOS build for developer-with-local-core corner case by @johnkerl in #1368
• [r] Split up large test-SeuratOutgest.R by @johnkerl in #1373
• [r] Provide reference to libtiledbsoma source via soft-link by @eddelbuettel in #1372
• [r] Implement stateful open/close by @johnkerl in #1325
• [r] Set a define so that spdlog uses REprintf by @eddelbuettel in #1381
• [r] Remove two debug statements by @johnkerl in #1382
• [r] Address remaining two R CMD check nags by @eddelbuettel in #1384
• [r] Update mode semantics for implicit open by @johnkerl in #1383
• [r] Update Seurat ingestor to skip the data slot when it's identical to counts by @aaronwolen in #1389

Ongoing C++ development

• [c++] Move installed header files one level deeper by @nguyenv in #1338
• [c++] Adjust two more include paths by @eddelbuettel in #1341
• [c++] Remove unused references to removed R_BUILD option by @eddelbuettel in #1344
• [c++] Remove remaining traces of BUILD_R from library CMakeLists.txt by @eddelbuettel in #1345
• [c++] Addition of SOMAGroup by @nguyenv in #1339
• [c++] Make SOMA{Array,Group} Changes According To Review in #1339 by @nguyenv in #1352

Minor changes

• [python] Update platform_config usage advice by @johnkerl in #1371
• [python] Reduce default goal_chunk_nnz by @johnkerl in #1393
• [python] Auto-consolidate fragment metadata and commits by @johnkerl in #1284
• [python] Add run-time type checks to PlatformConfig on ingest/create by @johnkerl in #1370
• [r/docs] Clean up documentation by @mojaveazure in #1379
• [docs] Added two badges (and ensure consistent two break before sections) by @eddelbuettel in #1375

Bug fixes

• [python] Update obsm-chunking logic by @johnkerl in #1380
• [python] PyPI-publish workflow by @bkmartinjr in #1388
• [python] Improve resume-mode experience by @johnkerl in #1396
• [docs] Fix signup link for TileDB Community Slack by @johnkerl in #1369

Full Changelog: 1.2.3...1.2.4

TileDB-SOMA 1.2.3

Overview

• Improve Seurat I/O
• Add support for unsigned/public AWS/S3 URIs
• Fix query conditions for boolean attributes in R

New features added

• [r] Seurat <> SOMA Integration Improvements by @mojaveazure in #1305

Minor changes

C++

• [c++] Rename SOMAArrayReader to SOMAArray by @nguyenv in #1283
• [c++] Add Array metadata methods by @nguyenv in #1323
• [c++] Add cstring Header to util.cc by @nguyenv in #1309
• [c++] Do not install source files by @nguyenv in #1319
• [c++] Install span header by @nguyenv in #1311
• [c++] Move two headers and adjust two include statements by @eddelbuettel in #1318
• [c++] Reorganize directory to separate public and private headers by @nguyenv in #1301

R

• [r] Add new example dataset and methods for accessing example datasets by @aaronwolen in #1298
• [r] Additional neatification of src/Makevars{,.in} by @eddelbuettel in #1321
• [r] Build against shared library libtiledbsoma by @johnkerl in #1302
• [r] Condition on more test on SeuratObject by @eddelbuettel in #1320
• [r] Remove all (already unused) cases of the R_BUILD define by @eddelbuettel in #1304
• [r/python] Seurat to AnnData test by @ebezzi in #1316

Python

• [python] Depend on somacore 1.0.2 by @johnkerl in #1292
• [python] Add a run-time type check on contexts provided to tiledbsoma.io by @johnkerl in #1297
• [python] Mark tiledbsoma as containing type annotations for users by @thetorpedodog in #1324
• [ci/python] Publish to TestPyPI on nightly or manual run by @johnkerl in #1291
• [docs/python] Expand docstrings for Experiment and Measurement by @thetorpedodog in #1230

Bug fixes

• [c++/r/python] Update to core 2.15.2, TileDB-Py 0.21.3, TileDB-R 0.19.1 by @johnkerl in #1303
• [python] Require TileDB-Py >= 0.21.2 by @johnkerl in #1296
• [python] Tighten TileDB pypi dependency to be a 0.21.x version. by @thetorpedodog in #1299

Full Changelog: 1.2.2...1.2.3

TileDB-SOMA 1.2.2

Highlights

• R Seurat I/O
• Full support for Python 3.11

New features added

• [r] Add Seurat -> SOMA Ingestor by @mojaveazure in #1146

• [r] Provide context() accessor by @eddelbuettel in #1205

• [r] Return lists of Arrow 'Table' objects in read(iterated=TRUE) by @eddelbuettel in #1214

• [r] Seurat outgestors expect sparse arrays in obsm and varm by @mojaveazure in #1225

• [python/ci] Include Python 3.11 in CI matrix by @johnkerl in #599

• [python] Store obsm and varm sparsely by @johnkerl in #1246

• [docs/python] Add tutorials.rst by @ebezzi in #1243

Minor changes

• [r] Coerce obs_joinids() and var_joinids() to vectors by @mojaveazure in #1281

• [r][python] Create an AnnData -> SOMA -> Seurat test by @ebezzi in #1263

• [r] Refine group member caching behavior by @aaronwolen in #1272

• [r] More renames SOMAReader -> SOMAArrayReader by @johnkerl in #1191

• [r] Improve R query performance with obs and var both specified by @johnkerl in #1195

• [r] Update result_order docs and defaults by @aaronwolen in #1196

• [r] Rerun roxygenise by @johnkerl in #1204

• [r] Rename NdArray file names to match NDArray class names by @johnkerl in #1218

• [r] Add pbmc3k.tiledb info to apis/r/README.md by @johnkerl in #1219

• [r] Static library is configured to install to lib by @nguyenv in #1221

• [r] Correct typo and minor edit to configure by @eddelbuettel in #1231

• [r] Port ephemeral classes to R by @mojaveazure in #1212

• [r] Port TileDBCreateOptions from Python to R by @johnkerl in #1216

• [r] Fix (venial) typo in R unit test by @johnkerl in #1244

• [r] Upcast SOMADataFrame coords to int64 by @aaronwolen in #1254

• [r/ci] Update R MacOS CI, given R 4.3.0 by @johnkerl in #1275

• [c++] Final rename SOMA_READER -> SOMA_ARRAY_READER by @johnkerl in #1201

• [c++] Fix build on MacOS 13.3 by @johnkerl in #1206

• [c++] Omit no-longer-needed libtiledbsoma commit-hash info from show_package_versions by @johnkerl in #1207

• [c++] Ensure cmake parameters are used and passed from superbuild by @Shelnutt2 in #1208

• [c++] Update MacOS 13 check from #1206 by @johnkerl in #1235

• [docs] Update name of CZ CELLxGENE Census by @bkmartinjr in #1211

• [docs] CZI Slack channel name change by @bkmartinjr in #1213

• [docs/python] Remove main-old branch documentation by @atolopko-czi in #1187

• [docs/python] More doc links/styling by @johnkerl in #1210

• [ci/python] Auto-create a GH issue on nightly CI fail by @johnkerl in #1233

• [python/r] Always write TileDB Cloud–specific metadata to Experiments by @aaronwolen in #1286

• [python] Write TileDB Cloud–specific metadata to Experiments by @thetorpedodog in #1260

• [python] Add tests for *_dim_zstd_level create options by @thetorpedodog in #1215

• [python] Parse TILEDB_PATH and TILEDBSOMA_PATH in setup.py by @nguyenv in #1189

• [python] Include PyPI wheel-publish in release CI by @johnkerl in #1190

• [python] Put capacity on the same footing as other PlatformConfig parameters by @johnkerl in #1229

• [python] Support CLI config arguments in devtools/ingestor by @johnkerl in #1227

• [python] Put allows_duplicates on the same footing as other PlatformConfig parameters by @johnkerl in #1234

• [python] Pin anndata to < 0.9 for Python 3.7 since it doesn't support Python 3.7 like we do by @johnkerl in #1252

• [python] Flexible --libtiledbsoma specification in setup.py by @johnkerl in #1253

Bug fixes

• [r] Upcast axis query coords to integer64 by @eddelbuettel in #1209

• [r] Error on incomplete (non-iterated) reads by @eddelbuettel in #1194

• [python] Explicit casting of Numpy nulls to Arrow nulls by @mojaveazure in #1265

• [python] Unwrap numpy scalar values when importing uns data by @thetorpedodog in #1268

• [python] Handle ingest of completely empty sparse arrays by @johnkerl in #1267

• [python] Avoid context leak on group-set-timestamp operation; use TileDB Embedded 2.15.1 by @johnkerl in #1199

• [python] Wire context through tiledbsoma.io.from_anndata by @johnkerl in #1224

• [ci] Fix publish_to_pypi.yml parse error by @mlin in #1236

Full Changelog: 1.2.1...1.2.2

TileDB-SOMA 1.2.1

Bug fixes

The primary purpose of this release is for Conda builds: TileDB-Inc/tiledbsoma-feedstock#9 (comment)

• [python] Don't use distro in show_package_versions by @johnkerl in #1186
• [R] Fix to_seurat() cell-level meta data by @mojaveazure in #1179

Full Changelog: 1.2.0...1.2.1

TileDB-SOMA 1.2.0

Summary

• Support for ingesting uns data from AnnData files
• Steady progress toward R 1.0 release candidate
• Doc-site evolution

New features added

• [r] Adding factory methods by @eddelbuettel in #1123

• [r] Pass config through shape() and nnz() by @mlin in #1138

• [r] Add general opener by @eddelbuettel in #1141

• [r] Make obj$close() public by @eddelbuettel in #1142

• [r] Add SOMA Experiment Query -> Seurat outgestors by @mojaveazure in #1119

• [r] Clarify status of DenseNDArray writes and matrices by @eddelbuettel in #1160

• [r] Also show osVersion when displaying package versions by @eddelbuettel in #1176

• [python] Ingest uns values from AnnData files by @thetorpedodog in #1157

• [python] Allow to_anndata when feature_id not var_id is in var columns by @johnkerl in #1136

Minor changes

• [r] Tweak CI on Ubuntu by taking advantage of newest bspm release by @eddelbuettel in #1168

• [r] Rename soma_reader -> soma_array_reader within source code by @johnkerl in #1174

• [python] Ensure that libtiledbsoma_dir is always a pathlib.Path by @thetorpedodog in #1155

• [python] A round of cross-references by @johnkerl in #1161

• [python] Depend on somacore 1.0.1 by @johnkerl in #1167

• [python] Include OS info in show_package_versions by @johnkerl in #1177

• [c++] Install cmake Target and Config for libtiledbsoma by @Shelnutt2 in #1159

• [c++] Pass TILEDBSOMA_ENABLE_TESTING and TILEDBSOMA_BUILD_CLI in INHERITED_CMAKE_ARGS by @Shelnutt2 in #1162

• [r/python] Support bytes metadata field + R SOMA_ENCODING_VERSION by @ebezzi in #1149

• [r/python] R-Python roundtrip testing by @ebezzi in #1139

• [r/python] DenseNDArray R-Python roundtrip testing by @ebezzi in #1158

• [r/python] fix TestDenseNDArrayWriteRReadPython by @ebezzi in #1163

• [r/python] SparseNDArray R-Python roundtrip testing by @ebezzi in #1165

• [c++/r/python] Modify SOMAReader To SOMAArrayReader by @nguyenv in #1147

• Filenames soma_reader -> soma_array_reader by @johnkerl in #1173

• [docs/python] Add more lifecycle-experimental indicators to docstrings [WIP] by @johnkerl in #1153

• [docs] Add notebooks as tutorials to the docsite by @ebezzi in #1140

• [docs/python] Fix hyperlinks in Sphinx docs by @johnkerl in #1152

• [docs/python] Add more external links for docsite by @johnkerl in #1154

• [docs/python] Neaten Sphinx config by @johnkerl in #1166

Bug fixes

• [r] Fix reading very sparse assays by @mojaveazure in #1156

Full Changelog: 1.1.1...1.2.0

TileDB-SOMA 1.0.1

What's Changed

The sole modification in this bugfix update to 1.0.0 is to update the library version of TileDB Embedded. The 1.1.1 release and above is appropriate for general-purpose use.

• [c++] Have TileDB-SOMA 1.0 depend on TileDB Embedded 2.14.2 by @johnkerl in #1145

Full Changelog: 1.0.0...1.0.1

TileDB-SOMA 1.1.1

Primary features

• This release brings in TileDB Embedded 2.15.0, along with TileDB-Py 0.21.1 and TileDB-R 0.19.0, in addition to updates for Conda installs
• Ongoing R development
• No significant changes for Python users

New features added

• Add examples to docstrings by @bkmartinjr in #1109

Bug fixes

• [python] Fix wheel builds given recent Conda-related changes by @johnkerl in #1134
• [ci] Ensure macOS builds use clang instead of gcc by @mlin in #1127
• [python] Pin typeguard to < 3.0 by @johnkerl in #1115

Minor changes

• [r] SOMASparseNDArray$read_sparse_matrix: set dims on the returned sparseMatrix by @mlin in #1110
• [r] SOMASparseNDArray$read_sparse_matrix: set dims on iterated sparseMatrix parts by @mlin in #1125
• [r] Fix testing logic for iterated reads from SOMASparseNDArray by @mlin in #1112
• [r] Limit pkgdown runs to releases and manual triggers by @eddelbuettel in #1131
• [r] Update documentation with roxygen2 by @eddelbuettel in #1133
• [python] Trivial typofix by @johnkerl in #1120
• [ci] Don't mark a run as failed if coverage upload fails. by @thetorpedodog in #1132

Full Changelog: 1.1.0...1.1.1

TileDB-SOMA 1.1.0 -- please use 1.1.1

Superseded

This release has no installable PyPI wheels and has been marked as pre-release. Release 1.1.1 fixes these issues.

Main goal

This release brings in TileDB Embedded 2.15.0, along with TileDB-Py 0.21.1 and TileDB-R 0.19.0. It also brings in updates for Conda installs.

New features added

• [python] Support Conda builds on POSIX by @nguyenv in #1043

Minor changes

• [c++,r,python] Update to core version 2.15.0 by @johnkerl in #1066
• [c++/r/python] Add libtiledbsoma info to show_package_versions() by @mlin in #1101
• [c++/python/r] Add shape to C++ SOMAReader by @nguyenv in #1083
• [r] Add test for nnz() when duplicates==TRUE by @eddelbuettel in #1107

Full Changelog: 1.0.0...1.1.0