[c++, r] Update and refactor nanoarrow

[eddelbuettel]

Issue and/or context:

We rely on nanoarrow to take advantage of higher-level structures for the (otherwise 'naked' void * of the C API of Arrow. nanoarrow helps here, and has made good strides in crecent releases.

Changes:

We update the 'vendored' nanoarrow to release 0.4.0, and make extended use of it. This works well: simple examples work fine and show the desired memory behaviour. However, on larger examples we see interactions with (local) handling of Arrow objects in libtiledbsoma so the scope of the PR will have to extended to handle allocating, releases and setting up Arrow objects via nanoarrow.

Notes for Reviewer:

This PR is informational to run some CI; all local tests pass. However, at present an invasice change in libtiledbsoma makes this unsuitable for a merge at present meaning it is also not yet fully ready for a review.
The secret cabal powers of CI are currently smiling upon us (after a lot of help from several people, and a last minute bug fix) so the 'draft' label is now off. There are remaining 'drafty' things here (such as a move of nanoarrow.* to a directory below external/ but we can do so in a later PR.

[shortcut-integration]

This pull request has been linked to Shortcut Story #42316: [r] Refactor to make use of nanoarrow.

[codecov]

Codecov Report

Merging #2188 (7bb42fb) into main (b34e727) will decrease coverage by 11.48%.
Report is 1 commits behind head on main.
The diff coverage is 7.30%.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2188       +/-   ##
===========================================
- Coverage   77.97%   66.49%   -11.48%     
===========================================
  Files         140      142        +2     
  Lines       10855    12743     +1888     
  Branches      217      511      +294     
===========================================
+ Hits         8464     8474       +10     
- Misses       2303     4169     +1866     
- Partials       88      100       +12     

Flag	Coverage Δ
libtiledbsoma	38.85% <0.17%> (-24.77%)	⬇️
python	90.59% <ø> (ø)
r	72.80% <65.21%> (-2.32%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
python_api	90.59% <ø> (ø)
libtiledbsoma	19.75% <0.17%> (-28.45%)	⬇️

[eddelbuettel]

The dual nanoarrows (from inside the R package, and inside libtiledbsoma) can likely be remedied. The location in tiledbsoma should be in externals/ anyway -- a follow-up PR seems apt.

[johnkerl]

The dual nanoarrows (from inside the R package, and inside libtiledbsoma) can likely be remedied. The location in tiledbsoma should be in externals/ anyway -- a follow-up PR seems apt.

I created #2362 to track this

[eddelbuettel]

@johnkerl One triplet of nanoarrow files is now gone.

[johnkerl]

🚢

[johnkerl]

Thanks @eddelbuettel !!

Just to set an expectation, this may not backport automatically to release-1.9 since apis/r/DESCRIPTION is touched and that differs between main and release-1.9 -- not to worry, I'll do a manual backport tomorrow morning if so

[nguyenv]

I know I'm the one who added this but I'm not sure that it's necessary here. I may have overcorrected.

[eddelbuettel]

I think it is very much needed. It is just stylistic to use strdup() which allocates. If you peek into what nanoarrow does it actually allocates explicitly with malloc (as do I over in tiledb-r)

ArrowErrorCode ArrowSchemaSetFormat(struct ArrowSchema* schema, const char* format) {
  if (schema->format != NULL) {
    ArrowFree((void*)schema->format);
  }

  if (format != NULL) {
    size_t format_size = strlen(format) + 1;
    schema->format = (const char*)ArrowMalloc(format_size);
    if (schema->format == NULL) {
      return ENOMEM;
    }

    memcpy((void*)schema->format, format, format_size);
  } else {
    schema->format = NULL;
  }

  return NANOARROW_OK;
}

This is different from what it also has 'view' inits for schema which just borrow the pointed to memory because the contract the viewer can assume the viewed object exists.

In short, your strdup() was fine. And after 30+ plus years of C it possibly taught me another K+R (at style) function so all good!

Also note that if you peek at the matching release function they free() so there is an assumption of allocated memory here.

[nguyenv]

Nanoarrow format strings here, not TileDB

[eddelbuettel]

Actually Arrow format string (which is the same as Arrow). Will adjust.