Higher resolution maps

[freeman-lab]

We have reason to think that maps / pyramids with significantly higher zoom levels may cause performance challenges, especially with respect to the way we generate tilesets in memory. We should get sample pyramid dataset with more levels (see e.g carbonplan/trace#123) and then start testing this out, and document any challenges and progress here.

[iferencik]

@freeman-lab

I'd like to inquire about this issue/feature and a bunch of others. I am not sure if this is the right place to inquire, but I am going to do it anyway because your repo does not feature discussions. I have a use case where I plan to publish country level high resolution GeoTiff data (30 meters). This data is a bit specific in the sense that it was produced by a model and many pixels are set to nodata (around 80%) so it looks almost like a sparse dataset.

My first step was to convert this to zarr using ndpyramid and I immediately hit a wall (size/memory). In the end i used rioxarray + rasterio + dask to reproject my dataset to EPSG:3857 and then I resampled the reprojected dataset for various zoom levels/resolutions using xarray.coarsen and cretaed a multiscale zarr.

Of course, when I reprojected it I did not make it a global dataset but rather I sticked to it's original spatial extent (country level). I started a local python server to serve the zarr over HTTP but the dataset would not load, but i also was not able to see any errors in the console.

I started to compare the multilevel zarr I have created with the one I created using [ndpyramid.reproject ].(https://github.com/carbonplan/ndpyramid/blob/main/ndpyramid/regrid.py#L104) and I realized my dataset was not global. Other than that there was not difference (.zmetadata was not missing any props) . So given this context, I'd like to know:

1. does carbonplan/maps lib work only with global datasets?
2. if yes, what would be required to make it work using nonglobal datasets (country level)?
3. The example data you use is of type float. Are int types zarr supported? (i1/u1) ?

Thanks