-
Notifications
You must be signed in to change notification settings - Fork 2
3 Read
Carmen Tawalika edited this page Nov 12, 2021
·
9 revisions
concept (70/100)
- location in which to calculate is determined by
load_collection
process from openEO, or user depending on which endpoint is called - actinia is responsible for reading in the needed collections, applying filters (space + time) to collect needed items and create the process chain for GRASS GIS
- see more details for filters here. It is necessary to support the filters first, which reduce data size best
- no development is needed in GRASS GIS
- r.in.gdal (r.import) will be used for all imports. Actinia will pass information (data location, bands, extent, ...)
- single scenes are registered directly as STRDS (no mosaic for prototype) -> no real difference for the user, it might take longer and use more storage space
- STAC will be a new type of importer (
import_descr
) - this needs to be enabled within actinia-core
- as soon as possible, a dedicated plugin should be called (actinia-stac-plugin or actinia-stac-importer-plugin)
- plugin should do as much as possible (as little as possible inside actinia-core)
- plugin should be called and only return process (or process chain piece) to actinia-core
- plugin should download to tmp folder / resource storage, where data will be deleted after processing / can be deleted manually -> look at other import options like sentinel
- for a first draft it is fine to do everything within actinia and move to plugin afterwards
- Reading of data has nothing to do with STAC but depends on where the items are and how they can be retrieved
- All needed data for retrieving it needs to be in STAC Metadata
- Type of storage (file mount, S3, copernicus API hub, ...)
- will be implemented on-demand (first for one example, one after another for different storage types / STAC instances / collections
- Determined by analysing path to resource (e.g. content of item.assets.B01.href)
- Use common band names where possible when registering data
- Implement look-up table for "common not common band names", eg as seen here
- only needed extent should be downloaded if possible - should be possible with COG
- pass Bandinformation
- in later stage, choose special GRASS GIS modules for metadata import (e.g. i.sentinel)
- usually determined by which endpoint is called (
/api/v1/locations/nc_spm_08/processing_async_export
) - currently, location is determined by openeo-grassgis-driver depending on location of input dataset
- what, if multiple inputs are used from different locations?
- currently, user is not allowed to create location, only mapser. Admin is allowed
- TODO: enable user to create locations but with limit configurable -> check if temporary location is possible
not nice: change projection in current location on-the-fly- Analyse input data and decide on this basis, in which location the processing should be done
- consider projection of assets and filter spatially, so that location is clear before real processing starts
- in prototypical implementation, reduce complexity:
- user should decide via openeo process
load_collections
- it is required to do any processing with openEO
- Default for CRS for BBOX filter is
4326
- It is meant to define BBOX for filter, not data, so it could confuse user more than it helps
- OpenEO API allows defining it, but can it be also manipulated with web editor?
- user should decide via openeo process
- Should GRASS GIS be able to read STAC?
- -> would be nice to have but currently no real requirement
- for each data type, a different module would be used -> a lot of metadata analysis is needed
- could be done by GRASS GIS addon (r.in.stac / t.in.stac ?) input would then be a URL to a Feature / FeatureCollection with bands. Example
- Should actinia analyse the data?
- If done via Importer, data would be cleaned
- actinia is able to do everything that GRASS GIS can do, including forward to dedicated GRASS GIS import modules -> no disadvantage
- Advantage of actinia: analysis of input data location can be done before processing
- => Decision for actinia-stac-plugin/actinia-stac-importer-plugin
- r.in.gdal (r.import) / r.external / special modules like i.sentinel / viscurl ?
- differentiate between storage (mount, local copy for landsat + Sentinel Copernicus, use GDAL for S3)
- prevent download where possible?
- where is the data stored? How large are the extents?
- do we need to determine by pixel limit?
- little extent => r.exernal
- if gdalinfo is able to read => r.external, else r.in.gdal
- => r.in.gdal (r.import) wins for now. If we have problems in certain use cases later, we can use r.external for them
- For prototypical implementation, we focus on single scenes/bands
- actinia downloads everything, location is determined by
load_collection
(used actinia endpoint) and single scenes are registered directly as STRDS