ornitela_to_movebank.Rmd

---
title: "Prepare Ornitela (meta)data for Movebank upload"
author: "Peter Desmet"
date: "`r Sys.Date()`"
output:
  html_document
---

This script prepares [Ornitela](https://www.ornitela.com/) bird tracking data for upload to [Movebank](https://www.movebank.org/) For a project, it downloads **reference data** from a Google spreadsheet where these data are maintained and transforms these to the [Movebank data format](https://www.movebank.org/node/2381) using `dplyr::mutate()` (see below). **GPS data** are automatically synced with Movebank (see `Live Feeds` in a Movebank study). This is not yet implemented for **acceleration data** which are currently not uploaded.

```{r setup, include = FALSE}
knitr::opts_chunk$set(echo = TRUE, warning = FALSE, message = FALSE)
```

Load libraries:

```{r}
library(tidyverse)
library(here)
```

Set user-defined values:

```{r}
project_id <- "LBBG_JUVENILE"
```

## Get Ornitela tag and bird information from spreadsheet

```{r}
metadata <- readr::read_csv("https://docs.google.com/spreadsheets/d/e/2PACX-1vSU838D5jOOcPSyL21iheYdC1hEwYaR88x9qhJNLOZtmaTuK6hv23Vpz-o1b5erXur4Vw2g9Z2mQchh/pub?gid=0&single=true&output=csv")
metadata_colnames <- colnames(metadata)
```

## Get reference data (metadata)

For a project, get data on animals, tags and deployments in the [Movebank reference data format](https://www.movebank.org/node/27).

Note: [UvA-BiTS reference data](../sql/movebank_ref.sql) has some fields that are not available in the Ornitela metadata. These are indicated with `NOT AVAILABLE` below and are not included in `movebank_ref_data.csv`, as their empty values would lead to import errors in Movebank. Since UvA-BiTS has more fields, use `bind_rows(uvabits_project_metadata, ornitela_project_metadata)` (UvA-BiTS first) to have all columns in the right order when you want to combine metadata.

Map data:

```{r get_ref_data}
movebank_ref_data <-
  metadata %>%
  dplyr::filter(project == project_id) %>%
  dplyr::mutate(
    # animal-comments: NOT AVAILABLE
    `animal-id` = metal_ring,
    `animal-nickname` = dplyr::case_when(
      str_detect(animal_name, "OT-") ~ NA_character_, # Exclude names that are the default tag name
      TRUE ~ animal_name
    ),
    `animal-ring-id` = colour_ring,
    `animal-sex` = dplyr::recode(sex,
      "F" = "f",
      "M" = "m",
      "X" = "u",
      .missing = "u"
    ),
    `animal-taxon` = scientific_name,
    `alt-project-id` = project_id,
    `animal-life-stage` = dplyr::recode(age,
      "A" = "adult",
      "J" = "juvenile"
    ),
    `animal-mass` = animal_weight,
    `attachment-type` = "harness",
    `deploy-off-timestamp` = format(
      as.POSIXct(track_session_end_date, tz = "UTC", format = "%Y-%m-%d %H:%M:%S"),
      "%Y-%m-%dT%H:%M:%SZ"
    ),
    # deploy-on-latitude: NOT AVAILABLE
    # deploy-on-longitude: NOT AVAILABLE
    `deploy-on-measurements` = animal_measurements,
    `deploy-on-timestamp` = format(
      as.POSIXct(track_session_start_date, tz = "UTC", format = "%Y-%m-%d %H:%M:%S"),
      "%Y-%m-%dT%H:%M:%SZ"
    ),
    `deployment-comments` = track_session_remarks,
    `deployment-end-type` = dplyr::case_when(
      stringr::str_detect(tolower(track_session_remarks), "dead") ~ "dead",
      stringr::str_detect(tolower(track_session_remarks), "predated") ~ "dead",
      stringr::str_detect(tolower(track_session_remarks), "victim") ~ "dead",
      stringr::str_detect(tolower(track_session_remarks), "defect") ~ "equipment failure",
      stringr::str_detect(tolower(track_session_remarks), "malfunction") ~ "equipment failure",
      TRUE ~ NA_character_
    ),
    # deployment-id: NOT AVAILABLE
    # location-accuracy-comments: NOT AVAILABLE, there is no location-error-numerical/vertical-error-numerical in the gps data
    `manipulation-type` = dplyr::case_when(
      stringr::str_detect(track_session_remarks, "VOC") ~ "manipulated other",
      # Birds from VOC were hatched from egg or raised as chicks
      # e.g. placed in controlled environment and subjected to behavioural studies
      TRUE ~ "none"
    ),
    `study-site` = release_location,
    `tag-readout-method` = "phone network",
    `sensor-type` = "GPS",
    `tag-id` = serial_number,
    `tag-manufacturer-name` = "Ornitela",
    `tag-mass` = tag_weight,
    `tag-serial-no` = serial_number,
    .keep = "none"
  ) %>%
  dplyr::arrange(`tag-id`)
```

Save to CSV:

```{r}
readr::write_csv(movebank_ref_data, file = here::here("data", "processed", project_id, "movebank_ref_data.csv"), na = "")
```

## Get gps data (detections)

Done via live feed. Important notes when comparing number of records:

```R
ornitela_export %>%
  # Exclude records with lat/long = 0: these are not fed to Movebank
  dplyr::filter(satcount != 0) %>%
  # Exclude duplicated timestamps: these are not fed to Movebank
  dplyr::distinct(device_id, UTC_datetime, .keep_all = TRUE) %>%
  # Use the same cutoff date
  dplyr::filter(UTC_datetime <= as.POSIXct("YYYY-MM-DD 23:59:59")
  
EQUALS

# Select "Include undeployed locations" when exporting
movebank_export %>%
  # Exclude SMS records
  dplyr::filter(`orn:transmission-protocol` == "GPRS")
  # Use the same cutoff date
  dplyr::filter(timestamp <= as.POSIXct("YYYY-MM-DD 23:59:59")
```

## Get acceleration data

Not implemented yet.