Skip to content

v0.11.1

Compare
Choose a tag to compare
@Ludee Ludee released this 16 May 22:49

[v0.11.0] 2022-05-16

The code becomes a python package and will be available on pypi.org
Additionally, a new datasource was identified and can be used: the xml bulk download.
The API was updated to the newest version and the data model was adapted.

Added

  • The class :class:open_mastr.mastr.Matr
    was introduced as the entrypoint for users, the API download was included in this entrypoint
    #203
  • A method for downloading and parsing the xml dump from the MaStR website
    was implemented #202
  • New data classes and attributes were introduced to orm.py #209
  • The documentation page was updated
  • Unit tests were created #207
  • A CI pipeline was introduced #208
  • The metadata was updated #219

[v0.10.0] 2020-10-08 - unreleased

A complete refactoring took place! Downloading data was entirely changed; introducing layers of code and removing
duplicated code while more of less following DRY.
Moreover, post-processing was changed to be more accessible and easier to execute. For example, docker now helps to
spin up a database container.
The documention on RTD was extended, update and improved to be more helpful for new users.
Read more about the details:

Added

  • added more technologies

  • added documentation for ReadTheDocs

  • improved parallel download

  • merged all stale branches

  • The class :class:open_mastr.soap_api.mirror.MaStRMirror
    was introduced for mirroring MaStR data with latest updates #149

  • Introduce project home ~/.open-MaStR/config/ #120

  • Documentation of post-processing #117

  • Updated documentation of downloading data #124 which is harmonized with the other parts of docs
    and with GitHubs README #135

  • Local execution of post-processing now possible, optionally in dockered database #116

  • Post-processing adapted to CSV data from :class:open_mastr.soap_api.mirror.MaStRMirror #172

  • Tests for changed download code are added #131

  • Metadata added for raw data as frictionless data package #160

  • Suffix columns instead of deferring in database CSV export #157

  • Code examples added for :class:open_mastr.soap_api.mirror.MaStRMirror explaining basic use of
    mirroring database #164

  • CSV file reader for MaStR raw data added #181

  • Zenodo data upload #173
    and the missing LICENSE file for Zenodo is fixed in #186

  • Add postgres database service in CI job for interacting with database in tests #159

  • Tests for :class:open_mastr.soap_api.mirror.MaStRMirror #191

  • Download functionality for Lokationen (with focus on :class:open_mastr.soap_api.mirror.MaStRMirror) #162

  • The CHANGELOG is now included in the documentation

Changed

  • Download of raw data has entirely been refactored. A
    python wrapper for querying
    the MaStR API was introduced #83
  • Based on that, for bulk data download,
    MaStRDownload provides handy query
    functions for power unit data #86. See also #128
  • configuration through config filen in ~/.open-MaStR/config/ with less hard-coded parameters in source files #120, #112
  • move code into one package named open_mastr #123
  • Switch to GitHub Actions for CI instead of Travis #143
  • Fixed unexpected line breaks during CSV export that corrupted data #170
  • Filtering of duplicates in MaStR data (see
    MaStR help <https://www.marktstammdatenregister.de/MaStRHilfe/subpages/statistik.html>_) got changed to filter
    units by leading three characters and select only directly entered data #180
  • Generalize CSV reading function #188

Removed

  • Most of prior code for downloading data