Merge pull request #17 from mjanez/main

Update latest

.env.example

@@ -13,4 +13,9 @@ DCATAP_FILES_VERSION=2.1.1
 # Update vocabs from the EU Publications Office
 UPDATE_VOCABS=True
 ## CKAN Metadata elements type: "ckan_uris" for GeoDCAT-AP schema with all elements described by URIs (e.g. dct:format = <http://publications.europa.eu/resource/authority/file-type/XML>) or "ckan" if used a default schema with elements (e.g. dct:format = "XML").
-CKAN_METADATA_TYPE=ckan_uris
+CKAN_METADATA_TYPE=ckan_uris
+
+#DEV
+MQA_DEV_PORT=5678
+# If you want to use the VSCode container debugger, set this variable to True
+MQA_DEV_VSCODE=False

README.md

@@ -7,7 +7,8 @@
     <a href="#overview">Overview</a> •
     <a href="#quick-start">Quick start</a> •
     <a href="#debug">Debug</a> •
-    <a href="#containers">Containers</a>
+    <a href="#containers">Containers</a> •
+    <a href="#references">DCAT-AP info</a>
 </p>
 
 **Requirements**:
@@ -18,7 +19,7 @@
 
 This Docker Compose configuration enhances a Python MQA software [^1] to integrates the powerful MQA toolset seamlessly with CKAN endpoints and European Data Portal catalogs, enabling users to perform in-depth assessments of metadata quality effortlessly. The setup provides an efficient way to run comprehensive quality checks on various metadata attributes, including data relevance, schema compliance, data format consistency, and adherence to standard vocabularies.
 
->**Note**<br>
+> [!TIP]
 > It can be tested with an open data portal of the CKAN type such as: [mjanez/ckan-docker](https://github.com/mjanez/ckan-docker)[^2]
 
 ### [Metadata Quality Assessment Methodology](https://data.europa.eu/mqa/methodology)
@@ -119,15 +120,13 @@ docker compose up --build
 docker compose up -d --build
 ```
 
->**Note**:<br>
+> [!NOTE]
 > Deploy the dev (local build) `docker-compose.dev.yml` with:
 >
 >```bash
 > docker compose -f docker-compose.dev.yml up --build
 >```
-
-
->**Note**:<br>
+>
 >If needed, to build a specific container simply run:
 >
 >```bash
@@ -168,10 +167,91 @@ List of *containers*:
 | mjanez/ckan-mqa| custom image | `mjanez/ckan-mqa:main` | 264 MB |  Dev version.  |
 
 
+## References
+### DCAT-AP Validator Validation Cases
+The different cases to validate in the [DCAT-AP Validator](https://www.itb.ec.europa.eu/shacl/dcat-ap/upload) are based on the level of completeness of the checks and the incorporation of background knowledge (vocabularies). Each case is designed for a specific data exchange scenario.
+The following describes each case and recommends which one you should use for a CKAN catalog:
+
+#### Case 1: DCAT-AP Base Zero (no background knowledge)
+Includes all constraints required for technical coherence, excluding range class membership constraints and controlled vocabulary usage.
+
+*SHACL Profiles*: 
+* [2.1.1](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_shapes.ttl)
+* [3.0.0](https://github.com/SEMICeu/DCAT-AP/blob/master/releases/3.0.0/html/shacl/shapes.ttl)
+
+
+#### Case 2: DCAT-AP Ranges Zero (no background knowledge)
+Includes all range class membership constraints.
+
+*SHACL Profiles*: 
+* [2.1.1](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_range.ttl)
+* [3.0.0](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/range.ttl)
+
+#### Case 3: DCAT-AP Base (with background knowledge)
+Extends Case 1 with background knowledge, including all vocabularies used in DCAT-AP.
+
+*SHACL Profiles*: 
+* 2.1.1: [`shapes`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_shapes.ttl) and [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_imports.ttl)
+* 3.0.0: [`shapes`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/shapes.ttl) and [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/imports.ttl)
+
+#### Case 4: DCAT-AP Ranges (with background knowledge)
+Extends Case 2 with background knowledge, adding validation of range class membership and vocabulary standards compliance.
+
+*SHACL Profiles*:
+* 2.1.1: [`range`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_range.ttl) and [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_imports.ttl)
+* 3.0.0: [`range`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/range.ttl) and [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/imports.ttl)
+
+#### Case 5: DCAT-AP Recommendations (with background knowledge)
+Includes all constraints related to recommended properties.
+
+*SHACL Profiles*: 
+* 2.1.1: [`shapes recommended`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_shapes_recommended.ttl) and [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_imports.ttl)
+* 3.0.0: [`shapes recommended`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/shapes_recommended.ttl) and [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/imports.ttl)
+
+
+#### Case 6: DCAT-AP Controlled Vocabularies
+Includes all constraints related to controlled vocabularies.
+
+*SHACL Profiles*: 
+* 2.1.1: [`vocabularies shape`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_mdr-vocabularies.shape.ttl) and [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_mdr_imports.ttl)
+* 3.0.0: [`vocabularies shape`](https://github.com/SEMICeu/DCAT-AP/raw/3.0.0/releases/3.0.0/html/shacl/mdr-vocabularies.shape.ttl) and [`mdr imports`](https://github.com/SEMICeu/DCAT-AP/raw/3.0.0/releases/3.0.0/html/shacl/mdr_imports.ttl)
+
+
+#### Case 7: DCAT-AP Full (with background knowledge)
+The union of Cases 3, 4, 5, and 6.
+
+*SHACL Profiles*: 
+* 2.1.1: [`shapes`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_shapes.ttl), [`shapes recommended`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_shapes_recommended.ttl), [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_imports.ttl), [`range`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_range.ttl) and [`deprecateduris`](https://github.com/SEMICeu/DCAT-AP/raw/2.1.1-draft/releases/2.1.1/dcat-ap_2.1.1_shacl_deprecateduris.ttl)
+* 3.0.0: [`shapes`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/shapes.ttl), [`shapes recommended`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/shapes_recommended.ttl), [`imports`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/imports.ttl), [`range`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/range.ttl) and [`deprecateduris`](https://github.com/SEMICeu/DCAT-AP/raw/gh-pages/releases/3.0.0-draft/html/shacl/deprecateduris.ttl)
+
+
+#### Recommendation:
+For most use cases, `Case 3: DCAT-AP Base (with background knowledge)` is recommended. It provides comprehensive validation of basic coherence and vocabulary standards compliance.
+If your CKAN catalog uses controlled vocabularies, consider using `Case 6: DCAT-AP Controlled Vocabularies` or `Case 7: DCAT-AP Full (with background knowledge)` for more exhaustive validation.
+Remember, the choice of the appropriate validation case depends on your specific needs and data exchange context.
+
+> [!TIP]
+> 
+> DCAT-AP:
+>  - https://github.com/SEMICeu/DCAT-AP/tree/master/releases
+>  - https://semiceu.github.io/DCAT-AP/releases/3.0.0/#validation-of-dcat-ap
+>
+>EU Vocabularies: https://op.europa.eu/en/web/eu-vocabularies/dcat-ap
+>
+>Validator: 
+>- https://www.itb.ec.europa.eu/shacl/dcat-ap/upload
+>-  https://github.com/ISAITB/validator-resources-dcat-ap/tree/master#
+>
+>DCAT-AP Country profiles:
+>  - https://github.com/diggsweden/DCAT-AP-SE
+>  - https://github.com/opendata-swiss/dcat_ap_ch
+>
+>SHACLs: https://github.com/ISAITB/validator-resources-dcat-ap/blob/baca3adf63d31ee415fa5e769249053ae211414c/resources/config.properties
+
 ## License
 Copyright (c) the respective contributors.
 It is open and licensed under the GNU Affero General Public License (AGPL) v3.0 whose full text may be found at:
 http://www.fsf.org/licensing/licenses/agpl-3.0.html
 
 [^1]: Program to test MQA evaluation: Javier Nogueras ([email protected]), Javier Lacasta ([email protected]), Manuel Ureña ([email protected]), F. Javier Ariza ([email protected]), Héctor Ochoa Ortiz ([email protected]). Trafair Project 2020.
-[^2]: A custom installation of Docker Compose with specific extensions for spatial data and [GeoDCAT-AP](https://github.com/SEMICeu/GeoDCAT-AP)/[INSPIRE](https://github.com/INSPIRE-MIF/technical-guidelines) metadata [profiles](https://en.wikipedia.org/wiki/Geospatial_metadata).
+[^2]: A custom installation of Docker Compose with specific extensions for spatial data and [GeoDCAT-AP](https://github.com/SEMICeu/GeoDCAT-AP)/[INSPIRE](https://github.com/INSPIRE-MIF/technical-guidelines) metadata [profiles](https://en.wikipedia.org/wiki/Geospatial_metadata).

ckan2mqa/assets/2.1.1/dcat-ap_2.1.1_shacl_imports.ttl

@@ -0,0 +1,28 @@
+@prefix owl: <http://www.w3.org/2002/07/owl#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+
+# 
+# This file provides the imports that are implicitly the result of reusing them in the DCAT-AP application profile.
+# The imports point to the URL of the RDF serializations (mostly the turtle serializations) as not all ontology URIs have content negotation implemented. 
+# The RDF format is required for the ISA testbed validator.
+# The following imports have been outcommented: 
+#   owl:imports <https://www.w3.org/ns/odrl/2/ODRL22.ttl> ; import is excluded because the shacl shape for Category applies to all instances of skos:Concept and the skos:Concepts in the ODRL do not comply to this. 
+
+
+<http://data.europa.eu/r5r>
+  rdf:type owl:Ontology ;
+  owl:imports <https://www.w3.org/ns/dcat2.ttl> ;
+  owl:imports <http://dublincore.org/2020/01/20/dublin_core_terms.ttl> ;
+  owl:imports <http://xmlns.com/foaf/spec/index.rdf> ;
+  owl:imports <https://www.w3.org/ns/locn.ttl> ;
+  owl:imports <https://spdx.org/rdf/terms/spdx-ontology.owl.xml> ;
+  owl:imports <https://schema.org/version/latest/schema.ttl> ;
+  owl:imports <http://www.w3.org/ns/prov-o.ttl> ;
+  owl:imports <http://www.w3.org/2006/time.ttl> ;
+  owl:imports <http://www.w3.org/2006/vcard/ns.ttl> ;
+  owl:imports <http://www.w3.org/ns/adms.ttl> 
+  .
+
+

ckan2mqa/assets/2.1.1/dcat-ap_2.1.1_shacl_mdr_imports.ttl

@@ -0,0 +1,29 @@
+@prefix owl: <http://www.w3.org/2002/07/owl#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+
+# 
+# This file provides the imports of the codelists recommended by the DCAT-AP application profile.
+#  http://publications.europa.eu/resource/authority/dataset-type (TODO)
+
+
+<http://data.europa.eu/r5r/codelist_imports>
+  rdf:type owl:Ontology ;
+  owl:imports <http://publications.europa.eu/resource/authority/language> ;
+  owl:imports <http://publications.europa.eu/resource/authority/corporate-body> ;
+  owl:imports <https://github.com/SEMICeu/DCAT-AP/raw/master/releases/2.0.1/dcat-ap_2.0.1.rdf> ;
+  owl:imports <http://publications.europa.eu/resource/authority/continent> ;
+  owl:imports <http://publications.europa.eu/resource/authority/country> ; 
+  owl:imports <http://publications.europa.eu/resource/authority/dataset-type> ;
+  owl:imports <http://publications.europa.eu/resource/authority/data-theme> ;
+  owl:imports <http://publications.europa.eu/resource/authority/file-type> ;
+  owl:imports <http://publications.europa.eu/resource/authority/frequency> ;
+  owl:imports <https://github.com/SEMICeu/ADMS-AP/raw/master/purl.org/ADMS_SKOS_v1.00.rdf> ; 
+  owl:imports <http://publications.europa.eu/resource/authority/place> ;
+  owl:imports <http://publications.europa.eu/resource/authority/planned-availability> ;
+  owl:imports <http://publications.europa.eu/resource/authority/access-right>
+.
+
+# import of the checksum vocabulary
+# owl:imports <https://spdx.org/rdf/terms/spdx-ontology.owl.xml> ;