Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New model builder #301

Merged
merged 78 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
6ab2d5a
address ambiguous template naming in inline_dependencies
gtfierro Oct 29, 2023
0499b15
fill non-graphs, propagate the optional arg requirement
gtfierro Oct 29, 2023
60bdcf4
update tests
gtfierro Oct 29, 2023
e3737a2
Merge remote-tracking branch 'origin/develop' into gtf-template-inlin…
gtfierro Oct 30, 2023
607ce65
Merge remote-tracking branch 'origin/develop' into gtf-template-inlin…
gtfierro Nov 28, 2023
28a950d
adjusting tests for inlining
gtfierro Nov 30, 2023
e69a94a
update 223p templates
gtfierro Nov 30, 2023
742e7ae
update 223
gtfierro Nov 30, 2023
c5e3a9a
add model builder impl
gtfierro Dec 6, 2023
c9030d7
Merge branch 'develop' into gtf-update-223p-templates
gtfierro Dec 6, 2023
3ce1cd1
add QUDT
gtfierro Dec 6, 2023
a5ec71f
Merge branch 'gtf-update-223p-templates' of github.com:NREL/BuildingM…
gtfierro Dec 6, 2023
85cf110
update templates
gtfierro Dec 6, 2023
41c650c
Merge remote-tracking branch 'origin/gtf-update-223p-templates' into …
gtfierro Dec 6, 2023
33eb00b
Merge remote-tracking branch 'origin/gtf-template-inline-fix' into gt…
gtfierro Dec 6, 2023
e371b54
allow adding triples directly to the builder graph
gtfierro Mar 5, 2024
04ca5b1
Merge remote-tracking branch 'origin/develop' into gtf-add-new-model-…
gtfierro Mar 5, 2024
606a8a4
allow Library to find template definitions from dependent libraries
gtfierro Mar 5, 2024
58cebb1
fixing tests to catch warnings, test dependency tracking
gtfierro Mar 6, 2024
550e263
use non-deprecated warnings; make sure warnings are emitted when nece…
gtfierro Mar 6, 2024
b483f71
clarify when dependencies are infered for SHACL shapes -> templates
gtfierro Mar 6, 2024
1bbdc3a
update dependencies
gtfierro Mar 6, 2024
82e575c
adding documentation on templates
gtfierro Mar 6, 2024
f036bc8
placeholder for further template docs
gtfierro Mar 6, 2024
569c468
clarify load order
gtfierro Mar 6, 2024
40a1b78
update jsonschema, skip mypy checking on imports
gtfierro Mar 6, 2024
87ade9e
fix API response and test to handle Brick
gtfierro Mar 6, 2024
1255b23
updating 223p and templates
gtfierro Mar 12, 2024
5231e71
filtering validationcontext by severity
gtfierro Mar 12, 2024
cdeba72
add shacl_validate/infer functions and use these as the entrypoint. A…
gtfierro Mar 12, 2024
95cee3f
fix interactions with shacl inference
gtfierro Mar 12, 2024
91e90d2
tightening up the implementation and use of the shacl_* methods
gtfierro Mar 14, 2024
936e830
support specifying shacl engine in the API
gtfierro Mar 14, 2024
7a1a786
update tests; test both pyshacl and topquadrant
gtfierro Mar 14, 2024
f5e9465
add brick-tq-shacl dep
gtfierro Mar 14, 2024
1c29f61
add TODOs
gtfierro Mar 14, 2024
37c29d2
Formatting
gtfierro Mar 14, 2024
1174e7b
no more 3.8!
gtfierro Mar 14, 2024
eef32da
ignoring some imported packages without type annotations
gtfierro Mar 14, 2024
1bd0a94
more type annotations
gtfierro Mar 14, 2024
9194ac0
add types, ignore type errors for imports
gtfierro Mar 14, 2024
5246f9e
update mypy, fix some issues and ignore some others
gtfierro Mar 14, 2024
ab4ee7d
fix union type annotation
gtfierro Mar 14, 2024
ea59b4a
update docker containers
gtfierro Mar 14, 2024
9804f10
3.8.1 python for higher
gtfierro Mar 15, 2024
0427587
add back python 3.8
gtfierro Mar 15, 2024
b68be1f
Merge remote-tracking branch 'origin/develop' into topquadrant-shacl
gtfierro Mar 17, 2024
ec350db
change 3.8 version
gtfierro Mar 17, 2024
f7116c6
add test for finding reasons with a given severity
gtfierro Mar 17, 2024
695290b
update brick-tq-shacl, fix type signature
gtfierro Mar 18, 2024
e4098ac
remove debug serializations
gtfierro Mar 18, 2024
2b8893f
bump shacl version
gtfierro Mar 18, 2024
b06f8c1
fixing skolemization for validation
gtfierro Mar 19, 2024
409f7ad
Merge remote-tracking branch 'origin/develop' into update-223p
gtfierro Mar 20, 2024
30c591b
Merge remote-tracking branch 'origin/topquadrant-shacl' into update-223p
gtfierro Mar 20, 2024
dce13f4
Merge remote-tracking branch 'origin/gtf-update-223p-templates' into …
gtfierro Mar 20, 2024
1e4c316
update 223p, fix merge error
gtfierro Mar 20, 2024
aa819d9
fix notebook
gtfierro Mar 20, 2024
89c6bcc
Merge remote-tracking branch 'origin/develop' into gtf-add-new-model-…
gtfierro Mar 20, 2024
fe1c2c0
Merge remote-tracking branch 'origin/update-223p' into gtf-add-new-mo…
gtfierro Mar 20, 2024
5fe5285
Merge remote-tracking branch 'origin/develop' into gtf-add-new-model-…
gtfierro Apr 12, 2024
a128823
fix the merge
gtfierro Apr 12, 2024
1b69e78
remove qudt from commit
gtfierro Apr 12, 2024
2d08b4d
remove more bad merge
gtfierro Apr 12, 2024
bf6118b
fix more bad merge
gtfierro Apr 12, 2024
b4d9786
remove more bad merge
gtfierro Apr 15, 2024
c333e1d
add s223 namespace, parameter prop on template builder
gtfierro Apr 15, 2024
44e6993
add model builder draft
gtfierro Apr 15, 2024
3eea3bf
add call syntax to simplify evaluating templates
gtfierro Apr 21, 2024
3e15200
do not waste cycles re-binding namespaces on copied graphs
gtfierro Apr 21, 2024
4a331af
update model builder notebook
gtfierro Apr 26, 2024
cd9ac2c
add java for topquadrant support
gtfierro Apr 26, 2024
e31d9f8
use topquadrant in 223 notebook
gtfierro Apr 26, 2024
bce3548
updating dockerfile; this started failing, possibly because of update…
gtfierro Apr 29, 2024
4b600fc
Merge branch 'develop' into gtf-add-new-model-builder
gtfierro May 13, 2024
7276c3c
Merge remote-tracking branch 'origin/develop' into gtf-add-new-model-…
gtfierro May 14, 2024
ddf2ad0
properly initialize graph superclass
gtfierro May 14, 2024
1c7d35d
Merge branch 'develop' into gtf-add-new-model-builder
gtfierro May 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ jobs:
steps:
- name: checkout
uses: actions/checkout@v4
- uses: actions/setup-java@v4 # for topquadrant shacl support
with:
distribution: 'temurin'
java-version: '21'
- name: setup-python
uses: actions/setup-python@v4
with:
Expand Down
2 changes: 1 addition & 1 deletion buildingmotif/dataclasses/validation.py
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ def diffset_to_templates(
continue

templ_lists = (diff.resolve(lib) for diff in diffset)
templs = list(filter(None, chain.from_iterable(templ_lists)))
templs: List[Template] = list(filter(None, chain.from_iterable(templ_lists)))
if len(templs) <= 1:
templates.extend(templs)
continue
Expand Down
149 changes: 149 additions & 0 deletions buildingmotif/model_builder.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
import secrets
from typing import Dict, List, Tuple

from rdflib import BNode, Graph, Literal, Namespace, URIRef
from rdflib.term import Node

from buildingmotif.dataclasses import Library, Template
from buildingmotif.namespaces import RDF, RDFS


class TemplateBuilderContext:
"""
gtfierro marked this conversation as resolved.
Show resolved Hide resolved
A context for building templates. This class allows the user to
add templates to the context and then access them by name. The
context also allows the user to compile all of the templates in
the context into a single graph.
"""

def __init__(self, ns: Namespace):
"""
Creates a new TemplateBuilderContext. The context will create
entities in the given namespace.

:param ns: The namespace to use for the context
"""
self.templates: Dict[str, Template] = {}
self.wrappers: List[TemplateWrapper] = []
self.ns: Namespace = ns
# stores triples outside of the templates
self._g: Graph = Graph()

def add(self, triple: Tuple):
"""
Adds a triple to the context

:param s: The subject of the triple
:param p: The predicate of the triple
:param o: The object of the triple
"""
self._g.add(triple)

def add_template(self, template: Template):
"""
Adds a template to the context with all of its dependencies
inlined. Allows the user of the context to access the template
by name.

:param template: The template to add to the context
"""
self.templates[template.name] = template.inline_dependencies()

def add_templates_from_library(self, library: Library):
"""
Adds all of the templates from a library to the context

:param library: The library to add to the context
"""
for template in library.get_templates():
self.add_template(template)

def __getitem__(self, template_name):
if template_name in self.templates:
w = TemplateWrapper(self.templates[template_name], self.ns)
self.wrappers.append(w)
return w
else:
raise KeyError(f"Invalid template name: {template_name}")

def compile(self) -> Graph:
"""
Compiles all of the template wrappers and concatenates them into a single Graph

:return: A graph containing all of the compiled templates
"""
graph = Graph()
graph += self._g
for wrapper in self.wrappers:
graph += wrapper.compile()
# add a label to every instance if it doesn't have one. Make
# the label the same as the value part of the URI
for s, p, o in graph.triples((None, RDF.type, None)):
if (s, RDFS.label, None) not in graph:
# get the 'value' part of the o URI using qname
_, _, value = graph.namespace_manager.compute_qname(str(o))
graph.add((s, RDFS.label, Literal(value)))
return graph


class TemplateWrapper:
def __init__(self, template: Template, ns: Namespace):
"""
Creates a new TemplateWrapper. The wrapper is used to bind
parameters to a template and then compile the template into
a graph.

:param template: The template to wrap
:param ns: The namespace to use for the wrapper; all bindings will be added to this namespace
"""
self.template = template
self.bindings: Dict[str, Node] = {}
self.ns = ns

def __call__(self, **kwargs):
for k, v in kwargs.items():
self[k] = v
return self

def __getitem__(self, param):
if param in self.bindings:
return self.bindings[param]
elif param not in self.template.all_parameters:
raise KeyError(f"Invalid parameter: {param}")
# if the param is not bound, then invent a name
# by prepending the parameter name to a random string
self.bindings[param] = self.ns[param + "_" + secrets.token_hex(4)]
return self.bindings[param]

def __setitem__(self, param, value):
if param not in self.template.all_parameters:
raise KeyError(f"Invalid parameter: {param}")
# if value is not a URIRef, Literal or BNode, then put it in the namespace
if not isinstance(value, (URIRef, Literal, BNode)):
value = self.ns[value]
# check datatype of value is URIRef, Literal or BNode
if not isinstance(value, (URIRef, Literal, BNode)):
raise TypeError(f"Invalid type for value: {type(value)}")
self.bindings[param] = value

@property
def parameters(self):
return self.template.parameters

def compile(self) -> Graph:
"""
Compiles the template into a graph. If there are still parameters
to be bound, then the template will be returned. Otherwise, the
template will be filled and the resulting graph will be returned.

:return: A graph containing the compiled template
:rtype: Graph
"""
tmp = self.template.evaluate(self.bindings)
# if this is true, there are still parameters to be bound
if isinstance(tmp, Template):
bindings, graph = tmp.fill(self.ns, include_optional=False)
self.bindings.update(bindings)
return graph
else:
return tmp
2 changes: 2 additions & 0 deletions buildingmotif/namespaces.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@
QUDTDV = Namespace("http://qudt.org/vocab/dimensionvector/")
UNIT = Namespace("http://qudt.org/vocab/unit/")

# ASHRAE namespaces
BACNET = Namespace("http://data.ashrae.org/bacnet/2020#")
S223 = Namespace("http://data.ashrae.org/standard223#")

BM = Namespace("https://nrel.gov/BuildingMOTIF#")
CONSTRAINT = Namespace("https://nrel.gov/BuildingMOTIF/constraints#")
Expand Down
2 changes: 0 additions & 2 deletions buildingmotif/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,6 @@ def copy_graph(g: Graph, preserve_blank_nodes: bool = True) -> Graph:
:rtype: Graph
"""
c = Graph()
for pfx, ns in g.namespaces():
c.bind(pfx, ns)
new_prefix = secrets.token_hex(4)
for t in g.triples((None, None, None)):
assert isinstance(t, tuple)
Expand Down
69 changes: 50 additions & 19 deletions libraries/ashrae/223p/nrel-templates/connections.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,32 +48,63 @@ duct:
@prefix s223: <http://data.ashrae.org/standard223#> .
P:name a s223:Duct ;
s223:hasMedium s223:Medium-Air ;
s223:connectsAt P:a, P:b .
s223:cnx P:a, P:b .
# issue here is that 'connectsAt' requires a,b to be conn points
# but we can't instantiate that class directly *and* being a conn point
# involves other properties that must be included (e.g. hasmedium).
# TODO: how to solve this?
# P:a a s223:ConnectionPoint .
# P:b a s223:ConnectionPoint .

zone-air-inlet-cp:
junction:
body: >
@prefix P: <urn:___param___#> .
@prefix s223: <http://data.ashrae.org/standard223#> .
P:name a s223:InletZoneConnectionPoint ;
s223:mapsTo P:mapsto ;
s223:hasMedium s223:Medium-Air .
P:mapsto a s223:InletConnectionPoint ;
s223:hasMedium s223:Medium-Air .
optional: ["mapsto"]

zone-air-outlet-cp:
body: >
@prefix P: <urn:___param___#> .
@prefix s223: <http://data.ashrae.org/standard223#> .
P:name a s223:OutletZoneConnectionPoint ;
s223:mapsTo P:mapsto ;
s223:hasMedium s223:Medium-Air .
P:mapsto a s223:OutletConnectionPoint ;
s223:hasMedium s223:Medium-Air .
optional: ["mapsto"]
P:name a s223:Junction ;
s223:hasMedium s223:Medium-Air ;
s223:cnx P:in1, P:in2, P:out1, P:out2, P:out3, P:out4, P:out5, P:out6, P:out7,
P:out8, P:out9, P:out10, P:out11, P:out12, P:out13, P:out14, P:out15, P:out16 .
optional: ["in2","in3","in4","in5","out2", "out3", "out4", "out5", "out6", "out7", "out8", "out9","out10","out11","out12","out13","out14","out15", "out16"]
dependencies:
- template: air-inlet-cp
args: {"name": "in1"}
- template: air-inlet-cp
args: {"name": "in2"}
- template: air-inlet-cp
args: {"name": "in3"}
- template: air-inlet-cp
args: {"name": "in4"}
- template: air-inlet-cp
args: {"name": "in5"}
- template: air-outlet-cp
args: {"name": "out1"}
- template: air-outlet-cp
args: {"name": "out2"}
- template: air-outlet-cp
args: {"name": "out3"}
- template: air-outlet-cp
args: {"name": "out4"}
- template: air-outlet-cp
args: {"name": "out5"}
- template: air-outlet-cp
args: {"name": "out6"}
- template: air-outlet-cp
args: {"name": "out7"}
- template: air-outlet-cp
args: {"name": "out8"}
- template: air-outlet-cp
args: {"name": "out9"}
- template: air-outlet-cp
args: {"name": "out10"}
- template: air-outlet-cp
args: {"name": "out11"}
- template: air-outlet-cp
args: {"name": "out12"}
- template: air-outlet-cp
args: {"name": "out13"}
- template: air-outlet-cp
args: {"name": "out14"}
- template: air-outlet-cp
args: {"name": "out15"}
- template: air-outlet-cp
args: {"name": "out16"}
Loading
Loading