Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EAGLE-1231: Added 'encoding' attribute to fields in Logical Graph schema #265

Closed
wants to merge 4,819 commits into from
Closed

EAGLE-1231: Added 'encoding' attribute to fields in Logical Graph schema #265

Changes from all commits
Commits
Show all changes
4819 commits
Select commit Hold shift + click to select a range
140a006
Very small change, but seems to work now again
awicenec May 2, 2023
be97072
Merge pull request #228 from ICRAR/liu-353
awicenec May 2, 2023
4deb7bb
Added parser fields to doxygen of apps
awicenec May 2, 2023
e48989c
Merge pull request #230 from ICRAR/liu-354
awicenec May 2, 2023
1b5c59f
Refactored _getIdText to _getPortName
awicenec May 3, 2023
817bf8e
Merge branch 'liu-348' into liu-355
awicenec May 3, 2023
ae60b2b
changed to generic "dropclass" keyword
awicenec May 7, 2023
6ea8df4
Missed two "dataclass" keywords
awicenec May 7, 2023
ccaa92a
added dataclass to all graphs, cleanup of lg_node
awicenec May 7, 2023
777853a
More streamlining
awicenec May 7, 2023
b3716b5
Re-distributed methods
awicenec May 7, 2023
bd322a3
Added backwards compatibility
awicenec May 7, 2023
ef6e53a
Fixed small issues with existing graphs
awicenec May 7, 2023
814e716
Trying to get name onto graph
awicenec May 7, 2023
37955ac
Fixed dropclasses in simple.py doxygen
awicenec May 8, 2023
26a281b
added dropclass to S3DROP
awicenec May 8, 2023
27ea2e0
fixed dropclass for all data drops
awicenec May 8, 2023
d3758f2
Fixed loading of parameters into PG
awicenec May 8, 2023
174ba94
Added dropclass to more internal classes
awicenec May 8, 2023
bd9813e
Final missing dropclasses
awicenec May 8, 2023
9186e10
Small doc update
awicenec May 8, 2023
397111d
Merge pull request #231 from ICRAR/liu-355
awicenec May 8, 2023
a522e3c
Merge pull request #232 from ICRAR/liu-294
awicenec May 9, 2023
67855cf
Minor spelling mistake in docs for Loop component
james-strauss-uwa May 11, 2023
dc221cd
Added shebang to run and stop scripts
awicenec May 16, 2023
71c4eca
Updated start and stop documentation
awicenec May 17, 2023
8168016
Updated start and stop documentation
awicenec May 18, 2023
e71a6f0
Replace merge branch 'master' into liu-348
awicenec May 18, 2023
ff1f46d
Merge branch 'liu-348'
awicenec May 18, 2023
abaa374
daliuge 3.0.0
awicenec May 18, 2023
de64c6b
changed init to bash
awicenec May 21, 2023
c13299f
Revert "changed init to bash"
awicenec May 21, 2023
33e1d30
Revert to de64c6b6
awicenec May 21, 2023
770862b
Remove unnecessary file
rtobar Mar 21, 2023
41c5b4c
Avoid deprecated python features
rtobar May 23, 2023
a21171b
First fix to enable browser submit into engine
awicenec May 23, 2023
68d3c25
Adjusted test include SERVER method
awicenec May 24, 2023
d1a7940
Fixed dropclass replacement for test runs
awicenec May 24, 2023
986e024
Make RE objects static members of AbstractDROP
rtobar May 24, 2023
c35df1c
Introduce new ProxyInfo object
rtobar Mar 21, 2023
d62cccd
Store session ID instead of full session in drops
rtobar May 24, 2023
f911bdb
Store RPC server endpoint instead of full server in drops
rtobar May 24, 2023
ddac025
A few more fixes for docker deployments
awicenec May 29, 2023
51d2e5f
Fixed test for new dropclass
awicenec May 31, 2023
fcce6f3
Merge pull request #233 from ICRAR/liu-365
awicenec May 31, 2023
0b363d8
Combine dummy input and output ports into one InputOutput port. Add a…
james-strauss-uwa Jun 1, 2023
142f6a5
Attempt to use dlg_paletteGen instead of xml2palette.py
james-strauss-uwa Jun 1, 2023
dece7d8
Minor cleanup
james-strauss-uwa Jun 2, 2023
e3dc2a3
Update dixygen to newest format
james-strauss-uwa Jun 14, 2023
11ea158
Add dummy input port to Branch component
james-strauss-uwa Jun 14, 2023
481f985
Re-named the tag and digest params on the Docker drop
james-strauss-uwa Jun 14, 2023
201ca2a
updated default for numpy load to allow pickle
awicenec Jun 30, 2023
69a1289
Fixed bug reading input port in sleepApp
awicenec Jul 3, 2023
7b02b40
Added named ports to APP components
awicenec Jul 4, 2023
a85de26
Fixed failing test
awicenec Jul 4, 2023
f00747d
Added missing import
awicenec Jul 4, 2023
635c154
Don't read from ports, but leave that to the app
awicenec Jul 5, 2023
5bb47ff
Changed doxygen for PyFunc
awicenec Jul 5, 2023
7a54b49
Merge pull request #234 from ICRAR/liu-343
awicenec Jul 5, 2023
76bd920
A few more fixes for docker deployments
awicenec May 29, 2023
4fee9e6
Fixed test for new dropclass
awicenec May 31, 2023
1e3c241
Combine dummy input and output ports into one InputOutput port. Add a…
james-strauss-uwa Jun 1, 2023
f14808a
Attempt to use dlg_paletteGen instead of xml2palette.py
james-strauss-uwa Jun 1, 2023
5ffea37
Minor cleanup
james-strauss-uwa Jun 2, 2023
ff5e86d
Update dixygen to newest format
james-strauss-uwa Jun 14, 2023
96730b5
Add dummy input port to Branch component
james-strauss-uwa Jun 14, 2023
8ac9b41
Re-named the tag and digest params on the Docker drop
james-strauss-uwa Jun 14, 2023
909fc05
Combine dummy input and output ports into one InputOutput port. Add a…
james-strauss-uwa Jun 1, 2023
e4a200b
Attempt to use dlg_paletteGen instead of xml2palette.py
james-strauss-uwa Jun 1, 2023
4fb6805
Update dixygen to newest format
james-strauss-uwa Jun 14, 2023
8a0079e
Add dummy input port to Branch component
james-strauss-uwa Jun 14, 2023
4424765
Re-named the tag and digest params on the Docker drop
james-strauss-uwa Jun 14, 2023
217bba8
Combine dummy input and output ports into one InputOutput port. Add a…
james-strauss-uwa Jun 1, 2023
ee55380
Attempt to use dlg_paletteGen instead of xml2palette.py
james-strauss-uwa Jun 1, 2023
5d2d4b3
Update dixygen to newest format
james-strauss-uwa Jun 14, 2023
9221a88
Add dummy input port to Branch component
james-strauss-uwa Jun 14, 2023
5eace11
Re-named the tag and digest params on the Docker drop
james-strauss-uwa Jun 14, 2023
9d99079
Merge branch 'master' into liu-367
awicenec Jul 5, 2023
4f6a85b
Adjusted doxygen strings
awicenec Jul 6, 2023
ce9b867
csv_load does not like quoted text inside field
awicenec Jul 6, 2023
1f290d8
Cleanup
awicenec Jul 6, 2023
8a51586
removed parsers from AppDROPs (not implemented yet)
awicenec Jul 7, 2023
c22a956
Removed the parsers from all data drops
awicenec Jul 7, 2023
40a6522
Revert pydata changes, dask delayed tests stuck
awicenec Jul 10, 2023
7a950dd
Fixed pydantic version
awicenec Jul 10, 2023
93cd723
typo
awicenec Jul 12, 2023
f3e4bdc
output_parser 'path' is now working
awicenec Jul 18, 2023
25ce050
Fixing pyfunc arguments
awicenec Jul 25, 2023
16e8b02
Doc and simple changes
awicenec Aug 8, 2023
507036f
Fixed version of pydantic and zeromq
awicenec Aug 8, 2023
0d86856
Aligned pyzmq versions
awicenec Aug 8, 2023
76388b9
Show IP address when staring containers
awicenec Aug 9, 2023
a55e5f8
Show IP address when starting containers
awicenec Aug 9, 2023
7f79a64
Enable to specify pydata directly through the graph
awicenec Aug 16, 2023
6c98b66
Fixed version of pydantic and zeromq
awicenec Aug 8, 2023
64871d7
Aligned pyzmq versions
awicenec Aug 8, 2023
456c1d2
Merge branch 'pydata' into liu-367
awicenec Aug 16, 2023
9134f49
Fixed SharedMemory drop
awicenec Aug 16, 2023
9f9e79e
Fix in SharedMemoryDROP
awicenec Aug 17, 2023
a143999
Changed data_volume field to a ConstraintParameter
james-strauss-uwa Aug 17, 2023
5f81369
Changed num_cpus field to a ConstraintParameter
james-strauss-uwa Aug 18, 2023
989e848
Changed execution_time field to a ConstraintParameter
james-strauss-uwa Aug 18, 2023
0ff471f
renamed nodeAttributes in PG to fields added constraintParams
awicenec Aug 25, 2023
44edda3
Merge pull request #237 from ICRAR/eagle-402
awicenec Aug 25, 2023
f8da606
Fixed version of pydantic and zeromq
awicenec Aug 8, 2023
0e5c461
Aligned pyzmq versions
awicenec Aug 8, 2023
37f94bb
Changed data_volume field to a ConstraintParameter
james-strauss-uwa Aug 17, 2023
dafae70
Changed num_cpus field to a ConstraintParameter
james-strauss-uwa Aug 18, 2023
ee38ba4
Changed execution_time field to a ConstraintParameter
james-strauss-uwa Aug 18, 2023
c205a0a
renamed nodeAttributes in PG to fields added constraintParams
awicenec Aug 25, 2023
ebe5789
Fix problem with remaining appArgs
awicenec Sep 21, 2023
0cf0b7c
removed allow_pickle for reduction functions
awicenec Sep 26, 2023
6f2a955
Adjusted for OOD
awicenec Nov 7, 2023
168f4a9
Delegate execution of AppDROP.run() to DropRunner
juliancarrivick May 23, 2023
a0d746e
Implement ProcessDropRunner
juliancarrivick Jan 11, 2024
2094405
Skip NGAS tests
juliancarrivick Jan 11, 2024
8b97854
Fix lgweb errors
juliancarrivick Jan 15, 2024
0b34a2f
Remove unnecessary file
rtobar Mar 21, 2023
34127cd
Avoid deprecated python features
rtobar May 23, 2023
71c0a12
Make RE objects static members of AbstractDROP
rtobar May 24, 2023
b4ad620
Introduce new ProxyInfo object
rtobar Mar 21, 2023
af33db4
Store session ID instead of full session in drops
rtobar May 24, 2023
911e50e
Store RPC server endpoint instead of full server in drops
rtobar May 24, 2023
11e9109
Delegate execution of AppDROP.run() to DropRunner
juliancarrivick May 23, 2023
1e431cf
Implement ProcessDropRunner
juliancarrivick Jan 11, 2024
eea4ac3
Skip NGAS tests
juliancarrivick Jan 11, 2024
ac31f87
Fix lgweb errors
juliancarrivick Jan 15, 2024
11bf09d
Merge pull request #239 from ICRAR/multiproc-drop-apps-julian
juliancarrivick Feb 28, 2024
3bdf3bd
Fixed pydantic, string2json docstring
awicenec Mar 9, 2024
21ba3e8
Fixed pydata and tests
awicenec Mar 11, 2024
170bfac
Fixed parset drop and added formats
awicenec Mar 12, 2024
f493b76
Updated paramset description
awicenec Mar 12, 2024
537540c
Added benedict to docs build
awicenec Mar 12, 2024
585ff99
Fixed test
awicenec Mar 12, 2024
44bf360
Small documentation change
awicenec Mar 15, 2024
d1240d4
Fixed parset tests
awicenec Mar 17, 2024
014d2cc
Merge branch 'multiproc-drop-apps-julian' into liu-367
awicenec Mar 17, 2024
f9b6ac8
Chasing test errors
awicenec Mar 18, 2024
54db33d
Still chasing
awicenec Mar 18, 2024
ff883fa
Finally tracked down the problem. Mereg complete!
awicenec Mar 18, 2024
3a86977
Added parser params to docstrings
awicenec Mar 18, 2024
d6a45c4
dropclass update for InMemory drop
awicenec Mar 18, 2024
00d6d0f
Update build-documentation.yml
awicenec Mar 18, 2024
75bdd6d
Update create-palettes.yml
awicenec Mar 18, 2024
cdf4f16
Update linting.yml
awicenec Mar 18, 2024
16bbc6b
Update run-unit-tests.yml
awicenec Mar 18, 2024
671d6e3
Update validate-graphs.yml
awicenec Mar 18, 2024
991c8db
Bumped minor version
awicenec Mar 18, 2024
9d8d612
daliuge 3.1.0
awicenec Mar 18, 2024
8567cd3
updated worflows
awicenec Mar 18, 2024
fa39900
Timeout for delayed requests
awicenec Mar 18, 2024
d0b974d
Fixed dlm test
awicenec Mar 18, 2024
f0a8b82
Fixed versions of zeroconf and zmq
awicenec Mar 18, 2024
3899bb5
PYTHONPATH fix for tests
awicenec Mar 18, 2024
590e0d8
Fixed typo in workflow
awicenec Mar 18, 2024
dd8219d
More path fixes
awicenec Mar 18, 2024
ba77770
Cleanup
awicenec Mar 18, 2024
bc75eef
Merge branch 'master' into liu-367
awicenec Mar 18, 2024
83fbcc7
Merge branch 'liu-367'
awicenec Mar 18, 2024
d40827e
Fixed reading from proxy
awicenec Mar 18, 2024
04777bc
Removed unused import
awicenec Mar 18, 2024
072ce08
Removed instance test
awicenec Mar 18, 2024
d2281b1
Removed the 'Description' node from the DALiuGE template palette
james-strauss-uwa Mar 20, 2024
1530ae3
Removed the 'Description' node from a sample graph
james-strauss-uwa Mar 20, 2024
63b1605
added entrypoint field
awicenec Mar 20, 2024
691d4b0
Removed width and height attributes from the node
james-strauss-uwa Mar 21, 2024
db41672
Removed width and height attributes from the list of required attributes
james-strauss-uwa Mar 21, 2024
1a509b9
Additions to the logical graph schema based on latest LG JSON format
james-strauss-uwa Mar 22, 2024
4e730ae
Fixed minor JSON error
james-strauss-uwa Mar 22, 2024
da55449
Added PythonObjects to palettes
awicenec Mar 25, 2024
fee9ef8
Merge branch 'eagle-pi-schema-update'
awicenec Mar 25, 2024
cd5b281
Updated test graphs to validate
awicenec Mar 25, 2024
96e04aa
Fixed graphs and tests
awicenec Mar 25, 2024
b11f970
daliuge 4.0.0
awicenec Mar 25, 2024
5e397f7
Merge branch 'master' into eagle-1165
james-strauss-uwa Mar 26, 2024
956a0ae
Fixed docker builds and CORS error
awicenec Mar 27, 2024
ba5a04a
Added https address to CORS
awicenec Mar 27, 2024
ad3d520
Fixed log output
awicenec Apr 2, 2024
77df9c4
added raw output, cleanup
awicenec Apr 10, 2024
2c54d27
avahi host names and bash command line arguments.
awicenec Apr 12, 2024
5d63ef7
Start avahi in translator image
awicenec Apr 12, 2024
417f351
Fixed log output
awicenec Apr 2, 2024
1cdf057
Allow NM port to be specified in CLI
awicenec Apr 15, 2024
e6b546c
Merge branch 'fix_nm_port'
awicenec Apr 15, 2024
5c99af0
Removed debug log
awicenec Apr 15, 2024
3b4ef66
daliuge 4.0.1
awicenec Apr 15, 2024
788b082
Added avahi to all targets
awicenec Apr 15, 2024
6a7bf50
Fix pgt assignment
awicenec Apr 16, 2024
4a61999
Fixed DIM node issue
awicenec Apr 16, 2024
df48d30
Updates to fix ZMQ with ports
awicenec Apr 17, 2024
0552ec3
Test update
awicenec Apr 17, 2024
bf6bfa0
More zmq updates
awicenec Apr 17, 2024
ee6b3c1
eval for memory pydata; dict gather app
awicenec Apr 21, 2024
d88af75
Allow appArg dict values without port
awicenec Apr 21, 2024
12438d2
Update merkletree to support Python3.10
myxie Apr 24, 2024
de0b52f
Updated regexp for CORS
awicenec Apr 26, 2024
ba97608
Merge pull request #240 from ICRAR/eagle-1165
james-strauss-uwa Apr 26, 2024
6d4b498
Fixed tests with manager ports
awicenec May 1, 2024
5e97bc9
Fix README.rst and Doc failures
myxie May 1, 2024
88714f3
Remove commented out create palettes link
myxie May 1, 2024
0c17c97
Fixed MM tests for node with ports
awicenec May 1, 2024
f542078
Add --show-coverage=no to stop polluting test runner output
myxie May 7, 2024
3d166e1
Fix spelling error
myxie May 7, 2024
988de16
Merge pull request #243 from myxie/dlg_modify_merkletree_install
awicenec May 13, 2024
c9ab31f
Bump pydantic from 1.10.7 to 1.10.13 in /daliuge-common
dependabot[bot] May 13, 2024
79be968
Merge pull request #242 from ICRAR/new-slim
awicenec May 13, 2024
95178a0
Update create-palettes.yml
awicenec May 13, 2024
86aac95
Removed the 'dataType' attribute from edges in the graph schema
james-strauss-uwa May 13, 2024
1300d84
Changed graph to fix translator tests
awicenec May 13, 2024
b919495
Removed 'dataType' attribute from existing graphs
james-strauss-uwa May 13, 2024
2427335
Merge branch 'graph-enabler' into eagle-1184
awicenec May 13, 2024
09eb7d0
Fixed translator test graph
awicenec May 13, 2024
51b7ea0
Merge pull request #251 from ICRAR/eagle-1184
awicenec May 13, 2024
3cc1d73
Fixed translator test graph
awicenec May 13, 2024
0fa4e5c
Removed 'dataType' attribute on edges from two graphs I missed
james-strauss-uwa May 13, 2024
3579fb5
Merge pull request #247 from ICRAR/fix_readme_and_docs
awicenec May 13, 2024
bf3a8fc
Merge pull request #244 from ICRAR/dependabot/pip/daliuge-common/pyda…
awicenec May 13, 2024
61a0c59
fixed translator test graph
awicenec May 13, 2024
7b89f00
Improve support for keyword-positional arguments
myxie May 15, 2024
ca276d2
Self-review
myxie May 20, 2024
89b8d34
Add manager_host to list of nodes when deploy is Server.
myxie May 23, 2024
e599c56
Fix missing rtd_sphinx_them install directive
myxie May 24, 2024
c7cf346
Merge pull request #257 from ICRAR/docs_fix-missing-install
myxie May 26, 2024
b5b65c3
Fix indentation error
myxie May 26, 2024
7e2b744
Merge pull request #258 from ICRAR/docs_fix-missing-install
myxie May 26, 2024
a3e225f
Address sourcery comments
myxie May 27, 2024
22fb53d
Merge pull request #256 from ICRAR/fix_server_genpgt_nodelist
myxie May 28, 2024
32cbacd
LIU-388: Update test_pg_gen.py to compare results to test data.
myxie May 29, 2024
a797a6a
Allow defining func_code as a plain string
awicenec May 29, 2024
bdd831c
Fixed backwards compatibility for dask graphs
awicenec May 29, 2024
9dd85e7
Final updates, all tests passing
awicenec May 29, 2024
caa24e3
LIU-385: Add support for SubGraphs in translator
myxie May 31, 2024
e9519ee
Reverted back to self instead of object
awicenec May 31, 2024
6ff595e
Merge pull request #250 from ICRAR/remove-logspam-from-testrunner
myxie Jun 4, 2024
ae0743d
Merge pull request #262 from ICRAR/LIU-385
myxie Jun 6, 2024
c0522c0
Merge pull request #254 from ICRAR/fix_positional_arguments
myxie Jun 6, 2024
9818b30
LIU-388: Code review updates
myxie Jun 7, 2024
6eccfb9
LIU-388: Remove globals from __init__.py
myxie Jun 7, 2024
061a7b2
LIU-388: __init__.py
myxie Jun 7, 2024
c2d8027
LIU-388: __init__.py
myxie Jun 7, 2024
db91b75
Merge pull request #260 from ICRAR/LIU-388_UpdatePGGenTests
myxie Jun 10, 2024
377cde8
Added 'encoding' attribute to fields in LG schema
james-strauss-uwa Jun 13, 2024
81702a5
Added 'encoding' attribute to fields in LG schema
james-strauss-uwa Jun 13, 2024
3eb6e49
Added allowed values for encoding attribute for fields to LG schema
james-strauss-uwa Jun 13, 2024
4ccb13d
Added allowed values for encoding attribute for fields to LG schema
james-strauss-uwa Jun 13, 2024
b15b783
Removed 'encoding' field attribute from required list
james-strauss-uwa Jun 13, 2024
aa73a5a
Removed 'encoding' field attribute from required list
james-strauss-uwa Jun 13, 2024
b574de0
Merge branch 'eagle-1231' of https://github.com/ICRAR/daliuge into ea…
james-strauss-uwa Sep 10, 2024
2178f5d
Feedback from Ryan and Andreas
james-strauss-uwa Sep 10, 2024
84216fe
Re-added 'utf-8' back to the list of options for encoding
james-strauss-uwa Sep 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions daliuge-translator/dlg/dropmake/lg.graph.schema
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,11 @@
"keyAttribute": {
"type": "boolean"
},
"encoding": {
"type": "string",
"enum": ["binary", "pickle", "dill", "npy", "base64", "utf-8"],
"default": "dill"
},
"id": {
"type": "string"
},
Expand Down
Loading