Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create python module to validate processed data #18

Closed
wants to merge 28 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
8c1dd85
add modularized validation
horstf Apr 13, 2022
d181201
add shape to test
horstf Apr 13, 2022
fc37a5e
add old dodo.py code from ont_dev branch (have to update)
horstf Apr 13, 2022
68aed79
update requirements and environment
horstf Apr 13, 2022
f49e040
add __init__.py for modularization of code
horstf May 3, 2022
644a4c9
rename emodul_validation.py to validation.py to imporve python modula…
horstf May 3, 2022
f752d17
add validation to dodo.py
horstf May 3, 2022
c4dd67b
add shape as dep to dodo.py
horstf May 3, 2022
230ac90
move and rename shape
horstf May 3, 2022
458da5e
merge main
horstf Jul 26, 2022
1e3697e
move validation into own lebedigital submodule
horstf Jul 26, 2022
f35255b
validation didnt need additional submodule structure
horstf Jul 26, 2022
379c107
add comments
horstf Jul 26, 2022
2a3e61c
add comments
horstf Aug 4, 2022
b9b3175
remove superflous comments
horstf Aug 4, 2022
c150d15
remove unused file from concrete
horstf Sep 5, 2022
8b25331
add files to test validation
horstf Sep 7, 2022
da2cebf
add validation tests
horstf Sep 7, 2022
de56554
add validation tests
horstf Sep 7, 2022
1a49eab
remove empty lines
horstf Sep 7, 2022
26e0925
add validation tests
horstf Sep 7, 2022
9e92d70
undo changes in requirements.txt
horstf Sep 8, 2022
3133735
undo changes in requirements.txt
horstf Sep 8, 2022
f0d889a
Merge remote-tracking branch 'origin/main' into validation_modulariza…
horstf Oct 7, 2022
0d501bc
add some dodo code
horstf Oct 7, 2022
3cd8a72
add output code
horstf Oct 7, 2022
a3917f9
add output code
horstf Oct 7, 2022
2678023
update dodo
horstf Jan 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add __init__.py for modularization of code
horstf committed May 3, 2022
commit f49e04025c3174d65f1014612e8bd59b563bf29a
20 changes: 17 additions & 3 deletions usecases/Concrete/dodo.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,30 @@
import graphlib
import os
from pathlib import Path
from emodul import emodul_validation as validation

baseDir = Path(__file__).resolve().parents[0]
emodulFolder = os.path.join(os.path.join(os.path.join(baseDir,'knowledgeGraph'),'emodul'),'E-modul-processed-data')
emodulFolder = os.path.join(os.path.join(baseDir,'knowledgeGraph'),'emodul')
emodulRawdataFolder = os.path.join(emodulFolder,'rawdata')
emodulProcesseddataFolder = os.path.join(emodulFolder,'processeddata')
emodulProcesseddataFolder = os.path.join(emodulFolder,'E-modul-processed-data')

compressionFolder = os.path.join(os.path.join(os.path.join(baseDir,'knowledgeGraph'),'compression'),'compression-processed-data')
compressionRawdataFolder = os.path.join(compressionFolder,'rawdata')
compressionProcesseddataFolder = os.path.join(compressionFolder,'processeddata')

graph_path = os.path.join(emodulProcesseddataFolder, 'EM_Graph.ttl')
shapes_path = os.path.join(os.path.join(baseDir, 'knowledgeGraph'), 'shape.ttl')

DOIT_CONFIG = {'verbosity': 2}

def validate_graph(graph_path, shapes_path):
g = validation.read_graph_from_file(graph_path)
s = validation.read_graph_from_file(shapes_path)
r = validation.test_graph(g, s)
assert validation.violates_shape(r, validation.SCHEMA.InformationBearingEntityShape)
assert not validation.violates_shape(r, validation.SCHEMA.SpecimenDiameterShape)
assert not validation.violates_shape(r, validation.SCHEMA.SpecimenShape)

def task_installation():
yield {
'basename': 'install python packages',
@@ -53,7 +66,8 @@ def task_emodul():
}
yield {
'basename': 'validate rdf files against shacl shape',
'actions': ['python knowledgeGraph/emodul/emodul_validation.py']
'actions': [(validate_graph, [graph_path, shapes_path])],
'file_dep': [graph_path]
}
yield {
'basename': 'run emodul query script',
Empty file.