Skip to content

Commit

Permalink
Merge pull request #6 from casework/revise_workflow
Browse files Browse the repository at this point in the history
Revise workflow and document new generation process
  • Loading branch information
kchason authored Sep 25, 2024
2 parents 63b5216 + c39997c commit 7b6228b
Show file tree
Hide file tree
Showing 451 changed files with 6,964 additions and 5,107 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@
__pycache__
.ipynb_checkpoints
.venv-pre-commit
all-*.mk
venv
20 changes: 20 additions & 0 deletions CONTRIBUTE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Maintenance

This repository derives JSON-LD "stub" dictionaries from the ontologies in CDO.

The generation process for these ontologies uses `make`, with some balance of hard-coding and source-including to save on code redundancy.

The maintenance necessary for this repository occurs when a new ontology is released. In most cases, this will be the necessary script to follow:

```bash
# (On a fresh Git clone.)
make clean
make -j
git add templates
git commit -s -m "Regenerate Make-managed files"
# Editing and updating README.md to refresh the implemented CASE version should come next.
```

(Note: Some `make`s assume infinite CPU resources if `-j` (`--jobs`) does not have a following numeric argument. On, e.g., macOS, you might want to follow that flag with the number of cores on your system.)

In the event a new ontology (i.e., a new namespace with `owl:Class`es) is added, a new directory under `/templates` will need to be created and given a `Makefile`. Copying `/templates/uco-core/Makefile` into the new directory and adapting its hard-coded prefix IRI will enable the workflow to function again.
2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ py_srcfiles := \
all: \
.venv-pre-commit/var/.pre-commit-built.log \
all-tests
$(MAKE) \
--directory templates

.PHONY: \
all-tests \
Expand Down
49 changes: 24 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,59 +1,58 @@
# CASE Mapping Template Stubs

[![Continuous Integration](https://github.com/casework/CASE-Mapping-Template-Stubs/actions/workflows/ci.yml/badge.svg)](https://github.com/casework/CASE-Mapping-Template-Stubs/actions/workflows/ci.yml)
![CASE Version](https://img.shields.io/badge/CASE%20Version-0.7.0-green)
![CASE Version](https://img.shields.io/badge/CASE%20Version-1.3.0-green)

This template repository is provided for those looking to draft JSON-oriented utilities using ontologies within the [Cyber Domain Ontology](https://cyberdomainontology.org) ecosystem, particularly [CASE](https://caseontology.org) and [UCO](https://unifiedcyberontology.org). "Stub" JSON dictionaries are provided for each class within CASE and UCO, generated from the ontologies.

Each "stub" provides a minimal JSON-LD context dictionary specific to a class, and null-valued entries for each property on that class and entailed by parent classes.
Each "stub" provides a minimal JSON-LD context dictionary specific to a class, and null-valued entries for each property on that class and entailed by parent classes. Cardinality is reflected by the null-ish value recorded: `null` is used when a property has maximum-cardinality 1, and an empty list (`[]`) is recorded when a property has a maximum-cardinality greater than 1 or is unbounded.


## Folders

1. `/templates` is generated by case_utils ontology and sub-ontologies alone.


## Install & setup
## Maintenance and refreshing

1. `pip install -r requirements.txt`
Resources in this repository do not require use of any code for their typical usage.

For those making updates to the repository (such as on ontology releases), see [`CONTRIBUTE.md`](CONTRIBUTE.md).

## Usage/run

- generate.py [-o: default: uses case_util's case ontology and sub ontologies,-s:Optional[],--output:default="templates", -a:default = False, -t:default = False]
- "-o", ontology directory or string of directories delimited by "," , required.
- "-s", specify a class by name that has at least one property, Optional. The name must be conventional to the original vocabulary eg. observable, tool, action, etc.
- "--output", specify a specific directory to put the stubs, Optional, default = "templates"
- "-a" True/False whether to allow case_utils to load in uco or not. The generated object will differ depending on what is within the graph. default = False.
- "-t" True/False whether to generate a short stub (True) or full stub (False). default = False

## Usage/run

## Examples
The script `src/generate_single_stub_json.py` is usable for generating a stub JSON-LD object for any class, if the ontology (including all ontologies reached by `owl:imports`) is provided on the command line.

generate all case and uco objects from case_utils alone:
```bash
python generate.py
```
usage: generate_single_stub_json.py [-h] [--debug] out_json class_iri [supplemental_graph ...]
generate a specific case object
```bash
python generate.py -s investigation:Authorization
positional arguments:
out_json
class_iri
supplemental_graph
options:
-h, --help show this help message and exit
--debug
```


### Examples

### Output
See [`tests/Makefile`](tests/Makefile) for examples of how to run `generate_single_stub_json.py` for specific classes of interest. Demonstrations are done for:

json-ld file with additional @version key for what version of case and case_utils generated the file.
* [`case-investigation:InvestigativeAction`](tests/InvestigativeAction.json)
* [`uco-observable:ArchiveFile`](tests/ArchiveFile.json)
* [`co:Bag`](tests/Bag.json) (from the [Collections Ontology](https://github.com/collections-ontology/collections-ontology))


### Note

1. we load the ontology ttl files directly, implying the graph is blank. The uco ontology can be loaded via case_utils load_subclass_hierarchy. Because it loads the uco ontology as well, some of the uco ontology will be generated too due to it being present in the triple preprocessing.

2. in `generate.py`, there is another 'obs_prefix' commented out which uses the direct uco ontology prefixes instead of being padded with "uco-", use it you want a stand-alone uco object with its original vocabulary prefixes: eg. case uses "uco-core" while the uco ontology uses "core".
1. Properties applicable to a class are drawn directly from the class definition, and from all parent class definitions.
1. Properties are also associated with classes from explicit `rdfs:domain` statements. When a `rdfs:domain` domain statement is absent, under OWL semantics this implies that the property applies to `owl:Thing` (i.e., is universal); but, to support use cases where domains are represented with other structural properties (e.g., `dcam:domainIncludes`, `schema:domainIncludes`), this project requires explicit statements of `rdfs:domain owl:Thing` for adding to a stub.

3. The code queries the graph for all triples with sh:property and sh:path to build the entire list, therefore classes that do not have direct properties (properties not from superclass inheritence) will not be geneated using the `--short True` flag. By default, the flag is set to `--short False` which additionally tries to pull triples that declare a subclass from a superclass, thereby adding superclass properties to the stub.
2. Stubs for `uco-core:Facet`s applicable to any eventual subclass of `uco-core:UcoObject` are also inlined in the stub for the `uco-core:UcoObject`.


## Licensing
Expand Down
Loading

0 comments on commit 7b6228b

Please sign in to comment.