Update trestle project structure to better define the SDK APIs

[jpower432]

Issue description / feature objectives

To support the trestle SDK user, create well-defined APIs so users can understand the SDK functionality what API signatures cannot be changed without breaking backward compatibility.

Caveats / Assumptions

• Keep package as one published package per discussion on feat(proposals): Decouple the trestle CLI from the SDK community#68

Completion Criteria

• Define the common utilities that are used in both the SDK and CLI (e.g. list-utils)
• Define logic that supports the CLI command functionality only and what supports the trestle SDK
• Split the SDK into its own sub-package of trestle

Proposed Child Issues

• Implement a Python class in the SDK to represent Trestle OSCAL extension. #1775
• Define Trestle OSCAL extensions in a language agnostic way #1779
• Create an SDK package with OSCAL-based Python classes and common types #1780
• Move model utilities to SDK package #1781
• Move validations to SDK package #1782
• Move transformers to SDK package #1783

Related to #924

[jpower432]

@degenaro @vikas-agarwal76 @butler54 I wanted to start collaboration on the structure here so I thought the most straightforward way might be to separate the existing sub-packages into categories. Let me know what you think or if you would rather look at a draft pull request.

To summarize an idea that we could use as a jumping off point:

• CLI: Trestle specific functionality - especially relating to Markdown validation/generation, agile authoring support, commands.
• Tasks (kept as is)
• Transforms: Keep implementations in the top transforms package and move the base to the SDK
• Utils: Constants, generic utilities like list utils or str_utils
• SDK: Models, model utils, model validation, model generation, resolver, model transformation (base)

cli
└── core
    ├── catalog
    │   ├── catalog_api.py
    │   ├── catalog_merger.py
    │   ├── catalog_reader.py
    │   ├── catalog_writer.py
    │   └── __init__.py
    ├── commands
    │   ├── add.py
    │   ├── assemble.py
    │   ├── author
    │   │   ├── catalog.py
    │   │   ├── command.py
    │   │   ├── common.py
    │   │   ├── component.py
    │   │   ├── consts.py
    │   │   ├── docs.py
    │   │   ├── folders.py
    │   │   ├── headers.py
    │   │   ├── __init__.py
    │   │   ├── jinja.py
    │   │   ├── prof.py
    │   │   ├── ssp.py
    │   │   └── versioning
    │   │       ├── __init__.py
    │   │       └── template_versioning.py
    │   ├── command_docs.py
    │   ├── common
    │   │   ├── cmd_utils.py
    │   │   ├── __init__.py
    │   │   └── return_codes.py
    │   ├── create.py
    │   ├── describe.py
    │   ├── href.py
    │   ├── import_.py
    │   ├── __init__.py
    │   ├── init.py
    │   ├── merge.py
    │   ├── partial_object_validate.py
    │   ├── py.typed
    │   ├── remove.py
    │   ├── replicate.py
    │   ├── split.py
    │   ├── task.py
    │   ├── validate.py
    │   └── version.py
    ├── control_context.py
    ├── control_reader.py
    ├── control_writer.py
    ├── crm
    │   ├── bycomp_interface.py
    │   ├── export_reader.py
    │   ├── export_writer.py
    │   ├── __init__.py
    │   ├── leveraged_statements.py
    │   └── ssp_inheritance_api.py
    ├── docs_control_writer.py
    ├── draw_io.py
    ├── __init__.py
    ├── jinja.py
    ├── markdown
    │   ├── base_markdown_node.py
    │   ├── control_markdown_node.py
    │   ├── docs_markdown_node.py
    │   ├── __init__.py
    │   ├── markdown_api.py
    │   ├── markdown_const.py
    │   ├── markdown_processor.py
    │   ├── markdown_validator.py
    │   └── md_writer.py
    ├── models
    │   ├── actions.py
    │   ├── file_content_type.py
    │   ├── __init__.py
    │   ├── plans.py
    │   └── py.typed
    ├── object_factory.py
    ├── parser.py
    ├── pipeline.py
    ├── profile_resolver.py
    ├── py.typed
    ├── remote
    │   ├── cache.py
    │   └── __init__.py
    ├── repository.py
    └── ssp_io.py

sdk
├── common_types.py
├── models
│   ├── base_model.py
│   ├── catalog_interface.py
│   ├── control_interface.py
│   ├── elements.py
│   ├── generators.py
│   ├── generic_oscal.py
│   ├── interfaces.py
│   ├── model_utils.py
│   └── trestle_base_model.py
├── oscal
│   ├── assessment_plan.py
│   ├── assessment_results.py
│   ├── catalog.py
│   ├── common.py
│   ├── component.py
│   ├── __init__.py
│   ├── poam.py
│   ├── profile.py
│   ├── py.typed
│   └── ssp.py
├── resolver
│   ├── _import.py
│   ├── __init__.py
│   ├── merge.py
│   ├── modify.py
│   └── prune.py
├── transforms
│   ├── __init__.py
│   ├── results.py
│   ├── transformer_factory.py
│   ├── transformer_helper.py
│   └── transformer_singleton.py
└── validation
    ├── all_validator.py
    ├── catalog_validator.py
    ├── duplicates_validator.py
    ├── links_validator.py
    ├── load_validate.py
    ├── refs_validator.py
    ├── rule_parameters_validator.py
    ├── validator_factory.py
    └── validator.py

[degenaro]

@jpower432 Seems like a good first swag. Maybe tests should be likewise structured?

[jpower432]

@vikas-agarwal76 @mrgadgil @AleJo2995 @butler54 @degenaro Following up from the review today. I am looking for feedback on this structure and the approach to implementing the change. Proposing the following approach:

• We get consensus on a high level structure, just so we can use that start defining a set of common functionality that the SDKs should implement that we can test for and at least to document what functionalities are supported or not supported yet.
• We create a set of issues to get these changes in develop in small increments. For example, we could start by creating the SDK package and moving the oscal sub-package underneath.

I also wanted to ask for everyone's thoughts on versioning. Since the public API is not fully defined yet, do we need to bump a major version if this results in no change to the trestle SDK end user?