diff --git a/data-processing-lib/doc/overview.md b/data-processing-lib/doc/overview.md index d8b0498d70..7ec1f4288c 100644 --- a/data-processing-lib/doc/overview.md +++ b/data-processing-lib/doc/overview.md @@ -22,6 +22,7 @@ To learn more consider the following: * [Transform Runtimes](transform-runtimes.md) * [Transform Examples](transform-tutorial-examples.md) * [Testing Transforms](transform-testing.md) +* [Utilities](transformer-utilities.md) * [Architecture Deep Dive](architecture.md) * [Transform project root readme](../../transforms/README.md) diff --git a/data-processing-lib/doc/python-runtime.md b/data-processing-lib/doc/python-runtime.md index 982bdeaccf..9b819b0b05 100644 --- a/data-processing-lib/doc/python-runtime.md +++ b/data-processing-lib/doc/python-runtime.md @@ -7,4 +7,6 @@ A `PythonTransformLauncher` class is provided that enables the running of the tr ```python launcher = PythonTransformLauncher(YourTransformConfiguration()) launcher.launch() -``` \ No newline at end of file +``` +The `YourTransformConfiguration` class configures your transform. +More details can be found in the [transform tutorial](transform-tutorials.md). \ No newline at end of file diff --git a/data-processing-lib/doc/ray-runtime.md b/data-processing-lib/doc/ray-runtime.md index dc40a44017..c815af4b3e 100644 --- a/data-processing-lib/doc/ray-runtime.md +++ b/data-processing-lib/doc/ray-runtime.md @@ -57,10 +57,10 @@ and class. In addition, it is responsible for providing transform-specific methods to define and capture optional command line arguments. ```python -class MyTransformConfiguration(TransformConfiguration): +class YourTransformConfiguration(TransformConfiguration): def __init__(self): - super().__init__(name="MyTransform", transform_class=MyTransform) + super().__init__(name="YourTransform", transform_class=YourTransform) self.params = {} def add_input_params(self, parser: ArgumentParser) -> None: @@ -69,16 +69,17 @@ class MyTransformConfiguration(TransformConfiguration): ... ``` Next we define the Ray-runtime specific transform configuration as an exension of -the RayTransformConfiguration and uses the `MyTransformConfiguration` above. +the RayTransformConfiguration and uses the `YourTransformConfiguration` above. ```python -class MyTransformConfiguration(RayTransformConfiguration): +class YourTransformConfiguration(RayTransformConfiguration): def __init__(self): - super().__init__(MyTransformConfiguration(), - runtime_class=MyTransformRuntime + super().__init__(YourTransformConfiguration(), + runtime_class=YourTransformRuntime ``` -This class provides the ability to create the instance of `MyTransformRuntime` class -as neede by the Ray runtime. +This class provides the ability to create the instance of `YourTransformRuntime` class (see below) +as needed by the Ray runtime. Note, that not all transforms will require a `runtime_class` +and can omit this parameter to default to an acceptable runtime class. Details are covered in the [advanced transform tutorial](advanced-transform-tutorial.md). ## Transform Runtime diff --git a/data-processing-lib/doc/transform-testing.md b/data-processing-lib/doc/transform-testing.md index 4d9e6c9c72..6a421ddf7a 100644 --- a/data-processing-lib/doc/transform-testing.md +++ b/data-processing-lib/doc/transform-testing.md @@ -1,6 +1,6 @@ # Transform Testing Once a transform has been built, testing can be enabled with the provided testing framework: -* [Standalone Transform Testing](transform-standalone-testing) - shows how to test a transform independent of the runtime. +* [Standalone Transform Testing](transform-standalone-testing.md) - shows how to test a transform independent of the runtime. * [End-to-End Testing](testing-e2e-transform.md) - shows how to test the transform running in a runtime. -* [Testing Transforms with S3](transform-s3-testing) -shows how to set up `minio` to test transforms with the S3 interfaces. \ No newline at end of file +* [Testing Transforms with S3](transform-s3-testing.md) -shows how to set up `minio` to test transforms with the S3 interfaces. \ No newline at end of file diff --git a/data-processing-lib/doc/transform-tutorial-examples.md b/data-processing-lib/doc/transform-tutorial-examples.md index 673b9ff10a..cd971d2b57 100644 --- a/data-processing-lib/doc/transform-tutorial-examples.md +++ b/data-processing-lib/doc/transform-tutorial-examples.md @@ -13,6 +13,3 @@ resources (models, configuration, etc) for a transform. -## Additional transform support - -We also started a library of [transform utilities](transformer-utilities.md) \ No newline at end of file