Skip to content

Commit

Permalink
more lib doc fixes, especially broken links.
Browse files Browse the repository at this point in the history
Signed-off-by: David Wood <[email protected]>
  • Loading branch information
daw3rd committed May 10, 2024
1 parent 54287c4 commit 5ef10ca
Show file tree
Hide file tree
Showing 5 changed files with 15 additions and 14 deletions.
1 change: 1 addition & 0 deletions data-processing-lib/doc/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ To learn more consider the following:
* [Transform Runtimes](transform-runtimes.md)
* [Transform Examples](transform-tutorial-examples.md)
* [Testing Transforms](transform-testing.md)
* [Utilities](transformer-utilities.md)
* [Architecture Deep Dive](architecture.md)
* [Transform project root readme](../../transforms/README.md)

4 changes: 3 additions & 1 deletion data-processing-lib/doc/python-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,6 @@ A `PythonTransformLauncher` class is provided that enables the running of the tr
```python
launcher = PythonTransformLauncher(YourTransformConfiguration())
launcher.launch()
```
```
The `YourTransformConfiguration` class configures your transform.
More details can be found in the [transform tutorial](transform-tutorials.md).
17 changes: 9 additions & 8 deletions data-processing-lib/doc/ray-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,10 @@ and class. In addition, it is responsible for providing transform-specific
methods to define and capture optional command line arguments.
```python

class MyTransformConfiguration(TransformConfiguration):
class YourTransformConfiguration(TransformConfiguration):

def __init__(self):
super().__init__(name="MyTransform", transform_class=MyTransform)
super().__init__(name="YourTransform", transform_class=YourTransform)
self.params = {}

def add_input_params(self, parser: ArgumentParser) -> None:
Expand All @@ -69,16 +69,17 @@ class MyTransformConfiguration(TransformConfiguration):
...
```
Next we define the Ray-runtime specific transform configuration as an exension of
the RayTransformConfiguration and uses the `MyTransformConfiguration` above.
the RayTransformConfiguration and uses the `YourTransformConfiguration` above.
```python

class MyTransformConfiguration(RayTransformConfiguration):
class YourTransformConfiguration(RayTransformConfiguration):
def __init__(self):
super().__init__(MyTransformConfiguration(),
runtime_class=MyTransformRuntime
super().__init__(YourTransformConfiguration(),
runtime_class=YourTransformRuntime
```
This class provides the ability to create the instance of `MyTransformRuntime` class
as neede by the Ray runtime.
This class provides the ability to create the instance of `YourTransformRuntime` class (see below)
as needed by the Ray runtime. Note, that not all transforms will require a `runtime_class`
and can omit this parameter to default to an acceptable runtime class.
Details are covered in the [advanced transform tutorial](advanced-transform-tutorial.md).

## Transform Runtime
Expand Down
4 changes: 2 additions & 2 deletions data-processing-lib/doc/transform-testing.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Transform Testing
Once a transform has been built, testing can be enabled with the provided testing framework:

* [Standalone Transform Testing](transform-standalone-testing) - shows how to test a transform independent of the runtime.
* [Standalone Transform Testing](transform-standalone-testing.md) - shows how to test a transform independent of the runtime.
* [End-to-End Testing](testing-e2e-transform.md) - shows how to test the transform running in a runtime.
* [Testing Transforms with S3](transform-s3-testing) -shows how to set up `minio` to test transforms with the S3 interfaces.
* [Testing Transforms with S3](transform-s3-testing.md) -shows how to set up `minio` to test transforms with the S3 interfaces.
3 changes: 0 additions & 3 deletions data-processing-lib/doc/transform-tutorial-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,3 @@ resources (models, configuration, etc) for a transform.



## Additional transform support

We also started a library of [transform utilities](transformer-utilities.md)

0 comments on commit 5ef10ca

Please sign in to comment.