Skip to content

Commit

Permalink
add reference to etl-entities
Browse files Browse the repository at this point in the history
  • Loading branch information
maxim-lixakov authored and dolfinus committed Nov 17, 2023
1 parent e8531b8 commit ff10be9
Show file tree
Hide file tree
Showing 5 changed files with 32 additions and 5 deletions.
7 changes: 7 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@
"sphinxcontrib.autodoc_pydantic",
"sphinxcontrib.towncrier", # provides `towncrier-draft-entries` directive
"sphinxcontrib.plantuml",
"sphinx.ext.extlinks",
]
numpydoc_show_class_members = True
autodoc_pydantic_model_show_config = False
Expand Down Expand Up @@ -119,6 +120,12 @@
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "sphinx"

# Create an alias for etl-entities lib in onetl documentation
extlinks = {
"etl-entities": ("https://etl-entities.readthedocs.io/en/latest/%s", None),
}


# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False

Expand Down
24 changes: 22 additions & 2 deletions docs/hwm_store/index.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,30 @@
.. _hwm-store:

YAML HWM Store
HWM
=========

Since ``onetl>=0.10.0`` version, the HWM Store and HWM classes have been moved to a separate library :etl-entities:`etl-entities <>`.

|
| **HWM Store**
- :etl-entities:`Base HWM Store <hwm_store/base_hwm_store.html>` This class facilitates the creation of custom hwm stores.
- :etl-entities:`Memory HWM Store <hwm_store/memory_hwm_store.html>` This class serving as an example for storing hwms in memory.
|
| **HWM**
Moreover, HWM classes that operate with HWM stores have also been relocated to the :etl-entities:`etl-entities <>` library. These include:

- :etl-entities:`Column Int HWM <hwm/column/int_hwm.html>`
- :etl-entities:`Column Date HWM <hwm/column/date_hwm.html>`
- :etl-entities:`Column DateTime HWM <hwm/column/datetime_hwm.html>`
- :etl-entities:`File List HWM <hwm/file/file_list_hwm.html>`

For detailed insights into the various HWM classes available, refer to the documentation on:

- :etl-entities:`What is HWM </hwm/column/index.html#what-is-hwm>`

Below is the **default HWM store** in onETL - YAML HWM Store:

.. toctree::
:maxdepth: 2
:caption: HWM store

yaml_hwm_store
2 changes: 1 addition & 1 deletion docs/hwm_store/yaml_hwm_store.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.. _yaml-hwm-store:

YAML HWM Store (local, default)
YAML HWM Store
=================================================================

.. currentmodule:: onetl.hwm.store.yaml_hwm_store
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@

.. toctree::
:maxdepth: 2
:caption: HWM and incremental reads
:caption: Read strategies and HWM
:hidden:

strategy/index
Expand Down
2 changes: 1 addition & 1 deletion onetl/hwm/store/yaml_hwm_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ class MyClass(BaseHWMStore):
@register_hwm_store_class("yaml")
@support_hooks
class YAMLHWMStore(BaseHWMStore, FrozenModel):
r"""YAML local store for HWM values. Used as default HWM store. |support_hooks|
r"""YAML **local store** for HWM values. Used as default HWM store. |support_hooks|
Parameters
----------
Expand Down

0 comments on commit ff10be9

Please sign in to comment.