Skip to content
This repository has been archived by the owner on Sep 4, 2024. It is now read-only.

Replace Airflow guide in docs/guides with Astro version of Marquez Airflow example #185

Merged
merged 14 commits into from
Jul 7, 2023

Conversation

merobi-hub
Copy link
Collaborator

@merobi-hub merobi-hub commented Jun 16, 2023

We need an easy-to-configure, functional Airflow quickstart, but airflow.md in docs/guides is not working currently and requires complex configuration across multiple services.

This:

  • edits and retitles airflow.md to reflect the fact that it's really a guide to the proxy (to give us a proxy guide)
  • with the blessing of @wslulciuc , updates the Airflow tutorial in marquez/examples to use Astro (with the help of the Astro tutorial here) and adds it to guides as airflow-quickstart

Note: with the changes needed to upgrade to Astro this is significantly different from the original in Marquez, so it seems that both are worth maintaining.

… retitle existing guide to emphasize proxy

Signed-off-by: Michael Robinson <[email protected]>
@merobi-hub merobi-hub changed the title Replace Airflow quickstart with tutorial in marquez/examples Replace Airflow guide in docs/guides with tutorial in marquez/examples Jun 16, 2023
@merobi-hub merobi-hub changed the title Replace Airflow guide in docs/guides with tutorial in marquez/examples Replace Airflow guide in docs/guides with Astro version of Marquez Airflow example Jun 27, 2023
Signed-off-by: Michael Robinson <[email protected]>
docs/guides/new-quickstart.md Outdated Show resolved Hide resolved
Signed-off-by: Michael Robinson <[email protected]>
docs/guides/new-quickstart.md Outdated Show resolved Hide resolved
docs/guides/new-quickstart.md Outdated Show resolved Hide resolved
docs/guides/new-quickstart.md Outdated Show resolved Hide resolved
docs/guides/new-quickstart.md Outdated Show resolved Hide resolved
docs/guides/new-quickstart.md Outdated Show resolved Hide resolved
@merobi-hub
Copy link
Collaborator Author

Thanks @kentdanas for all the improvements. Incorporating them into a single forthcoming commit.

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Michael Robinson <[email protected]>
Copy link
Contributor

@wslulciuc wslulciuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for improving the docs, @merobi-hub. I have some suggestion to simplify the setup steps:

  1. Use published Marquez image
  2. Use published OpenLineage proxy image
  3. Define a combined docker compose file that sets up both services; users would no longer have to clone repos to get started or run any scripts.

docs/guides/airflow_proxy.md Show resolved Hide resolved
docs/guides/airflow_proxy.md Show resolved Hide resolved
Signed-off-by: Michael Robinson <[email protected]>
@merobi-hub
Copy link
Collaborator Author

Thanks @wslulciuc . Propose opening a separate PR to make the more substantial suggested improvements to the proxy guide so that the new quickstart is available. The separate PR will:

  • use published images for services
  • combine services in one docker compose

@merobi-hub merobi-hub requested a review from wslulciuc July 7, 2023 13:09
Signed-off-by: Michael Robinson <[email protected]>
@merobi-hub
Copy link
Collaborator Author

Renamed new-quickstart as airflow-quickstart

Copy link
Contributor

@wslulciuc wslulciuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You officially have my blessing to us the Marquez example for Airflow! Great work! 💯

@merobi-hub merobi-hub merged commit 482a00e into main Jul 7, 2023
2 checks passed
merobi-hub added a commit that referenced this pull request Aug 4, 2023
…rflow example (#185)

* replace airflow quickstart with marquez/examples/airflow tutorial and retitle existing guide to emphasize proxy

---------

Signed-off-by: Michael Robinson <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants