Code

Topic/fault tolerance#34

Code

Open

pgzmnk wants to merge 7 commits intoanyscale:mainanyscale/airflow-provider-ray:mainfrom pgzmnk:mainpgzmnk/airflow-provider-ray:mainCopy head branch name to clipboard

+991-69

Commits

Commits on Aug 6, 2021

• Incorporate Fault Tolerance mechanism and its tests.

  pgzmnk

  committed
  1a515b9

  Copy full SHA for 1a515b9

Commits on Aug 11, 2021

• Implement checkpointing in AWS S3.

  pgzmnk

  committed
  c195a88

  Copy full SHA for c195a88

• Incorporate checkpoint flag.

  pgzmnk

  committed
  24338f8

  Copy full SHA for 24338f8

Commits on Aug 13, 2021

• Incorporate checkpoint_on_retry_callback to _retrieve_obj_id_from_xcom. Clean-up logs and docstrings. Standardize naming.

  pgzmnk

  committed
  94b0e2c

  Copy full SHA for 94b0e2c

Commits on Aug 22, 2021

• Create test to validate different Ray versions.

  pgzmnk

  committed
  70fc934

  Copy full SHA for 70fc934

Commits on Aug 29, 2021

• Update tests.

  pgzmnk

  committed
  7beb063

  Copy full SHA for 7beb063

Commits on Sep 17, 2021

• Add logging on file write to ray_decorators.py

  pgzmnk

  committed
  5cb7844

  Copy full SHA for 5cb7844