Skip to content

Commit

Permalink
Reduce NATS connectionBackoff in eventsource (#323)
Browse files Browse the repository at this point in the history
  • Loading branch information
grahamia authored Mar 15, 2024
1 parent fcfc03d commit 8b45755
Show file tree
Hide file tree
Showing 8 changed files with 10 additions and 50 deletions.
5 changes: 0 additions & 5 deletions docs-gen/content/docs/reference/run-completion.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,6 @@ metadata:
spec:
nats:
run-completion:
connectionBackoff:
duration: 10s
factor: 2
jitter: 0.2
steps: 5
jsonBody: true
subject: events
url: nats://eventbus-kfp-operator-events-stan-svc.kfp-operator.svc:4222
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,6 @@ metadata:
spec:
nats:
run-completion:
connectionBackoff:
duration: 10s
factor: 2
jitter: 0.2
steps: 5
jsonBody: true
subject: events
url: nats://eventbus-kfp-operator-events-stan-svc.kfp-operator.svc:4222
Expand Down
5 changes: 0 additions & 5 deletions docs/docs/examples/index.xml
Original file line number Diff line number Diff line change
Expand Up @@ -249,11 +249,6 @@ In this example we are updating a serving component with the location of the new
</span></span><span style="display:flex;"><span><span style="color:#f92672">spec</span>:
</span></span><span style="display:flex;"><span> <span style="color:#f92672">nats</span>:
</span></span><span style="display:flex;"><span> <span style="color:#f92672">run-completion</span>:
</span></span><span style="display:flex;"><span> <span style="color:#f92672">connectionBackoff</span>:
</span></span><span style="display:flex;"><span> <span style="color:#f92672">duration</span>: <span style="color:#ae81ff">10s</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">factor</span>: <span style="color:#ae81ff">2</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">jitter</span>: <span style="color:#ae81ff">0.2</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">steps</span>: <span style="color:#ae81ff">5</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">jsonBody</span>: <span style="color:#66d9ef">true</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">subject</span>: <span style="color:#ae81ff">events</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">url</span>: <span style="color:#ae81ff">nats://eventbus-kfp-operator-events-stan-svc.kfp-operator.svc:4222</span>
Expand Down
7 changes: 1 addition & 6 deletions docs/docs/examples/pipeline_training/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Create pipeline.py. Note that the pipeline definition itself is simpler because all infrastructure references, like pusher and pipeline root, will be injected by the operator before the pipeline is uploaded to Kubeflow:"><meta property="og:type" content="article"><meta property="og:url" content="https://sky-uk.github.io/kfp-operator/docs/examples/pipeline_training/"><meta property="article:section" content="docs"><meta itemprop=name content="Pipeline Training"><meta itemprop=description content="This tutorial walks you through the creation of a simple TFX pipeline on Kubeflow Pipelines and shows you how to manage pipelines via Kubernetes Custom Resources.
The examples for this tutorial can be found on GitHub.
1. Build the Pipeline We use the same pipeline as the TFX example with a few modifications.
Create pipeline.py. Note that the pipeline definition itself is simpler because all infrastructure references, like pusher and pipeline root, will be injected by the operator before the pipeline is uploaded to Kubeflow:"><meta itemprop=wordCount content="1134"><meta itemprop=keywords content><meta name=twitter:card content="summary"><meta name=twitter:title content="Pipeline Training"><meta name=twitter:description content="This tutorial walks you through the creation of a simple TFX pipeline on Kubeflow Pipelines and shows you how to manage pipelines via Kubernetes Custom Resources.
Create pipeline.py. Note that the pipeline definition itself is simpler because all infrastructure references, like pusher and pipeline root, will be injected by the operator before the pipeline is uploaded to Kubeflow:"><meta itemprop=wordCount content="1125"><meta itemprop=keywords content><meta name=twitter:card content="summary"><meta name=twitter:title content="Pipeline Training"><meta name=twitter:description content="This tutorial walks you through the creation of a simple TFX pipeline on Kubeflow Pipelines and shows you how to manage pipelines via Kubernetes Custom Resources.
The examples for this tutorial can be found on GitHub.
1. Build the Pipeline We use the same pipeline as the TFX example with a few modifications.
Create pipeline.py. Note that the pipeline definition itself is simpler because all infrastructure references, like pusher and pipeline root, will be injected by the operator before the pipeline is uploaded to Kubeflow:"><link rel=preload href=https://sky-uk.github.io/kfp-operator/scss/main.min.79e3930541ca05b5395c14b2a313b798fad1dc69f9a14aefa57b62eaa9f65f14.css as=style><link href=https://sky-uk.github.io/kfp-operator/scss/main.min.79e3930541ca05b5395c14b2a313b798fad1dc69f9a14aefa57b62eaa9f65f14.css rel=stylesheet integrity><script src=https://code.jquery.com/jquery-3.6.0.min.js integrity=sha384-vtXRMe3mGCbOeY7l30aIg8H9p3GdeSe4IFlP6G8JMa7o7lXvnz3GFKzPxzJdPfGK crossorigin=anonymous></script></head><body class=td-page><header><nav class="js-navbar-scroll navbar navbar-expand navbar-dark flex-column flex-md-row td-navbar"><a class=navbar-brand href=https://sky-uk.github.io/kfp-operator/><span class=navbar-logo></span><span class=font-weight-bold>KFP-Operator</span></a><div class="td-navbar-nav-scroll ml-md-auto" id=main_navbar><ul class="navbar-nav mt-2 mt-lg-0"><li class="nav-item mr-4 mb-2 mb-lg-0"><a class=nav-link href=https://sky-uk.github.io/kfp-operator/docs/><i class='fas fa-book pr-2'></i><span>Documentation</span></a></li><li class="nav-item mr-4 mb-2 mb-lg-0"><a class=nav-link href=https://github.com/sky-uk/kfp-operator/discussions target=_blank><i class='fas fa-comments'></i><span>Discuss</span></a></li><li class="nav-item mr-4 mb-2 mb-lg-0"><a class=nav-link href=https://github.com/sky-uk/kfp-operator target=_blank><i class='fab fa-github pr-2'></i><span>GitHub</span></a></li></ul></div><div class="navbar-nav d-none d-lg-block"></div></nav></header><div class="container-fluid td-outer"><div class=td-main><div class="row flex-xl-nowrap"><aside class="col-12 col-md-3 col-xl-2 td-sidebar d-print-none"><div id=td-sidebar-menu class=td-sidebar__inner><form class="td-sidebar__search d-flex align-items-center"><button class="btn btn-link td-sidebar__toggle d-md-none p-0 ml-3 fas fa-bars" type=button data-toggle=collapse data-target=#td-section-nav aria-controls=td-docs-nav aria-expanded=false aria-label="Toggle section navigation"></button></form><nav class="collapse td-sidebar-nav" id=td-section-nav><ul class="td-sidebar-nav__section pr-md-3 ul-0"><li class="td-sidebar-nav__section-title td-sidebar-nav__section with-child active-path" id=m-docs-li><a href=https://sky-uk.github.io/kfp-operator/docs/ class="align-left pl-0 td-sidebar-link td-sidebar-link__section tree-root" id=m-docs><span>Documentation</span></a><ul class=ul-1><li class="td-sidebar-nav__section-title td-sidebar-nav__section with-child" id=m-docsgetting-started-li><a href=https://sky-uk.github.io/kfp-operator/docs/getting-started/ class="align-left pl-0 td-sidebar-link td-sidebar-link__section" id=m-docsgetting-started><span>Getting Started</span></a><ul class="ul-2 foldable"><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsgetting-startedoverview-li><a href=https://sky-uk.github.io/kfp-operator/docs/getting-started/overview/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsgetting-startedoverview><span>Overview</span></a></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsgetting-startedinstallation-li><a href=https://sky-uk.github.io/kfp-operator/docs/getting-started/installation/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsgetting-startedinstallation><span>Installation</span></a></li></ul></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section with-child active-path" id=m-docsexamples-li><a href=https://sky-uk.github.io/kfp-operator/docs/examples/ class="align-left pl-0 td-sidebar-link td-sidebar-link__section" id=m-docsexamples><span>Examples</span></a><ul class="ul-2 foldable"><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsexamplespipeline_training-li><a href=https://sky-uk.github.io/kfp-operator/docs/examples/pipeline_training/ class="align-left pl-0 active td-sidebar-link td-sidebar-link__page" id=m-docsexamplespipeline_training><span class=td-sidebar-nav-active-item>Pipeline Training</span></a></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsexamplespipeline_dependencies-li><a href=https://sky-uk.github.io/kfp-operator/docs/examples/pipeline_dependencies/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsexamplespipeline_dependencies><span>Pipeline Dependencies</span></a></li></ul></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section with-child" id=m-docsreference-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/ class="align-left pl-0 td-sidebar-link td-sidebar-link__section" id=m-docsreference><span>Reference</span></a><ul class="ul-2 foldable"><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsreferenceconfiguration-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/configuration/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsreferenceconfiguration><span>Configuration</span></a></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section with-child" id=m-docsreferenceresources-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/resources/ class="align-left pl-0 td-sidebar-link td-sidebar-link__section" id=m-docsreferenceresources><span>Resources</span></a><ul class="ul-3 foldable"><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsreferenceresourcespipeline-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/resources/pipeline/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsreferenceresourcespipeline><span>Pipeline</span></a></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsreferenceresourcesrunconfiguration-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/resources/runconfiguration/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsreferenceresourcesrunconfiguration><span>RunConfiguration</span></a></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsreferenceresourcesrun-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/resources/run/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsreferenceresourcesrun><span>Run</span></a></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsreferenceresourcesexperiment-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/resources/experiment/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsreferenceresourcesexperiment><span>Experiment</span></a></li></ul></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsreferencerun-completion-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/run-completion/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsreferencerun-completion><span>Run Completion Events</span></a></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsreferenceproviders-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/providers/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsreferenceproviders><span>Using Multiple Providers</span></a></li><li class="td-sidebar-nav__section-title td-sidebar-nav__section without-child" id=m-docsreferencedebugging-li><a href=https://sky-uk.github.io/kfp-operator/docs/reference/debugging/ class="align-left pl-0 td-sidebar-link td-sidebar-link__page" id=m-docsreferencedebugging><span>Debugging</span></a></li></ul></li></ul></li></ul></nav></div></aside><aside class="d-none d-xl-block col-xl-2 td-sidebar-toc d-print-none"><div class="td-page-meta ml-2 pb-1 pt-2 mb-0"><a href=https://github.com/sky-uk/kfp-operator/tree/master/docs-gen/content/en/docs/examples/pipeline_training.md class=td-page-meta--view target=_blank rel=noopener><i class="fa fa-file-alt fa-fw"></i> View page source</a>
Expand Down Expand Up @@ -238,11 +238,6 @@
</span></span><span style=display:flex><span><span style=color:#f92672>spec</span>:
</span></span><span style=display:flex><span> <span style=color:#f92672>nats</span>:
</span></span><span style=display:flex><span> <span style=color:#f92672>run-completion</span>:
</span></span><span style=display:flex><span> <span style=color:#f92672>connectionBackoff</span>:
</span></span><span style=display:flex><span> <span style=color:#f92672>duration</span>: <span style=color:#ae81ff>10s</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>factor</span>: <span style=color:#ae81ff>2</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>jitter</span>: <span style=color:#ae81ff>0.2</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>steps</span>: <span style=color:#ae81ff>5</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>jsonBody</span>: <span style=color:#66d9ef>true</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>subject</span>: <span style=color:#ae81ff>events</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>url</span>: <span style=color:#ae81ff>nats://eventbus-kfp-operator-events-stan-svc.kfp-operator.svc:4222</span>
Expand Down
10 changes: 0 additions & 10 deletions docs/docs/index.xml
Original file line number Diff line number Diff line change
Expand Up @@ -544,11 +544,6 @@ In this example we are updating a serving component with the location of the new
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span>&lt;span style="color:#f92672">spec&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">nats&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">run-completion&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">connectionBackoff&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">duration&lt;/span>: &lt;span style="color:#ae81ff">10s&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">factor&lt;/span>: &lt;span style="color:#ae81ff">2&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">jitter&lt;/span>: &lt;span style="color:#ae81ff">0.2&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">steps&lt;/span>: &lt;span style="color:#ae81ff">5&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">jsonBody&lt;/span>: &lt;span style="color:#66d9ef">true&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">subject&lt;/span>: &lt;span style="color:#ae81ff">events&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">url&lt;/span>: &lt;span style="color:#ae81ff">nats://eventbus-kfp-operator-events-stan-svc.kfp-operator.svc:4222&lt;/span>
Expand Down Expand Up @@ -1184,11 +1179,6 @@ To use it, users can create an Argo-Events &lt;a href="https://argoproj.github.i
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span>&lt;span style="color:#f92672">spec&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">nats&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">run-completion&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">connectionBackoff&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">duration&lt;/span>: &lt;span style="color:#ae81ff">10s&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">factor&lt;/span>: &lt;span style="color:#ae81ff">2&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">jitter&lt;/span>: &lt;span style="color:#ae81ff">0.2&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">steps&lt;/span>: &lt;span style="color:#ae81ff">5&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">jsonBody&lt;/span>: &lt;span style="color:#66d9ef">true&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">subject&lt;/span>: &lt;span style="color:#ae81ff">events&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">url&lt;/span>: &lt;span style="color:#ae81ff">nats://eventbus-kfp-operator-events-stan-svc.kfp-operator.svc:4222&lt;/span>
Expand Down
5 changes: 0 additions & 5 deletions docs/docs/reference/index.xml
Original file line number Diff line number Diff line change
Expand Up @@ -243,11 +243,6 @@ To use it, users can create an Argo-Events &lt;a href="https://argoproj.github.i
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span>&lt;span style="color:#f92672">spec&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">nats&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">run-completion&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">connectionBackoff&lt;/span>:
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">duration&lt;/span>: &lt;span style="color:#ae81ff">10s&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">factor&lt;/span>: &lt;span style="color:#ae81ff">2&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">jitter&lt;/span>: &lt;span style="color:#ae81ff">0.2&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">steps&lt;/span>: &lt;span style="color:#ae81ff">5&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">jsonBody&lt;/span>: &lt;span style="color:#66d9ef">true&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">subject&lt;/span>: &lt;span style="color:#ae81ff">events&lt;/span>
&lt;/span>&lt;/span>&lt;span style="display:flex;">&lt;span> &lt;span style="color:#f92672">url&lt;/span>: &lt;span style="color:#ae81ff">nats://eventbus-kfp-operator-events-stan-svc.kfp-operator.svc:4222&lt;/span>
Expand Down
Loading

0 comments on commit 8b45755

Please sign in to comment.