Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: microsoft/MLOpsPython
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 93791
Choose a base ref
...
head repository: microsoft/MLOpsPython
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Nov 8, 2019

  1. enable App_Insight (#86)

    allaabdella2 authored and eedorenko committed Nov 8, 2019
    Copy the full SHA
    eb89654 View commit details

Commits on Nov 11, 2019

  1. update conda dependencies (#93)

    * update dependencies
    
    * cleanup spaces
    dtzar authored and eedorenko committed Nov 11, 2019
    Copy the full SHA
    fcc6fde View commit details

Commits on Nov 12, 2019

  1. Upgrade build train CI pipeline to multi-stage (#90)

    * add staged pipeline
    
    * remove release id
    
    * remove train in release pipeline
    
    * revert to BASE_NAME vars
    
    * Move train trigger to new stage
    
    * cleanup register comments
    
    * add conditional for triggering train pipe
    
    * update doc steps
    
    * string vs boolean
    
    * var to boolean
    
    * set to false
    
    * try with true
    
    * cleanup images
    
    * Use Coalesce so override works
    
    * add back build artifacts
    
    * address feedback
    
    * include code/scoring path for ci
    dtzar authored and eedorenko committed Nov 12, 2019
    Copy the full SHA
    d5864f3 View commit details

Commits on Nov 15, 2019

  1. Update conda_dependencies (#95)

    Adds back old PyPi package to fix the crash loop container for ACI/AKS deployments.
    dtzar authored and eedorenko committed Nov 15, 2019
    Copy the full SHA
    5372bb0 View commit details

Commits on Nov 20, 2019

  1. cleaner variables (#99)

    dariuszparys authored and dtzar committed Nov 20, 2019
    Copy the full SHA
    1eb4a42 View commit details

Commits on Nov 21, 2019

  1. Copy the full SHA
    969e6ca View commit details

Commits on Nov 22, 2019

  1. Copy the full SHA
    6c56eee View commit details

Commits on Nov 26, 2019

  1. Copy the full SHA
    d4a7633 View commit details
  2. Add code comments (#114)

    dtzar authored and eedorenko committed Nov 26, 2019
    Copy the full SHA
    9bee73f View commit details
  3. Cleanup getting started (#112)

    dtzar authored and eedorenko committed Nov 26, 2019
    Copy the full SHA
    50536c5 View commit details

Commits on Nov 28, 2019

  1. bump pip requirements versions (#104)

    dtzar authored and eedorenko committed Nov 28, 2019
    Copy the full SHA
    d360198 View commit details

Commits on Dec 6, 2019

  1. Copy the full SHA
    11716e8 View commit details

Commits on Dec 9, 2019

  1. Copy the full SHA
    2a22e75 View commit details

Commits on Dec 12, 2019

  1. Canary deployment (#125)

    eedorenko authored and dtzar committed Dec 12, 2019
    Copy the full SHA
    8d1be26 View commit details

Commits on Dec 16, 2019

  1. Copy the full SHA
    6c2f03f View commit details

Commits on Dec 26, 2019

  1. Copy the full SHA
    d79338f View commit details

Commits on Jan 7, 2020

  1. Copy the full SHA
    dc88c72 View commit details

Commits on Jan 9, 2020

  1. Fixed assignment bug for build_uri (#133)

    If the builduri_base is None, the build_uri would never been instantiated causing an error when calling register_aml_model(model_name, exp, run_id, build_id, build_uri). By adding the else statement, this is prevented.
    starlord-daniel authored and eedorenko committed Jan 9, 2020
    Copy the full SHA
    cb27dd6 View commit details

Commits on Jan 13, 2020

  1. Use training data from a managed datasource (#134)

    * Data from managed datastore
    
    * merge with unassigned variable fix
    
    * bugfix
    
    * typo
    
    * linting
    
    * linting
    
    * linting
    
    * added a link to az cli
    
    * doc update
    
    * reregistering a dataset
    
    * typo
    
    * rephrasing
    
    * rephrasing
    
    * auth enabled
    
    * revert auth enabled
    eedorenko authored and jotaylo committed Jan 13, 2020
    Copy the full SHA
    da60ab1 View commit details

Commits on Jan 22, 2020

  1. Copy the full SHA
    e4a931a View commit details
  2. Added notebooks for training and scoring a Diabetes Ridge regression …

    …model (#145)
    
    * Added notebooks for training and scoring a Ridge regression model with the Diabetes dataset
    
    * Fixed joblib import and remove request headers
    
    Co-authored-by: David Tesar <david.tesar@microsoft.com>
    2 people authored and sudivate committed Jan 22, 2020
    Copy the full SHA
    faeeef9 View commit details

Commits on Jan 23, 2020

  1. Copy the full SHA
    6cacc3c View commit details

Commits on Jan 25, 2020

  1. Change folder structure to support several training pipelines and mod…

    …els (#147)
    
    * change folder structure
    
    * rename variables file
    
    * fix linting
    
    * documentation fix
    
    * return model name due to a bug in scoring
    
    * make PR build working
    sbaidachni authored Jan 25, 2020
    Copy the full SHA
    65c0793 View commit details

Commits on Jan 28, 2020

  1. 1
    Copy the full SHA
    45afe29 View commit details

Commits on Jan 29, 2020

  1. added toggle for evaluation step (#152)

      # These are the default values set in ml_service\util\env_variables.py. Uncomment and override if desired.
    RUN_EVALUATION=true
    ALLOW_RUN_CANCEL=true
    sudivate authored Jan 29, 2020
    Copy the full SHA
    b3d598e View commit details
  2. always run the evaluation step (#154)

    Set the ci-build-train pipeline to always run the evaluation step (even if the pipeline is cancelled). This will cause the Azure Pipeline to log the reason for cancellation rather than just "silently failing".
    jotaylo authored Jan 29, 2020
    Copy the full SHA
    ecad47f View commit details

Commits on Jan 30, 2020

  1. Typo (#161)

    eedorenko authored Jan 30, 2020
    Copy the full SHA
    06c4ac7 View commit details
  2. Copy the full SHA
    ff95f0d View commit details
  3. Copy the full SHA
    a24870a View commit details
  4. Copy the full SHA
    57ef170 View commit details

Commits on Jan 31, 2020

  1. Tox + code coverage (#150)

    algattik authored Jan 31, 2020
    Copy the full SHA
    e0e6d22 View commit details
  2. Update diabetes_regression_build_train_pipeline_with_r.py (#167)

    Co-authored-by: Eugene Fedorenko <eugene.fedor@gmail.com>
    algattik and eedorenko authored Jan 31, 2020
    Copy the full SHA
    0b4f233 View commit details
  3. Manage environments in conda YAML files (#158)

    * .
    
    * .
    
    * Update code_test.py
    
    * .
    
    * Update Dockerfile
    
    * Do not use conda-merge
    
    * Move all 3 conda files to a single dir
    * Do not use conda-merge
    * Pin package versions
    
    * PR review fixes
    
    * Update Dockerfile
    
    * PR review fixes
    
    * Update training_dependencies.yml
    
    * Update code_test.py
    algattik authored Jan 31, 2020
    Copy the full SHA
    962778c View commit details
  4. Copy the full SHA
    8ae6701 View commit details

Commits on Feb 1, 2020

  1. Copy the full SHA
    312dfdf View commit details

Commits on Feb 4, 2020

  1. Update train.py (#174)

    algattik authored Feb 4, 2020
    Copy the full SHA
    424c317 View commit details
  2. Fail on R error (#173)

    algattik authored Feb 4, 2020
    Copy the full SHA
    9e3b007 View commit details
  3. Manage AML environments from conda YAML files (#168)

    * PR review fixes
    
    * Update Dockerfile
    
    * .
    
    * Update deploy_web_service.py
    
    * Update deploy_web_service.py
    
    * PR review fixes
    
    * PR review fixes
    
    * PR review fixes
    
    * Update training_dependencies.yml
    
    * Linting fixes
    
    * Fixed merge
    
    * Update code_test.py
    
    * Simplified environment management, restored deploy task
    
    * Simplified environment management, restored deploy task
    
    * Fixed doc for new file path
    
    * Added comments
    
    Co-authored-by: Sushant Divate <sudivate@microsoft.com>
    algattik and sudivate authored Feb 4, 2020
    Copy the full SHA
    8174d98 View commit details
  4. Copy the full SHA
    275ba5c View commit details

Commits on Feb 6, 2020

  1. Copy the full SHA
    08bc547 View commit details
  2. Copy the full SHA
    a39a453 View commit details

Commits on Feb 7, 2020

  1. Copy the full SHA
    fb1d951 View commit details

Commits on Feb 10, 2020

  1. Fixed and rearranged unit tests (#184)

    * Fixed test_train_model test in code_test.py
    * Renamed code_test.py to train_test.py
    * Moved train_test.py under diabetes_regression directory
    * Moved data_test.py under data directory
    * Added lint and unit test tasks to azdo-base-pipeline.yml
    * Deleted lint-and-test.sh and tox.ini
    * Removed test/unit directory
    
    
    Co-authored-by: David Tesar <david.tesar@microsoft.com>
    bjcmit and dtzar authored Feb 10, 2020
    Copy the full SHA
    6efc12c View commit details
  2. Clarifying comments (#187)

    eedorenko authored Feb 10, 2020
    Copy the full SHA
    e187158 View commit details

Commits on Feb 11, 2020

  1. Add Bootstrap code (#186)

    -bootstrap script
    -directory structure
    sudivate authored Feb 11, 2020
    Copy the full SHA
    977593a View commit details

Commits on Feb 12, 2020

  1. Copy the full SHA
    22d2526 View commit details
  2. bootstrap (#190)

    eedorenko authored Feb 12, 2020
    Copy the full SHA
    e108e07 View commit details

Commits on Feb 14, 2020

  1. Modified to use dataset and pipelinedata (#193)

    * Modified to use dataset and pipelinedata
    
    * Create diabetes dataset if no dataset specified
    
    * Reverted null build_id check
    
    * Added MSE tag
    
    * Reverted logic for NULL Build ID
    
    * Force new model if previous onehas no metric tag
    
    * Removed unused DATA variables
    
    * Removed unused DATA vars
    
    * Updated workaround for non-metric-tagged models
    
    * tidied up code
    GraemeMalcolm authored Feb 14, 2020
    Copy the full SHA
    88636df View commit details

Commits on Feb 20, 2020

  1. Copy the full SHA
    b97140c View commit details
  2. Getting started refactoring (#197)

    * State explicitly the flow order (Template as it is with Diabetes-> Bootstrap with your project code)
    * Fix “multistage pipeline structure” explanation
    * Clarify the R approach. It has only the model training. There is no evaluation/registration
    * Fix the confusion with AzureResourceConnection and WORKSPACE_SVC_CONNECTION
    * Explain using of a Docker image in the pipeline
    * Link to bring-your-own-code (Bryan’s) article
    * Fix broken links (e.g. diabetes_regression-ci-build-train.yml)
    * Provide ML Service connection screenshot
    * Explain explicit "diabetes" names. (e.g. The repo contains a sample “diabetes regression” project so here and there all names contain "diabetes")
    * Clarify the folder structure (Common folders (e.g. .pipelines, ml_service) vs Project folders (e.g. diabetes_regression))
    eedorenko authored Feb 20, 2020
    Copy the full SHA
    171a27f View commit details
Showing with 6,023 additions and 1,583 deletions.
  1. +58 −29 .env.example
  2. +4 −0 .gitignore
  3. +168 −0 .pipelines/abtest.yml
  4. +0 −26 .pipelines/azdo-base-pipeline.yml
  5. +0 −64 .pipelines/azdo-ci-build-train.yml
  6. +0 −18 .pipelines/azdo-pr-build-train.yml
  7. +27 −0 .pipelines/code-quality-template.yml
  8. +89 −0 .pipelines/diabetes_regression-batchscoring-ci.yml
  9. +161 −0 .pipelines/diabetes_regression-cd.yml
  10. +38 −0 .pipelines/diabetes_regression-ci-image.yml
  11. +97 −0 .pipelines/diabetes_regression-ci.yml
  12. +48 −0 .pipelines/diabetes_regression-get-model-id-artifact-template.yml
  13. +42 −0 .pipelines/diabetes_regression-package-model-template.yml
  14. +29 −0 .pipelines/diabetes_regression-publish-model-artifact-template.yml
  15. +129 −0 .pipelines/diabetes_regression-variables-template.yml
  16. +10 −0 .pipelines/helm-install-template.yml
  17. +20 −0 .pipelines/helm-upgrade-template.yml
  18. +24 −0 .pipelines/pr.yml
  19. +19 −44 README.md
  20. +3 −0 bootstrap/README.md
  21. +155 −0 bootstrap/bootstrap.py
  22. +5 −0 charts/abtest-istio/Chart.yaml
  23. +60 −0 charts/abtest-istio/templates/istio-canary.yaml
  24. +15 −0 charts/abtest-istio/values.yaml
  25. +5 −0 charts/abtest-model/Chart.yaml
  26. +30 −0 charts/abtest-model/templates/deployment.yaml
  27. +13 −0 charts/abtest-model/templates/service.yaml
  28. +13 −0 charts/abtest-model/values.yaml
  29. +8 −0 charts/load_test.sh
  30. +0 −113 code/evaluate/evaluate_model.py
  31. +0 −111 code/register/register_model.py
  32. +0 −3 code/training/R/train_with_r.py
  33. +0 −102 code/training/train.py
  34. +3 −0 data/README.md
  35. +5 −5 {tests/unit → data}/data_test.py
  36. +10 −0 diabetes_regression/.amlignore
  37. +29 −0 diabetes_regression/ci_dependencies.yml
  38. +15 −30 {code/scoring → diabetes_regression}/conda_dependencies.yml
  39. +31 −0 diabetes_regression/conda_dependencies_scorecopy.yml
  40. +32 −0 diabetes_regression/conda_dependencies_scoring.yml
  41. +154 −0 diabetes_regression/evaluate/evaluate_model.py
  42. +18 −0 diabetes_regression/parameters.json
  43. +214 −0 diabetes_regression/register/register_model.py
  44. +1 −2 {code → diabetes_regression}/scoring/deployment_config_aci.yml
  45. +4 −4 {code → diabetes_regression}/scoring/deployment_config_aks.yml
  46. +1 −1 {code → diabetes_regression}/scoring/inference_config.yml
  47. +157 −0 diabetes_regression/scoring/parallel_batchscore.py
  48. +91 −0 diabetes_regression/scoring/parallel_batchscore_copyoutput.py
  49. +90 −0 diabetes_regression/scoring/score.py
  50. +6 −0 diabetes_regression/scoring/scoreA.py
  51. +6 −0 diabetes_regression/scoring/scoreB.py
  52. 0 {code → diabetes_regression}/training/R/r_train.r
  53. +3 −0 diabetes_regression/training/R/train_with_r.py
  54. +2 −2 {code → diabetes_regression}/training/R/train_with_r_on_databricks.py
  55. 0 {code → diabetes_regression}/training/R/weight_data.csv
  56. +32 −0 diabetes_regression/training/test_train.py
  57. +59 −32 code/scoring/score.py → diabetes_regression/training/train.py
  58. +176 −0 diabetes_regression/training/train_aml.py
  59. 0 diabetes_regression/util/__init__.py
  60. +79 −0 diabetes_regression/util/model_helper.py
  61. +124 −0 docs/canary_ab_deployment.md
  62. +75 −22 docs/code_description.md
  63. +113 −0 docs/custom_container.md
  64. +124 −0 docs/custom_model.md
  65. +33 −0 docs/development_setup.md
  66. +464 −395 docs/getting_started.md
  67. BIN docs/images/ADO-CD-pipeline-to-webapp.png
  68. BIN docs/images/Architecture_DevOps_AI.png
  69. BIN docs/images/EditPipeline1.png
  70. BIN docs/images/EditPipeline2.png
  71. BIN docs/images/EditPipeline3.png
  72. BIN docs/images/EditPipeline4.png
  73. BIN docs/images/EditPipeline5.png
  74. BIN docs/images/EditPipeline6.png
  75. BIN docs/images/EditPipeline7.png
  76. BIN docs/images/EditPipeline8.png
  77. BIN docs/images/Install_Azure_Pipeline.png
  78. BIN docs/images/aci-in-azure-portal.png
  79. BIN docs/images/agent-specification.png
  80. BIN docs/images/appservice-webapp-deploymentcenter.png
  81. BIN docs/images/artifact-invoke-pipeline.png
  82. BIN docs/images/automate_deploy_model_pipeline.png
  83. BIN docs/images/automate_invoke_training_pipeline.png
  84. BIN docs/images/batch-child-run-scoringstep.png
  85. BIN docs/images/batchscoring-ci-result.png
  86. BIN docs/images/batchscoring-pipeline.png
  87. BIN docs/images/build-createpipeline.png
  88. BIN docs/images/build-createpipeline1.png
  89. BIN docs/images/build-run.png
  90. BIN docs/images/build-selectrepo.png
  91. BIN docs/images/ci-build-logs.png
  92. BIN docs/images/ci-build-pipeline-configure.png
  93. BIN docs/images/container-registry-webapp-image.png
  94. BIN docs/images/create-release.png
  95. BIN docs/images/create-rm-service-connection.png
  96. BIN docs/images/custom-container-variables.png
  97. BIN docs/images/deploy-model.png
  98. BIN docs/images/invoke-training-pipeline.png
  99. BIN docs/images/main-flow.png
  100. BIN docs/images/ml-ws-svc-connection.png
  101. BIN docs/images/model-deploy-configure.png
  102. BIN docs/images/model-deploy-get-artifact-logs.png
  103. BIN docs/images/model-deploy-result.png
  104. BIN docs/images/model-train-register-artifacts.png
  105. BIN docs/images/model-train-register.png
  106. BIN docs/images/multi-stage-aci-aks.png
  107. BIN docs/images/multi-stage-aci.png
  108. BIN docs/images/multi-stage-webapp.png
  109. BIN docs/images/new-build-pipeline.png
  110. BIN docs/images/new-build-pipeline1.png
  111. BIN docs/images/postmane.png
  112. BIN docs/images/release-create.png
  113. BIN docs/images/release-createarelease.png
  114. BIN docs/images/release-deployment-service-conn.png
  115. BIN docs/images/release-deployment.png
  116. BIN docs/images/release-deploymentacr.png
  117. BIN docs/images/release-deploymentcitrigger.png
  118. BIN docs/images/release-deploymentprodagent.png
  119. BIN docs/images/release-deploymentprodtrigger.png
  120. BIN docs/images/release-deploymentqaagent.png
  121. BIN docs/images/release-empty-job.png
  122. BIN docs/images/release-envtask-scriptpath.png
  123. BIN docs/images/release-envtask.png
  124. BIN docs/images/release-import.png
  125. BIN docs/images/release-link-vg.png
  126. BIN docs/images/release-new-pipeline.png
  127. BIN docs/images/release-retrainingagent.png
  128. BIN docs/images/release-retrainingartifact.png
  129. BIN docs/images/release-retrainingartifactsuccess.png
  130. BIN docs/images/release-retrainingpipeline.png
  131. BIN docs/images/release-retrainingtrigger.png
  132. BIN docs/images/release-retrainingtrigger1.png
  133. BIN docs/images/release-retraintask.png
  134. BIN docs/images/release-save-empty.png
  135. BIN docs/images/release-task-createimage.PNG
  136. BIN docs/images/release-webapp-pipeline.PNG
  137. BIN docs/images/release-workingdir.png
  138. BIN docs/images/retrain-pipeline-vg.png
  139. BIN docs/images/run-iac-pipeline.png
  140. BIN docs/images/run_training_pipeline_task.png
  141. BIN docs/images/running-training-pipeline.png
  142. BIN docs/images/scoring_image.png
  143. BIN docs/images/select-iac-pipeline.png
  144. BIN docs/images/service-connection-add.png
  145. BIN docs/images/service-connection.png
  146. BIN docs/images/trained-model.png
  147. BIN docs/images/training-pipeline.png
  148. BIN docs/images/workspace-connection.png
  149. +11 −8 environment_setup/Dockerfile
  150. +41 −16 environment_setup/arm-templates/cloud-environment.json
  151. +10 −5 environment_setup/docker-image-pipeline.yml
  152. +36 −0 environment_setup/iac-create-environment-pipeline-arm.yml
  153. +72 −0 environment_setup/iac-create-environment-pipeline-tf.yml
  154. +0 −36 environment_setup/iac-create-environment.yml
  155. +5 −9 environment_setup/{iac-remove-environment.yml → iac-remove-environment-pipeline.yml}
  156. +4 −4 environment_setup/install_requirements.sh
  157. +0 −8 environment_setup/requirements.txt
  158. +4 −0 environment_setup/tf-templates/backend.tf
  159. +71 −0 environment_setup/tf-templates/main.tf
  160. +353 −0 experimentation/Diabetes Ridge Regression Experimentation Pipeline.ipynb
  161. +211 −0 experimentation/Diabetes Ridge Regression Parameter Experimentation.ipynb
  162. +114 −0 experimentation/Diabetes Ridge Regression Scoring.ipynb
  163. +401 −0 experimentation/Diabetes Ridge Regression Training.ipynb
  164. 0 ml_service/pipelines/__init__.py
  165. +0 −128 ml_service/pipelines/build_train_pipeline.py
  166. +0 −78 ml_service/pipelines/build_train_pipeline_with_r.py
  167. +0 −70 ml_service/pipelines/build_train_pipeline_with_r_on_dbricks.py
  168. +428 −0 ml_service/pipelines/diabetes_regression_build_parallel_batchscore_pipeline.py
  169. +180 −0 ml_service/pipelines/diabetes_regression_build_train_pipeline.py
  170. +63 −0 ml_service/pipelines/diabetes_regression_build_train_pipeline_with_r.py
  171. +55 −0 ml_service/pipelines/diabetes_regression_build_train_pipeline_with_r_on_dbricks.py
  172. +18 −0 ml_service/pipelines/load_sample_data.py
  173. +134 −0 ml_service/pipelines/run_parallel_batchscore_pipeline.py
  174. +47 −36 ml_service/pipelines/run_train_pipeline.py
  175. 0 ml_service/util/__init__.py
  176. +18 −26 ml_service/util/attach_compute.py
  177. +51 −53 ml_service/util/create_scoring_image.py
  178. +17 −0 ml_service/util/create_scoring_image.sh
  179. +126 −0 ml_service/util/env_variables.py
  180. +41 −0 ml_service/util/manage_environment.py
  181. +0 −49 ml_service/util/register_model.py
  182. +91 −0 ml_service/util/smoke_test_scoring_service.py
  183. +0 −29 ml_service/util/workspace.py
  184. +0 −25 tests/unit/code_test.py
87 changes: 58 additions & 29 deletions .env.example
Original file line number Diff line number Diff line change
@@ -5,48 +5,77 @@ TENANT_ID = ''
BASE_NAME = ''
SP_APP_ID = ''
SP_APP_SECRET = ''
RESOURCE_GROUP = 'mlops-RG'

# Mock build/release ID for local testing - update ReleaseID each "release"
# Mock build/release ID for local testing
BUILD_BUILDID = '001'
RELEASE_RELEASEID = '001'

# Azure ML Workspace Variables
EXPERIMENT_NAME = ''
SCRIPT_FOLDER = './'
WORKSPACE_NAME = 'mlops-aml-ws'
EXPERIMENT_NAME = 'mlopspython'

# AML Compute Cluster Config
AML_COMPUTE_CLUSTER_NAME = ''
AML_COMPUTE_CLUSTER_CPU_SKU = ''
AML_CLUSTER_MAX_NODES = ''
AML_CLUSTER_MIN_NODES = ''
AML_ENV_NAME='diabetes_regression_training_env'
AML_ENV_TRAIN_CONDA_DEP_FILE="conda_dependencies.yml"
AML_COMPUTE_CLUSTER_NAME = 'train-cluster'
AML_COMPUTE_CLUSTER_CPU_SKU = 'STANDARD_DS2_V2'
AML_CLUSTER_MAX_NODES = '4'
AML_CLUSTER_MIN_NODES = '0'
AML_CLUSTER_PRIORITY = 'lowpriority'
# Training Config
MODEL_NAME = 'sklearn_regression_model.pkl'
MODEL_NAME = 'diabetes_regression_model.pkl'
MODEL_VERSION = '1'
TRAIN_SCRIPT_PATH = 'training/train.py'
TRAIN_SCRIPT_PATH = 'training/train_aml.py'


# AML Pipeline Config
TRAINING_PIPELINE_NAME = ''
PIPELINE_CONDA_PATH = 'aml_config/conda_dependencies.yml'
TRAINING_PIPELINE_NAME = 'Training Pipeline'
MODEL_PATH = ''
EVALUATE_SCRIPT_PATH = 'evaluate/evaluate_model.py'
REGISTER_SCRIPT_PATH = 'register/register_model.py'
SOURCES_DIR_TRAIN = 'code'

# These are not mandatory for the core workflow
# Remote VM Config
REMOTE_VM_NAME = ''
REMOTE_VM_USERNAME = ''
REMOTE_VM_PASSWORD = ''
REMOTE_VM_IP = ''
# Image config
IMAGE_NAME = ''
IMAGE_DESCRIPTION = ''
IMAGE_VERSION = ''
# ACI Config
ACI_CPU_CORES = ''
ACI_MEM_GB = ''
ACI_DESCRIPTION = ''
SOURCES_DIR_TRAIN = 'diabetes_regression'
DATASET_NAME = 'diabetes_ds'
DATASET_VERSION = 'latest'
# Optional. Set it if you have configured non default datastore to point to your data
DATASTORE_NAME = ''
SCORE_SCRIPT = 'scoring/score.py'

# Optional. Used by a training pipeline with R on Databricks
DB_CLUSTER_ID = ''
DATABRICKS_COMPUTE_NAME = ''

# Optional. Container Image name for image creation
IMAGE_NAME = 'mltrained'

# Run Evaluation Step in AML pipeline
RUN_EVALUATION = 'true'

# Set to true cancels the Azure ML pipeline run when evaluation criteria are not met.
ALLOW_RUN_CANCEL = 'true'

# Flag to allow rebuilding the AML Environment after it was built for the first time. This enables dependency updates from conda_dependencies.yaml.
AML_REBUILD_ENVIRONMENT = 'false'



USE_GPU_FOR_SCORING = "false"
AML_ENV_SCORE_CONDA_DEP_FILE="conda_dependencies_scoring.yml"
AML_ENV_SCORECOPY_CONDA_DEP_FILE="conda_dependencies_scorecopy.yml"
# AML Compute Cluster Config for parallel batch scoring
AML_ENV_NAME_SCORING='diabetes_regression_scoring_env'
AML_ENV_NAME_SCORE_COPY='diabetes_regression_score_copy_env'
AML_COMPUTE_CLUSTER_NAME_SCORING = 'score-cluster'
AML_COMPUTE_CLUSTER_CPU_SKU_SCORING = 'STANDARD_DS2_V2'
AML_CLUSTER_MAX_NODES_SCORING = '4'
AML_CLUSTER_MIN_NODES_SCORING = '0'
AML_CLUSTER_PRIORITY_SCORING = 'lowpriority'
AML_REBUILD_ENVIRONMENT_SCORING = 'true'
BATCHSCORE_SCRIPT_PATH = 'scoring/parallel_batchscore.py'
BATCHSCORE_COPY_SCRIPT_PATH = 'scoring/parallel_batchscore_copyoutput.py'


SCORING_DATASTORE_INPUT_CONTAINER = 'input'
SCORING_DATASTORE_INPUT_FILENAME = 'diabetes_scoring_input.csv'
SCORING_DATASTORE_OUTPUT_CONTAINER = 'output'
SCORING_DATASTORE_OUTPUT_FILENAME = 'diabetes_scoring_output.csv'
SCORING_DATASET_NAME = 'diabetes_scoring_ds'
SCORING_PIPELINE_NAME = 'diabetes-scoring-pipeline'
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -24,6 +24,7 @@ wheels/
.installed.cfg
*.egg
MANIFEST
venv/

# PyInstaller
# Usually these files are written by a python script from a template
@@ -46,6 +47,8 @@ coverage.xml
*.cover
.hypothesis/
.pytest_cache/
*-testresults.xml
test-output.xml

# Translations
*.mo
@@ -90,6 +93,7 @@ ENV/
env.bak/
venv.bak/
*.vscode
condaenv.*

# Spyder project settings
.spyderproject
168 changes: 168 additions & 0 deletions .pipelines/abtest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Pipeline for the canary deployment workflow.

resources:
containers:
- container: mlops
image: mcr.microsoft.com/mlops/python:latest

pr: none
trigger:
branches:
include:
- master
paths:
exclude:
- docs/
- environment_setup/
- ml_service/util/create_scoring_image.*
- ml_service/util/smoke_test_scoring_service.py

variables:
- template: diabetes_regression-variables-template.yml
- group: 'devopsforai-aml-vg'
- name: 'helmVersion'
value: 'v3.1.1'
- name: 'helmDownloadURL'
value: 'https://get.helm.sh/helm-$HELM_VERSION-linux-amd64.tar.gz'
- name: 'blueReleaseName'
value: 'model-blue'
- name: 'greenReleaseName'
value: 'model-green'
- name: 'SCORE_SCRIPT'
value: 'scoring/scoreA.py'

stages:
- stage: 'Building'
jobs:
- job: "Build_Scoring_image"
timeoutInMinutes: 0
pool:
vmImage: 'ubuntu-latest'
container: mlops
steps:
- task: AzureCLI@1
inputs:
azureSubscription: '$(WORKSPACE_SVC_CONNECTION)'
scriptLocation: inlineScript
inlineScript: |
set -e
export SUBSCRIPTION_ID=$(az account show --query id -o tsv)
python -m ml_service.util.create_scoring_image --output_image_location_file image_location.txt
displayName: 'Create Scoring Image'
name: 'buildscoringimage'

- publish: image_location.txt
artifact: image_location

- publish: $(System.DefaultWorkingDirectory)/charts
artifact: allcharts

- stage: 'Blue_Staging'
jobs:
- deployment: "Deploy_to_Staging"
timeoutInMinutes: 0
environment: abtestenv
strategy:
runOnce:
deploy:
steps:
- script: |
IMAGE_LOCATION="$(cat $(Pipeline.Workspace)/image_location/image_location.txt)"
echo "##vso[task.setvariable variable=IMAGE_LOCATION]$IMAGE_LOCATION"
displayName: 'Get Image Location'
- template: helm-upgrade-template.yml
parameters:
chartPath: '$(Pipeline.Workspace)/allcharts/abtest-model'
releaseName: $(blueReleaseName)
overrideValues: 'deployment.name=$(blueReleaseName),deployment.bluegreen=blue,deployment.image.name=$(IMAGE_LOCATION)'

- stage: 'Blue_50'
jobs:
- job: 'Blue_Rollout_50'
displayName: 50 50 rollout to blue environment
timeoutInMinutes: 0
steps:
- template: helm-upgrade-template.yml
parameters:
chartPath: '$(System.DefaultWorkingDirectory)/charts/abtest-istio'
releaseName: 'abtest-istio'
overrideValues: 'weight.blue=50,weight.green=50'

- stage: 'Blue_100'
jobs:
- deployment: 'blue_Rollout_100'
timeoutInMinutes: 0
environment: abtestenv
strategy:
runOnce:
deploy:
steps:
- template: helm-upgrade-template.yml
parameters:
chartPath: '$(Pipeline.Workspace)/allcharts/abtest-istio'
releaseName: 'abtest-istio'
overrideValues: 'weight.blue=100,weight.green=0'

- stage: 'Rollback'
dependsOn: 'Blue_100'
condition: failed()
jobs:
- deployment: 'Roll_Back'
displayName: 'Roll Back after failure'
environment: abtestenv
strategy:
runOnce:
deploy:
steps:
- template: helm-upgrade-template.yml
parameters:
chartPath: '$(Pipeline.Workspace)/allcharts/abtest-istio'
releaseName: 'abtest-istio'
overrideValues: 'weight.blue=0,weight.green=100'

- stage: 'Set_Production_Tag'
dependsOn: 'Blue_100'
condition: succeeded()
jobs:
- deployment: 'green_blue_tagging'
timeoutInMinutes: 0
environment: abtestenv
strategy:
runOnce:
deploy:
steps:
- script: |
IMAGE_LOCATION="$(cat $(Pipeline.Workspace)/image_location/image_location.txt)"
echo "##vso[task.setvariable variable=IMAGE_LOCATION]$IMAGE_LOCATION"
displayName: 'Get Image Location'
- template: helm-upgrade-template.yml
parameters:
chartPath: '$(Pipeline.Workspace)/allcharts/abtest-model'
releaseName: $(greenReleaseName)
overrideValues: 'deployment.name=$(greenReleaseName),deployment.bluegreen=green,deployment.image.name=$(IMAGE_LOCATION)'

- stage: 'Green_100'
jobs:
- job: 'Prod_Rollout_100'
timeoutInMinutes: 0
steps:
- template: helm-upgrade-template.yml
parameters:
chartPath: '$(System.DefaultWorkingDirectory)/charts/abtest-istio'
releaseName: 'abtest-istio'
overrideValues: 'weight.blue=0,weight.green=100'

- stage: 'Disable_blue'
condition: always()
jobs:
- job: 'blue_disable'
timeoutInMinutes: 0
steps:
- template: helm-install-template.yml
- task: HelmDeploy@0
displayName: 'helm uninstall blue'
inputs:
connectionType: 'Kubernetes Service Connection'
kubernetesServiceConnection: $(K8S_AB_SERVICE_CONNECTION)
command: delete
arguments: $(blueReleaseName) --namespace $(K8S_AB_NAMESPACE)
26 changes: 0 additions & 26 deletions .pipelines/azdo-base-pipeline.yml

This file was deleted.

64 changes: 0 additions & 64 deletions .pipelines/azdo-ci-build-train.yml

This file was deleted.

Loading