[PLINT-469] Add Octopus Deploy integration (#18488)

* Add scaffolding

* add link to readme

* clean up manifest

* validate ci

* validate models and config;

* add labeler

* remove dash

* fix manifest

* clean scaffolding and add base fixtures

* add base code and test

* fix changelog

* update and validate config

* add metric to check

* Add mocks and make true unit tests

* Add support for first level space autodiscovery

* Add support fot project groups discovery

* Add support for projects

* remove spaces discovery and add it as configuration option

* clean up code and tests

* move initializations to check init

* add support for task count metrics

* refactor to reference project_group

* submit project metrics in check method

* Add handle_error wrapper and add more logging

* bump python version

* Add e2e test

* lint

* add http template to config spec

* refactor test files and add caddy e2e

* lint

* fix tests

* Add deployment duration metric

* rename to seconds

* Add queue time metric

* Add deploy success metric

* Add can rerun metric

* add metric for has warnings or errors

* sort readme

* Sarah/add octopus server metrics (#18868)

* Add support for server node metrics

* add server node mocks

* remove task id tag

* update caddyfile

* remove dead code

* update codeowners

* add e2e lab and update config

* Address comments

* Update projects logic and refactor (#19087)

* Fix new deployment logic and support in progress and queued deployments

* sync metadata

* update caddyfile

* fix

* add to completed date

* Refactor

* fixed e2e tests

* fixed instance fixture

* fixed e2e test

* added some e2e asserts

* fixed lint error

* all metrics asserted

* Add test asserting metadata, add test and handle failure of tasks

* store base tags

* Don't report completed and executing metrics if no data

* Add support for octopus server node metrics

* remove unused constants

* validate

* lint

* update e2e

* update metadata

* Use project name and project group name tags

* Use space name instead of space id

* Update log lines

* update test instance

* Update metric description

---------

Co-authored-by: Jose Manuel Almaza <[email protected]>

* Make assertions more specific

* Add support for events (#19176)

* [PLINT-484]  Add support for deployment logs (#19152)

* Add support for deployment logs

* Add server node tag

* add e2e_metrics

* remove completed_time

* Address comments

* add extra assert

* [PLINT-469] Add support for pagination in octopus integration (#19228)

* Add support for pagination

* update caddyfile and remove unneeded args

* Apply suggestions

* update config docs (#19243)

* [PLINT-543] Add support for environment metrics and tagging by environment, deployment, and release (#19192)

* Add support for environment metrics

* Add config for autodiscovery

* Add support for environments via discovery

* Add environment, release, and deployment tags

* Add caching for environments, releases, and deployments

* add to caddyfile

* remove env fallback

* change environments cache to dictionary

* use tuple for deployment

* Add pagination for environments

* update caddyfile

* update environments config

* sync models

* Filter by environment

* only collect deploy tasks

* update caddyfile

* Add support for waiting, queued, executing metrics (#19291)

* support unified service tagging: (#19342)

* fix headers for new year

* [PLINT-542] Add machines metrics  (#19285)

* Add support for machine metrics

* add role tags

* change to startswith

* validate metadata

* Address comments

* Add octopus server tag to all metrics, events, and logs  (#19318)

* Add support for machine metrics

* add role tags

* change to startswith

* validate metadata

* Address comments

* Add octopus server tag

* Add collect_events option

* Update docs

* dont use future tense in config

---------

Co-authored-by: Jose Manuel Almaza <[email protected]>

.codecov.yml

@@ -454,6 +454,10 @@ coverage:
         target: 75
         flags:
         - nvidia_triton
+      Octopus_Deploy:
+        target: 75
+        flags:
+        - octopus_deploy
       OpenLDAP:
         target: 75
         flags:
@@ -1338,6 +1342,11 @@ flags:
     paths:
     - nvidia_triton/datadog_checks/nvidia_triton
     - nvidia_triton/tests
+  octopus_deploy:
+    carryforward: true
+    paths:
+    - octopus_deploy/datadog_checks/octopus_deploy
+    - octopus_deploy/tests
   openldap:
     carryforward: true
     paths:

.github/CODEOWNERS

@@ -145,6 +145,9 @@ manifest.json         @DataDog/documentation @DataDog/agent-integrations
 /openstack_controller/                  @DataDog/platform-integrations @DataDog/agent-integrations
 /openstack_controller/manifest.json     @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
 /openstack_controller/*.md              @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
+/octopus_deploy/                        @DataDog/platform-integrations @DataDog/agent-integrations
+/octopus_deploy/manifest.json           @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
+/octopus_deploy/*.md                    @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
 /teleport/                              @DataDog/platform-integrations @DataDog/agent-integrations
 /teleport/manifest.json                 @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation
 /teleport/*.md                          @DataDog/platform-integrations @DataDog/agent-integrations @DataDog/documentation

.github/workflows/config/labeler.yml

@@ -391,6 +391,8 @@ integration/nvidia_nim:
 - nvidia_nim/**/*
 integration/nvidia_triton:
 - nvidia_triton/**/*
+integration/octopus_deploy:
+- octopus_deploy/**/*
 integration/oke:
 - oke/**/*
 integration/oom_kill:

.github/workflows/test-all.yml

@@ -2754,6 +2754,26 @@ jobs:
       minimum-base-package: ${{ inputs.minimum-base-package }}
       pytest-args: ${{ inputs.pytest-args }}
     secrets: inherit
+  jc2a0c60:
+    uses: ./.github/workflows/test-target.yml
+    with:
+      job-name: Octopus Deploy
+      target: octopus_deploy
+      platform: linux
+      runner: '["ubuntu-22.04"]'
+      repo: "${{ inputs.repo }}"
+      python-version: "${{ inputs.python-version }}"
+      standard: ${{ inputs.standard }}
+      latest: ${{ inputs.latest }}
+      agent-image: "${{ inputs.agent-image }}"
+      agent-image-py2: "${{ inputs.agent-image-py2 }}"
+      agent-image-windows: "${{ inputs.agent-image-windows }}"
+      agent-image-windows-py2: "${{ inputs.agent-image-windows-py2 }}"
+      test-py2: ${{ inputs.test-py2 }}
+      test-py3: ${{ inputs.test-py3 }}
+      minimum-base-package: ${{ inputs.minimum-base-package }}
+      pytest-args: ${{ inputs.pytest-args }}
+    secrets: inherit
   j2cf0a0a:
     uses: ./.github/workflows/test-target.yml
     with:

octopus_deploy/CHANGELOG.md

@@ -0,0 +1,4 @@
+# CHANGELOG - Octopus Deploy
+
+<!-- towncrier release notes start -->
+

octopus_deploy/README.md

@@ -0,0 +1,63 @@
+# Agent Check: Octopus Deploy
+
+## Overview
+
+This check monitors your [Octopus Deploy][1] deployments through the Datadog Agent. Track information such as average deployment time per Environment, and deployment failure rate for a Project.
+
+## Setup
+
+Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the [Autodiscovery Integration Templates][3] for guidance on applying these instructions.
+
+### Installation
+
+The Octopus Deploy check is included in the [Datadog Agent][2] package.
+No additional installation is needed on your server.
+
+### Configuration
+
+1. Create an [API key][10] on your Octopus Server.
+
+2. Edit the `octopus_deploy.d/conf.yaml` file in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your `octopus_deploy` performance data. See the [sample `octopus_deploy.d/conf.yaml`][4] for all available configuration options. Limit the amount of projects you collect data for by using the `projects` configuration options:
+
+    ```
+    projects:
+        limit: 10
+        include:
+        - 'project.*'
+    ```
+
+3. [Restart the Agent][5].
+
+### Validation
+
+[Run the Agent's status subcommand][6] and look for `octopus_deploy` under the Checks section.
+
+## Data Collected
+
+### Metrics
+
+See [metadata.csv][7] for a list of metrics provided by this integration.
+
+### Events
+
+The Octopus Deploy integration does not include any events.
+
+### Service Checks
+
+The Octopus Deploy integration does not include any service checks.
+
+## Troubleshooting
+
+Need help? Contact [Datadog support][9].
+
+
+[1]: https://octopus.com/
+[2]: https://app.datadoghq.com/account/settings/agent/latest
+[3]: https://docs.datadoghq.com/agent/kubernetes/integrations/
+[4]: https://github.com/DataDog/integrations-core/blob/master/octopus_deploy/datadog_checks/octopus_deploy/data/conf.yaml.example
+[5]: https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent
+[6]: https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information
+[7]: https://github.com/DataDog/integrations-core/blob/master/octopus_deploy/metadata.csv
+[8]: https://github.com/DataDog/integrations-core/blob/master/octopus_deploy/assets/service_checks.json
+[9]: https://docs.datadoghq.com/help/
+[10]: https://octopus.com/docs/octopus-rest-api/how-to-create-an-api-key