Skip to content

Update ACN Pipeline for Compliance #3597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 75 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
69f9083
Edit Pipeline Trigger
sheylatrudo Mar 24, 2025
1b363e4
fixup! Edit Pipeline Trigger
sheylatrudo Apr 18, 2025
caeadeb
fixup! Edit Pipeline Trigger
sheylatrudo Apr 18, 2025
93d21b1
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
cd0a6a9
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
1c424af
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
e3ffcf3
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
f944057
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
278d530
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
7b50ef5
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
4336147
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
6a666ab
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
5c917fc
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
f532f03
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
7a51610
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
a85823b
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
38246c4
fixup! Edit Pipeline Trigger
sheylatrudo Apr 19, 2025
e1be764
fixup! Edit Pipeline Trigger
sheylatrudo Apr 20, 2025
d9e64fe
fixup! Edit Pipeline Trigger
sheylatrudo Apr 20, 2025
c9f0ba7
fixup! Edit Pipeline Trigger
sheylatrudo Apr 20, 2025
81fd29d
fixup! Edit Pipeline Trigger
sheylatrudo Apr 20, 2025
ac5b889
fixup! Edit Pipeline Trigger
sheylatrudo Apr 20, 2025
5a1fbda
fixup! Edit Pipeline Trigger
sheylatrudo Apr 20, 2025
5c7633e
fixup! Edit Pipeline Trigger
sheylatrudo Apr 20, 2025
e060f12
fixup! Edit Pipeline Trigger
sheylatrudo Apr 20, 2025
7051c42
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
15f8150
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
c967f65
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
7ce8861
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
3d4a9c6
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
a288a14
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
86c7653
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
4084a4b
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
6c25396
fixup! Edit Pipeline Trigger
sheylatrudo Apr 21, 2025
26b5fcc
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
d5bb07a
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
40b62f6
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
5faca86
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
86b7652
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
49c5dc7
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
cade678
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
dafdca6
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
a70d12b
fixup! Edit Pipeline Trigger
sheylatrudo Apr 22, 2025
4780e09
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
afd7206
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
3d6ab82
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
e81552c
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
5c75529
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
0338ab6
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
672d81b
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
e7a5055
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
b00e865
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
90607d1
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
e2ee305
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
e2c89d9
fixup! Edit Pipeline Trigger
sheylatrudo Apr 23, 2025
172881c
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
a77640c
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
1ad5d8e
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
f795f76
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
5b50d0b
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
707979e
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
7683a3f
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
6291be8
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
b25c25f
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
c7fd52e
fixup! Edit Pipeline Trigger
sheylatrudo Apr 24, 2025
19de055
fixup! Edit Pipeline Trigger
sheylatrudo Apr 25, 2025
8c25dfb
fixup! Edit Pipeline Trigger
sheylatrudo Apr 25, 2025
5150f08
fixup! Edit Pipeline Trigger
sheylatrudo Apr 25, 2025
2af6e22
fixup! Edit Pipeline Trigger
sheylatrudo Apr 25, 2025
63ddcef
fixup! Edit Pipeline Trigger
sheylatrudo Apr 25, 2025
7dbfe1b
fixup! Edit Pipeline Trigger
sheylatrudo Apr 25, 2025
b4841d5
fixup! Edit Pipeline Trigger
sheylatrudo Apr 25, 2025
730b499
ci: E2E work
jpayne3506 Apr 23, 2025
85f9044
Update Variables to Generated Values
sheylatrudo Apr 25, 2025
64e71ee
fixup! Update Variables to Generated Values
sheylatrudo Apr 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 0 additions & 28 deletions .pipelines/OneBranch.NonOfficial.CrossPlat.yaml

This file was deleted.

27 changes: 0 additions & 27 deletions .pipelines/OneBranch.Official.CrossPlat.yaml

This file was deleted.

45 changes: 45 additions & 0 deletions .pipelines/build/binaries.jobs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
parameters:
- name: binaries
type: jobList


jobs:
- ${{ each job_data in parameters.binaries }}:
- ${{ if eq(job_data.templateContext.action, 'build') }}:
- job: binaries_${{ job_data.job }}
displayName: "Build Binary - ${{ job_data.displayName }} -"
strategy: ${{ job_data.strategy }}
pool:
type: linux
${{ if eq(job_data.job, 'linux_arm64') }}:
hostArchitecture: arm64
variables:
ob_outputDirectory: $(Build.ArtifactStagingDirectory)/out
ob_artifactSuffix: _$(artifact)
ob_git_checkout: false
${{ if eq(job_data.job, 'linux_amd64') }}:
LinuxContainerImage: $(LinuxContainerImage2)
ARCH: amd64
OS: linux
${{ elseif eq(job_data.job, 'windows_amd64') }}:
LinuxContainerImage: $(LinuxContainerImage2)
ob_enable_qemu: true
ARCH: amd64
OS: windows
${{ elseif eq(job_data.job, 'linux_arm64') }}:
LinuxContainerImage: $(LinuxContainerImage3)
ob_enable_qemu: true
ARCH: arm64
OS: linux

steps:
- task: DownloadPipelineArtifact@2
inputs:
targetPath: $(Build.SourcesDirectory)/${{ job_data.templateContext.repositoryArtifact }}
artifact: '${{ job_data.templateContext.repositoryArtifact }}'

- template: binary.steps.yaml
parameters:
target: $(name)
os: $(OS)
arch: $(ARCH)
56 changes: 56 additions & 0 deletions .pipelines/build/binary.steps.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
parameters:
- name: target
type: string

- name: os
type: string

- name: arch
type: string


steps:
- task: GoTool@0
inputs:
version: '$(GOVERSION)'

- bash: |
sudo apt-get update -y
if [[ $GOARCH =~ amd64 ]]; then
sudo apt-get install -y llvm clang linux-libc-dev linux-headers-generic libbpf-dev libc6-dev nftables iproute2 gcc-multilib
for dir in /usr/include/x86_64-linux-gnu/*; do
sudo ln -sfn "$dir" /usr/include/$(basename "$dir")
done

elif [[ $GOARCH =~ arm64 ]]; then
sudo apt-get install -y llvm clang linux-libc-dev linux-headers-generic libbpf-dev libc6-dev nftables iproute2 gcc-aarch64-linux-gnu
for dir in /usr/include/aarch64-linux-gnu/*; do
sudo ln -sfn "$dir" /usr/include/$(basename "$dir")
done
fi
displayName: "Install Binary Pre-Reqs"
workingDirectory: $(ACN_DIR)
env:
TARGET: ${{ parameters.target }}
GOOS: ${{ parameters.os }}
GOARCH: ${{ parameters.arch }}

- bash: |
make "$TARGET"
ls -la output
displayName: "Build Binary - ${{ parameters.target }}"
workingDirectory: $(ACN_DIR)
env:
TARGET: ${{ parameters.target }}
GOOS: ${{ parameters.os }}
GOARCH: ${{ parameters.arch }}

- script: |
SOURCE_DIR="./output"
TARGET_DIR="$BUILD_ARTIFACTSTAGINGDIRECTORY"/out
mkdir -p "$TARGET_DIR"
find "$SOURCE_DIR" -name '*.tgz*' -print -exec mv -t "$TARGET_DIR"/ {} +
find "$SOURCE_DIR" -name '*.zip' -print -exec mv -t "$TARGET_DIR"/ {} +
ls -la "$TARGET_DIR"
displayName: "Prepare Artifacts"
workingDirectory: $(ACN_DIR)
22 changes: 22 additions & 0 deletions .pipelines/build/generate-manifest.steps.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
parameters:
- name: platforms
type: object
default: []


steps:
- bash: |
set -e
MANIFEST_DATA=$(echo "$IMAGE_PLATFORM_DATA" | \
jq -r '.[] |
.args = [ (.platform | split("/")[0]), (.platform | split("/")[1]) ] |
.args = [ ("--os " + .args[0] ), ("--arch " + .args[1] ) ] |
{ image: .imageReference, annotate: .args }' | \
jq -rcs)
echo >&2 "##vso[task.setvariable variable=MANIFEST_JSON;isOutput=true]$MANIFEST_DATA"
echo "$MANIFEST_DATA" | jq -r .
displayName: "Populate Image Manifest Data"
name: data
env:
IMAGE_PLATFORM_DATA: '${{ convertToJson(parameters.platforms) }}'

89 changes: 89 additions & 0 deletions .pipelines/build/image.steps.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
parameters:
- name: arch
type: string
default: ""

- name: name
type: string
default: ""

- name: os
type: string
default: ""

- name: build_tag
type: string
default: ""

- name: dockerfile_path
type: string
default: ""

- name: archive_file
type: string
default: '$(name)-$(os)-$(platform)-$(Tag)'

- name: source
type: string
default: drop_setup_env_source

- name: extra_args
type: string
default: ''


- name: default_args
type: object
default:
- "--target $(os) "
- "--platform $(os)/$(arch) "

- name: common_build_args
type: object
default:
- "PLATFORM=$(os)/$(arch) "
- "ARCH=$(arch) "
- "OS=$(os) "
- "VERSION=$(Tag) "


steps:
- task: DownloadPipelineArtifact@2
inputs:
targetPath: $(Build.SourcesDirectory)/dst/${{ parameters.source }}
artifact: '${{ parameters.source }}'

- task: onebranch.pipeline.containercontrol@1
displayName: "Login to ACR"
inputs:
command: login
endpoint: $(ACR_ARM_SERVICE_CONNECTION)

# Build and push the Docker image
- task: onebranch.pipeline.imagebuildinfo@1
displayName: Image Build
retryCountOnTaskFailure: 3
timeoutInMinutes: 30
inputs:
endpoint: $(ACR_ARM_SERVICE_CONNECTION)
registry: $(ACR).azurecr.io
repositoryName: $(os)-$(arch)/${{ parameters.name }}
os: '${{ parameters.os }}'
buildkit: 1
dockerFileRelPath: ${{ parameters.dockerfile_path }}/Dockerfile
dockerFileContextPath: ${{ parameters.source }}
enable_network: true
enable_pull: true
build_tag: ${{ parameters.build_tag }}
enable_acr_push: true

saveImageToPath: images/$(os)-$(arch)/${{ parameters.archive_file }}.tar.gz
#compress: true
#saveMetadataToPath: images/$(os)-$(arch)/metadata/${{ parameters.archive_file }}-metadata.json
#enable_isolated_acr_push: true

# Docker Build Arguments
${{ if parameters.common_build_args }}:
arguments: --build-arg ${{ join('--build-arg ', parameters.common_build_args) }} ${{ parameters.extra_args }} ${{ join(' ', parameters.default_args) }}
${{ else }}:
arguments: ${{ parameters.extra_args }} ${{ join(' ', parameters.default_args) }}
39 changes: 39 additions & 0 deletions .pipelines/build/manifests.jobs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
parameters:
- name: generate
type: jobList


jobs:
- ${{ each job_data in parameters.generate }}:
- job: ${{ job_data.job }}_generate_manifest
displayName: "Generate Image Manifest - ${{ job_data.job }}"
pool:
type: linux
variables:
ob_outputDirectory: $(Build.SourcesDirectory)/out
ob_git_checkout: false
steps:
- template: /.pipelines/build/generate-manifest.steps.yaml
parameters:
platforms: ${{ job_data.templateContext.platforms }}

- job: ${{ job_data.job }}_publish_manifest
displayName: "Publish Image Manifest - ${{ job_data.job }}"
dependsOn:
- ${{ job_data.job }}_generate_manifest
pool:
type: docker
os: linux
vmImage: $(LinuxContainerImage)
variables:
LinuxContainerImage: $(LinuxContainerImage)
ob_outputDirectory: $(Build.SourcesDirectory)/out
ob_git_checkout: false

MANIFEST_JSON: $[ dependencies.${{ job_data.job }}_generate_manifest.outputs['data.MANIFEST_JSON'] ]
steps:
- template: /.pipelines/build/publish-manifest.steps.yaml
parameters:
image_repository: ${{ job_data.templateContext.name }}
image_tag: ${{ job_data.templateContext.image_tag }}
manifest_data: $(MANIFEST_JSON)
Loading
Loading