Azure devops #216

souleb · 2023-04-28T15:11:52Z

Description

New azure devops implementation.

It can effectively reconcile azuredevops repositories using the azure zsk. It uses a PAT as ssh keys are not supported.

Commits and Pull requests are supported but teamspermissions are still to be implemented.

cc @nellyk

Test results

https://github.com/fluxcd/go-git-providers/actions/runs/4832004149/jobs/8610205652

nellyk · 2023-05-03T12:00:46Z

go.mod

@@ -15,6 +15,7 @@ require (
 	github.com/hashicorp/go-multierror v1.1.1
 	github.com/hashicorp/go-retryablehttp v0.7.2
 	github.com/ktrysmt/go-bitbucket v0.9.48
+	github.com/microsoft/azure-devops-go-api/azuredevops/v6 v6.0.1


latest release is v7 https://github.com/microsoft/azure-devops-go-api/releases/tag/azuredevops%2Fv7.1.0

we can't use V7 until they fix ContinuationToken field type. It is unconsistent depending on which struct you use.

@souleb is it related to this issue? microsoft/azure-devops-go-api#97 did you find it in a different struct?

i've mentioned this to the relevant team and will let you know when it gets resolved

.github/workflows/e2e.yaml

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]>

Signed-off-by: Soule BA <[email protected]> Signed-off-by: nellyk <[email protected]>

Signed-off-by: nellyk <[email protected]>

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]> Signed-off-by: nellyk <[email protected]>

Signed-off-by: nellyk <[email protected]>

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]> Signed-off-by: nellyk <[email protected]>

Signed-off-by: nellyk <[email protected]>

Signed-off-by: Soule BA <[email protected]>

Also cleaning up and refactoring where needed. Signed-off-by: Soule BA <[email protected]>

codecov-commenter · 2023-05-11T10:43:00Z

Codecov Report

Patch coverage: 60.93% and project coverage change: -0.49 ⚠️

Comparison is base (f57ef5c) 59.48% compared to head (f3bf893) 59.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #216      +/-   ##
==========================================
- Coverage   59.48%   59.00%   -0.49%     
==========================================
  Files          44       88      +44     
  Lines        3019     6891    +3872     
==========================================
+ Hits         1796     4066    +2270     
- Misses       1038     2251    +1213     
- Partials      185      574     +389

Impacted Files	Coverage Δ
azuredevops/client_repository_tree.go	`0.00% <0.00%> (ø)`
azuredevops/client_repository_pullrequest.go	`20.43% <20.43%> (ø)`
azuredevops/auth.go	`51.61% <51.61%> (ø)`
azuredevops/util.go	`64.15% <64.15%> (ø)`
azuredevops/azuredevopsclient.go	`64.70% <64.70%> (ø)`
azuredevops/resource_repository.go	`65.46% <65.46%> (ø)`
azuredevops/client_organizations.go	`72.09% <72.09%> (ø)`
azuredevops/client_repositories.go	`78.20% <78.20%> (ø)`
azuredevops/resource_pullrequest.go	`82.14% <82.14%> (ø)`
azuredevops/resource_organization.go	`85.00% <85.00%> (ø)`
... and 3 more

... and 31 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

makkes · 2023-06-01T16:09:17Z

DEVELOPMENT.md

@@ -86,6 +86,18 @@ make test-e2e-stash
 | Test organization | go-git-provider-testing      | `GIT_PROVIDER_ORGANIZATION` |
 | Test team         | fluxcd-test-team             | `STASH_TEST_TEAM_NAME`      |

+
+### Azure DevOps


Suggested change

### Azure DevOps

#### Azure DevOps

makkes · 2023-06-01T16:15:26Z

gitprovider/types_repository.go

+	// No default value at POST-time.
+	// Added for Azure devops compatibility since it allows updating the name
+	// +optional
+	Name *string `json:"name"`


This type is also returned by e.g. UserRepository.Get. We should adapt the other implementations to properly fill this field. Otherwise it will be nil which feels strange, esp. since the information is available from the API types.

makkes · 2023-06-01T16:17:35Z

azuredevops/integration_suite_test.go

+
+const (
+	azureTokenFile     = "/tmp/azure.token"
+	defaultDescription = "Foo description"


This variable is only ever consumed once and it's not very important to declare it here (as opposed to the variable above). We should just inline it. Alternatively give it a name that make it easier to determine its intended use.

makkes · 2023-06-01T16:18:14Z

azuredevops/integration_suite_test.go

+)
+
+const (
+	azureTokenFile     = "/tmp/azure.token"


Optional: inline this value.

makkes · 2023-06-01T16:21:28Z

azuredevops/integration_suite_test.go

+	testing.Init()
+	rand.Seed(time.Now().UnixNano())


Is it really necessary to do this in an init func as opposed to TestProvider or in the azure Provider spec? I personally avoid init wherever possible as it's really hard to control or discover when navigating the code.

makkes · 2023-06-01T16:33:03Z

azuredevops/client_repositories.go

+		// Create if not found
+		if errors.Is(err, gitprovider.ErrNotFound) {
+			resp, err := c.Create(ctx, ref, req, toCreateOpts(opts...)...)
+			return resp, true, err


Suggested change

return resp, true, err

return resp, true, fmt.Errof("failed creating repository %q: %w", ref, err)

makkes · 2023-06-01T16:33:28Z

azuredevops/client_repository_branch.go

+)
+
+const (
+	emptyObjectPlaceholder = "0000000000000000000000000000000000000000"


This definitely needs an explanatory comment. What is it used for and why do we need it?

makkes · 2023-06-01T16:34:32Z

azuredevops/client_repository_branch.go

+	gitRefPrefix           = "refs/heads/"
+)
+
+// BranchClient implements the gitprovider.BranchClient interface.


Suggested change

// BranchClient implements the gitprovider.BranchClient interface.

I think this is self-explanatory to a Go programmer.

makkes · 2023-06-01T16:36:48Z

azuredevops/client_repository_commit.go

+	repositoryId := c.ref.GetRepository()
+	projectId := c.ref.GetIdentity()
+	ref := gitRefPrefix + branch
+	all := make([]any, 0, len(files))


Suggested change

all := make([]any, 0, len(files))

allFiles := make([]any, 0, len(files))

?

makkes · 2023-06-01T16:37:56Z

azuredevops/client_repository_pullrequest.go

+	requests := make([]gitprovider.PullRequest, len(*prs))
+
+	for idx, pr := range *prs {
+		requests[idx] = newPullRequest(c.clientContext, &pr)


Suggested change

requests[idx] = newPullRequest(c.clientContext, &pr)

pr := pr

requests[idx] = newPullRequest(c.clientContext, &pr)

This is needed because Go reuses variables. You may end up with requests pointing to the same address for all items in the slice.

souleb requested review from makkes and yiannistri April 28, 2023 15:23

nellyk reviewed May 3, 2023

View reviewed changes

makkes reviewed May 5, 2023

View reviewed changes

.github/workflows/e2e.yaml Outdated Show resolved Hide resolved

nellyk and others added 26 commits May 10, 2023 18:28

add intial setup & get list of projects api

3a60af6

add intial setup & get list of projects api

7c0a256

adding a few tests and apis

08d5cc2

add get repository for project method for azure devops

dacd214

add get repository for project method for azure devops

8de69f7

address review comments

addba5d

remove space on auth file

a281f36

add support for Tree API

7ebb4c2

add create branch azuredevops

4e797ed

Update azuredevops/client_repository_tree.go

94bd68a

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]>

add intial setup & get list of projects api

b9a6ffd

add intial setup & get list of projects api

3d7c068

adding a few tests and apis

8f846ba

add get repository for project method for azure devops

25257c1

add get repository for project method for azure devops

a7ffc7c

address review comments

748da57

remove space on auth file

44c99a9

add support for Tree API

73d2cad

add create branch azuredevops

da06ba2

Add ability to create commits azure devops

218aa50

Add list commit azure devops

f1be3fe

Add ability to create pull request

65d258b

review comments

b09453d

Update azuredevops/client_repository_commit.go

aaad427

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]>

review comments

e80740b

review comments update commits function

f957e4a

nellyk and others added 19 commits May 10, 2023 18:31

commit review comments

577f59e

adding e2e tests for org repositories

97c8e28

Signed-off-by: Soule BA <[email protected]> Signed-off-by: nellyk <[email protected]>

added repo list and fixing integration tests

2d93283

Signed-off-by: nellyk <[email protected]>

Add ability to create repo

e3b554c

Signed-off-by: nellyk <[email protected]>

intergration tests

8a3c83a

Signed-off-by: nellyk <[email protected]>

fix integration tests

ab2b039

Signed-off-by: nellyk <[email protected]>

remove space

8aad5eb

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]> Signed-off-by: nellyk <[email protected]>

Update azuredevops/client_repositories.go

df7e404

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]> Signed-off-by: nellyk <[email protected]>

update review comments

545900b

Signed-off-by: nellyk <[email protected]>

update review comments

e2c87e1

Signed-off-by: nellyk <[email protected]>

Update azuredevops/util.go

2b7d1b1

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]> Signed-off-by: nellyk <[email protected]>

Update azuredevops/integration_suite_test.go

a9c2c92

Co-authored-by: souleb <[email protected]> Signed-off-by: Nelly Kiboi <[email protected]> Signed-off-by: nellyk <[email protected]>

update review comments

97994a4

Signed-off-by: nellyk <[email protected]>

Add pull request methods

0ee0913

Signed-off-by: nellyk <[email protected]>

review comments - add pr

c5b9980

Signed-off-by: nellyk <[email protected]>

remove panic comments

975a68c

Signed-off-by: nellyk <[email protected]>

adding missing DeployToken method

2e3b111

Signed-off-by: Soule BA <[email protected]>

adding azure env var to workflow

2256f08

Signed-off-by: Soule BA <[email protected]>

Adding azure devops projects tests

f3bf893

Also cleaning up and refactoring where needed. Signed-off-by: Soule BA <[email protected]>

souleb force-pushed the azure-devops branch from 8ba6009 to f3bf893 Compare May 11, 2023 10:39

makkes reviewed Jun 1, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Azure devops #216

Azure devops #216

souleb commented Apr 28, 2023

nellyk May 3, 2023

souleb May 11, 2023 •

edited

Loading

nellyk May 15, 2023

nellyk May 17, 2023

codecov-commenter commented May 11, 2023 •

edited

Loading

makkes Jun 1, 2023

makkes Jun 1, 2023

makkes Jun 1, 2023

makkes Jun 1, 2023

makkes Jun 1, 2023

makkes Jun 1, 2023

makkes Jun 1, 2023

makkes Jun 1, 2023

makkes Jun 1, 2023

makkes Jun 1, 2023

	return resp, true, err
	return resp, true, fmt.Errof("failed creating repository %q: %w", ref, err)

	all := make([]any, 0, len(files))
	allFiles := make([]any, 0, len(files))

	requests[idx] = newPullRequest(c.clientContext, &pr)
	pr := pr
	requests[idx] = newPullRequest(c.clientContext, &pr)

Azure devops #216

Are you sure you want to change the base?

Azure devops #216

Conversation

souleb commented Apr 28, 2023

Description

Test results

Choose a reason for hiding this comment

souleb May 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented May 11, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

souleb May 11, 2023 •

edited

Loading

codecov-commenter commented May 11, 2023 •

edited

Loading