Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subdirectories in cloned topologies and gitlab support #1664

Merged
merged 40 commits into from
Nov 6, 2023

Conversation

steiler
Copy link
Collaborator

@steiler steiler commented Oct 23, 2023

This PR aims at evolving the "fetching remote repositorys" functionality.

  • It is now possible to reference also files that do not reside in the root of a repository.
  • There is now a notion of different git webservices
  • Gitlab is kind of not that straight forward, since it can also be a selfhosted instance. Right now I have a strings.Contains(url,"gitlab) in place. This might lead to issues, on the other hand if there is no gitlab in the hostname you can simply add a query parameter ?gitlab or if parameters already exist in the URL &gitlab to trigger using the gitlab implementation.
  • The actual git action, is seperated from the parsing of the git webservice URLs
  • Two implementations of the git clone exist. ExecGit and GoGit. Whilst ExecGit is the os.exec() variant that uses the git binary and GoGit uses the go-git golang library.
  • The GoGit implementation on consecutive execution inspects the already downloaded repository, checks that the repository URL for the remote "origin" equals the provided URL and, if the same branch is referenced as on the previous execution, pulls the remote branch, to make sure the repo is up to date. In case a different branch is referenced, it would fetch the branch, create a local branch for the remote one and checkout the branch. If the branch already exists locally it will jsut be checked out and the remote is pulled to also forward it to the remote repo state. [This enhanced processing is not implemented in ExecGo. I would also assume that we abandon ExecGo and just go with GoGit. Kept it for now]

Open topics:

  • How to identify self hosted git deplyments (gitlab/gitea/etc.) github is quite easy, we use the domainname for it. For the others we need a feasible means to determine the git repo kind.
  • No authentication support in place right now.

@steiler steiler added the enhancement New feature or request label Oct 25, 2023
@steiler steiler requested a review from hellt October 26, 2023 12:36
@hellt
Copy link
Member

hellt commented Oct 29, 2023

@steiler can you make unit tests pass?

@steiler
Copy link
Collaborator Author

steiler commented Oct 29, 2023

@hellt unittests are fine now.
take a look at functinality, if thats all fine with you and then I'll also add some smoke tests.

git/github.go Outdated Show resolved Hide resolved
@hellt
Copy link
Member

hellt commented Nov 6, 2023

I have created https://gitlab.com/rdodin/clab-test-repo that matches the contents of the github repo.
To be used in integration tests for this feature.

@hellt hellt changed the title Allow referencing subdir in remote repo Subdirectories in cloned topologies and gitlab support Nov 6, 2023
Copy link

codecov bot commented Nov 6, 2023

Codecov Report

Merging #1664 (4ee5079) into main (29bf491) will increase coverage by 0.24%.
Report is 5 commits behind head on main.
The diff coverage is 52.66%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1664      +/-   ##
==========================================
+ Coverage   51.32%   51.56%   +0.24%     
==========================================
  Files         138      141       +3     
  Lines       13357    13640     +283     
==========================================
+ Hits         6855     7033     +178     
- Misses       5742     5840      +98     
- Partials      760      767       +7     
Files Coverage Δ
utils/file.go 52.76% <100.00%> (+3.57%) ⬆️
git/github.go 91.52% <91.52%> (ø)
git/gitlab.go 91.93% <91.93%> (ø)
git/repo.go 76.00% <76.00%> (ø)
cmd/root.go 80.00% <65.38%> (+28.86%) ⬆️
git/git.go 21.31% <21.31%> (ø)

... and 1 file with indirect coverage changes

@hellt hellt merged commit b388c3e into srl-labs:main Nov 6, 2023
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants