Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds state artifact fetcher #55

Merged
merged 23 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b0304ff
adding state artifact fetcher to fetch the state from harbor
Mehul-Kumar-27 Sep 23, 2024
8a5ba11
completing the url fetcher
Mehul-Kumar-27 Sep 25, 2024
c291e6a
adding schedulers and process to satellite
Mehul-Kumar-27 Sep 29, 2024
fe299f8
adding simple notifier to fetch state process
Mehul-Kumar-27 Sep 29, 2024
b6b5dea
added description to the scheduler
Mehul-Kumar-27 Sep 29, 2024
91222d4
coderabbit fixes and changes to fetcher and schedulers
Mehul-Kumar-27 Sep 29, 2024
09419ce
adding new format of the state file
Mehul-Kumar-27 Oct 2, 2024
c60f7a4
adding config to process new state artifact file
Mehul-Kumar-27 Oct 8, 2024
18b5633
Merge branch 'main' into satellite-state-fetcher
Mehul-Kumar-27 Oct 8, 2024
dec1ba0
coderabbit review
Mehul-Kumar-27 Oct 8, 2024
ef0d82a
added ./zot to gitignore
Mehul-Kumar-27 Oct 8, 2024
74fc4b9
fixing the replication process
Mehul-Kumar-27 Oct 10, 2024
3d0e209
fixing the replication and deletion process
Mehul-Kumar-27 Oct 12, 2024
6e54a14
fixing paning while removing the null tags
Mehul-Kumar-27 Oct 13, 2024
8904e99
using repository name instead of the image name while uploading the i…
Mehul-Kumar-27 Oct 16, 2024
a08405d
fixes
Mehul-Kumar-27 Oct 29, 2024
f754085
moving from toml config to json config
Mehul-Kumar-27 Nov 4, 2024
abb1a3e
making config.json work with the replicator
Mehul-Kumar-27 Nov 4, 2024
a641df3
avoid printing confedential information in log
Mehul-Kumar-27 Nov 4, 2024
2f11137
coderabbit fixes
Mehul-Kumar-27 Nov 4, 2024
5ca08c7
dagger version
Mehul-Kumar-27 Nov 19, 2024
9c6d293
Merge branch 'main' into satellite-state-fetcher
Mehul-Kumar-27 Nov 21, 2024
9782e7e
replication fix
Mehul-Kumar-27 Nov 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@ ZOT_URL="127.0.0.1:8585"
TOKEN=""
ENV=dev
USE_UNSECURE=true
GROUP_NAME=satellite-test-group-state
STATE_ARTIFACT_NAME=state
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,5 @@ dist/
zot/cache.db
secrets.txt
__debug_bin1949266242

/zot
1 change: 0 additions & 1 deletion ci/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@ func (m *HarborSatellite) Service(


// builds given component from source

func (m *HarborSatellite) build(source *dagger.Directory, component string) *dagger.Directory {
fmt.Printf("Building %s\n", component)

Expand Down
5 changes: 4 additions & 1 deletion config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ own_registry_adr = "127.0.0.1"
own_registry_port = "8585"

# URL of remote registry OR local file path
url_or_file = "https://demo.goharbor.io/v2/myproject/album-server"
# url_or_file = "https://demo.goharbor.io/v2/myproject/album-server"
url_or_file = "https://registry.bupd.xyz"
## for testing for local file
# url_or_file = "./image-list/images.json"

# Default path for Zot registry config.json
zotConfigPath = "./registry/config.json"
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,7 @@ require (
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/robfig/cron/v3 v3.0.1
github.com/rubenv/sql-migrate v1.5.2 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sagikazarmark/locafero v0.4.0 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1289,6 +1289,8 @@ github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qq
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
Expand Down
33 changes: 21 additions & 12 deletions image-list/images.json
Original file line number Diff line number Diff line change
@@ -1,16 +1,25 @@
{
"registryUrl": "https://demo.goharbor.io/v2/",
"repositories": [
"registry": "Satellite",
Mehul-Kumar-27 marked this conversation as resolved.
Show resolved Hide resolved
"artifacts": [
{
"repository": "myproject",
"images": [
{
"name": "album-server@sha256:39879890008f12c25ea14125aa8e9ec8ef3e167f0b0ed88057e955a8fa32c430"
},
{
"name": "album-server:busybox"
}
]
"repository": "satellite-test-group-state/alpine",
"tag": [
"latest"
],
"labels": null,
"type": "IMAGE",
"digest": "sha256:9cee2b382fe2412cd77d5d437d15a93da8de373813621f2e4d406e3df0cf0e7c",
"deleted": false
},
{
"repository": "satellite-test-group-state/postgres",
"tag": [
"latest"
],
"labels": null,
"type": "IMAGE",
"digest": "sha256:dde924f70bc972261013327c480adf402ea71487b5750e40569a0b74fa90c74a",
"deleted": false
}
]
}
}
83 changes: 55 additions & 28 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,27 @@ import (
var AppConfig *Config

type Config struct {
log_level string
own_registry bool
own_registry_adr string
own_registry_port string
zot_config_path string
input string
zot_url string
registry string
repository string
user_input string
scheme string
api_version string
image string
harbor_password string
harbor_username string
env string
use_unsecure bool
log_level string
own_registry bool
own_registry_adr string
own_registry_port string
zot_config_path string
input string
zot_url string
registry string
repository string
user_input string
scheme string
api_version string
image string
harbor_password string
harbor_username string
env string
use_unsecure bool
remote_registry_url string
group_name string
state_artifact_name string
state_fetch_period string
}

func GetLogLevel() string {
Expand Down Expand Up @@ -122,6 +126,26 @@ func GetHarborUsername() string {
return AppConfig.harbor_username
}

func SetRemoteRegistryURL(url string) {
AppConfig.remote_registry_url = url
}

func GetRemoteRegistryURL() string {
return AppConfig.remote_registry_url
}

func GetGroupName() string {
return AppConfig.group_name
}

func GetStateArtifactName() string {
return AppConfig.state_artifact_name
}

func GetStateFetchPeriod() string {
return AppConfig.state_fetch_period
}

func LoadConfig() (*Config, error) {
viper.SetConfigName("config")
viper.SetConfigType("toml")
Expand All @@ -142,17 +166,20 @@ func LoadConfig() (*Config, error) {
}

return &Config{
log_level: viper.GetString("log_level"),
own_registry: viper.GetBool("bring_own_registry"),
own_registry_adr: viper.GetString("own_registry_adr"),
own_registry_port: viper.GetString("own_registry_port"),
zot_config_path: viper.GetString("zotConfigPath"),
input: viper.GetString("url_or_file"),
harbor_password: os.Getenv("HARBOR_PASSWORD"),
harbor_username: os.Getenv("HARBOR_USERNAME"),
env: os.Getenv("ENV"),
zot_url: os.Getenv("ZOT_URL"),
use_unsecure: use_unsecure,
log_level: viper.GetString("log_level"),
own_registry: viper.GetBool("bring_own_registry"),
own_registry_adr: viper.GetString("own_registry_adr"),
own_registry_port: viper.GetString("own_registry_port"),
zot_config_path: viper.GetString("zotConfigPath"),
input: viper.GetString("url_or_file"),
harbor_password: os.Getenv("HARBOR_PASSWORD"),
harbor_username: os.Getenv("HARBOR_USERNAME"),
env: os.Getenv("ENV"),
zot_url: os.Getenv("ZOT_URL"),
use_unsecure: use_unsecure,
group_name: os.Getenv("GROUP_NAME"),
state_artifact_name: os.Getenv("STATE_ARTIFACT_NAME"),
state_fetch_period: os.Getenv("STATE_FETCH_PERIOD"),
}, nil
}

Expand Down
1 change: 1 addition & 0 deletions internal/notifier/email_notifier.go
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package notifier
28 changes: 28 additions & 0 deletions internal/notifier/notifier.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package notifier

import (
"context"

"container-registry.com/harbor-satellite/logger"
)

type Notifier interface {
// Notify sends a notification
Notify() error
}

type SimpleNotifier struct{
ctx context.Context
}

func NewSimpleNotifier(ctx context.Context) Notifier {
return &SimpleNotifier{
ctx: ctx,
}
}

func (n *SimpleNotifier) Notify() error {
log := logger.FromContext(n.ctx)
log.Info().Msg("This is a simple notifier")
return nil
}
Mehul-Kumar-27 marked this conversation as resolved.
Show resolved Hide resolved
Loading