[BUG] support copying to registries with paths with hauler store copy

[ngearhart]

Is this RFE related to an Existing Problem? If so, please describe: This is related to #232 as the implementation for this feature could solve both.

• When trying to push to a registry that requires a prepended path, hauler will fail. For example, JFrog Artifactory's docker/OCI registry requires you push to oci://artifactory.example.com/registry-name (pushing to oci://artifactory.example.com directly will fail).
  The goal would be for hauler to support copying to "path-required" registries.

Describe Proposed Solution(s):

• Add a --prepend-path feature to hauler store copy like hauler store copy --prepend-path registry-name registry://artifactory.example.com such that hauler would prepend that path to all image tags.

Describe Possible Alternatives:

• Allow rewriting in Hauler manifests, like:

apiVersion: content.hauler.cattle.io/v1alpha1
kind: Images
metadata:
  name: test
spec:
  images:
    - name: registry1.dso.mil/ironbank/big-bang/argocd:v2.9.4
      rewrite: registry-name/ironbank/big-bang/argocd:v2.9.4

such that hauler store sync pulls registry1.dso.mil/ironbank/big-bang/argocd:v2.9.4 but saves it as registry-name/ironbank/big-bang/argocd:v2.9.4 in the store.

[zackbradys]

Referenced again in Issue #232

[HoustonDad]

Bumping, another customer hit this today, they are trying to push to artifactory sub-projects

artifactory.domain.com/engineering/devops/images/carbide/

[deiberts86]

Bump as well, had a previous customer ran into the exact same issue.

[dweomer]

As mentioned here, since we are, effectively, expecting the copy target to be in URL/URI format, i.e. scheme://host why not extend such with an optional path segment, i.e. scheme://host/path?

(Yes, the dir scheme is special in that it is a more-compact version of file:///. I would submit that treating targets as URLs to be parsed opens us up for future option of drop-in handling for arbitrary copy-target schemes.)

[ngearhart]

Everyone here is welcome to build my fork if they want in the meantime.
#242
https://github.com/ngearhart/hauler/tree/main

It allows you to add --prepend-path to hauler store copy command to add that optional path segment as described.

[dweomer]

Oh, my suggested solution already Just Works ™️:

[ec2-user@c2s ~]$ hauler store --store=./store.test add image alpine:3.19
2024-06-10 19:06:40 INF adding 'image' [alpine:3.19] to the store
2024-06-10 19:06:44 INF successfully added 'image' [index.docker.io/library/alpine:3.19]
[ec2-user@c2s ~]$ hauler store --store=./store.test copy registry://harbor.ec2.internal/dweomer
2024-06-10 19:06:53 INF library/alpine:3.19
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 pushed blob: sha256:05455a08881ea9cf0e752bc48e61bbd71a34c029bb13df01e40e3e70e0d007bd
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 pushed blob: sha256:30c69795e46bd167df7f6152056f3c885cba4f5b4238e2327c73fb35c226d351
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 pushed blob: sha256:bca4290a96390d7a6fc6f2f9929370d06f8dfcacba591c76e3d5c5044e7f420c
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 pushed blob: sha256:ace17d5d883e9ea5a21138d0608d60aa2376c68f616c55b0b7e73fba6d8556a3
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 pushed blob: sha256:935b61847fc465ff70ecbd3436253a7596a500e649a16014646a99393ccbb661
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 pushed blob: sha256:fda0ff469afd28d9cfbb946e8e0a3c911c591a2691bea62be9187e45a1c50549
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 pushed blob: sha256:0dc2e6c0f9ded2daeca96bbf270526d182d2f4267f5c7610c222c05cad6f6b96
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 harbor.ec2.internal/dweomer/library/alpine@sha256:a0264d60f80df12bc1e6dd98bae6c43debe6667c0ba482711f0d806493467a46: digest: sha256:a0264d60f80df12bc1e6dd98bae6c43debe6667c0ba482711f0d806493467a46 size: 528
2024-06-10 19:06:54 INF 2024/06/10 19:06:54 harbor.ec2.internal/dweomer/library/alpine@sha256:ec299a7ba3c670e38642b0b62a0c779d84b249a3c889757e2b6f841433b4c6fe: digest: sha256:ec299a7ba3c670e38642b0b62a0c779d84b249a3c889757e2b6f841433b4c6fe size: 528
2024-06-10 19:06:55 INF 2024/06/10 19:06:55 harbor.ec2.internal/dweomer/library/alpine@sha256:b229a85166aadbde58e73e03c5e2b9737fb4642ffb2d98ba453adc90d144c1d8: digest: sha256:b229a85166aadbde58e73e03c5e2b9737fb4642ffb2d98ba453adc90d144c1d8 size: 528
2024-06-10 19:06:55 INF 2024/06/10 19:06:55 pushed blob: sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8
2024-06-10 19:06:55 INF 2024/06/10 19:06:55 pushed blob: sha256:4a0759b5afbffdc507fbb4e32b3a139063c3a5c0829f811973850447f98830ae
2024-06-10 19:06:55 INF 2024/06/10 19:06:55 pushed blob: sha256:2d433224a9f8f46c545c8fc4bc82ea382227d892e9f0c704d90ef585542bf497
2024-06-10 19:06:55 INF 2024/06/10 19:06:55 pushed blob: sha256:5b984dd0323cee557fb6a9d8796f4b4414317cf1fb88bb2047d2046ac9447d77
2024-06-10 19:06:55 INF 2024/06/10 19:06:55 pushed blob: sha256:f4968021da4ff8b74325e5aebf0f9448b44becfdd14df80ecba474e43cc92546
2024-06-10 19:06:55 INF 2024/06/10 19:06:55 harbor.ec2.internal/dweomer/library/alpine@sha256:6457d53fb065d6f250e1504b9bc42d5b6c65941d57532c072d929dd0628977d0: digest: sha256:6457d53fb065d6f250e1504b9bc42d5b6c65941d57532c072d929dd0628977d0 size: 528
2024-06-10 19:06:55 INF 2024/06/10 19:06:55 harbor.ec2.internal/dweomer/library/alpine@sha256:15c46ced65c6abed6a27472a7904b04273e9a8091a5627badd6ff016ab073171: digest: sha256:15c46ced65c6abed6a27472a7904b04273e9a8091a5627badd6ff016ab073171 size: 528
2024-06-10 19:06:56 INF 2024/06/10 19:06:56 harbor.ec2.internal/dweomer/library/alpine@sha256:b12b826de1ec8c4237aa09a0287e7be8bd317586f32bf6cd9395ec5dba52a3a2: digest: sha256:b12b826de1ec8c4237aa09a0287e7be8bd317586f32bf6cd9395ec5dba52a3a2 size: 528
2024-06-10 19:06:56 INF 2024/06/10 19:06:56 pushed blob: sha256:eb8fba61d86413beda3240c40c599041e040e658cd8314e38ee15e67ea57d349
2024-06-10 19:06:56 INF 2024/06/10 19:06:56 pushed blob: sha256:8fc740d8c40e45ea330a3f324fe009148dfc1f771bc90254eaf8ff8bbcecfe02
2024-06-10 19:06:56 INF 2024/06/10 19:06:56 harbor.ec2.internal/dweomer/library/alpine@sha256:5d0da60400afb021f2d8dbfec8b7d26457e77eb8825cba90eba84319133f0efe: digest: sha256:5d0da60400afb021f2d8dbfec8b7d26457e77eb8825cba90eba84319133f0efe size: 528
2024-06-10 19:06:56 INF 2024/06/10 19:06:56 harbor.ec2.internal/dweomer/library/alpine:3.19: digest: sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b size: 1638
2024-06-10 19:06:56 INF copied artifacts to [harbor.ec2.internal/dweomer]
[ec2-user@c2s ~]$

[ngearhart]

@dweomer That still does not work for a registry like Artifactory due to the login step. Artifactory is a special case where you must login to the root (artifactory.example.com) but push to the path (artifactory.example.com/repository).

[dweomer]

@dweomer That still does not work for a registry like Artifactory due to the login step. Artifactory is a special case where you must login to the root (artifactory.example.com) but push to the path (artifactory.example.com/repository).

I haven't worked with a private Artifactory. Can you show me the failure(s) you get running my PoC script above, assuming that you've already logged in at the root? Does it also fail if you attempt to login at artifactory.example.com/repository? There is some nuance here as Hauler leverages almost identical code to login as docker login, IIRC, which has baked-in the concept that the actual content/resource root is at a particular location relative to the one you provide on the command line, i.e.:

hauler login artifactory.example.com #implies https://artifactory.example.com/v2, whereas
hauler login artifactory.example.com/repository #implies https://artifactory.example.com/repository/v2

NM, don't try that part. I forget that we are using Crane and it's login does check for RFC 3986 authority compliance (whereas docker login harbor.ec2.internal/dweomer just rolls up to be equivalent to docker login harbor.ec2.internal). Please do provide failures/errors encountered trying out this simple script (ideally, please also provide sanitized ~/.docker/config.json auths):

hauler store --store=./store.test add image docker.io/library/alpine:3.20
hauler login artifactory.example.com -u bob -p haulin
hauler store --store=./store.test copy registry://artifactory.example.com/repository

[ngearhart]

Strangely enough, I am having trouble reproducing. I will work on it.

[zackbradys]

Hey @ngearhart, have you had any more luck trying to reproduce this issue? any new information?

[ngearhart]

@zackbradys I have not, sorry. I'm a bit swamped with other priorities at work right now, but will check in on this as soon as I can.

[mddamato]

Came here for this feature request. Very common to mirror internet registries inside internal registries with a subdirectory requirement of some sort but keeping all other paths identical. It would simplify if i had the option to add a flag like --prepend-path=$DEST_REPO_PREFIX of some kind.

docker.io/rancher/rancher:latest
would translate to

• harbor.com/public/rancher/rancher:latest for example or
• gitlab.com/project/rancher/rancher:latest

It's also helpful when using registries that require the extra step to create the repo before pushing like ECR or Harbor. While possible to overcome in other ways, it's fixed in many environments by simply creating global projects.

[HoustonDad]

This doesn't seem to work when doing a 'store copy' using --username and --password

root@hauler:~# hauler store copy --username perk --password "perkspw" -s rancher-store registry://harbor.domain.com/rancher --insecure
Error: error logging into registry: exit status 1, output: Error: registries must be valid RFC 3986 URI authorities: harbor.domain.com/rancher
main.go:74: error during command execution: registries must be valid RFC 3986 URI authorities: harbor.domain.com/rancher

Usage:
  hauler store copy [flags]

Flags:
  -h, --help              help for copy
      --insecure          Toggle allowing insecure connections when copying to a remote registry
  -p, --password string   Password when copying to an authenticated remote registry
      --plain-http        Toggle allowing plain http connections when copying to a remote registry
  -u, --username string   Username when copying to an authenticated remote registry

Global Flags:
      --cache string       (deprecated flag and currently not used)
  -l, --log-level string    (default "info")
  -s, --store string       Location to create store at (default "store")

2024-08-05 20:36:00 ERR error logging into registry: exit status 1, output: Error: registries must be valid RFC 3986 URI authorities: harbor.domain.com/rancher
main.go:74: error during command execution: registries must be valid RFC 3986 URI authorities: harbor.domain.com/rancher

[ngearhart]

I can confirm that

hauler store load <store>.tar.zst
hauler login https://artifactory.example.com --username ... --password ...
hauler story sync registry://artifactory.example.com/path

Does work for my use case. Therefore I am comfortable closing this, but want others to make the call if they feel it's still needed in some other way.

[zackbradys]

Glad to hear it and thank you for reporting back your results! @ngearhart

[HoustonDad]

I don't think this is ready to be closed.

We have a 'workaround', not a fix. We should either document this officially in our docs or 'fix' it properly

[a1994sc]

This is not closed, as @HoustonDad said it is a workaround....

If you try with a proxy like Nexus OSS, where you don't have the fancy namespace/path prefix for proxying.

Having a rewrite or something is very nice.

[zackbradys]

Hey @HoustonDad / @a1994sc, this issue was regarding problems with hauler store copy to registries with paths and that is doable with hauler without any workarounds...

hauler login registry.example.com -u bob -p haulin
hauler store copy registry://registry.example.com

If you're referencing the ability to rewrite (and/or persist upstream registry information)... that is tracked in Issue #232!

[HoustonDad]

I'm referencing this: #241 (comment)

Our docs advise to use '--username and --password' when doing a store copy. So either our docs need an update, or we need to fix this issue