chore(dev): replace the aqua local registry with the standard registry #21959

suzuki-shunsuke · 2024-12-04T22:05:06Z

Summary

This pull request replaces aqua's local registry with the standard registry.
Now aqua-registry supports all tools you defined in your local registry.

You don't need to manage a local registry.
aqua recommends using the standard registry rather than maintaining a local registry.

https://aquaproj.github.io/docs/develop-registry/

If tools are not private Repositories, we recommend sending a pull request to Standard Registry rather than maintaining them in your custom Registries, because

You can get the support from maintainers

You don't have to maintain Registry yourself

From aqua v2, you have to allow non Standard Registry explicitly for security, but this is a bit bothersome

Your local registry installs tools by cargo install, but some tools provide pre-built binaries.
The standard registry uses them, so users don't need to build tools themselves.

Registry settings:

Change Type

Bug fix
New feature
Non-functional (chore, refactoring, docs)
Performance

Is this a breaking change?

Yes
No

How did you test this PR?

I've confirmed I could install tools on my laptop (M3 Pro).

wasm-pack -V
cargo-deb --help
cross -V
cargo-nextest -V
cargo-deny -V
cargo-msrv -V
dd-rust-license-tool --help

Does this PR include user facing changes?

Yes. Please add a changelog fragment based on our guidelines.
No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

Please read our Vector contributor resources.
If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
run dd-rust-license-tool write to regenerate the license inventory and commit the changes (if any). More details here.

References

Closes #21922

Now aqua-registry supports all tools you defined in your local registry. - aquaproj/aqua-registry#29339 - https://github.com/aquaproj/aqua-registry/releases/tag/v4.265.0 You don't need to manage local registry.

bits-bot · 2024-12-04T22:05:11Z

All committers have signed the CLA.

suzuki-shunsuke · 2024-12-04T22:10:30Z

Unfortunately, it failed to install cargo-deb on my laptop (M3 Pro).
I'm not sure how to fix the issue.

$ cargo-deb --help
INFO[0000] Installing a crate                            aqua_version= env=darwin/arm64 exe_name=cargo-deb package_name=crates.io/cargo-deb package_version=1.41.3 program=aqua registry=standard
    Updating crates.io index
  Installing cargo-deb v1.41.3
    Updating crates.io index
error: failed to compile `cargo-deb v1.41.3`, intermediate artifacts can be found at `/var/folders/fc/1bgyy3_d3x90m_t04qbw5f8m0000gn/T/cargo-installTkLbCJ`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

Caused by:
  failed to select a version for the requirement `cargo_toml = "^0.13.0"`
  candidate versions found which didn't match: 0.20.5, 0.20.4, 0.20.3, ...
  location searched: crates.io index
  required by package `cargo-deb v1.41.3`
  if you are looking for the prerelease package it needs to be specified explicitly
      cargo_toml = { version = "0.19.0-beta.1" }

suzuki-shunsuke · 2024-12-04T22:17:06Z

Unfortunately, it failed to install cargo-deb on my laptop (M3 Pro).

I could solve the issue by updating cargo-deb 1.41.3 to the latest version 2.9.1

-  - name: crates.io/[email protected]
+  - name: crates.io/[email protected]

$ cargo-deb --help
INFO[0000] Installing a crate                            aqua_version= env=darwin/arm64 exe_name=cargo-deb package_name=crates.io/cargo-deb package_version=2.9.1 program=aqua registry=standard
    Updating crates.io index
  Installing cargo-deb v2.9.1
    Updating crates.io index
     Locking 85 packages to latest compatible versions
      Adding linux-raw-sys v0.4.14 (latest: v0.6.5)
      Adding unicode-width v0.1.14 (latest: v0.2.0)
      Adding windows-sys v0.52.0 (latest: v0.59.0)
   Compiling proc-macro2 v1.0.92
   Compiling unicode-ident v1.0.14
   Compiling libc v0.2.167
   Compiling serde v1.0.215
   Compiling memchr v2.7.4
   Compiling crossbeam-utils v0.8.20
   Compiling cfg-if v1.0.0
   Compiling shlex v1.3.0
   Compiling hashbrown v0.15.2
   Compiling pkg-config v0.3.31
   Compiling equivalent v1.0.1
   Compiling regex-syntax v0.8.5
   Compiling log v0.4.22
   Compiling rayon-core v1.12.1
   Compiling winnow v0.6.20
   Compiling cc v1.2.2
   Compiling utf8parse v0.2.2
   Compiling rustix v0.38.41
   Compiling aho-corasick v1.1.3
   Compiling anstyle-parse v0.2.6
   Compiling indexmap v2.7.0
   Compiling crc32fast v1.4.2
   Compiling colorchoice v1.0.3
   Compiling serde_json v1.0.133
   Compiling either v1.13.0
   Compiling is_terminal_polyfill v1.70.1
   Compiling once_cell v1.20.2
   Compiling bitflags v2.6.0
   Compiling adler2 v2.0.0
   Compiling anstyle-query v1.1.2
   Compiling anstyle v1.0.10
   Compiling miniz_oxide v0.8.0
   Compiling anstream v0.6.18
   Compiling crossbeam-epoch v0.9.18
   Compiling quote v1.0.37
   Compiling regex-automata v0.4.9
   Compiling crossbeam-deque v0.8.5
   Compiling ryu v1.0.18
   Compiling syn v2.0.90
   Compiling lzma-sys v0.1.20
   Compiling bumpalo v3.16.0
   Compiling cargo-deb v2.9.1
   Compiling unicode-width v0.1.14
   Compiling fastrand v2.2.0
   Compiling lockfree-object-pool v0.1.6
   Compiling itoa v1.0.14
   Compiling getopts v0.2.21
   Compiling zopfli v0.8.1
   Compiling flate2 v1.0.35
   Compiling itertools v0.13.0
   Compiling errno v0.3.10
   Compiling filetime v0.2.25
   Compiling tar v0.4.43
   Compiling xz2 v0.1.7
   Compiling rayon v1.10.0
   Compiling quick-error v2.0.1
   Compiling elf v0.7.4
   Compiling glob v0.3.1
   Compiling ar v0.9.0
   Compiling tempfile v3.14.0
   Compiling regex v1.11.1
   Compiling serde_derive v1.0.215
   Compiling env_filter v0.1.2
   Compiling env_logger v0.11.5
   Compiling serde_spanned v0.6.8
   Compiling toml_datetime v0.6.8
   Compiling toml_edit v0.22.22
   Compiling toml v0.8.19
   Compiling cargo_toml v0.20.5
    Finished `release` profile [optimized] target(s) in 31.76s
  Installing /Users/shunsukesuzuki/.local/share/aquaproj-aqua/pkgs/cargo/crates.io/cargo-deb/2.9.1/bin/cargo-deb
   Installed package `cargo-deb v2.9.1` (executable `cargo-deb`)
warning: be sure to add `/Users/shunsukesuzuki/.local/share/aquaproj-aqua/pkgs/cargo/crates.io/cargo-deb/2.9.1/bin` to your PATH to be able to run the installed binaries
Usage: cargo deb [options] [-- <cargo build flags>]
https://lib.rs/cargo-deb 2.9.1

        --no-strip      Do not strip debug symbols from the binary
        --strip         Always try to strip debug symbols
        --separate-debug-symbols 
                        Strip debug symbols into a separate .debug file
        --compress-debug-symbols 
                        Apply objcopy --compress-debug-sections
    -o, --output path   Write .deb to this file or directory
    -p, --package name  Select which Cargo workspace package to use
        --install       Immediately install the created deb package
    -q, --quiet         Don't print warnings
    -v, --verbose       Print progress
        --version       Show version of the cargo-deb tool
        --deb-version version
                        Override version string for the package
        --deb-revision num
                        Override revision suffix string for the package
        --maintainer name
                        Override Maintainer field
        --manifest-path ./Cargo.toml
                        Cargo project file location
        --offline       Passed to Cargo
        --locked        Passed to Cargo
        --frozen        Passed to Cargo
        --variant name  Alternative Cargo.toml configuration section to use
        --target triple Rust target for cross-compilation
        --multiarch none|same|foreign
                        Put libs in /usr/lib/$arch-linux-gnu/
        --profile release|<custom>
                        Select which Cargo build profile to use
        --no-build      Assume the project is already built
        --cargo-build subcommand
                        Override cargo build subcommand
        --fast          Use faster compression, which makes a larger deb file
    -Z, --compress-type gz|xz
                        Compress with the given compression format
        --compress-system 
                        Use the corresponding command-line tool for
                        compression
        --rsyncable     Use worse compression, but reduce differences between
                        versions of packages
    -h, --help          Print this help menu

I'm not familiar with Rust, Cargo, and cargo-deny at all, but is it okay to update cargo-deny?

jszwedko

Awesome, thank you @suzuki-shunsuke . Feel free to update cargo-deb in this PR (and cargo-deny if necessary).

jszwedko · 2024-12-04T22:31:41Z

aqua/aqua-policy.yaml

@@ -1,7 +0,0 @@
---


Love to see deleted code 💯

+1 thanks @suzuki-shunsuke

- #21959 (comment)

suzuki-shunsuke · 2024-12-04T22:37:57Z

Awesome, thank you suzuki-shunsuke . Feel free to update cargo-deb in this PR (and cargo-deny if necessary).

Thank you. I've updated cargo-deb. 0126ddd

JFYI. We can update tools by aqua up command, though we don't need to do this in this pull request.

-  - name: nextest-rs/nextest/[email protected]
-  - name: EmbarkStudios/[email protected]
-  - name: foresterre/[email protected]
-  - name: crates.io/[email protected]
+  - name: nextest-rs/nextest/[email protected]
+  - name: EmbarkStudios/[email protected]
+  - name: foresterre/[email protected]
+  - name: crates.io/[email protected]

jszwedko · 2024-12-04T22:38:55Z

Awesome, thank you suzuki-shunsuke . Feel free to update cargo-deb in this PR (and cargo-deny if necessary).

Thank you. I've updated cargo-deb.

JFYI. We can update tools by aqua up command, though we don't need to do this in this pull request.
-  - name: nextest-rs/nextest/[email protected]
-  - name: EmbarkStudios/[email protected]
-  - name: foresterre/[email protected]
-  - name: crates.io/[email protected]
+  - name: nextest-rs/nextest/[email protected]
+  - name: EmbarkStudios/[email protected]
+  - name: foresterre/[email protected]
+  - name: crates.io/[email protected]

Nice! I was thinking about seeing if we could integrate Renovate too, at some point, to automate upgrades.

suzuki-shunsuke · 2024-12-04T22:39:19Z

Oh. https://github.com/vectordotdev/vector/actions/runs/12169533629/job/33942782258?pr=21959

Error: Unknown scope "aqua" found in pull request title "chore(aqua): replace the local registry with the standard registry". Scope must match one of: new source, new transform, new sink, ARC, administration, api, architecture, auth, buffers, ci, cli, codecs, compression, config, core, data model, delivery, deployment, deps, dev, durability, enriching, enterprise, exceptions, external, external docs, filtering, healthchecks, internal, internal docs, logs, metrics, networking, observability, parsing, performance, platforms, privacy, processing, releasing, reliability, reload, replay, schemas, security, setup, shutdown, sinks, soak tests, sources, startup, templating, tests, topology, traces, transforms, unit tests, vrl, amazon-linux platform, apt platform, arm platform, arm64 platform, centos platform, debian platform, docker platform, dpkg platform, helm platform, heroku platform, homebrew platform, kubernetes platform, macos platform, msi platform, nix platform, nixos platform, raspbian platform, rhel platform, rpm platform, ubuntu platform, windows platform, x86_64 platform, yum platform, service providers, aws service, azure service, confluent service, datadog service, elastic service, gcp service, grafana service, heroku service, honeycomb service, humio service, influxdata service, logdna service, new relic service, papertrail service, sematext service, splunk service, yandex service, apache_metrics source, aws_ecs_metrics source, aws_kinesis_firehose source, aws_s3 source, aws_sqs source, datadog_agent source, demo_logs source, dnstap source, docker_logs source, exec source, file source, file_descriptor source, fluent source, gcp_pubsub source, heroku_logs source, host_metrics source, http source, http_scrape source, internal_logs source, internal_metrics source, journald source, kafka source, kubernetes_logs source, logstash source, mongodb_metrics source, new source, nginx_metrics source, opentelemetry source, postgresql_metrics source, prometheus_remote_write source, prometheus_scrape source, redis source, socket source, splunk_hec source, static_metrics source, statsd source, stdin source, syslog source, vector source, aws_ec2_metadata transform, dedupe transform, filter transform, geoip transform, log_to_metric transform, lua transform, metric_to_log transform, new transform, pipelines transform, reduce transform, remap transform, route transform, exclusive_route transform, sample transform, tag_cardinality_limit transform, throttle transform, apex sink, aws_cloudwatch_logs sink, aws_cloudwatch_metrics sink, aws_kinesis_firehose sink, aws_kinesis_streams sink, aws_s3 sink, aws_sqs sink, azure_blob sink, azure_monitor_logs sink, blackhole sink, clickhouse sink, console sink, datadog_archives sink, datadog_events sink, datadog_logs sink, datadog_metrics sink, elasticsearch sink, file sink, gcp_chronicle sink, gcp_cloud_storage sink, gcp_pubsub sink, gcp_stackdriver_logs sink, gcp_stackdriver_metrics sink, honeycomb sink, http sink, humio_logs sink, humio_metrics sink, influxdb_logs sink, influxdb_metrics sink, kafka sink, logdna sink, loki sink, nats sink, new sink, new_relic sink, new_relic_logs sink, opentelemetry sink, papertrail sink, prometheus_exporter sink, prometheus_remote_write sink, pulsar sink, redis sink, sematext_logs sink, sematext_metrics sink, socket sink, splunk_hec sink, statsd sink, vector sink, websocket sink, blog website, css website, guides website, highlights website, javascript website, search website, template website, website.

suzuki-shunsuke · 2024-12-04T22:39:49Z

Oh, thank you for fixing the title.

jszwedko · 2024-12-04T22:39:56Z

I updated the PR title to use on of the existing scopes which I think fits here (dev).

suzuki-shunsuke · 2024-12-04T22:41:39Z

Nice! I was thinking about seeing if we could integrate Renovate too, at some point, to automate upgrades.

Yeah, looks good. I love Renovate.

chore(aqua): replace the local registry with the standard registry

c55bc13

Now aqua-registry supports all tools you defined in your local registry. - aquaproj/aqua-registry#29339 - https://github.com/aquaproj/aqua-registry/releases/tag/v4.265.0 You don't need to manage local registry.

suzuki-shunsuke requested a review from a team as a code owner December 4, 2024 22:05

chore(aqua): update the registry

689859d

suzuki-shunsuke mentioned this pull request Dec 4, 2024

chore(aqua): Replace the local registry with standard registry #21922

Closed

jszwedko added the no-changelog Changes in this PR do not need user-facing explanations in the release changelog label Dec 4, 2024

jszwedko reviewed Dec 4, 2024

View reviewed changes

chore(aqua): update cargo-deny 1.41.3 to 2.9.1

0126ddd

- #21959 (comment)

jszwedko changed the title ~~chore(aqua): replace the local registry with the standard registry~~ chore(dev): replace the aqua local registry with the standard registry Dec 4, 2024

jszwedko approved these changes Dec 4, 2024

View reviewed changes

jszwedko enabled auto-merge December 4, 2024 22:40

jszwedko added this pull request to the merge queue Dec 4, 2024

Merged via the queue into vectordotdev:master with commit f9b7468 Dec 5, 2024
54 of 55 checks passed

suzuki-shunsuke deleted the chore-use-aqua-standard-registry branch December 5, 2024 00:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(dev): replace the aqua local registry with the standard registry #21959

chore(dev): replace the aqua local registry with the standard registry #21959

suzuki-shunsuke commented Dec 4, 2024 •

edited

Loading

bits-bot commented Dec 4, 2024 •

edited

Loading

suzuki-shunsuke commented Dec 4, 2024

suzuki-shunsuke commented Dec 4, 2024

jszwedko left a comment

jszwedko Dec 4, 2024

pront Dec 5, 2024

suzuki-shunsuke commented Dec 4, 2024 •

edited

Loading

jszwedko commented Dec 4, 2024 •

edited

Loading

suzuki-shunsuke commented Dec 4, 2024

suzuki-shunsuke commented Dec 4, 2024

jszwedko commented Dec 4, 2024

suzuki-shunsuke commented Dec 4, 2024

chore(dev): replace the aqua local registry with the standard registry #21959

chore(dev): replace the aqua local registry with the standard registry #21959

Conversation

suzuki-shunsuke commented Dec 4, 2024 • edited Loading

Summary

Change Type

Is this a breaking change?

How did you test this PR?

Does this PR include user facing changes?

Checklist

References

bits-bot commented Dec 4, 2024 • edited Loading

suzuki-shunsuke commented Dec 4, 2024

suzuki-shunsuke commented Dec 4, 2024

jszwedko left a comment

Choose a reason for hiding this comment

jszwedko Dec 4, 2024

Choose a reason for hiding this comment

pront Dec 5, 2024

Choose a reason for hiding this comment

suzuki-shunsuke commented Dec 4, 2024 • edited Loading

jszwedko commented Dec 4, 2024 • edited Loading

suzuki-shunsuke commented Dec 4, 2024

suzuki-shunsuke commented Dec 4, 2024

jszwedko commented Dec 4, 2024

suzuki-shunsuke commented Dec 4, 2024

suzuki-shunsuke commented Dec 4, 2024 •

edited

Loading

bits-bot commented Dec 4, 2024 •

edited

Loading

suzuki-shunsuke commented Dec 4, 2024 •

edited

Loading

jszwedko commented Dec 4, 2024 •

edited

Loading