Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variables/Connections/Pools specified in airflow_settings.yaml can't be imported or exported with Podman #1724

Open
1 of 3 tasks
joaomsa opened this issue Sep 26, 2024 · 0 comments
Assignees

Comments

@joaomsa
Copy link

joaomsa commented Sep 26, 2024

Describe the bug
Values in airflow_settings.yaml are not respected when using podman as the container binary

When trying to use the astro dev object import or astro dev object export commands while running with podman the commands fail without displaying any error messages.

For the first command, although astro indicates that variables have been successfully imported, the airflow command was never actually run. For the latter we have a fatal error.

These astro commands do not respect the container.binary: podman option specified in the .astro/config.yaml file and instead hardcode the docker executable.

https://github.com/astronomer/astro-cli/blob/main/docker/docker.go#L21

which podman

/opt/podman/bin/podman

which docker

docker not found

cat .astro/config.yaml

project:
    name: test-bug
container:
    binary: podman

cat airflow_settings.yaml

airflow:
    variables:
        - variable_name: hello
          variable_value: world

astro dev object import

Added Variable: hello

Airflow objects created from settings file

astro dev object export

panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
github.com/astronomer/astro-cli/settings.ExportPools({0x140001159c0?, 0x15?})
        github.com/astronomer/astro-cli/settings/settings.go:571 +0x748
github.com/astronomer/astro-cli/settings.Export({0x140001159c0, 0x40}, {0x102f1869a?, 0x0?}, 0x2, 0x1, 0x1, 0x1)
        github.com/astronomer/astro-cli/settings/settings.go:435 +0x7c
github.com/astronomer/astro-cli/airflow.(*DockerCompose).ExportSettings(0x14000262000, {0x102f1869a, 0x15}, {0x102f014ad, 0x4}, 0x0?, 0x0?, 0x0?, 0x0)
        github.com/astronomer/astro-cli/airflow/docker.go:1156 +0x150
github.com/astronomer/astro-cli/cmd.airflowSettingsExport(0x14000550f08, {0x103f0ed60?, 0x4?, 0x102f015fd?})
        github.com/astronomer/astro-cli/cmd/airflow.go:930 +0x108
github.com/spf13/cobra.(*Command).execute(0x14000550f08, {0x103f0ed60, 0x0, 0x0})
        github.com/spf13/[email protected]/command.go:940 +0x648
github.com/spf13/cobra.(*Command).ExecuteC(0x140001e6908)
        github.com/spf13/[email protected]/command.go:1068 +0x320
github.com/spf13/cobra.(*Command).Execute(0x1034f2160?)
        github.com/spf13/[email protected]/command.go:992 +0x1c
main.main()
        github.com/astronomer/astro-cli/main.go:21 +0x38

astro dev run variables export -

Running: airflow variables export -
[2024-09-26T09:43:56.575+0000] {plugin.py:32} WARNING - Astro managed secrets backend is disabled
{}

What CLI Version did you experience this bug?

astro version

Astro CLI Version: 1.29.0

This CLI bug is related to which Astronomer Platform?

  • Astro
  • Software
  • None/Unknown

What Operating System is the above CLI installed on?

🪜 Steps To Reproduce

Attempt to specify variables/connections/pools in airflow_settings.yaml without the docker client installed (using an alternative binary like podman)

📸 Screenshots

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants