New VMs: we need netavark 1.10.3 #327

edsantiago · 2024-02-05T12:47:30Z

Signed-off-by: Ed Santiago [email protected]

github-actions · 2024-02-05T14:04:06Z

Cirrus CI build successful. Found built image names and IDs:

Stage	Image Name	`IMAGE_SUFFIX`
base	debian	`do-not-use`
base	fedora	`do-not-use`
base	fedora-aws	`do-not-use`
base	fedora-aws-arm64	`do-not-use`
base	image-builder	`do-not-use`
base	prior-fedora	`do-not-use`
cache	build-push	`c20240205t124622z-f39f38d13`
cache	debian	`c20240205t124622z-f39f38d13`
cache	fedora	`c20240205t124622z-f39f38d13`
cache	fedora-aws	`c20240205t124622z-f39f38d13`
cache	fedora-netavark	`c20240205t124622z-f39f38d13`
cache	fedora-netavark-aws-arm64	`c20240205t124622z-f39f38d13`
cache	fedora-podman-aws-arm64	`c20240205t124622z-f39f38d13`
cache	fedora-podman-py	`c20240205t124622z-f39f38d13`
cache	prior-fedora	`c20240205t124622z-f39f38d13`
cache	rawhide	`c20240205t124622z-f39f38d13`
cache	win-server-wsl	`c20240205t124622z-f39f38d13`

edsantiago · 2024-02-05T14:07:16Z

Sigh. Got .2 in rawhide, but not f39. Will try a respin later in the week.

20240205t124622z-f39f38d13
20240201t143038z-f39f38d13 ⇑ (built in New VMs: include netavark in prior-fedora #325)

	debian	prior-fedora	fedora	fedora-aws	rawhide
base	13	38-1.6	39-1.5	?	40-0

aardvark-dns	1.4.0-5	1.9.0-1	1.10.0-1	1.10.0-1	1.10.0-1
				1.9.0-1 ⇑
buildah	1.33.3+ds1-2	1.34.0-1	1.34.0-1	1.34.0-1	1.34.0-3
		1.33.2-1 ⇑
conmon	2.1.6+ds1-1	2:2.1.8-2	2:2.1.8-2	2:2.1.8-2	2:2.1.8-4

container-selinux	?	2:2.228.1-1	2:2.229.0-1	2:2.229.0-1	2:2.229.0-2
		2:2.227.0-1 ⇑
containers-common	?	4:1-89	4:1-99	4:1-95	5:0.57.3-2
			4:1-95 ⇑		5:0.57.1-7 ⇑
criu	3.17.1-3	3.18-1	3.19-2	3.19-2	3.19-4

crun	1.14-1	1.12-1	1.14-1	1.14-1	1.14-1

golang	2:1.21~2	1.20.13-1	1.21.6-1	1.21.6-1	1.22~rc2-1

netavark	1.4.0-4	1.9.0-1	1.10.1-5	1.10.1-5	1.10.2-1
					1.10.1-5 ⇑
passt	2023-12-30	2023-12-30	?	2023-12-30	2023-12-30

podman	4.9.0+ds1-2	5:4.8.3-1	5:4.9.1-1	5:4.9.0-1	5:4.9.0-1
			5:4.9.0-1 ⇑
runc	1.1.12+ds1-1	2:1.1.8-1	2:1.1.12-1	2:1.1.8-1	2:1.1.12-1
	1.1.10+ds1-1 ⇑		2:1.1.8-1 ⇑		2:1.1.9-2 ⇑
skopeo	1.13.3+ds1-2	1:1.14.1-1	1:1.14.2-1	1:1.14.0-1	1:1.14.2-1
			1:1.14.1-1 ⇑		1:1.14.1-2 ⇑
slirp4netns	1.2.1-1	1.2.2-1	1.2.2-1	1.2.2-1	1.2.2-2

systemd-container	255.3-2	253.15-2	254.8-2	254.8-2	255.3-1

tar	?	?	?	?	2:1.35-3

edsantiago · 2024-02-05T14:32:22Z

Ah - there is no .2 yet for f39. @lsm5, when convenient, could you tell packit to build netavark-1.10.2 for f39 please? We need this in CI; .1 is broken for some cases.

lsm5 · 2024-02-06T11:54:21Z

Ah - there is no .2 yet for f39. @lsm5, when convenient, could you tell packit to build netavark-1.10.2 for f39 please? We need this in CI; .1 is broken for some cases.

bodhi has been submitted. If we need something right away, we can grab the koji build from https://koji.fedoraproject.org/koji/buildinfo?buildID=2396597 in the build.

github-actions · 2024-02-07T02:58:43Z

Cirrus CI build successful. Found built image names and IDs:

Stage	Image Name	`IMAGE_SUFFIX`
base	debian	`do-not-use`
base	fedora	`do-not-use`
base	fedora-aws	`do-not-use`
base	fedora-aws-arm64	`do-not-use`
base	image-builder	`do-not-use`
base	prior-fedora	`do-not-use`
cache	build-push	`c20240207t013159z-f39f38d13`
cache	debian	`c20240207t013159z-f39f38d13`
cache	fedora	`c20240207t013159z-f39f38d13`
cache	fedora-aws	`c20240207t013159z-f39f38d13`
cache	fedora-netavark	`c20240207t013159z-f39f38d13`
cache	fedora-netavark-aws-arm64	`c20240207t013159z-f39f38d13`
cache	fedora-podman-aws-arm64	`c20240207t013159z-f39f38d13`
cache	fedora-podman-py	`c20240207t013159z-f39f38d13`
cache	prior-fedora	`c20240207t013159z-f39f38d13`
cache	rawhide	`c20240207t013159z-f39f38d13`
cache	win-server-wsl	`c20240207t013159z-f39f38d13`

edsantiago · 2024-02-07T03:01:34Z

Yay, finally have new netavark. Testing in 17831.

20240207t013159z-f39f38d13
20240205t124622z-f39f38d13 ⇑ (built in New VMs: we need netavark 1.10.3 #327)

	debian	prior-fedora	fedora	fedora-aws	rawhide
base	13	38-1.6	39-1.5	?	40-0

aardvark-dns	1.4.0-5	1.9.0-1	1.10.0-1	1.10.0-1	1.10.0-1

buildah	1.33.3+ds1-2	1.34.0-1	1.34.0-1	1.34.0-1	1.34.0-3

conmon	2.1.6+ds1-1	2:2.1.8-2	2:2.1.8-2	2:2.1.8-2	2:2.1.8-4

container-selinux	?	2:2.228.1-1	2:2.229.0-1	2:2.229.0-1	2:2.229.0-2

containers-common	?	4:1-89	4:1-99	4:1-95	5:0.57.3-2

criu	3.17.1-3	3.18-1	3.19-2	3.19-2	3.19-4

crun	1.14-1	1.12-1	1.14-1	1.14-1	1.14-1

golang	2:1.21~2	1.20.13-1	1.21.6-1	1.21.6-1	1.22~rc2-1

netavark	1.4.0-4	1.10.2-1	1.10.2-1	1.10.2-1	1.10.2-1
		1.9.0-1 ⇑	1.10.1-5 ⇑	1.10.1-5 ⇑
passt	2023-12-30	2023-12-30	?	2023-12-30	2023-12-30

podman	4.9.0+ds1-2	5:4.8.3-1	5:4.9.2-1	5:4.9.0-1	5:4.9.0-1
			5:4.9.1-1 ⇑
runc	1.1.12+ds1-1	2:1.1.8-1	2:1.1.12-1	2:1.1.12-1	2:1.1.12-1
				2:1.1.8-1 ⇑
skopeo	1.13.3+ds1-2	1:1.14.1-1	1:1.14.2-1	1:1.14.0-1	1:1.14.2-1

slirp4netns	1.2.1-1	1.2.2-1	1.2.2-1	1.2.2-1	1.2.2-2

systemd-container	255.3-2	253.15-2	254.8-2	254.8-2	255.3-1

tar	?	?	?	?	2:1.35-3

edsantiago · 2024-02-07T12:45:21Z

@containers/podman-maintainers @containers/netavark-maintainers I'm really confused here. Every single fedora e2e test failed, and every single one of them in netavark tests: https://cirrus-ci.com/build/4795190137847808

(Debian failed too, but that's because of the new runc, and I'm not even going to bother looking at that today).

This is a diff against the packages in the current-podman-CI VMs. I have to conclude that netavark-1.10.2 is broken:

20240207t013159z-f39f38d13
20240201t143038z-f39f38d13 ⇑ (built in New VMs: include netavark in prior-fedora #325)

	debian	prior-fedora	fedora	fedora-aws	rawhide
base	13	38-1.6	39-1.5	?	40-0

kernel	6.6.13-1	6.7.3-100	6.7.3-200	6.7.3-200	6.8.0-0.rc0.20240112git70d201a40823.5

aardvark-dns	1.4.0-5	1.9.0-1	1.10.0-1	1.10.0-1	1.10.0-1
				1.9.0-1 ⇑
netavark	1.4.0-4	1.10.2-1	1.10.2-1	1.10.2-1	1.10.2-1
		1.9.0-1 ⇑	1.10.1-5 ⇑	1.10.1-5 ⇑	1.10.1-5 ⇑
buildah	1.33.3+ds1-2	1.34.0-1	1.34.0-1	1.34.0-1	1.34.0-3
		1.33.2-1 ⇑
conmon	2.1.6+ds1-1	2.1.8-2	2.1.8-2	2.1.8-2	2.1.8-4

container-selinux	?	2.228.1-1	2.229.0-1	2.229.0-1	2.229.0-2
		2.227.0-1 ⇑
containers-common	?	1-89	1-99	1-95	0.57.3-2
			1-95 ⇑		0.57.1-7 ⇑
criu	3.17.1-3	3.18-1	3.19-2	3.19-2	3.19-4

crun	1.14-1	1.12-1	1.14-1	1.14-1	1.14-1

golang	2:1.21~2	1.20.13-1	1.21.6-1	1.21.6-1	1.22~rc2-1

passt	2023-12-30	2023-12-30	2023-12-30	2023-12-30	2023-12-30

podman	4.9.0+ds1-2	4.8.3-1	4.9.2-1	4.9.0-1	4.9.0-1
			4.9.0-1 ⇑
runc	1.1.12+ds1-1	1.1.8-1	1.1.12-1	1.1.12-1	1.1.12-1
	1.1.10+ds1-1 ⇑		1.1.8-1 ⇑	1.1.8-1 ⇑	1.1.9-2 ⇑
skopeo	1.13.3+ds1-2	1.14.1-1	1.14.2-1	1.14.0-1	1.14.2-1
			1.14.1-1 ⇑		1.14.1-2 ⇑
slirp4netns	1.2.1-1	1.2.2-1	1.2.2-1	1.2.2-1	1.2.2-2

systemd-container	255.3-2	253.15-2	254.8-2	254.8-2	255.3-1

tar	1.34+dfsg-1.2	1.34-8	1.35-2	1.35-2	1.35-3

Luap99 · 2024-02-07T13:22:15Z

I have to conclude that netavark-1.10.2 is broken

That seems possible although I would not understand why it broke as the only fix was the network update issue and I see failing tests do not not call update at all. All failures seem related to dns so maybe the bug is on the aardvark-dns side. I will see if I can reproduce locally and given they failed on all tests that shouldn't be to hard.

edsantiago · 2024-02-07T13:24:49Z

But aardvark-dns didn't change! (Okay, AWS, but nowhere else)

Luap99 · 2024-02-07T13:38:22Z

yes it is netavark, the patch containers/netavark#912 is buggy

edsantiago · 2024-02-07T13:47:58Z

Thank you for looking into it. I'm really sorry to have dumped it on you.

Also, add "rpm -qa" (fedora) and "dpkg -l" (debian) so Ed's package-version script can get better data. It would be nice if we could save those to an artifact file, but we can't. Signed-off-by: Ed Santiago <[email protected]>

github-actions · 2024-02-12T13:39:22Z

Cirrus CI build successful. Found built image names and IDs:

Stage	Image Name	`IMAGE_SUFFIX`
base	debian	`do-not-use`
base	fedora	`do-not-use`
base	fedora-aws	`do-not-use`
base	fedora-aws-arm64	`do-not-use`
base	image-builder	`do-not-use`
base	prior-fedora	`do-not-use`
cache	build-push	`c20240212t122113z-f39f38d13`
cache	debian	`c20240212t122113z-f39f38d13`
cache	fedora	`c20240212t122113z-f39f38d13`
cache	fedora-aws	`c20240212t122113z-f39f38d13`
cache	fedora-netavark	`c20240212t122113z-f39f38d13`
cache	fedora-netavark-aws-arm64	`c20240212t122113z-f39f38d13`
cache	fedora-podman-aws-arm64	`c20240212t122113z-f39f38d13`
cache	fedora-podman-py	`c20240212t122113z-f39f38d13`
cache	prior-fedora	`c20240212t122113z-f39f38d13`
cache	rawhide	`c20240212t122113z-f39f38d13`
cache	win-server-wsl	`c20240212t122113z-f39f38d13`

edsantiago · 2024-02-12T13:46:56Z

20240212t122113z-f39f38d13
20240201t143038z-f39f38d13 ⇑ (built in New VMs: include netavark in prior-fedora #325)

	debian	prior-fedora	fedora	fedora-aws	rawhide
base	13	38-1.6	39-1.5	?	40-0

kernel	6.6.13-1	6.7.4-100	6.7.4-200	6.7.4-200	6.8.0-0.rc3.20240209git1f719a2f3fa6.31

aardvark-dns	1.4.0-5	1.10.0-1	1.10.0-1	1.10.0-1	1.10.0-1
		1.9.0-1 ⇑		1.9.0-1 ⇑
netavark	1.4.0-4	1.10.3-1	1.10.3-1	1.10.3-1	1.10.3-1
		1.9.0-1 ⇑	1.10.1-5 ⇑	1.10.1-5 ⇑	1.10.1-5 ⇑
buildah	1.33.5+ds1-4	1.34.0-1	1.34.0-1	1.34.0-1	1.34.0-3
	1.33.3+ds1-2 ⇑	1.33.2-1 ⇑
conmon	2.1.6+ds1-1	2.1.8-2	2.1.8-2	2.1.8-2	2.1.8-4

container-selinux	?	2.228.1-1	2.229.0-1	2.229.0-1	2.229.0-2
		2.227.0-1 ⇑
containers-common	?	1-89	1-99	1-99	0.57.3-3
			1-95 ⇑	1-95 ⇑	0.57.1-7 ⇑
criu	3.17.1-3	3.18-1	3.19-2	3.19-2	3.19-4

crun	1.14-1	1.14-1	1.14.1-1	1.14-1	1.14.1-1
		1.12-1 ⇑	1.14-1 ⇑		1.14-1 ⇑
golang	2:1.21~2	1.20.13-1	1.21.7-1	1.21.6-1	1.22.0-1
			1.21.6-1 ⇑		1.22~rc2-1 ⇑
passt	2023-12-30	2023-12-30	2023-12-30	2023-12-30	2023-12-30

podman	4.9.2+ds1-2	4.8.3-1	4.9.2-1	4.9.0-1	5.0.0~rc1-2
	4.9.0+ds1-2 ⇑		4.9.0-1 ⇑		4.9.0-1 ⇑
runc	1.1.12+ds1-1	1.1.12-1	1.1.12-1	1.1.12-1	1.1.12-1
	1.1.10+ds1-1 ⇑	1.1.8-1 ⇑	1.1.8-1 ⇑	1.1.8-1 ⇑	1.1.9-2 ⇑
skopeo	1.13.3+ds1-2	1.14.1-1	1.14.2-1	1.14.0-1	1.14.2-1
			1.14.1-1 ⇑		1.14.1-2 ⇑
slirp4netns	1.2.1-1	1.2.2-1	1.2.2-1	1.2.2-1	1.2.2-2

systemd-container	255.3-2	253.15-2	254.9-1	254.9-1	255.3-1
			254.8-2 ⇑	254.8-2 ⇑
tar	1.34+dfsg-1.2	1.34-8	1.35-2	1.35-2	1.35-3

edsantiago

@cevich PTAL. VMs are already in use for podman: containers/podman#21618 so this is an FYI review, just so you're aware of the stuff that's going on here.

I am deliberately not tagging this build because there's a nasty rawhide bug that I really don't want to have to track in more repos.

lib.sh

cache_images/fedora_packaging.sh

cevich

LGTM. Ed, there's a pair of minor things not worth rebuilding/testing for. Would you consider doing them in a [CI:DOCS] followup PR?

edsantiago · 2024-02-15T13:11:41Z

Since these VMs are already in use in podman, I would hope that this can be merged.

The issues you raised (adding comments re: netavark bodhi; using some variable instead of getting it myself) are timebombed and will be obsolete within 24-48 hours, so I choose not to submit a new CI:DOCS PR to address them.

cevich · 2024-02-15T16:56:56Z

will be obsolete within 24-48 hours

Oh sorry I didn't realize they were that quick. Np then.

edsantiago force-pushed the new_netavark branch from 1bbc8e4 to 11593ef Compare February 6, 2024 20:34

edsantiago changed the title ~~New VMs: we need netavark 1.10.2-1.fc40~~ New VMs: we need netavark 1.10.2 Feb 6, 2024

edsantiago force-pushed the new_netavark branch from 11593ef to 5cc0340 Compare February 7, 2024 01:32

edsantiago mentioned this pull request Feb 7, 2024

Vendor in latest containers/storage and image containers/buildah#5290

Merged

Luap99 mentioned this pull request Feb 7, 2024

fix netavark update to not start a new aardvark-dns containers/netavark#922

Merged

edsantiago changed the title ~~New VMs: we need netavark 1.10.2~~ New VMs: we need netavark 1.10.3 Feb 8, 2024

New VMs: we need netavark 1.10.2-1.fc40

15fe970

Also, add "rpm -qa" (fedora) and "dpkg -l" (debian) so Ed's package-version script can get better data. It would be nice if we could save those to an artifact file, but we can't. Signed-off-by: Ed Santiago <[email protected]>

edsantiago force-pushed the new_netavark branch from 5cc0340 to 15fe970 Compare February 12, 2024 12:22

edsantiago mentioned this pull request Feb 12, 2024

Bump CI VMs to ones with netavark 1.10.3 containers/podman#21618

Merged

edsantiago commented Feb 13, 2024

View reviewed changes

lib.sh Show resolved Hide resolved

lib.sh Show resolved Hide resolved

cache_images/fedora_packaging.sh Show resolved Hide resolved

cevich reviewed Feb 13, 2024

View reviewed changes

cache_images/fedora_packaging.sh Show resolved Hide resolved

cevich approved these changes Feb 13, 2024

View reviewed changes

cevich merged commit 5af77ad into containers:main Feb 15, 2024
39 checks passed

edsantiago deleted the new_netavark branch February 15, 2024 18:07

edsantiago mentioned this pull request Feb 15, 2024

Utilize the new library timebomb() function. #323

Merged

This was referenced Feb 16, 2024

DO NOT MERGE! Testing new rawhide kernel #328

Closed

New VMs again, keeping f38 #329

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New VMs: we need netavark 1.10.3 #327

New VMs: we need netavark 1.10.3 #327

edsantiago commented Feb 5, 2024

github-actions bot commented Feb 5, 2024

edsantiago commented Feb 5, 2024

edsantiago commented Feb 5, 2024

lsm5 commented Feb 6, 2024

github-actions bot commented Feb 7, 2024

edsantiago commented Feb 7, 2024

edsantiago commented Feb 7, 2024

Luap99 commented Feb 7, 2024

edsantiago commented Feb 7, 2024

Luap99 commented Feb 7, 2024

edsantiago commented Feb 7, 2024

github-actions bot commented Feb 12, 2024

edsantiago commented Feb 12, 2024

edsantiago left a comment

cevich left a comment

edsantiago commented Feb 15, 2024

cevich commented Feb 15, 2024

New VMs: we need netavark 1.10.3 #327

New VMs: we need netavark 1.10.3 #327

Conversation

edsantiago commented Feb 5, 2024

github-actions bot commented Feb 5, 2024

edsantiago commented Feb 5, 2024

edsantiago commented Feb 5, 2024

lsm5 commented Feb 6, 2024

github-actions bot commented Feb 7, 2024

edsantiago commented Feb 7, 2024

edsantiago commented Feb 7, 2024

Luap99 commented Feb 7, 2024

edsantiago commented Feb 7, 2024

Luap99 commented Feb 7, 2024

edsantiago commented Feb 7, 2024

github-actions bot commented Feb 12, 2024

edsantiago commented Feb 12, 2024

edsantiago left a comment

Choose a reason for hiding this comment

cevich left a comment

Choose a reason for hiding this comment

edsantiago commented Feb 15, 2024

cevich commented Feb 15, 2024