Build deb client pkgs

[sbernhard]

Prepare jenkins jobs to build e.g. theforeman/foreman-packaging#6958

[evgeni]

Are client bits OS independent, so it's sufficient to build them once? Given python is involved, I'd think not?

[sbernhard]

Are client bits OS independent, so it's sufficient to build them once? Given python is involved, I'd think not?

well, they have dependencies on python library. If they are dependent on the OS, what do I need to consider in this PR?

[ekohl]

We need to decide on a layout. With https://github.com/theforeman/foreman-packaging/tree/deb/develop we have 2 layouts:

• OS dependent (debian and dependencies) which builds a package for each OS
• OS independent (plugins) which builds a package on one OS and assumes it works on all

Generally speaking, OS dependent is a safe assumption. In this case Python is very OS dependent since the version changes for every release.

In this PR it would be much safer to copy the debian / dependencies layout than plugin. I'd go for dependencies since it's the most straight forward. There are no nightly packages.

You could even go a step further and create a common function that's used in setup_sources_dependency and setup_sources_client to avoid duplicating everything, but I'm not sure how feasible this is.

[sbernhard]

Have a look again @ekohl , please :-)

[ekohl]

This looks pretty close. See inline comments.

[ekohl]

I think this looks good.

@evgeni could you also have a look?

Also many thanks to @ehelms for rewriting the pipeline, making this so much easier than it used to be.

[evgeni]

this goes into the right direction, but is not yet sufficient!

with the current structure of the "publishing" workflow, this will land in the "core" repository, not in a dedicated "client" repo, which I think is the intention here?

jenkins-jobs/theforeman.org/pipelines/lib/packaging.groovy

Lines 141 to 161 in 47c3753

	if (type == 'plugin' && !pull_request) {
	suite = 'plugins'
	component = version
	release_type = 'release'
	echo "plugin build without PR: uploading directly to deb/${suite}/${component}"
	} else if (type == 'plugin' && pull_request) {
	suite = 'plugins'
	component = repoowner
	release_type = 'stage'
	echo "scratch build: uploading to stagingdeb/${suite}/${component}"
	} else {
	suite = os
	component = "${repoowner}-${version}"
	release_type = 'stage'
	echo "scratch build: uploading to stagingdeb/${suite}/${component}"
	}
	if (!repos["${suite}-${component}"]) {
	repos["${suite}-${component}"] = [suite: suite, component: component, deb_paths: [], type: release_type]
	}
	repos["${suite}-${component}"].deb_paths.add("${deb_path}/*deb")

Right now, we have the following repo structure

deb http://deb.theforeman.org/ <OS> <version>
deb http://deb.theforeman.org/ plugins <version>
deb http://stagingdeb.theforeman.org/ <OS> <USER>-<version>

To translate:

• a core/dependencies PR gets published to http://stagingdeb.theforeman.org/ <OS> theforeman-<version> on merge
• a plugins PR gets published to deb http://deb.theforeman.org/ plugins <version> on merge
• when pipelines pass, stuff from http://stagingdeb.theforeman.org/ <OS> theforeman-<version> is copied over to http://deb.theforeman.org/ <OS> <version>

I think we should not stage client stuff (we have no tests for them anyways, and we don't do it right now for RPM either), but make it publish to deb http://deb.theforeman.org/ <OS> <version>-client directly. If you agree, I can propose a patch to ⬆️ to do the same.

[evgeni]

I think the following should do it:

diff --git theforeman.org/pipelines/lib/packaging.groovy theforeman.org/pipelines/lib/packaging.groovy
index f5acc9a..ba49378 100644
--- theforeman.org/pipelines/lib/packaging.groovy
+++ theforeman.org/pipelines/lib/packaging.groovy
@@ -160,6 +160,16 @@ def build_deb_package_steps(packages_to_build, version, repoowner = 'theforeman'
             component = repoowner
             release_type = 'stage'
             echo "scratch build: uploading to stagingdeb/${suite}/${component}"
+        } else if (type == 'client' && !pull_request) {
+            suite = "${os}-client"
+            component = version
+            release_type = 'release'
+            echo "plugin build without PR: uploading directly to deb/${suite}/${component}"
+        } else if (type == 'client' && pull_request) {
+            suite = "${os}-client"
+            component = "${repoowner}-${version}"
+            release_type = 'stage'
+            echo "scratch build: uploading to stagingdeb/${suite}/${component}"
         } else {
             suite = os
             component = "${repoowner}-${version}"

[evgeni]

Or:

diff --git theforeman.org/pipelines/lib/packaging.groovy theforeman.org/pipelines/lib/packaging.groovy
index f5acc9a..ba49378 100644
--- theforeman.org/pipelines/lib/packaging.groovy
+++ theforeman.org/pipelines/lib/packaging.groovy
@@ -160,6 +160,16 @@ def build_deb_package_steps(packages_to_build, version, repoowner = 'theforeman'
             component = repoowner
             release_type = 'stage'
             echo "scratch build: uploading to stagingdeb/${suite}/${component}"
+        } else if (type == 'client' && !pull_request) {
+            suite = os
+            component = "${version}-client"
+            release_type = 'release'
+            echo "plugin build without PR: uploading directly to deb/${suite}/${component}"
+        } else if (type == 'client' && pull_request) {
+            suite = os
+            component = "${repoowner}-${version}-client"
+            release_type = 'stage'
+            echo "scratch build: uploading to stagingdeb/${suite}/${component}"
         } else {
             suite = os
             component = "${repoowner}-${version}"

[ekohl]

The former patch results in

deb http://deb.theforeman.org/ <OS> <version>
deb http://deb.theforeman.org/ <OS>-client <version>

While the latter results in:

deb http://deb.theforeman.org/ <OS> <version> <version>-client

I think I prefer the latter.

[evgeni]

I think I prefer the latter.

So do I.

@sbernhard what do you think?

[sbernhard]

Second options. Thanks @evgeni and @ekohl for your help to get this done (hopefully soon)

[ekohl]

This looks good to me, but I missed something earlier.

[ehelms]

Leaving it to @evgeni to catch things before we break things

[sbernhard]

Is there anything left before this can be merged @evgeni / @ekohl ?

[ekohl]

I must admit I mostly forgot about this due to the many other tasks. I'd be in favor of getting this live next week.