build packages with cmake presets and vcpkg #641

scareything · 2023-05-01T19:28:07Z

Set up and use vcpkg when building packages

also, fixes #639

…Linux

… a sufficient openssl version.

…an tlsuv needs

…untu instead of Debian; build with vcpack preset

qrkourier · 2023-05-03T23:27:29Z

The Focal DEB installs, and I got a few good functional Ziti service results, but sporadic failures as well with logs like these corresponding with each attempt to connect to a Ziti service. I verified the service was authorized with the tunnel_status command. I reproduced this result in a Focal VM and Focal LXC container and could not reproduce it during the same time frame on my Jammy workstation using an identity from the same ZEDS app.

$ journalctl -lfu ziti-edge-tunnel
#... truncated
May 03 23:20:26 ubuntu2004.localdomain ziti-edge-tunnel[639]: (639)[      349.625]   ERROR ziti-sdk:channel.c:853 on_channel_connect_internal() ch[1] failed to connect [-103/software caused connection abort]
May 03 23:20:26 ubuntu2004.localdomain ziti-edge-tunnel[639]: (639)[      349.626]   ERROR ziti-sdk:channel.c:853 on_channel_connect_internal() ch[2] failed to connect [-103/software caused connection abort]
May 03 23:20:26 ubuntu2004.localdomain ziti-edge-tunnel[639]: (639)[      349.627]    WARN ziti-sdk:connect.c:1523 process_edge_message() conn[0.7/Closed] data[432 bytes] received in state[Closed]
May 03 23:20:28 ubuntu2004.localdomain ziti-edge-tunnel[639]: (639)[      351.737]   ERROR ziti-sdk:channel.c:483 dispatch_message() ch[0] received message without conn_id or for unknown connection ct[ED72] conn_id[7]

Link to follow-up post the next day where the same symptom of an empty reply from the Ziti service coincides with the same WARN message.

qrkourier · 2023-05-04T00:03:50Z

Summary of package test

os	resolv.conf	install	tun	dns	summary
redhat7	NetworkManager	[x]	[x]	[ ]	OKAY w/ manual dns
redhat8	dhclient	[x]	[x]	[ ]	OKAY w/ manual dns
trusty	resolvconf	[ ]	[ ]	[ ]	install failed
xenial	resolvconf	[ ]	[ ]	[ ]	install failed
bionic	resolved	[ ]	[ ]	[ ]	install failed
focal	resolved	[x]	[x]	[x]	OKAY (ignoring ZEDS errors)
jammy	resolved	[x]	[x]	[x]	OKAY

qrkourier · 2023-05-04T19:09:55Z

Ubuntu 14 Trusty install error indicates /usr/lib/systemd/system dir is needed but doesn't exist.

EDIT: Ubuntu 16 Xenial has the same issue as Trusty.

$ yes|dpkg --install /tmp/ziti-edge-tunnel-*.deb || apt-get --yes --fix-broken install
# ... truncated
ln: failed to create symbolic link ‘/usr/lib/systemd/system/ziti-edge-tunnel.service’: No such file or directory
ln: failed to create symbolic link ‘/usr/lib/systemd/system/ziti-edge-tunnel.service’: No such file or directory
/usr/bin/deb-systemd-helper: error: unable to read ziti-edge-tunnel.service

----------------------------------------------------------------------------------------------------------------------------------------------------------------------
ziti-edge-tunnel was installed...
First install an OpenZiti identity or enroll token in: /opt/openziti/etc/identities
then start or restart this systemd service unit.
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
Processing triggers for libc-bin (2.19-0ubuntu6.15) ...

root@trusty-test:~# systemctl status ziti-edge-tunnel.service
ziti-edge-tunnel.service
   Loaded: error (Reason: No such file or directory)
   Active: inactive (dead)

root@trusty-test:~# ll /usr/lib/systemd
total 24
drwxr-xr-x  6 root root 4096 May  4 19:06 ./
drwxr-xr-x 57 root root 4096 May  4 19:06 ../
drwxr-xr-x  2 root root 4096 May  4 19:06 catalog/
drwxr-xr-x  2 root root 4096 Nov  7  2019 ntp-units.d/
drwxr-xr-x  2 root root 4096 May  4 19:06 user/
drwxr-xr-x  2 root root 4096 Apr  3  2019 user-generators/

root@trusty-test:~# ll /usr/lib/systemd/system
ls: cannot access /usr/lib/systemd/system: No such file or directory

scareything · 2023-05-04T19:25:22Z

The ln errors may be related to changes that were made in pr #622. cc @sabedevops

sabedevops · 2023-05-04T19:41:37Z

Seems that on the older distributions, the /lib is not a symlink to /usr/lib.

One approach is to set the unit directory for these distro packages builds via -D SYSTEMD_UNIT_DIR=/lib/systemd/system after adding an if (NOT SYSTEMD_UNIT_DIR) or similar.

qrkourier · 2023-05-04T20:22:00Z

On RedHat8, I encountered the same WARN log that I saw yesterday on Focal coinciding with an empty reply from the Ziti service. This is the first occurence of the anomaly during this batch.

May 04 20:17:04 rocky8.localdomain ziti-edge-tunnel[4997]: (4997)[      319.066]    WARN ziti-sdk:connect.c:1523 process_edge_message() conn[0.11/Closed] data[432 bytes] received in state[Closed]

Link to post about this above from yesterday, Wed 3rd at 19:27 EDT.

Update

I wasn't able to reproduce this issue after switching from ZEDS to my OpenZiti lab network.

qrkourier · 2023-05-04T20:36:58Z

Ubuntu 18 Bionic install errors

ln: failed to create symbolic link '/usr/lib/systemd/system/ziti-edge-tunnel.service': No such file or directory
ln: failed to create symbolic link '/usr/lib/systemd/system/ziti-edge-tunnel.service': No such file or directory
systemd-sysusers: unrecognized option '--replace=/usr/lib/sysusers.d/ziti-edge-tunnel.conf'
chown: invalid user: ‘ziti:ziti’
chown: invalid group: ‘root:ziti’
/usr/bin/deb-systemd-helper: error: unable to read ziti-edge-tunnel.service

--------------------------------------------------------------------------------
ziti-edge-tunnel was installed...
First install an OpenZiti identity or enroll token in: /opt/openziti/etc/identities
then start or restart this systemd service unit.
--------------------------------------------------------------------------------

qrkourier · 2023-05-04T20:49:39Z

os	resolv.conf	install	tun	dns	summary
redhat7	NetworkManager	[x]	[x]	[ ]	OKAY w/ manual dns
redhat8	NetworkManager	[x]	[x]	[ ]	OKAY w/ manual dns
trusty	resolvconf	[ ]	[ ]	[ ]	install failed
xenial	resolvconf	[ ]	[ ]	[ ]	install failed
bionic	resolved	[ ]	[ ]	[ ]	install failed
focal	resolved	[x]	[x]	[x]	OKAY
jammy	resolved	[x]	[x]	[x]	OKAY

Adjusts systemd unit directory for older Ubuntu distribution packages.

scareything · 2023-05-09T13:07:51Z

The install failures on bionic and earlier manifest with this:

ERROR ziti-edge-tunnel:resolvers.c:321 set_systemd_resolved_link_setting() Failure calling method: SetLinkLLMNR for link: (tun0): (org.freedesktop.DBus.Error.AccessDenied, Access to org.freedesktop.resolve1.Manager.SetLinkLLMNR() not permitted.)

The polkit configuration that the postinstall script drops into /var/lib/polkit-1/localauthority/10-vendor.d/ziti-edge-tunnel.pkla seems to be correct:

[Permit ziti-edge-tunnel to configure link DNS]
Identity=unix-user:ziti
Action=org.freedesktop.resolve1.*
ResultAny=yes

However the version of systemd on bionic and earlier does not expose the SetLinkLLMNR method through polkit. Indeed, the associated action isn't registered:

vagrant@ubuntu-18:~$ grep "action id" /usr/share/polkit-1/actions/org.freedesktop.resolve1.policy 
        <action id="org.freedesktop.resolve1.register-service">
        <action id="org.freedesktop.resolve1.unregister-service">

The method was exposed with this commit, which was released with systemd v243. Bionic has v237:

vagrant@ubuntu-18:~$ systemd --version
systemd 237
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid

So we need to do something else on bionic. @sabedevops mentioned the possibility of dropping privileges only after dns has been configured so that we can munge resolv.conf on the older rh distros. Delaying the drop would also help in this situation, too.

qrkourier · 2023-05-09T13:40:34Z

I confirmed that registering the additional actions with 18.04 Bionic's polkitd did not allow SetLinkLLMNR. I borrowed the actions from 20.04 Focal for this test.

qrkourier

advancing package fixes with pre-existing issues related to drop-privs that are being worked in another issue

scareything and others added 30 commits May 1, 2023 14:12

use cmake presets when building package

fcf7fc0

install and set up vcpkg in build image

a4e291f

fix cmake --build argument order

fbd7d22

look for zet binary in ${config} directory

d5fb04c

include config in run binary step

b39dbd1

use ssl libs from package manager when building with -DOPENSSL=ON on …

c47b1a7

…Linux

don't specify overlay ports from preset, since some OSs don't provide…

5a22a42

… a sufficient openssl version.

build openssl on rocky 8, since the repos provide an older version th…

ffe4f78

…an tlsuv needs

redhat 7 & 8 package builds with openssl

cafcaae

install vcpkg

2bfeea2

install vcpkg in ubuntu 14 builder image

bd160b3

try with system ssl libraries now that tlsuv builds with openssl 1.x

ecba7c6

correct cmake build arguments

06166a1

use zlib from system when building for linux

fc7cd7f

install openssl-devel from epel repo for newer version

242e7cc

re-add the Debian builder Dockerfile for reference

223db98

configure MultiArch for Ubuntu so we can build for Jammy, Focal on Ub…

a4c1ccc

…untu instead of Debian; build with vcpack preset

build bionic with vcpkg

7635848

stop running ZET with extra args

1faece1

delete the old Dockerfile for building ZET on Debian

2c4020f

try with system ssl libraries now that tlsuv builds with openssl 1.x

175b898

use separate vcpkg path for rh7.

ed5e324

dont use system-provided openssl on rh7

e4296a8

install zlib-dev packages for cross compiling

926599e

use vcpkg on ubuntu 16

a6007dc

remove line continuation

52b5cb7

install dependencies for vcpkg

4fe372d

install ninja

40304af

build opeenssl on ubuntu 16

3853af6

fix cmake command

c6744d5

scareything added 6 commits May 4, 2023 08:56

fall back to useradd in debian post script if systemd-sysuers is absent

c514d41

use distro subdirectories for docker context

ce5e391

construct dockerfile path with format

0395ac2

require distro input

afc7211

copy action metadata file into distro subdirectory before calling

e1889d8

fix paths. use workflow variable

c0e2a8b

qrkourier self-requested a review May 4, 2023 17:16

scareything added 3 commits May 4, 2023 13:19

fix paths

9fa5c98

fix line escape

8c38bd3

move all files from distro subdirectory when configuring build action

c22f872

sabedevops and others added 4 commits May 5, 2023 10:11

Adjusts systemd unit directory for older Ubuntu distribution packages.

0a5e511

Reverts to static path for systemd unit dir.

638c86d

Merge pull request #647 from openziti/hotfix/package-regression-pr-622

ed83643

Adjusts systemd unit directory for older Ubuntu distribution packages.

don't use older versions of systemd-sysusers

3d5bbe1

qrkourier mentioned this pull request May 9, 2023

older ubuntu packages fail #653

Closed

qrkourier approved these changes May 9, 2023

View reviewed changes

scareything merged commit 287dbc5 into main May 9, 2023

scareything deleted the fix.package.build branch May 9, 2023 17:33

scareything restored the fix.package.build branch August 18, 2023 15:08

scareything deleted the fix.package.build branch August 18, 2023 15:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build packages with cmake presets and vcpkg #641

build packages with cmake presets and vcpkg #641

scareything commented May 1, 2023 •

edited

Loading

qrkourier commented May 3, 2023 •

edited

Loading

qrkourier commented May 4, 2023 •

edited

Loading

qrkourier commented May 4, 2023 •

edited

Loading

scareything commented May 4, 2023

sabedevops commented May 4, 2023

qrkourier commented May 4, 2023 •

edited

Loading

qrkourier commented May 4, 2023

qrkourier commented May 4, 2023

scareything commented May 9, 2023 •

edited

Loading

qrkourier commented May 9, 2023

qrkourier left a comment

build packages with cmake presets and vcpkg #641

build packages with cmake presets and vcpkg #641

Conversation

scareything commented May 1, 2023 • edited Loading

qrkourier commented May 3, 2023 • edited Loading

qrkourier commented May 4, 2023 • edited Loading

Summary of package test

qrkourier commented May 4, 2023 • edited Loading

scareything commented May 4, 2023

sabedevops commented May 4, 2023

qrkourier commented May 4, 2023 • edited Loading

Update

qrkourier commented May 4, 2023

qrkourier commented May 4, 2023

scareything commented May 9, 2023 • edited Loading

qrkourier commented May 9, 2023

qrkourier left a comment

Choose a reason for hiding this comment

scareything commented May 1, 2023 •

edited

Loading

qrkourier commented May 3, 2023 •

edited

Loading

qrkourier commented May 4, 2023 •

edited

Loading

qrkourier commented May 4, 2023 •

edited

Loading

qrkourier commented May 4, 2023 •

edited

Loading

scareything commented May 9, 2023 •

edited

Loading