Packaging pcs with Nix

[matt1432]

I've been trying to package pcs for NixOS and I've been mostly successful. You can see my attempt here: https://github.com/matt1432/nixos-pcsd

However, a lot of changes to the build process were needed to get everything to work so I was looking for some help or pointers on how I could deal with this in a better way.

Here are the things I would like help with:

Modifications of build files

postUnpack = ''
      # Fix version of untagged build
      echo 'printf %s "${version}"' > $sourceRoot/make/git-version-gen


      # Fix pam path https://github.com/NixOS/nixpkgs/blob/5a072b4a9d7ccf64df63645f3ee808dc115210ba/pkgs/development/python-modules/pamela/default.nix#L20
      substituteInPlace $sourceRoot/pcs/lib/auth/pam.py --replace \
        'find_library("pam")' \
        '"${lib.getLib pam}/lib/libpam.so"'


      # Fix hardcoded paths to binaries
      substituteInPlace $sourceRoot/pcsd/bootstrap.rb --replace \
        "/bin/hostname" "${lib.getBin hostname}/bin/hostname"

      substituteInPlace $sourceRoot/pcsd/pcs.rb --replace \
        "/bin/cat" "${lib.getBin coreutils}/bin/cat"


      # Fix systemd path
      substituteInPlace $sourceRoot/configure.ac \
        --replace 'AC_SUBST([SYSTEMD_UNIT_DIR])' "SYSTEMD_UNIT_DIR=$out/lib/systemd/system
        AC_SUBST([SYSTEMD_UNIT_DIR])"


      # Fix paths to corosync and pacemaker executables
      substituteInPlace $sourceRoot/configure.ac \
        --replace 'PCS_PKG_CHECK_VAR([COROEXECPREFIX], [corosync], [exec_prefix], [/usr])' "COROEXECPREFIX=${corosync}
        AC_SUBST([COROEXECPREFIX])"

      substituteInPlace $sourceRoot/configure.ac \
        --replace 'PCS_PKG_CHECK_VAR([PCMKEXECPREFIX], [pacemaker], [exec_prefix], [/usr])' "PCMKEXECPREFIX=${pacemaker}
        AC_SUBST([PCMKEXECPREFIX])"

      substituteInPlace $sourceRoot/configure.ac \
        --replace "\$prefix/libexec/pacemaker" "${pacemaker}/libexec/pacemaker"


      # Don't create var files
      substituteInPlace $sourceRoot/pcsd/Makefile.am --replace \
        '$(MKDIR_P) -m 0700 $(DESTDIR)$(localstatedir)/log/pcsd' ""

      substituteInPlace $sourceRoot/pcsd/Makefile.am --replace \
	      '$(MKDIR_P) -m 0700 $(DESTDIR)$(localstatedir)/lib/pcsd' ""
    '';

Configure flags

configureFlags = [
  # This was the closest to what I needed, see PAM config
  "--with-distro=fedora"
  "--enable-use-local-cache-only"
  "--with-pcs-lib-dir=${placeholder "out"}/lib"
  "--with-default-config-dir=${placeholder "out"}/etc"
  "--localstatedir=/var"
];

PAM config

security.pam.services.pcsd.text = ''
  #%PAM-1.0
  auth       include      systemd-user
  account    include      systemd-user
  password   include      systemd-user
  session    include      systemd-user
'';

Some more info:

Untagged build

Since I wanted to get latest git to update to the latest gems, I tried building from source but I got this error:

> make/git-version-gen: WARNING: .gitarchivever doesn't contain valid version tag                                                            
> make/git-version-gen: ERROR: Can't find valid version. Please use valid git repository, released tarball or version tagged archive

which I fixed with this line: echo 'printf %s "${version}"' > $sourceRoot/make/git-version-gen
I could definitely use some help with git-version-gen

SystemD paths

In nix, all the built files go to $out which is a path in the nix store that only has the package's files. I found it quite hard to specify where pcs should place the systemd unit files so I had to a very ugly text replacement with a new line to get it to work.
I am also unsure which systemd environment variable I was supposed to change.

# This is basically a sed that replace the 2nd line with the 3rd and 4th lines in configure.ac
substituteInPlace $sourceRoot/configure.ac --replace \
'AC_SUBST([SYSTEMD_UNIT_DIR])' \
"SYSTEMD_UNIT_DIR=$out/lib/systemd/system
AC_SUBST([SYSTEMD_UNIT_DIR])"

Pacemaker and Corosync executables

To have the pacemaker and corosync executables available to pcs, I basically did the same thing as for the systemd path but for the env vars PCMKEXECPREFIX and COROEXECPREFIX
I also had to change the $prefix for $prefix/libexec/pacemaker because the prefix would be different for every package in nix

Var directories

In nix, the builds are sandbox and therefore cannot make directories. It would be nice to have a flag maybe to disable that instead of replacing the lines with nothing like I did, although there might be a better way of doing this.

In nix, the service takes care of this which I declare in my module in my git repo.

"--with-distro=*"

As far as I can tell, the major difference in the --with-distro parameters was the PAM configuration. In my module I had to make my own PAM config file based on the fedora one, but I replaced system-auth with systemd-user. I have no real knowledge of PAM so I have no idea if this is good or not but it works.

Obviously I am not expecting pcs to support nix but I would appreciate any help I can get since this was my most challenging software to package in nix and I am pretty certain there is still a lot I can improve.
Thank you

[mirecheck]

Hi @matt1432,

ad 1) untagged build

Build from an untagged commit is handled by creating a file .tarball-version
in the root of source directory. Just create a string which contains the latest
version and short commit hash like this: ${version}+${short_commit_hash}.
You can look at fedora pcs.spec
file how it is done.

ad 2) pam path

      # Fix pam path https://github.com/NixOS/nixpkgs/blob/5a072b4a9d7ccf64df63645f3ee808dc115210ba/pkgs/development/python-modules/pamela/default.nix#L20
      substituteInPlace $sourceRoot/pcs/lib/auth/pam.py --replace \
        'find_library("pam")' \
        '"${lib.getLib pam}/lib/libpam.so"'

I'm not sure how to help with this. Probably it is some issue of python package in nixos, that it cannot find the right path.

ad 3) fix other paths

     # Fix systemd path
      substituteInPlace $sourceRoot/configure.ac \
        --replace 'AC_SUBST([SYSTEMD_UNIT_DIR])' "SYSTEMD_UNIT_DIR=$out/lib/systemd/system
        AC_SUBST([SYSTEMD_UNIT_DIR])"


      # Fix paths to corosync and pacemaker executables
      substituteInPlace $sourceRoot/configure.ac \
        --replace 'PCS_PKG_CHECK_VAR([COROEXECPREFIX], [corosync], [exec_prefix], [/usr])' "COROEXECPREFIX=${corosync}
        AC_SUBST([COROEXECPREFIX])"

      substituteInPlace $sourceRoot/configure.ac \
        --replace 'PCS_PKG_CHECK_VAR([PCMKEXECPREFIX], [pacemaker], [exec_prefix], [/usr])' "PCMKEXECPREFIX=${pacemaker}
        AC_SUBST([PCMKEXECPREFIX])"

      substituteInPlace $sourceRoot/configure.ac \
        --replace "\$prefix/libexec/pacemaker" "${pacemaker}/libexec/pacemaker"

I think you do not have to modify configure.ac file. It is enough to define env variables before running configure script in order to override these settings:

SYSTEMD_UNIT_DIR_TMP="$out/lib/systemd/system"
COROEXECPREFIX="${corosync}"
PCMKEXECPREFIX="${pacemaker}"
PCMKDAEMONDIR="${pacemaker}/libexec/pacemaker"

ad 4) var files

      # Don't create var files
      substituteInPlace $sourceRoot/pcsd/Makefile.am --replace \
        '$(MKDIR_P) -m 0700 $(DESTDIR)$(localstatedir)/log/pcsd' ""

      substituteInPlace $sourceRoot/pcsd/Makefile.am --replace \
      '$(MKDIR_P) -m 0700 $(DESTDIR)$(localstatedir)/lib/pcsd' ""

Is it working without these directories? I think pcsd needs them in order to be able run.
I don't know how they should be handled in nix.

ad 5) PAM config
If your PAM config works then you can create pull request which adds a file pcsd/pam/pcsd.nixos
and also add a new distro section to configure.ac

I hope I helped a little bit.