Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Guix buildpack #1048

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/source/config_files.rst
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,27 @@ to produce a reproducible environment.
To see an example repository visit
`nix binder example <https://github.com/binder-examples/nix>`_.

.. _manifest.scm:

``manifest.scm`` - the Guix package manager
===========================================

Specify packages to be installed by the `Guix package manager <https://guix.gnu.org/>`_.
All packages specified in |manifest|_ will be installed in a container using |guix_package|_.If you do not install Jupyter explicitly, Repo2docker will no be able to start your container. In addition, you can use different `channels <https://guix.gnu.org/manual/en/html_node/Channels.html>`_ rather
than the ones available by default (official channels of GNU Guix 1.3.0).
You must describe such channels in a ``channels.scm`` file which will be used
alongside ``manifest.scm`` with the |guix_time-machine|_ command. Furthermore, using a ``channels.scm`` file lets you `pin a specific revision <https://guix.gnu.org/manual/en/html_node/Replicating-Guix.html>`_ of Guix, allowing you to unambiguously specific the software environment to reproduce.

For more information about Guix please read the `manual <https://guix.gnu.org/manual/en/guix.html>`_.

.. |manifest| replace:: ``manifest.scm``
.. _manifest: https://guix.gnu.org/manual/en/html_node/Invoking-guix-package.html#index-profile-manifesthy

.. |guix_package| replace:: ``guix package``
.. _guix_package: https://guix.gnu.org/manual/en/html_node/Invoking-guix-package.html#Invoking-guix-package

.. |guix_time-machine| replace:: ``guix time-machine``
.. _guix_time-machine: https://guix.gnu.org/manual/en/html_node/Invoking-guix-time_002dmachine.html

``Dockerfile`` - Advanced environments
======================================
Expand Down
2 changes: 2 additions & 0 deletions repo2docker/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
PipfileBuildPack,
PythonBuildPack,
RBuildPack,
GuixBuildPack,
)
from . import contentproviders
from .utils import ByteSpecification, chdir
Expand Down Expand Up @@ -95,6 +96,7 @@ def _default_log_level(self):
JuliaProjectTomlBuildPack,
JuliaRequireBuildPack,
NixBuildPack,
GuixBuildPack,
RBuildPack,
CondaBuildPack,
PipfileBuildPack,
Expand Down
1 change: 1 addition & 0 deletions repo2docker/buildpacks/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@
from .legacy import LegacyBinderDockerBuildPack
from .r import RBuildPack
from .nix import NixBuildPack
from .guix import GuixBuildPack
79 changes: 79 additions & 0 deletions repo2docker/buildpacks/guix/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
"""BuildPack for guix environments"""
import os

from ..base import BuildPack, BaseImage


class GuixBuildPack(BaseImage):
"""A Guix Package Manager BuildPack"""

def get_build_env(self):
""""""
env = super().get_build_env() + [("GUIX_DIR", "${APP_BASE}/guix")]
return env

def get_path(self):
"""Return paths to be added to PATH environment variable"""
path = super().get_path()
path.insert(0, "${GUIX_DIR}/bin")
return path

def get_build_scripts(self):
"""
Install a pinned version of Guix package manager,
precised in guix-install.bash.
"""
return super().get_build_scripts() + [
(
"root",
"""
bash /tmp/.local/bin/guix-install.bash
""",
),
]

def get_build_script_files(self):

"""Copying guix installation script on the image"""
return {
"guix/guix-install.bash": "/tmp/.local/bin/guix-install.bash",
}

def get_assemble_scripts(self):
"""
Launch Guix daemon with --disable-chroot to avoid the need
of root privileges for the user.
Make sure we never use Debian's python by error by renaming it
then, as an user install packages listed in manifest.scm,
use guix time-machine if channels.scm file exists.
Finally set Guix environment variables.
"""
assemble_script = """
/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon \
--build-users-group=guixbuild --disable-chroot & \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this single & intentional? Does this daemon continue to run in the background after this step is done?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, you need the guix-daemon running to be able to use guix!

su - $NB_USER -c '{}' && \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can return multiple scripts from here, some to be run as root and some as NB_USER. That's probably a bit cleaner than using su here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, so Im struggling a bit with this part. It seems I can't keep the guix-daemon running between distinct steps. In some cases I have some issues when the docker images is not completely rebuilt, steps by steps but reusing some steps already executed before. That's why I did that here, I launch the guix-daemon just before running the Guix command I need, therefore I'm absolutely sure guix-daemon is running. Maybe you have some clues or ideas about a better way to do so ?

echo 'GUIX_PROFILE="/var/guix/profiles/per-user/'$NB_USER'/guix-profile" ; \
source "$GUIX_PROFILE/etc/profile"'>> /etc/profile.d/99-guix.sh
"""

if os.path.exists(self.binder_path("channels.scm")):
assemble_script = assemble_script.format(
"guix time-machine -C "
+ self.binder_path("channels.scm")
+ " -- package -m "
+ self.binder_path("manifest.scm")
)
else:
assemble_script = assemble_script.format(
"guix package -m " + self.binder_path("manifest.scm")
)
return super().get_assemble_scripts() + [
(
"root",
assemble_script,
)
]

def detect(self):
"""Check if current repo should be built with the guix BuildPack"""
return os.path.exists(self.binder_path("manifest.scm"))
14 changes: 14 additions & 0 deletions repo2docker/buildpacks/guix/guix-install.bash
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/bash
# This downloads and installs a pinned version of Guix using a pinned version of the installation script.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# This downloads and installs a pinned version of Guix using a pinned version of the installation script.
#!/bin/bash
# This downloads and installs a pinned version of Guix using a pinned version of the installation script.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey thanks for that, I will add it in the next rebase!

# Here is the commit associated with Guix version 1.3.0 the latest version when this script was written.
set -euxo pipefail

GUIX_COMMIT="a0178d34f582b50e9bdbb0403943129ae5b560ff"
BIN_VER="1.3.0x86_64-linux"
GUIX_SHA256="bcdeaa757cd42d2c9de4791272737e9ee0d518398403955f113611f4a893380a"

wget "https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh?id=$GUIX_COMMIT"

echo "$GUIX_SHA256 guix-install.sh?id=$GUIX_COMMIT" | sha256sum -c

(yes || true) | BIN_VER=$BIN_VER bash guix-install.sh?id=$GUIX_COMMIT
4 changes: 4 additions & 0 deletions tests/guix/binder-dir/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
`manifest.scm` in a binder/ directory
-------------------------------------

Check if we can find and use `manifest.scm` when it is ina `binder/` sub-directory.
3 changes: 3 additions & 0 deletions tests/guix/binder-dir/binder/manifest.scm
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
(specifications->manifest
'("jupyter"
"hello"))
3 changes: 3 additions & 0 deletions tests/guix/binder-dir/verify
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh

hello
4 changes: 4 additions & 0 deletions tests/guix/ignore-outside/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
`manifest.scm` in main directory and in `binder/`
-------------------------------------------------

Check if `manifest.scm` located in the `binder/` sub-directory is prefered to the one in the main directory.
4 changes: 4 additions & 0 deletions tests/guix/ignore-outside/binder/manifest.scm
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
(specifications->manifest
'("jupyter"
"python"
"python-numpy"))
2 changes: 2 additions & 0 deletions tests/guix/ignore-outside/manifest.scm
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
(specifications->manifest
'("jupyter"))
3 changes: 3 additions & 0 deletions tests/guix/ignore-outside/verify
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env python3

import numpy
4 changes: 4 additions & 0 deletions tests/guix/simple-channels/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
`manifest.scm` and `channels.scm` in main directory
---------------------------------------------------

CHeck if we can use `manifest.scm` alongside a `channels.scm` file using `guix time-machine ...`
20 changes: 20 additions & 0 deletions tests/guix/simple-channels/channels.scm
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
(list (channel
(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git")
(commit
"f1bfd9f1948a5ff336d737c0614b9a30c2bb3097")
(introduction
(make-channel-introduction
"9edb3f66fd807b096b48283debdcddccfea34bad"
(openpgp-fingerprint
"BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA"))))
(channel
(name 'nonguix)
(url "https://gitlab.com/nonguix/nonguix")
(commit
"46c1d8bcca674d3a71cd077c52dde9552a89873d")
(introduction
(make-channel-introduction
"897c1a470da759236cc11798f4e0a5f7d4d59fbc"
(openpgp-fingerprint
"2A39 3FFF 68F4 EF7A 3D29 12AF 6F51 20A0 22FB B2D5")))))
3 changes: 3 additions & 0 deletions tests/guix/simple-channels/manifest.scm
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
(specifications->manifest
'("jupyter"
"hello"))
3 changes: 3 additions & 0 deletions tests/guix/simple-channels/verify
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh

hello
5 changes: 5 additions & 0 deletions tests/guix/simple/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

`manifest.scm` in the main directory
----------------------------------

Check if we can find and use `manifest.scm` when it is in the main directory.
3 changes: 3 additions & 0 deletions tests/guix/simple/manifest.scm
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
(specifications->manifest
'("jupyter"
"hello"))
3 changes: 3 additions & 0 deletions tests/guix/simple/verify
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh

hello