Hardlinks Can Cause ACIs to be Created Which Can't be Rendered

[g654]

If I modify the examples/nginx script to also install the postgresql apk the resulting ACI cannot be rendered by rkt.

$ git clone https://github.com/appc/acbuild
$ cd acbuild/
$ emacs examples/nginx/build-nginx.sh
$ git diff
diff --git a/examples/nginx/build-nginx.sh b/examples/nginx/build-nginx.sh
index 6ca6bc1..3422c29 100755
--- a/examples/nginx/build-nginx.sh
+++ b/examples/nginx/build-nginx.sh
@@ -25,6 +25,7 @@ acbuild --debug dep add quay.io/coreos/alpine-sh
 # Install nginx
 acbuild --debug run apk update
 acbuild --debug run apk add nginx
+acbuild --debug run apk add postgresql

 # Add a port for http traffic over port 80
 acbuild --debug port add http tcp 80
$ su
# bash examples/nginx/build-nginx.sh
# rkt run --insecure-options=image nginx-latest-linux-amd64.aci 
rkt: using image from local store for image name coreos.com/rkt/stage1-coreos:0.13.0
rkt: using image from file /tmp/acbuild/nginx-latest-linux-amd64.aci
rkt: using image from local store for image name quay.io/coreos/alpine-sh
run: error setting up stage0: error setting up image  sha512-0d4006b18e8950c08258c0b0774eef1d96e9aebafbcde3d49cdabf9ca531a127: error rendering tree image: treestore: cannot render aci: error extracting ACI: extracttar error: exit status 1, output: error extracting tar: error extracting tarball: link / /rootfs/etc/terminfo/v/vt200: operation not permitted

I can run the nginx example without my additional line, so I believe my rkt installation is correct. Looking at nginx-latest-linux-amd64.aci with tar t shows many 'Substituting `.' for empty hard link target' errors.

[cgonyeo]

I'm having problems reproducing this. As of last week the handling of hard links when extracting images changed, although this seems to be related to creating the images so I'm not positive if it'll have an impact here. Are you doing this with acbuild from master, or from the latest release? If you're using the latest release, would you be willing to build master and try this again?

When I add acbuild --debug run apk add postgresql to the nginx example and tar tfv nginx-latest-linux-amd64.aci | grep -i Substituting, I get no output. Also looking for the link that rkt errored on when you attempted to run the resulting ACI, things look the way I'm expecting:

derek@rokot ~/go/src/github.com/appc/acbuild> tar tfv nginx-latest-linux-amd64.aci | grep vt200          
-rw-r--r-- 0/0            1279 2015-04-24 02:39 rootfs/etc/terminfo/v/vt200
hrw-r--r-- 0/0               0 2015-04-24 02:39 rootfs/etc/terminfo/v/vt220 link to rootfs/etc/terminfo/v/vt200

[g654]

$ git clone https://github.com/appc/acbuild
Cloning into 'acbuild'...
remote: Counting objects: 1301, done.
remote: Total 1301 (delta 0), reused 0 (delta 0), pack-reused 1301
Receiving objects: 100% (1301/1301), 1.69 MiB | 904.00 KiB/s, done.
Resolving deltas: 100% (590/590), done.
Checking connectivity... done.
$ cd acbuild/
$ ./build
Building acbuild...
$ bin/acbuild version
acbuild version v0.2.1-22-g44c41e2
appc version 0.7.1+git
$ emacs examples/nginx/build-nginx.sh
$ git diff
diff --git a/examples/nginx/build-nginx.sh b/examples/nginx/build-nginx.sh
index 6ca6bc1..898cf74 100755
--- a/examples/nginx/build-nginx.sh
+++ b/examples/nginx/build-nginx.sh
@@ -6,6 +6,8 @@ if [ "$EUID" -ne 0 ]; then
     exit 1
 fi

+which acbuild
+acbuild version
 # Start the build with an empty ACI
 acbuild --debug begin

@@ -25,6 +27,7 @@ acbuild --debug dep add quay.io/coreos/alpine-sh
 # Install nginx
 acbuild --debug run apk update
 acbuild --debug run apk add nginx
+acbuild --debug run apk add postgresql

 # Add a port for http traffic over port 80
 acbuild --debug port add http tcp 80

# PATH=/tmp/acbuild/bin:$PATH bash examples/nginx/build-nginx.sh 
/tmp/acbuild/bin/acbuild
acbuild version v0.2.1-22-g44c41e2
appc version 0.7.1+git
Beginning build with an empty ACI
Setting name of ACI to example.com/nginx
Adding dependency "quay.io/coreos/alpine-sh"
Running: [apk update]
Downloading quay.io/coreos/alpine-sh: [========================] 2.65 MB/2.65 MB
fetch http://dl-4.alpinelinux.org/alpine/v3.2/main/x86_64/APKINDEX.tar.gz
v3.2.3-139-g3ce61e0 [http://dl-4.alpinelinux.org/alpine/v3.2/main]
OK: 5292 distinct packages available
Running: [apk add nginx]
(1/2) Installing pcre (8.37-r1)
(2/2) Installing nginx (1.8.0-r1)
Executing nginx-1.8.0-r1.pre-install
Executing busybox-1.23.2-r0.trigger
OK: 7 MiB in 17 packages
Running: [apk add postgresql]
(1/6) Installing ncurses-terminfo-base (5.9-r3)
(2/6) Installing ncurses-libs (5.9-r3)
(3/6) Installing libedit (20150325.3.1-r0)
(4/6) Installing libpq (9.4.5-r0)
(5/6) Installing postgresql-client (9.4.5-r0)
(6/6) Installing postgresql (9.4.5-r0)
Executing busybox-1.23.2-r0.trigger
OK: 23 MiB in 23 packages
Adding port "http"="tcp"
Adding mount point "html"="/usr/share/nginx/html"
Setting exec command [/usr/sbin/nginx -g daemon off;]
Writing ACI to nginx-latest-linux-amd64.aci
Ending the build
# tar tfv nginx-latest-linux-amd64.aci | grep vt200
tar: Substituting `.' for empty hard link target
hrw-r--r-- 0/0               0 2015-04-24 10:39 rootfs/etc/terminfo/v/vt200 link to .
hrw-r--r-- 0/0               0 2015-04-24 10:39 rootfs/etc/terminfo/v/vt220 link to rootfs/etc/terminfo/v/vt200
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target
tar: Substituting `.' for empty hard link target

I've attached the ACI it generates as aci.zip in case it's any use.

[cgonyeo]

Interesting, it looks like some of the files are being converted into hard links to . when the tar file is written. What distro are you running this on?

[g654]

$ uname -a
Linux think 4.2.0-1-amd64 #1 SMP Debian 4.2.6-3 (2015-12-06) x86_64 GNU/Linux

[g654]

I confirm these steps work fine on the Ubuntu Vagrant image in the rkt repo.

[cgonyeo]

I spent some time yesterday trying to get acbuild to work in Debian (issues getting overlayfs to work) in an attempt to reproduce this on my end. I dug through some of the code to see if anything jumped out at me, but at this point I'm not sure what's causing this. I'll keep poking.

[g654]

Thanks for the help. :-)