make-base-vm should upgrade OS before writing base image

[devrandom]

Reported in #86 by @Michagogo. This slows down builds.

[Michagogo]

Another possibility that comes to mind would be to simply have a way of running the upgrade within the base image... Any reason that wouldn't be feasible?

[Michagogo]

Any progress on this? The Bitcoin Core 0.11.0 release cycle has started, with the first RC tagged, and if at all possible I'd like to have this fixed (and ideally a way to upgrade an existing base, but that's less important) before I start doing 3 builds for each RC and then for the release.

[devrandom]

A simple workaround for now is to execute the following after an LXC gbuild:

cp -a --sparse=always target-trusty-amd64 base-trusty-amd64

[devrandom]

(this is equivalent to running gbuild -i BTW, so it's not as clean... the build artifacts will remain, but that normally shouldn't affect future builds)

[Michagogo]

That's not a good solution, because it's the state of a VM after a whole
build, with some packages installed and files present. That's a problem --
I've tried using -i after builds targeting the same OS, and it didn't work
unless I on-targeted in and deleted everything manually. Even a gbuild
with a special gitian descriptor that does nothing might not work, since I
know a whole bunch of stuff gets done to the target after it's copied and
before the build-specific stuff works...

On Thu, Jun 4, 2015 at 10:14 PM, Dev Random [email protected]
wrote:

A simple workaround for now is to execute the following after an LXC
gbuild:

cp -a --sparse=always target-trusty-amd64 base-trusty-amd64

—
Reply to this email directly or view it on GitHub
#91 (comment)
.

[Michagogo]

Actually, what about doing that copy after a manual make-clean-vm plus a
manual package upgrade?

On Thu, Jun 4, 2015 at 10:17 PM, Micha Bailey [email protected] wrote:

That's not a good solution, because it's the state of a VM after a whole
build, with some packages installed and files present. That's a problem --
I've tried using -i after builds targeting the same OS, and it didn't work
unless I on-targeted in and deleted everything manually. Even a gbuild
with a special gitian descriptor that does nothing might not work, since I
know a whole bunch of stuff gets done to the target after it's copied and
before the build-specific stuff works...

On Thu, Jun 4, 2015 at 10:14 PM, Dev Random [email protected]
wrote:

A simple workaround for now is to execute the following after an LXC
gbuild:

cp -a --sparse=always target-trusty-amd64 base-trusty-amd64

—
Reply to this email directly or view it on GitHub
#91 (comment)
.

[devrandom]

I was just going to suggest that. Here's the outline of the commands:

export SUITE=trusty
export ARCH=amd64
make-clean-vm
on-target -u root apt-get update
on-target -u root bash < target-bin/upgrade-system.sh
cp -a --sparse=always target-trusty-amd64 base-trusty-amd64

[Michagogo]

Is there anything in make-clean-vm (or rather, in
target-bin/bootstrap-fixup) that shouldn't be repeated?

On Thu, Jun 4, 2015 at 10:35 PM, Dev Random [email protected]
wrote:

I was just going to suggest that. Here's the outline of the commands:

export SUITE=trusty
export ARCH=amd64
make-clean-vm
on-target -u root apt-get update
on-target -u root bash < target-bin/upgrade-system.sh
cp -a --sparse=always target-trusty-amd64 base-trusty-amd64

—
Reply to this email directly or view it on GitHub
#91 (comment)
.

[devrandom]

That's a good point. I just pushed a commit to make it idempotent: 7219c49

[Michagogo]

I seem to be getting an error, something about already having added a user, but it appears to be harmless.

[devrandom]

Right, I have a || true on that line to make it idempotent.

On 2015-06-05 07:22 AM, Michagogo wrote:

I seem to be getting an error, something about already having added a
user, but it appears to be harmless.

—
Reply to this email directly or view it on GitHub
#91 (comment).

devrandom / Miron

[Michagogo]

Hm. So now every time some packages are updated, gbuild fails unless I go through that process of upgrading the base container again.

[devrandom]

What is the exact error?

[Michagogo]

I don't have it written down. I think it failed when running grab-packages
or something like that, but I can't recreate it now until Ubuntu releases
more updates.

On Wednesday, July 1, 2015, Dev Random [email protected] wrote:

What is the exact error?

—
Reply to this email directly or view it on GitHub
#91 (comment)
.

[laanwj]

Agreed - a command or flag to upgrade the base image would be great. Or it could do the upgrading in the base image before making the copy.

Seems that debootstrap installs the earliest version of every package, so when the distribution grows older, more and more packages have to be updated every time, making the build take longer than necessary.

[josephbisch]

Can this be closed, because gbuild has an upgrade flag?

[Michagogo]

No -- it's still something that needs to be done on each build, and it
takes a very long time. There's a workaround to manually update the base,
but it's still something that shouldn't have to be done long-term.

On Monday, October 10, 2016, Joseph Bisch [email protected] wrote:

Can this be closed, because gbuild has an upgrade flag?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#91 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACcNnk-LU6JcFkC_RDVIdicc3J4TzTX7ks5qyYgsgaJpZM4EfsUO
.

[laanwj]

I agree. There is a workaround for this, but having a built-in command to upgrade the base image (instead of the copy) would be nice. This costs a lot of time every run, especially with LXC which starts off with a un-upgraded distribution.

[Michagogo]

(#120 would help with that last part. I meant to research and figure out
how to get that to work - or even better, to use LXC's tools to create a
container, rather than hacking one up - but unfortunately I stopped having
much free time before I got around to it…)

On Tuesday, October 11, 2016, Wladimir J. van der Laan <
[email protected]> wrote:

I agree. There is a workaround for this, but having a built-in command to
upgrade the base image (instead of the copy) would be nice. This costs a
lot of time, especially with LXC which starts off with a un-upgraded
distribution.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#91 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACcNnm0TXXKdoDU6CV58hIzz0kuxQouIks5qy509gaJpZM4EfsUO
.

[unsystemizer]

Among repetitive time-consuming actions I saw the locales built 2 times at least (I don't know why all those languages are needed in the first place, but that's another issue).