Noticed a couple of dupes (gz and rom) so I figured I should sort it now
to hopefully avoid another one in the future.

Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Remove hard coded paths from shebangs and other references because they
do not play well in nix-land. Either use /usr/bin/env to do runtime PATH
based lookup or avoid absolute paths so PATH look up happens instead.

Signed-off-by: Thierry Laurion <[email protected]>
Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Just some minor clean ups like fixing whitespace and sorting things. I
added (bash)/removed (libusb) white space in order to look like the
other modules.

I sorted the --enable/--disable/--with blocks so that common stuff
looked similar which should aid in comparing modules. I also removed a
couple of duplicate config options (--disable-fallback-curses &
--disable-regex).

Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Thierry Laurion <[email protected]>
Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

More pending work needed to fix fragility of buildsystem and fix nix
build issues as well like:

https://app.circleci.com/pipelines/github/mmlb/osresearch-heads/11/workflows/32cc883c-5074-4f28-94b8-a83a2ec44414/jobs/252
https://app.circleci.com/pipelines/github/mmlb/osresearch-heads/11/workflows/32cc883c-5074-4f28-94b8-a83a2ec44414/jobs/221
https://app.circleci.com/pipelines/github/tlaurion/heads/1781/workflows/ee402ead-6739-4549-88ae-105b695fb3cd
https://app.circleci.com/pipelines/github/tlaurion/heads/1783/workflows/2b35826c-aff4-4f48-8809-4e66259f9aa4/jobs/25877/parallel-runs/0/steps/0-103

Signed-off-by: Thierry Laurion <[email protected]>
Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Thanks to @JonathonHall-Purism, that pointed to me that sysroot was
neglected in tpm2-tools configure step.

Signed-off-by: Thierry Laurion <[email protected]>
Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

… args

Thanks to @JonathonHall-Purism, that pointed to me that sysroot was
neglected in tpm2-tools configure step.

I wonder why this is not respected if not forced with --with-sysroot and
TSS2_ESYS_3_0_LIBS="-ltss2-esys -L$(INSTALL)/lib"?

Signed-off-by: Thierry Laurion <[email protected]>
Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

These are generated during the build process so should be ignored.

Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Got rid of long lines in favor of more lines for readability. Cleaned up
some comments/typos and unnecessary cruft*. Finally ran prettier on the
file for its automatic formatting, including whitespace clean ups.

cruft:

-  && when already set -e
- run commands with trailing \
- deleted commented out "OLD STUFF"
- sorted listy looking things because unsorted stuff bothers me :) (I
  held back on sorting the board build definitions though, thats
  probably too much).

Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Gives the exact same output:

```
docker run --rm -ti debian:11 bash -c '
  mkdir -p build/subdir1/ build/subdir2
  echo "subdir1 error" >build/subdir1/fail.log
  echo "subdir2 error" >build/subdir2/fail.log
  find build -type f -name "*.log" -exec tail -n +1 "{}" +
'
==> build/subdir1/fail.log <==
subdir1 error

==> build/subdir2/fail.log <==
subdir2 error
```

Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Until nix PR is merged to not interfere with master/other pr caches

Signed-off-by: Manuel Mendez <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

…maximized

Signed-off-by: Thierry Laurion <[email protected]>

…now passed from golbal Makefile sysroot (TODO: generalize)

Signed-off-by: Thierry Laurion <[email protected]>

…es to not delete already downloaded packages to economize bandwidth

Signed-off-by: Thierry Laurion <[email protected]>

/nix/store/5lr5n3qa4day8l1ivbwlcby2nknczqkq-bash-5.2p26/bin/bash ./libtool  --tag=CC   --mode=link /home/user/heads/crossgcc/x86/bin/x86_64-linux-musl-gcc -fdebug-prefix-map=/home/user/heads=heads -gno-record-gcc-switches -D__MUSL__ --sysroot  /home/user/heads/install/x86 -isystem /home/user/heads/install/x86/include -L/home/user/heads/install/x86/lib  -I./tools -I./lib -Wall -Wextra -Wformat -Wformat-security -Wstack-protector -fstack-protector-all -Wstrict-overflow=5 -O2 -fPIC -fPIE -D_GNU_SOURCE -std=gnu99 -Wstringop-overflow=4 -Wstringop-truncation -Wduplicated-branches -Wduplicated-cond -Wbool-compare -fdata-sections -ffunction-sections -I/home/user/heads/install/x86/include -I/home/user/heads/install/x86//include -I/home/user/heads/install/x86//include/tss2 -I/home/user/heads/install/x86/nix/store/yg75achq89wgqn2fi3gglgsd77kjpi03-openssl-3.0.13-dev/include  -I/home/user/heads/install/x86//include -I/home/user/heads/install/x86//include/tss2 -I/home/user/heads/install/x86//include -I/home/user/heads/install/x86//include/tss2 -I/home/user/heads/install/x86//include -I/home/user/heads/install/x86//include/tss2 -DTPM2_TOOLS_MAX="101" -fdebug-prefix-map=/home/user/heads/install/x86=. -shared -pie -Wl,-z,relro -Wl,-z,now -Wl,--gc-sections   -o tools/tpm2 tools/tpm2-tpm2_tool.o tools/misc/tpm2-tpm2_certifyX509certutil.o tools/misc/tpm2-tpm2_checkquote.o tools/misc/tpm2-tpm2_encodeobject.o tools/misc/tpm2-tpm2_eventlog.o tools/misc/tpm2-tpm2_print.o tools/misc/tpm2-tpm2_rc_decode.o tools/misc/tpm2-tpm2_tr_encode.o tools/tpm2-tpm2_activatecredential.o tools/tpm2-tpm2_certify.o tools/tpm2-tpm2_changeauth.o tools/tpm2-tpm2_changeeps.o tools/tpm2-tpm2_changepps.o tools/tpm2-tpm2_clear.o tools/tpm2-tpm2_clearcontrol.o tools/tpm2-tpm2_clockrateadjust.o tools/tpm2-tpm2_create.o tools/tpm2-tpm2_createak.o tools/tpm2-tpm2_createek.o tools/tpm2-tpm2_createpolicy.o tools/tpm2-tpm2_setprimarypolicy.o tools/tpm2-tpm2_createprimary.o tools/tpm2-tpm2_dictionarylockout.o tools/tpm2-tpm2_duplicate.o tools/tpm2-tpm2_getcap.o tools/tpm2-tpm2_gettestresult.o tools/tpm2-tpm2_encryptdecrypt.o tools/tpm2-tpm2_evictcontrol.o tools/tpm2-tpm2_flushcontext.o tools/tpm2-tpm2_getrandom.o tools/tpm2-tpm2_gettime.o tools/tpm2-tpm2_hash.o tools/tpm2-tpm2_hierarchycontrol.o tools/tpm2-tpm2_hmac.o tools/tpm2-tpm2_import.o tools/tpm2-tpm2_incrementalselftest.o tools/tpm2-tpm2_load.o tools/tpm2-tpm2_loadexternal.o tools/tpm2-tpm2_makecredential.o tools/tpm2-tpm2_nvdefine.o tools/tpm2-tpm2_nvextend.o tools/tpm2-tpm2_nvincrement.o tools/tpm2-tpm2_nvreadpublic.o tools/tpm2-tpm2_nvread.o tools/tpm2-tpm2_nvreadlock.o tools/tpm2-tpm2_nvundefine.o tools/tpm2-tpm2_nvwrite.o tools/tpm2-tpm2_nvwritelock.o tools/tpm2-tpm2_nvsetbits.o tools/tpm2-tpm2_pcrallocate.o tools/tpm2-tpm2_pcrevent.o tools/tpm2-tpm2_pcrextend.o tools/tpm2-tpm2_pcrread.o tools/tpm2-tpm2_pcrreset.o tools/tpm2-tpm2_policypcr.o tools/tpm2-tpm2_policyauthorize.o tools/tpm2-tpm2_policyauthorizenv.o tools/tpm2-tpm2_policynv.o tools/tpm2-tpm2_policycountertimer.o tools/tpm2-tpm2_policyor.o tools/tpm2-tpm2_policynamehash.o tools/tpm2-tpm2_policytemplate.o tools/tpm2-tpm2_policycphash.o tools/tpm2-tpm2_policypassword.o tools/tpm2-tpm2_policysigned.o tools/tpm2-tpm2_policyticket.o tools/tpm2-tpm2_policyauthvalue.o tools/tpm2-tpm2_policysecret.o tools/tpm2-tpm2_policyrestart.o tools/tpm2-tpm2_policycommandcode.o tools/tpm2-tpm2_policynvwritten.o tools/tpm2-tpm2_policyduplicationselect.o tools/tpm2-tpm2_policylocality.o tools/tpm2-tpm2_quote.o tools/tpm2-tpm2_readclock.o tools/tpm2-tpm2_readpublic.o tools/tpm2-tpm2_rsadecrypt.o tools/tpm2-tpm2_rsaencrypt.o tools/tpm2-tpm2_send.o tools/tpm2-tpm2_selftest.o tools/tpm2-tpm2_setclock.o tools/tpm2-tpm2_shutdown.o tools/tpm2-tpm2_sign.o tools/tpm2-tpm2_certifycreation.o tools/tpm2-tpm2_nvcertify.o tools/tpm2-tpm2_startauthsession.o tools/tpm2-tpm2_startup.o tools/tpm2-tpm2_stirrandom.o tools/tpm2-tpm2_testparms.o tools/tpm2-tpm2_unseal.o tools/tpm2-tpm2_verifysignature.o tools/tpm2-tpm2_setcommandauditstatus.o tools/tpm2-tpm2_getcommandauditdigest.o tools/tpm2-tpm2_getsessionauditdigest.o tools/tpm2-tpm2_geteccparameters.o tools/tpm2-tpm2_ecephemeral.o tools/tpm2-tpm2_commit.o tools/tpm2-tpm2_ecdhkeygen.o tools/tpm2-tpm2_ecdhzgen.o tools/tpm2-tpm2_zgen2phase.o tools/tpm2-tpm2_sessionconfig.o tools/tpm2-tpm2_getpolicydigest.o lib/libcommon.a -ltss2-esys -L/home/user/heads/install/x86/lib -L/home/user/heads/install/x86//lib -ltss2-mu -L/home/user/heads/install/x86/nix/store/7nmrrad8skxr47f9hfl3xc0pfqmwq51b-openssl-3.0.13/lib -lcrypto -L/home/user/heads/install/x86//lib -ltss2-tctildr -L/home/user/heads/install/x86//lib -ltss2-rc -L/home/user/heads/install/x86//lib -ltss2-sys
libtool:   error: cannot find the library '//lib/libtss2-sys.la' or unhandled argument '//lib/libtss2-sys.la'
make[1]: *** [Makefile:2478: tools/tpm2] Error 1
make[1]: Leaving directory '/home/user/heads/build/x86/tpm2-tools-5.6'
make: *** [Makefile:521: /home/user/heads/build/x86/tpm2-tools-5.6/.build] Error 1

Signed-off-by: Thierry Laurion <[email protected]>

…test CPUS=8 to maximize loadavg on CircleCI with 4 CPUs & 8GB ram

See first lines of output of any make command. Change aimed to be respectful of CI resource (8GB ram 4CPUs)

With CPUS=8 AVAILABLE_MEM_GB=4, CircleCI outputs:
!!!!!! BUILD SYSTEM INFO !!!!!!
System CPUS: 36
System Available Memory: 4 GB
System Load Average: 12.99
----------------------------------------------------------------------
Used **CPUS**: 8
Used **LOADAVG**: 8
Used **AVAILABLE_MEM_GB**: 4 GB
----------------------------------------------------------------------
**MAKE_JOBS**: -j8 --max-load 8

Variables available for override (use 'make VAR_NAME=value'):
**CPUS** (default: number of processors, e.g., 'make CPUS=4')
**LOADAVG** (default: same as CPUS, e.g., 'make LOADAVG=4')
**AVAILABLE_MEM_GB** (default: memory available on the system in GB, e.g., 'make AVAILABLE_MEM_GB=4')
**MEM_PER_JOB_GB** (default: 1GB per job, e.g., 'make MEM_PER_JOB_GB=2')
----------------------------------------------------------------------

Let's try without any limitation...

Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Thierry Laurion <[email protected]>

…at were missing it to propogate build optimizations per module, while still impossible to call make -j 12 on main make call

Signed-off-by: Thierry Laurion <[email protected]>

…ch_rebuild_what_changed: reextract packages, repatch sources (might fail, easy to fix) and rebuild only what changed (not a lot)

if patch fails to apply, its because patch file creates a file and doesn't expect it to exist.
just call rm on the file reported to exist, and relaunch build.

Deletes ./install/*/* and permits to rebuild all dependencies in order, just based on freshly extracted and patched code.
Bonus, this saves your SDD from unneeded wear and rebuilds faster then all other Mafile helpers.

That's my favorite.

Signed-off-by: Thierry Laurion <[email protected]>

…t loosing traces of reproduction notes

Signed-off-by: Thierry Laurion <[email protected]>

… creation. Cleaner NIX_REPRO_NOTES

Signed-off-by: Thierry Laurion <[email protected]>

…elop) which fails at tpm2-tss

- switch cache to nix-docker-heads to not interfere with nixos develop layer on same PR
- remove nix develop calls; replace by direct script calls and make calls
- make sure save/restore/root is ~/heads

Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Thierry Laurion <[email protected]>

….com/pipelines/github/tlaurion/heads/2500/workflows/23674215-8b22-4852-adf4-2a6df9e44353/jobs/45080?invite=true#step-102-16530_106 zlib-dev not found on coreboot buildstack buildstep...

Signed-off-by: Thierry Laurion <[email protected]>

… to zlib-ng

Signed-off-by: Thierry Laurion <[email protected]>

…circleci.com/pipelines/github/tlaurion/heads/2500/workflows/23674215-8b22-4852-adf4-2a6df9e44353/jobs/45080?invite=true#step-102-16530_106 zlib-dev not found on coreboot buildstack buildstep..."

This reverts commit d6c4996.

Signed-off-by: Thierry Laurion <[email protected]>

…zlib.dev to zlib-ng"

This reverts commit 9052d2b.

Signed-off-by: Thierry Laurion <[email protected]>

…ard additional step

Signed-off-by: Thierry Laurion <[email protected]>

…le since there is no easy way of fixing this

nix doesn't provide an equivalent of libltdl-dev, so just wipe the remnant of old ages if present
tpm2-software/tpm2-tss#2161

Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Thierry Laurion <[email protected]>

… flake.nix for talos-2 board's linux config

Signed-off-by: Thierry Laurion <[email protected]>

…p without hacking around

Signed-off-by: Thierry Laurion <[email protected]>

…v.sh.

Was causing another ac macro misbehavior since host ac was not considered by aclocal and autoreconf

TODO: Might want to revert 6a17911 if talos-2 board is able to build 3rdparty/sb-sign-tool

Signed-off-by: Thierry Laurion <[email protected]>

…rated file since there is no easy way of fixing this"

This reverts commit 6a17911.

Signed-off-by: Thierry Laurion <[email protected]>

…xport of develop env into the docker image. Works locally for talos-2 board build.

Signed-off-by: Thierry Laurion <[email protected]>

…qemu-coreboot-whiptail-tpm2 with swtpm and canokey for smartcard

- include nix tools inside of the docker to be able to call the garbage collector prior of creating docker.
- protect roots from garbage collection (WiP)
  - Requires external preparation call so that nix (the binary) is not wiped as well. See NIX_REPRO_NOTES at the end of the file for repro notes
   - Could probably be improved. Works as of now and created a 4Gb vs 3.02Gb docker image I'm uploading now.
- CircleCI bumped to use v0.0.9 version including this
- CircleCI now depending on flake.lock for all cache layers. Will rebuild clean once again

So now we have qemu with canokey support in image, nix basic tools inside of container. Possible to call docker with DISPLAY, see NIX_REPRO_NOTES as of now.
That feels nice. No need of USB security dongle to have TPM based TPMTOTP nor detach sign? Not tested but feature is there

TODO:
- make docker creating nicer in the Nix way.
- Add canokey support under targets/qemu.mk
- add canokey board version

At least we have reproducible stack and testing stack being in same docker image. Docker image moved from 991.18MB (v0.0.8) to 1.18GB (v0.0.9)
- And I tried to clean binaries of symbols here! Seems like I do not know enough of the Nix way here.

Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Thierry Laurion <[email protected]>

Signed-off-by: Thierry Laurion <[email protected]>

…d makes the docker image bigger

TODO: push v0.1.2 with those changes pointing circleci to use it
Signed-off-by: Thierry Laurion <[email protected]>

… of garbage collection inside of nix prior of making the docker

Signed-off-by: Thierry Laurion <[email protected]>

…nix developed created docker image

- push v0.1.3 and have latest point to the same image, add repro notes inside of README.md
- modify qemu.md to also refer to using docker images

TODO: remove NIX_REPRO_NOTES prior of merging

Signed-off-by: Thierry Laurion <[email protected]>

…esOS Template/AppVM for usage. Expand on nix repro instructions fro NIX_REPRO_NOTES for review

Signed-off-by: Thierry Laurion <[email protected]>

…or native kvm support: added

Signed-off-by: Thierry Laurion <[email protected]>

…emu_kvm for kvm testing in docker image

Signed-off-by: Thierry Laurion <[email protected]>

…nce cosntant interest in the builds)

Modify .circleci/config.yml to also not reuse past caches if CircleCI config changes as part of calculated hashes for the 3 layers
Signed-off-by: Thierry Laurion <[email protected]>

Specifying LIBS_BASE causes flashrom's Makefile to link in an RPATH,
using the Heads workspace path, which is not what we want.  It does
other things too, but we already pass the parts we need to the make
invocation for flashrom.

Signed-off-by: Jonathon Hall <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

Use relative paths in configs generated from templates, so the final
build doesn't depend on the absolute location of the repository.  The
coreboot config is part of the final ROM.

Signed-off-by: Jonathon Hall <[email protected]>
Signed-off-by: Thierry Laurion <[email protected]>

… Nix users, remove NIX_REPRO_NOTES, Makefile dev helpers self-explain themselves

Signed-off-by: Thierry Laurion <[email protected]>

…instructions

Signed-off-by: Thierry Laurion <[email protected]>