Skip to content

Commit

Permalink
Merge branch 'commaai:master' into pyenvtouv
Browse files Browse the repository at this point in the history
  • Loading branch information
coder351 authored Jul 29, 2024
2 parents 58ad598 + c59455d commit a3c52b4
Show file tree
Hide file tree
Showing 18 changed files with 134 additions and 149 deletions.
7 changes: 0 additions & 7 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,6 @@ jobs:
id: kernel-submodule
run: echo "ref=$(git ls-tree HEAD | awk '$4 == "agnos-kernel-sdm845"' | awk '{print $3}')" | tee -a $GITHUB_OUTPUT

- name: Checkout agnos-kernel-sdm845
uses: actions/checkout@v4
with:
repository: commaai/agnos-kernel-sdm845
ref: ${{ steps.kernel-submodule.outputs.ref }}
path: agnos-kernel-sdm845

- name: ccache
uses: hendrikmuhs/ccache-action@c92f40bee50034e84c763e33b317c77adaa81c92
with:
Expand Down
9 changes: 0 additions & 9 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,12 +1,3 @@
[submodule "edk2_tici"]
path = edk2_tici
url = ../../commaai/edk2_tici.git
[submodule "agnos-firmware"]
path = agnos-firmware
url = ../../commaai/agnos-firmware.git
[submodule "agnos-kernel-sdm845"]
path = agnos-kernel-sdm845
url = ../../commaai/agnos-kernel-sdm845.git
[submodule "agnos-binaries"]
path = agnos-binaries
url = ../../commaai/agnos-binaries.git
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
# agnos-builder

This is the buidler repo for AGNOS, our Ubuntu based OS. AGNOS runs on the comma three devkit and [comma 3X](https://comma.ai/shop/comma-3x).
This is the builder repo for AGNOS, our Ubuntu based OS. AGNOS runs on the comma three devkit and [comma 3X](https://comma.ai/shop/comma-3x).

NOTE: the `edk2_tici` and `agnos-firmware` submodules are internal, private repos.
> [!NOTE]
> Use https://flash.comma.ai if you're looking to reflash your device to a clean factory state.
AGNOS includes all the software to boot and run a comma 3/3X.
This repo builds the two main parts of that (system and kernel) and packages it all up into a "release".

## Development

Expand Down Expand Up @@ -32,7 +36,7 @@ Flashing to a comma 3/3X:
Validating changes:
* Running openpilot is a good smoketest for general AGNOS functionality
* [CI](https://github.com/commaai/agnos-builder/blob/master/.github/workflows/build.yaml) ensures the kernel and system builds work (and pushes the images for you to download)
* [this](https://github.com/commaai/agnos-builder/blob/master/internal/README.md) is the checklist we go through before shipping new AGNOS releases to openpilot
* [this](https://github.com/commaai/agnos-builder/blob/master/TESTING.md) is the checklist we go through before shipping new AGNOS releases to openpilot

### macOS

Expand Down Expand Up @@ -71,6 +75,6 @@ Some concrete things on the roadmap:
- [ ] make the image tiny, for fast updating and flashing https://github.com/commaai/agnos-builder/issues/225
- [ ] update to Ubuntu 24.04 https://github.com/commaai/openpilot/issues/32386
- [ ] mainline Linux kernel https://github.com/commaai/openpilot/issues/32386
- [ ] fully open source
- [ ] anything from `agnos-firmware`: XBL, ABL, etc.
- [ ] open source Weston https://github.com/commaai/agnos-builder/issues/16
- [ ] fully open source
- [ ] anything from [firmware.json](firmware.json): XBL, ABL, etc.
- [ ] Weston https://github.com/commaai/agnos-builder/issues/16
50 changes: 50 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# testing

## Release Checklist

- [ ] [`test_onroad`](https://github.com/commaai/openpilot/blob/master/selfdrive/test/test_onroad.py) passes
- [ ] Wi-Fi: lists networks and connects
- [ ] Modem: connects to cell network
- [ ] Image sizes haven't increased
- [ ] Sounds work
- [ ] `python` is our python, not system version
- [ ] Clean openpilot build: `scons -c && scons -j8`
- [ ] Factory reset
- [ ] from openpilot menu
- [ ] tapping on boot
- [ ] corrupt userdata
- [ ] Color calibration
- [ ] from /persist/comma/
- [ ] directly from panel over sysfs
- [ ] Clean setup: factory reset -> install openpilot -> openpilot works
- [ ] AGNOS update works on warm boot
- [ ] previous -> new
- [ ] new -> previous

### ABL

- [ ] Fastboot USB enumeration
- [ ] Boot time hasn't regressed (3.8s)

### XBL

- [ ] Display init works in cold and hot temperatures
- [ ] Boot time hasn't regressed (2.4s)

### Setup

#### Networking

- [ ] Continue button disabled when no connection
- [ ] Forget/connect to wifi

#### Custom URL
- (a) Not a real URL (e.g. `comma`, `abc123`, `...`)
- [ ] "Ensure the entered URL is valid"
- [ ] Start over
- [ ] Reboot device
- (b) Website but not an installer URL (e.g. `github.com`, `comma.ai`, `installer.comma.ai`)
- [ ] "No custom software found at this URL."
- (c) Valid installer URL (e.g. `openpilot.comma.ai`)
- [ ] Download successful (comma logo or installer appears)
- [ ] `/tmp/installer_url` should contain the installer URL
1 change: 0 additions & 1 deletion agnos-binaries
Submodule agnos-binaries deleted from 72a4c0
1 change: 0 additions & 1 deletion agnos-firmware
Submodule agnos-firmware deleted from 8dbaed
21 changes: 0 additions & 21 deletions build_bootloader.sh

This file was deleted.

1 change: 0 additions & 1 deletion edk2_tici
Submodule edk2_tici deleted from 676004
32 changes: 32 additions & 0 deletions firmware.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
[
{
"name": "abl",
"url": "https://commadist.azureedge.net/agnosupdate/abl-eeb89a74c968a5a2ffce96f23158b72e03e2814adf72ef59d1200ba8ea5d2f39.img.xz",
"hash": "eeb89a74c968a5a2ffce96f23158b72e03e2814adf72ef59d1200ba8ea5d2f39",
"size": 274432
},
{
"name": "xbl",
"url": "https://commadist.azureedge.net/agnosupdate/xbl-bcef195b00a1ab685da601f4072722569773ab161e91c8753ad99ca4217a28f5.img.xz",
"hash": "bcef195b00a1ab685da601f4072722569773ab161e91c8753ad99ca4217a28f5",
"size": 3282672
},
{
"name": "xbl_config",
"url": "https://commadist.azureedge.net/agnosupdate/xbl_config-19791056558c16f8dae787531b5e30b3b3db2ded9d666688df45ce1b91a72bac.img.xz",
"hash": "19791056558c16f8dae787531b5e30b3b3db2ded9d666688df45ce1b91a72bac",
"size": 98124
},
{
"name": "devcfg",
"url": "https://commadist.azureedge.net/agnosupdate/devcfg-be44b73dda5be840b09d5347d536459e31098da3fea97596956c0bdad19bdf27.img.xz",
"hash": "be44b73dda5be840b09d5347d536459e31098da3fea97596956c0bdad19bdf27",
"size": 40336
},
{
"name": "aop",
"url": "https://commadist.azureedge.net/agnosupdate/aop-5d764611a683d6a738cf06a1dcf8a926d0f47b5117ad40d3054167de6dd8bd0f.img.xz",
"hash": "5d764611a683d6a738cf06a1dcf8a926d0f47b5117ad40d3054167de6dd8bd0f",
"size": 184364
}
]
10 changes: 5 additions & 5 deletions flash_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd $DIR

for part in aop xbl xbl_config devcfg; do
tools/edl w ${part}_a $DIR/agnos-firmware/$part.bin
tools/edl w ${part}_b $DIR/agnos-firmware/$part.bin
done
# TODO: download firmware from firmware.json
#for part in aop abl xbl xbl_config devcfg; do
# tools/edl w ${part}_a $DIR/agnos-firmware/$part.bin
# tools/edl w ${part}_b $DIR/agnos-firmware/$part.bin
#done

./flash_bootloader.sh
./flash_kernel.sh
./flash_system.sh
13 changes: 0 additions & 13 deletions flash_bootloader.sh

This file was deleted.

51 changes: 0 additions & 51 deletions internal/README.md

This file was deleted.

23 changes: 0 additions & 23 deletions internal/build_and_package.sh

This file was deleted.

14 changes: 14 additions & 0 deletions scripts/build_and_package.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/bash
set -e

DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd $DIR/..

./build_kernel.sh
./build_system.sh
scripts/package_ota.py

# push to azure on an internal machine
if [ "$USER" == "batman" ]; then
scripts/ota_push.sh staging
fi
File renamed without changes.
32 changes: 22 additions & 10 deletions internal/package_ota.py → scripts/package_ota.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env python3
import json
import os
import hashlib
import subprocess
from copy import deepcopy
from pathlib import Path
Expand All @@ -9,15 +10,16 @@
ROOT = Path(__file__).parent.parent
OUTPUT_DIR = ROOT / "output"
OTA_OUTPUT_DIR = OUTPUT_DIR / "ota"
FIRMWARE_DIR = ROOT / "agnos-firmware"

AGNOS_UPDATE_URL = os.getenv("AGNOS_UPDATE_URL", "https://commadist.azureedge.net/agnosupdate")
AGNOS_STAGING_UPDATE_URL = os.getenv("AGNOS_STAGING_UPDATE_URL", "https://commadist.azureedge.net/agnosupdate-staging")


def checksum(fn):
return subprocess.check_output(["sha256sum", fn]).decode().split()[0]

sha256 = hashlib.sha256()
with open(fn, 'rb') as f:
for chunk in iter(lambda: f.read(4096), b""):
sha256.update(chunk)
return sha256.hexdigest()

def compress(fin, fout) -> None:
subprocess.check_call(f"xz -T4 -vc {fin} > {fout}", shell=True)
Expand All @@ -32,7 +34,7 @@ def process_file(fn, name, sparse=False, full_check=True, has_ab=True, alt=None)
if sparse:
with NamedTemporaryFile() as tmp_f:
print(" converting sparse image to raw")
subprocess.check_call(["simg2img", fn, tmp_f.name])
subprocess.check_call(["simg2img", fn, tmp_f.name], shell=True)
hash_raw = checksum(tmp_f.name)
size = Path(tmp_f.name).stat().st_size
print(f" {size} bytes, hash {hash} (raw)")
Expand Down Expand Up @@ -76,18 +78,28 @@ def process_file(fn, name, sparse=False, full_check=True, has_ab=True, alt=None)

files = [
process_file(OUTPUT_DIR / "boot.img", "boot"),
process_file(FIRMWARE_DIR / "abl.bin", "abl"),
process_file(FIRMWARE_DIR / "xbl.bin", "xbl"),
process_file(FIRMWARE_DIR / "xbl_config.bin", "xbl_config"),
process_file(FIRMWARE_DIR / "devcfg.bin", "devcfg"),
process_file(FIRMWARE_DIR / "aop.bin", "aop"),
process_file(OUTPUT_DIR / "system.img", "system", sparse=True, full_check=False, alt=OUTPUT_DIR / "system-skip-chunks.img"),
]
configs = [
(AGNOS_UPDATE_URL, "ota.json"),
(AGNOS_STAGING_UPDATE_URL, "ota-staging.json"),
]

# pull in firmware not built in this repo
with open(ROOT/"firmware.json") as f:
fws = json.loads(f.read())
for fw in fws:
files.append({
"name": fw["name"],
"url": fw["url"],
"hash": fw["hash"],
"hash_raw": fw["hash"],
"size": fw["size"],
"sparse": False,
"full_check": True,
"has_ab": True,
})

for remote_url, output_fn in configs:
processed_files = []
for f in deepcopy(files):
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion userspace/usr/comma/comma.sh
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ fi
# symlink vscode to userdata
mkdir -p /data/tmp/vscode-server
ln -s /data/tmp/vscode-server ~/.vscode-server

ln -s /data/tmp/vscode-server ~/.cursor-server

while true; do
pkill -f "$SETUP"
Expand Down

0 comments on commit a3c52b4

Please sign in to comment.