Unable to Install master branch

[morrislee]

Describe the bug

Master branch or anything based on master recently has not been able to complete install. Looks like it fails at Agnos both using SSH command or regulat update UI button

screenshot shows some sort of size chunk issue?

Provide a route where the issue occurs

dc7716b32bf25574

openpilot version

0.9.8

Additional info

No response

[jakethesnake420]

I have the same issue. The system img url provided in agnos.json is not working.
https://commadist.azureedge.net/agnosupdate-staging/system-93a86656670d6d8d99ea401bd5735cd1060c2355d65f2c14de522c77a80c57ea.img.xz

...well at least it cannot be downloaded.

Edit:

Tried again and i can download it now.

Edit:

its failing again;

wget https://commadist.azureedge.net/agnosupdate-staging/system-93a86656670d6d8d99ea401bd5735cd1060c2355d65f2c14de522c77a80c57ea.img.xz -O system.img
--2024-12-03 03:16:27--  https://commadist.azureedge.net/agnosupdate-staging/system-93a86656670d6d8d99ea401bd5735cd1060c2355d65f2c14de522c77a80c57ea.img.xz
Resolving commadist.azureedge.net (commadist.azureedge.net)... 13.107.246.38, 2620:1ec:bdf::38
Connecting to commadist.azureedge.net (commadist.azureedge.net)|13.107.246.38|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 800363048 (763M) [application/x-xz]
Saving to: ‘system.img’

system.img                                       1%[>                                                                                                   ]   8.00M  19.0MB/s    in 0.4s

2024-12-03 03:16:28 (19.0 MB/s) - Connection closed at byte 8388608. Retrying.

--2024-12-03 03:16:29--  (try: 2)  https://commadist.azureedge.net/agnosupdate-staging/system-93a86656670d6d8d99ea401bd5735cd1060c2355d65f2c14de522c77a80c57ea.img.xz
Connecting to commadist.azureedge.net (commadist.azureedge.net)|13.107.246.38|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 800363048 (763M), 791974440 (755M) remaining [application/x-xz]
Saving to: ‘system.img’

system.img                                      24%[+======================>                                                                            ] 184.00M  55.0MB/s    in 3.6s

2024-12-03 03:16:33 (49.5 MB/s) - Connection closed at byte 192937984. Retrying.

--2024-12-03 03:16:35--  (try: 3)  https://commadist.azureedge.net/agnosupdate-staging/system-93a86656670d6d8d99ea401bd5735cd1060c2355d65f2c14de522c77a80c57ea.img.xz
Connecting to commadist.azureedge.net (commadist.azureedge.net)|13.107.246.38|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 800363048 (763M), 607425064 (579M) remaining [application/x-xz]
Saving to: ‘system.img’

system.img                                      25%[++++++++++++++++++++++++>                                                                           ] 192.00M  22.3MB/s    in 0.4s

2024-12-03 03:16:35 (22.3 MB/s) - Connection closed at byte 201326592. Retrying.

--2024-12-03 03:16:38--  (try: 4)  https://commadist.azureedge.net/agnosupdate-staging/system-93a86656670d6d8d99ea401bd5735cd1060c2355d65f2c14de522c77a80c57ea.img.xz
Connecting to commadist.azureedge.net (commadist.azureedge.net)|13.107.246.38|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 800363048 (763M), 599036456 (571M) remaining [application/x-xz]
Saving to: ‘system.img’

system.img                                      26%[+++++++++++++++++++++++++>                                                                          ] 200.00M  9.49MB/s    in 0.8s

2024-12-03 03:16:40 (9.49 MB/s) - Connection closed at byte 209715200. Retrying.

[nelsonjchen]

Possibly related to #34138 ?

[morrislee]

Possibly related to #34138 ?

looks like the same concern. can be replicated sinply try to download the system image off http link

[morrislee]

tried installing it just now, started working, hopefully it stays working when I flip back and forth on forks

[nelsonjchen]

I think this is a bit out of comma's control and a bit in.

openpilot/system/hardware/tici/agnos.py

Lines 21 to 54 in a22d6cd

	class StreamingDecompressor:
	def __init__(self, url: str) -> None:
	self.buf = b""
	self.req = requests.get(url, stream=True, headers={'Accept-Encoding': None}, timeout=60)
	self.it = self.req.iter_content(chunk_size=1024 * 1024)
	self.decompressor = lzma.LZMADecompressor(format=lzma.FORMAT_AUTO)
	self.eof = False
	self.sha256 = hashlib.sha256()
	def read(self, length: int) -> bytes:
	while len(self.buf) < length and not self.eof:
	if self.decompressor.needs_input:
	self.req.raise_for_status()
	try:
	compressed = next(self.it)
	except StopIteration:
	self.eof = True
	break
	else:
	compressed = b''
	self.buf += self.decompressor.decompress(compressed, max_length=length)
	if self.decompressor.eof:
	self.eof = True
	break
	result = self.buf[:length]
	self.buf = self.buf[length:]
	self.sha256.update(result)
	return result

openpilot/system/hardware/tici/agnos.py

Lines 165 to 187 in a22d6cd

	with open(path, 'wb+') as out:
	# Flash partition
	last_p = 0
	raw_hash = hashlib.sha256()
	f = unsparsify if partition['sparse'] else noop
	for chunk in f(downloader):
	raw_hash.update(chunk)
	out.write(chunk)
	p = int(out.tell() / partition['size'] * 100)
	if p != last_p:
	last_p = p
	print(f"Installing {partition['name']}: {p}", flush=True)
	if raw_hash.hexdigest().lower() != partition['hash_raw'].lower():
	raise Exception(f"Raw hash mismatch '{raw_hash.hexdigest().lower()}'")
	if downloader.sha256.hexdigest().lower() != partition['hash'].lower():
	raise Exception("Uncompressed hash mismatch")
	if out.tell() != partition['size']:
	raise Exception("Uncompressed size mismatch")
	os.sync()

@jakethesnake420 's example with wget even show some resilience on wget's part with auto-resumes. However, the agnos downloader is a bit basic and assumes Azure's CDN isn't closing connections prematurely.

[morrislee]

Would it be a good idea if it is a "submodule" and simply caches the image file to the disk via github, and then it writes to the disk then deletes it later? Or this is too much overhead for github terms and conditons?

[jakethesnake420]

The correct solution is to move this issue onto Microsoft Azure team if it's an issue with their service or fix the agnos staging code if there is an issue with that.

[adeebshihadeh]

6c02d5c