Initial Archiver Service Implementation #1
Conversation
Co-authored-by: Qi Wu <[email protected]>
danyalprout
force-pushed
the
initial-archiver
branch
from
4977c6a to
8b58d7e
Compare
Enable HTTP Healthcheck for Archvier
danyalprout
force-pushed
the
initial-archiver
branch
4 times, most recently
from
d90dd45 to
3674105
Compare
danyalprout
force-pushed
the
initial-archiver
branch
from
3674105 to
7927f72
Compare
| if err != nil { | ||
| return nil, newIndicesError(index) | ||
| } | ||
| blobIndex := deneb.BlobIndex(parsedInt) |
There was a problem hiding this comment.
would return error here if the blobIndex is out of range, or catch any unresolved indices in the later loop.
There was a problem hiding this comment.
My intention here is to mirror the CL client behavior as close as possible, if the index is out of range/duplicated etc it's just ignored (at least on Lighthouse)
❯ curl -s "<cl-url>/eth/v1/beacon/blob_sidecars/0x8f0d0ca5a16fc064514574dddde8d11b39e81f29b642d9f8b149a369b7c9fa9e?indices=10"
{"data":[]}%
❯ curl -s "<cl-url>/eth/v1/beacon/blob_sidecars/0x8f0d0ca5a16fc064514574dddde8d11b39e81f29b642d9f8b149a369b7c9fa9e?indices=1,1,10"
#.. returns a single blob at index 1
There was a problem hiding this comment.
is that in the spec or is that just what Lighthouse does? Duplicated I get, out of range seems worth alerting the caller over.
There was a problem hiding this comment.
It's the behavior of lighthouse, the spec doesn't specify :D (it's very possible there's a more detailed spec I'm missing though).
| // Start starts the archiver service. It begins polling the beacon node for the latest blocks and persisting blobs for | ||
| // them. Concurrently it'll also begin a backfill process (see backfillBlobs) to store all blobs from the current head | ||
| // to the previously stored blocks. This ensures that during restarts or outages of an archiver, any gaps will be | ||
| // filled in. |
There was a problem hiding this comment.
What happens if the backfill period exceeds the L1 retention window?
There was a problem hiding this comment.
It'll treat the beacon node of the source of truth and keep walking back, writing empty blobs until it hits the last known block.
I wonder if we should extend this to not fetch blobs older than the blob expiry window (though we would have to make it configurable as some people may adjust the pruning time on their CL nodes).
There was a problem hiding this comment.
Right we don't know how the beacon node is configured. I think this is fine for now, just make sure the behavior is documented. We can only archive what's there :)
|
Overall LGTM modulo my mostly cosmetic comments, thank you for doing this! |
cb-heimdall
dismissed
roberto-bayardo’s stale review
Approved review 1881782545 from roberto-bayardo is now dismissed due to new commit. Re-request for approval.
Description
This PR creates the archiver service. It consists of two components the
archiverand theapi.archivercontinuously polls the beacon chain and writes any blobs it finds to the configured data store.apiuses this datastore and the beacon client to resolve get blob sidecar requestsThis image shows a visualization of how the key components interact.
