feat: implement Indexing Agreements #1134
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| # Still pending | ||
|
|
||
| * `require(provision.tokens != 0, DisputeManagerZeroTokens());` - Document or fix? | ||
| * Check code coverage | ||
| * Don't love cancel agreement on stop service / close stale allocation. | ||
| * Arbitration Charter: Update to support disputing IndexingFee. | ||
|
|
||
| # Done | ||
|
|
||
| * DONE: ~~* Remove extension if I can fit everything in one service?~~ | ||
| * DONE: ~~* One Interface for all subgraph~~ | ||
| * DONE: ~~* Missing Upgrade event for subgraph service~~ | ||
| * DONE: ~~* Check contract size~~ | ||
| * DONE: ~~Switch cancel event in recurring collector to use Enum~~ | ||
| * DONE: ~~Switch timestamps to uint64~~ | ||
| * DONE: ~~Check that UUID-v4 fits in `bytes16`~~ | ||
| * DONE: ~~Double check cancelation policy. Who can cancel when? Right now is either party at any time. Answer: If gateway cancels allow collection till that point.~~ | ||
| * DONE: ~~If an indexer closes an allocation, what should happen to the accepeted agreement? Answer: Look into canceling agreement as part of stop service.~~ | ||
| * DONE: ~~Switch `duration` for `endsAt`? Answer: Do it.~~ | ||
| * DONE: ~~Support a way for gateway to shop an agreement around? Deadline + dedup key? So only one agreement with the dedupe key can be accepted? Answer: No. Agreements will be "signaled" as approved or rejected on the API call that sends the agreement. We'll trust (and verify) that that's the case.~~ | ||
| * DONE: ~~Test `upgrade` paths~~ | ||
| * DONE: ~~Fix upgrade.t.sol, lots of comments~~ | ||
| * DONE: ~~How do we solve for the case where an indexer has reached their max expected payout for the initial sync but haven't reached the current epoch (thus their POI is incorrect)? Answer: Signal in the event that the max amount was collected, so that fisherman understand the case.~~ | ||
| * DONE: ~~Debate epoch check protocol team. Maybe don't revert but store it in event. Pablo suggest block number instead of epoch.~~ | ||
| * DONE: ~~Should we set a different param for initial collection time max? Some subgraphs take a lot to catch up. Answer: Do nothing. Make sure that zero POIs allow to eventually sync~~ | ||
| * DONE: ~~Since an allocation is required for collecting, do we want to expect that the allocation is not stale? Do we want to add code to collect rewards as part of the collection of fees? Make sure allocation is more than one epoch old if we attempt this. Answer: Ignore stale allocation~~ | ||
| * DONE: ~~If service wants to collect more than collector allows. Collector limits but doesn't tell the service? Currently reverts. Answer: Allow for max allowed~~ | ||
| * DONE: ~~What should happen if the escrow doesn't have enough funds? Answer: Reverts~~ | ||
| * DONE: ~~Don't pay for entities on initial collection? Where did we land in terms of payment terms? Answer: pay initial~~ | ||
| * DONE: ~~Test lock stake~~ | ||
| * DONE: ~~Reduce the number of errors declared and returned~~ | ||
| * DONE: ~~Support `DisputeManager`~~ | ||
| * DONE: ~~Check upgrade conditions. Support indexing agreement upgradability, so that there is a mechanism to adjust the rates without having to cancel and start over.~~ | ||
| * DONE: ~~Maybe check that the epoch the indexer is sending is the one the transaction will be run in?~~ | ||
| * DONE: ~~Should we deal with zero entities declared as a special case?~~ | ||
| * DONE: ~~Support for agreements that end up in `RecurringCollectorCollectionTooLate` or ways to avoid getting to that state.~~ | ||
| * DONE: ~~Make `agreementId` unique globally so that we don't need the full tuple (`payer`+`indexer`+`agreementId`) as key?~~ | ||
| * DONE: ~~Maybe IRecurringCollector.cancel(address payer, address serviceProvider, bytes16 agreementId) should only take in agreementId?~~ | ||
| * DONE: ~~Unify to one error in Decoder.sol~~ | ||
| * DONE: ~~Built-in upgrade path to indexing agreements v2~~ | ||
| * DONE: ~~Missing events for accept, cancel, upgrade RCAs.~~ | ||
|
|
||
| # Won't Fix | ||
|
|
||
| * Add upgrade path to v2 collector terms | ||
| * Expose a function that indexers can use to calculate the tokens to be collected and other collection params? | ||
| * Place all agreement terms into one struct | ||
| * It's more like a collect + cancel since the indexer is expected to stop work then and there. When posting a POI that's < N-1 epoch. Answer: Emit signal that the collection is meant to be final. Counter: Won't do since collector can't signal back to data service that payment is maxed out. Could emit an event from the collector, but is it really worth it? Right now any collection where epoch POI < current POI is suspect. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,131 @@ | ||||||
| // SPDX-License-Identifier: GPL-3.0-or-later | ||||||
| pragma solidity 0.8.27; | ||||||
|
|
||||||
| import { ProvisionTracker } from "./ProvisionTracker.sol"; | ||||||
| import { IDataServiceFees } from "../interfaces/IDataServiceFees.sol"; | ||||||
| import { IHorizonStaking } from "../../interfaces/IHorizonStaking.sol"; | ||||||
| import { LinkedList } from "../../libraries/LinkedList.sol"; | ||||||
|
|
||||||
| library DataServiceFeesLib { | ||||||
|
There was a problem hiding this comment.
Suggested change
How about we rename this? I'm personally not very fond of the There was a problem hiding this comment. Agreed, it was meant to be a placeholder until we agreed on the validity of the approach. |
||||||
| using ProvisionTracker for mapping(address => uint256); | ||||||
| using LinkedList for LinkedList.List; | ||||||
|
|
||||||
| /** | ||||||
| * @notice Locks stake for a service provider to back a payment. | ||||||
| * Creates a stake claim, which is stored in a linked list by service provider. | ||||||
| * @dev Requirements: | ||||||
| * - The associated provision must have enough available tokens to lock the stake. | ||||||
| * | ||||||
| * Emits a {StakeClaimLocked} event. | ||||||
| * | ||||||
| * @param feesProvisionTracker The mapping that tracks the provision tokens for each service provider | ||||||
| * @param claims The mapping that stores stake claims by their ID | ||||||
| * @param claimsLists The mapping that stores linked lists of stake claims by service provider | ||||||
| * @param graphStaking The Horizon staking contract used to lock the tokens | ||||||
| * @param _delegationRatio The delegation ratio to use for the stake claim | ||||||
| * @param _serviceProvider The address of the service provider | ||||||
| * @param _tokens The amount of tokens to lock in the claim | ||||||
| * @param _unlockTimestamp The timestamp when the tokens can be released | ||||||
| */ | ||||||
| function lockStake( | ||||||
| mapping(address => uint256) storage feesProvisionTracker, | ||||||
| mapping(bytes32 => IDataServiceFees.StakeClaim) storage claims, | ||||||
| mapping(address serviceProvider => LinkedList.List list) storage claimsLists, | ||||||
| IHorizonStaking graphStaking, | ||||||
| uint32 _delegationRatio, | ||||||
| address _serviceProvider, | ||||||
| uint256 _tokens, | ||||||
| uint256 _unlockTimestamp | ||||||
| ) external { | ||||||
| require(_tokens != 0, IDataServiceFees.DataServiceFeesZeroTokens()); | ||||||
| feesProvisionTracker.lock(graphStaking, _serviceProvider, _tokens, _delegationRatio); | ||||||
|
|
||||||
| LinkedList.List storage claimsList = claimsLists[_serviceProvider]; | ||||||
|
|
||||||
| // Save item and add to list | ||||||
| bytes32 claimId = _buildStakeClaimId(_serviceProvider, claimsList.nonce); | ||||||
| claims[claimId] = IDataServiceFees.StakeClaim({ | ||||||
| tokens: _tokens, | ||||||
| createdAt: block.timestamp, | ||||||
| releasableAt: _unlockTimestamp, | ||||||
| nextClaim: bytes32(0) | ||||||
| }); | ||||||
| if (claimsList.count != 0) claims[claimsList.tail].nextClaim = claimId; | ||||||
| claimsList.addTail(claimId); | ||||||
|
|
||||||
| emit IDataServiceFees.StakeClaimLocked(_serviceProvider, claimId, _tokens, _unlockTimestamp); | ||||||
| } | ||||||
|
|
||||||
| /** | ||||||
| * @notice Processes a stake claim, releasing the tokens if the claim has expired. | ||||||
| * @dev This function is used as a callback in the stake claims linked list traversal. | ||||||
| * @param feesProvisionTracker The mapping that tracks the provision tokens for each service provider. | ||||||
| * @param claims The mapping that stores stake claims by their ID. | ||||||
| * @param _claimId The ID of the stake claim to process. | ||||||
| * @param _acc The accumulator data, which contains the total tokens claimed and the service provider address. | ||||||
| * @return Whether the stake claim is still locked, indicating that the traversal should continue or stop. | ||||||
| * @return The updated accumulator data | ||||||
| */ | ||||||
| function processStakeClaim( | ||||||
| mapping(address serviceProvider => uint256 tokens) storage feesProvisionTracker, | ||||||
| mapping(bytes32 claimId => IDataServiceFees.StakeClaim claim) storage claims, | ||||||
| bytes32 _claimId, | ||||||
| bytes memory _acc | ||||||
| ) external returns (bool, bytes memory) { | ||||||
| IDataServiceFees.StakeClaim memory claim = claims[_claimId]; | ||||||
| require(claim.createdAt != 0, IDataServiceFees.DataServiceFeesClaimNotFound(_claimId)); | ||||||
|
|
||||||
| // early exit | ||||||
| if (claim.releasableAt > block.timestamp) { | ||||||
| return (true, LinkedList.NULL_BYTES); | ||||||
| } | ||||||
|
|
||||||
| // decode | ||||||
| (uint256 tokensClaimed, address serviceProvider) = abi.decode(_acc, (uint256, address)); | ||||||
|
|
||||||
| // process | ||||||
| feesProvisionTracker.release(serviceProvider, claim.tokens); | ||||||
| emit IDataServiceFees.StakeClaimReleased(serviceProvider, _claimId, claim.tokens, claim.releasableAt); | ||||||
|
|
||||||
| // encode | ||||||
| _acc = abi.encode(tokensClaimed + claim.tokens, serviceProvider); | ||||||
| return (false, _acc); | ||||||
| } | ||||||
|
|
||||||
| /** | ||||||
| * @notice Deletes a stake claim. | ||||||
| * @dev This function is used as a callback in the stake claims linked list traversal. | ||||||
| * @param claims The mapping that stores stake claims by their ID | ||||||
| * @param claimId The ID of the stake claim to delete | ||||||
| */ | ||||||
| function deleteStakeClaim( | ||||||
| mapping(bytes32 claimId => IDataServiceFees.StakeClaim claim) storage claims, | ||||||
| bytes32 claimId | ||||||
| ) external { | ||||||
| delete claims[claimId]; | ||||||
| } | ||||||
|
|
||||||
| /** | ||||||
| * @notice Gets the next stake claim in the linked list | ||||||
| * @dev This function is used as a callback in the stake claims linked list traversal. | ||||||
| * @param claims The mapping that stores stake claims by their ID | ||||||
| * @param claimId The ID of the stake claim | ||||||
| * @return The next stake claim ID | ||||||
| */ | ||||||
| function getNextStakeClaim( | ||||||
| mapping(bytes32 claimId => IDataServiceFees.StakeClaim claim) storage claims, | ||||||
| bytes32 claimId | ||||||
| ) external view returns (bytes32) { | ||||||
| return claims[claimId].nextClaim; | ||||||
| } | ||||||
|
|
||||||
| /** | ||||||
| * @notice Builds a stake claim ID | ||||||
| * @param serviceProvider The address of the service provider | ||||||
| * @param nonce A nonce of the stake claim | ||||||
| * @return The stake claim ID | ||||||
| */ | ||||||
| function _buildStakeClaimId(address serviceProvider, uint256 nonce) internal view returns (bytes32) { | ||||||
|
There was a problem hiding this comment. Since this is now a linked lib I think it could be useful to expose this internal function with an external getter as well. |
||||||
| return keccak256(abi.encodePacked(address(this), serviceProvider, nonce)); | ||||||
| } | ||||||
| } | ||||||
|
There was a problem hiding this comment. Changes to this file can now probably be reverted since we are at <23kb |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would remove the dependency with
IDataServiceFeesany event or error that we use from there I think should probably instead live in this lib contract. Note that if we move any errors into this file we would need to rename their prefixIDataServiceFees.DataServiceFeesClaimNotFound->StakeClaimsClaimNotFoundThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also the struct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah yes that makes sense. to avoid name collision we can rename the struct to
Statesimilar to what we have withAllocation.Stateand a few others.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
look at this last, maybe drop if not enough time or will power.