Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Competitor AMMs KPIs #61

Merged
merged 25 commits into from
Nov 27, 2024
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
fcdb0bd
New queries to measure the kpis from competitors
PoloX2021 Nov 6, 2024
7bb4668
new parameter added because of the previous queries
PoloX2021 Nov 6, 2024
dd0dccc
Excludes possible negative tvls because of issues with ERC20/ETH tran…
PoloX2021 Nov 6, 2024
45d3f2a
Formatting
PoloX2021 Nov 6, 2024
752bf52
Added a missing blockchain parameter
PoloX2021 Nov 6, 2024
9fc431a
Simplified the querying of all tranactions from a curve pool
PoloX2021 Nov 15, 2024
7060d6d
Few comments and adapting to remarks for Curve
PoloX2021 Nov 15, 2024
db8459a
Added filter by x largest pools
PoloX2021 Nov 15, 2024
cb41f52
Added/propagated through queries number_of_pools parameter and variou…
PoloX2021 Nov 18, 2024
27ac2db
Removed the union for inactive pools. THe left join does the work
PoloX2021 Nov 19, 2024
1f328eb
Fixed the sqlfluff
PoloX2021 Nov 19, 2024
0bf6622
Moved the folder to kpis/competitors
PoloX2021 Nov 20, 2024
0d0f7f5
New query to replace the uni style querying. they are now grouped in …
PoloX2021 Nov 20, 2024
72f5c6a
Added the project value to classify the pools: uni/ pancake/sushi
PoloX2021 Nov 22, 2024
73a55b2
Comments
PoloX2021 Nov 22, 2024
77c8121
Added parameter for time window, and removed avg(tvl), avg(fee)
PoloX2021 Nov 22, 2024
a6478fa
upadated default parameter
PoloX2021 Nov 22, 2024
d53d5c0
Added syncs inline to kpis.
PoloX2021 Nov 22, 2024
7f4dbb9
Added flag uni/pancake/Sushi and parameter for time window
PoloX2021 Nov 22, 2024
5b26df3
New default value for day, few comments and renaming
PoloX2021 Nov 26, 2024
cd66d9f
Changed deployer contract addres for uniswap
PoloX2021 Nov 26, 2024
694d5cf
Query to regroup all the different projects kpis together
PoloX2021 Nov 26, 2024
df49fb1
Typo for chain_id
PoloX2021 Nov 26, 2024
f162872
fixed typo and added base
PoloX2021 Nov 26, 2024
f08a4b5
updated kpis to filter before joining and improve performance
PoloX2021 Nov 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cowamm/.sqlfluff
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,5 @@ start_time='2024-08-20 00:00:00'
end_time='2024-08-27 00:00:00'
results=final_results_per_solver,cow_surplus_per_batch
cow_budget=30000
number_of_pools=500
end_time = 2024-01-01
25 changes: 25 additions & 0 deletions cowamm/kpi/competitors/curve/curve_kpis_4232873.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
-- Computes volume, tvl and APR for Curve pools
-- APR is measured as the fees earned per $ invested, over the last 24 hours, projected over 1 year
-- Parameters:
-- {{blockchain}}: The blockchain to query
-- {{end_time}}: The end time of the time window (end_time - 1 day; end_time), defaults to now()
select
r1.contract_address,
r1.fee,
r1.tvl,
sum(amount_usd) as volume,
365 * sum(amount_usd * r.fee / r.tvl) as apr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation, I get it now. Just as a comment (not actionable assuming performance of the join isn't an issue), but this could also be computed using a "rolling sum" which would avoid the self join.

Something like

select
    contract_address,
    fee,
    tvl,
    sum(amount_usd) over (partition by contract_address order by latest_per_pool) as volume,
    365 * sum(amount_usd * fee / tvl) over (partition by contract_address order by latest_per_pool) as apr

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is indeed a performance issue overall for all queries. So I implemented it. Result: it divides by 2 the execution time

from "query_4232976(blockchain='{{blockchain}}')" as r
inner join "query_4232976(blockchain='{{blockchain}}')" as r1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this self join needed? Can you please document it in the code.

on
r.contract_address = r1.contract_address
left join curve.trades as t
on
r.contract_address = t.project_contract_address
and r.tx_hash = t.tx_hash
where
t.block_time >= date_add('day', -1, (case when '{{end_time}}' = '2024-01-01' then now() else timestamp '{{end_time}}' end))
and r1.latest = 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4232976 already filters for latest = 1 doesn't it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it might feel confusing in the last part of the code: it filters by latest_per_pool only to find the largest pools, then it includes all of the tvls after every event which happened

select * from recent_tvl where contract_address in ( select contract_address from recent_tvl where latest_per_pool = 1 order by tvl desc limit {{number_of_pools}} )

-- This test avoids any possible issue with reconstructing the reserves of the pool
and r1.tvl > 0
group by 1, 2, 3
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
-- Finds all the curve pools with 2 tokens and their TVLs at every moment in time
-- For every pool that is currently one of the {{number_of_pools}} largest by TVL:
-- It returns their TVL at every moment in time
-- Parameters:
-- {{blockchain}}: The blockchain to query
-- {{number_of_pools}}: The number of largest pools to return

with
-- filters pools with 2 tokens
pools as (
select
pool_address as contract_address,
coin0 as token0,
coin1 as token1,
mid_fee * power(10, -10) as fee
from curvefi_{{blockchain}}.view_pools
-- Curve pools can have more than 2 token, if coin2 = 0x00 it means there are 2 tokens at most
where coin2 = 0x0000000000000000000000000000000000000000
),

-- finds all transfers in and out of the pools to rebuild the reserves
transfers as (
select
p.contract_address,
token0,
token1,
evt_block_time as block_time,
evt_index,
evt_tx_hash as tx_hash,
fee,
case
when t.contract_address = token0 and "from" = p.contract_address then -value
when t.contract_address = token0 and to = p.contract_address then value else 0
end as transfer0,
case
when t.contract_address = token1 and "from" = p.contract_address then -value
when t.contract_address = token1 and to = p.contract_address then value else 0
end as transfer1
from erc20_{{blockchain}}.evt_transfer as t
inner join pools as p
on
(
t."from" = p.contract_address
or t.to = p.contract_address
)
and (token0 = t.contract_address or token1 = t.contract_address)
),

-- rebuilds the reserves from the transfers
-- ETH transfers are not considered
reserves as (
select
contract_address,
token0,
token1,
tx_hash,
block_time,
fee,
sum(transfer0) over (partition by contract_address order by block_time, evt_index) as reserve0,
sum(transfer1) over (partition by contract_address order by block_time, evt_index) as reserve1,
Comment on lines +59 to +60
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the performance of the query and the fact that we are already joining with prices for sell and buy token here, I wonder if we wouldn't dramatically speed up this query by also computing the amount_usd per tx_hash here instead of computing it in the other query.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it would speed up or not, because we would have to look for the transaction events and not simply the transfers

row_number() over (partition by tx_hash, contract_address order by evt_index desc) as row_num,
row_number() over (partition by contract_address order by block_time desc) as latest
from transfers
),

-- finds the TVL of the pools
recent_tvl as (
select
r.contract_address,
token0,
token1,
block_time,
tx_hash,
reserve0,
reserve1,
fee,
latest,
(reserve0 * p0.price / pow(10, p0.decimals)) + (reserve1 * p1.price / pow(10, p1.decimals)) as tvl
from reserves as r
inner join prices.usd as p0
on
date_trunc('minute', block_time) = p0.minute
and token0 = p0.contract_address
inner join prices.usd as p1
on
date_trunc('minute', block_time) = p1.minute
and token1 = p1.contract_address
where row_num = 1
)


select * from recent_tvl
where contract_address in (
select contract_address
from recent_tvl
where latest = 1
order by tvl desc
limit {{number_of_pools}}
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
-- Computes volume, tvl and APR for Uni Swap style pools (Uni, Pancake, Sushi)
-- APR is measured as the fees earned per $ invested, over the last 24 hours, projected over 1 year
-- The type of pool (Uni/Pancake/Sushi) is not specified
-- Parameters:
-- {{blockchain}}: The blockchain to query
-- {{number_of_pools}}: The number of largest pools to return
-- {{end_time}}: The end time of the time window (end_time - 1 day; end_time), defaults to now()

-- select the pool with the largest latest k
with pool as (
select
pool_address as contract_address,
project,
token0,
token1,
tvl
from "query_4303563(blockchain='{{blockchain}}', number_of_pools = '{{number_of_pools}}')"
),

syncs as (
select
pool.*,
tx_hash as evt_tx_hash,
index as evt_index,
block_time as evt_block_time,
block_number as evt_block_number,
varbinary_to_uint256(substr(data, 1, 32)) as reserve0,
varbinary_to_uint256(substr(data, 33, 32)) as reserve1,
rank() over (partition by (logs.contract_address) order by block_time desc) as latest
from {{blockchain}}.logs
inner join pool
on logs.contract_address = pool.contract_address
where
block_time >= date_add('day', -1, (case when '{{end_time}}' = '2024-01-01' then now() else timestamp '{{end_time}}' end))
and topic0 = 0x1c411e9a96e071241c2f21f7726b17ae89e3cab4c78be50e062b03a9fffbbad1 -- Sync
),

swaps as (
select
tx_hash as evt_tx_hash,
index as evt_index,
block_time as evt_block_time,
block_number as evt_block_number,
contract_address,
varbinary_to_uint256(substr(data, 1, 32)) as amount0In,
varbinary_to_uint256(substr(data, 33, 32)) as amount1In,
varbinary_to_uint256(substr(data, 65, 32)) as amount0Out,
varbinary_to_uint256(substr(data, 97, 32)) as amount1Out
from {{blockchain}}.logs
where
block_time >= date(date_add('day', -1, now()))
and topic0 = 0xd78ad95fa46c994b6551d0da85fc275fe613ce37657fb8d5e3d130840159d822 -- Swap
and contract_address in (select contract_address from pool)
),

-- gets the swapped volume and tvl at the time of the swap for each swap
tvl_volume_per_swap as (
select
syncs.contract_address,
syncs.evt_block_time,
syncs.evt_tx_hash,
(amount0In * p0.price / pow(10, p0.decimals)) + (amount1In * p1.price / pow(10, p1.decimals)) as volume_in,
(amount0Out * p0.price / pow(10, p0.decimals)) + (amount1Out * p1.price / pow(10, p1.decimals)) as volume_out,
(reserve0 * p0.price / pow(10, p0.decimals)) + (reserve1 * p1.price / pow(10, p1.decimals)) as tvl
from syncs
inner join swaps
on
syncs.evt_tx_hash = swaps.evt_tx_hash
and syncs.contract_address = swaps.contract_address
and syncs.evt_index + 1 = swaps.evt_index
inner join pool
on syncs.contract_address = pool.contract_address
inner join prices.usd as p0
on
date_trunc('minute', syncs.evt_block_time) = p0.minute
and syncs.token0 = p0.contract_address
inner join prices.usd as p1
on
date_trunc('minute', syncs.evt_block_time) = p1.minute
and syncs.token1 = p1.contract_address
)

select
pool.contract_address,
project,
case
when project = 'pancakeswap' then 0.0025
else .003
end as fee,
coalesce(sum((volume_in + volume_out) / 2), 0) as volume,
-- the average pool is conceptually unnecessary because the table pool has only one tvl per pool
-- but it is necessary for the group by statement
avg(pool.tvl) as tvl,
case
when project = 'pancakeswap' then coalesce(365 * sum((volume_in + volume_out) / 2 / t.tvl) * 0.0025, 0)
else coalesce(365 * sum((volume_in + volume_out) / 2 / t.tvl) * 0.003, 0)
end as apr
from pool
left join tvl_volume_per_swap as t
on pool.contract_address = t.contract_address
group by pool.contract_address, project
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
-- Finds the largest Uni Style pools (Pancake, Sushi, Uni) and their TVLs
-- Parameters:
-- {{blockchain}}: The blockchain to query
-- {{number_of_pools}}: The number of largest pools to return

with pools as (
select
substr(data, 13, 20) as contract_address,
substr(topic1, 13, 20) as token0,
substr(topic2, 13, 20) as token1,
case
when contract_address = 0x1097053Fd2ea711dad45caCcc45EfF7548fCB362 then 'pancakeswap'
when contract_address = 0xDBf403764Ba35552b8Be25d1583Bd1BB540d1d8a then 'uniswap'
when contract_address = 0xC0AEe478e3658e2610c5F7A4A2E1777cE9e4f2Ac then 'sushiswap'
end as project
from {{blockchain}}.logs
where
topic0 = 0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9
and contract_address in
(
0x1097053Fd2ea711dad45caCcc45EfF7548fCB362,
0xDBf403764Ba35552b8Be25d1583Bd1BB540d1d8a,
0xC0AEe478e3658e2610c5F7A4A2E1777cE9e4f2Ac
)
),

syncs as (
select
pools.*,
tx_hash as evt_tx_hash,
index as evt_index,
block_number as evt_block_number,
date_trunc('minute', block_time) as evt_block_time,
varbinary_to_uint256(substr(data, 1, 32)) as reserve0,
varbinary_to_uint256(substr(data, 33, 32)) as reserve1,
rank() over (partition by (logs.contract_address) order by block_time desc, index asc) as latest
from {{blockchain}}.logs
inner join pools
on logs.contract_address = pools.contract_address
where
topic0 = 0x1c411e9a96e071241c2f21f7726b17ae89e3cab4c78be50e062b03a9fffbbad1 -- Sync
)

select distinct
s.contract_address as pool_address,
project,
token0,
token1,
reserve0,
reserve1,
evt_block_time,
reserve0 * p0.price * power(10, -p0.decimals) + reserve1 * p1.price * power(10, -p1.decimals) as tvl
from syncs as s
inner join prices.usd as p0
on
token0 = p0.contract_address
and p0.minute = evt_block_time
inner join prices.usd as p1
on
token1 = p1.contract_address
and p1.minute = evt_block_time
where latest = 1
order by 7 desc
limit {{number_of_pools}}
Loading