Use web3 for gas calculations usage/price #609

akremstudy · 2023-03-01T13:45:24Z

Summary

Closes #602
Closes #362
Closes #355
Closes #367

in command/report.py:

removed speed flag used for gas api endpoint responses.
add gas multiplier to up the gas price returned by web3 (optional)

in reporters/interval.py:

change fetch_gas_price function to fetch gas price from web3 api instead of other apis' and returns a float to maintain the decimals especially when the number is close to zero. The returned number is increased by the multiplier (default increase by 1 percent)
switched flow between ensure_profitability and gas calculations to instead build transaction function, get gas price, then call ensure_profitability which checks if there is profit based on the carried over gas price and gas limit and the rest of the other calculations ie token prices.

in reporters/rng_interval.py:

remove the profit calculations from the file since thats already in the parent class. It looks like it was there because native tokens weren't checked for profitability but now they are.

in reporters/tellorflex.py:

removed the method since its no different than the parent class
adapted the changes for ensure_profitability.

in utils/reporter_utils.py:

added function tkn_symbol that returns native token symbol for viewing in the cli when seeing cost info

in tests/reporters/test_flex_reporter and test_interval_reporter:

changes to some tests that add the gas_info attribute

Steps Taken to QA Changes

Ran all the tests/ and they all pass. (some tests need to be ran individually to pass)
transactions executed with these flags: -qt eth-usd-spot -p YOLO
Flashbot txn
goerli
arbitrum
optimism
mumbai

Checklist

This pull request is:

A documentation error, docs update, or typographical error fix
- No tests or issue needed
A code fix
- Please reference the related issue by including "Closes <link to issue>" in this Pull Request's summary section.
  - If no issue exists, please create a bug report issue
- Please include tests. Fixes without tests will not be accepted unless it's related to the documentation only.
- Please make sure docs are updated if need be
A feature implementation
- Please reference the related issue by including "Closes <link to issue>" in this Pull Request's summary section.
  - If no issue exists, please create a feature enhancement issue
- Please include tests
- Please make sure docs updates are both thorough and easy to reproduce by somebody with limited knowledge of the feature that you are submitting

Happy engineering!

oraclown

Thanks for this @akremstudy ! Lots of great updates. I added a few suggestions as comments. And here's what I found when testing:

Testing manually

I'm having trouble reporting w/ the following command: telliot report -a dev-acct-bags -gm 1 -tx 2 -ncr
It gets stuck printing this over and over:

ERROR   | telliot_feeds.reporters.tips.listener.funded_feeds | Error getting supported feeds
ERROR   | telliot_feeds.reporters.tips.listener.funded_feeds | Error getting supported feeds
ERROR   | telliot_feeds.reporters.tips.listener.funded_feeds | Error getting eligible funded feeds: No funded feeds returned by autopay function call
ERROR   | telliot_feeds.reporters.tips.listener.funded_feeds | Error getting eligible funded feeds: No funded feeds returned by autopay function call
INFO    | telliot_feeds.reporters.interval | Unable to suggest datafeed
INFO    | telliot_feeds.reporters.interval | Sleeping for 7 seconds

I was able to report using this command: telliot report -a dev-acct-bags -gm 1 -tx 2 -ncr -rf, which skips getting the feed from autopay.

INFO    | telliot_feeds.reporters.interval | Current query: {"type":"SpotPrice","asset":"bch","currency":"usd"}
WARNING | telliot_feeds.sources.price.spot.kraken | Asset not supported: BCH
INFO    | telliot_feeds.sources.price_aggregator | Running median on [130.92, 130.93, 130.85]
INFO    | telliot_feeds.sources.price_aggregator | Feed Price: 130.92 reported at time 2023-03-02 18:38:11.668299+00:00
INFO    | telliot_feeds.sources.price_aggregator | Number of sources used in aggregate: 3
DEBUG   | telliot_feeds.reporters.interval | IntervalReporter Encoded value: 00000000000000000000000000000000000000000000000718e1465b2a23cf26
DEBUG   | telliot_feeds.reporters.interval | Sending submitValue transaction
INFO    | telliot_feeds.reporters.interval | View reported data:

submitValue tx here
I'd like to see the new tx info log output, but the first command I posted above (w/o -ncr flag) doesn't get to ensure_profitable bc of the datafeed fetch error. Is it actually an error getting it from autopay, or that there's just no funded feeds? Also, why does it log that message twice?

Checking local test runs

The following tests are failing for me (also when run individually):

360 reporter

FAILED tests/reporters/test_360_reporter.py::test_report - assert 0 == 1000000000000000000
FAILED tests/reporters/test_360_reporter.py::test_adding_stake - AssertionError: assert False
FAILED tests/reporters/test_360_reporter.py::test_checks_reporter_lock_when_manual_source - AssertionError: assert False

evm call

FAILED tests/reporters/test_evm_call.py::test_evm_call_e2e - assert 'Current query: {"type":"EVMCall","chainId":1,"contractAddress":"0x88d' in 'INFO telliot_feed...
FAILED tests/reporters/test_evm_call.py::test_no_endpoint_for_tipped_chain - AssertionError: assert 'Endpoint not found for chain_id=123456789' in 'INFO telliot_...

rng reporter

FAILED tests/reporters/test_rng_reporter.py::test_invalid_timestamp - AssertionError: assert False
FAILED tests/reporters/test_rng_reporter.py::test_invalid_timestamp_in_future - AssertionError: assert False

src/telliot_feeds/reporters/flashbot.py

src/telliot_feeds/reporters/interval.py

src/telliot_feeds/reporters/rng_interval.py

src/telliot_feeds/reporters/tellor_360.py

src/telliot_feeds/reporters/tellor_flex.py

src/telliot_feeds/utils/reporter_utils.py

akremstudy · 2023-03-04T02:13:55Z

Thanks for this @akremstudy ! Lots of great updates. I added a few suggestions as comments. And here's what I found when testing:

Testing manually

I'm having trouble reporting w/ the following command: telliot report -a dev-acct-bags -gm 1 -tx 2 -ncr

It gets stuck printing this over and over:
ERROR   | telliot_feeds.reporters.tips.listener.funded_feeds | Error getting supported feeds
ERROR   | telliot_feeds.reporters.tips.listener.funded_feeds | Error getting supported feeds
ERROR   | telliot_feeds.reporters.tips.listener.funded_feeds | Error getting eligible funded feeds: No funded feeds returned by autopay function call
ERROR   | telliot_feeds.reporters.tips.listener.funded_feeds | Error getting eligible funded feeds: No funded feeds returned by autopay function call
INFO    | telliot_feeds.reporters.interval | Unable to suggest datafeed
INFO    | telliot_feeds.reporters.interval | Sleeping for 7 seconds
I was able to report using this command: telliot report -a dev-acct-bags -gm 1 -tx 2 -ncr -rf, which skips getting the feed from autopay.
INFO    | telliot_feeds.reporters.interval | Current query: {"type":"SpotPrice","asset":"bch","currency":"usd"}
WARNING | telliot_feeds.sources.price.spot.kraken | Asset not supported: BCH
INFO    | telliot_feeds.sources.price_aggregator | Running median on [130.92, 130.93, 130.85]
INFO    | telliot_feeds.sources.price_aggregator | Feed Price: 130.92 reported at time 2023-03-02 18:38:11.668299+00:00
INFO    | telliot_feeds.sources.price_aggregator | Number of sources used in aggregate: 3
DEBUG   | telliot_feeds.reporters.interval | IntervalReporter Encoded value: 00000000000000000000000000000000000000000000000718e1465b2a23cf26
DEBUG   | telliot_feeds.reporters.interval | Sending submitValue transaction
INFO    | telliot_feeds.reporters.interval | View reported data:
submitValue tx here

I'd like to see the new tx info log output, but the first command I posted above (w/o -ncr flag) doesn't get to ensure_profitable bc of the datafeed fetch error. Is it actually an error getting it from autopay, or that there's just no funded feeds? Also, why does it log that message twice?

Checking local test runs

The following tests are failing for me (also when run individually):

360 reporter

FAILED tests/reporters/test_360_reporter.py::test_report - assert 0 == 1000000000000000000

FAILED tests/reporters/test_360_reporter.py::test_adding_stake - AssertionError: assert False

FAILED tests/reporters/test_360_reporter.py::test_checks_reporter_lock_when_manual_source - AssertionError: assert False

evm call

FAILED tests/reporters/test_evm_call.py::test_evm_call_e2e - assert 'Current query: {"type":"EVMCall","chainId":1,"contractAddress":"0x88d' in 'INFO telliot_feed...

FAILED tests/reporters/test_evm_call.py::test_no_endpoint_for_tipped_chain - AssertionError: assert 'Endpoint not found for chain_id=123456789' in 'INFO telliot_...

rng reporter

FAILED tests/reporters/test_rng_reporter.py::test_invalid_timestamp - AssertionError: assert False

FAILED tests/reporters/test_rng_reporter.py::test_invalid_timestamp_in_future - AssertionError: assert False

tests are passing for me. I think it might be failing cause it depends on the telliot-core changes. can you pip install telliot-core then try again. I just merged the new changes.

for the error messages, thats cause there are not tips. also i was checking the behavior with the main branch and it doesn't seem that the behavior changed because of the additions in this branch.

codecov-commenter · 2023-03-04T02:33:59Z

Codecov Report

Merging #609 (c24e3c8) into main (d5cf93d) will decrease coverage by 0.02%.
The diff coverage is 0.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main     #609      +/-   ##
==========================================
- Coverage   30.87%   30.86%   -0.02%     
==========================================
  Files          99       99              
  Lines        3459     3457       -2     
==========================================
- Hits         1068     1067       -1     
+ Misses       2391     2390       -1

Impacted Files	Coverage Δ
tests/reporters/test_flex_reporter.py	`27.35% <0.00%> (+0.50%)`	⬆️
tests/reporters/test_interval_reporter.py	`25.17% <0.00%> (-0.70%)`	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

oraclown

Thanks for the updates @akremstudy

akremstudy · 2023-03-08T13:27:15Z

Hey @oraclown, I made a few changes after your review. Mainly the type 2 transaction fees now are generated using web3 instead of the etherscan api. Also made type 2 transactions the default, and a minor fix to get rid of the double logging that was happening with the tip listener.

oraclown

@akremstudy Changes look good! Thanks a lot for these

Use web3 for gas calculations usage/price

1226224

oraclown requested changes Mar 2, 2023

View reviewed changes

suggested changes

59be856

akremstudy added 2 commits March 7, 2023 09:33

Merge branch 'main' into fetch-gasinfo-via-web3

df5a4bc

fix bug

65fe00b

oraclown approved these changes Mar 7, 2023

View reviewed changes

akremstudy added 3 commits March 7, 2023 11:35

logic is repeated so removed extra password prompt

c24e3c8

Calculate type 2 gas fees using web3

d97d573

fix double logging in tip listener

396aab0

akremstudy requested a review from oraclown March 8, 2023 13:27

oraclown approved these changes Mar 8, 2023

View reviewed changes

oraclown merged commit 7e49e8e into main Mar 8, 2023

oraclown deleted the fetch-gasinfo-via-web3 branch March 8, 2023 17:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use web3 for gas calculations usage/price #609

Use web3 for gas calculations usage/price #609

akremstudy commented Mar 1, 2023 •

edited

Loading

oraclown left a comment

akremstudy commented Mar 4, 2023

Testing manually

Checking local test runs

360 reporter

evm call

rng reporter

codecov-commenter commented Mar 4, 2023 •

edited

Loading

oraclown left a comment

akremstudy commented Mar 8, 2023

oraclown left a comment

Use web3 for gas calculations usage/price #609

Use web3 for gas calculations usage/price #609

Conversation

akremstudy commented Mar 1, 2023 • edited Loading

Summary

Steps Taken to QA Changes

Checklist

oraclown left a comment

Choose a reason for hiding this comment

Testing manually

Checking local test runs

360 reporter

evm call

rng reporter

akremstudy commented Mar 4, 2023

Testing manually

Checking local test runs

360 reporter

evm call

rng reporter

codecov-commenter commented Mar 4, 2023 • edited Loading

Codecov Report

oraclown left a comment

Choose a reason for hiding this comment

akremstudy commented Mar 8, 2023

oraclown left a comment

Choose a reason for hiding this comment

akremstudy commented Mar 1, 2023 •

edited

Loading

codecov-commenter commented Mar 4, 2023 •

edited

Loading