All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Chainlink header images to the following
README.mdfiles: root, core, evm-contracts, and evm-test-helpers. - Database migrations: new log_consumptions records will contain the number of the associated block. This migration will allow future version of chainlink to automatically clean up unneeded log_consumption records. This migration should execute very fast.
Previously when the node was overloaded with heads there was a minor possibility it could get backed up with a very large head queue, and become unstable. Now, we drop heads instead in this case and noisily emit an error. This means the node should more gracefully handle overload conditions, although this is still dangerous and node operators should deal with it immediately to avoid missing jobs.
A new environment variable is introduced to configure this, called ETH_HEAD_TRACKER_MAX_BUFFER_SIZE. It is recommended to leave this set to the default of "3".
A new prometheus metric is also introduced to track dropped heads, called head_tracker_num_heads_dropped. You may wish to set an alert on a rule such as increase(chainlink_dropped_heads[5m]) > 0.
- Performance improvements when using BulletproofTxManager.
- Added a workaround for Infura users who are seeing "error getting balance: header not found".
This behaviour is due to Infura announcing it has a block, but when we request our balance in this block, the eth node doesn't have the block in memory. The workaround is to add a configurable lag time on balance update requests. The default is set to 1 but this is configurable via a new environment variable
ETH_BALANCE_MONITOR_BLOCK_DELAY.
- Job specs now support pinning to multiple keys using the new
fromAddressesfield in the ethtx task spec.
- Using
fromAddressin ethtx task specs has been deprecated. Please usefromAddressesinstead.
- Support for RunLogTopic0original and RunLogTopic20190123withFullfillmentParams logs has been dropped. This should not affect any users since these logs predate Chainlink's mainnet launch and have never been used on mainnet.
IMPORTANT: The selection mechanism for keys has changed. When an ethtx task spec is not pinned to a particular key by defining fromAddress or fromAddresses, the node will now cycle through all available keys in round robin fashion. This is a change from the previous behaviour where nodes would only pick the earliest created key.
This is done to allow increases in throughput when a node operator has multiple whitelisted addresses for their oracle.
If your node has multiple keys, you will need to take one of the three following actions:
- Make sure all keys are valid for all job specs
- Pin job specs to a valid subset of key(s) using
fromAddresses - Delete the key(s) you don't want to use
If your node only has one key, no action is required.
- Incorrect sequence on keys table in some edge cases
- Added a check on sensitive file ownership that gives a warning if certain files are not owned by the user running chainlink
- Added mechanism to asynchronously communicate when a job spec has an ethereum interaction error (or any async error) with a UI screen
- Gas Bumper now bumps based on the current gas price instead of the gas price of the original transaction
- Support for multiple node addresses
ethtxtasks now support a new parameter,minRequiredOutgoingConfirmationswhich allows you to tune how many confirmations are required before moving on from anethtxtask on a per task basis (only works with BulletproofTxManager). If it is not supplied, the default ofMIN_OUTGOING_CONFIRMATIONSis used (same as the old behaviour).
- HeadTracker now automatically backfills missing heads up to
ETH_FINALITY_DEPTH - The strategy for gas bumping has been changed to produce a potentially higher gas cost in exchange for the transaction getting through faster.
admin withdrawcommand has been removed. This was only ever useful to withdraw LINK if the Oracle contract was owned by the Chainlink node address. It is no longer recommended to have the Oracle owner be the chainlink node address.- Fixed
txs createto send the amount in Eth not in Wei (as per the documentation)
This release contains a number of features aimed at improving the node's reliability when putting transactions on-chain.
- An experimental new transaction manager is introduced that delivers reliability improvements compared to the old one, especially when faced with difficult network conditions or spiking gas prices. It also reduces load on the database and makes fewer calls to the eth node compared to the old tx manager.
- Along with the new transaction manager is a local client command for manually controlling the node nonce -
setnextnonce. This should never be necessary under normal operation and is included only for use in emergencies. - New prometheus metrics for the head tracker:
head_tracker_heads_in_queue- The number of heads currently waiting to be executed. You can think of this as the 'load' on the head tracker. Should rarely or never be more than 0.head_tracker_callback_execution_duration- How long it took to execute all callbacks. If the average of this exceeds the time between blocks, your node could lag behind and delay transactions.
- Nodes transmit their build info to Explorer for better debugging/tracking.
ENABLE_BULLETPROOF_TX_MANAGER- set this to true to enable the experimental new transaction managerETH_GAS_BUMP_PERCENTdefault value has been increased from 10% to 20%ETH_GAS_BUMP_THRESHOLDdefault value has been decreased from 12 to 3ETH_FINALITY_DEPTHspecifies how deep protection should be against re-orgs. The default is 50. It only applies if BulletproofTxManager is enabled. It is not recommended to change this setting.EthHeadTrackerHistoryDepthspecifies how many heads the head tracker should keep in the database. The default is 100. It is not recommended to change this setting.- Update README.md with links to mockery, jq, and gencodec as they are required to run
go generate ./...
- The node now logs the eth client RPC calls
- More reliable Ethereum block header tracking
- Limit the amount of an HTTP response body that the node will read
- Make Aggregator contract interface viewable
- More resilient handling of chain reorganizations
- The chainlink node can now be configured to backfill logs from
nblocks after a connection to the ethereum client is reset. This value is specified with an environment variableBLOCK_BACKFILL_DEPTH. - The chainlink node now sets file permissions on sensitive files on startup (tls, .api, .env, .password and secret)
- AggregatorInterface now has description and version fields.
- Solidity: Renamed the previous
AggregatorInterface.soltoHistoricAggregatorInterface.sol. Users are encouraged to use the new methods introduced on theAggregatorInterface(getRoundDataandlatestRoundData), as they return metadata to indicate freshness of the data in a single cross-contract call. - Solidity: Marked
HistoricAggregatorInterfacemethods (latestAnswer,latestRound,latestTimestamp,getAnswer,getTimestamp) as deprecated onFluxAggregator,WhitelistedAggregator,AggregatorProxy,WhitelistedAggregatorProxy. - Updated the solidity compiler version for v0.6 from 0.6.2 to 0.6.6.
- AccessControlledAggregatorProxy checks an external contract for users to be able to read functions.
- Fluxmonitor jobs now respect the
minPaymentfield on job specs and won't poll if the contract does not have sufficient funding. This allows certain jobs to require a larger payment thanMINIMUM_CONTRACT_PAYMENT.
- Fluxmonitor initiators may now optionally include an
absoluteThresholdparameter. To trigger a new on-chain report, the absolute difference in the feed value must change by at least theabsoluteThresholdvalue. If it is unspecified or zero, fluxmonitor behavior is unchanged. - Database Migrations: Add created_at and updated_at to all tables allowing for better historical insights. This migration may take a minute or two on large databases.
- Fix incorrect permissions on some files written by the node Prevent a case where duplicate ethereum keys could be added Improve robustness and reliability of ethtx transaction logic
- Added Changelog.
- Database Migrations: There a number of database migrations included in this release as part of our ongoing effort to make the node even more reliable and stable, and build a firm foundation for future development.
- New cron strings MUST now include time zone. If you want your jobs to run in
UTC for example:
CRON_TZ=UTC * * * * *. Previously, jobs specified without a time zone would run in the server's native time zone, which in most cases is UTC but this was never guaranteed.
- Fix crash in experimental gas updater when run on Kovan network