Job Pipeline

The core functionality of a single Chainlink node is to run a set of sequential processes, feeding the results of each process into the subsequent process, until a final result is reached. This is known as the "job pipeline."

When creating a job specifcation, you must declare the job's Initiators and the job's Tasks. Initiators are the entry points for the job pipeline, and are covered in more detail here.

Task specifications define the actual processes that will be run in order to produce a result. Every task spec has a type, and many take additional parameters to specify their behavior. Each task type is handled by an Adapter which specializes in running the computations associated with each task.

The result of each Adapter's computation is passed on as the input to the next adapter, until the final adapter is reached and its result is the result of the entire Job Run.

By using a modular pipeline of tasks and adapters, Chainlink creates a modular pipeline. This design makes it easy to leverage common logic across different computations. The full list of Adapters can be found on the Adapters page. Details on plugging in custom computations can be found on the External Adapters page.

Example

If you find the above description to be too abstract, it may help to look at a concrete example. Take the common job spec:

{
  "initiators": [{ "type": "web" }],
  "tasks": [
    { "type": "HttpGet", "url": "https://bitstamp.net/api/ticker/" },
    { "type": "JsonParse", "path": ["last"] },
    { "type": "EthBytes32" },
    {
      "type": "EthTx",
      "address": "0x356a04bce728ba4c62a30294a55e6a8600a320b3",
      "functionSelector": "0x609ff1bd"
    }
  ]
}

This job spec has a single initiator, web, which means new runs can be created via POST requests to the Chainlink node. Once a new run is created, it starts processing the job pipeline with the first task, in this case HttpGet.

HttpGet performs a GET request to the url parameter, which may respond with {"last":"11394.86","timestamp":"1519087731","volume": "12229.25870700"}. That response becomes the result of the HttpGet adapter, which is passed on to the JsonParse adapter.

JsonParse takes the input, parses it into JSON and looks up the value in the JSON path specified as the path parameter. In this case, it will find "11394.86" for the last key, and pass that along as the result of this task to the EthBytes32 adapter.

EthBytes32 takes the input, converts it to a string and then formats that string into Solidity's Bytes32 format. In this example, "11394.86" would be formatted as "0x31313339342e3836000000000000000000000000000000000000000000000000", which is passed on to the EthTx adapter.

EthTx is an adapter which will write transactions to the Ethereum blockchain. It sends the transaction to the Ethereum account specified in the address field. Additionally, it selects which function to call on an Ethereum contract with the functionSelector field. Once EthTx has built up and signed a transaction it sends that transaction into the blockchain and returns the transaction hash as the result. Being the last task in this job, the resulting transaction hash is also the result of the entire job run.

Run Result Format

The run result is a simple JSON object, made up of 3 fields: output, error, and pending.

output holds a JSON object which is where the data which is passed between adapters is held. By default, the value key holds the primary result from each adapter, as a string. Other keys/values can be added to output as needed. When getting values from output, it is the adapter reading the value whose responsibility it is to make sure the value is the right format.

error is null by default, and if an error is found there then the run is assumed to have errored out, and does not proceed.

pending is a boolean, which is assumed to be false if not set. Task Runs can be marked pending, if they take a long time to complete or are blocked. It is up to the adapter which marked it pending to resume the task.