Ethereum is "the World Computer". That’s one of the more common descriptions of the Ethereum platform. But what does that mean? Let’s try to start with a computer science focused description, and then try to decipher that with a more practical analysis of Ethereum’s capabilities and characteristics, while comparing it to bitcoin and other distributed ledger technologies (that - for simplicity’s sake - we will often refer to as "blockchains").
From a computer science perspective, Ethereum is a deterministic but practically unbounded state-machine with two basic functions; the first being a globally accessible singleton state, and the second being a virtual machine that applies changes to that state.
From a more practical perspective, Ethereum is an open-source, globally decentralized computing infrastructure that executes programs called smart contracts. It uses a blockchain to synchronize and store the system state along with a cryptocurrency called ether to meter and constrain execution resource cost.
The Ethereum platform enables developers to build powerful decentralized applications with built-in economic functions. While providing continuous uptime, it also reduces or eliminates censorship, third party interface, and counterparty risk.
Many people will come to Ethereum with some prior experience of cryptocurrencies, specifically bitcoin. Ethereum shares many common elements with other open blockchains: a peer-to-peer network connecting participants, a consensus algorithm for synchronization of state (Proof-of-Work), a digital currency (ether) and a global ledger (the blockchain).
The components of an open, public, blockchain are (usually):
-
A peer-to-peer network connecting participants and propagating transactions and blocks, based on a standardized "gossip" protocol.
-
A set of consensus rules, implemented in the state machine.
-
Messages, in the form of transactions, representing state transitions.
-
A state machine that processes transactions according to the consensus rules.
-
A distributed database, the blockchain, that records a journal of all the state transitions.
-
A consensus algorithm (e.g. Proof-of-Work) that decentralizes control over the blockchain, by forcing participants to compete and constraining them by the consensus rules.
-
One or more open-source software implementations of the above.
All or most of these components are usually combined in a single software client. For example, in Bitcoin, the reference implementation is developed by the bitcoin Core open source project, implemented as the bitcoind client. In Ethereum, rather than a reference implementation, there is a reference specification, a mathematical description of the system in the [yellowpaper]. There are a number of clients, which are built according to the reference specification.
In the past, we used the term blockchain to represent all of the components above, a short-hand reference to the combination of technologies that encompass all of the characteristics above. Today, however, the term blockchain has become diluted by marketers and profiteers, looking to hype their projects and attain unrealistic valuations for their startups. It’s effectively meaningless on its own. We need qualifiers to help us understand the characteristics of the blockchain in question, such as open, public, global, decentralized, neutral, and censorship-resistant, to identify the important emergent characteristics of a "blockchain" system that these components allow.
Not all blockchains are created equal. When you are told that something is a blockchain, you have not received an answer, rather you need to start asking a lot of questions to clarify what "blockchain" means. Start by asking for a description of the components above, then ask about whether this "blockchain" exhibits the open, public, etc. characteristics.
In many ways, both the purpose and construction of Ethereum are strikingly different from the open blockchains that preceded it, including Bitcoin.
Ethereum’s purpose is not primarily a digital currency payment network. While the digital currency ether is both integral and necessary for the operation of Ethereum, ether is intended as a utility currency to pay for use of the Ethereum platform.
Unlike Bitcoin, which has a very limited scripting language, Ethereum is designed to be a general purpose programmable blockchain that runs a virtual machine capable of executing code of arbitrary and unbounded complexity. Where Bitcoin’s Script language is constrained to simple TRUE/FALSE evaluation of spending conditions, Ethereum’s language is Turing Complete, meaning that it is equivalent to a general purpose computer that can run any computation a theoretical Turing Machine can run.
All great innovations solve real problems, Ethereum is no exception. Ethereum was conceived at a time when people recognized the power of the Bitcoin model, and were trying to move beyond application of cryptocurrency applications, into other projects. But developers faced a conundrum. They either needed to build on top of Bitcoin or start a new blockchain. Building upon Bitcoin meant living within the intentional constraints of the network and trying to find workarounds. The limited types and sizes of data storage seemed to limit the types of applications that could run on top, as second layer solutions. Programmers needed to build systems that utilized only the limited set of variables, transaction types, and data. For projects that needed more freedom, more flexibility, starting a new blockchain was the only option. But starting a new blockchain meant bootstrapping all the infrastructure elements, testing, etc.
Towards the end of 2013, Vitalik Buterin, a young programmer and Bitcoin enthusiast, started thinking about further extending the capabilities of Bitcoin and Mastercoin (an overlay protocol that extended Bitcoin to offer rudimentary smart contracts). In October of 2013, Vitalik proposed a more generalized approach to the Mastercoin team, one that allowed flexible and scriptable (but not Turing complete) contracts to replace the specialized contract language of Mastercoin. While the Mastercoin team was impressed, this proposal was too radical a change to fit into their development roadmap.
In December 2013, Vitalik started sharing a "white paper", which outlined the idea behind Ethereum: a Turing complete programmable and general purpose blockchain. A few dozen people saw this early draft and offered feedback to Vitalik, helping him gradually evolve the proposal.
Both of the authors of this book received a copy of the white paper and commented on this early draft. Andreas M. Antonopoulos was intrigued by the idea and asked Vitalik many questions about the use of a separate blockchain to enforce consensus rules on smart contract execution and the implications of a Turing complete language. Andreas continued to follow Ethereum’s progress with great interest but was in the early stages of writing his book "Mastering Bitcoin" and did not participate directly in Ethereum until much later. Dr. Gavin Wood, however, was one of the first people to reach out to Vitalik and offer to help with his C++ programming skills. Gavin became Ethereum’s co-founder, co-designer and CTO.
As Vitalik recounts in his "Ethereum Prehistory" post:
This was the time when the Ethereum protocol was entirely my own creation. From here on, however, new participants started to join the fold. By far the most prominent on the protocol side was Gavin Wood.
…
Gavin can also be largely credited for the subtle change in vision from viewing Ethereum as a platform for building programmable money, with blockchain-based contracts that can hold digital assets and transfer them according to pre-set rules, to a general-purpose computing platform. This started with subtle changes in emphasis and terminology, and later this influence became stronger with the increasing emphasis on the “Web 3” ensemble, which saw Ethereum as being one piece of a suite of decentralized technologies, the other two being Whisper and Swarm.
Starting in December 2013, Vitalik and Gavin refined and evolved the idea, together building the protocol layer that became Ethereum.
Ethereum’s founders were thinking about a blockchain that didn’t aim for a specific purpose, but instead could support a broad variety of applications by being programmed. The idea was that by using a general purpose blockchain like Ethereum, a developer could program their particular application without having to bootstrap the underlying mechanisms of peer-to-peer networks, blockchains, consensus algorithms, etc. The Ethereum platform was designed to abstract these details and provide a deterministic and secure programming environment for decentralized blockchain applications.
Much like Satoshi, Vitalik and Gavin didn’t just invent a new technology, they combined new inventions with existing technologies in a novel way and delivered the prototype code to prove their ideas to the world.
The founders worked for years, building and refining the vision. And on July 30th 2015, the first Ethereum block was mined. The world’s computer started serving the world…
Vitalik Buterin’s article "A Prehistory of Ethereum" was published in September 2017 and provides a fascinating first-person view of Ethereum’s earliest moments.
You can read it at http://vitalik.ca/general/2017/09/14/prehistory.html
The birth of Ethereum was the launch of the first stage, named "Frontier". Ethereum’s development is planned over four distinct stages, with major changes occurring in each new stage. Each stage may include sub-releases, known as "hard forks" that change functionality in a way that is not backwards compatible.
The four main development stages are codenamed Frontier, Homestead, Metropolis and Serenity. The intermediate hard forks are codenamed "Ice Age", "DAO", "Tangerine Whistle", "Spurious Dragon", "Byzantium", and "Constantinople". They are listed below, by the block number in which the hard fork occurred:
- Block #0
-
"Frontier" - The initial "test" stage of Ethereum, lasted from July 30th 2015 to March 2016.
- Block #200,000
-
"Ice Age" - A hard fork to introduce an exponential difficulty increase, motivating a transition to Proof-of-Stake.
- Block #1,150,000
-
"Homestead" - The second stage of Ethereum, launched in March 2016.
- Block #1,192,000
-
"DAO" - The hard fork that reversed the hacked DAO contract and caused Ethereum and Ethereum Classic to split into two competing systems.
- Block #2,463,000
-
"Tangerine Whistle" - A hard fork to change the gas calculation for certain IO-heavy operations and to clear the accumulated state from a denial of service attack, which exploited the low gas cost of those operations.
- Block #2,675,000
-
"Spurious Dragon" - A hard fork to address more denial of service attack vectors, and another state clearing. Also, a replay attack protection mechanism.
We are currently in the Metropolis stage, which was planned as two sub-release hard forks (see [hard_fork]) codenamed Byzantium and Constantinople. Byzantium went into effect in October 2017 and Constantinople is anticipated by mid-2018.
- Block #4,370,000
-
"Metropolis Byzantium" - Metropolis is the third stage of Ethereum, current at the time of writing this book, launched in October 2017. Byzantium is the first of two hard forks for Metropolis.
After Metropolis' Byzantium hard fork, there is one more hard fork planned for Metropolis. Metropolis is followed by the final stage of Ethereum’s deployment, codenamed Serenity.
- Constantinople
-
- The second part of the Metropolis stage, planned for mid-2018. Expected to include a switch to hybrid Proof-of-Work/Proof-of-Stake consensus algorithm, among other changes.
- Serenity
-
The fourth and final stage of Ethereum. Serenity does not yet have a planned release date.
The "original" blockchain, bitcoin’s blockchain, tracks the state of units of bitcoin and their ownership. You can think of bitcoin as a distributed consensus state machine, where transactions cause a global state transition, altering the ownership of coins. The state transitions are constrained by the rules of consensus, allowing all participants to (eventually) converge on a common (consensus) state of the system, after several blocks are mined.
Ethereum is also a distributed state machine. But instead of tracking only the state of currency ownership, Ethereum tracks the state transitions of a general-purpose data store. By general purpose we mean any data that can be expressed as a key-value tuple. A key-value data store simply stores any arbitrary value, referenced by some key. For example storing the value "Mastering Ethereum", referenced by the key "Book Title". In some ways, this serves the same purpose as the data storage model of Random Access Memory (RAM) used by a general purpose computer. Ethereum has memory that stores both code and data and it uses the blockchain to track how this memory changes over time. Like a general-purpose stored-program computer, Ethereum can load code into its state machine and run that code, storing the resulting state changes in the blockchain. Two of the critical differences from a general purpose computer are that Ethereum state changes are governed by the rules of consensus and the state is distributed globally on a shared ledger. Ethereum answers the question: "What if we could track any arbitrary state and program the state machine, to create a world-wide computer operating under consensus?".
In Ethereum, the components of a blockchain system described in Components of a blockchain are, more specifically:
- P2P Network
-
Ethereum runs on the Ethereum Main Network, which is addressable on TCP port 30303, and runs a protocol called ÐΞVp2p.
- Consensus rules
-
Ethereum’s consensus rules, are defined in the reference specification, the [yellowpaper].
- Transactions
-
Ethereum transactions (see [transactions]) are network messages, that include (among other things) a sender, recipient, value and data payload.
- State Machine
-
Ethereum state transitions are processed by the Ethereum Virtual Machine (EVM), a stack-based virtual machine that executes bytecode (machine-language instructions). EVM programs called "smart contracts" are written in high-level languages (e.g. Solidity) and compiled to bytecode for execution on the EVM.
- Blockchain
-
Ethereum’s blockchain is stored locally on each node as a database (usually Google’s LevelDB), which contains the transactions and system state in a serialized hashed data structure called a Merkle Patricia Tree.
- Consensus Algorithm
-
Ethereum currently uses a Proof-of-Work algorithm called Ethash, but there are plans to transition to a Proof-of-Stake system codenamed Casper in the near future.
- Clients
-
Ethereum has several interoperable implementations of the client software, the most prominent of which are Go-Ethereum (Geth) and Parity.
The Ethereum Yellow Paper: https://ethereum.github.io/yellowpaper/paper.pdf
The "Beige Paper": a rewrite of the "Yellow Paper" for a broader audience in less formal language: https://github.com/chronaeon/beigepaper
ÐΞVp2p network protocol: https://github.com/ethereum/wiki/wiki/%C3%90%CE%9EVp2p-Wire-Protocol
Ethereum Virtual Machine - a list of "Awesome" resources: https://github.com/ethereum/wiki/wiki/Ethereum-Virtual-Machine-(EVM)-Awesome-List
LevelDB Database (used most often to store the local copy of the blockchain): http://leveldb.org
Merkle Patricia Trees: https://github.com/ethereum/wiki/wiki/Patricia-Tree
Ethash Proof-of-Work Consensus Algorithm: https://github.com/ethereum/wiki/wiki/Ethash
Casper Proof-of-Stake v1 Implementation Guide: https://github.com/ethereum/research/wiki/Casper-Version-1-Implementation-Guide
Go-Ethereum (Geth) Client: https://geth.ethereum.org/
Parity Ethereum Client: https://parity.io/
As soon as you start reading about Ethereum, you will immediately hear the term "Turing Complete". Ethereum, they say, unlike bitcoin, is "Turing Complete". What exactly does that mean?
The term "Turing Complete" is named after English mathematician Alan Turing who is considered the father of computer science. In 1936 he created a mathematical model of a computer consisting of a state machine that manipulates symbols, by reading and writing them on sequential memory (resembling an infinite-length magnetic tape). With this construct, Alan Turing went on to provide a mathematical foundation to answer (in the negative) questions about universal computability, meaning whether all problems are solvable. He proved that there are classes of problems that are uncomputable. Specifically, he proved that the Halting Problem (trying to evaluate whether a program will eventually stop running) is not solvable.
Alan Turing further defined a system to be Turing Complete, if it can be used to simulate any Turing Machine. Such a system is called a Universal Turing Machine (UTM).
Ethereum’s ability to execute a stored program, in a state machine called the Ethereum Virtual Machine, while reading and writing data to memory makes it a Turing Complete system and therefore a Universal Turing Machine. Ethereum can compute any algorithm that can be computed by any Turing Machine, given the limitations of finite memory.
Ethereum’s groundbreaking innovation is to combine the general-purpose computing architecture of a stored-program computer with a decentralized blockchain, thereby creating a distributed single-state (singleton) world computer. Ethereum programs run "everywhere", yet produce a common (consensus) state that is secured by the rules of consensus.
Hearing that Ethereum is Turing Complete, you might arrive at the conclusion that this is a feature that is somehow lacking in a system that is Turing Incomplete. Rather, it is the opposite. It takes a very focused effort to constrain a system so that it is not Turing Complete. Turing completeness arises in even the simplest state machines. In fact the simplest Turing Complete state machine known (Rogozhin, 1996) has 4 states and uses 6 symbols, with a state definition that is only 22 instructions long.
Not only is Turing completeness achievable in the simplest of systems, but systems that are designed to be constrained so that they are Turing Incomplete, are often found to be "Accidentally Turing Complete". A constrained system that is Turing Incomplete is harder to design and must be carefully maintained so that it remains Turing Incomplete.
A fun reference of systems that are "Accidentally Turing Complete" can be found here: http://beza1e1.tuxen.de/articles/accidentally_turing_complete.html
The fact that Ethereum is Turing Complete means that any program of any complexity can be computed in Ethereum. But that flexibility brings some thorny security and resource management problems.
Turing proved that you cannot predict whether a program will terminate, by simulating it on a computer. In simple terms, we cannot predict the path of a program without running it. Turing Complete systems can run in "infinite loops", a term used (in oversimplification) to describe a program that does not terminate. It is trivial to create a program that runs a loop that never ends. But unintended never-ending loops can arise without warning, due to complex interactions between the starting conditions and the code. In Ethereum, this poses a challenge: every participating node (client), must validate every transaction, running any smart contracts it calls. But as Turing proved, Ethereum can’t predict if a smart contract will terminate, or how long it will run, without actually running it (possibly running forever). Whether by accident, or on purpose, a smart contract can be created such that it runs forever when a node attempts to validate it, effectively a denial of service attack. Of course, between a program that takes a millisecond to validate and one that runs forever there is an infinite range of nasty, resource hogging, memory-bloating, CPU overheating programs that simply waste resources. In a world computer, a program that abuses resources gets to abuse the world’s resources. How does Ethereum constrain the resources used by a smart contract if it cannot predict resource use in advance?
To answer this challenge, Ethereum introduces a metering mechanism called gas. As the EVM executes a smart contract, it carefully accounts for every instruction (computation, data access, etc.). Each instruction has a pre-determined cost in units of gas. When a transaction triggers the execution of a smart contract, it must include an amount of gas that sets the upper limit of computation that can be consumed running the smart contract. The EVM will terminate execution if the amount of gas consumed by computation exceeds the gas available in the transaction. Gas is the mechanism Ethereum uses to allow Turing Complete computation while limiting the resources that any program can consume.
In 2015 an attacker exploited an EVM instruction that cost far less gas than it should have. this allowed the attacker to create transactions that use a lot of memory and take several minutes to validate. To fix this attack, Ethereum had to change its gas accounting formula for certain instructions in a backwards incompatible (hard fork) change. Even with this change, however, Ethereum clients have to skip validating these transactions or waste weeks trying to validate them.
Ethereum started as a way to make a general purpose blockchain that could be programmed for a variety of uses. But very quickly, Ethereum’s vision expanded to become a platform for programming Decentralized Applications (DApps). DApps represent a broader perspective than "smart contracts". A DApp is, at the very least, a smart contract and a web user-interface. More broadly, a DApp is a web application that is built on top of open, decentralized, peer-to-peer infrastructure services.
A DApp is composed of at least:
-
Smart contracts on a blockchain.
-
A web front-end user-interface.
In addition, many DApps include other decentralized components, such as:
-
A decentralized (P2P) storage protocol and platform.
-
A decentralized (P2P) messaging protocol and platform.
|
Tip
|
You may see DApps spelled as ÐApps. The Ð character is the Latin character called "ETH", alluding to Ethereum. To display this character, use decimal entity #208 in HTML, and Unicode characters 0xCE (UTF-8), or 0x00D0 (UTF-16). |
In 2004, the term "Web 2.0" came to prominence, describing an evolution of the web towards user-generated content, responsive interfaces and interactivity. Web 2.0 is not a technical specification, but rather a term describing the new focus of web applications.
The concept of DApps is meant to take the World Wide Web to its next natural evolution, introducing decentralization with peer-to-peer protocols into every aspect of a web application. The term used to describe this evolution is Web3, meaning the third "version" of the web. First proposed by Gavin Wood, web3 represents a new vision and focus for web applications: from centrally owned and managed applications, to applications built on decentralized protocols.
In later chapters we’ll explore the Ethereum web3js JavaScript library which bridges JavaScript applications that run in your browser with the Ethereum blockchain. The web3.js library also includes an interface to a P2P storage network called Swarm and a P2P messaging service called Whisper. With these three components included in a JavaScript library running in your web browser, developers have a full application development suite that allows them to build web3 DApps:
So far we’ve talked about how Ethereum’s goals and technology differ from other blockchains that preceded it, like bitcoin. Ethereum also has a very different development culture.
In bitcoin, development is guided by conservative principles: all changes are carefully studied to ensure that none of the existing systems are disrupted. For the most part, changes are only implemented if they are backwards compatible. Existing clients are allowed to "opt-in", but will continue to operate if they decide not to upgrade.
In Ethereum, by comparison, the development culture is focused on speed and innovation. The mantra is "move fast and break things". If a change is needed, it is implemented, even if that means invalidating prior assumptions, breaking compatibility, or forcing clients to update. Ethereum’s development culture is characterized by rapid innovation, rapid evolution and a willingness to engage in experimentation.
What this means to you as a developer, is that you must remain flexible and be prepared to rebuild your infrastructure as some of the underlying assumptions change. Do not assume anything will be static or permanent. One of the big challenges facing developers in Ethereum is the inherent contradiction between deploying code to an immutable ledger and a development platform that is still rapidly evolving. You can’t simply "upgrade" your smart contracts. You must be prepared to deploy new ones, migrate users, apps and funds, and start over.
Ironically, this also means that the goal of building systems with more autonomy and less centralized control cannot be realized. Autonomy and decentralization requires a bit more stability in the platform than you’re likely to get in Ethereum, in the next few years. In order to "evolve" the platform, you have to be ready to scrap and restart your smart contracts, which means you have to retain a certain degree of control over them.
But, on the positive side, Ethereum is moving forward very fast. There’s very little opportunity for "bike-shedding" - an expression that means arguing over minor details such as how to build the bicycle shed in the back of the building. If you start bike-shedding, you might suddenly discover the rest of the development team changed the plan, and ditched bicycles in favor of autonomous hovercrafts. There are very few sacred principles, final standards, or fixed interfaces in Ethereum.
Eventually, Ethereum core protocol development will slow down and its interfaces will become fixed. But in the meantime, innovation is the driving principle. You’d better keep up, because no one will slow down for you.
Blockchains have a very steep learning curve, as they combine multiple disciplines into one domain: programming, information security, cryptography, economics, distributed systems, peer-to-peer networks etc. Ethereum makes this learning curve a lot less steep, so you can get started very quickly. But just below the surface of a deceptively simple environment, lies a lot more. As you learn and start looking deeper, there’s always another layer of complexity and wonder.
Ethereum is a great platform for learning about blockchains and it’s building a massive community of developers, faster than any other blockchain platform. More than any other blockchain, Ethereum is a developer’s blockchain, built by developers, for developers. A developer familiar with JavaScript applications can drop into Ethereum and start producing working code very quickly. For the first years of Ethereum, it was common to see t-shirts announcing that you can create a token in just five lines of code. Of course, this is a double-edged sword. It’s easy to write code, but it’s very hard to write good code and secure code.
This book dives into Ethereum and examines every component. You will start with a simple transaction, dissect how it works, build a simple contract, make it better and follow its journey through the Ethereum system.
You will learn how Ethereum works, but also why it is designed the way it is. You will be able to understand how each of the pieces work, but also how they fit together and why.