The consensus mechanism

If someone buys a bottle of water for $1, that person cannot spend the same $1 to buy a can of coke. If a person is free to double-spend a dollar, money would be worthless since everyone would have unlimited amounts and the scarcity, which gives the currency its value, would disappear. This is called the double-spending problem. With BTC, double spending is the act of using the same Bitcoin more than once. If this problem is not resolved, BTC loses its scarcity and cannot be used to facilitate a trade between two untrusting parties. The Bitcoin Core network protects against double spends via a consensus mechanism. To explain how the Bitcoin consensus mechanism works, we first describe the concepts of PoW (Proof-of-Work) and mining.

As explained earlier, a miner needs to solve a difficult mathematical puzzle ahead of other miners in order to receive the role of being a builder of the current new block and receive a reward for doing the work. The work of resolving the math problem is called PoW.

Why is PoW needed? Think of this: in a network consisting of mutually untrusting parties, more honest parties are needed than dishonest attackers in order to make the network function. Imagine if upon collecting sufficient transactions for a new block, a miner is allowed to build the new block immediately. This simply becomes a race for whoever can put enough transactions together quickly. This leaves a door wide open for malicious attackers to hack the network by including invalid or fake transactions and always win the race. This would allow hackers to double-spend BTCs freely.

Therefore, to prevent attackers from introducing bad transactions, a sufficient window of time is needed for participating nodes to verify every transaction's validity by making sure a BTC has not been spent yet. Since every node maintains a copy of the ledger, an honest miner can trace the history and ensure the following to confirm the validity of a transaction:

• The requestor of a transaction does own the BTCs.
• The same BTCs have not been spent by any other transactions in the ledger.
• The same BTCs have not been spent by other transactions within the candidate block.

This window of time is currently set to be around 10 minutes. To enforce the 10-minute waiting time, Bitcoin asks a miner to solve a sufficiently difficult mathematical puzzle. The puzzle requires only a simple computation. Miners have to repeat the same computation many times in order to burn enough CPU time to reach the network's goal of building a new block every 10 minutes on average. The process of repeated guessing is called mining and the device (specially made) is called a mining rig.

Since, in order to win the mining race, a miner needs to invest heavily in hardware, these miners are dedicated to the work of mining and aim to receive sufficient BTCs to cover the cost of running the mining operation and make a profit. As of the first half of 2018, the reward given to a winning miner is 12.5 BTCs. One can find the price of BTC by visiting the CoinMarketCap website (https://coinmarketcap.com/). As of September 21, 2018, one BTC is traded at around $6,710. Therefore, 12.5 BTC is worth about $83,875 USD.

Per Bitcoin protocol, mining is the only way for a new BTC to be issued (minted). Having a miner be rewarded handsomely serves three purposes:

• Compensates a miner's investment on hardware.
• Covers mining operational costs such as utility bills, which can be significant due to the large mining rigs being deployed at a mining site, human salaries, and site rentals.
• Gives miners incentives to safeguard the network from being attacked by malicious hackers. Miners are motivated to maintain the Bitcoin network in order not to lose value in their BTCs and their mining infrastructure. If Bitcoin is breached by hackers, Bitcoin's reputation will suffer badly and BTC prices would freefall. This is exactly what the Bitcoin inventor hoped for: having more good miners than bad miners to address the double-spending issue.

The total number of BTC that can be issued is fixed to be 21 million. As of today (September 19, 2018), around 17 million BTCs have been issued. The Bitcoin protocol defines a rule for dynamically adjusting the payout rate and the remaining 4 million coins aren't expected to be mined completely for another 122 years. The following point explains how the block creation payout rate is dynamically adjusted:

• The rate changes at every 210,000 blocks. It is a function of block height on the chain with genesis=0, and is calculated using 64-bit integer operations such as: (50 * 100000000) >> (height / 210000). The rate initially started with 50 BTCs, and fell to 25 BTCs at block 210,000. It fell to 12.5 BTCs at block 420,000, and will eventually go down to 0 when the network reaches 6,930,000 blocks.

Forking

A Bitcoin blockchain can diverge into two potential paths since miners do not necessarily collect transactions and contract block candidates in the same way, nor at the same time. Other reasons such as hacking or software upgrades can also lead to path divergence. The splitting patches are called forks. There are temporary forks and permanent forks.

If a permanent fork occurs due to, for example, malicious attacks, a hard fork occurs. Similarly, there is the concept of soft fork. Both hard fork and soft fork refer to a radical change to the protocol. Hard fork makes previously invalid blocks/transactions valid and a soft fork makes previously valid blocks/transactions invalid. 

To remove a temporary fork, Bitcoin protocol dictates that the longest chain should be used. In other words, when facing two paths, a winning miner will choose the longer chain to link a new block. As a result, the longer path continues to grow and the blocks on the losing (shorter) path becomes orphaned. Bitcoin nodes will soon discard or not take the orphaned blocks. They only keep the blocks on the longest chain as being the valid blocks.

In the case of a permanent fork, nodes on the network have to choose which chain to follow. For example, Bitcoin Cash diverged from Bitcoin due to a disagreement within the Bitcoin community on how to handle the scalability problem. As a result, Bitcoin Cash became its own chain and shares the transaction history from the genesis block up to the forking point. As of September 21, Bitcoin Cash's market cap is around $8 billion, ranking fourth, versus Bitcoin's $215 billion.

Mining and difficulty level

There is one more issue that needs to be resolved: how to maintain the new block building rate of 10 minutes. If nothing is done, the mining rate will change due to the following factors:

• The number of miners on the network can vary in response to the BTC price
• Technology advancements make mining rigs progressively faster
• The total number of mining rigs varies

Bitcoin adjusts the difficulty level of the mathematical puzzle in order to keep the building rate at 10 minutes. The difficulty level is calculated from the rate at which the most recent blocks were added in. If the average rate of new blocks being added is less than 10 minutes, the difficulty level will be increased. If the average rate takes more than 10 minutes, it's decreased. The difficulty level is updated every 2,016 blocks. The following graph displays the historical trend in Bitcoin difficulty level.

We have yet to talk about the actual mining algorithm. Assume the current difficulty level is to find the first hash value with the leading character to be 0. In Bitcoin, the process of solving a puzzle, that is, mining, requires a miner to follow these steps:

• First, find the SHA-256 hash of the block in construction.
• If the resulting hash has a leading 0, the miner solves the puzzle. The miner links the block to the ledger on the node and claims the trophy, 12.5 BTCs. The miner's node broadcasts the news to all nodes. All other nodes and miners on the network validate the answer (by mapping the block information plus nonce to get the same hash) and validate the entire history of the ledger, making sure that the block contains valid transactions.
• If it passes the checks, all nodes on the network add the block to their copies of the ledger. Miners start to work on the next new block.
• If the winning miner is a malicious attacker and includes bad transactions in the block, the validation of these transactions will fail and other miners will not include the block in their ledger copies. They will continue to mine on the current block. As time passes, the path containing the bad block will no longer be the longest path and, therefore, the bad block will become an orphaned block. This is essentially how all nodes on the network reach consensus to add only good blocks to the network and prevent bad blocks from sneaking in, therefore resolving the double-spending issue.
• If the resulting hash does not start with 0, then the miner is allowed to append a sequence number, known to be a nonce, starting from 0 to the input text, and retry the hash.
• If the resulting hash still does not contain a leading 0, the miner will add another sequence number, 1, to the input text and obtain a new hash. The miner will keep trying in this way until it finds the first hash with a leading zero.

The following is an example of how the plaintext and nonce work together. The original plaintext is input string and the nonce varies from 0 to 1:

• input string: f23f4781d6814ebe349c6b230c1f700714f4f70f735022bd4b1fb69421859993
• input string0: 5db70bb3ae36e5b87415c1c9399100bc60f2068a2b0ec04536e92ad2598b6bbb
• input string1: 5d0a0f2c69b88343ba44d64168b350ef62ce4e0da73044557bff451fd5df6e96

In Bitcoin, adjusting difficult level largely refers to changing the required number of leading zeros. (The actual adjustment involves some other miner tuning to the requirement.) Each addition of a leading zero will increase the average number of tries significantly and therefore will increase the computing time. This is how Bitcoin manages to maintain the average rate of 10 minutes for new blocks being added in. The current Bitcoin difficulty level is 18 leading zeros.

Hacking – the 51% problem

Thanks to the rising price of BTC, the mining operation has become more attractive. Investments are rushing in and large mining pools involving thousands of rigs or more have joined the network in order to gain an advantage in the race to solve the puzzle first and get the reward. For players without large capital from investments, they have a choice to participate in a mining pool. When the pool wins a race, the award will be allocated to each participant based on the computational power contributed.

This ever-growing computational power of a pool poses a real threat due to the so-called 51% problem. This problem occurs when a miner manages to build up computational power to total at least 51% of the total computing power of the network. When this occurs, the miner will have a chance to outrun other miners. The miner can continue to grow the ledger with blocks containing bad transactions since this miner has more than a 50% chance of solving the puzzle first. Soon, the malicious miner's ledger will grow to be the longest path and all other nodes have to save this path based on Bitcoin's consensus protocol.

For a large and well-established network such as Bitcoin, the 51% problem is not as critical an issue, mainly due to the following reasons:

• A well-established network will attract a much larger number of participating parties and connect a very significant number of nodes. It will take an exorbitantly high initial investment for a hacker to purchase the necessary mining rigs. When such a network is attacked, the price of cryptography will drop quickly when the news becomes public and the hacker will have a low chance of recovering the investment.
• In the history of Bitcoin, there have been cases when a mining pool that accumulated dangerously high computing power approached this line. When the participating miners in the pool realized the problem, many of them chose to leave the pool. Soon, the computational power of the pool fell to a safe level.
• In the case of a small and immature network, it is not difficult for a miner to muster computing power of more than 51%. However, the cryptocurrency value of these networks is minimal and it gives hackers very little financial incentive to take advantage of the 51% problem.

Private keys and Bitcoin wallets

As discussed earlier, BTCs do not physically exist. The only evidence of their existence is when they are associated with addresses, which are referred to in transactions. When an address is initially created, a pair of public and private keys are generated with it. The public key is made known to the public and the private key is kept only by the owner of the address. When the owner wants to spend all or a portion of their BTCs, the owner provides a digital signature signed with the private key and sends the BTC request to the Bitcoin network. In other words, one has to know both the address and its private key to spend the BTC. 

If an owner loses a private key, its associated BTCs will be lost permanently. Therefore, it is advised to keep this information in a safe place. It is generally good practice to keep the address and private keys in separate places. To prevent a digital copy getting lost, an owner should maintain physical copies of printouts. To make conversion easier, an owner can print a QR code and later scan the QR code whenever it is needed.

Bitcoin wallet applications are available to help a user manage keys and addresses. One can use a wallet to do the following:

• Generate addresses and corresponding public/private keys
• Save and organize a BTC's information
• Send a transaction request to the Bitcoin network

In Bitcoin, a private key is a 256-bit-long hash and a public key is 512 bits long. They can be converted into shorter lengths in hexadecimal representation. The following screenshot gives an example of a pair of public/private keys along with an address:

Bitcoin private keys can also be expressed in a string of 51 characters starting with a 5 and a public key in a string of 72 characters. A sample private key is 5Jd54v5mVLvy
RsjDGTFbTZFGvwLosYKayRosbLYMxZFBLfEpXnp and a sample public key is BFCDB2DCE28D959F2815B16F81798483ADA7726A3C4655DA4FBFC0E1108A8FD17B448A68.

Bitcoin scripting

One can install the following development tools for programming Bitcoin operations:

• NodeJS: This is an open source, cross-platform JavaScript runtime environment that executes JavaScript code outside of a browser. It allows a programmer to write and execute scripts quickly and easily. These scripts can be written to be run in a web browser or on a server.
• BitcoinJS: This is a JavaScript library for working with Bitcoin and its cryptographic functions. BitcoinJS can be used to generate public/private keys and addresses.
• Blockchain.info: This is a public API that can be used to query the blockchain to find out balances and broadcast transactions to the network. It can be used to implement a Bitcoin node and install and run a Bitcoin node.

After installing the preceding tools, one can execute the following operations:

• Generate a new private key and compute a public key
• Check the balance for a certain address
• Generate addresses
• Construct a new transaction
• Send a transaction, which involves three steps:
  • Build a transaction with a list of inputs and outputs
  • Sign the transaction with the required private keys
  • Broadcast the transaction to the network
• Build an escrow account
• Broadcasts the transaction

Altcoins

Thanks to Bitcoin, blockchain technology has attracted worldwide attention. Like any new technology, it has its limitations. Many variations of Bitcoin were created to address a particular limitation of Bitcoin. Here, we mention a few of them:

• Bitcoin Cash: This is a hard fork of the Bitcoin chain that was created because a group of Bitcoin core developers wanted to use a different way of addressing the scalability issue.
• Litecoin: This is almost identical to Bitcoin except that the time for adding a new block was reduced from 10 minutes to 2 minutes.
• Zcash: This is based on Bitcoin but offers total payment confidentiality.
• Monero and Zcash: Both altcoins address the privacy issue by making transaction history untraceable, but they implement two different solutions.
• Dash: This mainly improves user-friendliness. For example, transactions are made untraceable and a user does not have to wait for several additional new blocks to be added before considering a transaction to be committed to the chain.
• Namecoin: This extends the use case of Bitcoin, which is for trading BTCs only, to providing domain name services.
• Peercoin: This altcoin addresses the deficiencies of PoW, which is environmentally unfriendly and is low in throughput. Instead, it adopts proof of stake for achieving consensus. Based on this rule, a miner validates block transactions according to how many coins a miner holds. In other words, the mining power of a miner is in proportion to the number of peercoins owned.
• Primecoin: A primecoin miner competes to be the first to find the next biggest prime number.

Ethereum

Regardless of the efforts made from the steps-mentioned altcoins in addressing some part of the Bitcoin's limitations, there are several fundamental issues that are not being addressed yet:

• Bitcoin and these altcoins are specific to one purpose: trading either BTC or an altcoin.
• Although a programmer can use tools such as BitcoinJS to interact with the network, the resulting code sits outside of the blockchain and is not guaranteed to run. The chain itself does not have a Turing complete programming language for coding directly on a blockchain.
• These blockchains are stateless and one has to search through the entire ledger to find an answer such as the total number of BTC minted.

In response to these problems, Vitalik Buterin, a Canadian cryptocurrency researcher and programmer, proposed the idea of Ethereum in late 2013. Funded by an online crowdsale, the system went live on 30 July 2015, with 11.9 million coins premined for the crowdsale.

The core idea for Ethereum was to build a general-purpose blockchain so users could solve a wide range of business problems not just limited to cryptocurrency transfer. Ethereum introduced a few new and critical concepts:

• The concept of saving a smart contract on a blockchain
• The concept of implementing a smart contract with a Turing complete programming language such as Solidity and running the piece of code on the blockchain

Solidity was initially proposed in August 2014 by Gavin Wood. The Ethereum project's Solidity team led by Christian Reitwiessner later developed the language. It is one of the five languages, (Solidity, Serpent, LLL, Vyper, and Mutan) designed to target the Ethereum virtual machine (EVM).

Nick Szabo, a programmer and lawyer, initially proposed the term smart contract in 1996. In his blog, Nick Szabo described it as the granddaddy of all smart contracts, the vending machine.
A vending machine shares the exact same properties as a smart contract on a blockchain today. A vending machine is built with hardcoded rules that define what actions to execute when certain conditions are fulfilled, for example:

• If Susan inputs a dollar bill in the vending machine, then she will receive a bag of pretzels.
• If Tom puts in a five-dollar bill, Tom will receive a bag of pretzels and also change of four dollars.

In other words, rules are defined and enforced by a vending machine physically. Similarly, a smart contract contains rules in program code that are run on the blockchain and triggered when certain conditions are met.

The introduction of the smart contract concept is significant:

• A smart contract is a scripted legal document.
• The code built into the contract is stored on the Ethereum blockchain and cannot be tampered with or removed. This greatly increases the credibility of the legal document.
• This code cannot be stopped, meaning any party—regardless of how powerful the party is—cannot order or interfere with the running of the smart contract code. As long as certain conditions are met, the code will run and the legally defined actions will be fulfilled.
• Ethereum to blockchain is like an OS to a computer. In other words, the platform is generic, no longer serving only one specific purpose.
• It now has a Turing complete language: Solidity.