Book Image

Mastering Blockchain Programming with Solidity

By : Jitendra Chittoda
Book Image

Mastering Blockchain Programming with Solidity

By: Jitendra Chittoda

Overview of this book

Solidity is among the most popular and contract-oriented programming languages used for writing decentralized applications (DApps) on Ethereum blockchain. If you’re looking to perfect your skills in writing professional-grade smart contracts using Solidity, this book can help. You will get started with a detailed introduction to blockchain, smart contracts, and Ethereum, while also gaining useful insights into the Solidity programming language. A dedicated section will then take you through the different Ethereum Request for Comments (ERC) standards, including ERC-20, ERC-223, and ERC-721, and demonstrate how you can choose among these standards while writing smart contracts. As you approach later chapters, you will cover the different smart contracts available for use in libraries such as OpenZeppelin. You’ll also learn to use different open source tools to test, review and improve the quality of your code and make it production-ready. Toward the end of this book, you’ll get to grips with techniques such as adding security to smart contracts, and gain insights into various security considerations. By the end of this book, you will have the skills you need to write secure, production-ready smart contracts in Solidity from scratch for decentralized applications on Ethereum blockchain.
Table of Contents (21 chapters)
Free Chapter
1
Section 1: Getting Started with Blockchain, Ethereum, and Solidity
5
Section 2: Deep Dive into Development Tools
9
Section 3: Mastering ERC Standards and Libraries
16
Section 4: Design Patterns and Best Practices

Introduction to Ethereum

Ethereum was proposed in 2013 by Vitalik Buterin, a cryptocurrency researcher and programmer. The first production release launched in 2015. Ethereum is an implementation of blockchain, and is an open source, public, and decentralized computing platform. Ethereum gained popularity because of its smart contract's features. Developers can program their smart contracts and execute them on the blockchain platform so that the full potential of the blockchain's different properties can be used.

Ethereum is for writing decentralized applications

As discussed before, Bitcoin uses blockchain technology for payments using bitcoins as a currency. Bitcoin also supports a scripting language, with which you can write small scripts. Bitcoin scripts have limited functionalities and it is difficult to write complex scripts on it. Its scripting language is also not Turing-complete (it does not support loops). Bitcoin transactions are slow; normally, it gets processed in 10 minutes or more.

With Ethereum, writing complex smart contracts and utilizing other properties of blockchain becomes easy. Any programmer who knows Java, JavaScript, and other programming languages can easily learn to code in the Solidity language. Solidity is a Turing-complete language. Transactions on Ethereum are processed much faster as compared to Bitcoin transactions. Using all of these potentials, you can write decentralized applications that would run on the Ethereum blockchain.

Ethereum architecture

Ethereum architecture consists of multiple entities that make its blockchain network. An Ethereum network has all the nodes connected to each other using the P2P network protocol; each node keeps the latest copy of the Ethereum blockchain ledger. A user can interact with the Ethereum network via the Ethereum client. The Ethereum client can be a desktop/mobile/web page:

Ethereum high-level architecture

Let's discuss these entities.

P2P networks

In a P2P network, two or more computers are connected to share their resources without going to a centralized server or machine. Nodes connected in a P2P network can share or consume network resources.

Nodes

Nodes are devices in the blockchain network, which make the mesh of the blockchain network. Nodes can perform various types of tasks, such as being an active computer, cell phone, or disk drive as long as they are connected to the internet, and they participate in a blockchain network.

A node keeps the latest copy of the blockchain data (ledger) and, in some cases, processes the transactions as well. Miners put nodes on the network and willingly share their computing power or disk space. Nodes can be of two types—full nodes or light nodes.

Full nodes

A full node in a blockchain network maintains the complete data structure of the blockchain ledger and participates in the blockchain network by verifying every block and transaction using their consensus rules. Full nodes always keep the latest copy of the ledger.

Lightweight nodes

A light node does not maintain the full copy of the blockchain ledger. It verifies the transaction using a method called Simple Payment Verification (SPV). SPV allows a node to verify whether a transaction has been included in the blockchain or not. This is verified without the need to download the entire blockchain on a machine. A light node just downloads the header information of the blocks, hence lower storage space is required in running light nodes.

Miners

Owners of the nodes willingly contribute their computing resources or hard disk space to store and validate transactions; in return, they collect block reward and transaction fees from the transactions. The nodes that perform transaction execution and verification are called miners.

Blocks

The blockchain maintains a series of blocks. Each block is linked together with the last generated block. A block contains multiple transactions. At the time of writing this book, blocks are generated using a PoW consensus algorithm called Ethash in Ethereum. In future it is planned to change to PoS.

Miners share their processing power with the Ethereum blockchain network to process all pending transactions and generate a new block. Each block is generated in approximately 15 seconds. At the time of writing, a miner who generates the block is rewarded with 3 ETH in their wallet. It is proposed that, in the future, it will be reduced from 3 ETH to 2 ETH per block. It may change in future as well.

Each block has a defined gas limit: as of now, it is 8 million gas per block. This means that the total gas consumed by the transactions included in the block cannot exceed the 8 million gas limit per block.

A block contains a block header and transactions. A block can contain at least one transaction. A new block is added to the blockchain and linked to the previous block. The very first block in the blockchain is called the genesis block:

Chain of connected blocks

The block header contains metadata about the block and the previous block. It contains multiple fields, but it's only necessary to understand a few to have an idea of block generation:

Block header and transactions root

The Parent Hash always points to the parent block of the newly generated block. This child-to-parent link goes on until the genesis block.

The Nonce is the number that is found by the miner to solve the cryptographic puzzle.

The Receipts Root is the Keccak 256-bit hash of the root node of the tree structure, which is populated with the receipts of each transaction.

The Timestamp is the time when the block is found and added to the blockchain.

The Transactions Root is the Keccak 256-bit hash of the root node of the tree structure, which is populated with each transaction.

The State Root is the Keccak 256-bit hash of the root node of the state tree, after all transactions are executed.

You should know how hash functions work and what the properties of the hash functions are. Also, you should have a basic knowledge of asymmetric cryptography.

Ethereum Virtual Machine (EVM)

EVM is the runtime environment for smart contracts. Ethereum supports multiple scripting languages for writing smart contracts. These smart contracts are then compiled and converted into EVM bytecodes. Bytecodes are then executed by EVM to perform operations on smart contracts.

Ether currency

Ethereum has its own cryptocurrency called ether (symbol: ETH). Ether is a fungible coin, which means that a coin can be subdivided into smaller units. For example, 1 ETH can be subdivided into a maximum of 18 decimal places; the smallest value is called wei. Ether is a crypto fuel for the Ethereum network.

For any transaction you perform on the Ethereum network, some gas is consumed to execute that transaction, and the gas is to be paid in ether only. This transaction fee that you pay to process your transaction goes to the miners.

Ether is publicly traded on many centralized and decentralized exchanges. Some decentralized exchanges are also built upon the Ethereum blockchain itself using Solidity smart contracts.

Smallest unit – wei

The smallest unit of ether is denoted as wei. The conversion table for ether and wei is as follows:

Unit Wei value Wei Ether value Ether
Wei 1 wei 1 10-18 ether 0.000000000000000001
Kwei (KiloWei/babbage) 103 wei 1,000 10-15 ether 0.000000000000001
Mwei (MegaWei/lovelace) 106 wei 1,000,000 10-12 ether 0.000000000001
Gwei (GigaWei/shannon) 109 wei 1,000,000,000 10-9 ether 0.000000001
microether (szabo) 1012 wei 1,000,000,000,000 10-6 ether 0.000001
milliether (finney) 1015 wei 1,000,000,000,000,000 10-3 ether 0.001
ether 1018 wei 1,000,000,000,000,000,000 1 ether 1

 

The most commonly used terms are ether, gwei, and wei. These units are used in many places in the Ethereum ecosystem, such as when creating a new transaction, or when setting the gas price for a transaction. These units can also be used in the Solidity language, hence Solidity supports weifinneyszabo, and ether units.

Gas

Gas is the fuel of the Ethereum blockchain network. Gas is always paid in ether. Gas is the transaction fee paid to miners for executing and adding your transaction into the Ethereum blockchain.

For every transaction, you have to specify the value of the gas price and gas limit you want to set. Based on these values, the network calculates how much of a fee you are willing to pay for a transaction.

Every bytecode operation in EVM has fixed gas units assigned. When that operation is executed by EVM, fixed quantity of gas is consumed by the transaction.

Gas limit

The gas limit is the maximum unit of gas your transaction may take, in order to be executed on the Ethereum blockchain. If your transaction takes less gas, then the extra gas that you have provided will be refunded back to your wallet. If your transaction consumes all the gas and requires more gas to be executed, your transaction will fail, and gas will be consumed as part of the transaction fees.

Gas price

The gas price is the price per gas unit you are willing to pay for executing your transaction. The unit of gas price is always defined in gwei. According to your gas price, miners decide to process your transaction. If the transaction has a higher gas price, there is a chance that your transaction will be processed early. If your gas price is low, then your transaction might be processed after a delay or when the blockchain is free to process low gas price transactions.

Formulas

The following are some formulas to calculate the gas usage and transaction fee per transaction:

Example

Let's take an example of a transaction where you want to transfer 0.1 ETH to your friend. The following are the values for the transaction fields you would have to fill while creating a transaction:

Field name Value
From 0xff899af34214b0d777bcd3c230ef637b763f0b01
To 0xc4fe5518f0168da7bbafe375cd84d30f64cda491
Value 0.1 ether
Gas limit 30,000
Gas price 0.000000021 ether (21 gwei)

 

You can also refer to Chapter 4, Learning MetaMask and Remix, where we have demonstrated how to use MetaMask to initiate transactions. 

Once the transaction is sent and confirmed on blockchain, you can see its status on block explorer (etherscan.io) as follows:

Screenshot from etherscan.io of the transaction

Let's calculate the values using these formulas: 

  • TxFeeMax = 30,000 * 0.000000021 = 0.00063 ether
  • TxFeePaid = 21,000 * 0.000000021 = 0.000441 ether
  • TxFeeReturned = 0.00063 - 0.000441 = 0.000189 ether
  • GasUnused = 30,000 - 21,000 = 9,000 units of gas
  • BalanceRequiredAtTransactionInitiation = 0.1 + 0.00063 = 0.10063 ether

Let's go through the preceding calculations. When you initiate the preceding ether transfer transaction, it must have a balance of 0.10063 ether. If the balance in your wallet is lower than 0.10063 ether, then you will not be allowed to initiate transactions and this will be rejected by the client itself as it's an invalid transaction.

If you have sufficient balance and the transaction is initiated, then, along with your transaction, the BalanceRequiredAtTransactionInitiation amount will be locked. So, in this example, 0.10063 ether is locked.

Now, your transaction is processed, 21,000 gas has been consumed in order to execute the transaction on the EVM, and the transaction executed successfully. As it consumed only 21,000 gas, it refunds 9,000 units of gas. This means 0.000189 ether will be refunded back to your wallet and you have paid 0.000441 ether for your transaction processing.

You can remember this GasLimit and GasPrice with a car and fuel example. Let's say you want to go from city A to city B and the distance between these two cities is 500 km. Your car fuel average is 20 km/liter (GasRequired). There is no fuel station on the way when you travel, so you would have to get it filled before you start. You need 500/20 = 25 liters (TxFeeRequired) of fuel to reach city B. If you filled your car fuel tank with 30 liters (TxFeeMax), then you would successfully reach city B and you will still have 5 liters (GasUnused) left with you that you can use later on. However, if you had filled it with 20 liters when you started the journey, then you would have stopped at 400 km and you couldn't have done anything. This would be called a failed transaction and your gas/fuel is consumed so there's no gas refund.

An ether transfer requires a fixed 21,000 gas limit per transaction. If you also include data along with the ether value, it may require a greater gas limit to execute the transaction.

Ethereum accounts

On the Ethereum platform, you can have two types of accounts:

  • Externally Owned Accounts (EOA)
  • Contract accounts

Both of these accounts can hold balances of ETH; however, there are some differences between these accounts, which we will cover in the following sections. Both account addresses are in hexadecimal, which starts with the 0x prefix. The Ethereum public address of an EOA and a contract is always a 20 bytes (160 bits) address represented in hexadecimal form:

Ethereum account/contract public address format

You need these accounts to perform transactions on the Ethereum network.

Externally owned accounts

An Ethereum wallet can hold one or more EOA. Wallets can be created by anyone via wallet creation websites, browser plugins such as MetaMask, or mobile apps. These wallet creation services generate a new private key using a specific account creation algorithm. Using this private key, the public key is derived. The combination of a private key and a public key is called an account or EOA. A public key can be shared with anyone to receive ETH/ERC20 tokens. However, the private key must be kept secret by the user. For the creation of wallets using MetaMask, you can refer to Chapter 4, Learning MetaMask and Remix.

An EOA is controlled by its private key to initiate any transaction with it. An EOA owner can send ETH/ERC20 tokens to other EOA or contract addresses. They can also initiate a transaction on a deployed contract with EOA:

User interacting with Ethereum network using the EOA present in their wallet

As shown in the preceding diagram, the user can interact with the Ethereum network via one of the EOA accounts present in his wallet. A user holds a wallet that can have one or more EOA accounts. Each EOA account has a private key, which the user must use so they can sign the transaction data and send it to the Ethereum network to be executed. The user can also check their account balance from the Ethereum network using their public key.

New wallets are created off-chain using wallet generation algorithms used by MetaMask or other available websites. They are not created on-chain; however, smart contracts and their addresses are created on-chain when a contract is deployed on the blockchain.

Contract accounts

This type of account is also known as a contract. Contracts only have a public address, and their code logic controls the flow of the funds and other states of the contract. Contracts do not have any private keys associated. You need an EOA account to create a new contract or to interact with an existing contract. Contract accounts have a Solidity code that defines their behavior.

Any Solidity code you deploy creates new contract accounts or contracts:

A user accessing a contract account via their EOA account

As shown in the preceding diagram, the user has their own EOA account. Using this EOA account, the user can interact with any deployed contract accounts (smart contracts) present on the Ethereum blockchain. The user can interact by initiating transactions. Also, with an EOA account, the user can deploy a new contract.

Contract accounts can only be created on-chain and their contract bytecode is included in the block when they are deployed successfully. Contract accounts are not created off-chain, just like EOA accounts.

The difference between an EOA and a contract

There are some differences between an EOA and a contract account. The following table distinguishes between an EOA and a contract:

EOA (account) Contract account (smart contract)
Creation A new account can be created at any time, using a wallet generation utility such as MetaMask or with https://www.myetherwallet.com/. A new contract can only be created from an EOA or from an existing deployed contract. 
Public address An EOA account's public address is derived from its private key. A contract's public address is created by the combination of a public address of creating account + nonce.
Private keys An EOA can only be controlled using its private key. A contract which do not has a private key can be controlled using other EOA or contract. They respond to transactions initiated from an EOA.
Control Whoever owns a private key can control the funds of an EOA. Control is defined by the code stored with it. An EOA or another contract may have control over a deployed contract.
Code An EOA does not have any associated Solidity code. A contract always has Solidity code associated with it, which is stored together with it.
Access Using the private key of an EOA, we can make transactions via different wallet software such as MetaMask and Ethereum wallet. Using Abstract Binary Interfaces (ABIs) of a contract, an EOA can access the functions of the contract via Remix, Truffle, and others tools.
Transaction initiation Using the private key, one can directly access the funds of EOA and initiate any transactions. So, direct access to funds is allowed. A contract cannot initiate a transaction on its own. An EOA need to initiate the transaction first, then only contracts can initiate other transactions.

 

As we have discussed the basic differences between an EOA and a contract, let's discuss Ethereum transactions.

Ethereum transaction

A transaction on an Ethereum blockchain is required to transfer ETH from EOA or for smart contracts to change state, perform an action, or read a state of a contract. A transaction can only be initiated from an EOA and it can further initiate other internal transactions.

For example, an EOA wallet initiates a transaction to call a function of contract A, and this contract calls another function of contract B. In this case, a transaction is initiated from an EOA to contract A and then an internal transaction is initiated from contract A to contract B.

Transaction fields

A user first fills the required fields for creating a new transaction. Then, they sign the transaction with their EOA's private key. This signed transaction data is then broadcast to the Ethereum blockchain network via any Ethereum client. The Ethereum client also first verifies that the transaction is valid and signed using the EOA's private key only, the address of which is in the from field.

From

From is always the public address of the EOA that initiated the transaction. In Ethereum, only an EOA account can initiate a transaction. Contract accounts cannot initiate a transaction on their own. The transactions further initiated by contracts are called internal transactions (on etherscan.io) in general terms; however, as per the Ethereum's yellow paper, these transactions are called message calls.

To

To is always the public address of an EOA or a contract. When To is an EOA address, then this transaction is initiated to send some ETH. When To is a contract address, then this transaction is initiated to call a method of a contract or to send ETH to contract..

Value

A transaction can have either zero or nonzero for the value field. Normally, the value field is nonzero when sending ETH to an EOA. However, when initiating a transaction to a contract, the value field either has a zero or nonzero value based on the contract function.

Gas limit

The gas limit is the maximum limit of gas units that will be required by the EVM to execute the transaction. We discussed this in detail earlier in this chapter.

Gas price

The gas price is the cost per gas unit you are willing to pay. We discussed this in detail earlier in this chapter.

Nonce

The nonce field in a transaction always starts with 0 (zero) for an EOA that initiates the transaction. The nonce keeps increasing by one for every transaction an EOA initiates. The nonce also specifies the execution order of the transactions initiated from an EOA. When multiple transactions initiated from an EOA are in the pending state, then the EVM executes a transaction that has a lower nonce value for that EOA.

Data

Data represents the hexadecimal data that you want to transfer along with a transaction. This data field is required to call a method of a contract.

When transferring ETH from one EOA to another EOA, the data field is empty, which means the value of the data field will be 0x. However, when we want to execute a method of a contract, the data field will hold the hexadecimal value of the method and its arguments:

Transaction data in hexadecimal from etherscan.io

The preceding hexadecimal transaction data, once decoded, represents the transfer() method call (called on an ERC20 contract) with the _to and _value parameters passed in. As you can see, 0xa9059cbb (four bytes data) is the hex code for the transfer() method, and the following method parameters are passed in:

Hexadecimal transaction data decoded on etherscan.io

Once the previous transaction data is decoded on the etherscan.io website, it shows the first four bytes of the function signature and the parameters passed to the function. 

Transaction hash

A user doesn't have to set the transaction hash field. A transaction hash is assigned to every transaction when it contains valid transaction fields and is broadcast to the Ethereum blockchain. The transaction ID is the SHA-256 hash of the signed transaction data. Using this transaction ID, we can track the status of the transaction and can cancel or drop the transaction before it gets executed.

Transaction status

The transaction status lets the user know the status of a transaction. Transaction change or dropping a transaction is allowed in different states. A transaction can be in any of the following three statuses:

Transaction status diagram

As shown in the preceding diagram, transactions are always initiated by an EOA. Let's go through each of the statuses.

Pending status

When a transaction is submitted to the Ethereum network, it first goes into the pending status, waiting to be executed by the nodes. A transaction can be in the pending state for a longer duration if the gas price is set very low in the transaction and the nodes are busy processing other higher gas price transactions.

During the pending state, the transaction initiator is allowed to change the transaction fields at any time. They can do so by sending another transaction with the same nonce. 

Success status

A transaction that is executed successfully without any failure is called a successful transaction. A successful transaction means the intended operation is executed successfully by the Ethereum blockchain and verified by other nodes present in the Ethereum blockchain network:

A successful transaction on etherscan.io

Fail status

A transaction is executed and, during the execution, it may fail due to an error in the contract method execution. Transaction failure also depends on the contract definition. Transaction execution failure could also occur in case of a low gas limit, and transactions will fail with the OutOfGasException error:

Failed transaction on etherscan.io

Dropped status

A transaction is sometimes dropped from the Ethereum network for several reasons:

  • If the transaction was in the pending state for a very long duration
  • It has a very low gas price as compared to other pending transactions
  • The maximum number of pending transactions the Ethereum blockchain can handle are in the pending transaction pool

Once a transaction is dropped, all  the gas and ETH is returned to the originating wallet:

Transaction dropped on etherscan.io
The dropped transaction is not included in any block on the blockchain, as if it didn't happen or wasn't initiated. There is no trace of those transactions on the blockchain; you can only see these dropped transactions on etherscan.io (block explorer) for some time. Later, they may be removed from etherscan.io as they are not part of the blockchain. However, successful and failed transactions are included in the block of the Ethereum blockchain and their status can be checked from any blockchain client.

Transaction operations

Two types of operations are allowed on an already initiated transaction. To do either of these operations, a transaction must be in the pending status.

Replace/update

If a user wants to change some transaction fields they have initiated, they can do so by changing the values and keeping the same nonce in the new transaction. A replace transaction is mostly used when a transaction is in the pending status for a longer duration because of the low gas price, and just increasing the gas price of the same transaction could increase the likelihood of it getting executed by the Ethereum network.

Cancel

A transaction can be canceled by the same EOA that initiated the same transaction. If a user feels that they have initiated a wrong transaction accidentally, they can cancel that transaction by initiating another transaction having the same nonce and changing the To fields to their own EOA address. This new transaction must have a higher gas price. A higher gas price is set to ensure that the new transaction will be given priority over the older transaction and can cancel the transaction as soon as possible. This type of transaction is called a self transaction

Testnets

There are multiple testnets available where you can deploy and test your contract's functionality. These testnets are the replica of the mainnet (production) Ethereum blockchain, so these testnets also behave the same. If your contracts are working on testnets, then they will work perfectly on the mainnet as well.

However, to deploy and test your contracts on these testnets, you need to have some testnet ethers for that testnet. You can get those testnet ethers from a faucet for free from the following locations:

Testnet Faucet link Block explorer
Kovan https://faucet.kovan.network/ https://kovan.etherscan.io/
Rinkeby https://faucet.rinkeby.io/ https://rinkeby.etherscan.io/
Ropsten https://faucet.metamask.io/ https://ropsten.etherscan.io/

 

If you want to test your contracts locally, then you can either install and set up your private testnet or you can use the Ganache tool.