Now the fun part for the more technically minded folks who read my blog, which is most of you. In this final installment on crypto and blockchain I’ll dig into the more technical aspects of these technologies.
This is where I started, several months back: I wanted to dig into blockchain and smart contracts to revisit the state-of-the-art. I hadn’t intended to dig more deeply into crypto history and variants, legal questions, and blockchain use cases. But I soon discovered that it is difficult to understand how the technologies work and fit together unless you have an understanding of the domain and jargon. And it was interesting…
Web 3.0
One term commonly used when referring to blockchain related technologies is Web 3.0, or W3. The definition for W3 has not completely stabilized, but it is intended to indicate a new generation of web technologies, including blockchain.
Web 1.0 (1990s and 2000s)
Retroactively, the initial web technologies are referred to as Web 1.0. These technologies include HTML, URLs, HTTP, Perl, and relatively simple server technologies. The web pages created were usually static–you had to surf to a different page to see fundamentally different content.
Web 2.0 (2000s to present)
The creation of more powerful browser and server technologies such as CSS, HTML5, Ajax, powerful UI frameworks, Node.js, PHP, etc., led to more dynamic pages. A common W2 implementation uses a Single Page Application (SPA) where a single page hosts all of the controls and data, much like a traditional desktop or native mobile application. Rather than surfing to new webpages, new content is dynamically loaded into sections of the current page.
Web 2.0 is generally understood to include a grab bag of other ideas including responsive applications (which can adjust to differing screen dimensions and orientation), user generated content, and algorithms to customize and target content.
The rise of dominant internet goliaths like Google, Amazon, and Meta is considered both a driver and consequence of these new technologies.
Web 3.0 (2010s to present)
The definition of Web 3.0 is still evolving. It usually includes decentralized apps–internet products not completely controlled by a single company, group, or government. Technologies that facilitate decentralized, trustless, and permissionless coding, such as blockchain, consensus algorithms, cryptographic techniques, cryptocurrencies are considered key components.
Sometimes AI, ML, LLMs, IoT, AR, and other modern alphabet-soup technologies are included, to the extent that they facilitate strong personalization of content, smart access to information, and different ways of creating, viewing, or processing it.
Some W3 proponents invoke the same sort of utopian ideals that were common in the very early days of the internet. They hope that the new technologies will fulfill the promise of free and democratic access to information (and applications) that the nascent web held out, but never delivered. Instead of a wonderland of free, truthful information, and access to modern functionality, they point out that we have a deluge of ads, disinformation, and tightly controlled functionality managed by a small number of mega-company gatekeepers. It is understandable that people are hopeful and excited by the new paradigm.
Time will tell if W3 develops as clear of a separation as the W1-W2 transition, and if it does in fact put more power in the hands of people. But there is no doubt that W3 technology continues the evolution of web technology.
Successful Decentralization
Creating a distributed platform that supports independent access to valuable data requires a bit of thought and planning. How do you strike a balance between distributed information that can be safely and widely shared while still protecting sensitive information?
In the Web 2.0 world you would avoid the question by creating a private application running on the cloud. Users would pay you for the right to access the data through a custom app. Under the hood their access would be through a REST API, using JSON. Access would be protected by OAUTH. The actual data would be stored in a relational or NoSQL db. Even if your service was free (or ad supported), you would still need to pay for the infrastructure to host the service.
In the Web 3.0 world this is addressed by creating trustless, permissionless systems which are hosted by volunteers. Don’t interpret those words as meaning that you cannot trust the system or that no one can get permission to use it. Rather, they mean that there is no need to trust in any particular entity or to obtain permissions from them. The system is set up so that trust (security) is built in from the start, safely allowing everyone access to data and functionality without the need for special permissions.
Blockchain systems meet this criteria by working as freely distributed systems with layers of cryptographic security built in. These systems include an incentive to run them (earning crypto) otherwise they would be freely distributed in theory, but unused in practice.
Security
Blockchain systems use a combination of cryptographic techniques including encoding of sensitive data, digital signatures, hashing, zero-knowledge proofs and Merkle trees. All of this is intelligently combined to allow visibility into the broad strokes of a transaction, but obscure the private details. So it is possible to verify that a particular account transferred crypto to another account, but not to know who the people behind the accounts are, or to modify the transaction after the fact.
This extensive use of cryptography is where the “crypto” in cryptocurrencies comes from. Each of these techniques is interesting and well worth a deeper dive for technical folks, but beyond the scope of this high level survey. The security they provide has proven to be sufficient for modern applications, but it is possible that quantum computing will eventually be able to break it, once it is powerful enough to break current encryption in general.
Agreement
Cryptographic techniques provide the low-level security of blockchain systems, supporting their free distribution while maintaining privacy. But there is still a need to determine which transactions are valid and should be added to the permanent record.
For example, you could use two accounts to create a transaction where you grant yourself a million Bitcoin, from one to another. Even though this transaction is fraudulent, it could be properly encrypted and signed.
These and other sorts of problematic transactions are prevented by the consensus mechanisms. Each blockchain system has a procedure in place for validating the transactions that they are given, before permanently committing them to the chain.
Every node of a blockchain system attempts to validate the transactions that it receives, perform the necessary calculation to earn crypto, and revalidate the transactions that other nodes have passed. The transactions must pass a certain threshold of agreement before the transactions (grouped into blocks) are added to the chain. This means that a rogue node can attempt to add fraudulent transactions but these will be invalidated and rejected by other nodes. There is quite a bit of depth to this issue, known as the Byzantine Generals Problem. Note that revalidating blocks is orders of magnitude easier than validating them in the first place, since the extensive calculations required to earn crypto do not need to be replicated.
Permissioned systems rely upon a central or limited number of authorities to reach consensus. In this way permissioned systems bear some resemblance to Web 2.0 approaches.
Nodes and Mining
Because permissionless blockchains are open, you are free to run a node yourself. You can get the source code for Bitcoin from github and the instructions on running a node online.
Requirements and Costs
The Bitcoin instructions claim that it only takes 2GB of memory and 7GB of storage. For Ethereum it is suggested you have 16GB of memory and 1TB of storage (this seems more likely). In addition to system resources you are expected to have good, unlimited internet access and will be willing to run the services for at least six hours a day. Since Ethereum switched to Proof of Stake last year, you must deposit 32ETh (about $50K) into a deposit contract, in order to join the actual mining effort.
The total hash rate for Bitcoin is estimated to be around 31 petahash/second (31,000,000,000,000,000). That means that all of the high power computers stacked up in all of the massive facilities around the world calculate that many hashes, a key part of the work involved in earning Bitcoins. With all of that competition it is estimated that it currently takes roughly 155,000 kWh to earn a single Bitcoin (a US household uses 900 kWh per month). Given all of this your odds of actually earning a Bitcoin on your own computer are something like 1 in 1.1 billion.
With all of the associated costs and the odds stacked against you, very few individuals are trying to mine popular coins on their own systems at this point in time. A more popular approach is to join a mining pool, where your results and those of the others in your pool are joined, based upon some criteria. If one person succeeds you share the proceeds.
You could run a node for technical expertise, research or fun, but probably not for profit.
Build Your Own Chain
There are quite a few ways you can create entirely new chains to either improve upon existing ones, or solve new problems. Common bases these are built off of include Ethereum, Corda, Hyperledger. The tools do the heavy lifting of preparing a blockchain system. You need to determine how yours will add unique value.
- Hyperledger Fabric is an open-source blockchain platform that is designed for enterprise use. Hyperledger Fabric is supported by a number of large companies, including IBM, Intel, and JP Morgan.
- R3 Corda is an open-source blockchain platform that is designed for financial services use. R3 Corda is supported by a number of large banks and financial institutions.
- Ethereum is both the basis for the Ethereum chain, and available as an open source project you can modify and build yourself. It is popular as a basis for new chains because of its strong support for smart contracts.
As noted in the blog installment discussing use cases, providing a blockchain system is just part of the solution–you also need to ensure that you have a compelling pitch to get a critical mass of others involved in running your chain, whether it is within an specific industry or the public at large.
Adding to a Chain
One topic that has not yet been addressed is the procedure for adding new transactions (records or data) to a chain. In the world of W2 you would call an API. Since blockchain runs in a distributed fashion, where do you send your request?
The simplest way for individuals is to use a digital wallet like MetaMask or a full fledged crypto financial company like Coinbase. These services take care of most of the low-level details involved in creating transactions, sending them to a crypto chain, and informing you with the results.
More technically inclined folks who need to programmatically connect to a chain can use one of several easy-to-use libraries to create valid transactions and send them to a chain. If you go down this route you will be expected to provide specific ids for participants, and which specific crypto networks you want to connect to.
The most advanced approach to creating and submitting new transactions is to host your own node for whichever crypto network you want to participate in. This way you can directly add your transactions. There are many caveats to this approach, depending upon the network you choose. For example, Ethereum requires a Proof of Stake–money held in escrow to ensure your good behavior. If you submit invalid or fraudulent transactions you may lose this money.
Developing Smart Contracts
All the interesting cryptographic technologies and consensus mechanisms combine to create useful cryptocurrency applications. It is the addition of programmable functionality that transforms them from applications into extensible platforms. In theory a smart contract can do anything that any other program can do.
Because smart contracts can run on multiple distributed nodes of a chain simultaneously they are often referred to as Decentralized Applications (dapps), particularly when combined with external (W2 style) frontends.
What Can a Smart Contract Do?
The advantage of smart contracts is that they execute their code (contract) in exactly the same way every time. You don’t need a lawyer, arbiter, or middleman. Fees can be set in stone. Transactions that use smart contracts can be anonymous.
They are considered legal contracts in some, but not necessarily all jurisdictions. This means that it is possible to use them in circumstances where a digital signature can indicate acceptance or compliance with a contract.
Practical usage is more nuanced. While the contract code should run predictably and not require outside assistance, humans often need advice, clarification, assistance, and follow up. You can write smart contracts for real estate transactions and legal contracts, but will people comfortably use them?
Another peculiarity of smart contracts is that they often require payment up front (or at least escrow up front), as there is no simple mechanism to collect after the fact. This is not a problem with simple exchanges of money for an immediately delivered digital asset like an NFT. It gets more complicated when there are real world assets or future payments involved. There are elaborate mechanisms that can be included in contracts for refunds or reimbursement, but properly implementing these is difficult.
There has been progress in addressing the foibles of smart contracts, but there still is more work needed. Over time standardized systems and libraries for smart contracts, along with time-tested paradigms will improve their creation, comprehension, utility, and reliability.
What Smart Contracts Look Like
Smart contracts are executed by VMs, much like Java and so many other languages. While you can code directly in the byte code for the VMs, various high-level languages are available to create programs. Solidity is the most popular language, with the most developed ecosystem, at least for Ethereum-based systems.
In the example below I show a simple Hello World app, written in Solidity. Any programmer can read it and get the general gist, but, as always, the devil is in the details. Rather than attempting to teach the language, I will point out some of the higher level considerations:
const contract = require("../artifacts/contracts/HelloWorld.sol/HelloWorld.json"); const API_URL = process.env.API_URL; const API_KEY = process.env.API_KEY; const PRIVATE_KEY = process.env.PRIVATE_KEY; const CONTRACT_ADDRESS = process.env.CONTRACT_ADDRESS; // Provider - updated as per: https://docs.alchemy.com/discuss/64a6c38b109220000a8e357d const alchemyProvider = new ethers.providers.JsonRpcProvider(API_URL); // Signer const signer = new ethers.Wallet(PRIVATE_KEY, alchemyProvider); // Contract const helloWorldContract = new ethers.Contract( CONTRACT_ADDRESS, contract.abi, signer ); async function main() { const message = await helloWorldContract.message(); console.log("The message is: " + message); console.log("Updating the message..."); const tx = await helloWorldContract.update("This is the new message."); await tx.wait(); const newMessage = await helloWorldContract.message(); console.log("The new message is: " + newMessage); } main();
Simple Solidity Hello World, from Visual Studio Code
Smart contracts will be stored on a chain where they need to be verified and uniquely identified, so they require information such as the chain (network) to use, their address (a unique identifier), and the person who is creating the contract (signer). The Alchemy SDK is used in this code to simplify the processes involved with this information.
There is no interactive mode so a log is used to output the messages. This information will be stored on the chain, properly identified by the contract and transaction. It is possible to avoid the costs associated with storing information on the chain by using off-chain storage for cases where your contract needs to refer to or generate medium to large amounts of data.
The Ethereum Virtual Machine (EVM), which this example is designed for, is single-threaded. The hint is the number of times async and await occurred in the code. There are many sources that can describe Ethereum and its EVM. At least a basic knowledge is required to understand how the code works and why it does what it does.
Contract Costs
If you submit this code to an actual Ethereum-based system it will cost you crypto both to add it to the chain and each time it is executed. Storage and run time will also raise costs.
- From $500 to $5000 to deploy the contract to a production chain, depending upon contract size, initial storage requirements, and network congestion.
- The cost of running the contract, which can be shifted on to the user, can easily be $100.
- A single Kb of storage is about $50.
- The maximum cost of executing your code can be estimated ahead of time (these fees are called gas), and paid by each user. You must optimize your code to avoid exorbitant gas fees.
- As prices are actually in the native currency for Ethereum, ETH, they vary dramatically with the price of the currency.
Given that it will be permanently ensconced on the chain once submitted, developers work with testnets, such as Sepolia, where they can freely submit their applications and test them, until they have them exactly right. Even on a testnet you need test crypto in order to simulate the proper functionality and costs, so you use a faucet to generate test currency.
Complications
Every useful software technology has some level of complexity. For example, modern UI frameworks like React and Vue require expertise with JavaScript, CSS, HTML, and a whole host of specialized tools and libraries. Smart contracts have the usual set of special cases, kludgy workaround, and outright bugs, but also present two additional types of complications:
1. Pervasive Limitations
The execution environment, including memory, storage, communication, environmental information, user interaction, and usage of outside functionality is restricted due to the append-only nature of blockchains and security considerations.
Smart contracts can call other contracts, supporting some modularization of solutions. They can even make external API calls with the assistance of an oracle, a service built specifically to provide external data to chains. One common design pattern with dapps is for the front end to gather data, call appropriate contracts, and process outputs, repeating this process until the desired solution is reached. All of these interactions can take a significant amount of time, beyond the processing delay and execution time of the contract alone. And the introduction of external oracles or dapps adds another layer which may reduce visibility and security.
2. Critical Quality Constraints
Since it is not possible to modify the code once it is placed on the chain, it must be completely free of bugs the first time it is released.
With most modern software you can ship once you have reached a certain quality threshold. Generally this is when there are possibly minor issues, but no major ones. After release, you can issue patches and updates as needed. With smart contracts you must absolutely ensure that your code is both free from bugs and algorithmically correct from day one. Once submitted, any issues are set in stone. While you can issue a new application in place of an old one, you cannot modify or delete the original.
Unintentionally Complex Example
One example I found that unintentionally illustrates the complexity inherent in smart contracts is a video description of Lightning, an L2 network that sits on top of Bitcoin. This video features an example of a smart contract to set up a bar tab, which starts out simple enough. However, once they start to get into the edge cases the complexity mounts. Real world challenges, such as life cycle stages, protecting against bad actors, and timing issues layer increasing intricacy on their solution.
Lightning, along with other L2 and sidechain solutions, are specifically designed to speed up transactions, add functionality, and reduce usage costs. They are part of the growing blockchain ecosystem which attempts to build off of and resolve issues in earlier implementations. But as my discussions have shown, in spite of attempts at improvement, the current state of blockchain and crypto remains challenging, obfuscated, and often not enterprise-ready.
The Future of Crypto and Blockchain
Despite the length of my posts, I was only able to skim the surface of crypto and blockchain. It is impossible to predict their future, but during the course of my research I definitely developed some opinions.
Digital Currency, not Necessarily Crypto
Digital currencies will continue to gain traction over time. But while a few major cryptocurrencies have tremendous momentum right now, it is not certain that permissionless blockchain systems will dominate in the future. Many popular cryptocurrencies behave much more like speculative investments in the midst of a bubble, and governments are starting to regulate them as such. Permissioned blockchains and even non-blockchain based crypto could take advantage of the flaws inherent in the first generation of currencies and, particularly with backing from governments and/or large financial institutions, supplant them.
Choose the Right Tech for the Job
The vast sums of money made in crypto have been both boon and bane for blockchain. One the one hand encouraging a phenomenal number of proposals for the technology. On the other, leading to a significant number of false starts and failures in areas where blockchain should never have been attempted. As a result blockchain interest has varied in a series of technology bubbles and bursts.
Weekly blockchain interest over the past 5 years, from Google
To date most of the sustained success for blockchain has occurred in solutions that are tied closely to crypto. This does not mean that it cannot be used elsewhere, but it has proven to be much harder to find success in other areas. Indeed, there have been so many failures that companies may eventually shy away from most blockchain solutions, even where it could be a reasonable choice.
While preparing these conclusions my news feed presented me with yet another blockchain stumble: Reddit is moving away from their blockchain system for community rewards. After reading my posts you should be able to read this article and have a good understanding of both their hopes for the initial system and the types of challenges they eventually encountered.
Blockchain is an ingenious combination of cryptography and other technologies to create a new class of distributed applications. There is constant innovation in the field as new solutions attempt to address unique problems or improve upon existing implementations. I enjoyed learning more about both the use cases and technologies involved, and believe that for technology professionals it is a topic worth investigating.
However, I still believe in the advice I wrote a few years back: Pick technologies not because they are new and exciting, but because they currently provide your business with tangible benefits in excess of their risks and costs.