TL;DR
Solana is a high-performance blockchain platform that uses a unique technical architecture to achieve high throughput and low latency. Its core technologies include the Proof of History (POH) algorithm to ensure transaction order and global clock, Leader Rotation Schedule and Tower BFT consensus mechanism to increase the block rate. The Turbine mechanism optimizes large block propagation through Reed-solomon encoding. Solana Virtual Machine (SVM) and Sealevel parallel execution engine speed up transaction execution. These are all architectural designs for Solana to achieve high performance, but they also bring some problems, such as network downtime, transaction failures, MEV problems, rapid state growth and centralization problems. We also focus on the problems caused by this mechanism in this article.
Solana's ecosystem is developing rapidly, and various data indicators have grown rapidly in the first half of the year, especially in DeFi, infrastructure, GameFi/NFT, DePin/AI and consumer applications. Solana's high TPS and consumer application-oriented strategy and the ecological environment with weak brand effect provide entrepreneurs and developers with abundant entrepreneurial opportunities. In terms of consumer applications, Solana has demonstrated its vision for promoting the application of blockchain technology in a wider range of fields. By supporting applications such as Solana Mobile and building SDKs specifically for consumer applications, Solana is committed to integrating blockchain technology into everyday applications to increase user acceptance and convenience. For example, applications such as Stepn provide users with novel fitness and social experiences by combining blockchain and mobile technologies. Although many consumer applications are still exploring the best business model and market positioning, the technical platform and ecosystem support provided by Solana undoubtedly provide strong support for these innovative attempts. With the further development of technology and the maturity of the market, Solana is expected to achieve more breakthroughs and success cases in the field of consumer applications.
Although Solana has gained significant market share in the blockchain industry with its high throughput and low transaction costs, it also faces fierce competition from other emerging public chains. As a potential rival in the EVM ecosystem, Base’s number of active addresses on the chain is growing rapidly. At the same time, although Solana’s total lock-up volume (TVL) in the DeFi field has hit a record high, competitors such as Base are also rapidly occupying market share. , the financing amount of Base Ecosystem also surpassed Solana in Q2 quarter for the first time.
Although Solana has made certain achievements in technology and market acceptance, it needs to continue to innovate and improve to meet the challenges from competitors such as Base. In particular, in terms of improving network stability, reducing transaction failure rate, solving MEV problems, and slowing down the growth rate of state, Solana needs to continue to optimize its technical architecture and network protocols to maintain its leading position in the blockchain industry.
Technical Architecture
Solana is famous for its POH algorithm, Tower BFT consensus mechanism, and the high TPS and fast Finality brought by Trubine data transmission network and SVM virtual machine. We will briefly introduce how its various components work, how to achieve its high-performance goal for architectural design, and the disadvantages and derived problems caused by this architectural design.
POH Algorithm
POH (Proof of History) is a technology for determining global time. It is not a consensus mechanism, but an algorithm for determining the order of transactions. POH technology is derived from the most basic cryptographic SHA256 technology. SHA256 is usually used to calculate the integrity of data. Given an input X, there is only one output Y, so any change to X will result in a completely different Y.
POH sequence, source: Solana white paper
POH sequence diagram, source: Solana white paper
In Solana's POH sequence, the integrity of the entire sequence can be ensured by applying the sha256 algorithm, which also determines the integrity of the transactions. For example, if we package the transactions into a block and generate the corresponding sha256 hash value, then the transactions in this block are determined, and any changes will cause the hash value to change. After that, this block hash will be used as part of the X that will be used as the next sha256 function, and then the hash of the next block will be added, so the previous block and the next block are both determined, and any changes will result in a different new Y.
This is the core meaning of its Proof of History technology. The hash of the previous block will be used as part of the next sha256 function, similar to a chain. The latest Y always contains proof of history.
Transaction Flow architecture diagram, source: Solana white paper
In Solana's transaction flow architecture diagram, the transaction process under the POH mechanism is described. Under a rotation mechanism called Leader Rotation Schedule, a Leader node will be generated among all on-chain validators. The Leader node collects transactions and sorts them for execution, generates a POH sequence, and then generates a block to propagate to other nodes.
Leader election mechanism, source: Helius
In order to avoid single point failure at the leader node, a time limit is introduced. In Solana, the time unit is divided by epoch, each epoch contains 432,000 slots (time slots), each slot lasts 400ms, and in each slot, the rotation system will assign a leader node in each slot. The leader node must publish a block within the given slot time (400ms), otherwise, the slot will be skipped and the leader node of the next slot will be re-elected.
In general, the Leader node uses the POH mechanism to finalize all historical transactions. Solana's basic time unit is Slot, and the Leader node needs to broadcast blocks within a slot. Users send transactions to the Leader through the RPC node, and the Leader node packages and sorts the transactions and then executes to generate blocks. The blocks are propagated to other validators, and the validators need to reach a consensus through a mechanism to reach a consensus on the transactions and order within the block. The consensus uses the Tower BFT consensus mechanism.
Tower BFT consensus mechanism
Tower BFT protocol, source: Helius
The Tower BFT consensus protocol comes from the BFT consensus algorithm and is a specific engineering implementation of it. The algorithm is still related to the POH algorithm. When voting on a block, if the validator's vote itself is a transaction, then the block hash formed by the user transaction and the validator's exchange can also be used as a historical proof, and the transaction details of the user and the voting details of the validator can be uniquely confirmed.
The Tower BFT algorithm stipulates that if all validators vote on the block and more than 2/3 of the validators vote approve, then the block can be confirmed. The advantage of this mechanism is that it saves a lot of memory, because only the hash sequence needs to be voted on to confirm the block. However, in the traditional consensus mechanism, block flooding is generally used, that is, a validator receives a block and then sends it to the surrounding validators, which will cause a lot of redundancy in the network because a validator receives the same block more than once.
In Solana, due to the large number of validator voting transactions, the efficiency brought by the centralization of Leader nodes and the 400ms slot time, the overall block size and block frequency are particularly high. When large blocks are propagated, they will also cause great pressure on the network. Solana uses the Turbine mechanism to solve the problem of large block propagation.
Turbine
Voting Icon
Turbine block propagation mechanism, source: Helius
The Leader node splits the block into shredded sub-blocks through a process called Sharding, with the size of the block measured in MTU (maximum transmission unit, the maximum amount of data that can be sent from one node to the next without splitting it into smaller units). The integrity and availability of the data are then guaranteed by using the Reed-solomon erasure code scheme.
Reed-solomon coding scheme, source: Helius
By dividing the block into four Data Shreds, and then using Reed-solomon encoding to encode four packets into eight packets to prevent packet loss and damage during data transmission, this solution can tolerate a packet loss rate of up to 50%. In actual tests, Solana's packet loss rate is about 15%, so this solution is well compatible with the current Solana architecture.
In the underlying data transmission, the UDP/TCP protocol is generally considered. Since Solana has a high tolerance for packet loss rate, the UDP protocol is used for transmission. The disadvantage is that it will not retransmit when the packet is lost, but the advantage lies in faster transfer rates. On the contrary, the TCP protocol will retransmit multiple times when packets are lost, which will greatly reduce the transmission rate and throughput. With Reed-solomon, this solution can significantly increase Solana's throughput. In a real environment, Throughput can be increased by 9 times.
Schematic diagram of layered propagation, source: Helius
After Turbine shards the data, it uses a multi-layer propagation mechanism to propagate it. The Leader node will hand over the block to any block validator before the end of each slot. The validator will then shard the block into Shreds and generate erasure codes. The validator will then start Turbine propagation. First, it must be propagated to the root node, and then the root node will determine which validators are located at which layer. The process is as follows:
Create a node list: The root node aggregates all active validators into a list, and then sorts them according to each validator’s stake in the network (that is, the amount of SOL staked). Those with higher weights are in the first layer, and so on.
Node Grouping: Each validator in the first layer will then create its own list of nodes to build its own first layer.
Layer formation: Divide the nodes into layers from the top of the list. By determining the depth and breadth values, the general shape of the entire tree can be determined. This parameter affects the propagation rate of shreds.
Nodes with a higher stake, when divided into layers, can obtain complete shreds in advance and restore the complete block. However, due to transmission loss, the probability of obtaining complete shreds for nodes in the later layers will be reduced. If these shreds are not enough to build a complete fragment, the Leader will be required to retransmit directly. At this time, data transmission will be carried out inside the tree, and the nodes in the first layer have already built a complete block confirmation. The longer it takes for the validators in the later layers to vote after completing the block construction.
The idea of this mechanism is similar to the single node mechanism of the Leader node. There are also some priority nodes in the block propagation process. These nodes first obtain shreds fragments to form complete blocks to achieve voting consensus. Pushing redundancy to a deeper level can significantly speed up Finality and maximize throughput and efficiency. Because in fact the first few layers may represent 2/3 of the nodes, then the votes of subsequent nodes will not matter.
SVM
Solana can process thousands of transactions per second, mainly due to its POH mechanism, Tower BFT consensus and Turbine data propagation mechanism. However, as a virtual machine for state transition, if the Leader node is executing transactions and the SVM processing speed is slow, the throughput of the entire system will be reduced. Therefore, Solana proposed the Sealevel parallel execution engine to speed up the execution of transactions for SVM.
Sealevel parallel execution diagram, source: Xangle
In SVM, instructions consist of 4 parts, including program ID, program instructions, and a list of accounts that read/write data. By determining whether the current account is in a read or write state and whether the operation to change the state is conflicting, the account's transaction instructions can be parallelized without conflicting states, and each instruction is represented by a Program ID. This is one of the reasons why Solana's validators have high requirements, because the validator's GPU/CPU is required to support SIMD (single instruction multiple data) and AVX advanced vector expansion capabilities.
Ecological Development
Solana Ecosystem Landscape
In the current development of the Solana ecosystem, it is increasingly leaning towards practical utility, such as Blinks, Actions, and even Solana Mobile, and the official supported application development direction is also more inclined towards consumer applications rather than unlimited involution of infrastructure. With Solana's current sufficient performance, the application types are more abundant. As for Ethereum, due to its low TPS, the Ethereum ecosystem is still based on infrastructure and expansion technology. When the infrastructure cannot carry applications, it is impossible to build consumer applications. This has caused an unbalanced state in which funds are invested too much in infrastructure but too little in applications.
DeFi
DeFi Landscape
Among the DeFi protocols on Solana, there are a large number of projects that have not issued tokens, including Kamino (First Lending), Marginfi (Lending + Restaking), SoLayer (Restaking), Meteora, etc. Due to Solana’s united ecological atmosphere, usually when a project is issuing tokens, other projects will try to avoid it as much as possible to attract enough market attention.
DEX market share, source: Dune
Currently, competition in the entire DEX sector is fierce, and its leaders have also undergone multiple migrations, from Raydium, Orca to the current dominant position of Jupiter.
The initiator of DEX transaction, source: Dune
It is worth noting that about 50% of DEX transactions are initiated by MEV bots, mainly because of its low fees and active Meme transactions, which have fostered the profitability of MEV. This is also one of the main reasons for the frequent failures and downtime of user peak transactions.
Solana TVL, source: Deflama
As the price of SOL rises, the DeFi protocol on Solana has also experienced an explosive increase in its USD nominal TVL. The rising trend of its TVL has not stopped, and a new wave of rising trend has formed.
In short, although the competition in the Solana track is fierce, there are still changes. Unlike Uniswap on Ethereum, which occupies the brand mind of users, DEX, which should have high stickiness and network effects, will also face the risk of being replaced. The transactions of the Solana main chain are filled with MEV bots, which has caused great user experience problems for users and still needs to be solved. In the overall direction, Solana's TVL is still growing very rapidly, and its subsequent DeFi ecological development is still very worth looking forward to. In addition, the brand mind of these applications is not strong in occupying users, which is a potential driving force for entrepreneurs to choose chains.
Infrastructure
Infrastructure Landscape
In terms of infrastructure construction, the main leaders are the oracle Pyth and the cross-chain bridge Wormhole, as well as some targeted solutions that the public may not know much about, such as:
Jito Labs: Focusing on building MEV solutions on Solana, its Jito Labs client built Bundle and pseudo-emempool to give researchers MEV. Its current market share exceeds 50%. In addition, the staked SOL of its LSD protocol Jito is close to 12 million, and is still growing rapidly.
Helius: As an active contribution community on Solana, Helius has the most comprehensive research on Solana and contributes code through research.
GenesysGo: Its product ShdwDrive is a data storage project in Solana. It is committed to supporting projects that can be commercialized, including social data, website hosting and other businesses. It is still in the testnet stage. At the same time, its parent company GenesysGo is also building various public goods and research for the Solana community.
In addition, Solana still has a large number of projects worth exploring waiting for the Chinese community to explore. We do find that these infrastructures have a huge influence on Solana's protocol-level construction, ecological development, and community, and there may be opportunities to further explore their potential, whether through investment or cooperation.
Gaming / NFT
Gaming / NFT Landscape
Solana also has a rich GameFi and NFT ecosystem, among which Mad Labs occupies a relatively important position in the entire Solana ecosystem. Many project airdrops will give priority to Mad Labs holders. In the past, DeGods was the leader. The NFT market has also undergone changes. In the past, Magic Eden was used by the most people, but now it has changed to Tensor.
DePin / AI
DePin Landscape
DePin data overview, source: DePin Scan
Currently, Render is the well-deserved leader in Solana's DePin market with actual business. With Solana's development strategy centered on practical applications, it has also caught the narrative of Depin in this round of recovery. In the first half of the year, a large number of new Depin projects were built on Solana, including io.net, Nosana, Shadow, etc.
Consumer
Consumer Landscape
Whether it is Solana Mobile or the column built specifically for consumer applications on the Solana Ecosystem website, the invention of Actions and Blinks, all illustrate Solana's vision for the commercialization and practicality of blockchain. The release of its Mobile collection is also to put the dapp on the web side on the mobile side, which is very in line with human nature and the development trend of the Internet. Therefore, applications are prone to explode on this soil, the most typical example being Stepn.
Looking at the consumer applications currently in operation, most of them have not yet found a good breakthrough, so they are still unable to realize a real application in the business world. This includes many factors, including single product innovation, single business model, weak Web2 marketing, demand for Gas Fees, and entry barriers for tokens.
However, consumer applications are the ultimate landing scenario for blockchain technology, and also determine the ceiling of the public chain. Therefore, Solana's exploration of mobile consumer applications is very necessary, and our long-term exploration in this direction is also necessary. Especially in the current Ethereum ecosystem, infrastructure is far greater than applications. Ultimately, infrastructure serves applications.
Payments
Payment Landscape
Wallets on Solana include Phantom, Backpack, TipLink, etc. Like DEX, the brand effect here is not strong, so entrepreneurs have more opportunities. In the past, the leading wallet was Phantom, but now it has changed to Backpack, which was built by Mad Labs. It is worth mentioning that Mad Labs is now also the NFT leader on Solana.
Solana stablecoin issuance status, source: Deflama
It currently cooperates with Paypal, Visa and other companies to conduct stablecoin payment transfers on the chain. This business scenario itself is very conducive to the fast Finality and low Gas Fee Solana chain. Currently, its on-chain stablecoin is in a state of slow growth.
Stablecoin transfer stack chart YTD, source: Artemis
Solana had impressive stablecoin transfer market share in the first half. But since June, its market share has declined significantly. Solana's performance was absolutely outstanding in the first half of the year, but its initial transfer data in the second half of the year showed a clear downward trend.
Competitor data
Number of active addresses on the chain, source: Artemis
Among all the public chains, Base is seen as a potential competitor of Solana in the EVM ecosystem. The number of active addresses on Base is increasing rapidly, and Solana is still in a stage of rapid growth despite its first-mover advantage. NEAR remains at a high level, but Aptos and Sui lag behind in the competition among public chains.
TVL comparison, source: Artemis
Solana’s TVL has also made significant progress in the Defi field. Its TVL has reached a record high and is still far behind other public chains. However, it is worth noting that Base is also in a stage of rapid growth.
Public chain stablecoin reserves, source: Allium
Currently, Solana's market share of stablecoin supply has been sluggish. Ethereum's market share has naturally shrunk due to the emergence of multiple chains, while Base's market share has quietly increased.
Financing data, source: Messari
In terms of capital market financing, the frequency of financing for the Base ecosystem has increased significantly in the last quarter, and has surpassed that of the Solana ecosystem. Therefore, the market share and capital financing of various on-chain data also show that the competition between Base and Solana in the market is valid, and this competitive pressure will be greater as Base matures. In addition, Base and Solana have similar visions, both of which hope to achieve the vision of Mass Adoption's Cryptio Native Consumer App with high TPS.
Technical challenges
Downtime
Solana's network architecture, such as the Gulfstream Leader election mechanism and the single-node risk of the Leader node, makes subsequent predictions of the Leader node feasible. As network transactions increase, the Leader node will be under great memory pressure, and the Leader node needs to be ready to retransmit blocks to the nodes in the Turbine tree at any time, otherwise the consensus vote cannot be completed. When a large number of DDoS attacks occur, system downtime caused by single-node failures becomes extremely frequent.
In short, the downtime is mainly due to the problem of not being able to produce blocks. It may be due to a single node failure caused by the Leader mechanism, causing problems in the block formation, or it may be that the consensus layer cannot reach a consensus on the block, resulting in the problem of not being able to produce blocks. Overall, this is closely related to Solana's own architecture and software testing process.
Transaction Failure
The proportion of failed transactions by users, source: Dune
Users who have used Solana should know that our transactions often fail to be submitted normally. After a while, the transaction is found to have failed, which results in a very poor user experience. As shown in the figure above, according to statistics, about 35% of transactions submitted by users fail and require users to submit multiple times. When there are large fluctuations on the chain, this proportion will be even greater.
The main reason for this is the network layer technology QUIC, which is a relatively new technology.
Network protocol hierarchy — 5-layer structure, source: Research Gate
QUIC (Quickl UDP Internet Connections) is a transport layer improvement proposed by Google for the HTTP 2.0 protocol. This experimental protocol is developed based on the UDP transport layer protocol and is also known as HTTP 3.0.
HTTP/2 and QUIC diagram, source: EMQX
TCP is more reliable than UDP, but UDP has a higher rate than TCP because TCP has a congestion control mechanism when packets are lost and retransmits lost packets. UDP has a high rate but low reliability. Google hopes to build a high-reliability and high-rate transport layer protocol, QUIC. The core feature of QUIC is independent logical streams. It allows multiple data streams to be transmitted in parallel on a single connection, and each stream can be processed independently. In contrast, TCP only supports a single data stream and needs to receive and confirm each message in the order it is sent.
Failed transaction diagram, source: bread
The main reason for Solana's downtime is the use of the QUIC experimental application layer protocol. Due to the speed of UDP and multi-channel transmission, and the desire to maintain the transmission of complete data, it will also design a mechanism to retransmit multiple packets in the event of packet loss. When the Leader node receives multiple transactions, it opens multiple channels through the QUIC protocol. However, the Leader node is a computer after all, and it can only handle a limited transaction capacity. Therefore, when a large number of transactions flow in, the Leader node will cut off certain channel connections, which will cause the transaction to be dropped. There is no set standard for how to choose the connection to be cut off (such as cutting off all connections with a fee lower than xxx), and whether all connections will be cut off is random. Therefore, this leads to a certain amount of black box operation space. The Leader node may prefer profitable MEV transactions and abandon users' low-value transactions.
MEV
In Solana's block generation mechanism, since RPC interacts directly with the Leader and adopts the FCFS principle, it does not have the Mempool like Ethereum. Due to the existence of Mempool and Ethereum's permissionless principle, Ethereum faces a more severe MEV problem in comparison.
MEV architecture, source: Helius
Jito Labs client currently occupies 50% of the client market share, so Jito Labs built a pseudo mempool by itself. Users enter a pseudo mempool through RPC and stay for about 200ms. Jito Labs provides an off-chain inclusion guarantee to ensure that all transactions in the bundle are included in the block. Searchers can bid for the opportunity to attack pending transactions. Searchers bid for the Bundle that maximizes profits, and then the Block Engine is responsible for finding the Bundle with the highest bid and submitting it to the Leader running the Jito Labs client.
This is the root cause of MEV, but MEV has its positive externalities and demand. If Jito Labs does not do pseudo mempool, other projects will do it, so Jito Labs chooses to take over this market to improve the mechanism of MEV and reduce negative externalities. Of course, this demand for MEV bots puts users in the most vulnerable position, because validators will charge fees, mev bots will get arbitrage profits, but users will suffer higher slippage and possible failed transactions.
State Growth
Solana's POH mechanism and Turbine consensus cause its blocks to be too large, which will cause the problem of state growth. At present, there is no definite answer to the size of the ledger, and the ledger is still growing at a rate of one block every 450ms in a practical environment, about 4PB per year (running at a maximum performance of 1GBPS). Currently, Solana's historical construction occurs after 2 epochs, which is about 4 days (a total of 100-200GB). The past data is stored in the Google Bigtable database.
Solana’s ledger data is not transparent, and the official has not disclosed much about the extremely high block size and potential impact caused by the pursuit of large blocks and high TPS throughput. The storage of the ledger is also completely dependent on a third party, because the official has also found that centralized databases such as Google have higher performance than Genesys Go, Arweave, etc. At present, these decentralized databases still have problems with commercialization. This extremely growing state and centralized hosting are one of the reasons why Solana has been criticized.
Outlook
Solana also released its future roadmap, including:
Improve the protocol for issuing tokens, including transfer encryption, Hooks, and metadata pointers.
Client improvements, including the lightweight client Tinydancer, the transitional client Frankendancer, and the final client Firedancer.
Supporting development components of the ecosystem: Gmaeshift focuses on game SDK, armada markets focuses on token life cycle improvement, SPE focuses on enterprise-level SVM blockchain, virtual machine improvement, etc.
We can see that Solana's POH algorithm and Turbine consensus mechanism both prioritize performance in the blockchain trilemma. The benefit is that they have the best performance in the current environment, which brings a wider range of applications that can be explored. And with Solana's strategic goal of consumer applications, there is a great possibility that some Mass Adoption applications will emerge. At the same time, the brand effect of projects on Solana is weak, so there are more opportunities for entrepreneurs.
Solana's main advantages in ecological development are DePin/AI and Meme, but we can also see that its ecological development has not yet reached the expected development, and the Consumer App has not yet been commercialized. In terms of competitors, there are also rising stars such as Base, and Base's financing amount and market share are rapidly increasing.
Solana has also faced some technical problems, including downtime, transaction failures, MEV, rapid state growth, and centralization, but the positive side of Solana is that it does not focus on redundant infrastructure construction, but relies more on the current TPS capacity to build consumer-oriented applications, and its roadmap also revolves around this. With the construction of more and more Layer2 and the launch of clients, the TPS of the SVM ecosystem will be taken to a higher level. Solana is still an oasis, with many ecological projects that have not been fully reached by capital, and there are many opportunities for entrepreneurs to explore.
