Written by: Kam
Compiled by: Luffy, Foresight News
This letter is addressed to the Hyperliquid development team, hoping the team can take the time to review this feedback regarding Hyperliquid blockchain management.
Key Points:
Due to the lack of open-source code, poor documentation, and reliance on centralized application programming interfaces (APIs), validators face significant challenges, leading to frequent jailing and unstable performance.
The testnet incentives have given rise to a black market for HYPE tokens, favoring transactions with whales rather than fairly selecting validators.
The rewards for mainnet validators are too low to meet high self-collateral requirements, and the degree of decentralization is limited, as 81% of the staked share is controlled by foundation nodes.
To compete with mainstream L1 blockchains, Hyperliquid must increase transparency, reduce staking centralization, implement a fair validator selection process, and strengthen interaction with external validators.
I started exploring Hyperliquid in December 2023, and it is an amazing crypto application. It is easy to use, offers a great user experience, and provides some unique features like vaults and the famous HLP. Currently, the assets managed by HLP exceed $350 million, and anyone can participate in Hyperliquid passively.
Seeing this platform perform so well and learning that Hyperliquid operates as a native L1 blockchain, I hope Chorus One (a staking solution) can participate in the Hyperliquid chain as an operator. I am an employee of Chorus One, which is one of the largest node operators in this field. Chorus One has been very active in the proof-of-stake industry since 2018.
After obtaining whitelist eligibility on October 17, 2022, Chorus One joined the Hyperliquid testnet. I would like to share our overall experience on the testnet with the Hyperliquid engineering team because even three months after joining the testnet, we have yet to have an opportunity to communicate with them.
During this time, we witnessed one of the most successful token launches of 2024: the launch of the HYPE token. At the same time, we experienced a testnet environment that was both interesting and challenging. I would like to mention some observed issues that I hope will be addressed in the coming days, weeks, or months.
Testnet Experience
So far, the experience on the testnet has been extremely challenging. Operators have almost no knowledge of how to run nodes, and the available resources are very limited. Besides that, we are basically exploring in the dark, and we have identified several issues, including the following:
Frequent jailing without clear reasons.
Initially, we faced frequent jailing but did not understand the reasons. Due to the lack of open-source code, it was impossible to assess the reasons accurately. The only thing we could do was to communicate with other validators on Discord and speculate about the reasons together. After talking to several validators, we found that everyone was also frequently jailed and unclear about the reasons.
Node Location
Later, we found that the jail issue might be because we were not running nodes in Tokyo. Moving the nodes to Tokyo might help. Unfortunately, the team never made this clear, and we only discovered it after encountering numerous issues.
After moving the nodes to Tokyo, the situation improved. This is likely because many testnet nodes holding a large amount of staked tokens are also located in Tokyo, allowing our nodes to miss fewer blocks and keep pace. However, even after relocating the nodes, we still continuously encountered jail issues without knowing the exact cause. This problem is primarily due to the lack of open-source code.
Reliance on automatic exit scripts
We realized that maintaining good uptime on the Hyperliquid testnet depends on the speed at which scripts can automatically exit nodes from jail. The only way to improve uptime is to rely on scripts that can quickly automate the exit from jail. Validators cannot fully understand or resolve the root issues and can only automatically exit nodes without a deep understanding.
The centralized Hyperliquid API, as a single point of failure.
There were several times when our attempts to exit jail failed due to Hyperliquid API failures. If the API fails, validators cannot exit jail on their own, as they must send a request to the Hyperliquid server to exit jail.
The team may be aware of this, but this design needs to be reconsidered, as it makes the API a significant single point of failure for the network. If the goal is to build a Byzantine fault-tolerant system, there should be no nodes with special privileges, such as nodes relying on a centralized API.
Mainnet Validator Selection
Recently, Hyperliquid has taken significant steps to decentralize its validator set by selecting about 16 validators. Previously, Hyperliquid was managed by four validators from the core team, which led to much criticism. Recently, Hyperliquid took a major step by expanding the number of validators from 4 to 16.
Regarding validator selection, four validators were announced through the following post on Discord:
The four validators are Validao, Harvest, Hypurrstake, and Prrposefulnode. They were selected based on uptime and successfully maintained over 90% uptime in the past 7 or 30 days.
This is an incredible achievement for many reasons. The main reason is that validator performance is also affected by external factors, such as Hyperliquid API failures, jail issues, and persistent crashes of the binaries, all of which have a significant impact on performance.
In addition to these four validators selected based on testnet performance, five validators from the Hyperliquid Foundation are also running on the mainnet. Additionally, seven other validators were selected for the mainnet, but the reasons for their selection have not been publicly disclosed.
Then, a black market for HYPE testnet tokens began to emerge.
The Hyperliquid testnet initially had 50 validators. At first, specific entities were whitelisted to join the testnet, but on December 12, validators were fully open.
The conditions are simple: 10,000 HYPE testnet tokens are required to register as a validator. However, to become an active validator, one must rank among the top 50; otherwise, the validator will be inactive.
This led to a surge in the price of HYPE testnet tokens. Initially, the price rose to over 3,000 testnet USDC, and a few days later, it even surged to over 28,000 testnet USDC. As of this writing, the current token price is about 700 testnet USDC.
Unfortunately, the faucet only dispenses 100 testnet USDC every 4 hours. To rank among the top 50 active validators on the testnet, over 528,747 HYPE testnet tokens are currently needed. Assuming a token price of 700 testnet USDC and relying solely on the faucet to release USDC, the calculation is as follows:
Days Required = (528747×700)÷(100×6) = 616871.5 days
This means that relying solely on the faucet to obtain the required HYPE testnet tokens would take approximately 616,871.5 days, or 1690 years, to become an active validator on Hyperliquid.
However, those who received HYPE airdrops on the mainnet also received an equivalent amount of tokens on the testnet. This created opportunities for validators to collaborate with these community members, allowing them to stake testnet HYPE tokens with validators, thereby securing a place in the active set.
Meanwhile, this situation has also led to other ideas among holders of testnet HYPE tokens. Given the fierce competition to join the testnet validator set, many validators are eager to acquire as many testnet HYPE tokens as possible. As a result, a black market has emerged, where whales holding large amounts of testnet HYPE tokens begin selling tokens to validators in exchange for real USDC on the mainnet.
I have never seen such a chaotic situation. Although the Hyperliquid team clearly disapproves of these practices, they are fully capable of resolving this issue. One possible solution is to implement a proper validator selection process on the testnet.
In most other proof-of-stake networks, the core team usually shares a form that any validator can fill out to express their willingness to run the chain. The team then reviews these applications and performs initial screening based on various criteria, such as the validator's experience running nodes, past contributions, community involvement, or other factors.
This group of initially selected validators can contribute to the testnet and work closely with the engineering team to provide feedback, ensuring everything runs smoothly. We have attempted multiple times to provide feedback, but have not succeeded so far.
Mainnet vs. Decentralization
As mentioned earlier, the current validator set on the Hyperliquid mainnet consists of 16 validators, which can be viewed here: https://app.hyperliquid.xyz/staking.
Five of them are from the Hyperliquid Foundation. Four are selected based on testnet performance and maintained over 90% uptime. Seven were selected by the Hyperliquid team.
Out of 404,495,250 staked HYPE tokens, about 329,578,724 HYPE tokens are staked on foundation nodes, accounting for approximately 81.4% of the total staked share. We know little about HyperBFT, but assuming it operates as a Byzantine fault-tolerant system, the core assumption of most BFT systems is that no more than 33% of the voting power can behave maliciously. If a single entity controls one-third of the staked share, they can halt the blockchain. If they control two-thirds of the staked share, they can take full control of the network.
The Hyperliquid Foundation initially staked 60 million HYPE tokens for each foundation node. However, many HYPE holders also choose to stake at foundation nodes, which is detrimental to decentralization. The team should engage more with the community to encourage a more distributed staking.
There are three potential solutions:
Promote to the community the importance of staking with external validators to enhance the chain's security and decentralization.
Foundation nodes implemented a 100% commission rate to incentivize users to stake with external validators, promoting decentralization.
Reallocate the foundation's staked share to external validators, which is a common practice among most chains.
Distributing the staked share to external validators also helps them achieve greater economic sustainability. Hyperliquid is a blockchain focused on high throughput, and the infrastructure costs to run nodes (especially in Tokyo) can be quite high. Currently, bottom-tier validators earn between $3,000 and $5,000 per year, which is not enough to cover their costs. Particularly challenging is the requirement to stake 10,000 HYPE tokens (approximately $250,000 at current prices) on the mainnet to validate.
Currently, users interact with Hyperliquid by bridging USDC from Arbitrum to the Hyperliquid chain. After reviewing the contracts for the cross-chain bridge, it appears that the bridge is still managed by four validators. These validators do not seem to be associated with the chain's consensus mechanism or the 16 validators on the mainnet.
Hyperliquid has a great product, but the team still needs to improve several aspects of its infrastructure to truly compete with mainstream L1 blockchains.
Some improvement measures are as follows:
Listen to validator feedback. While the team’s current approach of working independently without much interaction with many external parties is effective for building their Perp product, validators are the backbone of L1 blockchains. Listening to their feedback is equally important to ensure everything runs smoothly.
Open source the code. This would help validators better understand the challenges they face when running nodes on the Hyperliquid chain and help users trust the product. Open-sourcing the code would also allow validators to gain more insight into the architecture and consensus algorithm. Currently, information about HyperBFT is very limited, and open sourcing could provide the much-needed transparency and understanding.
Create a proper validator selection process to eliminate the black market for HYPE testnet tokens. Selecting validators based on uptime is a fair approach, but achieving good uptime should also be fair. It should not depend on external factors such as having connections to acquire testnet tokens, purchasing testnet tokens, or relying on the uptime of the Hyperliquid API.
Overall, Hyperliquid can compete with mainstream L1 blockchains without significant changes. The main focus should be on interacting more with external parties and considering their feedback. I look forward to seeing how things develop in the coming weeks and months, and our team is ready to provide help and feedback. Wishing the Hyperliquid team all the best!