Hyperliquid has a great product, but the team still needs to improve several aspects of its infrastructure to truly compete with mainstream L1 blockchains.
Written by: Kam
Compiled by: Luffy, Foresight News
This letter is addressed to the Hyperliquid development team, hoping the team can take the time to review this feedback on Hyperliquid blockchain management.
Key Points:
Due to the code not being open source, lack of documentation, and reliance on centralized APIs, validators face numerous significant challenges, leading to frequent jailing and unstable performance.
Testnet incentives have spawned a black market for HYPE tokens, favoring deals with large holders rather than fair selection of validators.
Validator rewards on the mainnet are too low to meet high self-collateral requirements, and the level of decentralization is limited, as 81% of the staked share is controlled by foundation nodes.
To compete with mainstream L1 blockchains, Hyperliquid must improve transparency, reduce staking centralization, implement a fair validator selection process, and strengthen interactions with external validators.
I started getting involved with Hyperliquid in December 2023, and it is an amazing crypto application. It is easy to use, provides an excellent user experience, and offers some unique features such as vaults and the famous HLP. Currently, assets managed by HLP exceed $350 million, and anyone can participate in Hyperliquid passively.
Seeing this platform perform so well, and understanding that Hyperliquid operates as its own L1 blockchain, I hope that Chorus One (staking solutions) can participate as an operator on the Hyperliquid chain. I am an employee of Chorus One, which is one of the largest node operators in the field. Chorus One has been very active in the proof of stake industry since 2018.
After obtaining whitelist status on October 17 last year, Chorus One joined the Hyperliquid testnet. I want to share our overall experience on the testnet with the Hyperliquid engineering team, as even though we joined the testnet nearly 3 months ago, we have yet to have the opportunity to communicate with them.
During this time, we witnessed one of the most successful token launches of 2024: the launch of the HYPE token. At the same time, we experienced a testnet environment that was both interesting and challenging. I would like to mention some observed issues, hoping they will receive attention in the coming days, weeks, or months.
Testnet Experience
So far, the testnet experience has been very challenging. Operators have almost no knowledge of how to run nodes, and the resources available are very limited. Besides that, we are basically exploring in the dark, and we have encountered several issues, including the following:
Frequent jailing without a clear reason
At first, we were jailed multiple times without understanding the reason. Due to the code not being open source, it was impossible to accurately assess the cause. The only thing we could do was communicate with other validators on Discord and guess the reasons together. After talking to several validators, we found that everyone was frequently jailed and not completely clear about the reasons.
Node Location
Later we discovered that the jail issue might be because we did not run nodes in Tokyo. Moving the nodes to Tokyo might help. Unfortunately, the team never clearly stated this, and we found out only after encountering numerous issues.
After moving the nodes to Tokyo, the situation improved somewhat. This is likely because many testnet nodes holding a large amount of staked tokens are also located in Tokyo, allowing our nodes to miss fewer blocks and keep pace. However, even after relocating the nodes, we still faced continuous jailing issues without knowing the exact reasons. The main frustration stems from the code not being open source.
Reliance on automatic unjail scripts
We realized that maintaining good uptime on the Hyperliquid testnet depends on the speed at which scripts can automatically unjail nodes. The only way to improve uptime is to rely on scripts that can quickly unjail automatically. Validators cannot fully understand or solve the underlying problems; they can only unjail nodes automatically without a deep understanding.
Centralized Hyperliquid API as a single point of failure
There were instances when our unjail attempts failed due to Hyperliquid API malfunctions. If the API malfunctions, validators cannot unjail themselves, as they must send requests to Hyperliquid servers to unjail.
The team may have realized this, but this design needs to be reconsidered, as it makes the API a significant single point of failure for the network. If the goal is to build a Byzantine fault-tolerant system, there should be no nodes with special permissions, such as those relying on a centralized API.
Mainnet Validator Selection
Hyperliquid has recently selected about 16 validators in the process of decentralizing its validator set. Previously, Hyperliquid was managed by 4 validators from the core team, which drew significant criticism. Recently, Hyperliquid took a significant step by expanding the number of validators from 4 to 16.
Regarding the selection of validators, 4 validators were announced through the following posts on Discord:
These 4 validators are Validao, Bharvest, Hypurrstake, and Prrposefulnode. They were selected based on uptime, maintaining over 90% uptime in the last 7 or 30 days.
This is an outstanding achievement for many reasons. The main reason is that validator performance is also affected by external factors such as Hyperliquid API malfunctions, jail issues, and binary crashes, all of which have a significant impact on performance.
In addition to these 4 validators selected based on testnet performance, 5 validators from the Hyperliquid Foundation also operate on the mainnet. Furthermore, 7 other validators were selected for the mainnet, but the reasons for their selection have not been publicly disclosed.
Then, a black market for HYPE testnet tokens began to emerge.
The Hyperliquid testnet initially had 50 validators. At first, specific entities were whitelisted to join the testnet, but on December 12, validators were completely open.
The conditions are simple: 10,000 HYPE testnet tokens are required to register as a validator. However, to become an active validator, one must also be among the top 50; otherwise, the validator will be inactive.
This led to a surge in the price of HYPE testnet tokens. Initially, the price rose to over 3,000 testnet USDC, and a few days later, it even reached over 28,000 testnet USDC. At the time of writing, the current token price is about 700 testnet USDC.
Unfortunately, the faucet only dispenses 100 testnet USDC every 4 hours. To rank among the top 50 active validators on the testnet, one currently needs over 528,747 HYPE testnet tokens. Assuming the token price is 700 testnet USDC each and relying solely on the faucet to release USDC, the calculation is as follows:
Days required = (528747×700)÷(100×6) = 616871.5 days
This means that relying solely on faucets to obtain the required HYPE testnet tokens would take approximately 616871.5 days, or 1690 years, to become an active validator on Hyperliquid.
However, those who received HYPE airdrops on the mainnet also received an equal amount of tokens on the testnet. This created opportunities for validators to collaborate with these community members, allowing them to stake their testnet HYPE tokens with validators and secure a place in the active pool.
Meanwhile, this situation also gave holders of testnet HYPE tokens new ideas. Given the fierce competition to join the testnet validator set, many validators were eager to acquire as many HYPE testnet tokens as possible. As a result, a black market emerged, with large holders of testnet HYPE tokens beginning to sell tokens to validators in exchange for real USDC on the mainnet.
I have never seen such a chaotic situation. While the Hyperliquid team clearly disapproves of these practices, they are fully capable of solving this issue. One possible solution is to implement an appropriate validator selection process on the testnet.
In most other proof of stake networks, the core team usually shares a form that any validator can fill out expressing their willingness to run a node on the chain. The team then reviews these applications and conducts a preliminary screening based on various criteria, such as the validator's experience running nodes, past contributions, community engagement, or other factors.
This group of initially selected validators can contribute to the testnet and closely collaborate with the engineering team to provide feedback to ensure everything runs smoothly. We have made numerous attempts to provide feedback, but have yet to succeed.
Mainnet vs Decentralization
As mentioned earlier, the current validator set on the Hyperliquid mainnet consists of 16 validators, which can be viewed here: https://app.hyperliquid.xyz/staking.
Among them, 5 are from the Hyperliquid Foundation. 4 were selected based on testnet performance and maintained over 90% uptime. 7 were chosen by the Hyperliquid team themselves.
Out of the 404,495,250 HYPE tokens staked, approximately 329,578,724 HYPE tokens are staked at foundation nodes, accounting for about 81.4% of the total staked share. We know little about HyperBFT, but assuming it operates as a Byzantine fault-tolerant system, the core assumption of most BFT systems is that no more than 33% of voting power can act maliciously. If a single entity controls one-third of the staked share, they can halt the blockchain. If they control two-thirds of the staked share, they can completely dominate the network.
The Hyperliquid Foundation initially staked 60 million HYPE tokens for each foundation node. However, many HYPE holders also chose to stake at the foundation nodes, which is detrimental to decentralization. The team should engage more with the community to encourage a more dispersed staking distribution.
There are three potential solutions:
Promote the importance of staking with external validators to enhance the security and decentralization of the chain within the community.
Foundation nodes implement a 100% commission rate to incentivize users to stake with external validators and promote decentralization.
Reallocate the foundation's staking share to external validators, which is the practice for most chains.
Distributing staking shares to external validators also helps them to be more economically sustainable. Hyperliquid is a blockchain focused on high throughput, and the infrastructure costs of running nodes (especially in Tokyo) can be quite high. Currently, validators at the bottom earn between $3,000 and $5,000 per year, which is not enough to cover their costs. Particularly challenging is that they must stake 10,000 HYPE tokens (about $250,000 at current prices) on the mainnet to validate.
Currently, users interact with Hyperliquid by bridging USDC from Arbitrum to the Hyperliquid chain. After reviewing the contracts of the cross-chain bridge, it appears that the bridge is still managed by 4 validators. These validators do not seem to be affiliated with the consensus mechanism of the chain or with the 16 validators on the mainnet.
Hyperliquid has a great product, but the team still needs to improve several aspects of its infrastructure to truly compete with mainstream L1 blockchains.
Some improvement measures are as follows:
Listen to the feedback from validators. While the team’s current approach of working independently without interacting with many external parties has been effective in building their Perp product, validators are the backbone of L1 blockchains. Listening to their feedback is equally important to ensure everything runs smoothly.
Open-source the code. This would help validators better understand the issues faced while running nodes on the Hyperliquid chain and also help users trust the product. Open-sourcing the code could also allow validators to gain more insight into the architecture and consensus algorithms. Currently, information about HyperBFT is very limited, and open-sourcing could provide the much-needed transparency and understanding.
Create an appropriate validator selection process to eliminate the black market for HYPE testnet tokens. Selecting validators based on uptime is a fair method, but obtaining good uptime should also be fair. It should not depend on external factors such as having connections to acquire testnet tokens, purchasing testnet tokens, or relying on the uptime of the Hyperliquid API.
Overall, Hyperliquid can compete with mainstream L1 blockchains without making significant changes. The main focus should be on interacting more with external parties and considering their feedback. I look forward to seeing how things develop in the coming weeks and months, and our team is always ready to provide assistance and feedback. Best wishes to the Hyperliquid team!
