At the heart of a high-throughput blockchain is its ability to handle numerous transactions swiftly and securely. However, when transactions in Sui involve writing to the same shared object, they must be executed sequentially. This can translate to longer checkpoint times which may reduce state synchronization efficiency.
The first goal of shared object congestion control is to improve efficiency in checkpoint execution. By controlling the number of transactions touching a congested, or hot, shared object within each checkpoint, the system ensures that processing times remain consistent, preventing delays.
This mechanism also promotes transaction fairness by ensuring that transactions with higher gas fees are prioritized in checkpoint inclusion. Users will expect more costly transactions to process more quickly.
Addressing Sui's earlier limitations
Sui previously managed shared object congestion through its transaction manager. This system monitors the total number of transactions that are pending execution, waiting for the required objects to become available. If pending transactions exceeded a threshold, the transaction manager would stop accepting new requests for transaction signing or execution.
The previous mechanism, though effective in certain scenarios, fell short in several areas. For example, it often led to partial transaction rejections and object lockups due to inconsistencies across validators. It did not accurately estimate the execution times for sequences of dependent transactions, leading to inefficiencies and potential congestion in processing. Finally, when an object congested, the previous solution rejected all incoming transactions until the current transactions were executed, meaning that there was no ability to prioritize inclusion using gas payments.
A closer look at congestion control
The new Consensus Handler design introduces a more nuanced approach to managing execution dependencies across hot shared objects. This includes two new functions added to validator logic:
The ability to defer transactions: The consensus handler now has the capability to defer transactions to future consensus commits, managing execution dependencies in checkpoints more effectively.
Active transaction cancellation: Validators can now actively cancel transactions that have been deferred excessively. When a transaction is canceled, it is still processed but with a command to the execution engine to stop immediately. Once it receives this instruction, the execution engine releases any objects it has locked and quickly sends back a cancellation error to the client without completing the transaction.
The Consensus Handler sorts and manages transactions. This diagram uses colored circles to represent transactions with different gas amounts.
When the Consensus Handler receives a consensus commit, it first merges the transactions in the commit with any that were previously deferred, sorting them by gas price. It then examines each transaction one by one, creating a per-object execution dependency graph that outlines the crucial steps needed for checkpoint execution. This process ensures that transactions are handled efficiently and in order of their cost.
To add a transaction to the dependency graph, the handler evaluates all the shared objects involved in the transaction. It identifies the object with the longest queue to start the transaction’s execution, aligning the queues of all involved objects to this maximum length. The transaction’s cost is then added to each object’s queue, updating the execution order.
The dependency graph provides an estimate of execution latency for each object based on the longest queue, which also indicates the overall latency for the consensus commit. There is a maximum limit for queue length in each consensus commit. If a transaction exceeds this limit, it’s deferred to a future commit. If deferred repeatedly exceeding a certain threshold, a transaction is canceled and removed from processing. This typically occurs when a transaction targets a highly demanded object but offers a gas price too low to be competitive.
The new design increases efficiency by monitoring execution dependencies and limiting transactions that involve highly demanded shared objects within each consensus commit, thus protecting checkpoint execution. Crucially, transactions that do not involve these high-demand objects are not affected by these limits.
Solving previous challenges
As mentioned, the previous system struggled with tracking transaction queues for each object, leading to inefficiencies. The shared object congestion control mechanism not only tackles these challenges but also introduces significant improvements to streamline the process. The new approach ensures uniform decision-making among all validators on whether to execute, defer, or cancel a transaction. This change effectively eliminates the problems associated with locked objects and allows for the quick release of objects held by transactions of lower priority.
The congestion control mechanism also enhances accuracy by tracking the full transaction dependency graph within each consensus commit. This means it carefully notes the sequence and dependencies of transactions, providing a much clearer view of the actual time it takes to execute checkpoints. By doing so, it addresses complexities and inefficiencies that were previously overlooked.
Additionally, by deferring transactions and incorporating them with new ones in subsequent consensus commits, the new method supports local fee markets. This setup benefits transactions with higher gas prices, ensuring they have a better chance of being processed during busy periods. Users gain the ability to pay for inclusion, which can be especially powerful for DeFi activities.
Finally, the new approach tends to achieve a better performance in hot shared object workload before any degradation in checkpoint execution occurs. This improvement stems from the consistent and collective decision-making by validators, which leads to more efficient transaction processing and increased overall network throughput. This uniformity ensures that the system can handle more transactions before any performance issues arise.
Sui clears the path
The implementation of shared object congestion control is more than a technical upgrade, it’s a strategic enhancement that significantly improves the scalability and efficiency of Sui. With this foundation, users and apps on Sui can now enjoy greater transaction efficiency and reliability. The streamlined congestion control mechanism provides a springboard for deploying more complex and responsive applications, further expanding the ecosystem’s capabilities.
