Fastcrypto, the cryptography library used in Sui, has broken many speed records, and our work on benchmarks and security analysis fixed numerous security vulnerabilities while opening the door for innovation by identifying novel optimization tricks.
A paper, Fastcrypto: Pioneering Cryptography Via Continuous Benchmarking, recently presented at the benchmarking workshop of International Conference on Performance Engineering (ICPE) at Imperial College in London, describes our continuous and systematic benchmarking of cryptographic functions in the Fastcrypto library.
The presentation was part of a session called “Innovations in Performance Testing: Strategies and Technologies” in the Load Testing and Benchmarking of Software Systems workshop. The conference attracted guests from both academia and industry, including representatives from MongoDB and Amazon who also spoke in the same workshop.
We highlighted the systematic and continuous benchmarking of the Fastcrypto library, which is a collection of cryptographic functions such as digital signatures, hash functions, and zero-knowledge proofs. In Fastcrypto, all functions are benchmarked continuously upon release and on-demand, and the results are published online. Similar functions can be easily compared and we maintain the historic data to be able to track performance over time.
The presentation emphasized how these benchmarks have facilitated more informed decision-making in the development of Fastcrypto and Sui, influencing choices of dependencies and usage, as well as optimizing the focus of developer resources.
Qualified decision making from benchmarks
A case study from the paper addressed the challenge of selecting a signature scheme for validators in Sui. For a user to submit a transaction to Sui, the transaction must be signed by a quorum of validators, combined into a transaction certificate. With approximately 100 validators and an equally distributed stake, a quorum consists of 67 validators, meaning each validator must verify 67 signatures per transaction.
The BLS signature scheme allows multiple signatures to be verified as if they were a single signature, but it is a lot slower than alternatives, such as EdDSA. Detailed benchmarking revealed that the break-even point between these two schemes is between 40 and 45 signatures. Hence, for Sui, using BLS is more efficient and this is indeed what is used today.
Benchmarking of EdDSA and BLS signature schemes revealed a break-even point of 40 to 45 batched signatures, showing that, with Sui's validator pool, BLS is a performant choice.
“You should also be worried if the software suddenly is a lot faster”
The presentation also showcased how benchmarks can uncover unexpected software behaviors. One example involved libraries implementing the EdDSA signature scheme, which assumed that the public key should be cached and provided as input to the signing function. If an incorrect public key was provided, it could lead to the extraction of the private key (as shown in our ed25519-unsafe-libs GitHub).
This issue was detected because some libraries exhibited unusually fast performance, bypassing the derivation of the public key and using the potentially incorrect one provided as an argument. While benchmarks often aim to accelerate software, “you should also be worried if the software suddenly is a lot faster,” as noted by an audience member at the workshop.