Interviewer: Faust, Wu Yue, Geek Web3

Interviewee: Luis, ScaleBit

Editor: Faust, Jomosis

On July 1, Geek Web3 invited Luis, co-founder of Web3 security audit company ScaleBit, to answer many questions about code audit and Web3 security. During the discussion, both parties discussed code audit and Web3 security from the perspectives of business and technology, as well as ZK, AI, Bitcoin ecology, etc. The issues involved included:

  • Why ScaleBit initially chose the Web3 security direction and took root in the MOVE ecosystem;

  • The business logic and customer tier division of the code audit industry;

  • The difference and connection between Web3 security and Web2 security;

  • What are the complexities of ZK circuit auditing and what efforts Scalebit has made to achieve this?

  • Views on the Bitcoin ecosystem and Layer 2 from an operational and technical perspective;

  • The impact and help of AI tools such as Chatgpt on the code auditing industry;

This article is a text version of the interview, about 7,000 words, during which Luis combined his own experience to provide a relatively detailed popular science on many issues in the Web3 security industry. For those who are not familiar with the Web3 security and code auditing industry, this article will be an excellent opportunity to learn about auditing agencies. It is strongly recommended to read, collect and forward.

1. Faust: The first question I want to ask is about the direction of entrepreneurship. When ScaleBit was first founded, why did you choose Web3 security as the direction?

Luis: We initially focused on Web3 security for the following reasons:

First, many people in our team entered the blockchain circle relatively early from Silicon Valley, and we hope to find long-term and stable user needs. Code auditing happens to be a very basic track that can survive for a long time, so we chose this direction, and we hope to become a security company respected by the industry.

Second, we believe that Web3 security is at a very early stage, but security issues in Web3 are much more important than traditional Internet security because the former is directly related to financial assets and is definitely more valuable than traditional Internet security.

Although some people now believe that the business ceiling related to contract auditing is limited, in fact, in addition to contract auditing, there are more new businesses emerging in Web3 security, and various new demands will continue to emerge, so we think that the Web3 security/code auditing track is still very good.

Third, the backgrounds of our team members overlap a lot with code auditing/blockchain security. We have a lot of experience in the security industry. I was a founding member of another blockchain security company. Our chief scientist, Mr. Chen Ting, has also been doing research related to blockchain security. He is a professor with a lot of experience in the field of Web3 security. Other members also have backgrounds in exchange security, formal verification, and static analysis.

For these reasons, we finally chose the Web3 security path.

2. Faust: I heard that ScaleBit was originally called MoveBit, and later upgraded its brand to ScaleBit. Can you tell us why you chose the Move ecosystem in the first place, and why you changed the name later?

Luis: This is actually our brand upgrade. The parent brand is BitsLab. We are using the three sub-brands MoveBit, ScaleBit, and TonBit. It is a multi-brand strategy. We have also expanded from the Move ecosystem to more ecosystems. The overall positioning is security and infrastructure in emerging ecosystems.

As for why we chose the Move ecosystem in the first place, here's a little story: When we started working on security audits in 2022, we spent three months researching the most suitable niche areas. At the time, we thought that if we were to become a full-category security company, it would be difficult to surpass competitors in the market, so we had to find a separate track to enter.

We discussed several directions, one was Move, and then ZK. In addition, we also considered creating a vertical audit brand, for example, focusing on GameFi or a certain type of business. In short, the strategy was to make a breakthrough in a single point. Later, we comprehensively considered various factors and chose the Move ecosystem.

At that time, we were only seven or eight people, and we were quite successful in the Move ecosystem. About 80% to 90% of the top 20 projects in the Move ecosystem by TVL were audited by us. We also audited important underlying chain components such as MoveVM and Aptos Framework, and discovered many underlying chain vulnerabilities. Therefore, we have a high market share in the Move track.

We are also currently accepting code auditing business in the Move ecosystem, which currently accounts for about 50% of our revenue and about 40% of our auditing workload. Because the Move ecosystem has more overseas customers with code auditing needs, the unit price of this type of business will be higher.

Currently, TonBit is mainly engaged in Ton ecosystem auditing, and ScaleBit is engaged in BTC Layer2 ecosystem, ZK and other new ecosystems. The overall positioning of our BitsLab is to focus on emerging ecosystems and those with the potential for Mass Adoption.

3. Faust: I would like to ask a question about ZK. Auditing related to ZK is very troublesome. Vitalik also said before that the circuits of systems like zkEVM are too complicated. Even with functional testing or auditing, there is no guarantee that there will be no problems with the circuits. Can you talk about this based on your own experience?

Luis: ZK-related audits are divided into many aspects, mainly circuit audits, source language audits and general computing audits. Let me talk about circuit audits first.

One of the major difficulties in circuit auditing is that circuit code is less readable than traditional programming languages. In addition, the ecosystem related to circuit languages ​​is very fragmented. There are probably more than a dozen languages ​​and frameworks for writing circuits, including Circom, Halo2, Artwork, Bellman, etc. Therefore, there is currently no unified standard for circuit writing.

Obviously, it is basically impossible for any security organization to be proficient in all circuit languages ​​at the same time. Therefore, we have selectively entered the field of ZK. At present, we have mainly done two things in the ZK track. One is to co-organize the ZK Security Capture the Flag Competition (zkCTF) with Scroll, EthStorage and Mr. Guo Yu of Anbi Lab. This competition is held approximately once a year. The main purpose of holding this competition is to cultivate more ZK security talents.

Another thing is that we have developed a vulnerability detection tool, zkScanner, which mainly uses some formal and static analysis methods to scan for vulnerabilities in ZK circuits. After the zkScanner initially scans the circuit, it will find some suspicious points, which will then be handed over to manual confirmation, which can serve as a supplement to manual auditing. Of course, this automated auditing tool cannot completely replace manual auditing, but it is still relatively effective in discovering some more hidden constraints, that is, constraint problems.

Wu Yue: Is the automated audit tool you mentioned a static detection tool like the one used to detect ERC-20 Tokens?

Luis: A bit like that, but not quite. The similarity lies in the workflow: scanning static code and giving the location and cause of the vulnerability. The difference is that the errors that the circuit is concerned about are mainly divided into two categories, namely Under-Constrain and Over-Constrain; at the same time, ordinary lexical analysis has difficulty finding these errors.

Wu Yue: If we just talk about Constraint, it’s a bit abstract. Can you give an example of what it is like?

Luis: I think we can look at this from a side perspective. In essence, circuits are more mathematical than smart contract languages, and they need to be converted to R1CS, which can be understood as a pure polynomial expression. Therefore, some problems that may occur in conventional programs are relatively rare in circuits.

Because the circuit is actually "not wrong", each circuit needs to use the correct input and output to generate the corresponding proof. If there is an error in the circuit, it will not pass the compilation. This ensures that the result of the circuit calculation must be "correct". But being "correct" alone is not enough for the circuit. It needs to be "correct" in all cases, which leads to the two constrain problems mentioned above.

If it is Over Constraint, some inputs that meet the requirements will not be able to pass through the circuit; if it is Under Constraint, those that do not meet the requirements will be made to meet the requirements, which are both fatal problems.

Wu Yue: So these problems cannot be discovered by the compiler. They are the prerequisites that the compiler sets before designing the circuit. There are problems when it expresses them. Is that right?

Luis: Yes, because these problems are not entirely at the grammatical level, they also involve the developer's intentions and some common specifications in cryptography. Specifically, these problems often require formal tools such as SMT-Solver to discover them.

4. Faust: From a business perspective, what do you think about ZK-related auditing?

Luis: ZK-related auditing business is worth paying attention to in the long term. We have been continuously accumulating experience in ZK auditing. Including our later auditing in the Bitcoin ecosystem, it was also because we found that the Bitcoin ecosystem was well integrated with ZK. The ZK Layer2 narrative of the Ethereum ecosystem has receded. The next wave of narratives in the ZK track has not officially arrived yet, and it may be related to FHE.

Of course, it is neither too early nor too late for us to officially enter the ZK audit field. It is more of a stage of long-term attention and long-term accumulation. At the business level, we continue to focus on the two things mentioned above, one is zkCTF, and the other is zkScanner, which is the ZK circuit vulnerability detection tool mentioned above.

Wu Yue: Can you briefly introduce the zkCTF event?

Luis: This is a CTF (Security Capture the Flag) related to the ZK field that we initiated. It is held once a year and we invite the top security researchers in the industry and ZK researchers to participate. We will work with Scroll, EthStorage and Professor Guo Yu from Anbi Lab to set topics for the contestants. We have also received strong support from Ingonyama, zkMove, HashKey and other institutions.

We have counted the list of contestants, and they are basically from all over the world, and their level is also top-notch, including:

OpenZepplin, Offside, Salus, Amber Group, Sec3, etc., as well as some PhD students in security and ZK from Georgia Tech and Berkeley.

5. Faust: I would like to ask about ScaleBit’s views on the Bitcoin ecosystem. I heard that you have audited more than 30 Bitcoin ecosystem projects before. What do you think about Bitcoin’s second layer?

Luis: The more than 30 Bitcoin ecosystem projects mentioned here include projects on the second layer or within the second layer ecosystem, such as UniSat, Arch Network, Merlin Chain, RGB++, B² Network, etc., as well as Liquidium and other projects related to inscriptions and runes. Many other projects are Defi protocols within the second layer ecosystem.

Regarding the second layer of Bitcoin, I agree with the previous view of Kevin He, co-founder of Bitlayer, that the competition of the second layer of Bitcoin will be divided into three stages. The first stage is to increase TVL, the second stage is to attract developers, and the third stage is the competition of technical routes. I think we are still at the end of the first stage, or at the beginning of attracting developers and building an ecosystem.

Faust: When you audit projects in the Bitcoin ecosystem, what aspects do you mainly focus on, or what indicators do you use to audit?

Luis: If it refers to Bitcoin Layer 2, we divide it into several dimensions. For example, some projects need to review their scripts on the Bitcoin chain, and some need to review their contracts deployed on Bitcoin Layer 2. Some audit objects are cross-chain bridges or the bottom layer of the chain. Some second layers may not use EVM. There are audit work to be done at these levels.

We mainly look at the attack surfaces in the codes of these projects and see if there are any vulnerabilities from various dimensions. This is actually very complicated because Bitcoin Layer 2 is a system similar to a public chain. We will look at the points that the industry audits public chains to examine, such as whether the project has some double-spending attacks, eclipse attacks, witch attacks, external dependency security, centralization issues, man-in-the-middle attacks, etc. We need to look at the specifics, which are very detailed, and we can talk about this topic another day.

Our ScaleBit's chain auditing capabilities are at least first-class in Asia. Team members have discovered vulnerabilities in well-known public chains such as Sui, OKX Chain, GalaChain, Nervos, and recently discovered a High and a Low level vulnerability in Babylon's public audit competition.

6. Faust: Based on your experience in security audits, is the cross-chain bridge the most vulnerable place to security vulnerabilities? As far as I understand, many cross-chain bridges are actually extensions of DeFi, so they are as vulnerable to attacks as DeFi protocols. What do you think about this?

Luis: In terms of frequency, DeFi-related places are the easiest to lose money, but in terms of amount, the amount lost after the cross-bridge attack is the largest, and once there is a problem, it will be a big problem. Of course, when I talk about DeFi, I am more referring to the contract level. As long as there is a problem with the contract of the DeFi protocol, it can be attacked, and there are relatively few remedial measures.

As for cross-chain bridges, they are indeed the most prone to problems, because the amount of funds usually associated with cross-chain bridges is relatively large, and many of them use multi-signatures, which can be easily messed up.

7. Faust: How much help or impact do you think LLM tools like Chatgpt can have on code auditing?

Luis: It is actually quite helpful, but it is more of an auxiliary function. Sometimes when auditors look at some code, in order to quickly understand the purpose of the code, they will use Chatgpt to parse what the code does. Of course, this is only auxiliary, and ultimately it is up to humans to determine many details.

Another area is writing documents and audit reports, especially writing in English. Some auditors are not very native in English, so they will ask Chatgpt to polish these documents, which is quite helpful.

However, from the perspective of auditing, we are also training some specific LLMs internally, using some open source large language models for training, but currently it is only auxiliary. Although it can improve work efficiency, we cannot rely entirely on AI for auditing. It can only improve efficiency by about 20%, which is still a long way from the step of large-scale reduction in the number of auditors.

At present, LLM still has two obvious shortcomings. The first is the problem of missed reports, and the second is false reports. We can use LLM to do some vulnerability mining, but we must pay attention to the false alarm rate. If you use AI to help you find code vulnerabilities, but it has a high false alarm rate, it will waste your time and bring burdens. However, we will continue to pay attention to the progress of AI, such as whether it can achieve efficient vulnerability mining at the tool level. This is still relatively cutting-edge, and everyone is still exploring, but no one has said that it can really achieve the above effect.

Wu Yue: Regarding AI automated code auditing, do you think this will be a focus in the future? As far as I understand, AI can read code almost instantly, and it has accumulated much more experience and can exhaustively enumerate more states than humans, which is a great advantage of AI. If a security company has a deep layout in this area, trains special AI to do automated code auditing, and surpasses its competitors, what do you think of this?

Luis: We have been paying attention to this direction. I think we should look at it from two aspects:

First, if you believe that AI automated auditing is really feasible, then a situation will arise:

Theoretically, LLM can harmonize the entire code auditing industry, because if everyone uses LLM to generate code, LLM can ensure that the code it generates has no problems or bugs, so you don’t need to audit. At this time, it will not only kill the auditing industry, but also programmers. But it is very difficult to achieve such an effect.

If we think that LLM can eliminate auditors, it must be more difficult than eliminating developers. It is much more difficult to write code without vulnerabilities than to simply implement code that meets the requirements, so I think it is more difficult for AI to eliminate auditors than to replace programmers.

Another aspect is that AI does not solve security audits directly, but achieves breakthroughs in certain directions. For example, as mentioned earlier, AI can help you discover vulnerabilities. Although it cannot help you find all bugs, it can help you find one or two types of problems that manual audits may overlook. Application scenarios like this are exactly what we focus on.

8. Faust: I would like to ask you about your views on the audit work itself. When you do an audit, what does the specific workflow include? It is not as simple as issuing a certificate, right? In the process of reviewing the code, will you help the project party optimize the code?

Luis: It depends on customer needs. Sometimes we will help customers optimize their original code, such as making certain DeFi operations consume less gas.

As for the audit process, I will briefly introduce that we have at least two independent rounds of auditing: initial review and re-review. During the initial review, a group of people will do a separate audit, and the project party will modify the initial code; then enter the re-review, and another group of people will continue to check the code. The final effect is that at least two groups of people will cross-audit the code.

Speaking of the difference between ScaleBit and other auditing companies, we are best at auditing innovative businesses. We like to recruit people with a CTF (Capture the Flag) background to do audits. They have strong learning abilities and a strong understanding of various attack methods.

In addition, we are different from other audit companies in that we prefer the boutique audit route. If the code we audited misses Major or above vulnerabilities, we will refund 30%~50% of the fee. This is something other audit companies dare not promise.

9. Faust: Some people believe that security audits are actually about brand endorsement, just like Wall Street rating agencies. The Matthew effect is very strong in this area. Old companies like SlowMist have first-mover advantages and a strong moat. It is difficult for up-and-coming companies to compete with old powerhouses like SlowMist. What do you think about this?

Luis: I partially agree with this view, but it depends on different situations. For example, we divide clients into three categories according to audit needs, namely low, medium and high. Local dog projects are the lowest level, and above are medium projects, which are those with certain strength but not star teams. The highest level is the star team with relatively strong financial strength.

Let's start with the highest-level customers. These customers usually look for 2-3 audit companies and are very concerned about the quality of code audits. They may first look for some of the world's top auditing agencies, but these agencies have too much business to handle and may not be able to prioritize the needs of certain customers. Therefore, many star project teams will also find some less well-known but high-quality auditing agencies to check at the same time.

The type of clients mentioned above are the favorites of audit companies because they are rich, but this type of clients are also very concerned about the quality of audits, so they generally look for multiple audit companies. As long as you have strong auditing capabilities, you have the opportunity to reach this type of clients, so this type of client is one of our main customer groups.

The second tier is the mid-tier clients, who are “more concerned about audit quality but not necessarily have a lot of money”, but have the potential to become top clients in the future. Although they hope to find a star auditing agency, they may not be able to afford it.

There are only 4-5 organizations in this circle that can be called "top security companies", such as OpenZeppelin and Trail of Bits. Everyone knows that these organizations are very powerful, but their prices are also very expensive, which may be 3-10 times higher than that of regular auditing companies.

Mid-tier clients may not necessarily accept the offer even if they go to the top audit firms. So for mid-tier clients, instead of spending all their budget on a top audit firm, it is better to give the budget to an audit firm with excellent audit quality, or to look for multiple audit firms. This type of client is our largest group, and we hope to grow together with them.

The last type of client is the one mentioned earlier, the cheap project at the bottom. This type of client will basically look for whoever is cheap, or spend money to find a well-known auditing agency to review the contract.

So from the above points, your comments on the audit industry are somewhat reasonable. For audit firms with a longer history and a better reputation, there is indeed a Matthew effect, but you can see that some old audit firms, although well-known, have suffered from various problems in recent years.

However, as an audit company that emerged later, we must have differentiated advantages, so our strategy is to make a breakthrough in a single area:

First, enter a certain niche and gain an absolute advantage. For example, in the Bitcoin Layer2 ecosystem, our current coverage rate has exceeded 50%, and our coverage rate in the Move ecosystem has exceeded 80%. Even top auditing agencies like OpenZepplin may not be able to compete with us in these niches. So the so-called "Matthew effect" depends on the environment.

10.Faust: From your personal perspective, what is the biggest difference between Web2 and web3 security? Can you talk about it based on your past experience?

Luis: First of all, I think that Web3 security is in a very early stage of development, and the market for Web3 security must be larger than that for Web2 security because Web3 has stronger security requirements.

Here I will tell you a story. There was a Chinese executive in the Silicon Valley security circle who once worked as a VP of a Silicon Valley listed company. He said that the two main groups of people working on security in Silicon Valley are Chinese and Jewish. So why can Chinese do security so well? Because the security industry is a typical industry that has no presence in normal times, but has to take the blame when something goes wrong. Indians and white people are unwilling to do it, so Chinese people emerge. This is for Web2 security companies;

But Web3 security is different, because blockchain deals directly with money, so the presence of Web3 security is many orders of magnitude greater, and in this field, many "security practitioners" can directly monetize. Someone once joked that hackers are the most successful in the transition from Web2 to Web3.

From a technical perspective, Web3 security includes parts of Web2 security, and reuses many of the latter's technologies. Many systems now, for example, many DeFi applications have servers and interfaces, which also require traditional penetration testing, DoS defense, etc., which are actually part of Web2 security.