Everyone is experimenting with ChatGPT, even crypto exchanges.

Coinbase has experimented with artificial intelligence to see how correctly #ChatGPT could do a token security review—a requirement for all tokens listed on the exchange.

After reviewing 20 different smart contracts, the mega-popular #ai tool produced the same conclusions as the manual assessment 12 times.

Unfortunately, five of the eight misses occurred when ChatGPT mislabeled a high-risk asset as low-risk, which is the worst-case failure.

The investigation also indicated that the AI delivered inconsistent results, with the same prompt producing varied outputs, particularly when switching from one ChatGPT iteration to the next.

Still, the Coinbase team is optimistic that with more rapid engineering, they may improve ChatGPT's accuracy to the point where it can be utilized as a secondary quality assurance check. Spokeswoman told Decrypt:

"We are not shocked because such smart contracts may also be automatically reviewed by [other traditional programming] tools," a BlockSec, a blockchain security infrastructure provider.Unfortunately, it cannot work for complex business logic, which are the key attack surfaces and flaws that smart contract audits should concentrate on."

Several security specialists in the crypto-security area agreed with Coinbase's confidence about employing the tool for extra reassurance. Independent security researcher Officer's Notes via Twitter said:

"At this moment, it [AI] cannot replace a person, but it is an invaluable aid, including for fatigued or inattentive auditors, I believe that in the future, Q/A [quality assurance] and fuzzing will be unable to function without AI technologies."

Engineers Are Being Replaced By ChatGPT

Though cautious, the blockchain security industry appears to be open to the possibility of implementing AI capabilities.

Yet, in The future, may AI replace manual security auditors?

Certik's head of solutions architecture Connie Lam told Decrypt via email:

"Perhaps one day we'll get to that point, but we're still a long way off. What's more likely is a complementary approach. There are some things humans do better than machines and vice versa. Tools help us build new things, but they don't replace us. The invention of the calculator didn't make accountants obsolete, it made them better at their job."

For the time being, however, non-AI security solutions are significantly more useful than anything new on the market for locating vulnerabilities. An OpenZeppelin official told Decrypt:

"Existing security audit methods are still superior to OpenAI. They're testing it for listing ERC-20 tokens, which is a well-known pattern, making it more ideal for automation."

But, this could change.

The significant advancements shown between ChatGPT 3.5 and ChatGPT 4 are apparent, implying that future enhancements will continue to impress.

Integrating these tools should be "encouraged" as they do so.

Lam added:

"Using ChatGPT during the development phase should be encouraged; it's a strong tool, and refusing to work with it and learn what it can do would be a setback. ChatGPT is also a highly effective tool for searching for information and creating a knowledge hub, allowing users to easily comprehend complex issues and stay up to date on the most recent security information."

This news is republished from https://coinaquarium.io/