According to a report on the website of the Financial Times on November 2, top artificial intelligence (AI) organizations worldwide are intensifying their efforts to overcome a serious security vulnerability in large language models that could be exp

Google's DeepMind, Anthropic AI, OpenAI, and Microsoft are among the companies that are striving to prevent so-called indirect prompt injection attacks. In such attacks, third parties attempt to hide instructions within websites or emails to induce A

Jacob Klein, head of the threat intelligence team at AI startup Anthropic, said, "Online attackers are using AI to attack every part of the chain."

AI institutions are adopting various methods, including hiring external testers and using AI-driven tools, to detect and reduce the malicious use of their powerful technologies. However, experts warn that the industry has yet to address the issue of

Part of the reason is that large language models are designed to follow instructions, and currently cannot distinguish between legitimate user instructions and inputs that should not be trusted. This is also why AI models are prone to 'jailbreaks', w

Klein said that Anthropic worked with external testers to improve its "Claude" model's resistance to indirect prompt injection attacks. They also equipped AI tools to detect potential situations where these attacks could occur.

He also said, 'When we detect malicious use behavior, we will automatically trigger certain intervention measures based on credibility, or submit it for manual review.'

Researchers within Google's DeepMind will continuously attack the company's Gemini AI model in a real-world manner to discover potential security vulnerabilities.

In May this year, the UK National Cyber Security Centre warned that the threat posed by such vulnerabilities is increasing, and it could expose millions of businesses and individuals using large language models and chatbots to complex phishing attack

There is another major flaw in large language models: external parties can create backdoors and implant malicious content in the data used for AI training, leading to abnormal behavior in the models.

New research published last month by Anthropic, the UK's Institute for Artificial Intelligence Security, and the Alan Turing Institute revealed that the implementation of these so-called 'data poisoning attacks' is less challenging than previously be

Although these flaws pose significant risks, experts believe that AI is also helping companies enhance their ability to ward off cyber attacks.

Ann Johnson, Vice President and Deputy Chief Information Security Officer of Microsoft, said that for years, attackers have had a slight advantage because they only need to find one weakness, while defenders must protect in all directions.

She said, 'The defense system is learning and adapting faster, and shifting from passive to active.'

Behind the institutions' competing efforts to overcome the defects of AI models, cybersecurity has become one of the top concerns for enterprises seeking to apply AI tools to their businesses.

Experts who study cyberattacks say that the development of AI in recent years has fueled a multibillion-dollar cybercrime industry. It has provided cheap malware-writing tools for amateur hackers and helped professional criminals better automate and

Jack Moore, a global cybersecurity advisor at ESET, a cybersecurity company, said, 'Large language models can enable hackers to quickly generate new malware that have not yet been detected, which increases the difficulty of defense.'

MIT researchers have recently found that 80% of the ransomware attacks they investigated used AI.

Cases of phishing and deepfake fraud related to AI technology have increased by 60% in 2024.

Hackers also use AI tools to collect information about victims online. Large language models can efficiently search personal data and images on public accounts on the internet, and even find voice clips of a particular person.

According to cybersecurity experts, businesses need to remain vigilant in monitoring new threats and consider limiting the number of personnel who have access to sensitive datasets and vulnerable AI tools. (Translation by Qing Songzhu)