AI Security - Embedded Trojans in Third-Party Models
On 14 September 2023, IIA Singapore hosted a highly informative and eye-opening webinar titled "AI Security - Embedded Trojans in Third-Party Models", featuring Gerry Chng, Executive Director of Deloitte & Touche Enterprise Risk Services, as the speaker. The event had over 270 participants from Singapore and the region in attendance.
The event commenced with a demonstration, illustrating how a deceptively simple 4x4 pattern could be exploited to misclassify AI decisions. Gerry delved into various aspects of AI security. He began by laying the groundwork with AI security basics, providing a comprehensive understanding of the challenges and vulnerabilities that accompany the rapidly expanding use of AI models and systems across different industries. He made it clear that while AI offers remarkable capabilities, it also introduces new and complex security risks that demand our attention.
A central theme of Gerry’s presentation was the importance of third-party models, which organisations frequently employ to reduce development time and costs. These models, though convenient, can contain embedded Trojans, which are malicious triggers designed to intentionally misclassify results, often to the benefit of malicious actors at a later stage. Depending on the application of these models, these vulnerabilities can have severe implications, particularly in systems that rely on accurate data classification, such as facial recognition for authentication.
To address this growing threat, organisations must gain a comprehensive understanding of the entire model development process, from the origin and validation of third-party models to the deployment phase, while also developing a fundamental awareness of how and why these attacks occur.
