New AI attack hides data-theft prompts in downscaled images
By inserting malicious prompts into photos that are analyzed by AI systems before being sent to a big language model, researchers have created a novel attack that steals user data.
The technique uses full-resolution photographs that contain instructions that are hidden from view but become obvious when resampling algorithms are used to reduce the image quality.
The attack, which was created by Trail of Bits researchers Kikimora Morozova and Suha Sabi Hussain, is based on a theory put forth in a 2020 USENIX paper by TU Braunschweig, a German university that investigated the potential for an image-scaling attack in machine learning.
AI systems automatically downscale user-uploaded photographs to a lesser quality for cost and performance reasons.
Image resampling algorithms may u...