When Microsoft released Bing Chat, an AI-powered chatbot co-developed with OpenAI, it didn’t take long before users found creative ways to break it. Using carefully tailored inputs, users were able to get it to profess love, threaten harm, defend the Holocaust and invent conspiracy theories. Can AI ever be protected from these malicious prompts?
What set it off is malicious prompt engineering, or when an AI, like Bing Chat, that uses text-based instructions — prompts — to accomplish tasks is tricked by malicious, adversarial prompts (e.g. to perform tasks that weren’t a part of its objective. Bing Chat wasn’t designed with the intention of writing neo-Nazi propaganda. But because it was trained on vast amounts of tex …
