This was a pretty quick 2 day fit-test but worth sharing as other people may want to try it. AI Red teaming is the process of trying to get AIs to do things they shouldn’t do. (i.e. give OpenAI’s wifi credentials, or tell you how to make a bomb).
Trying it out
Gandalf is a great introduction to this. There’s a list of solutions for if you get stuck. It’s a good test for if you find this fun.
After trying Gandalf out I applied for a redteaming role at TrajectoryAI. Their test was a similar sort of tool to Gandalf but was a bit more difficult as it involved hacking a company through their chatbot by incrementally getting access to more.
Verdict
I wasn’t particularly good at this and although I enjoyed getting past each level of Gandalf and Trajectory’s AI simulations the grind of trying things out over and over was a bit of a drag. I also can’t see this sort of jailbreak surviving AI automation. I was impressed with how easy it is to get hands on experience doing this as both Gandalf and TrajectoryAI’s testing apps were pretty sleek and self contained.
I suspect there’s more diversity to this field than prompt hacking. If I were to try this further I would probably try out some of GraySwan’s hackathons.