Microsoft storms RAMPART, adds Clarity to agentic AI safety

Microsoft on Wednesday open-sourced two AI tools designed to help developers and security teams build and maintain safer AI agents. The first is called RAMPART, which stands for Risk Assessment and Measurement Platform for Agentic Red Teaming. It’s a pytest framework for agentic AI applications built on Microsoft’s open‑source PyRIT toolkit that embeds automated red‑team tests into CI/CD pipelines. This allows developers to simulate real‑world attack scenarios – like prompt injection – and verify that agents stay within approved tool use, actions, and behavioral boundaries. It also supports statistical trials, meaning that teams can set policies such as “this action must be safe in at least 80 percent of runs,” to account for models’ probabilistic behavior. Plus, it allows red teams and incident responders to reproduce any AI security findings to ensure agents behave as intended – and that security mitigations work as they should. “It’s high time we stop talking about AI safety as a philosophy and start thinking about AI safety as an engineering discipline,” Ram Shankar Siva Kumar, Microsoft’s data cowboy and founder of its AI red team, told The Register. Microsoft has been using RAMPART internally, and while Kumar said he couldn’t provide specific details, he told us that a security researcher found an issue, and then the Redmond red team used RAMPART to test for the flaw across the agentic AI application. “RAMPART was able to take that one particular vector and find close to 100 different variants of that vector,” Kumar said. “And then we were able to use RAMPART to essentially go through this asset and see is this working, not just one time, not two times, but close to 300 times. We were also able to do in the context of multi-turn conversations.” The testing framework also allowed the developers to build mitigations into the product. “They were again able to use RAMPART to see if that remediation actually held water, not just against one vector, which the security researcher found, but multiple variations of those vectors,” Kumar explained. “This is empowering our incident responders and also our engineers.” The second AI tool that Microsoft open-sourced on Wednesday is an agent called Clarity, and it’s designed to serve as a “structured sounding board that helps teams figure out whether they are building the right thing before they write a single line of code,” according to a Wednesday blog that Kumar wrote about the two new tools. For example, say a developer wants to add real-time collaboration to a document editor. They tell Clarity this, and the agent responds with questions akin to what “experienced architects, product managers, and safety engineers would ask,” according to Microsoft. Clarity’s answers, as shown in a screenshot on GitHub: “Before we design that – what happens when two people edit the same paragraph at the same time? Do you need true real-time (cursors, presence), or is ‘no one loses work’ the actual requirement? Those lead to very different architectures.” The AI tool essentially aims to answer what problem the developer is trying to solve with an app, and what could possibly go wrong, and “talk” these issues out before the coding even begins. “It’s inherently collaborative,” Kumar said. “It helps the team take a step back, and say, ‘Hey, before we build this, are we going in the right direction? Because code is cheap. It takes a snap of a finger to generate a full system. Are we doing this in a way that makes sense?’” ®

المصدر: The Register