OpenAI just acquired the top AI testing tool — it stays free
OpenAI acquired promptfoo, the most popular open-source tool for testing AI apps. 17K GitHub stars, MIT licensed, and it's not going anywhere.
OpenAI just made a quiet but significant move: it acquired promptfoo, the most popular open-source tool for testing and securing AI applications. The tool has 17,400 GitHub stars, 265+ contributors, and will remain free and open-source under the MIT license.
Why This Matters More Than It Sounds
Promptfoo solves a problem every company building with AI faces: how do you know your AI app actually works well? Before promptfoo, most teams tested their AI by manually trying different inputs and hoping for the best. Promptfoo turns that guesswork into a systematic process — like unit testing, but for AI.
With OpenAI's backing, this tool is now positioned to become the standard way companies test their AI applications, regardless of which AI provider they use.
What Promptfoo Actually Does
Compare AI models side by side — Run the same questions through Claude, GPT, Gemini, or any other model and see which gives better answers in a visual dashboard.
Catch security vulnerabilities — Built-in "red teaming" (automated hacking tests) finds weaknesses in your AI app before attackers do.
Automate quality checks — Set up tests that run automatically whenever you update your AI prompts, catching regressions before they reach users.
100% local — Your prompts never leave your machine. Everything runs privately.
The Bigger Picture: OpenAI's Safety Play
This acquisition fits a pattern. As AI applications multiply, the question isn't just "can AI do this?" but "can AI do this reliably and safely?" By owning the most popular testing tool, OpenAI is positioning itself as the company that cares about AI quality — not just AI capability.
Co-founders Ian Webster (CEO) and Michael D'Angelo (CTO) announced the deal on March 9, emphasizing that the open-source project will continue under OpenAI. This matters because promptfoo works with every AI provider — not just OpenAI — and that cross-platform support appears to be staying.
Who Benefits
Developers building AI apps — You now have an OpenAI-backed, MIT-licensed tool to systematically test your AI before shipping. No more "let me try a few prompts and hope it works."
Companies using AI — If your team is deploying chatbots, AI assistants, or automated workflows, promptfoo can catch embarrassing or dangerous outputs before customers see them.
Anyone comparing AI models — Not sure whether Claude or GPT is better for your use case? Promptfoo lets you run head-to-head comparisons with real data.
Try It Now
npm install -g promptfoo
promptfoo init --example getting-started
cd getting-started
promptfoo eval
promptfoo viewAlso available via brew install promptfoo or pip install promptfoo. No API key needed to get started with local models.
With 398 releases, 7,700+ commits, and now OpenAI's resources behind it, promptfoo is the most battle-tested AI testing tool available. The fact that it stays open-source and provider-agnostic makes this acquisition a win for the entire AI ecosystem.
Related Content — Get Started with Easy Claude Code | Free Learning Guides | More AI News
Stay updated on AI news
Simple explanations of the latest AI developments