May Updates: Top open-source AI Agent on SWE-bench; What sparked CTOs' interest at Dublin Tech Summit?
This edition of our Changelog is a little different. No product features or release notes — instead, two big milestones to remind you of:
- 🥇Refact.ai became the #1 open-source Agent on SWE-bench Verified
- 👏… And caught the strong interest from dev teams at Dublin Tech Summit.
Details below!
Refact.ai: #1 open-source Agent on SWE-bench Verified
We’re proud that Refact.ai Agent is now the best open-source AI Agent in SWE-bench Verified, the leading benchmark for evaluating AI Agents on software engineering tasks
Score: 70.4%. Tasks solved: 352 / 500. Fully autonomously.
We even outperformed AI Agents running Claude 4 Sonnet (we used 3.7 Sonnet).
Our full SWE-bench pipeline is open-source — check it out on GitHub.
Key elements of the approach:
- Claude 3.7 as an orchestrator.
debug_script()sub-agent using pdb.strategic_planning()tool with o3 for debug-to-solution reasoning- Auto guardrails (messages sent as if from a simulated ‘user’) to course-correct the Agent mid-run.
- One-shot runs only — with one clean solution per task.
If you feel like sharing this milestone on LinkedIn or X, we’d really appreciate the support 🫶 Thank you for building with Refact.ai!
And yes, the new score with Claude 4 Sonnet is coming soon.
What enterprise devs liked most about Refact.ai in Dublin?
Just got back from Dublin Tech Summit! We spent two days talking with devs, CTOs, and tech leaders about what matters in enterprise AI. Refact.ai stood out — on-prem, autonomous, and ready for real-world workflows.
Here’s what teams told us they need — and what Refact.ai delivers:
✅ On-prem security. No code or telemetry ever leaves servers. Refact.ai offers on-prem deployment, giving companies 100% data privacy.
✅ Fully autonomous & transparent. Solves tasks end-to-end while letting developers preview and guide every step.
✅ Truly context-aware. Understands your project, codebase, repo structure, dependencies, and more, adapting to how your team actually works.
✅ Shared knowledge for teams. Captures successful interactions with AI and builds a team-wide memory, speeding up collaboration across developers.
Want to see what Refact.ai could do inside your company?
- Explore our Enterprise Agent at refact.ai/enterprise — or share this link with your CTO.
- Book a quick demo call.
Discover how Refact.ai Enterprise turns AI into a real programming partner for faster, smarter development across complex codebases!
Get Refact.ai for your IDE
If you’re ready to work with an AI that understands your environment, works across your tools, and delivers reliable, product-ready results — Refact.ai is ready for you.
Autonomous when you want it, collaborative when you step in. Get Refact.ai for VS Code or JetBrains today and start doubling your output — without doubling working hours.