Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases

Real-SWE is a benchmark that evaluates frontier AI coding agents on proprietary, production-grade codebases licensed from real companies, requiring models to navigate company-specific conventions, business logic, and integrated services such as AWS, Docker, Kubernetes, TaxJar, and various databases. Tasks include fixing billing to apply correct taxes across diverse configurations and involve full-stack environments spanning TypeScript, Go, Python, and Node.js, reflecting genuine engineering challenges.