SWE-Lancer

A benchmark for evaluating large language models on real-world freelance software engineering tasks from Upwork. Contains over 1,400 tasks valued at $1 million USD total, ranging from $50 bug fixes to $32,000 feature implementations. Includes both independent engineering tasks graded via end-to-end tests and managerial tasks assessed against original engineering managers' choices.

Released

Unknown

Readiness

Paper only

Primary field

General AI

Benchmark Radar records only publicly supported details and links back to primary sources for verification.