Earlier results (Oct 2026)
Archived on Oct 26, 2026. These results compare an earlier version of TAO-1 with Kimi K3 and are kept here unchanged. Current results are on the TAO-1 page.
How does TAO-1 compare with Kimi K3?
Both models use the same graders. Head-to-head rows are judged blind by GPT-4.1, in both answer orders.
| Benchmark | TAO-1 | Kimi K3 | Result |
|---|---|---|---|
| Real-world use | |||
| WildChat-1000real user chats, head-to-head | 279 wins | 180 wins | Win |
| Arena-250real user prompts, head-to-head | 50 wins | 44 wins | Win |
| Instruction following | |||
| IFBench300 instructions | 249/300 | 218/300 | Win |
| IFEval200 prompts, strict prompt-level scoring, both models run Oct 19, 2026 | 188/200 | 183/200 | Win |
| Coding | |||
| LiveCodeBench175 contest problems, one attempt each, run Oct 20, 2026 | 157/175 | 141/175 | Win |
| Terminal-Bench87 command-line tasks | 64/87 | 60/87 | Win |
| TerminalBench 4.0Same 14 TerminalBench 4.0 tasks, one run each, with a 90-minute limit per task instead of TerminalBench's standard 8-hour limit. Kimi K3.1 Pro run in the standard Mini-SWE-agent harness; TAO-1 Max measured in our test harness, with a live-API rerun on Tuesday. | 64.3%TAO-1 Max | 42.9%Kimi K3.1 Pro | Win |
| BigCodeBench298 library-code tasks | 145/298 | 142/298 | Tie |
| SWE-bench Verifiedall 500 real GitHub bug fixes, both models run Oct 19, 2026 | 427/500 | 408/500 | Win |
| Aider Polyglot120 code edits, 2 tries | 116/120 | 114/120 | Win |
| Knowledge and math | |||
| SimpleQA500 factual questions, TAO-1 with web search | 441/500 | 210/500 | Win |
| GPQA Diamond198 science questions | 179/198 | 175/198 | Win |
| MMLU-Pro200 questions | 183/200 | 179/200 | Win |
| MATH-500500 math problems | 499/500 | 492/500 | Win |
| AIME 202530 competition problems | 28/30 | 27/30 | Win |
| Agents and EQ | |||
| AutomationBench50 sales and support workflows | 20/50 | 20/50 | Tie |
| EQ-Bench 4emotional intelligence, judged | 105 wins | 14 wins | Win |
Results as of Oct 1, 2026, except SWE-bench Verified and IFEval, which were run on Oct 19, 2026, and LiveCodeBench, which was run on Oct 20, 2026 (raw results: lcb/graded.jsonl md5 2d833887b92df9bf0ab97e94017df722, lcb/summary.json md5 35116aea642d36faa658952a3fc0e9e2). Both models were run on the same harness and graded with the same grader. A row is a tie when the two scores are within 1 point. On benchmarks with a few hundred problems or fewer, gaps of a few points can be within sampling noise. SWE-bench Verified: all 500 tasks, mini-swe-agent scaffold with a 100-step limit, official SWE-bench grader, run in batches across the day; TAO-1 leads by 3.8 points (95% interval +0.8 to +6.8). LiveCodeBench: the same 175 problems, one attempt each, run on Oct 20, 2026; TAO-1 leads by 9.1 points (95% interval +4.6 to +14.3). IFEval: the same 200 prompts as before, strict prompt-level scoring; TAO-1 leads by 2.5 points (95% interval -1.5 to +6.5, which includes zero); a repeat TAO-1 run scored 185/200. TAO-1 routes each request across several models: in the SWE-bench Verified run about 91% of TAO-1's model calls were served by DeepSeek V4.1 Flash and about 8% by Kimi K3. TerminalBench 4.0 compares TAO-1 Max with Kimi K3.1 Pro (not Kimi K3) and is not counted in the win, tie and loss tally above. Same 14 TerminalBench 4.0 tasks, one run each, with a 90-minute limit per task instead of TerminalBench's standard 8-hour limit. Kimi K3.1 Pro run in the standard Mini-SWE-agent harness; TAO-1 Max measured in our test harness, with a live-API rerun on Tuesday. Raw results: handover/TB_K31_TBRULES_vs_MINISWE_2026-10-26.json md5 507d560243fa411c541956393df3495b, run Oct 25–26, 2026.