KNOWLEDGE · LANGUAGE MODELS
Kimi K3.1 Pro: same base model, so what raised its AA index to 55?
Not the base model but post-training. Moonshot gave K3’s base model far more reinforcement learning (RL). That training centres on “how to use tools and see a job through to the end.” And the AA index puts half of its weight on exactly such tests.
Introduction
The first Chinese model above 50
The AA index (Artificial Analysis Intelligence Index) is the weighted average of ten tests that the evaluation company Artificial Analysis (AA) runs itself, on many AI models under the same conditions. In the current version (v4.3.2), the only Chinese model above 50 is Moonshot AI’s Kimi K3.1 Pro (55).
| Leaderboard row | Model | AA index |
|---|---|---|
| 1 | Claude Opus 5.5 (max) | 58 |
| 2 | Claude Sonnet 5.5 (max) | 56 |
| 3 | Claude Opus 5.5 (xhigh) | 56 |
| 4 | Muse Wonder Full (max) | 56 |
| 5 | Kimi K3.1 Pro | 55 |
| … | MiMo-V2.6-Pro (next Chinese model) | 46 |
| … | Kimi K3 (max) | 44 |
The leaderboard counts the same model separately for each reasoning effort (how long it is allowed to think). Counting such variants as one, K3.1 Pro is fourth. The next models from Chinese companies are MiMo-V2.6-Pro (Xiaomi) at 46, and Qwen3.8 Max (Alibaba), DeepSeek V4.1 Flash and GLM-5.3 at 45. K3.1 Pro has the same base model (the model built by pre-training) as K3 (44), released in July.
50 is a moving line
With v4.3 on 2026-09-07, AA changed its tests. It moved Terminal-Bench from 2.1 to the harder 4.0, replaced τ³-Banking with the business-automation test AutomationBench-AA, and raised the weight of private tests, whose questions are not published, from 40% to 45%. The reason: “to prevent gaming and reduce saturation.” Saturation is when almost every model gets nearly full marks, so a test no longer tells models apart.
As the tests got harder, the same models scored lower. Claude Fable 5 scored 62.1 on v4.1.1 and 49.70 on v4.3. When the version changes, the scoring changes, so scores can be compared only within one version. So “passed 50” is a statement about the current version (v4.3.2), and what means something is the comparison within it. In the current version, 55 is 3 points below the top (58) and 9 points above the next Chinese model (46).
Sources: ranking: AA leaderboard (v4.3.2, read 2026-10-26) · base model: Moonshot K3.1 Pro blog (2026-10-23) · the test changes: AA Intelligence Index v4.3 announcement · Fable 5’s scores: wojciech.io’s v4.3 explainer
01 Performance
Which tests rose
The AA index is the weighted average of ten tests. An agent test makes an AI use tools (the command line, a browser, apps) to finish a job over many steps. Agents (30%) and coding (20%) together make half of the index tests of “using tools to get a job done.” About half of the 11 points gained (+5.4) came from the single test Terminal-Bench 4.0, and about 40% (+4.6) from four office-work agent tests.
| Test | K3 → K3.1 Pro | Index points |
|---|---|---|
| Agents 30% | ||
| AA-Briefcase 15%privateReal knowledge-work tasks | Elo 1499 → 1678 | +1.34 |
| GDPval-AA 10%Real work products across occupations | Elo 1534 → 1846 | +1.56 |
| AutomationBench-AA 5%privateBusiness workflow automation | 58% → 73% | +0.75 |
| Coding 20% | ||
| Terminal-Bench 4.0 10%Finishing jobs in a terminal | 13% → 67% | +5.40 |
| SciCode 10%Scientific computing code | 59% → 61% | +0.20 |
| General 30% | ||
| AA-Omniscience 15%privateKnowledge accuracy and not making things up | Accuracy 48% → 51% Hallucination rate 53% → 44% | +0.75 |
| GDP.pdf 10%Reasoning over long professional documents | 22% → 31% | +0.90 |
| AA-LCR 5%Long-context reasoning | 89% → 86% | −0.15 |
| Scientific reasoning 20% | ||
| Humanity's Last Exam 10%Very hard expert-knowledge questions | 47% → 52% | +0.50 |
| CritPt 10%privateResearch-level physics problems | 23% → 25% | +0.20 |
Index points are weight × the change in score. Elo is a score from pitting two models’ work against each other, turned into 0–1 by (Elo − 500) ÷ 2000. AA-Omniscience’s non-hallucination is 1 − the hallucination rate (how often the model made up an answer when it did not know). The index was recomputed from the weights and test scores AA publishes and matches AA’s values within rounding. AA shows the index as a whole number; this page gives one decimal to tell 49.6 from 50.
The test that lifted the index most is Terminal-Bench 4.0: 66 tasks that each finish a real job in a terminal (a window where you give commands in text), and a task counts only if all its tests pass. K3 scores 13% and K3.1 Pro 67%, the top score on AA’s page for the test. Run in its own harness (the frame a model works in; here Kimi Code CLI), K3 still scored 21%, and on the older version (v2) it scored 84%, so it fell sharply on the harder 4.0 tasks. Why it fell so far is unknown. Unknown
What it takes to pass 50
Start from K3’s scores and swap the tests one by one for K3.1 Pro’s, and you can see what lifted the index past 50.
| Tests swapped to K3.1 Pro’s scores | AA index |
|---|---|
| None (all K3) | 43.6 |
| Terminal-Bench 4.0 alone | 49.0 |
| The nine other than Terminal-Bench 4.0 | 49.6 |
| Terminal-Bench 4.0 + AA-Briefcase | 50.3 |
| Terminal-Bench 4.0 + GDPval-AA | 50.5 |
| All (K3.1 Pro) | 55.0 |
The terminal test alone adds about half, yet still leaves the index 1.0 point short of 50, and without it, swapping all nine others stops at 49.6. Only the terminal test plus one big office-work test (AA-Briefcase or GDPval-AA) passes 50. The knowledge and science tests barely move the index (HLE +0.5, CritPt +0.2, SciCode +0.2). So 55 came not from a base model that knows more but from training to finish jobs.
“Private” in the table marks a test whose questions are not published. A model can be practised in advance on questions like those of a public test. In the current version, private tests carry 45% of the weight, and 3 of the 11 points gained came from private tests.
Sources: weights, public or private, and the Elo mapping: AA methodology v4.3.2 · scores: AA comparison page, K3 vs K3.1 Pro (read 2026-10-26) · Omniscience accuracy and hallucination rate: AA’s values as reported by VentureBeat · Terminal-Bench 4.0: AA’s page for the test · K3’s 21%: AA coding agent comparison (Coding Agent Index v2.0) · the 84% on v2: AA’s post on X (seen only in a search summary) · index points and the swapped indexes: this page’s computation
02 Training
Same base model, much more post-training
A language model is a mass of numbers called weights. Training means changing those numbers little by little, and it happens in two stages. Pre-training has the model read an enormous amount of text and predict the next word. There the model gains knowledge and learns to write. This page calls the result the base model. Post-training teaches that base model how to work.
Moonshot says “K3.1 Pro is a post-trained version of K3, with substantially larger post-training compute.” Its size is the same as K3’s too: about 2.8 trillion weights, of which only about 104 billion are picked for each piece of a word (token) it writes.
The same base model
K3’s post-training
K3.1 Pro’s post-training
K3’s post-training: copy examples, nine experts, merge into one
- Copy good examples (SFT): the model is shown records of work done well and learns to write the same way. The records were made by earlier Kimi models, checked several times and reviewed by people.
- 9 RL experts: the model learns by trying things and getting scores (see “One turn of reinforcement learning” below). For each of three domains (general tasks, general agents, coding agents), experts that think briefly, long or longest (low, high, max) are grown separately. Thinking length is taught with scores too: each problem gets a suitable length, and thinking past a multiple of it earns a penalty (−1 point). A generous multiple gives max; smaller ones give high and low.
- Merge into one (MOPD): the student model writes an answer itself, and word by word it is pulled toward what the expert for that domain and thinking length would pick. That is how nine experts become one model.
One turn of reinforcement learning
Reinforcement learning does not hand the model the right answer. It lets the model try things itself and gives the result a score (reward) to learn from. For an agent, it goes round like this.
- ① Task
- Programming tasks with tests; assistant work spanning days in mock email, messaging and document apps (up to thousands of tool uses in one go); web development; GPU programming; investment-banking, data-analysis and legal work (tens to hundreds of steps).
- ② Try
- Inside a sandbox (a virtual computer set apart for experiments) the model runs commands and edits files over many steps. Training and evaluating K3 created 51,219,741 sandboxes. The frame the model works in (the harness) varies too, with Kimi Code, Claude Code, Codex and others set up for different task groups, so the model does not get used to just one tool.
- ③ Check
- When the model says “done,” nobody takes its word for it. A checking program (a verifier) looks at the state after the job and scores it. The scoring checks are kept hidden and submissions are limited, to stop tricks that only pass the checks. For work with no right answer, such as writing, another AI grader writes a rubric and compares two answers.
- ④ Update
- The model is updated to make high-scoring attempts more likely and low-scoring ones less likely. This goes round over and over.
The update is loss and backpropagation, too
How does ④ Update work? Exactly as in pre-training. Two words are all you need.
- Loss
- A number for how wrong the current answer is. Smaller is better.
- Backpropagation
- A way to work out, tracing back from the answer, which way and how much to change each weight to make the loss smaller.
Measure the loss, find the direction to change by backpropagation, and move the weights a little that way. Pre-training and post-training follow the same order. What differs from stage to stage is what the loss is measured against.
| Stage | Measured against | How it changes the model |
|---|---|---|
| Pre-training | The next word that actually came in the text | Raises the chance of predicting that word |
| Copy good examples | Examples of good answers | Raises the chance of writing the example’s words |
| Reinforcement learning | The score of its own answer (there is no right answer) | Makes answers that score above average more likely, and those below less likely |
| Merge into one | The words the expert model would pick | Moves each word toward the expert’s choice |
Only reinforcement learning is different. There is no right answer, only a score, and the score is a number set by a checking program, so backpropagation cannot trace back to it. So the model’s own answer is treated as if it were the right one to measure the loss, and the model is changed toward writing that answer more if its score beat the average for the same problem, and less if it fell short.
As formulas, the loss of pre-training and of copying examples is −log p(the right word), and the loss of reinforcement learning is −(r − r) × log p(the word it wrote). p is the chance the model writes that word, r is the score, and r is the average score of the answers to the same problem. Changing the model by this loss moves it, on average, toward higher scores (the policy gradient). K3 uses K2.5’s formula, which adds guards against changing too much at once, and when merging into one it uses log pexpert − log pstudent as the score for each word.
Take one critical step. A model that has almost finished a job has two paths. Path a is “run the tests and fix”: choose it and the task succeeds 80% of the time. Path b is “say it’s done right away”: 20%. At first the model chooses a only three times in ten. The numbers are assumptions made for the explanation.
Of four tries, two succeed, so the average is half a success. A try that succeeded did better than average, so its path is made more likely; a try that failed did worse, so its path is made less likely.
| Try | Path | Result | So |
|---|---|---|---|
| 1 | a test and fix | Success | a more likely |
| 2 | b finish at once | Failure | b less likely |
| 3 | b finish at once | Success (luck) | b more likely |
| 4 | b finish at once | Failure | b less likely |
A lucky b like the third try pushes toward b. Still, the four together push toward a, because with only two paths, making b less likely is the same as making a more likely.
Worked out, the four tries push toward choosing a by +0.35, +0.15, −0.15, +0.15, which add up to +0.5. Each push is (score − average score) times how much the log probability of the chosen path moves toward a (1 − 0.3 = 0.7 if a was chosen, −0.3 if b was).
- 30%Start
- 42%1
- 56%2
- 70%3
- 79%4
- 85%5
Nobody told the model the right answer. Scores alone made the good habit grow. K3 went round this loop on tasks full of such critical steps.
Sources: policy gradient: Williams (1992) · K3’s reinforcement learning formula: K2.5 report (arXiv 2602.02276) §4.4.2 Eq. 1, which the K3 report §4.1.2 follows · the score when merging into one: K3 report §4.1.3 Eq. 15 · the four tries: this page’s computation (assumed numbers)
What K3.1 Pro added
K3.1 Pro ran this reinforcement learning “jointly across hundreds of downstream domains.” For that, Moonshot reworked its training software, long-text processing, training efficiency, sandboxes, the scheduling of practice runs (rollouts) and the quality checks of its training environments, and says the aim was stable, predictable returns from each increase in reinforcement learning compute, with consistent gains in every domain. The K3 report also showed that with more reinforcement learning compute, the model chains more tool uses and its capability rises with them.
Finishing projects in sandboxes; long-running coding
Terminal-Bench 4.0 13% → 67%
Financial modelling and research; running a simulated business
GDPval-AA Elo 1534 → 1846AA-Briefcase Elo 1499 → 1678GDP.pdf 22% → 31%
Getting work done with tools and apps
AutomationBench-AA 58% → 73%
Factual accuracy
AA-Omniscience Hallucination rate 53% → 44%
(not on the list)
HLE 47% → 52%CritPt 23% → 25%SciCode 59% → 61%AA-LCR 89% → 86%
Tests similar to the added work rose a lot; scientific reasoning and long context, which are not on the list, barely rose or fell. Moonshot has not said which training raised which test. Hypothesis
In a long job, small mistakes add up
Why did the tests of finishing jobs rise so much more? An agent moves many times to finish one job. In AA’s coding agent tests, the model moved 146 to 186 times per task on average (turns). Among those moves are several critical points where a big mistake ruins the job. Count these as steps, and say the job is done only if every step is passed. Then task success is per-step success multiplied by itself once per step. For example, if per-step success rises from 95.0% to 99.0%, a one-step test gains 4.0 points, but a 50-step task goes from 7.7% to 60.5%. So making each step just a little more reliable lifts success on long tasks a lot. Hypothesis
This is a toy model. Real agents fix mistakes and try again, and steps differ in difficulty. Neither the number of steps in Terminal-Bench tasks nor K3’s and K3.1 Pro’s per-step success is published. Still, it shows the direction: tests of finishing jobs magnify small improvements from post-training.
The architecture is the footing
The architecture did not raise the score directly, because the base model is the same. But K3 is built to read long text with little memory. It mixes layers that carry a small summary of what came before (linear attention, KDA) with layers that look at all of it again (full attention), 3:1, which cuts the memory a 1-million-token context takes. Reinforcement learning that runs long jobs over and over needs that kind of footing. Hypothesis
Sources: Moonshot K3.1 Pro blog · size: The Decoder · K3’s post-training: Kimi K3 technical report (arXiv 2607.24653) §4.1 (SFT, nine experts, reasoning effort, MOPD), §4.2 (tasks and rewards, harnesses), §5.3.2 (sandboxes), §4.1.2 Fig. 8 (RL compute and capability) · turns: AA coding agent comparison (Coding Agent Index v2.0) · toy model (task success = per-step successsteps): this page’s computation · attention: the same report §2.1
Summary
Same base model; what rose is the power to finish the job
| K3 → K3.1 Pro | |
|---|---|
| Base model (architecture, size) | The same |
| Post-training | Much larger: reinforcement learning across hundreds of domains, reworked infrastructure |
| How the model is changed | By loss and backpropagation, as in pre-training. Reinforcement learning uses a score instead of a right answer |
| Where it rose | Terminal +5.4, office work +4.6, knowledge and science +1.7, long context −0.2 |
| What it took to pass 50 | The terminal test was required, plus at least one office-work test |
| Thinking longer | Both default to the longest-thinking setting (max), and the amount they wrote for one run of the index tests (148M–161M output tokens) is similar too. This is not why the score rose |
MythIt rose because the base model got bigger.
In factThe base model is the same as K3’s. What grew is post-training.
MythIt rose by practising only for the public tests.
In factThe private agent tests, whose questions nobody can see (AA-Briefcase, AutomationBench-AA), rose a lot too: 3 of the 11 points came from private tests. But Terminal-Bench 4.0, the test that lifted the index most, is public, so whether the model practised on similar tasks is unknown.
MythReinforcement learning teaches the model the right answers.
In factIt gives only scores, no right answers. It just changes the model to do more often what scored above average among the tries it made itself.
Confirmed · Hypothesis · Unknown
- ConfirmedThe scores, the ranking and the index weights were checked against AA’s pages (read 2026-10-26). The index recomputed from the published weights and test scores matches AA’s values for K3, K3.1 Pro, Claude Opus 5.5 and Claude Fable 5.1. That K3.1 Pro is a post-trained version of K3, and its reinforcement learning domains, are Moonshot’s statements. K3’s post-training stages and its reinforcement learning formula are as written in the K3 and K2.5 technical reports.
- HypothesisThe pairing of the work Moonshot added to reinforcement learning with the tests that rose most, the training that raised Terminal-Bench 4.0, and the architecture being the footing for large-scale reinforcement learning. The toy model of long jobs and the four-try example are worked from numbers set in advance.
- UnknownWhy K3 fell so far on Terminal-Bench 4.0, how much post-training compute grew, whether K3.1 Pro’s reinforcement learning had tasks similar to Terminal-Bench 4.0, and whether the base model really is the same (the weights are not out yet).
Sources: AA comparison page · AA methodology v4.3.2 · Moonshot K3.1 Pro blog · the recomputed index: this page’s computation
Questions
Frequently asked questions
What is Kimi K3.1 Pro?
A language model released by China’s Moonshot AI on 2026-10-23. It is Kimi K3, released in July (an MoE that uses only 104B of its 2.8T parameters per token, with a 1M-token context), given far more post-training. Moonshot says it will publish the weights by November 4 (Moonshot K3.1 Pro blog).
What does the Artificial Analysis Intelligence Index (AA index) measure?
It is the weighted average of ten tests that the evaluation company Artificial Analysis runs on each model itself. The current version (v4.3.2) weights agents 30%, coding 20%, general 30% and scientific reasoning 20%, and private tests carry 45% of the weight (AA methodology).
What is Kimi K3.1 Pro’s AA index score?
55, fourth when the variants of each model count as one (2026-10-26). Ahead of it are Claude Opus 5.5 (58), Claude Sonnet 5.5 (56) and Muse Wonder Full (56). K3 (max) scored 44.
Where did K3.1 Pro gain over K3?
About half of the 11 points (+5.4) came from Terminal-Bench 4.0 (13% → 67%), and about 40% (+4.6) from four office-work agent tests. Knowledge and scientific reasoning together rose by just over 1.7 points (this page’s computation from AA’s test scores).
What is Terminal-Bench 4.0?
A test of 66 tasks that each finish a real job in a terminal. They span software, machine learning, science, operations, security, hardware and media, and a task counts only if all its tests pass. AA runs it in the mini-swe-agent harness and averages three runs per task.
Which Chinese AI models score above 50 on the AA index?
On AA’s leaderboard of 2026-10-26 (v4.3.2), only Kimi K3.1 Pro (55). The next models from Chinese companies are MiMo-V2.6-Pro (46) and Qwen3.8 Max, DeepSeek V4.1 Flash and GLM-5.3 (45). The scoring differs from version to version, so scores can be compared only within one version.
Does Kimi K3.1 Pro have a different architecture from K3?
Moonshot mentions no architectural change. It says only that it is “a post-trained version of K3, with substantially larger post-training compute.” Once the weights are published, by November 4, the configuration file will show it.
How does post-training (reinforcement learning) change the model?
As in pre-training, it measures a loss and changes the weights by backpropagation, but it uses a score instead of a right answer. Among answers the model wrote itself, those scoring above the average for the same problem are made more likely and those below less likely (the policy gradient). K3 took this method over from K2.5 (Kimi K3 technical report §4.1.2).
Can Kimi K3.1 Pro’s scores be trusted as they are?
The AA index is measured by AA, not by the model company. The private agent tests, whose questions nobody can see (AA-Briefcase, AutomationBench-AA), also rose a lot. Moonshot’s own scores use a different harness and settings, so they differ slightly from AA’s (Terminal-Bench 4.0: Moonshot 65.7%, AA 67%).
Further reading
- Moonshot AI, Kimi K3.1 Pro (blog, 2026-10-23)
- Kimi K3: Open Frontier Intelligence (Moonshot AI technical report, arXiv 2607.24653, 2026)
- Kimi K2.5: Visual Agentic Intelligence (Moonshot AI technical report, arXiv 2602.02276, 2026)
- Ronald J. Williams, Simple statistical gradient-following algorithms for connectionist reinforcement learning (Machine Learning 8, 1992)
- Artificial Analysis, Kimi K3.1 Pro model page
- Artificial Analysis, Kimi K3 vs Kimi K3.1 Pro
- Artificial Analysis, Intelligence Benchmarking Methodology (Intelligence Index v4.3.2)
- Artificial Analysis, Announcing the Intelligence Index v4.3 (2026-09-07)
- Artificial Analysis, Terminal-Bench 4.0
- Artificial Analysis, Claude Code vs Kimi Code CLI (Coding Agent Index)
- The Decoder, Chinese AI model Kimi K3.1 Pro is closing in on Anthropic's Claude Opus 5.5
- VentureBeat, Moonshot's Kimi K3.1 Pro flirts with open weights leadership
- wojciech.io, Artificial Analysis Intelligence Index v4.3 explained
- Computation: the index was recomputed from the weights and test scores AA publishes (Python). Five of six models match AA’s values within rounding; one (GLM-5.3) matches only at the edge.