devadmin/OpenCodeGoEvals
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

chore: weekly evaluation refresh 2026-10-26

Browse Source
This commit is contained in:
opencode-go-evals
2026-10-26 05:24:20 +00:00
parent 99489ca17b
commit f3259fdc0a
6 changed files with 1508 additions and 1184 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MODELS.md
+59 -19
View File
@@ -5,7 +5,7 @@ Short, practical notes on every model in the OpenCode Go catalogue: what each on
what it costs, and what to watch out for.
Evaluated scores come from Artificial Analysis (single consistent harness) plus vendor model cards.
Catalogue metadata comes from models.dev's OpenCode Go provider. Data snapshot: **2026-10-21**.
Catalogue metadata comes from models.dev's OpenCode Go provider. Data snapshot: **2026-10-26**.
The interactive version with the full task matrix lives at `site/index.html`.
> No model in the Go catalogue generates images. Every Go model is text-output; several accept image,
@@ -28,6 +28,16 @@ The interactive version with the full task matrix lives at `site/index.html`.
### undefined
**moonshotai** · released 2026-10-23 · $2 in / $10 out per 1M tokens · 1.05M context · text, image, video in → text out
Multimodal Kimi model with 1M context and always-on reasoning for coding, 3D creation, financial research, and agentic work
_No hand-written write-up yet._
*Evaluated:* Intelligence Index 55.1 · FrontierCode 47.1 · TB4.0 67.2 · AutomationBench 73.3 · HLE 52.5 · LCR 86.3 · MMMU-Pro 83.9
### undefined
**google** · released 2026-10-13 · $4 in / $20 out per 1M tokens · 1.05M context · text, image, video, audio, pdf in → text out
Targeted for: **Frontier coding, Deep research, Legal & financial work, Security, Multimodal**
@@ -46,6 +56,16 @@ Hardest frontier work: large codebase refactors and optimisation (Google used it
### undefined
**google** · released 2026-10-13 · $2 in / $10 out per 1M tokens · 1.05M context · text, image, video, audio, pdf in → text out
Google's flagship reasoning model for complex software engineering, deep math and science, and long-horizon autonomous agents
_No hand-written write-up yet._
*Evaluated:* Intelligence Index 52.6 · FrontierCode 43.5 · TB4.0 57.1 · AutomationBench 77.5 · HLE 57.1 · LCR 79.7
### undefined
**meta** · released 2026-09-02 · $0.1 in / $0.2 out per 1M tokens · 1.05M context · text, image, video, pdf, audio in → text out
Targeted for: **Long-horizon agents, Multi-agent workflows, Coding, Multimodal, Cheap**
@@ -183,6 +203,16 @@ Anything that has to answer fast and often: interactive completions, classificat
### undefined
**minimax** · released 2026-10-19 · $0.15 in / $0.6 out per 1M tokens · 1M context · text, image, video in → text out
Frontier multimodal coding model with 1M context window and tunable thinking depth
_No hand-written write-up yet._
*Evaluated:* Intelligence Index 42.7 · FrontierCode 38.6 · TB4.0 25.8 · AutomationBench 72.5 · HLE 48 · LCR 79.7 · MMMU-Pro 79.1
### undefined
**minimax** · released 2026-10-19 · $0 in / $0 out per 1M tokens · **free tier (limited time)** · 1M context · text, image, video in → text out
Targeted for: **Coding, Multimodal, Agentic, Free tier**
@@ -200,7 +230,7 @@ Day-to-day coding-agent work on real repositories: reading long codebases, editi
### undefined
**zhipuai** · released 2026-08-26 · $0.15 in / $0.5 out per 1M tokens · 1M context · text, image, video, pdf in → text out
**zhipuai** · released 2026-08-26 · $0.15 in / $0.5 out per 1M tokens · 1M context · text, image, video in → text out
Targeted for: **Efficient coding, Long-horizon agents, Multimodal, Cheap**
@@ -360,7 +390,7 @@ Reflection's first open-weight model: a 501B-parameter sparse Mixture-of-Experts
Cost-efficient coding and agent loops. The MoE design activates only 23B parameters per token, so it is a cheap way to run a lot of terminal work, tool calls and multi-step reasoning. Text-only, so it is a pure code/reasoning worker rather than a perception model.
*Evaluated:* Intelligence Index 30.3 · TB4.0 3 · AutomationBench 48.5 · HLE 34.7 · LCR 78
*Evaluated:* Intelligence Index 30.3 · FrontierCode 24.3 · TB4.0 3 · AutomationBench 48.5 · HLE 34.7 · LCR 78
**Caveats**
@@ -505,20 +535,20 @@ Long-context reasoning with tool calling at a modest price — useful for repo-s
### undefined
**tencent** · released 2026-08-28 · $0.834 in / $2.501 out per 1M tokens · 1.02M context · text in → text out
**undisclosed** · released 2026-10-12 · $0 in / $0 out per 1M tokens · **free tier (limited time)** · 1.05M context · text in → text out
Targeted for: **Agentic workflows, Productivity, Complex task execution, Open weights**
Targeted for: **Free tier, Coding, Agentic workflows, Structured output**
A next-generation productivity model with significantly enhanced agent and complex task execution capabilities.
A text-only reasoning model for coding and agentic workflows, with tool calling and structured JSON outputs.
Productivity and workflow automation: multi-step office/enterprise tasks, tool orchestration and long-running assistants. Tencent positions it around agentic capability and complex task execution rather than raw chat quality, with a 1M window for large corpora.
An anonymous free tier aimed at agents: tool calling and structured JSON output make it useful as a sub-agent or a formatter that has to hand machine-readable results to the next step.
*Evaluated:* no public third-party evaluation published yet.
**Caveats**
- Preview release with no third-party evaluations published yet, so performance is unverified outside the vendor's own numbers.
- Text input only.
- No vendor, no model card, no public evaluation — verify behaviour before depending on it.
- Not listed in the live Go catalogue endpoint.
### undefined
@@ -538,6 +568,23 @@ Same model as the paid Step 5 Preview, without the token bill — useful for lon
### undefined
**tencent** · released 2026-08-28 · $0.834 in / $2.501 out per 1M tokens · 1.02M context · text in → text out
Targeted for: **Agentic workflows, Productivity, Complex task execution, Open weights**
A next-generation productivity model with significantly enhanced agent and complex task execution capabilities.
Productivity and workflow automation: multi-step office/enterprise tasks, tool orchestration and long-running assistants. Tencent positions it around agentic capability and complex task execution rather than raw chat quality, with a 1M window for large corpora.
*Evaluated:* no public third-party evaluation published yet.
**Caveats**
- Preview release with no third-party evaluations published yet, so performance is unverified outside the vendor's own numbers.
- Text input only.
### undefined
**meituan** · released 2026-09-25 · $0 in / $0 out per 1M tokens · **free tier (limited time)** · 1M context · text, image in → text out
Targeted for: **Coding, Agentic, Image understanding, Free tier**
@@ -573,21 +620,14 @@ An unreleased preview model with full multimodal input and tool use, priced like
### undefined
**undisclosed** · released 2026-10-12 · $0 in / $0 out per 1M tokens · **free tier (limited time)** · 1.05M context · text in → text out
**undisclosed** · released 2026-10-24 · $0 in / $0 out per 1M tokens · **free tier (limited time)** · 0.26M context · text in → text out
Targeted for: **Free tier, Coding, Agentic workflows, Structured output**
Stealth model, free for a limited time
A text-only reasoning model for coding and agentic workflows, with tool calling and structured JSON outputs.
An anonymous free tier aimed at agents: tool calling and structured JSON output make it useful as a sub-agent or a formatter that has to hand machine-readable results to the next step.
_No hand-written write-up yet._
*Evaluated:* no public third-party evaluation published yet.
**Caveats**
- No vendor, no model card, no public evaluation — verify behaviour before depending on it.
- Not listed in the live Go catalogue endpoint.
## Free models outside the Go plan
### MiMo-V2.6-Flash Free (Xiaomi)