OpenCodeGoEvals/README.md

OpenCodeGoEvals

Benchmarks and plain-language notes for every model in the OpenCode Go catalog: a task-by-task performance matrix, cost/value analysis, and a description of what each model is actually targeted for (agentic work, coding, data analysis, image understanding, deep research, high-volume chores).

Static site, no build step — it is plain HTML/CSS/ES modules, deployed to Cloudflare Pages.

What's here

Path	What it is
site/index.html	The site: performance matrix, price/value charts, model guide
site/assets/data.js	Model catalogue — pricing, context, modality, target descriptions, caveats
site/assets/evals.js	Benchmark scores (0–100) per model per task family
site/assets/app.js	Rendering logic for the matrix, charts and guide
site/assets/style.css	Styling
MODELS.md	The same model descriptions in Markdown
scripts/sync-catalog.mjs	Weekly refresh: models.dev catalogue + Artificial Analysis → catalog.js, evals.js, snapshot.json
scripts/build-models-md.mjs	Regenerates MODELS.md from the three data layers
scripts/notify-new-models.mjs	Files a Gitea issue when a new model needs a write-up
scripts/deploy.sh	Manual Cloudflare Pages deploy
.gitea/workflows/weekly-eval.yml	Weekly scheduled evaluation refresh
wrangler.toml	Cloudflare Pages project configuration
.gitea/workflows/pages.yml	CI: builds and deploys to Cloudflare Pages on every push to main

Data sources

• OpenCode Go docs — pricing, monthly allowances, privacy/retention, client support
• OpenCode Data — catalog metadata, context windows, usage and retention stats
• Artificial Analysis — benchmark scores and the Intelligence Index
• Vendor announcements — Google (Gemini 4 Argon), Reflection AI (Beam), Meituan (LongCat), MiniMax (M3.1 Flash), xAI (Grok 4.7), StepFun (Step 5), Qwen, Zhipu, Moonshot, DeepSeek, Xiaomi, Tencent

Data snapshot date: 2026-10-21. Re-run the collection script to refresh.

Publishing to Cloudflare Pages

1. Create the Pages project

npm install
npx wrangler login                       # or export CLOUDFLARE_API_TOKEN
npx wrangler pages project create opencodogoevals

2. Give CI the credentials

Add these as repository secrets (Gitea: Settings → Actions → Secrets):

Secret	Value
CLOUDFLARE_API_TOKEN	Cloudflare API token with Cloudflare Pages: Edit (and Account Settings: Read)
CLOUDFLARE_ACCOUNT_ID	The Cloudflare account that owns the Pages project

The workflow in .gitea/workflows/pages.yml then:

1. checks out the repo and installs Wrangler,
2. stages MODELS.md next to the site,
3. creates the Pages project if it does not exist yet (idempotent),
4. deploys site/ on every push to main, on a manual Run workflow, and hourly.

A runner must be registered on the Gitea instance for the workflow to execute.

3. Deploy manually (optional)

./scripts/deploy.sh     # uses env vars or your wrangler login session
npm run deploy          # same thing, wrangler only
npm run dev             # local preview on http://localhost:8788

Custom domain

Set it in Cloudflare Pages → Custom domains, then add a CNAME record pointing at <project>.pages.dev.

Local development

The site is static — open site/index.html directly, or use npm run dev for a Cloudflare-compatible local server.

The weekly cycle

Two workflows run in Gitea Actions:

Workflow	Trigger	What it does
weekly-eval.yml	Mondays 05:23 UTC (cron), or manual	Re-scrapes the catalogue and benchmark scores, rebuilds MODELS.md, opens an issue for any new model needing a write-up, and commits the result
pages.yml	every push to main	Deploys site/ to Cloudflare Pages

The chain is: schedule → data refresh → commit → Pages deploy, so the live site updates itself without anyone touching it. The weekly run is idempotent — if nothing changed, nothing is committed and nothing is redeployed.

A run leaves behind a trail you can audit:

• site/assets/snapshot.json — when the data was taken, which models are new or gone, and every benchmark score that moved since the previous run
• the commit chore: weekly evaluation refresh <date> in the history
• the Actions run log for the week
• a Gitea issue when a new model appears in Go without a write-up

What is automated vs. hand-written

File	Owner	Contents
site/assets/catalog.js	auto-generated	every model in Go: pricing, context, modalities, release date
site/assets/evals.js	auto-generated	benchmark scores per model per task family
site/assets/data.js	hand-written	what each model is targeted for, caveats, tags
site/assets/snapshot.json	auto-generated	timestamp, added/removed models, score deltas

When you want to change a description, edit site/assets/data.js and run node scripts/build-models-md.mjs; the weekly refresh will never overwrite it.

Running it by hand

node scripts/sync-catalog.mjs        # refresh catalogue + scores
node scripts/build-models-md.mjs     # rebuild MODELS.md
node scripts/notify-new-models.mjs   # file the "needs write-up" issue

If the scheduler is not firing

Gitea only runs scheduled workflows when a runner is registered and awake. If Mondays slip past, trigger it manually from the Actions tab (Run workflow), or run the three commands above on any machine with network access and push the result.

Caveats

• Task-matrix cells use the best published evaluation per task family; when a model has not been run on a benchmark the column falls back to another one, so the underlying benchmark can differ between rows in a column. Hover a cell to see which one it is.
• Free-tier models (MiniMax M3.1 Flash Free, Beam Free, LongCat 2.5 Preview Free) are limited-time offers.
• Contributor-tier models (Gemini 4 Argon, Muse Spark 1.3) may use your prompts and completions for training — check the Go docs' privacy table.
• No Go model generates images; they are all text-output.

License

MIT — see LICENSE.