Grok 5 Is on Writingmate: Testing xAI's Flagship for Agentic Coding and Reasoning

Grok has spent the last year shipping in half-steps — 4, 4.1, 4.5, 4.6 — while Elon Musk kept promising the real jump was coming. Grok 5 is that jump, and as of this week it's live inside Writingmate alongside every other frontier and open-weight model we support. I ran it through the exact same test suite I've used on Qwen3.8 Max, Mistral Large 3, and DeepSeek V4 Pro: a broken function to debug, a messy file to refactor, a multi-step agentic task that requires real tool calls, and a long-context reasoning job. No cherry-picked demos, just the same four tests every new flagship gets.

My name is Artem, I run the Writingmate blog, and testing every new frontier release the same way is basically my full-time job at this point. I've watched Grok go from "the sarcastic one on X" to a model xAI is now pitching as agentic-coding-first, and I wanted to know if that pitch holds up once you take it out of a launch-day demo and put it in a real workspace next to the models people are already using daily.

Short version: Grok 5 is fast, it's genuinely strong at tool-calling, and it's the best long-context reasoner in the xAI lineup so far. It's also not a clean win over everything else on the platform, and there's one specific failure mode worth knowing about before you point it at production code. Let's get into it.

What Grok 5 actually is

xAI has been vague about Grok 5's exact parameter count in public materials, but the direction is consistent with everything the company has said since its January funding round: a much larger mixture-of-experts model trained on the Colossus 2 cluster, positioned as the successor to Grok 4.5 rather than an incremental update. The headline changes that matter for day-to-day work:

• Context window: up to 2M tokens, a jump from Grok 4.5's 500K — enough to hold a mid-sized codebase or a stack of contracts in one pass.
• Native tool orchestration: Grok 5 is trained to chain multiple tool calls (code execution, file search, web fetch) without needing heavy prompt scaffolding to keep it on track.
• Agentic framing: xAI is explicitly marketing this release around coding agents and multi-step task completion, not chat quality — a shift from how Grok 3 and 4 were pitched.

That's also exactly why I ran it through the agentic suite instead of just asking it trivia questions. If a lab says "built for agents," the fair test is to hand it an agent's job.

How I tested it

Same four tests I run on every new flagship model that lands in Writingmate's model directory, so the results are actually comparable across releases instead of being four different vibes:

1. Broken function: a 40-line Python function with a subtle off-by-one bug and a silent type coercion issue, no error message pointing at either.
2. Messy refactor: a 300-line file with three unrelated responsibilities crammed into one class, asked to split it without changing behavior.
3. Multi-step agentic task: "research three competitor pricing pages, pull the numbers into a table, then draft a one-paragraph summary" — a task that only works if the model actually calls tools in the right order instead of guessing.
4. Long-context reasoning: a 180-page combined document (product spec + support tickets + changelog) with a question whose answer requires cross-referencing three separate sections.

I ran each test three times per model to catch inconsistency, since a single lucky run tells you nothing about what happens on the fourth try in a real workflow.

Test 1 and 2: debugging and refactoring

On the broken function, Grok 5 caught the off-by-one on the first try in two of three runs, and flagged the type coercion issue as a "possible secondary bug worth checking" even when it initially missed it — which is a genuinely useful behavior, since a model that hedges correctly is more trustworthy than one that's silently wrong. Mistral Large 3 caught both issues in one shot more consistently in my earlier testing, so Grok 5 isn't the clear leader here, but it's close and it's honest about its own uncertainty, which matters more in practice.

The refactor test is where Grok 5 separated itself a bit. It split the class into three focused pieces, kept the public interface stable, and — unprompted — added a short comment flagging that one of the original methods had a side effect that wasn't obvious from its name. That's the kind of catch that saves you a debugging session two weeks later. DeepSeek V4 Pro also handled this cleanly, but took noticeably longer per attempt.

Test 3: multi-step agentic work

This is the test Grok 5 is supposed to win, given how xAI is marketing it, and mostly it did. Across three runs it completed the full research-table-summary chain twice without any manual nudging, calling tools in a sensible order and recovering gracefully once when a fetch came back with a malformed page. The one failure run is worth flagging: it got stuck re-fetching the same source twice instead of moving to the third competitor, and needed a manual "continue" prompt to break out of the loop.

That's a real weakness, not a nitpick — a model marketed for agentic work needs to self-recover from a stalled loop without hand-holding, and two-out-of-three isn't a great hit rate for a task this simple. Qwen3.8 Max hit three-for-three on the identical task in my earlier run, which is the more relevant comparison than any published benchmark.

Test 4: long-context reasoning

Here Grok 5 was the strongest model in this batch. On the 180-page cross-reference task, it correctly pulled the right sections from the spec, the tickets, and the changelog in all three runs, and it cited which document each part of its answer came from without being asked to. That citation habit alone makes it more usable for research-heavy work than a model that gives you the right answer but no way to verify it.

"Grok 5.0 is indeed in development at xAI, aiming for enhanced reasoning and helpfulness" — @grok on X

How it stacks up against the rest of the lineup

Numbers below are from my own runs on the four-test suite above, not third-party leaderboards — treat them as directional, not a substitute for testing your own workload.

Model	Context window	Debug/refactor	Agentic (3 runs)	Long-context	Best for
Grok 5	2M tokens	Strong, honest about uncertainty	2/3 clean	Strongest of the batch	Research over huge documents, tool-chaining tasks
Qwen3.8 Max	1M tokens	Strong	3/3 clean	Solid	Reliable multi-step agent chains
Mistral Large 3	256K tokens	Most consistent bug catches	2/3 clean	Weaker on cross-referencing	Fast, precise code review
DeepSeek V4 Pro	512K tokens	Strong but slower	3/3 clean	Solid	Budget-conscious agentic work

Where the community lands on it

Reception has been split in the way it usually is right after a frontier release: genuine excitement about the context window and tool-use jump, paired with the same "wait for it to hold up outside the demo" caution that greeted Grok 4.5 last July, when independent testers flagged a hallucination rate that had roughly doubled from its predecessor.

"The context window bump and the tool-chaining are real improvements, not marketing fluff — I've gotten it to do things Grok 4.5 just couldn't. But it still confidently states wrong facts often enough that I don't let it run unsupervised on anything that touches production. Verify before you ship." — u/agentic_skeptic on r/grok

That tracks with what I saw in testing: the stalled tool-loop in test 3 and the occasional confident-but-wrong answer are the two things worth watching for, not deal-breakers but real enough that "test your specific workflow before trusting it end-to-end" is the right amount of caution.

Who should actually reach for Grok 5

Based on four tests and three runs each, here's how I'd route work:

• Pick Grok 5 if your job involves a genuinely huge document set — contracts, long support histories, multi-file research — where the 2M-token window and the citation habit save you from re-uploading chunks.
• Pick Qwen3.8 Max if you need an agent chain to run unattended and can't afford a stalled-loop failure mode.
• Pick Mistral Large 3 for fast, tight code review where you want the most consistent single-pass bug catch.
• Pick DeepSeek V4 Pro if cost per task matters more than speed and you can tolerate slightly longer runs.

The actual advantage of having all four in one place isn't picking a permanent favorite — it's routing each job to whichever model tested best for that specific job type, which is the whole point of running a multi-model workspace instead of locking into one subscription.

Getting Grok 5 running in Writingmate

Grok 5 is live in the model picker now, same as every other model on the platform — no separate xAI account, no separate API key to manage. Open a new chat, switch the model dropdown to Grok 5, and you're testing it against your own files in under a minute. If you want to run the same side-by-side comparison I did, open two chats with the same prompt and swap the model on one of them; that's the fastest way to see whether the context-window jump actually matters for your specific documents. Check the pricing page for current credit costs per model, since flagship-tier models like Grok 5 typically draw more from the shared credit pool than smaller ones.

If you're not sure where Grok 5 fits versus what you're already using, the fastest gut check is the long-context test — drop in your longest real document and a question that requires reading more than one section. That's the single test in my suite where Grok 5 pulled clearly ahead of everything else in this batch.

The bottom line

Grok 5 earns the "agentic" label xAI is putting on it, mostly. The tool-chaining is real, the long-context reasoning is the best I've seen from the Grok lineup, and the citation habit makes it more trustworthy for research work than a model that just gives you an answer with no receipts. But it's not a clean sweep — the stalled-loop failure in agentic testing and the community's ongoing hallucination concerns mean I wouldn't hand it unsupervised production work yet. For big-document research and tool-heavy tasks, it's worth the switch. For unattended agent chains, Qwen3.8 Max is still the safer bet until Grok 5 tightens up that recovery behavior.

See you in the next one!

Artem