Why “92% Accurate” AI Graders Might Just Be Guessing

Most IAs score a 5 or 6, so guessing the average lands within one grade 89% of the time. How to read an AI grader's accuracy claim, and what we found testing ChatGPT, Claude and Gemini.

Marksy Team, November 8, 2026

AI graders for IB coursework love one number: how often their mark lands within one grade of the IB's. Ours is 92%. One competitor's is 92% too.

Here is the problem. You can get about 89% without reading a single essay.

The trick hiding in "within one grade"

Most real IAs score a 5 or a 6. Very few get a 2 or a 7. So if a grader ignores the essay and gives every student their subject's average mark, it is still within one grade most of the time.

We tested this on 1,317 IAs and TOK pieces that students shared with the mark the IB gave them after moderation.

Within one grade of the IB
A teacher's own mark94%
Marksy92%
Guessing each subject's average mark89%

The gap between a real grader and a guess is three points. A headline of "92%" on its own tells you almost nothing.

What a good accuracy claim shows

When you look at any AI grader's accuracy page, check for these:

  • What guessing gets. If they don't show it, you can't tell how much the grader adds.
  • Results for weak and strong essays. A grader that gives everyone a 5 or 6 looks great until you look at the 3s and 7s.
  • Test essays kept out of training. Otherwise the grader has already seen the answers.
  • Where the marks come from. IB marks after moderation, a teacher's mark and the company's own reader are very different answer keys.
  • Which criteria. The Extended Essay criteria change for May 2027, so older results may not apply to your essay.

Counting every grade equally

A fairer test gives a 7 the same weight as a 5. We did that on 226 essays with the same number of weak, middle and strong ones. Guessing falls apart here, because it has no idea which essays are the 3s and the 7s.

We also pasted the official criteria and the same 226 essays into ChatGPT, Claude and Gemini, the way a student would. Then we scored everyone out of 100: 0 is what random marks get, and 100 is as close as a student's own teacher.

Score out of 100Essays 2+ grades off, per 100
Marksy7417
ChatGPT6523
Claude6126
Gemini5722

ChatGPT did no better than guessing the average mark. Claude and Gemini did worse.

Why chatbots get it wrong

Pasted into a chat, ChatGPT and Claude mark strong work far too harshly. Of the essays the IB gave a 7, Claude put 71% two or more grades too low. Gemini went the other way and was too generous to weak work.

Marksy isn't perfect either. It marks strong work a little low and weak work a little high, and a teacher is still closer. But it is the only grader in the test that beat guessing.

The nonsense essay test

We also sent one competitor a Business Management Extended Essay with nothing right in it: wrong definitions, a method based on counting car-park spaces, and a made-up final number. Its own summary said every conclusion rested on invalid evidence. Then it gave the essay a B, 22.1 out of 34. Marksy gave it 7 out of 30.

See the full comparison with MyRevisionAgent

The short version

  • "Within one grade" is easy to hit. Always ask what guessing gets.
  • Look for results on 3s and 7s, not just the average.
  • Pasting your IA into a chatbot gives marks no better than a guess.

See every number and how we tested

Check your own IB draft

Marksy marks your draft against the IB assessment criteria for your subject and shows what to fix first. The mark is an estimate; your teacher and the IB decide the real one.

92%

of marks land within one grade of the IB's final mark

Tested on 1,317 real IAs and TOK essays. See how we tested

Back to all guides