How classification accuracy is measured

Tekst measures how well a classification model is performing by comparing its predictions against a test set of validated examples. This article explains how that score is calculated so you can interpret the accuracy you see and know where to focus your improvements.

If you have not built a test set yet, see the "Set up your first classification model" article first.

The test set is the source of truth

Accuracy is always measured against your test set: a collection of messages for which you have confirmed the correct label (the "ground truth"). For each item, Tekst compares the label the model predicted against the label you confirmed. The more representative your test set, the more meaningful the accuracy figure.

This means accuracy cannot be calculated until you have added test items with confirmed labels. Until then, the model header shows an Evaluate model button instead of an accuracy figure.

How the score is calculated

Scoring is straightforward: a prediction is correct only if it exactly matches the confirmed label, and wrong otherwise.

Tekst reports these in the Model accuracy panel, which opens when you select the accuracy figure in the model header:

• The overall accuracy percentage across all test items.
• A Tag evaluation list with a per-label breakdown, marking each label All correct or with the number of conversations it got wrong, so you can see which categories perform well and which struggle.
• A detail view per label that shows, for each conversation, the label you confirmed beside the label the model predicted. This is the fastest way to spot two labels that the model keeps mixing up.

Spotting labels that get mixed up

Select a label in the Tag evaluation list to open its detail view. The mix-ups are listed first, each showing the confirmed label in the Correct column and the model's prediction next to it, and the correct predictions are collapsed behind a Show N correct toggle.

When the same two labels keep getting mixed up, their descriptions usually overlap too much - a sign that you should sharpen the wording of those labels or add more examples.

Improving the score

Use the breakdowns above to decide where to act. The most effective levers are usually clearer label descriptions and a richer, more balanced test set. As your team corrects predictions over time, those corrections also feed back into the model.

For related background, see:

• How is model accuracy calculated?
• What data does Tekst store and how does the learning process work?