How we measure AI extraction accuracy
A number without a methodology is marketing. At Monokaido we have an automated evaluation set with gold-label data — on it we currently measure 92% or more.
For a large customer, the difference between “we claim” and “we measure” is fundamental
When AI fills in parameters across the whole catalogue, you need to know whether you can rely on it. Real measurement means that we know the correct answer in advance — and regularly check how close AI gets to it. Without this discipline, every accuracy figure is just an estimate.
“As a data manager, I want to see how the accuracy of AI extraction is measured and how it develops over time, so that I can rely on it for thousands of products.”
What our measurement is built on
Practices from ML model development, applied to extracting parameters from product data.
Gold-label dataset
A set of products with manually verified parameters (ground truth), maintained by in-house data specialists — not by the AI itself.
Automated evaluation run
It goes through the whole dataset, calls AI extraction and compares the output with the verified values.
Accuracy by parameter type
Not just an overall score — dimensions, materials and performance data each have their own number.
Repeatable runs
Every run has its own version — you can compare accuracy after a change of model, prompt or parameter template.
Trend over time
You see the accuracy curve for recent months, not a single number from a single run.
Auditable output
Every comparison leaves a record — you can trace where AI got it wrong and why.
Want to see the actual numbers?
We'll show you the results of the latest evaluation runs and how accuracy changed after each change of model.
