Evaluation outcomes

Interpret measurement scores, pass rates, and rule and term counts through their actual answers and denominators.

Evaluation-outcome reports summarize measurement answers. The score panel on Radar → Overview follows the current month; an Explore report with the Measurements source narrows answers by measurement, language, engine, or event type. Use either to locate a pattern, then open individual answers to understand the text behind it.

The overview score and Explore reports summarize pass-or-fail and numeric-score answers, including their N/A results where counts apply. Results shows those individual answers with their text and available reasoning. Edit-pattern classifications are outside these reports and result lists. Keep the measurement and event type explicit: similar names can describe different evidence.

Overview score #

The panel is labelled with the month, and its bars are daily average scores. Each day pools that day's pass-or-fail and 0–100 answers from every measurement: a pass counts 1, a fail 0, and a score its value divided by 100. N/A answers are left out, and structured answers such as edit patterns are not included. The headline averages the days that have a score without weighting them by answer count. The line below compares the month so far with the same days of the previous month, in points.

Because the panel pools every measurement, it mixes criteria whose numbers point in different directions. A higher edit distance means more text changed, and a passing edit-rate answer means the text was edited. Treat a movement in the panel as a reason to open Explore with one measurement selected, rather than as a quality verdict.

Scores #

Under Data, choose the field and calculation for the measurement answers you want to examine:

DataCalculation
MeasurementCount: pass-or-fail and numeric-score answers, N/A included. Unique: how many different measurements answered
ScoreAverage or Sum of the 0 to 100 answers that are not N/A
Value BoolAverage: passed pass-or-fail answers divided by those that are not N/A
Passed, FailedPass-or-fail answers that passed or failed
Not applicableN/A answers
Value kind, Event type, Target locale, Source locale, Decided by, Judge model, Engine, ProviderCount of the answers that carry the field, or Unique values it takes

Select one Measurement as a filter or breakdown, or start from its preset, before comparing scores. Without it, Score combines unrelated criteria. Use Judge model, Value kind, Decided by, locales, engine, or provider as a breakdown to examine the answer population.

For Unique over a whole period, use Time → Total. Adding daily distinct counts can count a measurement or engine more than once.

Glossary adherence and rule adherence answers show how consistently your glossary terms and rules are applied; edit rate and edit distance show how often and how much people change translations. For edit rate on a Zendesk Help Center or a GitHub repository, Explore counts each translation Lingo.dev wrote once - passed when a person changed it - so Value Bool with Average is the share of those translations that people changed, counted in the period Lingo.dev wrote them.

A count of stored answers does not establish the whole eligible population. Failed attempts and translations that never reached evaluation are absent. Events outside a condition's languages, engines, or sample can have N/A answers. Preserve that selection alongside completion and determinate coverage.

Read a whole-period average correctly #

Suppose an illustrative day has 1 pass from 1 answer and another has 0 passes from 9 answers. Averaging daily rates gives 50%; pooling all ten answers gives 10%. The overview headline averages days, so it reports 50%. In Explore, choose Value Bool → Average, Time → Total, and a Number chart over the same range to get 10%. The query response's totals.averageScore gives the same pooled result for these pass-or-fail answers in the report query contract.

A line chart by day shows each day's pooled value; it cannot show how many answers stand behind a point. Add Measurement with Count and inspect Not applicable when a day's population matters. Measurement names can change, so keep IDs in a recurring analysis.

A changed average may describe a different criterion, sample, language mix, or answer format. Inspect those before attributing the difference to better translations. A numeric score is not inherently a probability of correctness, and a high average can coexist with a severe failure in one passage.

Rules and terms #

Rule adherence scores the share of applicable rules that the translation followed. Glossary adherence scores the share of returned term findings marked as applied. Each answer's score is passed / checked × 100; when nothing was checked, the answer is N/A.

Use Score with Average to compare the measurement's numeric answers in Explore. That gives each determinate answer equal weight, however many rules or terms applied. Results exposes the score and available explanation, but not separate passed and checked totals. The report therefore cannot tell you the pooled share of individual rule or term checks that passed.

The average score is also not the percentage of translations that followed every rule. An N/A answer with no applicable checks differs from a translation that failed them. Open Reasoning and the available Judge request to inspect the findings behind a score before changing the translation or guidance.

These checks read the engine's current glossary and rules when they run. They can therefore disagree with what a request actually received at generation time. Inspect failed rules and missed terms with the text before correcting a target. If expected guidance was absent from the request, investigate Context retrieval. If a rule itself is unclear, clarify it in Rules; for terms, use Glossaries and Define a term with boundaries.

Compare languages and investigate causes #

Use the same criterion, source population, interval, and selection policy when comparing locales. Put scored counts and missingness beside outcomes. A language with fewer sampled answers can appear stronger simply because different content reached the judge.

Workload, quality findings, and edit activity remain distinct measures; counts of repository changes cannot establish linguistic corrections. Read source and context as well as target text. An ambiguous source or conflicting instruction can produce recurring findings across languages. Diagnosing that pattern requires source revisions and an explicit classification, rather than inferring source defects from low target scores alone.

For a deliberate comparison, Compare translation models preserves matched content and criteria. For a suspicious finding, Validate quality findings establishes whether the reported problem is real.