Validate quality findings
Establish whether a flagged translation contains a real problem, explain disagreements, and route the accepted diagnosis to the right repair.
A finding is useful when a reviewer can examine the same words and understand why they were flagged. Validation connects the evaluator's claim to the source, target, and requirement, then decides what the evidence supports. The outcome may be a correction, a clearer source, or a better measurement.
Use this guide for findings already returned by a check. Measurement design covers the criterion itself, and feedback triage covers the repair after a diagnosis is accepted.
Reconstruct the comparison #
Open the result and retain its available answer, reasoning, and request or event identity. Pair it with the actual evaluated target and corresponding source revision. Use only authorized evidence needed for the investigation; a result read does not imply a complete revision archive or export feature.
Confirm that the criterion applies. A navigation label has no delivery commitment to preserve. A delivery sentence with no paired source is still relevant to a claim-preservation assessment, but the evidence cannot yet settle it. Keep missing evidence, unclear applicability, and a completed indeterminate answer distinct.
If the target was translated before a source change, it may be stale rather than wrong under its original source. If the evaluated revisions cannot be established, retain the finding as unresolved instead of judging the older answer against the latest text.
Test the claimed change in meaning #
Consider the illustrative source Your order will arrive within five business days. A target equivalent to may arrive within five business days weakens its commitment. A target equivalent to will arrive within five working days may preserve it, depending on the locale and business policy.
Read the deadline, qualification, and degree of certainty together. Matching the number is insufficient, but different wording is not itself an error. Ask the reviewer to identify the source and target passages, explain the intended meaning, and show where the target preserves or changes it.
Use explicit outcomes in the team's adjudication record:
| Outcome | Evidence and next action |
|---|---|
| Confirmed translation defect | Identify the changed meaning or violated requirement, then propose a correction. |
| False positive | Explain why the target is acceptable under the criterion and retain the case for evaluator improvement. |
| Source problem | Identify ambiguity or contradiction and ask the source owner to resolve it. |
| Unresolved | Name the missing evidence or disagreement without assigning a pass or defect label. |
These are method labels for the record, rather than promised product states. A confident evaluator explanation remains a claim to inspect.
Resolve disagreement at the right level #
Product knowledge and language knowledge answer different parts of the question. If reviewers disagree about whether “business days” excludes a local holiday, resolve the product policy with its owner. If they agree on the policy but disagree about the target expression, obtain the linguistic evidence needed to settle that expression.
Record the rationale and decision owner. Averaging contradictory categorical labels does not validate a defect. An unresolved case can expose an unclear requirement, even when both reviewers understand the language well.
Use resolved examples to improve the criterion, then test the change on a separate set that was not used to write its instructions. Retain the earlier criterion version so later outcomes can be interpreted against the question actually asked.
Validate a set of findings #
Select examples across the relevant languages, content classes, and apparent severities, including likely defects, likely false positives, and ambiguous cases. Record the selection method. Deliberately chosen cases can reveal failure modes without estimating their prevalence in all content.
A sample of flagged translations can show how often the selected flags are justified. It cannot reveal all the errors that the evaluator failed to flag. To examine missed errors, inspect a separately defined sample of unflagged translations or independently judged cases. Keep those populations distinct in the analysis.
For an illustrative sample of 20 flags, suppose 12 are confirmed defects, 5 are false positives, and 3 remain unresolved. Among the 17 resolved findings, 12/17 are confirmed; resolved coverage is 17/20. Report both fractions and the unresolved count. They describe this selected flagged sample, rather than the defect rate of all translations or the evaluator's recall.
If a broader quality pass rate is also needed, use the eligible population, determinate denominator, and coverage definitions in measurement design. That calculation answers a different question from how often a flag was confirmed.
Turn the diagnosis into a repair #
Correct a confirmed target defect where accepted text is maintained. Send a source ambiguity to its owner. Revise an evaluator that repeatedly flags acceptable writing. Investigate delivery or rendering in the system that loads the text.
A local correction and a shared instruction have different reach. Before turning the finding into a glossary entry, rule, or voice change, examine where that guidance applies and whether it conflicts with existing requirements. Then use feedback triage to propose the authorized repair and recheck the resulting revision.