Review Lingo.dev translations

Run measurements on translations your localization engines produce, choose engines, languages, and sampling, and read each answer with its request.

A translation that Lingo.dev generates can be examined by the same measurements that examine collected activity. Each completed localization becomes a localization.completed event in Radar, and every measurement that runs on that event type answers its question about the output. These answers help explain a generated translation without requiring another generation attempt.

There is no separate evaluator resource for Lingo.dev requests. A review is a measurement in Radar → Measurements: the localization engines, language pair, and sampling that select requests are settings of its localization.completed condition, and its answers appear with the event they examined.

Turn on automatic review for an engine #

Open the localization engine's Reviewers tab. Automatic review has a switch for Glossary adherence, which checks the terms the engine's glossaries pin, and Rule adherence, which checks the rules its rulesets carry. Turning a switch on adds the engine to that built-in measurement and turns the measurement on; turning it off removes the engine. The measurement's language and sample settings stay as they are.

When a built-in runs on every engine, each engine shows its switch on. Turning one engine off stores the list of the other current engines, so an engine created afterwards is not reviewed until it is added. Read the built-in's Engines selector in Radar to confirm the stored scope.

MQM and custom measurements are not on this tab. Turn MQM on in Radar, and select engines for a custom measurement through its condition. Where your plan meters review, Automatic review shows Billed per run with the rate.

Write a criterion for generated translations #

Create a custom measurement and choose localization.completed as its trigger. Give it a name and prompt, and choose Pass or fail or 0 to 100. Define what each answer means. A boolean fits a requirement such as preserving a qualification; a percentage needs an explicit scale for partial fulfillment.

When Allow not applicable is on, define when the judge should abstain and how it should explain that answer. An irrelevant requirement and a relevant translation with missing evidence need different follow-up, even if both return no determinate judgment.

For example, a source promise of delivery within five business days carries a deadline, a business-day qualification, and a commitment. A check of that promise should explain which of those relationships changed. Matching the number alone cannot establish preservation of the claim.

The judge receives the event's payload. For a document job, it includes source and target text grouped by file, the locale pair, and the engine ID. It does not include a request-level context field. The judge is a platform-managed evaluation model and need not be the model that generated the text.

Choose which translations are reviewed #

On the measurement's General tab, the localization.completed row under Runs on selects requests:

SelectorSelects
LanguagesTarget locales. Any language selects every one.
Source languagesSource locales. Any source language selects every one.
EnginesLocalization engines whose translations are reviewed. Any engine includes engines added later.
SampleEvery event, or 50, 25, 20, 10, 5, or 1% of the otherwise selected events.

Select Save changes to apply them. Locale selection follows the locale hierarchy, as described in event selection: fr also selects fr-CA, while fr-CA does not select fr-FR. A request recorded without a localization engine does not pass an engine-limited condition.

Sampling selects a fraction of matching requests, rather than guaranteeing an exact count in a small batch. A request outside the sample, language, or engine selection still receives an N/A answer whose reasoning begins “Condition not met”; it is not billed. Keep the eligible population and the selected requests distinct in reports. A request deliberately omitted by sampling is not a failed evaluation, but its absence still matters when explaining how much of the population received a determinate answer.

Know which guidance a check reads #

Glossary adherence and rule adherence read the engine's current glossary terms and rules when they run. MQM adds the engine's current brand voice, rules, and matching terms when the event names an engine. None of them reconstructs a snapshot of the guidance recorded with the request, so a term changed after generation changes what a later historical evaluation examines.

A missing finding does not establish which guidance generation used. Inspect the source batch and model attempts in Requests, and compare the submitted context and engine configuration with Context retrieval.

Read an asynchronous answer #

Measurement is queued after the localization completes. Its queueing and evaluation are separate from translation, so successful output can exist without an answer.

Radar → Logs → Measurements lists answers. Filter by Measurement and Value, then open one for its value, reasoning, suggestion, and judge request. Match the underlying event to the job and locale you are investigating. Results distinguishes determinate answers, N/A, and missing execution.

A failing or low custom-measurement answer on an engine's translation can also become evidence for localization engine suggestions. The answer itself does not change the translation.

An asynchronous evaluation does not block the translation pipeline or accept its output for publication. Optional post-editing and human decisions are configured separately in Pipeline, Review policy, and Inbox.

Stop reviewing or remove the measurement #

Turn off the engine's switch, remove the engine from the condition, or turn the measurement off to stop future reviews. Already queued work may still complete.

Deleting a custom measurement removes its definition and conditions. Its recorded answers remain, shown as Unknown measurement. Preserve the definition with any investigation that relies on it before deletion.

For a missing answer, inspect the original request, its event, the measurement's engines, languages, and sample, and the N/A reasoning before trying another translation. To review translations generated before a measurement was turned on, use historical evaluation. Reproduction creates new work and may select a different model or guidance; it cannot recover the exact original attempt by itself.