Custom measurements

Create a content-specific check, choose its answer and event scope, and test whether it detects the intended change.

A custom measurement makes one of your content requirements explicit enough to test. It can ask whether a translation preserves a qualification, follows a defined voice requirement, or keeps two product concepts distinct. The prompt tells the judge which difference matters and which differences are acceptable.

This page creates a measurement in Radar → Measurements. To review translations your localization engines generate, choose localization.completed as the trigger; Review Lingo.dev translations covers engine selection and where those answers appear.

Write the requirement as a comparison #

Suppose the source says a feature may become available after approval. A translation can retain the words for the feature and approval while turning that possibility into a promise. A useful measurement must examine their relationship, rather than only matching terms.

For that requirement, a boolean prompt could read:

Example prompt: preserve certainty and conditions

Compare the source and target claims. Return true when the target preserves the source's degree of certainty or obligation, its conditions, and the relationship between them. Allow natural changes in wording and sentence structure.

Return false when a possibility becomes a promise, an obligation changes, or a condition is removed or attached to the wrong claim. Identify the affected source and target wording in the explanation and suggest a correction when the evidence supports one.

Return null when the text contains no applicable claim or when the available evidence cannot support the comparison. Explain which situation applies and what evidence is missing, if any.

The prompt defines a narrow question with a reason to disagree with its answer. It does not ask the judge to invent the team's entire quality standard.

Create the measurement #

  1. In Radar → Measurements, start a new measurement and enter the prompt.
  2. Give it a recognizable name, such as Preserve certainty and conditions.
  3. Under Value, choose Pass or fail for a boolean answer, or 0 to 100 for a numeric scale whose intermediate values you can define.
  4. Keep Allow not applicable on when the judge should be able to answer N/A, as in the example prompt. Turned off, the judge must give a pass, a fail, or a score.
  5. Choose the Trigger and Languages. The form initially selects completed localization; change it when your intended evidence arrives through another event type. No languages selected means every target locale.
  6. Select Create measurement. The definition is saved with the selected event condition enabled. On its General tab, narrow that condition by source language, engine, or sample, or add other event types with Choose an event.

The judge uses the platform-managed evaluation model. The measurement's prompt does not select the generation model, change localization engine guidance, or add a content-upload route. Configure a supported intake to supply its evidence.

Understand the answer contract #

Answer typeReturned judgment
boolpass: true, false, or null.
scorescore: a finite value from 0 through 100, or null.

Both types include nullable reasoning and suggestion. A null judgment is available only while Allow not applicable is on. Null means the judge could not give the requested judgment; its reason may indicate inapplicability, uncertainty, or missing evidence. An event that fails a condition's selection is answered N/A without asking the judge, whatever this setting. A percentage is a value on your defined scale, not automatically a probability that the translation is correct.

In result history, the boolean or score is presented through value, valueKind, and notApplicable. Read the N/A indicator before interpreting a numeric zero. Results explains this representation and the available judge trace.

Test the distinction the prompt makes #

Generate a small set of supported test activity after saving the condition, or use a bounded historical evaluation for existing events. Include known errors and acceptable translations: a changed commitment, a removed condition, an unrelated label, a case missing its source, and a natural paraphrase that preserves the meaning.

Read the answer beside the actual text. If the check flags an acceptable paraphrase, inspect which instruction encouraged that false alarm. If it misses a removed condition, make the requirement more explicit and test it on examples not used to revise the prompt. The measurement-design guide develops calibration and reporting beyond this first setup.

Maintain the definition and scope #

Custom measurements are owned by the organization. Their prompt, name, value type, and Allow not applicable are editable on the General tab; their conditions select individual event types and narrow them by languages, source languages, engines, and sample. Language selection follows the locale hierarchy: fr also selects fr-CA, while fr-CA does not select fr-FR. See event selection.

Save the prior definition and useful outcomes before changing what constitutes a failure. Historical evaluation uses the current definition and, when it measures a period again, replaces answers in its interval, so it cannot preserve both sides of a criterion comparison. Deleting the definition leaves earlier recorded values without that definition; turning the measurement off is the narrower action when the aim is to stop future checks.