Run a localization workflow trial
Test a complete source change, correction, and recovery before depending on a localization workflow at larger scope.
A first translation can demonstrate useful output without showing whether the workflow will preserve a correction or recover a missing target. A trial examines the complete work the team intends to rely on, including the events that would make it reject the workflow.
Write those rejection conditions first. Examples include replacing reviewed text during an unrelated update, activating stale output, or being unable to explain why a required locale has no result. They turn a demonstration into a decision with evidence.
For an isolated model choice, use Compare translation models. For the quality check itself, use Measurement design.
Select representative work #
Use authorized content from the surfaces involved. Include an ambiguous term, a variable, a plural, connected prose, a consequential claim, and an update that should preserve unrelated accepted text. Cover the important formats and locale classes; a JSON example cannot establish Apple catalog or ARB behavior.
Retain source and target revisions, accepted baselines, localization-engine and model configuration, guidance, format options, trigger, and review policy. When comparing workflows, keep the sample and requirements the same. Existing accepted translations are useful evidence with provenance, not an automatically error-free reference.
Define what completion means at each stage #
| Stage | Evidence |
|---|---|
| Intake | The intended source revision and scope were selected. |
| Execution | Run identity, actual configuration, status, and failed work are known. |
| Output | Required locales have retrieved valid resources or explicit unresolved outcomes. |
| Review | Qualified reviewers can see the source, target, and relevant guidance and return decisions. |
| Correction | Accepted edits survive; proposed shared guidance is decided separately. |
| Consumption | The application uses current accepted output without concealing missing locales through fallback. |
| Recovery | Interrupted work can be handled without broad duplication or destructive resets. |
Choose a concrete completion criterion for each. A native app needs resource generation and representative rendering. Application records need revision checks and correct per-locale reads. Implement the steps through the actual tool references.
Exercise a correction and an interruption #
Run a first translation, then edit one source message and verify preservation of unrelated accepted targets where reuse is supported. In a safe local fixture, introduce an invalid variable or resource construct and check the consuming build. Simulate interrupted retrieval or a failed target without creating production errors solely for the trial.
Take one language defect through evidence inspection, correction, and a bounded rerun. If it generates shared-guidance suggestions, test applicability and a counterexample before approving them. Saving guidance does not train the model or necessarily rerun existing content.
Record what actually executed, what was source-inspected, and what remained a planned scenario. An unrun recovery case is useful preparation but is not a passing result.
Examine quality and unresolved work together #
Report quality with its denominator and coverage. Inspect model/fallback use, retrieval, reviewer corrections, and known cost. A dashboard change can reflect different content, language mix, or policy rather than improvement caused by the workflow.
Decide the next permitted scope #
Summarize accepted results, failed rejection conditions, untested paths, and unresolved dependencies. A successful trial for two locales supports a bounded expansion, not a guarantee for all languages and formats.
Repair lifecycle gaps through pipeline design and unresolved acceptance through review policy. Retain the trial's inputs and results so the next operator can understand and reproduce the decision.