TAUS Dynamic Quality Framework

2015/07/03

Translation quality revisited

Translation quality is one of the key concepts in the translation industry: measuring and tracking translation quality is essential for all the industry players. Unfortunately, quality measurement is still not always linked to customer satisfaction. Often, quality evaluation is the task of quality managers on the supply and demand side who make use of one specific evaluation model, ignoring the fact that several models are available for this purpose. This model is usually based on error-typologies that assign different weights to different error types. Input from customers is frequently missing or ignored and every translation receives the same treatment.

Standardizing translation quality evaluation

In an attempt to standardize translation quality evaluation TAUS developed the Dynamic Quality Framework (DQF) in 2011. The framework was co-created by over fifty companies and organizations including translation buyers, translation service providers, translation technology suppliers and academic institutions. Today practitioners continue defining requirements and best practices through events and regular meetings.

How it all started: benchmarking pilot

In Q1 2011 TAUS carried out a benchmarking exercise to review evaluation models. The exercise showed that existing QE models are relatively rigid. In most cases penalties applied and pass/fail thresholds were the same in spite of the communication parameters involved. Also, most QE models are of such a detailed nature that applying them is time-consuming and evaluation can only be done for a small sample of words. Lastly, current QE models are predicated on a static and serial model of translation production, which is not suited to the emerging models of ubiquitous computing.
DQF offers a more flexible approach to the common static QE models since it is based on the three parameters of utility, time and sentiment (UTS). The model considers the communication channel – B2C, B2B and C2C - and is informed by the results from the content profiling exercise performed by collaborating TAUS enterprise members.

TAUS Presents the DQF API and Quality Dashboard

Since 2014, DQF is part of the TAUS Evaluate platform which includes the DQF content profiling wizard, DQF knowledge base and DQF tools. The DQF Content Profiling wizard is used to help select the most appropriate QE model for specific requirements. In the knowledge base you find best practices, metrics, step-by-step guides, reference templates, and use cases. The DQF tools then allow users to do different types of evaluations like adequacy, fluency, productivity measurement, MT ranking and comparison.
 
In December 2014, TAUS started working on an API and dashboard for DQF. The release of an API for collecting translation data makes it easy for translation buyers and vendors (including freelance translators) to measure and benchmark the productivity, efficiency and quality of translation. TAUS aggregates data to offer industry benchmarking in the form of the Quality Dashboard available as a business intelligence platform.

The benefits of the TAUS Quality Dashboard

Just like other crowdshaping technologies, the TAUS Quality Dashboard will bring many benefits to users who are willing to share their translation data. It will tell them which translator to choose for a certain job, the origin of translated segments and how efficient a vendor or a technology is in a given project. And these are just some examples. All the information you need will soon be available at your fingertips by using an intelligent solution aggregating data: the TAUS Quality Dashboard. Developers and integrators are invited to use the API and connect with DQF from within their tool environments. MateCat, TRADOS Studio and Memsource are among the translation tools that will connect to the Quality Dashboard.
 
For more information on DQF, please visit our DQF web page.
 

アウトライン

翻訳業界の誰もが翻訳品質の重要性を認めますが、残念なことに品質基準が常に顧客満足につながるとは限りません。TAUSは翻訳品質評価の標準モデルを提示することを目指して、顧客企業、翻訳会社、ツール開発元、研究機関など50社を超える組織の協力を得て2011年にDQF(Dynamic Quality Framework)を開発しました。DQFはそれ以前のエラー発生頻度に基づく画一的品質基準と比較して柔軟性が高く、文書目的に適合した統計的品質評価モデルを複数のモデルから選択して用います。2014年にAPIとダッシュボードを新たに提供したことでDQFは単純な品質評価モデルからプラットフォームへと進化しました。ユーザーは使い慣れたツールからAPIを経由してDQFの評価結果を参照することにより、たとえば適切な翻訳者を選ぶ判断の参考にするなど多くのメリットを享受できます。