ROI and the Value of Machine Translation
Presented by Nicholas McMahon Moderated by Paula Shannon July 15, 2009
Introductions Nicholas McMahon VP, Global Technology Solutions, Lionbridge Nicholas.McMahon@lionbridge.com Over 13 years experience working with some of the world’s most successful global organizations Provided strategic and tactical global leadership as a foundational executive for two US divisions of European companies, as well as a VAR business
Paula Shannon CSO, SVP and General Manager, Lionbridge Paula.Shannon@lionbridge.com More than 23 years experience in the translation and localization industry Responsible for sites in 26+ countries, driving new services and sustainable solutions, ensuring the continued delivery of innovation and execution excellence to a broad range of Global 1000 customers
Company confidential – distribution prohibited without permission
About Lionbridge
Global Scale
Market Leadership
4,600 employees 26 countries
Leader in $14B services industry • Translation and adaptation of products
• Global network of 25,000 translators
and content for international markets
Hosted Technology
Global Clients
Web-based language technology platform
Recurring relationships with 500+ global clients
• Enhances competitive advantage • Drives efficiency
Company confidential – distribution prohibited without permission
• 80% of revenue comes from recurring clients • 12 of the Fortune 20 companies are client
Agenda
Basic MT Introduction 3 Basic Considerations for MT Evaluation
Top 3 Applications of MT Enterprise Cost Considerations for MT
Business considerations and ROI Modeling Live Q&A
Company confidential – distribution prohibited without permission
Basic MT Introduction
Company confidential – distribution prohibited without permission
Three Types Of Machine Translation
Rules Based iTranslator Statistical Google Hybrid Systran Enterprise Server 7 Company confidential – distribution prohibited without permission
Basic MT Introduction
Company confidential – distribution prohibited without permission
Agenda
3 Basic Considerations for MT Evaluation
Company confidential – distribution prohibited without permission
Top Three Considerations for MT
MT performance in one language is NOT MT performance in another language or even in another language direction
Company confidential – distribution prohibited without permission
Top Three Considerations for MT
MT is not an “every language solution” for any of the MT Engines
Company confidential – distribution prohibited without permission
Top Three Considerations for MT
MT at an Enterprise Level is NOT free
Company confidential – distribution prohibited without permission
Agenda
Top 3 Applications of MT
Company confidential – distribution prohibited without permission
Top Three Applications of MT: Chat
Company confidential – distribution prohibited without permission
Top Three Applications of MT: Chat MT Chat Approaches Human to human instant dialogue Human to semi auto response based on key word findings
Evolution Early adoption – Value focus, very little distinct ‘ROI’ modeling Company confidential – distribution prohibited without permission
Top Three Applications of MT: Support Content
Company confidential – distribution prohibited without permission
Top Three Applications of MT: Support Content FAQ MT Approaches Raw MT output for untranslated content Tiered Quality approach
Evolution Volume Success and reasonable adoption “Self serve” & “New content” increases user patience
Company confidential – distribution prohibited without permission
Top Three Applications of MT: User Guides
Company confidential – distribution prohibited without permission
Top 3 Applications of MT Approaches TM, MT + Post Editing of content Considerations Early Adoption – strong indicators of success Goal is to MT to be ‘invisible’ in process
Company confidential – distribution prohibited without permission
POLLING QUESTION Are you currently considering or using MT in any of these situations? Considering MT for Chat Considering MT for FAQs Considering MT for Volume User Content Using MT for Chat Using MT for FAQs Using MT for Volume User Content
Company confidential – distribution prohibited without permission
Agenda
Enterprise Cost Considerations for MT
Company confidential – distribution prohibited without permission
Enterprise Cost Considerations “Start up Costs” Costs Data Identification/ Corpus creation $0k - $100k Customization or training of content $5k - $250k Licensing fees $0k - $200k
Integration fees $10k - $200k
Company confidential – distribution prohibited without permission
Enterprise Cost considerations “Post Editing” Costs
Training of resources for post editing $5k - $10k Post editing rates 40% – 65% of word rate approx 10 cents – 16 cents per word Ongoing customization/ corpus management annually $5k - $200k
Company confidential – distribution prohibited without permission
Agenda
Business considerations and ROI Modeling
Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication User 1
User 2
My Second figure is an animal with a tail and four feet
My role of a young handsome beau is a boy with a tail and 4 feet
What kind of meaning and nice boy?
What do you mean? A handsome boy?
My second figures looks like an animal
My second figure is like an animal
It has four feet and a tail
It has four feet and a tail
Oh, I understand and am the 8th I got it, it is my 8th figure Source: NTT Labs and Kyoto University, Effects of Machine Translation on Collaborative Work, Dec 06 Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication Establishing the value of instant communications
Ability to resolve support requests Cost per multilingual call handling Ability to communicate better Value of increased time to market
Company confidential – distribution prohibited without permission
ROI Modeling for MT within Instant Communication MT in Chat Benefits Variables Strong defined reference sets produce better results Strong defined language use (specific to a domain or industry) Patience and Flexibility of user (not for complaints!) Favors Value of ‘currency’ of content versus quality
Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication Results Defined Data Sets 5 - 25% average success rate (resolution of issue) User Satisfaction generally 30 – 50%
Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication ROI Model – Statistical MT considered best match 10 Languages, 100,000 chat support requests Alignment of data for Corpus Creation: Product Licenses Integration of tools Human system/ process training System tweaking :
$30k $100k $20k $10k $20k
Total Cost
$180k
Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication Averaged Results for Calculation 10% support cases resolved Average cost of alternate solution: $15 (FAQ routing/ human Call) Benefit 100,000 * 10% * $15
$150,000
Less Costs
$180,000
Basic Financial ROI Year 1
-$30,000
Company confidential – distribution prohibited without permission
POLLING QUESTION Based on the considered MT chat model do you think the suggested ROI is: Very Low Low About Right High Very High
Company confidential – distribution prohibited without permission
ROI modeling for MT applied to FAQs Establishing the value of FAQs? Ability of users to autonomously resolve support requests Increased Customer Satisfaction
Company confidential – distribution prohibited without permission
ROI modeling for MT applied to FAQs Benefits Variables Strong defined reference sets produce better results Quality of source content construction Expectations of user – disclaimers on quality Favors un-translated content Rules based or Hybrid best solutions Provision of English source links provide additional context Cultural expectations Volume of source data available Company confidential – distribution prohibited without permission
ROI modeling for MT applied to FAQs Results Defined Data Sets 20% - 35% average success rate (resolution of issue) On average about 8% less than English
User Satisfaction generally 50 – 65%
Company confidential – distribution prohibited without permission
ROI modeling for MT applied to FAQs ROI Model – Hybrid MT Engine considered 10 Languages, 100,000 FAQs docs 450 words per doc. Alignment of data for Corpus Creation: Product Licenses Integration of tools/ solution Rating, feedback reporting Human system/ process training System tweaking :
$50k $250k $60k $20k $10k $50k
Total Cost
$390k
Company confidential – distribution prohibited without permission
ROI modeling for MT applied to FAQs Averaged Results for Calculation : 25% support cases resolved Average cost of alternate solution: $35(Human Call Response) Benefit 100,000* 25% * $35 =
$875,000
Less Costs
$390000
Basic Financial ROI Year 1
$485,000
Company confidential – distribution prohibited without permission
POLLING QUESTION Based on the considered MT FAQ model do you think the suggested ROI is: Very Low Low About Right High Very High
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides Establishing the value of MT within User Documentation Improvement in output volume Reduction in total attracted cost of translation Increased user content available
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides Benefits Variables Strong defined reference sets produce better results Quality of source content construction Knowledge and skills of human resources Systems and tools integration of MT and TM Target Language complexity
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides Results Defined Data Sets -10% - +25% productivity improvement Reduction in total attracted cost per word
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides ROI Model – Rules Based MT considered 4 Languages, 2,000,000 words total Alignment of data for Corpus Creation: Product Licenses Human system/ process training System tweaking :
$5k $20k $20k $5k
Total Cost
$50k
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides Averaged Results 15 % increased in productivity Average fully loaded per word cost (accounting for fuzzy matches)
$0.25
Benefit 2,000,000 * 15%* 0.25 =
$75,750
Less Costs
$50,000
Basic Financial ROI Year one (about 10% of expected costs)
$25,750
Company confidential – distribution prohibited without permission
MT ROI Take Away
Early Adoption, but MT based projects are reporting MT solutions are dramatically improving MT source data is growing exponentially
Company confidential – distribution prohibited without permission
Agenda
Live Q&A
Company confidential – distribution prohibited without permission
Knowledge Center
www.lionbridge.com Download White Papers • Building Stronger Brands Around the World: A Guide to Effective Global Marketing • Strengthening Global Brands: Key Steps for Meaningful Communications around the World • Building a Global Web Strategy: Best Practices for Developing your International Online Brand • Machine Translation: Saving Time, Lowering Costs, and Improving Services
View Webinars • Localization 2.0 • The Art and Science of Global Navigation • The Best Global Web Sites (and Why)
Company confidential – distribution prohibited without permission
Questions?
Nicholas McMahon VP, Global Technology Solutions Nicholas.McMahon@lionbridge.com
http://localization2dot0.lionbridge.com http://blog.lionbridge.com http://twitter.com/Lionbridge
Lionbridge www.lionbridge.com
Company confidential – distribution prohibited without permission