Weblations knowledge bits

Sniff your web

Robert — Fri, 25 Feb 2005 18:41:00 +0000

Before you publish any web content in a fancy character set, be sure to sniff your web. No, it shouldn't smell. Sniff for your web server's HTTP response header.

The problem
If there is a clash between the HTTP header and the character set of the page you are publishing, most browsers will try to render the page according to the HTTP header. The result can be gibberish.

Here for example is a Japanese page, encoded in UTF-8, being delivered by a server that claims it is encoded in ISO-8859-1, the encoding for standard Western European languages. Not a pretty site!

We were hoping to see something like this. What happened?

Background
When a browser renders a web page, it needs to know what character set the page is encoded in. There are three ways it can find out:

Read the META tag in the header of the file,

study the text on the page and guess, or

read the web server's HTTP response header.

Before going any further, let's be clear: the file header mentioned in method 1 is the stuff between the <HEAD> and </HEAD> tag at the beginning of an HTML file. The HTTP response header of method 3 is the data that the server sends to your browser before it begins to send the HTML file. You can't see it without the aid of a web sniffer.

Method 3 is the least widely known and the most powerful. It's the gotcha method.

The <META> tag of the first method is the most popular, with good reason, because it allows you to specify (and change) your encoding from one file to the next. Here is the meta tag for UTF-8:

<meta http-equiv="Content-Type" content="text/html;charset=utf-8">

I copied this tag from the header of the Japanese page above. Unfortunately, its server was singing a different tune.

Web sniffing
Here's where web sniffing comes in. Got to http://www.web-sniffer.net, enter the URL of the website you are sniffing, and look at the results...

The culprit is in the last line: "charset=ISO-8859-1". That's a direct conflict with the UTF-8 encoding that we actually intended.

Solution
Should you run down the hall to tell your webmaster to change the HTTP header to your new encoding? No! To do so would render your pages correctly while breaking those of everyone else.

If your webmaster is extremely kind and patient (possibly True) and has a lot of extra time (definitely False), you could ask him or her to set a special encoding for your folders. But why bother. The easier and better practice is not to specify any charset at all in the HTTP header. It should say simply "text/html", so that the encoding of each page can vary according to the capricho of its author.

ClearType reduces eye fatigue

Robert — Wed, 02 Feb 2005 09:40:00 +0000

Ten years ago, in the epoch of flickering 640 x 480 screens, conscientious proofreaders would print out their texts and work on paper. Errors that were invisible on the screen seemed to jump out from the page. But printing involved an extra step. It was tempting to be lazy.

Now, screens have improved. Most people review text on the screen directly to save time and benefit from improved tools for spell checking, revision tracking, glossaries and so on. But eye fatigue is still a factor, and may have suddenly worsened thanks to flat LCD screens, which sharpen the edges around each pixel and render type with a choppier look than did the old CRT screens.

To solve all that, Microsoft has released a great little tool especially useful for flat screens, the ClearType Tuner. According to the download page: "This PowerToy lets you use ClearType technology to make it easier to read text on your screen, and installs in the Control Panel for easy access." (Windows XP only.)

Give it a try. The result is frankly luxurious, a whole lot easier on the eyes. You'll feel like you paid an extra couple of hundred bucks for your display.
Here's an example with ClearType on the left and standard type on the right.

Ditch the RFP, and other translation outsourcing t...

Robert — Wed, 16 Feb 2005 18:07:00 +0000

A lot of people who need to buy a website translation have never bought one before. If you fall in this category, read this posting. It includes advice that can make you a tougher and more informed buyer, written by someone (me) who has written or reviewed more than 1,600 translation proposals over the last few years.

Insist on an itemized quote. Ask each potential vendor to itemize and price every service that they will deliver. You should be satisfied that you really understand what your are buying, and what factors affect each line item. For example, some services are priced according to the word count while others vary according to the technology that your website uses.
Understand the word count. The basic variable in a website translation proposal is, or should be, the word count. It takes time to translate words carefully. One way or the other, you're going to have to pay for that time. But you shouldn't have to pay as much to translate repeated strings of text -- a very common characteristic of websites. In a transparently written proposal, both the raw word count and the corrected word count after subtracting some portion of the repetitions will be clearly displayed.

Once you understand the word count, insist that if the word count of your project varies between the bidding and production phases, that the final price vary proportionately, either up or down.
Negotiate your maintenance agreement up front. Maintaining a multilingual website can cost more than translating it in the first place. If you plan to maintain your site for the long haul, be sure to discuss the options and negotiate the price up front while you still have the negotiating leverage. A common pricing strategy (to use a euphemism for "trick"), is to buy your account by under bidding on the initial project, then ratcheting up the fees later when you are locked in.
Don't get locked in. Speaking of which, you should be aware of two fundamental ways that a translation service provider can gain leverage as your incumbent. Both strategies are not adversarial tricks. On the contrary, when used with your interests in mind, they can be beneficial to you.

The first power factor in the client-incumbent relationship is translation memory (TM), a software technology that sophisticated translators use to remember and reuse everything that they translated for you. If your website contains a lot of repeated descriptions of similar products and such, then with time, your incumbent translator will gain a pricing advantage over all other competitors, making it prohibitively expensive for you to change. If the pricing advantage is passed on to you, then fine, but if it is used to increase your vendor's profit margin and tactical pricing ability, then you lose. The solution? I'll discuss it in a separate posting.

The second power factor is any kind of multilingual content management software that the vendor sells and installs on your server. These systems are also known as global management systems, GMS. Granted, certain sections of most websites can benefit from structured, multilingual content management. But not all of the content on all websites, and not if the license costs all of the money in the world. The downside to these systems, in addition to their cost and complexity, is the leverage that they give to the system vendor.
Ask for the name of, and interview, your project manager. Your most contact at the vendor is the project manager that they assign to your account. If you are being sold by an account representative, respect the work that he or she is doing for you during the selling process, but ask to speak to the project manager. The P.M. will take the reins, build your project team, set the calendar, handle your inevitable change order requests, manage the quality control process and all the file transfers. If you like the qualifications, profile, experience and personality of the designated project manager, you will be in good hands.
Ask for the names, resumes and work locations of your linguists. Most translation vendors justifiably make a lot of noise about using native, professional linguists who are resident in the target language country. They may also claim to use more than just a single translator, and thusly to differentiate themselves as an agency from a less expensive freelance translator.

Linguists are the heart and quality of the service that you are purchasing. So it makes sense to have the vendors show you the résumés of your linguists. They should also be able show that your documents have been edited, reviewed or proofread by someone in addition to the original translator -- if that is what they claim to do.
Compare apples to apples. Price and service comparisons for a complex solution have to be compared with care. A website translation can range greatly in completeness, or lack of it. At one extreme are the agencies that simply translate a Word document for you and leave it to you to paste the translation into your HTML code, database or whatever. At the other extreme are agencies (like mine) that offer a lot of very helpful consulting during the planning phase, work on a turnkey basis with native assets and do a conscientious and systematic job of quality control. The price for a complete service can be higher while the true cost to you is significantly lower.
Ditch the RFP. Last but not least, if you are about to write a Request for Proposal (RFP), think the better of it. Did you know that less than 10% of all RFPs for website translations result in actual projects? The very fact that someone in your organization is asking for an RFP should be a warning to you that the project is going to be studied and tabled, forever. Save yourself the trouble. If you are forced to write an RFP, let me know; I could be persuaded to blog about what every RFP should and should not contain.

Welcome to Knowledge Bits

Robert — Sat, 01 Jan 2005 17:10:00 +0000

I'm Robert Hopkins, the president of Weblations. Since early 1996, when I founded Weblations in Barcelona, my colleagues and I have been translating and localizing websites -- some 600 projects in all. We've learned a lot over the years, and developed some opinions as well. In this blog, we'll share some of the knowledge bits that we've picked up over the years. If you are running your own translation project, we hope that they make your life easier. If you're looking for an agency to outsource your project to, this blog will give you an insider's look at the issues we face on a daily basis, and hence the kind of skills you should look for.

To translate a website, you and your team need to know about culture and languages, copy writing, the technologies used by the website that you are translating, graphic design and the tools that you use to do your work. In our case, when we started the company, there were no adequate tools to assist a professional, high volume website translation team, so we decided to develop our own. Fortunately, the Web wasn't very complicated back then, and we were able to put something together in relatively short order. Since then, as the Web has enriched, we have enriched our two core applications, Weblations Cypher and the Weblations Workspace. You'll hear a lot about them in this blog, I bet, since developing them takes up a big part of my working day.

I didn't really start this blog on New Year's Day, but I will pretend I did to give me some breathing room and to remind me that this is a New Year's resolution to be honored.

Enough of introductions...on to the blog!

BOM -- Huh?

Robert — Tue, 11 Jan 2005 16:31:00 +0000

Every now and then, we wake up to an ugly technical surprise usually just before we're supposed to deliver a project to a client. One of the more memorable ones that has had a lot of repercussions was the day we met our first BOM.

What's a BOM?
I'm embarrassed to admit it now, but there was a time when I didn't know what a BOM was. Had I Googled on it, I would have seen it just a few entries below the Australian Bureau of Meteorology, but Google isn't much help when you don't know the name of what you're looking for. (Aside: why did the intelligent scientists at this bureau incorrectly elevate the status of "of" to the "O" in BOM? Is it that they don't want to be known as the "BM?")

According to the BOM FAQ on the Unicode home page, BOM is short for byte order mark, three fancy characters that you might find at the beginning of a UTF-8 file. (UTF-8 is a compact but universal character set that we use a lot for multilingual websites. This blog is published in UTF-8. More about that in another posting.) Ironically, UTF-8 comes in only one order, so a BOM doesn't matter in the way it does with full-bore Unicode.

The problem with the BOM is that some text editors interpret and hide the characters. You don't know that they're there. Here's what they look like:

ï»¿

These babies mean "You are about to see some UTF-8!" Are they necessary? Usually not, because a well-formed web page should make that announcement anyway with a META tag. Are they dangerous? You bet!

Notepad for WindowsXP and the BOM
If you're like us and you want to review and approve every single byte that you deliver to a customer, an unexpected BOM can play havoc with your plans, with your and your tools' concept of how long the file is, and so on.

We learned about the BOM at the same time that we were upgrading to WindowsXP from Win98. One of the many gotchas was seeing our humble Win98 Notepad be replaced by Notepad for WindowsXP, a much more take-charge program. The old Notepad shows you the BOM characters, while the new one does not! With the new Notepad, if you save a file and select at the UTF-8 as the encoding in the Save As dialog box, the BOM will be pre-pended to your file, an unnecessary but seemingly harmless little detail.

But is it really that harmless?

Let's look closely at the definition of a BOM:

BOM. The character code U+FEFF at the beginning of a data stream, where it can be used as a signature defining the byte order and encoding form, primarily of unmarked plaintext files.

The operative term is "beginning." How can we be sure that a given text file that has been saved in Notepad, will be the beginning of a data stream? It may the footer of a web page, deployed with an include statement to the middle of another file. Now, unfortunately, our BOM has gone from the beginning of one file to the middle of another, and in that position it can do a lot of damage.

Another ugly place for a BOM to land is in a database entry, where it can ruin your best laid plans for running exact searches and concatenating strings.

Forewarned is forearmed. Hope this helps.

Naming files in a multilingual site

Robert — Sun, 16 Jan 2005 15:31:00 +0000

If you plan on translating your website to several languages, one of your first issues will be devising a naming scheme for the translated files and folders. I have some advice for you.

Don't change the file names! Don't change the folder names!

Instead, put the translated set of files in a new parent folder named for the target language, e.g., /ger for German. All the original files and folders (now translated to German, of course) should branch out from here. That way, all of your relative links from file to file should work without modification.

The above illustration shows a well-structured trilingual website in English, German and Spanish. The site was originally designed in English to contain three folders: /images, /press and /products. At the time, no one thought the site would be translated, so a folder for English was never contemplated, but that's OK.

Later, German and Spanish were added under the folders /ger and /spa. Below these language folders, the structure of the original site is replicated. The names of the HTML and image files remain exactly the same so that the relative links don't need to be touched. That's it.

PDF files
I wrote this very basic posting to mention an exception that proves the rule: PDF file names. PDF files should incorporate their language in the file name itself, e.g., brochure_eng.pdf and brochure_spa.pdf. Why? Because PDF files have a life beyond the website where they are stored, as attachments to emails and as loose files on someone's hard disk. The file name should show the language that the file is written in. Furthermore, if the brochure is written in three languages under the same name and you want to attach all three versions to the same email message, you'll have a nasty little naming conflict.

If you want your website to sound like a robot...

Robert — Wed, 05 Jan 2005 17:59:00 +0000

...then use a machine to translate it!

You can use software to translate any kind of text. There are several sites that will do it for free. Perhaps the best are run by Systran and Google.

As a demonstration of the state of the art of computational science, as a personal tool or as a novelty item, machine translation is wonderful, but the quality of the result isn't reliable enough to use for production purposes. It's one thing for an end user to say, "I need to understand the gist of this article about my company in a German newspaper." It's another thing altogether for a company to say, "Let's use a machine to translate our web site." In the former case, the user knowingly takes the risk of reading a botched or awkward translation. In the latter, users read a corporate website, unaware of the circumstances behind its translation. Their critical defenses are lowered, they're vulnerable to any kind of misinterpretation.

How good is machine translation?
Machine translation is surprisingly good, but in the end it's not good enough.

Look what Systran's engine does to the following sentence translated from English to Swedish and (for those of us who don't speak Swedish), back to English. I chose Swedish because it was at the top of their list.

English original: I took my daughter for a walk in the park.
Swedish translation: Jag tog den min dottern för en gå i parkera.
English back translation: I took that my daughter for a to go in parking.

Admittedly, this is not a fair test, because the back translation tends to magnify the errors in the first translation. The translation to Swedish is probably only half as bad as the back translation to English, but half of horrible is not good enough.

What good is machine translation?
We've already described the most popular use of MT -- to allow end users to selectively and instantly understand the gist of an otherwise unintelligible document, especially on the Web, where the content and the tool are both simultaneously available. But this is largely an underground activity. It can't be controlled by the publisher of a website and it doesn't appear on any ledger as a cost savings.

MT could save some serious money for a web publisher in one discrete area: the translation of a corporate knowledge base. KB's cost a lot to create and maintain in English, and are rarely translated to other languages. They utilize a constrained terminology and are written by a small group of expert authors who are trained to write in a precise and clear way. It wouldn't take much extra effort to condition their writing to be translated by a machine translation program armed with the corporate terminology.

The resulting translated KB should carry an armor-plated disclaimer to the effect that the translation is not guaranteed to be any good at all, and should allow the user to read the original English article. Why not give it a try? Many of the non-American readers of KBs are technical trained professionals who are nominally bilingual of necessity, but would find it much easier to scan several articles in their native language before finding and studying the one that matters.

That's all for now. I have to take my daughter for a to go in parking.