Jump to content
Mark
 Share


IPS 4.0: Internationalization and Localization

One of the things we wanted to focus on for IPS Social Suite 4.0 right from the beginning was providing better support for sites which do not use English or use multiple languages (or, as it was scribbled on my whiteboard, "++ i18n/L19n"). In this blog entry I'm going to cover some of those changes and new features.


Translatable Everything

Currently when you create a forum, user group, custom profile field, etc. you have to give it a title and can only do this in one language. If you have more that one language installed, you might want to provide different titles for different languages.

In 4.0 you can do exactly that - if you have only one language installed, these fields will continue to show as normal text boxes - however, if you have more than one installed you'll see several text boxes like this:



Visual Language Editor

One feature that has been really popular in IP.Board is the Visual Skin Editor - a tool which allows you to browse your site, and click on elements to bring up a colour selector to change it. What if we could take this idea and apply it to translating as well? Allowing you to click on any word or phrase on your site and translate it there immediately. In 4.0, you can.


Easier Language Management

In addition to the visual translation we've also made several improvements to the traditional translation method:

  • As you search for a language string, results appear as you type.
  • Editing a language string saves immediately without needing to click a save button.
  • Filter tabs can show you words/phrases which have not yet been translated or the translation is out of date (meaning we've changed the default English value for the word/phrase since it was translated).

We've also made importing/exporting much faster and more reliable - no matter how large your language is (it will grow as you add more applications of course) there is now no risk of hitting an error importing/exporting (for those interested in the technical side of how this is achieved, see this blog entry).

An exported language pack will also now maintain information on the version of each application it was exported from, so that the filter which shows outdated language strings is always accurate.


Automatic Language Detection

Let's say you have Spanish and French languages installed on your site - up until now, you'd have to choose one default language, and users who want the other would have to manually choose it (which can be extremely difficult to find how to do when you're browsing a site in a foreign language).

In 4.0, we automatically examine the information that the user's browser sends (which includes their preferred language) to choose the best one out of what's available, if that user hasn't already set an explicit preference.

Pluralisation

In English, pluralisation is very simple - for most nouns, you just append "s" on the end, with some variation for certain words.

This however, isn't the case in all languages - for example, I was speaking with the owner of a site in Slovak recently who was telling me that the word "records" changes depending on the number of records there are - for 2 records, it's "2 články", but for 5 records it's "5 článkov". Currently, most language strings only have a singular and plural form (as is all that's needed in English) - meaning having the site show "2 články"/"5 článkov" was impossible.

In 4.0, we've introduced some really basic logic into language strings to accommodate this. Rather than having, for example, two language strings with the singular and the plural, there is now one with a value like this:

{# [1:record][?:records]}


The # indicates where the number will go, then each set of square brackets represents a possible value - the number before the : indicating the number which will cause that to show, and ? meaning "all other numbers".

So for our Slovak example, we'd set the value to:

{# [1:článok][5:článkov][?:články]}


On display, it will automatically show the appropriate version.


Lists

Along a similar thread to pluralisation, we've also made the way lists are formatted to be customised through a special language string. For example, a list in English looks like "one, two and three". However, in Japanese, it's "一、二、三。" (the comma symbol is different and there's no "and") - similarly Arabic, Thai and others have similar differences. In 4.0, simply by changing an example language string, this can be changed.

In the default language, this language string is:

a, b and c


For our Japanese example, we'd just change it to:

a、b、c



UTF-8

Without wanting to get into too much technical detail - UTF-8 is the most common of many ways text can be encoded for storage and display on webpages. UTF-8 has been the default encoding in our software since IP.Board 3.0.

Some sites which have been around for a long while though may not be using UTF-8. This can cause issues with some features where UTF-8 encoding is expected (for example, many features which rely on JavaScript require UTF-8 due to JSON only supporting it and nothing else). In addition, some sites may try to use UTF-8, but content is actually stored differently as the database is set to a different encoding, which can also cause issues.

In 4.0, we're going all UTF-8. If you're not already on it, the upgrader will convert data. This means a much more reliable and compatible way of handling text.

 Share

Comments

Recommended Comments



What about those that have changed to UTF-8 a few years ago and have lots of broken posts? I'm talking a few million posts.

 

The upgrader will convert non-UTF-8 sites to UTF-8.  If your site is already on UTF-8 but the content is broken, the upgrader isn't going to be able to 'fix' that for you.

Link to comment
Share on other sites

It looks like your constructive alternative suggestion after your pointless sarcastic comment got cut off, could you post it again?

 

In all seriousness, this is exactly the kind of thing that leads to a system that is considered powerful but hopelessly complicated. Especially so when there is little or no consistency from one place to the next (I don't know). Although I understand the benefit and can't see an easy alternative, it concerns me to see such things being built into the core on this rare chance to rearchitecture the system. I just hope that you're being careful with such considerations, and that IPS has some really good documentation and guidance in mind for this sort of thing. (Sorry, this blog entry doesn't count.)

Link to comment
Share on other sites

  • Management

 

In all seriousness, this is exactly the kind of thing that leads to a system that is considered powerful but hopelessly complicated. Especially so when there is little or no consistency from one place to the next (I don't know). Although I understand the benefit and can't see an easy alternative, it concerns me to see such things being built into the core on this rare chance to rearchitecture the system. I just hope that you're being careful with such considerations, and that IPS has some really good documentation and guidance in mind for this sort of thing. (Sorry, this blog entry doesn't count.)

 

There is no other parallel or similarity elsewhere in the entire system that does something like this, neither is there a natural language construct or a programming construct we can use, so we had to make one up.

 

If it helps, we had a long and very involved internal discussion about the syntax. We have long and very involved discussions about almost every minor detail so be assured that we're not just throwing out these things from our collective cuffs.

 

As always, it's worth keeping perspective; this is only a very tiny area of the language system that isn't used in the majority of text strings - but it does overcome a massive headache our translators encounter regularly. 

Link to comment
Share on other sites

 

The upgrader will convert non-UTF-8 sites to UTF-8.  If your site is already on UTF-8 but the content is broken, the upgrader isn't going to be able to 'fix' that for you.

 

What is a non UTF-8 site? All I did was switch from iso to utf in charactor conversion set, I assume you don't just go by this as my database isn't utf, it is latin swedish and I haven't been able to convert it successfully.

Link to comment
Share on other sites

 

What is a non UTF-8 site? All I did was switch from iso to utf in charactor conversion set, I assume you don't just go by this as my database isn't utf, it is latin swedish and I haven't been able to convert it successfully.

 

In that case it possibly will fix your old data. It's difficult to say for sure though at this stage.

Link to comment
Share on other sites

 

In all seriousness, this is exactly the kind of thing that leads to a system that is considered powerful but hopelessly complicated. Especially so when there is little or no consistency from one place to the next (I don't know). Although I understand the benefit and can't see an easy alternative, it concerns me to see such things being built into the core on this rare chance to rearchitecture the system. I just hope that you're being careful with such considerations, and that IPS has some really good documentation and guidance in mind for this sort of thing. (Sorry, this blog entry doesn't count.)

 

I'm not that technical, and honestly, I had the same fear as you when we started to build this. Don't worry though - it's really easy. All the different arbitrary syntaxes in 3.x bug me too, but that's not the case in 4.x. Pluralization called for a special syntax so it could work, but we aren't just making them up willy nilly - for the most part, everything else uses the simple template syntax, which we'll talk more about later. 

Link to comment
Share on other sites

 

You have to upload a language pack for anything other than English.

So there will never be an official language other than English, you should always wait for some user needs to do the translation, as at this time with version 3.4.5, I get well? Thanks Mark

Link to comment
Share on other sites

So there will never be an official language other than English, you should always wait for some user needs to do the translation, as at this time with version 3.4.5, I get well? Thanks Mark

 

Yeah, right now. We don't have the resources (i.e. people that speak other languages well enough) to provide other translations as defaults unfortunately.

Link to comment
Share on other sites

I got one more important question/suggestion.

 

- will it be possible to search for a phrase within a certain word pack instead of all of them?

 

- and how many strings per page will it have? will it be configurable?

 

basically what I'm saying is that this mod http://community.invisionpower.com/files/file/5550-acp-manage-languages-improvements/ improves translation progress to some degree and you should copy what it does if you believe what it does is usefull.

 

also a dumb question perhaps, but is Visual Language Editor a part of ips suite or is it external like visual skin editor and available only to active clients?

Link to comment
Share on other sites

 

Yeah, right now. We don't have the resources (i.e. people that speak other languages well enough) to provide other translations as defaults unfortunately.

Thanks for the reply Rikki, I try to translate as best I can IP.Borad in Italian and I expect a lot of work for version 4. Have a nice day.

Link to comment
Share on other sites

Is it worth having the UTF8 converter separate, as I guess any conversion would add a long period to any upgrade?

 

Might be good for large sites to be able to convert first.

As for the Italian language, I resolved using the unicode, to avoid the problem of the characters with accent, even in the presence of a different encoding of the database

Link to comment
Share on other sites

You have got to be kidding me! We just launched some new sub forums on our growing forum which are dedicated to International speakers. Our product happens to appeal on a global scale and we've been trying to figure out how to support multiple languages on what is currently an English dominated forum. We just had a meeting up it this evening. I pop over to IPB and find this, fantastic! You sure the spam service only tracks spammers ;) 

Link to comment
Share on other sites

I'm extremely excited about IPB 4.0 and seeing that better language functions are being implemented but I only have one concern. Will there ever be an official language pack for Japanese & Korean in the future for IPB?

It's been over 7 or 8 years (I think) since I've started using IPB and even though my site main language is English, waiting for someone to bring a language pack for these two languages has been one long wait with nothing shown or such so I'm just wondering.

 

Regardless of that, things looks to be shaping up really well with the new version. Keep it up!! ^_^
 

Link to comment
Share on other sites

I got one more important question/suggestion.

 

- will it be possible to search for a phrase within a certain word pack instead of all of them?

 

- and how many strings per page will it have? will it be configurable?

 

basically what I'm saying is that this mod http://community.invisionpower.com/files/file/5550-acp-manage-languages-improvements/ improves translation progress to some degree and you should copy what it does if you believe what it does is usefull.

 

also a dumb question perhaps, but is Visual Language Editor a part of ips suite or is it external like visual skin editor and available only to active clients?

 

The concept of "packs" doesn't really exist any more. When you search, it searches everything - but it's an AJAX search (the table updates as you type) so it's quite quick to find what you need.

 

25 per page but again, it's AJAX pagination so moving is very quick... although perhaps this is an area where infinite scrolling would be a good idea.

 

 

It's part of the suite and does not require any external services.

 

 

I'm extremely excited about IPB 4.0 and seeing that better language functions are being implemented but I only have one concern. Will there ever be an official language pack for Japanese & Korean in the future for IPB?

It's been over 7 or 8 years (I think) since I've started using IPB and even though my site main language is English, waiting for someone to bring a language pack for these two languages has been one long wait with nothing shown or such so I'm just wondering.

 

Regardless of that, things looks to be shaping up really well with the new version. Keep it up!! ^_^
 

 

There are no plans for us to distribute anything other than English, but the Marketplace is a great place to find other language packs.

Pretty much the only thing I can say in Japanese is "kagi wa teberu no ue ni arimasu" :p

Link to comment
Share on other sites

 

The concept of "packs" doesn't really exist any more. When you search, it searches everything - but it's an AJAX search (the table updates as you type) so it's quite quick to find what you need.

 

25 per page but again, it's AJAX pagination so moving is very quick... although perhaps this is an area where infinite scrolling would be a good idea.

 

infinite scrolling is okay as long as pagination still remains.

 

in regard to packs - why?, when you search everything it gives way too many results. try searching for some language strings that is found multiple times in every app and that includes admin area translation. that's frustrating.

 

at least there should be ability to search for ACP strings or public front-end strings. but overall searching within a wordpack would be great. please do not discard that idea too fast.

Link to comment
Share on other sites

We're reusing language string in the front-end and the ACP so there's no separation. If you're looking for a specific string, searching for what it is in English will bring up only that string.

Link to comment
Share on other sites




Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...