Thread

I’m building a site that will provide info in a bunch of exotic languages like Arabic, Thai, Russian, etc… I have limited knowledge myself regarding possible issues in dealing with the document encoding in those languages. The back end can stay in english… But The client will post translations of articles in many different languages… Is there any type of limitation on EE database hand for handling these character types?

I’ll also have to figure out how to switch language direction on the spot for arabic… but I digress… 😉

Sorry if it’s somewhat of a vague question… I need to do my homework on the subject… But I’d like to know in the meantime if EE can handle it.

Thanks!

Well, I tried something completely practical, rather than speculation, as EE ‘should’ do well with your request.  The base character setting is utf8, which should handle most languages in the world.

- first I tried Chinese, the Simplified variant that is the language of continental China.  This, with mixed English as an entry from a friend’s website, looked fine.  I can read only a tiny bit of it, but am familiar with the characters, and these look to be all correct.  The interspersed English was fine also.

- then I tried Korean, borrowing from newspapers there.  Here the body text looked fine, and I do read a bit of it, as well as being quite familiar with the Korean writing.  There was trouble, though, with the article titles.  Something goes wrong there, and cuts off the end character or two.  I suspect the ‘punctuation clearing’ etc. that’s done to clean up EE article titles.  Such things brought to the attention of staff usually result in remarkably quick bug fixes - that’s one of the strengths and uniquenesses of the EE community.

- there’s a general issue also for titling, but that is very simple.  EE makes up a ‘short title’ when you are editing the main article title, which is used as the url name for the article.  This automatic creation of the short title doesn’t work with non-Roman scripts, but it’s right there in the form for you, and you can just type in a short non-accented name for the article.  This ‘short title’ anyway wants to be roman and non-accented, to assure compatibility with web servers and browsers out there in a wide world.

In short, it looks pretty good, and I don’t think you’ll have trouble convincing the EE people to clean up whatever is sometimes interfering with the title, if you give them an example or two to work with.  It might be that an option just needs to be added to defeat the cleaning-up, for such cases, if they can’t work out what is going on to alter higher-address characters.  But I suspect they can actually fix the root difficulty rather easily.

On this aspect, though, you’ll need to hear from them.  I think their intention is always to have a very clean as well as powerful ExpressionEngine.

Kind regards,
Clive

p.s.  You can try all this out with your languages yourself, by downloading the EE Core, which is free.

This is a good idea anyway, as it is simple to install (but not too simple), and will give you a direct experience in how you like the EE style.

Regards again, and I will report the Korean title bug,
Clive

thanks a bunch for the detailed report!

I have EE installed locally, but I actually don’t know exactly yet all the languages I will have to deal with on this project… That’s why I wanted to get the heads as much as possible… It sounds like most likely it should be fine though… The short titles/links can be in english… not a problem.

Thanks again.

Any language you can display with HTML you can display with EE.

You’re most welcome, minimal-design.

As well, I suddenly had an insight, and tried according to it, and the EE bug with Korean titles is apparently pretty simple and easy to get around until a better solution.

It appears the problem is just truncation itself.  The insight was that even though Korean looks like ‘Chinese characters’, it is actually written with an alphabet, and the letters positioned in clusters. 

So a short-looking Korean title is actually a lot of individual, wide characters.  The EE title-shortening automation then cuts in, and if you are unlucky, breaks apart one of these characters or clusters.  I verified it by adding or subtracting spaces, to move the cut point, and the proper Korean text appeared or disappeared, sometimes with a little nonsense shown.

I’m not sure if the fix is simply to assure full-width Unicode truncations, or something like breaking only on spaces.  Spaces is likely the right solution, actually, and in truth it will make for a very simple fix.

I’ll put that into the bug report.

Kind regards, and hope it works out for you,
Clive

thanks a bunch for the detailed report!

I have EE installed locally, but I actually don’t know exactly yet all the languages I will have to deal with on this project… That’s why I wanted to get the heads as much as possible… It sounds like most likely it should be fine though… The short titles/links can be in english… not a problem.

Thanks again.

p.s. again -

I got a little tied up in describing the problem at the last, but your own solution is very simple.

If you see the end of a title getting messed up in a given language, just reduce the length of the actual article title you enter. 

Then it will fit without needing to be shortened, and there will be no problems seen, even before they make a fix.

Regards again,
Clive

narration- you might want to try creating a foreign language title field. That’s how we handle the title issues for various post. We have the post title in English as the primary title, then the foreign_language_title. On the template, we just use a simple {if} {if:else} statement. If there’s data in foriegn_language_title display that else display title.  The other thing we do for our foreign language articles is also have a summary field in both english and the language the article is submitted as.

We’ve found both to be very beneficial for things like: insuring that foreign language posts get displayed in search results when appropriate.

Thanks, allgood2, and that’s a good idea, for all your reasons.

It’s not going to help this truncation issue, I think, and we are working on that at the moment, but a nice demonstration of EE’s flexibility to make things nice.

Best regards,
Clive

Sorry that doesn’t help you. It’s what we did when we notice titles in korean, japanese, and tagalog would occasionally get screwed up; but the issue was just on the titles.