To take this to the next level I loaded the sister application in our family of applications, Open-Realty.

Reset O-R to UTF8 and created two entries (city = Île-de-France & city = Franche-Comté).

Looking in the DB I see identical glyphs for Î and é and using Ingmar’s SQL script I get the same sort-order result (based on glyph order).

So both applications are communicating with the back-end DB in the same way and producing identically shoddy results.

2 positives; i) it’s not unique to EE; and ii) the fix should improve both applications.

1 negative; this issue is not specific to EngineHosting, we have the same issue on the DS.

Interestingly when I run the same test on mySQL5 I get errors like “Warning: cannot yet handle MBCS in html_entity_decode()!” Admittedly this is not an EE error but if the automagical stuff is intended to do entity replication/replacement that will go back to the heart of the issue - I can make EE work in French using ISO-8859-1 and I can make EEWiki work in French using UTF8 but there’s no way in the world to make them both work together - until someone can actually prove me wrong

jiF

Ingmar Greil - 01 July 2008 01:46 AM

Jules In France - 30 June 2008 04:34 PM

Aha - you cheated!

What exactly do you mean? I have a few entries with high ASCII characters and have EE sort them, is that not what we are after?

Here’s the code I whipped up quickly:

<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
{exp:weblog:entries orderby="{segment_3}" sort="asc" entry_id="197|195|196|198|199|200" dynamic="off" }
{title}<br />
{/exp:weblog:entries}

Fair enough? Here’s the result. If I change the orderby parameter to, say, “date”, the order of entries changes as well. So unless I completely misunderstood you, I don’t see where I should have “cheated” here.

A false proof! Your list on page looks like this;

Ärger
Île de Fleurön
Österreich
Überkonto
Zachä
Zachö

Mine looks the same, because Î appears as AZ in my database therefor in an alpha-sort comes after Ä

You need to add a value between A & I to prove your point.

.....and doesn’t {segment_3} read a different field in the DB........

jiF

We’re looking into this internally. I can reproduce that behavior to some degree.

Ingmar Greil - 02 July 2008 02:36 PM

We’re looking into this internally. I can reproduce that behavior to some degree.

Thank you; you could liaise with EngineHosting as I have set up accounts with these errors on their servers and opened a ticket with them too. Please PM if you would like the account number.

I found this though I am not sure what it means;

http://bugs.mysql.com/bug.php?id=33791

and;

http://www.php.net/manual/en/intro.mbstring.php

jiF

Edited to add link

Jules, my apologies for this one lingering so long without a resolution.  The development crew put in a good deal of work on the issue and 2.0 may be handling some of these character set issues differently.

For now, are you still stuck on this?  At the moment, it may boil down to a mysql limitation.

Thank you for your observations Robin.

In this particular case we moved to a different application; not as elegant but more effective.

I look forward to v2 and the opportunity to revisit the decision.

jiF

Thanks Jules, sorry EE did not work for you in this instance.