Thread

Hi,

I’ve tried many solutions, but therefore i have to ask for some help.
I’m upgrading from 1.6.9 to 2.1 AND changing server.
Here is my protocol.

The site is encoded in latin 1 (iso-8859-1). I export the database from the first server. Then I open the .sql dump file and save it in UTF-8 to get the right encoding.

Then I upload the new database in utf8, with the intercalssemnt collution utf8_general_ci
Everything is OK

When i applie the upgrade script everything run smooths, no errors.

The problem is that in between, all my data has been cut as soon as i get a special accent.

Exemple : if i had before upgrade : Le monde va bientôt arriver à la fin
I get after the uprgade : Le monde va bient
And all the text after the special character is erased…

I don’t know why…

Any idea ?

helain le blanc,

Is it possible for you to do the upgrade on your original server and then attempt migrating the site?
Or is the reason you are moving because of requirements?

I would not try and change encoding before upgrade. I would just try and replicate what you have on original server as much as possible.

Hello John,

Thanks for the answer, the reason of migration are various, more power, and indeed, PHP5, mysql 5 and apache 2.

I cannot apply the update on the old server, because of requirements. The old server run with php 4 and MySQL 3.23.58 and cannot be updated (production server)

I’ve tried all kinds of solutions

First with my.cnf default-character-set=latin1 for all options

- Dump and import in latin 1 without re-encoding in utf8 and collution latin1: after update : all latin characters are changed in “?”
- Dump and import in latin1 without re-encoding in utf8 and collution utf8 : after update : all latin characters are changed in “?”
- Dump and import in utf8 : all latin characters are changed in “?” before update and still after update.

Change my.cnf to default-character-set=utf8 and restart mysql

Dump and import in latin 1 without re-encoding in utf8 and collution latin 1 : after update : all latin characters are changed in “?”

Dump, re-encoding in utf8, importing in utf8 with collution utf8 : all good. Applying update i get this new problem, erasing all content after the first latin character founded.

I’m gonna try to make this last test with my.cnf back to latin1 because i did not perform this last option with re-encoding first.

Meanwhile if you get any idea of what went wrong, i’ll appreciate.

Helain.

Dump, re-encoding in utf8, importing in utf8 with collution utf8 : all good. Applying update i get this new problem, erasing all content after the first latin character founded.

Does that also happen for any new entries, or just for the imported ones? Do you’ve got a link where can see this for ourselves?

I’m gonna try to make this last test with my.cnf back to latin1 because i did not perform this last option with re-encoding first.

We’ll be here.

So the last test didn’t work, same result…

http://ns204153.ovh.net/~efap/

If you see blank screen is just because the erased charaters after L’EFAP L’ cause it’s an é

Here some exemple of one table result

And the problem doesn’t repeat in new publish article, so that’s only a database update problem…

The only solution i’ve found is to re-import data after update but with taking care of the new modifications, ans it’s a hard job. indeed the problems are in differrent tables, all that uses latin1 characters:

exp_categories
exp_channel_data
exp_channel_titles
exp_channel_fields
exp_members
exp_templates

And i’d rather not to have to do this manually, re-importing data after update works without charsets problems.. this problem appears ONLY when update applied !!!

Thanks for your help.

Hélain.

Hélain, are you seeing any sort of errors? Anything being written to your server logs? What about mod_security? Is that turned on?

Mod_security is turn to off
No errors in logs

Mysql variables are :
character set client     utf8
character set connection   utf8
character set database   utf8
character set filesystem   binary
character set results   utf8
character set server   utf8
character set system   utf8
character sets dir   /usr/share/mysql/charsets/
collation connection   utf8_general_ci
collation database   utf8_general_ci
collation server   utf8_general_ci

data base collation is set to utf8_general_ci

Dump old database in latin1, iconv in utf-8 resulting

# file dump_utf8.sql
dump_utf8.sql: UTF-8 Unicode text, with very long lines

Importing dump_utf8 in utf8 format : No errors !

Applying uprgrade script : kaboooom ! again all data is truncate after the first special character founded!!!!

I give up…. any change to skip the re-encoding process in the update-script ???

Thanks.

Well… i’ve open the ud_200.php and make in ultraedit a “find and replace”
- latin1 to utf8
- latin1_swedish_ci to utf8_general_ci

replacing this changed ud_200.php in installer folder
Applied the modified script and…. (rolling drums) : it works !!! No more truncates…

I know that’s not a good idea to change an update script… but it works…

If that could help you…

Thanks

And the good news is you never have to run it again. I know this isn’t ideal, but are new entries working as expected?

Yes all good. Thanks Sue.

Very glad to hear it. Post again as needed, please.