Thread

Ok, I just moved my database/site from the localhost off my own computer to my companies server for testing.

We are a Japanese company, and I am the ONLY person here who speaks english, so obviously we need expression engine to run smoothy in Japanese. 

I had no problem building a japanese/english on my localhost.  It worked without any problems. 

However, when I moved the database and install to the sever, all of the Japanese encoding went haywire.

This includes control panel along with the pages.  I can’t get a single character to encode correctly.

I have set the general configuration to every setting possible,  I have changed the encoding declarations on the templates and it is NOT encoding any japanese at all throughout the entire site.  the odd thing is it worked on my localhost (I am running mac os10.6)

  I have spent months building this database, and we need to get our bi-lingual page up and running ASAP… so any help you can offer to help deal with these encoding woes will be appreciated!!!  thanks!!!

Hi, jsk16. Are the databases using the same encoding?

Hi, jsk16. Are the databases using the same encoding?

yes they are.  Everything is set at UTF-8.  I even set it to UTF-8 when i transferred the data into the database. 

this is so strange :/

What version and build are you using? Do you’ve got a link?

What version and build are you using? Do you’ve got a link?

I am using 1.6.9  
and I suppose I can open the site up for a bit to let a few people see,  But I will need to turn it off because it isn’t officially released yet

The japanese page is what is having most of the problems.  you can see it here,  (but none of the images are on the server)

It looks like an encoding issue. Have you been using utf-8 locally as well? If I force Shift_JIS site looks much better. Can you check that?

It looks like an encoding issue. Have you been using utf-8 locally as well? If I force Shift_JIS site looks much better. Can you check that?

everything is set to UTF-8.  It is still messy on Shift_Jis for me.  The control panal also is not encoding right.  I have made the control panal, the field names and instructions all in Japanese.  But now it is completely unreadable. 

I am beginning to think that this is more of a server issue than and EE issue.  I just did a test on my personal webspace.  This is the exact same database and the same installation of EE , but it looks fine here

It is still messy on Shift_Jis for me.

What exactly does “messy” mean in that context? I certainly get some text resembling Japanese characters, don’t you?

I am beginning to think that this is more of a server issue than and EE issue.

Yes, me too. I have compared the headers of the two servers and they seem to be identical, however. Can you ask the host about this issues? Are they, perhaps, using a default character set or anything like that?

It is still messy on Shift_Jis for me.

What exactly does “messy” mean in that context? I certainly get some text resembling Japanese characters, don’t you?

I am beginning to think that this is more of a server issue than and EE issue.

Yes, me too. I have compared the headers of the two servers and they seem to be identical, however. Can you ask the host about this issues? Are they, perhaps, using a default character set or anything like that?

Messy in that it is still unreadable.  Shift-Jis throws it into random characters, so everything is jumbled.  It is like having a cat walk across your keyboard, which is a great way to name things in Sci-fi novels.  However we are selling clothes, not sci-fi novels

I have talked to the host and I asked them about that.  They first told me to add some lines to .htaccess to declare utf-8 for all php throughout the site. that of course did not have any affect. 

Ok, the way I understand things is that most mySql servers are generally set in UTF-8 because most of the major CMS’s require a UTF-8 connection.  I have read somewhere EE also requires the database UTF-8. 

If the sever is connected to Something else, like shift-jis, is there anything I can do to make EE work?

also, is there a shell command or something to check the default character set of the sql server? 

thanks!

jsk16, I don’t know enough about encoding to answer your additional questions, other than to note that EE2 does require UTF-8.

Will mention this thread to the dev team. Thanks in advance for your patience.

Messy in that it is still unreadable.  Shift-Jis throws it into random characters, so everything is jumbled.

I understand. Since I don’t read or write Japanese I have know way of knowing if these (seemingly Japanese) characters make any sense.

I have talked to the host and I asked them about that.  They first told me to add some lines to .htaccess to declare utf-8 for all php throughout the site. that of course did not have any affect.

No, I don’t think that would be needed. The decoding is declared properly in your templates.

I have read somewhere EE also requires the database UTF-8.

EE 2 does, should you ever decide to upgrade. That said, UTF-8 is certainly the way to go.

also, is there a shell command or something to check the default character set of the sql server?

Your host should be able to tell you, or you can use a tool like phpMyAdmin or similar to check.

But again, since it’s working on your secondary account, perhaps we should focus on what differences between these server environments there are? Are they forcing a default encoding, anything like that? What version of PHP are you running on both servers?

[quote

Your host should be able to tell you, or you can use a tool like phpMyAdmin or similar to check.

But again, since it’s working on your secondary account, perhaps we should focus on what differences between these server environments there are? Are they forcing a default encoding, anything like that? What version of PHP are you running on both servers?

Hi again!  Ok I finally got information about my company’s host server
Their set up is as follows:

Linux-2.6.30
Apache-2.2.15
mysql-5.1.45 ( set at UTF-8)
PHP-5.3.2

They also told me that their PHP default character set is ECU-JP.  I am wondering if this may be causing the encoding problems I am having. 

I checked the PHP info in the EE control panal and all the charset local values are set to UTF-8, and all the master values are set to ECU-JP.  could this be the problem??

Also I checked out the php info for both my companies server and my server.  Of course the local default_charset was UTF-8 and the master setting was EUC-JP

Also, the mbsettings were really different between the servers. 

My companies mbsettings were as follows:

Multibyte (japanese) regex support       enabled
Multibyte regex (oniguruma) backtrack check   On
Multibyte regex (oniguruma) version   4.7.1

mbstring.detect_order Local:  auto Master:  auto
mbstring.encoding_translation Local: ON Master: ON
mbstring.func_overload   Local:  7 Master 7
mbstring.http_input   Local:  pass Master: pass
mbstring.http_output Local:  pass   Master: pass
mbstring.internal_encoding Local:  no value   Master no value
mbstring.language Local:  Japanese Master: Japanese
mbstring.strict_detection Local Off Master Off
mbstring.substitute_character Local: no value Master: no value

My server was:

mbstring.detect_order Local:  no value Master:  no value
mbstring.encoding_translation Local: off Master: off
mbstring.func_overload   Local:  0 Master 0
mbstring.http_input   Local:  Auto Master Auto
mbstring.http_output Local:  pass   Master pass
mbstring.internal_encoding Local:  no value   Master no value
mbstring.language Local:  neutral Master: neutral
mbstring.strict_detection Local Off Master Off
mbstring.substitute_character Local: no value Master: no value

The mbstring.encoding_translation　mbstring.http_input 　mbstring.http_output and mbstring.language settings were different. 

Do you think this may have caused a problem with EE?

jsks16,

thank you for the detailed reply. We will review this and get back to you via this thread.

jsks16,

thank you for the detailed reply. We will review this and get back to you via this thread.

Thanks,  I found some more information which MAY be helpful in debugging this.  I checked the system variables in the SQL Manager between both the working and non working server

Here they are on the non working server
character_set_client   utf8
character_set_connection   utf8
character_set_database   utf8
character_set_filesystem   binary
character_set_results   utf8
character_set_server   utf8
character_set_system   utf8
character_sets_dir   /usr/local/mysql/share/mysql/charsets/
collation_connection   utf8_general_ci
collation_database   utf8_general_ci
collation_server       utf8_general_ci

Here they are on my working server

character_set_client   latin1
character_set_connection   latin1
character_set_database   utf8
character_set_filesystem   binary
character_set_results   latin1
character_set_server   latin1
character_set_system   utf8
character_sets_dir   /data/mysql/woge/share/mysql/charsets/
collation_connection   latin1_swedish_ci
collation_database   utf8_general_ci
collation_server   latin1_swedish_ci

I find it odd that my working server is set to a swedish_ci   The server is located in america, and I wouldn’t ever find the need of using it.

anyway, do you think the problem may be in these setting?

Thanks for your help! I hope we can get this working ASAP so my boss will be happy