Thread

Oh noes! I’m having the same problem. After moving to a new server, text encoding issues galore. Except, I used the EE backup utility for the transfer. This stuff befuddles me, so please help.

The dreaded question mark symbol now shows up for all quotes and apostrophes. An example:
this article. Even when I view the source, quotes still display in the view source window as ?s, using firefox on both windows and mac.

You’ll notice in the source that charset=utf-8 in a meta tag. Utf-8 was also set as the default character set in the EE CP under general configuration on the old server. (I cannot access the CP on the new server yet, but that’s a whole other thing). I presume the setting has remained.

But it gets better. If you do a search, the character errors do not show up. On the search page, no charset is set by the template’s XHTML, and the article excerpts appear to work fine. Click here to see. Now, the links on this search page still incorrectly point to the article on the old server. But I believe it is pulling the excerpts from the database on the new surver. (Again, can’t get to the new cp right now.) Though I could be wrong.

In phpMyAdmin (which I’m really not familiar with), the SQL database appears to be using utf8 unicode.

Taking JT’s suggestion, I uploaded a file html file with only © - and it showed up fine. I don’t want to use Derek’s export/import data method yet, it sounds too difficult and risky to try without looking for another solution.

Whew. So, any ideas? Did I do something wrong on the transfer? Some setting I need to change? Could re-doing the transfer differently solve it?

When I switch encoding to ISO (overriding the meta tag), the character set issues are cleared up.  I suspect that’s why the search, without a character set specified, shows ok.  Except the search wasn’t working for me- think there’s no index.php in the links and they haven’t been removed via htaccess on the new site.

Why this would have changed, I’m not sure.

Hm- what version of mysql were you running on the old server?  How about the new one?  Pull up your exported file and take a look at the first table- exp_actions- mine looks like:

CREATE TABLE `exp_actions` (
  `action_id` int(4) unsigned NOT NULL auto_increment,
  `class` varchar(50) NOT NULL default '',
  `method` varchar(50) NOT NULL default '',
  PRIMARY KEY  (`action_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

The character set may/may not be there, depending on version.

The new server is running MySQL 4.1.21-standard. The old server I can’t get to right now (though I can find out tomorrow). I’m betting that it was and older version though. Because the old server sucked like that. Not so much with the updates.

The charset is not set in the exp_actions table. Behold an excerpt from the exported file I used for the transfer:

CREATE TABLE `exp_actions` (
  `action_id` int(4) unsigned NOT NULL auto_increment,
  `class` varchar(50) NOT NULL default '',
  `method` varchar(50) NOT NULL default '',
  PRIMARY KEY  (`action_id`)
) TYPE=MyISAM;

So here’s a silly question. Can I just take the meta tag out? Will it then display properly on all browsers? Are there other downsides to doing that? I don’t know too much about this stuff, obviously.

And yeah, I don’t know what happened with the links either. We’re working on that right now. In the meantime, if you just manually add index.php into the URL, it loads fine. But that’s another topic, I suppose.

UPDATE

Until just now, I was unable to access my control panel. After much more time spent fiddling with the server than any sane man would endure, I finally got into it. There I found a bunch of blank fields and strangely changed options. Fixed the problem with the links straight away. In general configuration, the default character set was big5. (And the default XML language was Afar—like big5, it was the first option on the list.) Changing that solved a few of my garbled characters like the » , but all quotation marks are still off.

So my original questions remain - Can I wisely take charset out of my meta tags? Put a charset decleration in the SQL table?

And I add this - did my files just get corrupted in the move? Should I try starting over then? Most, though not all, of my EE Admin setting on the new server are blank. No radio buttons selected or anything. Site status mysteriously set to offline.

P.S. The old server ran MySQL 4.0.15-max-debug. The EE SQL manager also indicates that the new server’s database is .5 MB and about 3800 records smaller than the old one. Um, what?

The technical term for this is ‘Argh!’ 

Hm- yep, a lot of the collation settings came with mysql 4.1- thus no character set specified in your imported tables.  You can take the metatag out- no problem.  I’d try that right now- if all is ok, might leave it in and change the character set to ISO. 

I don’t think things got corrupted in the move, and if you exported from EE and did a ‘Crucial data only’ option, the db could be smaller.  (No point in moving some of the cached data, which will likely need to be rebuilt, other stuff like that.)

Let’s see- you were already running 1.6- and that’s the version that was moved.  Correct?  Not sure why the settings would be goofed like that.  When you save new settings, you don’t get errors, do you?  And have you cleared the cache file, cleared caches in the database, and double checked config.php is chmod 666?

(And I may call in some dev crew backup to take a look at the table/character set issue.)

Well, the mystery continues…

With the database size, I too thought of the nonessential data thing. But I would have sworn I unchecked that and had it download all data. Perhaps my memory fails me.

And I hate to pull an Alberto Gonzalez on you, but I really don’t recall when and if I cleared caches. I did a few minutes ago and everything looks the same.

I took the meta tag out, no apparent change.

Yes, I was running 1.6 and that’s the version that moved. When I save new settings, no errors (except, you know, if I put in something stupid). Config.php is 666. Could it be there was a setting in the FTP program off kilter when uploading the database?

Thanks for your help.

Are you sure you removed the meta tag- I’m still seeing it in the parsed output:

<meta http-equiv="content-type" content="text/html;charset=utf-8" />

I’m still thinking removing it will get rid of the character set issue.

The settings issue…  Let’s get character set squared first.

Okay, taking the meta tag out fixes some of the templates, but in others problems persist. I can’t make rhyme or reason of it yet. Here goes…

In #1, everything is fine. It’s clear in the parsed source.
In #2, it takes care of a few problems, but apostrophes are still screwy. In the source, the appear as the HTML text codes #whatever.
In #3, apostrophes are also off. In the source, these characters appear as the ? symbol.

All these templates are practically copies of each other, except for a few words, so it makes no sense to me that they would be different. The articles are all entered on different blogs (for each template) and by different people from different computers.

In all instances, the character errors also show up in the control panel entries, both in headlines and the text fields.

So, I have a headache. I’m not worried about the blank control problems thing—so far it’s just an annoyance that I can deal with. The encoding though is driving me crazy. I’m fully ready at this point to delete the whole thing and start over IF that will take care of it.

I’ll keep fiddling a little more to see if I can give you more specific information.

😠

Nevermind! It works without the meta tag now. Some combination of things I changed got most of the errors to go away, except for a couple characters and three entries. These I can take care of by hand. Could it be that the tag in index.php affected all articles loaded with index.php in the URL? That doesn’t make any sense to me, but that’s the way it seemed. But, no matter.

The character errors remain for all entries when edited in CP. That’s somewhat bothersome but not critical. New entries (posted since the switch) are clean.

So I’m not going to worry about the rest. I still would be curious as to why what worked on the old server causes problems on the new or what caused the control panel to freak out. For future reference. But for now, stuff works, I’m good.

Thanks again for you help, Robin.

PS. Whoever invented character encoding ought to be shot. The rest of the world should just learn to speak English displayed by one single character set. That’s my philosophy.