Thread

I’m working on an upgrade for a large site - taking them from 1.7 to 2.3.1 and I am noticing an issue with special characters being replaced by a question mark.

As an example, it seems that all apostrophes in entries and in templates were replaced with a question mark (ex: Don’t is now Don?t)

Is this a known issue? And is there a way to avoid this when I do the live upgrade?

Note: I am working on a test server while I test the upgrade process - so the upgraded site/db is accessible if you need to take a look.

Thanks for your help.

Mark,

This is just a guess, but is your original database encoded in something other than utf8?

Default Character Set is set to utf-8. But I did notice that the 1x database is NOT set to Automatically Convert High ASCII Text to Entities. Could that be the cause?

that would be my guess.

Thanks for the assist, Sean!

Mark, are you able to put back the DB to 1.7.x and let us login and try a few things?

The live site is still on 1x. Can you login to that to test some things without risk? If not I can deploy the 1x site to a dev server (but that will take some time).

What we’ll want to do is re-run the update script- commenting out the utf-8 conversion bit (I suspect the characters are being converted twice- which would result in what you’re seeing).

So yes- we’d want to do it on the dev server.

That said?  No need to get all of the ee files back up- a DB with the 1.x version and a copy of the 1.x config file in a backup folder somewhere will do the trick.

Now that you say that, I remember that during the test upgrade I had to run the upgrade.sh script twice because it threw an error. So let me re-run that locally to eliminate that as the cause.

Hrm- that could be it.  I’ve run into it one other time where I suspect a hack had been used to make ee1 utf8 throughout (hence it did not need re-converting).

But yes- definitely worth a shot.  We want it converted- we just don’t want it unnecessarily converted, which will bugger it up.

Anyway- subscribed.  Give a yell when you know something!

Okay, I think I’m good here. I ran the update script again and I’m not seeing the same behavior. Thanks for pointing me in the right direction.

Thanks for thinking to double check that, Mark.  You saved us both a fair bit of effort.  Effort that wouldn’t have worked 😉.

Glad this looks squared away.  If you do run into any other trouble with it?  Just start up a new thread and we’ll dig into it.