Thread

Chances are that only a few particular things are causing your problem(s).

Did you turn the google crawler rate way down?
Check on the apache logging…turn it off.

I don’t use the EE built in caching, because I think the server and mysql automatically do this.
I have only the online user tracking selected….

If the problem is in the settings of mysql and/or php and apache, then all this stuff will hardly matter. You have to start with the big things…....
Your site loaded pretty quick for me, BTW….yesterday.

I’ve got the crawl rate down now, yes. Apache logging I don’t have control over myself, but have passed on the suggestion to my server admin. However, as you noted, site load yesterday was fine. Basically, its only when we have a bit more of a traffic spike that it becomes a problem. So I think we can say that it handles our general everday traffic fine right now but I need to look at what I can do to minimize the issues that occur during traffic spikes.

I am not quite following this:

“If the problem is in the settings of mysql and/or php and apache, then all this stuff will hardly matter.”

That’s sort of what I have been trying to find out. What settings there matter? If I knew more what to ask about, I could get more detailed information from my host.

We’re somewhat beyond what we can offer in terms of regular support here. Let me move this tread down to the CSC. Thanks.

Ingmar,

I am sorry, how is this not a support question? I need some help figuring out what specific settings I should have in Expression Engine to have it work best for my system? I have asked several times on this thread for specific advice about what information I should get from my host so that I can advice get some advise about which setup is best.

If that isn’t a support question, I don’t know what. I am sorry, but this is very disappointing. I may be in a situation where I have to abandon EE if I can’t sort out some way for it to work better during traffic spikes and I really want to avoid that.

For example, how do I determine:

*If the database connection type should be persistent or non-persistent?
*If SQL Query Caching should be on or off?
*If it is best to use page caching or tag caching? Is there a drawback to having every page set to cache?

Furthermore, I have asked what could cause caching not to work. As I noted, I have page caching of 60 minutes for every page, yet my server admin reports that everything appears to render dynamically. Could this be because I have a too low max of cachable URLs? Its currently set to 500.

Additionally, I asked whether the view counter tag on a Gallery page means that the page will never be cached but fully loaded each time.

Hi, Linda,

The reason this was moved as it had gotten far afield of EE and gone into managing the server settings and Google to help with load.  However, your EE questions are indeed technical support, so I have moved this back.  I am sorry for the confusion.

*If the database connection type should be persistent or non-persistent?

Usually non-persistent; but confirm with your server admin.

*If SQL Query Caching should be on or off?

This one again depends on your server.  For instance, EngineHosting prefers SQL Query Caching off since they handle query caching on the servers.  Again, you need to ask your admin this.

*If it is best to use page caching or tag caching? Is there a drawback to having every page set to cache?

Template caching is preferable.

As far as caching not working - did you check your system/cache folders for files?  Those will have files in them if caching is working.

If you cache your pages, your view counter may be off, so that is something to be aware of.

Does that help?

I am sorry, how is this not a support question?

Linda,

This is all about server configuration which, unfortunately, we won’t be able to assist you with. At this point I have no reason to believe that this an EE issue as such. We talked about optimization, static templates and a few other options you might have, but ultimately this is about setting up, configuring and fine-tuning your server.

I need some help figuring out what specific settings I should have in Expression Engine to have it work best for my system?

There is no hard and fast answer here, either. There are the various caching options that you’re aware of, of course, but results will largely depend on your system, setup and configuration. I can only suggest that you try a few options, see which ones work best for you.

ETA: I see there were new questions, and Lisa moved the thread back for now.

The EE crew is correct, because server config ends up being an infinite number of possibilities - so anything they (or even I, as a layman) might be true…or not…for a certain config.

What I meant by my statement about configs….is that they MUST be set somewhat correctly….that is, the apache/php/mysql balance…..

From afar and from what your server admin said, the first settings to look at are the Mysql.
It is easy to see if mysql is being stretched just by running a free program or two….

http://blog.mysqltuner.com/download/
https://launchpad.net/mysql-tuning-primer/+download

Your admin can easily run these and get some results - they look at the past mysql logs and let you know if anything is way off.

An example….of me running one of those…
——————-
OPEN FILES LIMIT
Current open_files_limit = 1134 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine

TABLE CACHE
Current table_cache value = 512 tables
You have a total of 210 tables
You have 317 open tables.
The table_cache value seems to be fine

TEMP TABLES
Current max_heap_table_size = 16 M
Current tmp_table_size = 32 M
Of 1389207 temp tables, 2% were created on disk
Effective in-memory tmp_table_size is limited to max_heap_table_size.
Created disk tmp tables ratio seems fine

TABLE SCANS
Current read_buffer_size = 2 M
Current table scan ratio = 3166 : 1
read_buffer_size seems to be fine
——————————————————-

That would seem to be your fist step.

Chances are that PHP is fine…...but if Mysql does not show any problems, then you want to move on to apache (the thingy that serves web pages).....etc.

This has settings which tell it how many connections to handle at one time, etc. etc….
It all depends on the version as to what applies.

I think you should do the mysql checking…...before anything! Why complicate matters if most of the problem is there….

Of course, we assume you regularly optimize your tables…....

Thank you Lisa. Those were basically the questions I tried to get across from the start. I suppose that the fact that handyman brought up some additional suggestions for what I could do may have muddied the water, though I am not exactly in a position to turn down any suggestion to get the situation improved.

Asking about persistent vs non-persistent, okay, that’s straightforward enough. However, in terms of the SQL query caching, the docs say “EllisLab does not recommend using SQL Query Caching in the majority of environments”.  This setting just comes down to whether the server is handling the caching or not?

In terms of template caching, I see a number of folders under the page_cache folder. Is it this number that is controlled by the maximum URLs setting? It appears you are right that the view counter is not incrementing every time, so clearly there is caching going on. Maybe I need to get the max cachable urls up to the max, which I think I saw was 1000. Need to determine how much diskspace 500 is currently taking.

Basically, what it keeps coming down to is what my server admin said: “Nothing’s ever just loaded from disk, almost every hit involves asking the database for data.” Is this always the case with EE, even with page caching turned on?

The caching page mentions “Because ExpressionEngine requires some scripting and a few database queries in order to manage core resources, we can’t achieve 100% static pages, but we can get close.” Is there anything I can do from a user point of view to minimize that number of queries, or is that integral EE workings that aren’t affect by what’s in the templates?

Edited to add: Is there any way to tell from the debug output for a particular page whether it was served from cache or dynamically rendered?

Linda,

if you turn on your query caching or indeed your database caching can you see relevant files and folders being created in your cache folder as Lisa asked?
This is a quick way of telling if your caching is working.

If you want to speed up the site in general, the YSlow firefox extension is quite good in finding suspect areas.

Do you have any custom PHP in your templates or are they just straightforward EE tags?

There is an excellent document on performance in EE availabe for download here

This gives a great overview as well as good advice on solutions for speed and performance and delves into issues like handyman brings up

I posted something similar in http://ellislab.com/forums/viewthread/182248/. Unfortunatley someone seems to think it fine for the db to be hit with so many connection issues!

Hi Sky Small Projects,

I have replied to your other thread.

@Linda A, what was the outcome of your testing? Did you manage to optimise your hosting appropriately? Do you need further assistance from us?

Cheers

Greg