Thread

I have a question regarding template caching embedded templates.

Let’s say I have a sidebar that lists the latest blog posts.  This sidebar is included on every page.

Is this template cached once, or is it cached for every URI?

The reason I ask:
I was looking in the system/cache/page_cache directory.  There’s a bunch of directories, but I see the same file in many of these directories: “c_342c188469be6669a830e2135d8f8a81”.  Everytime I see that file, it has the same content (the latest blog posts sidebar html).

Is the directory the URI and each file in the directory a cached template?

Do these duplicate files count against the 1000 cached files limit?

Thanks!

Template caching happens on a per-template basis. That said, what’s the issue? Does caching not work?

Template caching happens on a per-template basis

  Then, why do I have multiple exact copies of the same template in the cache?

I’m trying to understand the caching system better (specifically limits, flushing, content of cache).

The web pages appear not to be cached.  I’m wondering if the caches are only being stored for a few minutes and are getting flushed out.  So, I’m trying to understand the limits/flushing to figure out how better to more effectively set up my template caching.

Then, why do I have multiple exact copies of the same template in the cache?

Are you using other kinds of caching? Also, all caches are refreshed at some point, even if nothing changed. You can read more about caching in the docs.

Then, why do I have multiple exact copies of the same template in the cache?

Are you using other kinds of caching?

I’m curious here, also—why would it matter if other caching was being used? Does one cache cause the others to malfunction and duplicate the work as Stephen described above?

Also, all caches are refreshed at some point, even if nothing changed.

What point is that, exactly? How do we know what conditions and—most importantly—how often the caches are refreshed? A cache that’s being refreshed every minute is a lot less effective than one that gets refreshed daily.

Thanks for your help!

Hi Dave,

In the documentation that Ingmar referenced there is a description of the different types of caching that ExpressionEngine supports. You have fine-grained control over how long certain types of data are cached.

Cheers

Greg

In the documentation that Ingmar referenced there is a description of the different types of caching that ExpressionEngine supports. You have fine-grained control over how long certain types of data are cached.

I get that, and I get that Tag caching is overridden by template caching. But am I correct in understanding that the fine-grained control goes away as soon as the 1000-file limit is hit and the cache is rebuilt from scratch every time?

Also, Ingmar seemed to indicate that some caching types would make others malfunction and duplicate content. Perhaps I misinterpreted the response above, but that’s what it came across as to me. If so, this would be good to know about.

The bottom line is that we’re having trouble figuring out how to make EE’s caching work properly for a site serving millions of pages per month as I’m trying to figure out how to get our site back up to speed.

We’re both very familiar with the documentation, and I’ve done extensive searching of the forums and wikis. I have not found answers for the specific questions I asked in my original post. Getting answers to those first before we continue, would be the most helpful to me.

Now as for our setup: we do have SQL caching, we do have dynamic weblog caching, our uri limit is 1000, we donot use tag caching, most templates are caches 15-60 minutes, this specific one is 15. I see our cache flushing much more frequently than that (presumably because of the 1000 limit), and I’m trying to figure out what steps to take to get my templates to stay in the cache longer. The first step for me is to gain a more in depth understanding than the documentation provides

Caching is done at a URI level not a template level, if you have the same template say an “articles” template that is used to power each of your articles you will have a copy for each URL, same with embedded templates, if you embed the sidebar template into each page and you cache the sidebar template, you will have a copy for each URI.

Unless you actually are seeing a specific need, for a specific template, to be cached, I would highly recommend to not cache unless you have to.  That goes for query caching to as likely MySQL is also doing caching so you are adding overhead keeping track of multiple levels of cache rather than benefiting from caching specific items that need caching.  As certain things can not be cached at the template level, like conditionals, php you maybe using, etc. EEs caching does not make a “static” version of a page fully.  If you need that level of caching functionality I would recommend looking at Solspace’s Static Page Caching, as well as Template Morsels.

I see hosting clients over-caching in EE much more than under-caching for what its worth.

A good extra source of information if you have a high traffic site would be Handling Extreme Traffic, but really those settings and the disk i/o section can impact even moderate traffic sites.

Edit:  As a side note, clients we have handling many 10’s of millions of EE generated page views per month, or more, are using limited template caching, no sql/db caching, and no tag caching except in extreme cases.  In extreme cases other options are also good to look at too, ie: my references above.

Nevin—Thank you for all your answers and advice here. This is, indeed, valuable stuff and helps us quite a bit. One specific question, if I may:

Edit:  As a side note, clients we have handling many 10’s of millions of EE generated page views per month, or more, are using limited template caching, no sql/db caching, and no tag caching except in extreme cases.  In extreme cases other options are also good to look at too, ie: my references above.

Would you mind providing a link to one or two of these sites (or in Private Message, if you prefer). I’d just like to see what’s *possible* in terms of browser experience loading pages like this. It may very well be that we’re at the limit of what’s doable with EE, and I’d like to know/compare before chasing our tails.  For example, I believe http://ilounge.com/ is hosted on EE, and it, too, is slow to begin the initial load, IMHO (especially as compared to a site like http://arstechnica.com/ that pops in very quickly).

Thanks again!

Keep in mind that everything from logic within the EE template, specific usage of 3rd party add-on modules, etc. and even the amount of server infrastructure powering a specific site, all play a role on how quickly or slowly certain sites may load, as you can imagine no two sites will load the same. 

Even the simple fact that http://arstechnica.com/ has a 63k html home page, iLounge has a 135k.  Ars has a rather simple side navigation system, iLounge is pulling in a lot of extra data including rss feeds, polls, and including pulling in external data, in the data laden left nav section, like the recent posts from their vB forums as an external data source, etc.  Unless the sites being compared are even close to as complex data wise being presented its not a valid comparison in all honesty.

I will assume you have analyzed the template debugging output in EE to see each step used to create a specific page, many times you will find things that are easily identifiable as slow down points in your templates.

Unless the hosting clients have specificly stated publicly they are hosted with EngineHosting, I can not share client details, even privately.

I guess a few good question would be:

Where is your bottleneck?
  Have you fully analyzed the logic in your templates?
  3rd party add-on impacts to render times?
Are you seeing heavy web server resource usage (cpu/ram) per page request?
Are you seeing an impact at the db server level?
Have you considered pulling in outside consulting to assist in analyzing your bottlenecks? ie: Solspace Performance Analysis?

Note: I bring up Solspace a lot as we have worked with their group on a lot of high traffic, high profile sites, and EE and performance is one of their focal points.  Just an FYI.

Thanks, Nevin. Indeed, we’re seeing some CPU/Apache/PHP spikes, though that’s gotten better using eaccelerator. The biggest hit we’re seeing is to our MySQL server, which serves many other sites just fine (including some on Drupal as well as other CMS), but as soon as we added EE sites to it, usage spiked.

And I think it’s our misinterpretation/misunderstanding of EE’s caching that’s lead us astray. As we learn more about it, we find that it’s not at all what we interpreted it to be from our initial read of the documentation.

Honestly, an in-depth explanation of the logic behind the caching would be infinitely helpful if anyone at EE has any such document. For example, we recently came to understand that having something like a global “headlines list” at the bottom of each article isn’t just cached once, but is cached individually with each article. With a site that publishes 20+ articles per day (and has thousands of articles worth of history that are regularly indexed by search spiders), combined with the 1000-item cache limit (and the fact that the entire cache is erased when a new entry is published!), makes EE’s caching not really an efficiency point for us.

If I’m (still) making any false assumptions or interpretations here, please correct me, but it seems like this is all true, as we’ve come to find out.

Not sure what the magic answer is there. Yes, I’m sure we could continue tweak our template logic for small/incremental gains, but the fact remains that we don’t want to give up on something as simple (and valuable) as a global headlines list on each article page. It just seems to me we shouldn’t have to “pay” (in CPU and DB cycles and valuable reader/browser wait times) to generate it for every single page when the data is exactly the same. In fact, it need only be generated each time an article is added/edited.

Dave:

It just seems to me we shouldn’t have to “pay” (in CPU and DB cycles and valuable reader/browser wait times) to generate it for every single page when the data is exactly the same. In fact, it need only be generated each time an article is added/edited.

How are you making the global headlines list? Do you need more than just the title and the date?