Thread

We have a fairly complex and extensive site, and some of our key page types seem to not respond to their various contributing templates being cached.

Others do, and there template caching helps a great deal, reducing queries by 75% or so and server time by about 90%. But our more complex pages, with a few generations of nested embeds, seem to load at the same (slow) speed regardless of whether their contributing templates are set to cache or not. The output debugging confirms this.

We’re wondering if there are any rules dictating whether template caching gets cancelled, and if we can change them, or hack them. We need that template caching!

(The one thing we can think of is that the templates that won’t cache contain PHP.)

Running EE 2.1.3 build 20101220

Have you tried other forms of caching, like query or tag caching? Just how large / complex are these templates?

Ingmar, every channel:entries tag has the cache parameter set, and we were advised by a certain Mr D Jones not to query cache sitewide as mysql is set to do it and does it better. (We did have it on before and the difference either way seems negligible.)

Ingmar, every channel:entries tag has the cache parameter set, and we were advised by a certain Mr D Jones not to query cache sitewide as mysql is set to do it and does it better.

It’ll depend on your server setup, but yes, that’s a definite possibility. When you turn on template debugging, do these templates show any significant differences to the others?

How much traffic are you getting? I suppose you have done the obvious things?

We’re wondering if there are any rules dictating whether template caching gets cancelled, and if we can change them, or hack them. We need that template caching!

About the only thing I can suggest is checking out Solspace’s cache module.

It’ll depend on your server setup, but yes, that’s a definite possibility. When you turn on template debugging, do these templates show any significant differences to the others?

How much traffic are you getting? I suppose you have done the obvious things?

Yes, we’ve switched off the hit tracking, things like that. Also doing what we can re reducing queries, but the content is complexly interrelated, which is our problem.

About the only thing I can suggest is checking out Solspace’s cache module.

We’re using Solspace’s Static Page Caching for a few dozen of the first-tier pages, but the site is pretty big and we don’t want to do it for all of them. Though maybe we’ll have to.

Nonetheless, the initial question remains—does anything cancel page/template caching when embedding gets too complex? Derek is helping us out via email so I’ll report back here with the findings.

Very good. Just keep us in the loop, please 😊

Well, it turns out that our problems were twofold, both fixed now, and we’re crackling along. I’ll detail them below, maybe someone else will have similar issues.

Due to SEO, we have a strict URL structure. Every page in our part of the site is at:

/sport, eg, /basketball/
/sport/sponsor, eg /basketball/nike/
/sport/sponsor/area, eg, /basketball/nike/california/
/sport/sponsor/type, eg, /basketball/nike/overnight/
/sport/sponsor/camp, eg, /basketball/nike/nike-girls-basketball-camp-uc-santa-cruz

(Generally, pages with trailing slashes are multi-entry; without are single.)

The problem is we’re struggling against the EE grain here, as the natural way to do the above would be:

/sports/basketball
/sponsors/nike/basketball
/types/overnight/basketball/nike
/camps/nike-overnight-basketball-camp-at-santa-cruz

with the first segments being template groups or templates in the default template group.

Instead, they’re all produced by the website/index template and its embedded templates. The drawback here is that its template caching doesn’t work; somehow website/index is overloaded and won’t cache all the pages it’s handling.

Our solution—though it won’t work for everyone—is to create a template group for every sport, and a template for every sponsor within every sport, each set to cache, and containing the same single line:

{embed="website/index"}

This reenables caching for each page; the basketball/index template’s cache holds the /basketball page, and presumably the basketball/nike template’s cache holds the /basketball/nike and /basketball/nike/* pages.

http://www.ussportcamps.com permits it because the number of sports and sponsors is relatively static and small, about a dozen of each. It wouldn’t work for a similarly-structured site where many more entries are going in the segment_1 and segment_2 parts of the URL.

So now we could cache pages fully—no channel:entries tags called! But unfortunately some embedded templates cannot be cached because they’re called more than once within the same pageload; if cached, the content simply repeats. Also unfortunately, it was those uncachable templates that were taking so long to load; they had huge numbers of queries. In all, template caching barely helped us.

The offenders were the two templates that display our camps grouped by area and city—and often there are more than one of these displays on a page. Camps contain a relationship field to cities, and cities in turn to areas. From our list of camps derived from programs (in turn derived from sport and sponsor), how do we display grouped by area then by city? Embarrassingly, it was a challenge.

We passed the list of camps to the entry_id parameter of a channel:entries tag, then used the related entries tag to get each camp’s city and the Tied Entries plugin to get each camp’s area. We passed all this data using PHP variables to the embedded template. Unfortunately we passed them as lists of entry_ids without intact relationships, so when looping through the areas we had to use Tied Entries again to get each particular area’s camps. Then we displayed each area’s cities, filtered by whether it was one of the cities we brought over. Then we looped through each city’s camps, filtered again by whether we brought the camp over. It worked, but all these reverse_related tags and Tied Entries instances—sometimes called repeatedly on a page—were producing huge numbers of MySQL queries. And remember, it couldn’t be cached.

Our eventual solution was to swap out these two heavily looped templates for a single query tag with a MySQL join from camps to cities then another from cities to areas, ordered by area, city and camp title. To avoid repeating the area and city on each camp line they’re only displayed if their values differ from the previous iteration through the single loop. This new method reduced to one query what was in some cases almost 2000!

With these changes we reduced page load times from up to around 20 seconds to just 2 on the first load and 0.2 on subsequent fully cached loads.

Thanks, Derek J, for proddings and encouragements, and pointing out the untenable number of query tag calls.

Good engineering work there, Adam—and it is certainly a very attractive site. Lots of nice concepts and integrations in how you did things, and it works very well.

Thanks for writing your full experience. I think it will put a much clearer picture to those who need to hear what ‘enterprise’ means for EE use, and those persons will understand the aspects.

Regards,
Clive

Adam,

Thanks for coming back with your write-up of this. I have bookmarked it. I had seen the other thread on this site and had actually mentioned it internally. Glad DJ got you sorted. Closing this out for now. Feel free to create a new thread if you have any more questions.