Thread

Hey guys and girls, I’m developing an EE install for a fairly high trafficked site and while load testing the environment we’re seeing some awfully strange results.

First some background: the site has some similar features to http://gizmodo.com/ with a dynamic header containing recent, rotating stories. The main body has, in general two page types. A listing page contains upwards of thirty of the most recent entries organized by category, (solspace) tags and date. A detail page contains a single entry with comments and up to 10 cross-linked articles.

I have two questions on caching now, first, are all caches (template and tag) based on the URL? Meaning will my dynamic header (that’s the same on every page) be cached for each and every detail page that’s viewed? If so does that mean that I’ll have the same header cache for /article/detail/1, /article/detail/2, /article/detail/3, /article/detail/4, etc…

Secondly, we’re seeing cache files aren’t being written quite as fast as we’d like (if they’re even written at all). For example when testing an out-of-the-box install of EE 1.6.7, targeting the default homepage, over 60 concurrent requests only 22 use the cache, where I would expect 59 to. Hacking into the core of EE I’ve narrowed this down to the `write_cache_file` method, which is being called 38 times, each time writing a new cache file. Presumably this is because the 38 requests are concurrent and running on separate processes. All of which check the cache directory at the same time, realize there’s no cache so all 38 concurrent processes decide to create the cache individually. This problem is compounded when we throw more concurrent requests at the box. 70 concurrent requests or higher and the cache file is never written (or, I should say is written 70 times and not used once).

So, what can I do? Is there a way to create cache files that will not be dependent on the URL? Is the cache writing problem a problem with my box’s disk speed? Permissions problem? Lock problem?

Thanks for any insight you can offer!

Mark,

Thanks for your post. My insight does not extend to answering all your questions so I am giving a head’s up to a dev who might answer you more precisely.

Yes, template caches are all based on the URI, since any tags contained in a template can have their output modified depending on what URL segments are used.

For the cache files on concurrent requests issue, there could be two things at play.  One is that when your maximum number of files have been written (specified in your preferences, with an internal limit of 1000), you’re going to be destroying and rewriting cache files.  The second is that the disk array is simply not quick enough, whether it be from the time it takes to establish a file lock or simply writing the file. For that short burst of concurrent visitors, it could indeed cause an issue, but should not affect the long-term effectiveness of that page’s cache.

The second is that the disk array is simply not quick enough, whether it be from the time it takes to establish a file lock or simply writing the file. For that short burst of concurrent visitors, it could indeed cause an issue, but should not affect the long-term effectiveness of that page’s cache.

Thanks Derek, I’ll look into disk speeds and see if there’s anything to be gained here. But you’re right, once the cache file gets written all is well (or should I say fantastic), it just sometimes takes upwards of two minutes to create the cache file as all the concurrent requests continue to overwrite the file.

Eep, yeah, two minutes is not good.  Let me know the results of your findings and I’ll see if I can help further.