Traffic Management Features in the 20081212 Build

For those of you using ExpressionEngine in high traffic environments: political/government sites like Change.gov or gov.ca.gov, social action groups like We Can Solve It, daily newspapers like Periódicos Zócalo, entertainment sites like iLounge and the Sundance Film Festival, business sites like BMI and paidContent.org, or even high profile blogs like Subtraction and Veerle Pieters, there’s a hidden treat in the last build of ExpressionEngine, released this past Friday.  Ok, well maybe not hidden, but as we don’t normally add features to ExpressionEngine in build updates, this one’s a bit under the radar.

The impact that the feature can have though is profound, and if you don’t actively enable these features, there’s no functionality change, so we made the decision to release it now instead of waiting for a version update.  What am I going on about?  Tracking Preferences, and an associated and highly technical page in the user guide for handling extreme traffic with ExpressionEngine.

MySQL’s default storage engine (MyISAM) is designed for efficiency with reading, which is good since ExpressionEngine is a read-heavy application in terms of database usage, not write-heavy.  When writes are made to tables with the MyISAM storage engine, it must establish a lock on the entire table for the write to occur.  ExpressionEngine has a few things that it tracks to provide bits of information to the site owner.  How many times a template has been accessed.  How many users are currently online.  Where inbound links are coming from.  How many times a particular entry has been viewed.  On high traffic sites, or under extreme and unusual traffic events (i.e. Digg), these normally innocuous bits of tracking suddenly become a problem, as you have thousands of visitors hitting the site simultaneously, and for each one of those visitors, MySQL is locking the associated table to try to write to it.  Before you know it, there’s a queue of table locks and MySQL just can’t keep up.

Now certainly hardware, infrastructure, configuration, and monitoring play a huge role in whether or not the environment can handle this when it occurs, but sometimes it simply may not be able to, or there’s not a budget for the toys that need to go into place to do so.  And these bits of data may not be meaningful to you anyway, particularly if you are using third party traffic analysis software.  With the exception of Referrer and Entry View tracking, prior to this build, you would have had to hack the core files to disable these features.  Now you can disable them temporarily or permanently in your control panel or your config file (handy if the sudden traffic burst prevents you from being able to access your control panel!).  There’s even an emergency “switch” you can put in your config file that will shut them all off at once.

Additionally, if you have a highly capable server administrator, the “master switch” can be triggered dynamically based on your traffic.  For instance, in your environment, if you and your server admin know that these locks start to become a problem anytime you have more than 800 online users, you can dynamically disable all tracking when that threshold is reached; no manual action required, no more emergency pages when you’re on the beach sipping the refreshing beverage of your choice, ExpressionEngine will take care of it for you.