Thread

We posted a guide to getting started with Varnish for your ExpressionEngine site. Post here if you have any comments or questions.

Thanks!

http://ellislab.com/blog/entry/making_sites_fly_with_varnish

We’ve had some really great success with Varnish on a few projects. It’s an excellent package, but suffers a pretty serious flaw in that it requires fairly advanced systems knowledge in order to install, configure, and maintain properly. This is a great article on the complexity that goes into doing that stuff!

I’d also like to point out that there are other options as well - such as my own CacheCracker as well as the excellent CE Cache which I feel are a lot easier to setup for the typical ExpressionEngine dev. Just thought I’d throw it out there in case anyone wasn’t aware and is scared off from taking the plunge into Varnish.

We’ve had excellent (shall I say amazing?) results using CE Cache and Template Morsels, in some cases in a single site.

We are able to get the perceived page speed that beats dedicated/virtual hosting solutions costing $2-300/mo., even on a shared hosting account.

I’d highly recommend both add-ons for any site, not just with high traffic concerns.

For one of our lasts projects (http://www.torfs.be/) we used Varnish. It was the first time we used Varnish and we ran into several issues; especially for the ‘dynamic’ part of that website. As it is a webshop including commenting, ratings and a continuously changing stock we realized that Varnish would not be the easiest part of our projects.

We ran the first few weeks with CE Cache enabled, but it couldn’t handle the massive content (more than 4000 unique pages) and all http requests were still handled by the webservers. With handling 100.000 to 300.000 pageviews a day we needed a better solution as our servers (5 virtual web nodes) were melting. Running with Varnish, the server load was seriously decreased and it could handle more than 500 concurrent users at lightning speed.

Things we learned/did;

- We used some AJAX calls for some ‘dynamic’ parts of our website (eg; comments) but they slow down (and you loose the benefits of your Varnish cache) and with a lot of visitors those AJAX calls still take your server down.
- We receive data from our clients ERP system, when that data is indexed in Solr we automatically run our purge scripts that remove all caches that are related to that data.
- Test a lot! Especially on user-driven content websites.
- Avoid excluding too many pages from the cache.
- Try using ESI snippets where possible but use them wisely.

Hi Guys,
Totally new to varnish but gotta start somewhere. As per http://ellislab.com/blog/entry/making_sites_fly_with_varnish I went thru all the steps I believe. I’m running Ubuntu 10.4 and installed Varnish 3.0 using https://www.varnish-cache.org/installation/ubuntu

I created a file main.vcl at /etc/varnish/main.vcl and put just

backend default {
    .host = '127.0.0.1';  # IP address of your backend (Apache, nginx, etc.)
    .port = '8080';       # Port your backend is listening on
}

sub vcl_recv {

    # Set the URI of your system directory
    if (req.url ~ '^/system/' ||
        req.url ~ 'ACT=' ||
        req.request == 'POST')
    {
        return (pass);
    }

    unset req.http.Cookie;

    return(lookup);
}

sub vcl_fetch {

    # Our cache TTL
    set beresp.ttl = 1m;

    return(deliver);
}

..into it and restarted apache. Do I change the 127.0.0.1 to my actual hostname IP of eg 150.151.152.153?

My system folder is the default “system”.
The full path to my system folder is /srv/www/mysite.com/public_html/system
Should I change /system/ in the vcl file to something else? Changing /system/ to /srv/www/mysite.com/public_html/system/ didnt work :(
/etc/varnish/main.vcl is correct and exists at that location on my server.

Using the recommended settings from Kevin I get the error:

root@li482-221:~# root@li482-221:~# varnishd -f /etc/varnish/main.vcl -s malloc,200M
-bash: root@li482-221:~#: command not found
root@li482-221:~# Message from VCC-compiler:
-bash: Message: command not found
root@li482-221:~# Syntax error at
-bash: Syntax: command not found
root@li482-221:~# ('input' Line 9 Pos 19)
-bash: input: command not found
root@li482-221:~#     if (req.url ~ '^/system/' ||
>
> ------------------#--------------
>
> Running VCC-compiler failed, exit 1
>
> VCL compilation failed

Line 9 being

if (req.url ~ '^/system/' ||

..so could someone identify what I am doing wrong please. I also tried ” instead of ’ quotes which didnt throw an error in SSH but instead i get

root@li482-221:~# varnishd -f /etc/varnish/main.vcl -s malloc,200M
WARNING: Taking over SHMFILE marked as owned by running process (pid=20740)
root@li482-221:~# sudo service apache2 restart
 * Restarting web server apache2                                                 ... waiting                                                             [ OK ]
root@li482-221:~# sudo service varnish start
 * Starting HTTP accelerator varnishd                                    [fail]

I also changed the apache port to 8080 in ports.conf using http://www.umarani.com/varnish as a loose guide.

Using Putty for SSH.
Thanks!!!

Hi Ralph,

Do I change the 127.0.0.1 to my actual hostname IP of eg 150.151.152.153?

If Apache is listening on the same machine you have Varnish, which I think it is in your case, then you can leave it as 127.0.0.1.

Should I change /system/ in the vcl file to something else? Changing /system/ to /srv/www/mysite.com/public_html/system/ didnt work :(

Varnish doesn’t need to know the server path, just the path it sees in the URL. In this case, you should be fine leaving it as /system/.

It looks like double quotes may have worked for you, there wasn’t a compilation error at least. Running `sudo service varnish start` may not have worked because I think it was already running even though you got that warning. Always try to kill the existing process before starting another one.

Great thanks Kevin. Thats all working now. Just one issue though and thats with PHPMYADMIN. After installing varnish I cant log in to PHPMYADMIN.
I even added

req.url ~ "^/phpmyadmin/" ||

to

sub vcl_recv {

  
    remove req.http.X-Forwarded-For;
    set req.http.X-Forwarded-For = client.ip;

    
    if (req.url ~ "^/system/" ||
 req.url ~ "^/phpmyadmin/" ||
        req.url ~ "ACT=" ||
        req.request == "POST" ||
        (req.url ~ "member_box" && req.http.Cookie ~ "exp_sessionid"))
    {
        return (pass);
    }

    unset req.http.Cookie;

    set req.grace = 1h;

    return(lookup);
}

Log in worked fine without Varnish. Is there anyway to fix this?
Thanks

Strange, adding the check for /phpmyadmin/ should have done the trick. I assume you restarted Varnish after adding this?

If it still doesn’t work after a restart, it’s hard to diagnose what phpMyAdmin is doing from here that Varnish isn’t allowing to hit the backend.

As a last resort, you can access it straight through Apache by going to http://yoursite.com:8080/phpmyadmin/

thanks it a worked after a few minutes. I have a symlink in the root called myadmin that redirects to phpmyadmin also to overcome a network firewall issue. I was hoping to apply the check to “myadmin” but that didnt work. If anyone knows how to get the symlink working it would be great. Great article by the way!

Kevin how do you stop

varnishd -f /etc/varnish/main.vcl -s malloc,200M

using the ssh terminal. Whats the command

I tried service varnish stop…but no joy. Kill varnish also didnt work.

I just want to switch off varnish to see if it is causing a php sessions issue on my AWS hosting

Thanks