Thread

I’m designing an image-intensive site and plan to use EE to run the static/blog/news sections and possibly the whole enchilada.  In evaluating the latter,  I’d like a sense of whether EE is likely to have any performance issues with a site of the following characteristics, assuming reasonable hosting and recommended caching systems/coding techniques:
    - 10 weblogs
    - 7000 entries
    - 200 custom fields (~30 related)
    - several key templates displaying 100 custom fields (~30 related) as well as 10’s of images.
    - modest javascript for most templates (base jquery + a little more)

    - 30 galleries/weblogs (haven’t decided which)
    - 10,000 images

I’ve seen several useful threads about EE driving “big” and/or “busy” sites, but nothing so far that address my largest concern: page loading times for the key templates I’ve outlined above.
    - What are some of the bigger/most visited EE-based sites? 24 April 2007
    - EE scalability 29 August 2007
     
I’ve built just one site with EE so far (clients like it and are running it on their own), but it’s much more modest in size. Performance is OK without having done *any* server or database fiddling, but still nothing like the sites I’ve built previously with Dreamweaver’s PHP+MySQL behaviors, extensions, and some hand coding (hey, I’m not a developer 😉).  Of course, those sites required me to build custom databases with 10’s of inter-related tables and a lot of SQL code, not to mention the admin process…

Another concern is data import and export.  I’ll look into plugins like CSVGrab but I suspect it may be easier to just learn the EE database schema and write some SQL (?)

BTW, the site will start out with only 1500 entries and 1500 images but will build up to the above numbers (assuming we get more funding).  If curious, check out preliminary interactive wireframes of our design (scroll down to find links to PDF and HTML versions; any feedback on wireframes appreciated).

Thanks for your thoughts!

The wire frames are great - what did you use to put hem togther?

I’m going to chime in and also inquire about your wireframing technique - really nice. What are you using?

As for scalability I’ve built one site only a quarter of the size you’re proposing and its fine under 10’s of thousands of hits compacted into 4 or 5 days. However, have you looked at iPod lounge? That’s a big ee site under lots of traffic. Perhaps you could contact their developer through this board?

The only potential red flag I see is the number of custom fields you purpose. EE can have 200 custom fields but you may have some performance issues if you’re pulling them excessively on a single template. How are you planning to use the custom fields?

@Leslie
yes, it’s the number of custom fields that concerns me the most (EE seems to handle many entries and lots of traffic pretty well).  There would be one template in particular that would pull about 75-100 custom fields from several webblogs; that same template would also deal with several regular and reverse relationships.  It may be just be too much for EE?... presumably at least partly due to how EE structures its data?  My understanding is that EE’s structure was never designed for lots of custom fields and its relatively few mysql tables end up being pretty inefficient with so many columns and lots of empty “cells”.  If so, here’s hoping that EE v.2 has a more adaptable/extensible database structure.  Maybe a built-in way to work with other mysql tables or even other databases.

I’ll have to give some thought to how I could run some parts of my site with EE and others with my own tables/database and code.  There will need to be some integration. And I don’t look forward to having to build a second admin backend; using the EE CP for the whole site would have been sweet!

@simoncox & Andy Detskas
I made my wireframes with Omnigraffle Pro (OS X only).  It isn’t ideal, but most tasks are pretty intuitive and exporting the ‘frames to interactive PDF or HTML literally takes a second or two.

Just to add to this - when EE pulls custom fields the performance will be poorer then any “raw” query you are making yourself.  If you knew your database structure well, and knew SQL well, then you could use the query module to write much leaner, more responsive queries.  That said, we are talking about considerable work here.

Also of note, with your own custom queries, none of the “EE magic” will happen (ie: related or reverse related entries).

Nothing else you wrote strikes me as particularly “scary”, except to note that a site of this magnitude will probably bottleneck on 3 places:
1) custom fields/relationships (as already discussed)
2) developer “smarts”.  Things like not nesting too many tags, embedding templates, etc.
3) raw server speed and power.  You’ll definately want a dedicated database server for this.

Derek has it right.  I’ve got a few sites that have 180+ custom fields.  Those are not a problem generally in terms of performance if you have a site with only moderate traffic.  However, related entries within those fields can make things heavier.  Keeping the number of entries tags per page down (if possible) helps immensely if you need to use the entries tag.

If your site is going to have a lot of traffic then some custom queries and probably some custom PHP is probably more in order. 

Or if you have the monetary resources you can always throw more horsepower at it.

One of the sites I have that uses this number of fields is all stock EE in terms of code.  The others though have quite a bit of custom coding that went into them to get the performance and functionality that was needed. 

However, in only one of those cases was the client’s traffic enough that they had to go with a dedicated server setup.

The biggest question in terms of performance though is the amount of traffic you need to handle.

Jamie

180 Custom Fields?! Is that in one field group?

I toyed with the idea of using one weblog for products with a lot of custom fields but it seemed too cumbersome to have to scroll down to where the client needed to just add one product.

Instead I created multiple weblogs and custom field groups. The downside to this is when producing search results.

Yep, that’s one weblog.  The client uses a front end edit/publish form that hides or shows the fields based on what they have set in pervious fields.  So its pretty efficient for them to use.

Jamie

Mmm is that via an SAEF? I’d love to find out how you did all that! It would have been perfect and probably made my EE implementation a lot cleaner.

Perhaps a tutorial is in order? 😉

Yes, combination of the SAEF, Solspace’s Form Helper plugin, and a little Javascript to do the hide/show stuff.

Jamie

That’s pretty nifty. We just did a weblog with 105 custom fields describing a school directory, with board positions (multi-text!), staff, contact info, blah blah blah. 😊 That Form Helper plugin saved my life as well.

Derek, Jamie, AJP,  et al, thanks for your additions, clarifications, and examples 😊

Being the optimist that I am, I’m interpreting your posts to suggest that EE might be able to handle my whole site after all.  But probably not with entirely stock EE code… even with lots of horsepower (?).  For decent performance it sounds like I’ll need some Query Module queries and custom PHP .  And as the site grows out to the size I described, the hosting will have to keep pace, possibly with a dedicated database server.

Jamie, can you give me a sense of the page loading times (for basic DSL connection speeds) for your most complicated templates for sites with 180+ custom fields?  3-4 seconds?  less?  And roughly how many entries do these sites have?  Thanks.

To handle data entry for weblogs with tons of custom fields, I’ll explore the SAEF-based approach discussed by Jamie.  I may also explore Rob Q’s CP-based approach of adding a field header and divider based on an extension written by Huot.  See his comments at Jambor-ee.com, including his image example.

Of course, I’ll need a rough backup plan in case performance is not adequate.  I’ll probably spend a little time skimming the forums for ideas for integrating my own data with EE.  I assume that putting my data into tables within the same mysql database gives me more options, including using the Query Module.  Having some referential integrity between my and EE data sounds tricky.  I haven’t looked at EE’s tables yet, but I imagine they’re myIsam.

Load time depends on the template obviously. 

On the most complicated example the single entry and add/edit pages pull up in sub 1 second time frames.

Search and multi entry listings pages come in anywhere from sub 1 second to 1.5 or so seconds depending on the number of entries being pulled in at any given time.

The example is a custom Constituent Relations Management database so the data is pretty varied and does contain one field that relates entries to another in the same weblog.  The relationships go in both directions in the templates as well.  So viewing one entry may pull up 15 more entries in relation to it.  Viewing one those entries will likewise pull up the other entry they were related to.  There are currently 1380 entries in the weblog.

I should note that in this case due to the relationships and the need to pull large number of entries at one time into some of the pages this site did not end up using the entries tag on the search and listings pages because it was too server intensive.  I instead am running custom queries on those pages.  But as described above the performance is more than fine after having done that.  The single entry pages are still using the entries tag as they never had a performance issue to begin with.  The site is hosted on a shared host, though the shared host does not have a large number of clients on it (maybe 5 or 6 at this point) so its probably more accurate to call it a semi dedicated server.

At one point the site was on a shared hosting environment provided by Pair and it proved too much for that host.  The main issue was Pair’s anemic server ram which simply couldn’t handle the larger multi entry queries and would simply provide a blank page when it ran out of memory.  A more dynamic software oriented host (ala EE Hosting) will have more ram available to scripts and won’t have that problem unless you write a really awful query.

Jamie

That extension doesn’t seem to be on Mark’s Docs page so here is what I am getting each time I use a Field Header -

Notice: Undefined variable: field_type in C:\dev\sitefolder\admin\cp\cp.publish_ad.php on line 7308