Thread

I am currently using ee version 1.69 for a classified website.  I’m running into issues with server performance with over 100,000 ads.

My server admin states to me in an email: “The main problem is the classifieds component employing stock CMS tables for storing data (mainly exp_weblog_titles and exp_weblog_data) which is anything but optimal for a large number of records. Ideally, the data should be stored either in specialized tables (if the classified record structure is relatively static) or using an EAV model and have pivot tables generated using a cron job (so the queries will be heavily accelerated at the cost of not having real time entries - this should not be a problem as all major systems are employing this technique). “

My question is if ee2.0 still utilizes separate tables (exp_weblog_titles and exp_weblog_data)? 

Are there any performance improvements that ee 2 has over the legacy version in terms of having over 400,000 entries?

Thanks

I’ll let you hear a more official answer, but I made a few inquiries as to performance tuning of EE2 over 1 and was told each time by EE staff that speed and capacity were not reasons to switch (and I have not yet).....

It would seem difficult for the custom fields to not be in the same place as before, as migration would be impossible if that were the case.

I understand what your ISP is saying and he, in theory, is 100% correct. Nothing beats a slimmed down and properly designed system which does just what you need done and nothing more. But a flexible CMS simply is not designed for that. It’s more like the all-in-one tool.

Thanks for the assist, Craig.

Ryan, if you wanted to use a different table to store your ads in you certainly could, it doesn’t matter which version you chose at that point.

I’ve used sites where we parsed .csv files on a daily basis and used custom tables. It was easier to do it that way.

Are your classified ads weblog tags making use of the disable parameters?

See: EE1.x Docs: Disable Parameters

I would also point you to the blog post that Greg recently wrote about troubleshooting performance issues

http://ellislab.com/blog/entry/troubleshooting_site_performance_issues/

Yes, and here’s a thread where a site owner decreased their queries by 40% and decreased their server load by 60%, using Greg’s pointers.

They were also under pressure from their host, mainly due to a monster page that their client had insisted upon. Suggestions for a redesign can speedily respond to that, now that the general performance looks good.

Ryan, in some ways I found yours one of the more interesting posts recently, for its outside directions. I looked at EAV especially, having once dealt once in related things.

But actually, I believe this is probably kind of a bum steer for you. EAV is an attempt to get at what an object database can naturally do, and it apparently in practice is very inefficient, just as one would expect. It and attributes themselves often turn out also to be overly complicated to deal with for what they might gain.

In fact, I believe Magento, the e-commerce outfit of some persuasion, has redesigned to remove their original EAV, even though I suspect it was one of the magics they thought to offer, and maybe was part of their name (Mage, this arrangement was called, if I’ve read that right).

As a functionality for deeply classified classifieds, attribute database could be good, if the practical aspects showed well. There have been excellent object databases, such as the one invented by the person who also invented the Scrum methodology Ellis has advantageously taken on.

I’m thinking, though, that search so often works better than deep attempts at classification. After all, if your tags get too complicated, who can realize what the right ones are?

Practical examples of doing better include traditional newspapers with carefully not so many classifications, and online discovery places like Craigslist and Ebay, where search is your actually powerful tool, and categories are used just to narrow the context enough so you don’t get too many un-wished-for results.

Thus I’m thinking that for your site, the first order is to be sure it’s really operating efficiently. Sue’s suggestion is spot on for removing the heavy database joins, and then you can go on to the other areas in Greg’s article.

After that, you might look into what more advanced Search add-on modules might be able to offer. There might be a little further advancement coming here also, as I hear people recognizing the importance and talking about needs. You can search (!) on Devot-ee.com to see where the useful add-ons are being made.

Good fortune, and I’m sure the community will be interested in how this goes for you, and what you come up with.

Regards,
Clive

Ryan, if you wanted to use a different table to store your ads in you certainly could, it doesn’t matter which version you chose at that point.

...to me this reads as
“If you want to use something other than EE you can….”

What is the benefit of using EE if we are dealing with different table structure?

Do post back if you apply the tricks in the article above, I am too very curious about results as I am getting ready to develop hybrid of intranet/job application website that could potentially bubble up to 1,000s of entries each holding a few file fields.
Thanks.

lebisol,

That’s not how I read it 😉

If you have a component of a website that requires a specialized table structure for a particular feature, that can certainly be done. Add-ons (or even just the Query module) can pull data from whatever tables they need, back into EE where you can display it using the standard EE tag format. The rest of the site would continue to store other data as channel entries.

Such is text based communication as well as me being a devil’s advocate 😉
Oh I see what you are saying and I am familiar with query module I thought there was more to it.

Hi Ryan,

Do the suggestions in this thread help you with this? Do you need further assistance?

Cheers

Greg

I’ll go ahead and close this one, Ryan, but feel free to post again if you have any other questions!