Thread

Hi everyone,

I am using AJW Feed Parser (http://devot-ee.com/add-ons/feed-parser/) to parse an XML feed into my EE templates. All going fine except I’m experiencing slow loading times when fetching 24 objects of /root/advertentie of the following feed: http://www2.autodealers.nl/advertentieXml.aspx?did=2756

This is the code I am using:

{exp:ajw_feedparser 
    url="http://www2.autodealers.nl/advertentieXml.aspx?did=2756"
    itempath="/root/advertentie"
    cache_refresh="60"
    limit="{embed:per_page}"
    offset="{embed:xml_offset}" 
    parse="inward"
}
[some html and variables here]
{/exp:ajw_feedparser}

It’s taking Expression Engine about 4.5 - 6 seconds to load the page and the while using a limit of 24 objects, it doesn’t really matter how much objects I fetch.

Debug output shows the delay is clearly within the AJW Feed Parser:

(3.545572)      Calling Class/Method: Ajw_feedparser/
(3.545679)       -> Class Called: Ajw_feedparser
(5.211717)       -> Method Called: Ajw_feedparser
(5.211856)       -> Data Returned

Is there any way to speed up this process or am I just loading to much data into the feed parser at once?

Are you sure the delay is not with the remote site?

Are you sure the delay is not with the remote site?

Well the XML-feed seems to load quite quickly. Or is that not what you mean?

I have been looking at DataGrab with Cron as an alternative, but this solution is all set up and ready as we speak, so it’s definitely got my preference now. It’s just that it’s (too) slow.

You could try loading the feed in a browser that presents XML, like Google Chrome.

I’m getting times pretty much in your range (or slower) on that feed, read this way.

If you don’t go faster, being nearer the source, it may be just because on a dataset of its size, the (infernal 😉 ) internal consistency-checking of an XML parser takes resources.

Regards,
Clive

MH1985, I had another look at this, as I wanted to check out this feed parser for another purpose.

- first, I would say your numbers don’t indicate the feed parser as a great source of delay. Those timings indicate that before it has been called, there has already been 3.5 seconds of delay, which is a lot. The add-in parser is adding 1.7 seconds, which under the load, and again considering what it and the system library parser it calls have to do for that large feed you use (it is 300K bytes), sounds not out of order.

- Then, for 24 items of that auto feed, you also need to be reckoning the latency time to go out and check each of the the images, depending on how many of those you are using per item. That adds up, since browser open only a few sockets at a time. And for the initial downloads, much more time will be used. Question: do you really need to do 24 items on a page? That’s a lot of cars to look at.

- Still, making a test page using 6 images (when there are six) and 24 items of autos, I am getting quite reasonable response testing here. The delta time for the parser is about 0.9 seconds—half yours. The difference is probably just due to a more sensible (lighter, managed) host loading.

- Finally, you can speed the parsing page response up a lot, by using template caching. I am going from 0.9 sec overall time to 0.045 for the page itself. There is the PHP startup load as usual when not on fastcgi, so total times as indicated by Pingdom go from about 1.6 sec down to about 0.65.

If you want to try this method, you’ll need to think about the template cache time vs. required following accuracy to the feed. You may want to set the internal cache time for the parse tag itself to be very short, so that it will be past each time the template cache wants to update.

A bit of monitoring will be in order to be sure you’ve arranged various cachings to act as you expect them to. You can manually look at the feed for changes, using a web browser which interprets XML. You can check internal cache for update by looking at its date using ftp (be sure to refresh the view each time).

Hoping these thoughts turn out helpful.

Regards,
Clive

MH1985, I had another look at this, as I wanted to check out this feed parser for another purpose.

- first, I would say your numbers don’t indicate the feed parser as a great source of delay. Those timings indicate that before it has been called, there has already been 3.5 seconds of delay, which is a lot. The add-in parser is adding 1.7 seconds, which under the load, and again considering what it and the system library parser it calls have to do for that large feed you use (it is 300K bytes), sounds not out of order.

- Then, for 24 items of that auto feed, you also need to be reckoning the latency time to go out and check each of the the images, depending on how many of those you are using per item. That adds up, since browser open only a few sockets at a time. And for the initial downloads, much more time will be used. Question: do you really need to do 24 items on a page? That’s a lot of cars to look at.

- Still, making a test page using 6 images (when there are six) and 24 items of autos, I am getting quite reasonable response testing here. The delta time for the parser is about 0.9 seconds—half yours. The difference is probably just due to a more sensible (lighter, managed) host loading.

- Finally, you can speed the parsing page response up a lot, by using template caching. I am going from 0.9 sec overall time to 0.045 for the page itself. There is the PHP startup load as usual when not on fastcgi, so total times as indicated by Pingdom go from about 1.6 sec down to about 0.65.

If you want to try this method, you’ll need to think about the template cache time vs. required following accuracy to the feed. You may want to set the internal cache time for the parse tag itself to be very short, so that it will be past each time the template cache wants to update.

A bit of monitoring will be in order to be sure you’ve arranged various cachings to act as you expect them to. You can manually look at the feed for changes, using a web browser which interprets XML. You can check internal cache for update by looking at its date using ftp (be sure to refresh the view each time).

Hoping these thoughts turn out helpful.

Regards,
Clive

Hi Clive,

First of all, thank you for your extensive post. The 3.5 seconds you mention is indeed a lot, but I haven’t turned any caching on yet so there is time to gain. Same goes for the hosting.

The images however are an absolute must. I am using Causing Effect’s image plugin to display remotely hosted images, and since the page is an occasion overview page, I really can’t do without them. There’s 24 occasions on a page and visitors are encouraged to click on images more than text links. Once cached however, this plugin works like a treat.

I’ll give the caching part a try and see what can be improved. Thanks again 😊