The aim of this plugin it to allow you to grab data from RSS feeds and insert it into your own weblog.

The motivation behind it is to produce something similar to http://www2.jeffcroft.com/tumblelog/ where various sources of 3rd party information are brought together in a single place. On a simpler level it could be used to import links to, say, your flickr images so that the image appears as part of your weblog, on the correct day.

The plug in has only been tested for my own purposes, but I’m posting it here in case any one else might find it useful.

Basic usage is:

1) Create a new template

2) Add a call to the plugin, eg:

{exp:feedgrab url=“http://api.flickr.com/services/feeds/photos_public.gne?id=25509357@N00&format=rss_200”
weblog=“1”
title=“title”
date=“dc:date.Taken”
use=“link|description”
fields=“extended|body” }

(This tells the plugin to grab the feed and for each item in the feed put its link and description in the weblog entry’s extended and body custom fields, using the feed item’s title as the entry title and its dc:data.Taken field as the entry’s publish date. The weblog parameter allows you to specify which weblog to put the data in.)

3) Call the template, either by hand or through a cron job

A known issue is that it breaks the Plugin Manager! I’m pretty sure that this is because it ‘shares’ code with the magpie plugin. Any help to get around this would be appreciated…

Edit: Please see the FeedGrab page to download current version

Awesome!  This is just what I have been looking for.  Will it work with Atom feeds?

Thanks for sharing this plugin.

This plugin is totally awesome, and just what I was looking for to duplicate some of my Flickr photos on my photolog!

The only problem I have spotted with it so far, is that it sometimes creates duplicate entries. Not for everything, but maybe every other entry or so. The first time it shows up in the RSS feed, it gets added fine. But if I then run the plugin again at a later date, when there are new items in the feed as well as old items, it will add some of the old items as new entries *again*, even though they already exist within EE. The title and date are still exactly the same for them - any ideas why it would be adding some as duplicate entries? It’s a pain having to weed through it every time to delete the duplicate entries!

In case it helps any, I am using this RSS feed (linked from this page):

http://api.flickr.com/services/feeds/photos_public.gne?id=95239135@N00&tags=stillburningcom&format=rss_200

And here is an example of the output I got when it created several duplicate entries, while adding a few new ones as well:

Other than that, thanks a ton for creating this plugin! :D

asarbogast - 25 September 2006 11:14 PM

Will it work with Atom feeds?

It should work with Atom feeds. It uses the Magpie RSS library (version 0.6a) which claims to support RSS 0.9x, RSS 2.0, RSS 1.0, and Atom 0.3.

Sasha - 01 October 2006 01:41 AM

The only problem I have spotted with it so far, is that it sometimes creates duplicate entries.

I’m sure there will be more problems than that!

Rather than checking for duplicate entries by title, the plugin attempts to look for entries created more recently than the previous time the plugin was called. This doesn’t seem to work very well, especially across different timezones and with daylight savings. It also requires the site that generates the feed gets its timestamp right, which I’ve noticed doesn’t always happen.

I’ll see if I can improve the method of checking for duplicates.

Many thanks for your feedback.

Thanks Andrew! For me personally, checking on entry title would be perfect - or possibly a combination of entry title and entry date…my photos would sometimes have the same title, but they’d all have different “taken on” dates, even if it’s just by minutes or seconds. If you need someone to do some testing, please don’t hesitate to let me know!

Sasha - 02 October 2006 03:41 AM

checking on entry title would be perfect - or possibly a combination of entry title and entry date

This version checks title and entry date. Its relatively untested, but doesn’t do anything dangerous - it just checks for matching titles and dates before adding a new entry.

Andrew

Edit: Please see the FeedGrab page to download current version

Andrew, does this new version still break the plugin manager?

Yes, the plugin manager still breaks.

I’m pretty sure it is because the plugin manager uses the same Magpie RSS code as the FeedGrab plugin. It would be nice to share the Magpie plugins code somehow, but I could code around this, I guess.

OK, another update…

I think this fixes the problem with the plugin manager not showing. The RSS code is still duplicated between this plugin and the Magpie one, but FeedGrab now checks to make sure it doesn’t redeclare the same functions (which was causing the error).

You really do need to fix that. I use the plugin manager all the time, so not being able to get to it is a major problem.

Edit: Works now! Thanks for fixing it.

Sue Crocker - 02 October 2006 07:21 AM

Works now! Thanks for fixing it.

Thanks for the feedback, Sue.

Andrew - thanks for the fix earlier! I downloaded the plugin and ran it, but it had now completely stopped adding any entries. I made one small change in the source code that fixed that for me though. I changed line 183 to:

The < was the other way around first (>). I’m a bit confused regarding the logic around that…but it works for me now, even if I can’t figure out why. Thanks again!

Thanks Sasha.

The plugin is trying to check a timestamp so that previously added entries aren’t added again, but this test is now superceded by the test on entry name and date.

I’ll remove the timestamp check completely from the next version and just rely on checking the title and date. It hits the database a little more - which is why I didn’t do it initially - but it will work better.

I’ve removed any attempt to check timestamp - they were too problematic. A new version of the plugin is available below:

One question: Can you run magpie at the same time? I am getting these error messages…

Fetching: feed://news.google.com/news?svnum=10&as_scoring=r&hl=en&ned=&ie=UTF-8&ned=:ePkh8BM9E8JmByvQDgNWHLYAAJLgBts&as_drrb=q&as_qdr=&as_mind=20&as_minm=1&as_maxd=19&as_maxm=1&q=design+toronto&output=atom&ned=:ePkh8BM9E8JmByvQDgNWHLYAAJLgBts

Warning: MagpieRSS: Failed to fetch feed://news.google.com/news?svnum=10&as_scoring=r&hl=en&ned=&ie=UTF-8&ned=:ePkh8BM9E8JmByvQDgNWHLYAAJLgBts&as_drrb=q&as_qdr=&as_mind=20&as_minm=1&as_maxd=19&as_maxm=1&q=design+toronto&output=atom&ned=:ePkh8BM9E8JmByvQDgNWHLYAAJLgBts. (HTTP Error: Invalid protocol “feed”) in /home/xxx/plugins/pi.feedgrab.php on line 1257

Notice: Constant RSS already defined in /home/xxx/plugins/pi.magpie.php on line 553

Notice: Constant ATOM already defined in /home/xxx/plugins/pi.magpie.php on line 554

Fatal error: Cannot redeclare class magpierss in /home/xxx/plugins/pi.magpie.php on line 557

Oops, bad URL. No more error messages but it is not populating the blog for some reason.

Thanks to all of you who have sent in reports that it does not seem to be adding the entries to your weblogs.

To help me fix this I’d be very grateful if can you send me some more details so I can try and replicate the problem.

It would be useful to know:

any error messages you are getting
what version of EE you are using
the contents of template you are using to call FeedGrab
details of the weblog (eg, field names) you are trying to insert the entry into

You’re welcome to email these if you don’t want your details on the forum.

Many thanks again to everyone testing this.

Hi Andrew:

Terrific concept, and it’s almost there!  I’ve tried dragging the info out of my last.fm rss feed into my weblog, and I keep running into a problem in which the template will grab the first 6 entries then stop.  I’ve posted the output below:

First run with zero entries in weblog:

Subsequent runs:

Those songs in the above listing were from approx 1 hour ago.

Deleting the entries from the weblog resets the process of successfully grabbing the first six entries and stalling.

Hope this helps!