3 of 11
3
Plugin: FeedGrab
Posted: 29 January 2007 03:26 PM   [ Ignore ]   [ # 37 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

Thanks - I’ll take a look at the plugins you’ve listed and see if I can see any problems.

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 30 January 2007 07:56 AM   [ Ignore ]   [ # 38 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

I’ve added all those plugins (except the Amazon one that needs PHP5) to my install and don’t get any problems.

If you are happy dabbling with your EE installation you could try editing the file: system/cp/cp.utilities.php

On line 758 (on my EE install v.1.5.2) change:

if ( ! @include_once(PATH_PI.$file))

to

if ( ! include_once(PATH_PI.$file))

The @ sign suppresses errors, so removing it might gives us some clues to what’s breaking. Ta.

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 09 February 2007 03:39 AM   [ Ignore ]   [ # 39 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

OK - I’ve a new version of the FeedGrab plugin ready to release.

A couple of exciting (and slightly scary) new features:

Attributes

You can now access attribute values from feed elements, eg, flickr uses fields like:

<media:content url="http://farm1.static.flickr.com/116/301933412_fad89245a8_o.jpg"
                       
type="image/jpeg"
                       
height="1200"
                       
width="1600"/>

You can now access the url, type, etc attributes using media:content@url, media:content@type at the field name

User defineable ‘uniqueness’

Previously, the plugin always used the title and entry date to determine whether a feed entry already existed in the weblog. Now you can use any (combination) of your weblog fields to test. (Most feeds provide a global unique identifier for each entry - I’d suggest adding a guid field to your weblog and sticking the guid in there.)

The parameter is called unique and takes a comma separated list of weblog fields to test for duplicates.

(Note for those interested: this is my first step towards allowing entries to be updated)

Here’s an example plugin call (to flickr) using the new features:

{exp:feedgrab url="http://api.flickr.com/services/feeds/photos_public.gne?id=YOURIDHERE@N00&format=rss_200"
                          
weblog="2"
                          
title="title"
                          
date="dc:date.Taken"
                          
use="media:content@url|media:thumbnail@url|guid"
                          
fields="feeds_url|feeds_thumbnail|feeds_guid"
                          
unique="feeds_guid" }

There’s also a debug=“off” parameter that stops the plugin reporting what it is doing, if you would like it to run silently.

Edit: Please see the FeedGrab page to download current version

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 17 February 2007 05:05 AM   [ Ignore ]   [ # 40 ]  
Grad Student
Avatar
Rank
Total Posts:  92
Joined  12-12-2002

This is unbelievable.  I wish I had found it a year ago!!  Thank you.

 Signature 

Gridwork Design * Site design for small organizations

Profile
 
 
Posted: 20 February 2007 05:18 PM   [ Ignore ]   [ # 41 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  486
Joined  07-12-2005

I’m gonna cry.

The new version does the same thing. I get nothing in my plugin manager, and this is on a completely different (client’s) EE site and install…

 Signature 

Technology Consulting
Jobs, apartments, working and moving to Spain.

Profile
 
 
Posted: 21 February 2007 11:04 AM   [ Ignore ]   [ # 42 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

Hi wondermonkey,

I’m keen to find what’s wrong here (although I’m a bit snowed under at the moment, so excuse me if I’m slow to respond).

Could you email or PM me the results of: Control Panel > Admin > Utilities > PHP Info

That might give me a clue…

(The php change in post #38 above would be better, but I understand if you don’t want to go changing your codebase)

Andrew

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 13 March 2007 04:52 AM   [ Ignore ]   [ # 43 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  400
Joined  09-09-2006

Thanks for this - this is a really great. You should be proclaimed King for a week or at least a day!

A feature request [I’m sure you are fed up of these!] would be to have the categories from feeds dynamically inserted into the weblog category field. This is <dc:subject> in del.icio.us. Currently I think you need to create the categories before the feed comes in.

Of course you can get round this but it would be nice to have the categories assigned automatically.

I’m planning to map my delicious categories to my weblog categories and use them to serve targeted links according to posts. All doable with this great plugin and cron!

This plugin also breaks the plugin manager for me - on PHP 5.0.4 / Apache 2.0 / MySQL 5

Thanks again!

 Signature 

Shape Shed | Usable, elegant web design, London, UK
Twitter | Flickr | LastFm | Blog | EE addons

Profile
 
 
Posted: 19 March 2007 03:52 AM   [ Ignore ]   [ # 44 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

Sorry for the delayed reply…

I *think* the plugin can already do this (but it might not be documented).

There are 3 parameters:

cat_field - which would be, in your case, cat_field=“dc:subject”
cat_delimiter - I think del.icio.us categories are space separated, so use cat_delimiter=“SPACE”
cat_group - this is the id number of the category group you want to add new categories to (I cannot derive this anymore as weblogs can now have multiple category groups)

As I say, I *think* this should work smile

I need to try and fix the plugin manager problem. As I cannot reproduce it, can I contact you for some help when I get a chance to look at it?

Thanks,

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 19 March 2007 09:32 AM   [ Ignore ]   [ # 45 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  486
Joined  07-12-2005

Hey Andrew, I know I’ve been a little AWOL lately, but today I think I’m going to try to troubleshoot the plugin manager problem a bit myself. I’ll let you know if I figure it out.

 Signature 

Technology Consulting
Jobs, apartments, working and moving to Spain.

Profile
 
 
Posted: 19 March 2007 02:33 PM   [ Ignore ]   [ # 46 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  400
Joined  09-09-2006

Hi Andrew,

Yep no problem - let me know how I can help.

When I turn off the error supression I get:

Fatal error: Cannot redeclare class magpierss in /..snip../plugins/pi.magpie.php on line 0

I couldn’t get the categories to work I’m afraid. Here’s the code I’m using:

{exp:feedgrab url="http://del.icio.us/rss/dodland/dom"
                          
weblog="5"
                          
cat_group="1"
                          
cat_delimiter="SPACE"
                          
cat_field="dc:subject"
                          
title="title"
                          
date="pubDate"
                          
use="link|description"
                          
fields="delicious_url|delicious_body" }

It pulls in the links ok but doesn’t insert any categories.

Would be unbelievable if this worked - you could use a cron with wget to sync pretty much any rss feed with categories.

Still pretty good without categories though!

 Signature 

Shape Shed | Usable, elegant web design, London, UK
Twitter | Flickr | LastFm | Blog | EE addons

Profile
 
 
Posted: 20 March 2007 03:15 AM   [ Ignore ]   [ # 47 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

My mistake withe the parameter name, sorry. They should be category_ not cat_.

Your example would be:

{exp:feedgrab url="http://del.icio.us/rss/dodland/dom"
                          
weblog="5"
                          
category_group="1"
                          
category_delimiter="SPACE"
                          
category_field="dc:subject"
                          
title="title"
                          
date="dc:date"
                          
use="link|description"
                          
fields="delicious_url|delicious_body" }

Note, I think the date parameter should be dc:date for delicious feeds.

I think this should work now.

Thanks for the error message - I think I know what the problem is now (if not the solution yet).

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 20 March 2007 06:42 AM   [ Ignore ]   [ # 48 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  400
Joined  09-09-2006

Thanks Andrew

Works like a dream. This is really great.

I’m using a daily cron job to sync my delicious bookmarks each day.

wget -q http://www.pathtoyour/feedgrab_page

They are all categorised and in the db so I can do pretty much anything with them.

If you need any more help debugging let me know.

 Signature 

Shape Shed | Usable, elegant web design, London, UK
Twitter | Flickr | LastFm | Blog | EE addons

Profile
 
 
Posted: 21 March 2007 08:16 AM   [ Ignore ]   [ # 49 ]  
Summer Student
Avatar
Total Posts:  2
Joined  06-13-2006

First time poster and new to EE and I would like to first thank Andrew for the feedgrab plugin it totally saves me time for doing multiple things.

Second I keep seeing posts about Cron jobs and I installed the Expression Engine Cron but I can’t or I should say don’t understand how to implement it. Do I just create a blank template insert the cron plugin on it like this:

{exp:cron minute="*" hour="*" day="*" month="*" plugin="feedgrab"}{/exp:cron}

or do I stick it into the page template like this:

{exp:cron minute="*" hour="*" day="*" month="*" plugin="feedgrab"}
{exp
:feedgrab
url
="http://ma.gnolia.com/rss/full/groups/SocialMedia"
weblog="11"
title="title"
date="pubDate"
use="link|description|dc:creator|guid"
fields="magnolia_link|magnolia_body|magnolia_founder|feeds_guid"
debug="off"}

{exp
:feedgrab
url
="http://twitter.com/statuses/user_timeline/776411.rss"
weblog="19"
title="title"
date="pubDate"
use="link|description|guid"
fields="magnolia_link|summary|feeds_guid"
debug="off"}


{exp
:feedgrab
url
="http://api.flickr.com/services/feeds/photos_public.gne?id=98116400@N00&format=rss_200"
weblog="17"
title="title"
date="dc:date.Taken"
use="media:content@url|media:thumbnail@url|media:text@type|guid|media:title"
fields="full_photo|thumb_photo|summary|feeds_guid|subject"
debug="off"}
{
/exp:cron}

I could really use some help.

Thanks,
Cesar

 Signature 

Cesar H. Castro Jr.
Founder and Chief Creative Director
Standard Society
http://www.standardsociety.com

Profile
 
 
Posted: 22 March 2007 02:03 PM   [ Ignore ]   [ # 50 ]  
Grad Student
Rank
Total Posts:  34
Joined  03-06-2004

Am I missing something? I installed FeedGrab and set up a weblog and a template. I can bring in feeds with no problem and they are imported into the database. But when I try to click on a title under the Edit tab, I get an error message that I have tried to access a weblog that does not exist. Yet, when I go under the Publish tab and choose my feeds weblog, I can publish to it and also edit the entry. I just cannot get into the entries generated by FeedGrab. Is that how it is supposed to work?

Profile
 
 
Posted: 22 March 2007 02:16 PM   [ Ignore ]   [ # 51 ]  
Summer Student
Avatar
Total Posts:  2
Joined  06-13-2006

are you using the right weblog id number like weblog=“17” meaning the weblog you setup has the id number of 17. Or did you do what I did the first time around and wrote in the actual name of the blog. Try inserting the id number into the feedgrab script.

 Signature 

Cesar H. Castro Jr.
Founder and Chief Creative Director
Standard Society
http://www.standardsociety.com

Profile
 
 
Posted: 22 March 2007 04:29 PM   [ Ignore ]   [ # 52 ]  
Grad Student
Rank
Total Posts:  34
Joined  03-06-2004

Yes, I am using the weblog ID rather than the name.

Profile
 
 
Posted: 24 March 2007 12:23 PM   [ Ignore ]   [ # 53 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  486
Joined  07-12-2005
Andrew Weaver - 21 February 2007 11:04 AM

Hi wondermonkey,

I’m keen to find what’s wrong here (although I’m a bit snowed under at the moment, so excuse me if I’m slow to respond).

Andrew

Hey Andrew,
I did a bit of fiddling with the plugin code this morning and figured out that it’s definitely the declaration of the MagpieRSS class that causes the Plugin Manager to break. I tested all the rest of the code and it’s fine. If I remove everything in the MagpieRSS class (but not the class itself) I get no Plugin Manager, but as soon as I remove the

class MagpieRSS {

it comes back.

Since Magpie is already installed, is there some way to just import it instead of replicating it in FeedGrab?

 Signature 

Technology Consulting
Jobs, apartments, working and moving to Spain.

Profile
 
 
Posted: 24 March 2007 04:09 PM   [ Ignore ]   [ # 54 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  486
Joined  07-12-2005
Andrew Weaver - 10 January 2007 04:41 PM

It should be fixed in the download below - to make the parameter name a bit more consistent I’ve renamed it author rather than author_id.

So I was testing it out this morning and now it’s gone and done something real bad I think.

I imported a couple blog feeds and tried the parameter author=“author” because “author” is one of the xml items right? It imported them but the entries aren’t coming up on the Edit page so I can’t access them. There’s one entry that appeared in the Recent Entries list and when I check the Author of the entry it’s blank.

So two things I guess: how does the author parameter work and any ideas on fixing this? Since I can’t access the entries from the CP does that mean I have to go hunting through phpmyadmin?

Also, does the plugin grab tags from the feed’s title, link, description etc that aren’t items (about the feed and source, not the entries)?

UPDATE: I fixed it by running an UPDATE query to set author_id=‘1’ where author_id=’’ and now I can access the entries in the Edit panel of CP. Above question about how to use ‘author’ remains… wink

UPDATE II: After more testing I’m still unable to pull content from Atom feeds such as this one. The tag/item is called “content” but I end up with the word “Array” in my entry’s content field instead of the real content from the feed.

 Signature 

Technology Consulting
Jobs, apartments, working and moving to Spain.

Profile
 
 
   
3 of 11
3
 
Post Marker Legend
New Topic New posts Hot Topic Hot Topic with new posts New Poll New Poll Moved Topic Moved Topic Sticky Topic Sticky topic
Old Topic No new posts Hot Old Topic Hot Topic with no new posts Old Poll Old Poll Closed Topic Closed Topic Announcement Announcements
Theme
Change Theme
Visitor Statistics
The most visitors ever was 1149, on July 16, 2007 09:33 AM
Total Registered Members: 64533 Total Logged-in Users: 30
Total Topics: 81111 Total Anonymous Users: 19
Total Replies: 436427 Total Guests: 210
Total Posts: 517538    
Members ( View Memberlist )