13 of 15
13
Plugin: FeedGrab
Posted: 08 April 2009 07:47 AM   [ Ignore ]   [ # 217 ]  
Lab Assistant
RankRank
Total Posts:  154
Joined  04-04-2004

Hi Andrew,
your plugin is really great. Lately I ran into some problems and tried to find the answer in the forums.
I found this post in this thread

Notice: Undefined index: date in ~/pi.feedgrab.php on line 221

My problem is similar. Feedgrab reports

Notice: Undefined index: date in /plugins/pi.feedgrab.php on line 569

All feeds are coming from Feedburner. feedgrab grabs them, but there are issues with the date, because when I use the exp:weblog:entires tag as shown below, the order of the items is wrong. When I clear the database and let feddgrab pull the feeds again sometimes the order is correct and sometimes not.

{exp:weblog:entries weblog="news" orderby="date" limit="10" show_future_entries="yes" author_id="2"}

I checked the source of the feeds and replaced some instances of

date="pubDate"

with date=“published” or date=“updated”, since some feeds seem to use this tags for the date. But I had no luck.

So, I´m wondering, if somebody can help me here.

Thanks
awa

Profile
 
 
Posted: 23 April 2009 11:49 PM   [ Ignore ]   [ # 218 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  522
Joined  10-09-2007

Thank you for creating FeedGrab!  I used it with some additional rss fields like comments (used to hold a phone number), needed to bring over more custom fields from one install to another.  That is how I needed to use it this time, I set up an RSS feed for my listings on one site and grabbed what I needed and it work seemlessly!!!

 Signature 

Vancouver Travel & Tourism http://www.discovervancouver.com/

Profile
 
 
Posted: 29 April 2009 09:06 AM   [ Ignore ]   [ # 219 ]  
Grad Student
Rank
Total Posts:  50
Joined  04-16-2008

sorry for cross post to “how to”
10 days and no response so..

I’ve got a Feedgrab issue where a twitter feedgrab import is not working as it should when output to weblog:entries

I have the “FeedGrab” plugin pulling in data from twitter.

it looks more or less like this:

{exp:feedgrab url="http://twitter.com/statuses/user_timeline/11111111.rss"
weblog="21"
title="title"
date="pubDate"
use="link|description"
fields="twtlink|twtbody"
unique="twtlink" }

twtlink and twtbody are custom fields within a field group dedicated to twitter feeds.

twtbody is a text input field of 161 characters w text formatting set to none
twtlink is a text input field of 256 characters w text formatting set to xhtml


the import appears to succeed pulling in 20 or so items, generating a response in the following format:

Fetching: http://twitter.com/statuses/user_timeline/111111.rss
Checking: twitterusername: twitter post data
Found new entry: twitterusername: twitter post data

the data is visible and editable in the CP.
new posts may be entered directly in the CP

the issue comes when I try to use it in an entry form as follows:

{exp:weblog:entries weblog="tweet_feed" orderby="date" limit="10" dynamic="off" show_future_entries="yes"}

{title}
<br/>
{twtbody}<br/>
{twtlink}
{
/exp:weblog:entries}

entries entered directly in the CP render fine.
entries pulled in via FeedGrab
- while they look fine on import and in the CP -
do not render…

any bright ideas as to what is happening and how to solve it?

thanks!

Profile
 
 
Posted: 12 May 2009 04:45 PM   [ Ignore ]   [ # 220 ]  
Grad Student
Avatar
Rank
Total Posts:  59
Joined  01-15-2007

Has anyone been using this plugin to make entries from an RSS feed and relate those entries to existing entries?

I guess you could do this on by writing the exact numbers of a relate entry (if all of the things in the feed relate to that entry). But has anyone done this dynamically? I’d like to be able to relate entries I pull from a RSS feed individually to different entries on my site, but I’m not sure this has the capacity to do this.

Todd

 Signature 

Todd Jackson

Dead-Frog - A Comedy Blog
http://www.dead-frog.com
Home of the Stand-Up Comic Database

Profile
 
 
Posted: 12 May 2009 04:50 PM   [ Ignore ]   [ # 221 ]  
Summer Student
Total Posts:  15
Joined  01-08-2009

The way I’ve done this so far:

Created one weblog for channel level information.  Wrote a plugin to update that based on some parameters.

Created a second weblog for item level information.  I use Feedgrab here to pull in items and create the database entries.

I modified Feedgrab to accept another parameter, the entry_id for the Feed level weblog item that it is the child of.  I also added the ability for it to make a relationship between the item level weblog entries and the feed level weblog entries.

In the page that updates (which I’m going to set as a cron job), I loop through the entities of the Feed weblog, and dynamically pass the parameters (feed url, feed weblog entry id) to the Feedgrab plugin.  So basically, all you have to enter in the Feed level info is a title and the url of the feed, and it will pull in the channel level and child item level information.

Profile
 
 
Posted: 12 May 2009 06:06 PM   [ Ignore ]   [ # 222 ]  
Grad Student
Avatar
Rank
Total Posts:  59
Joined  01-15-2007

An incredibly detailed response to something I felt was possibly garbled and unintelligible. Bafadam - if you’re up for your sharing your mods to the plugin, I’d love to see them.

In between, I attempted to see if an embed would make this work, but no such luck.

It seems like this would require some soft of rewrite of the actual plugin to make what it do what I want it to.

Unless anybody else has an answer of how one could automatically relate pulled in feed entries dynamically to different weblog entries with the plugin as is.

Todd

bafadam - 12 May 2009 04:50 PM

The way I’ve done this so far:

Created one weblog for channel level information.  Wrote a plugin to update that based on some parameters.

Created a second weblog for item level information.  I use Feedgrab here to pull in items and create the database entries.

I modified Feedgrab to accept another parameter, the entry_id for the Feed level weblog item that it is the child of.  I also added the ability for it to make a relationship between the item level weblog entries and the feed level weblog entries.

In the page that updates (which I’m going to set as a cron job), I loop through the entities of the Feed weblog, and dynamically pass the parameters (feed url, feed weblog entry id) to the Feedgrab plugin.  So basically, all you have to enter in the Feed level info is a title and the url of the feed, and it will pull in the channel level and child item level information.

 Signature 

Todd Jackson

Dead-Frog - A Comedy Blog
http://www.dead-frog.com
Home of the Stand-Up Comic Database

Profile
 
 
Posted: 13 May 2009 12:35 PM   [ Ignore ]   [ # 223 ]  
Summer Student
Total Posts:  15
Joined  01-08-2009
toddajackson - 12 May 2009 06:06 PM

An incredibly detailed response to something I felt was possibly garbled and unintelligible. Bafadam - if you’re up for your sharing your mods to the plugin, I’d love to see them.

They’re neither fancy nor flexible.  The first thing I did was write a completely separate function that swaps out custom field names for actual field names.

function customFieldSwap($arrCustomFields, $weblogId)
    
{
        
global $DB;
        
        
// Get custom fields from database
        
$customfieldsArray = $DB->query("SELECT exp_weblog_fields.field_id,
                                exp_weblog_fields.field_name,
                                exp_weblog_fields.field_label, exp_weblog_fields.field_fmt
                                 FROM exp_weblogs, exp_weblog_fields
                                 WHERE exp_weblogs.field_group = exp_weblog_fields.group_id
                                AND exp_weblogs.weblog_id = '"
.$weblogId."'");
        
        
// Map weblog custom fields to id's
        
$data = Array();
    
        foreach (
$customfieldsArray->result as $row ) {
        
            
if (array_key_exists($row['field_name'], $arrCustomFields))
            
{
                $fieldid
= 'field_id_' . $row['field_id'];
                
$fieldname = $row['field_name'];
                
$data[$fieldid] = $arrCustomFields[$fieldname];            
            
}
        }
        
return $data;
    
}

It accepts an array of (customFieldName => customFieldData) and returns an array with (actualFieldName => actualFieldData).  It’s not very flexible, though.  Since title, pubdate, etc aren’t custom field names, they won’t get swapped, so if you’re using those fields in there, add them after you do your swapping.  I suppose it could be updated to deal with system fields, but I’m honestly too lazy for that business, so… moving on.

Andrew is a better programmer than I am.  So my code is infinitely less flexible than his, so bear that in mind as you continue reading on.

I added a new class variable called $parent_weblog_id.  I created a new fetch param line for, you guess it, $parent_weblog_id.  When you call your function, it’s now looking for the entry id of the related (I keep calling it parent) weblog entry.

Since I’m looping through the Feed level entries, the idea is that I’ve got access to this field pretty handily and can just pass it in real easy-like.  Like I said, inflexible.

After this line:

$sql = $DB->insert_string('exp_weblog_data', $data);
                if (
$this->runsql ) {
                    $DB
->query($sql);
                
} else {
                    
if ( $this->debug ) { print $sql . "<br/>\n"; }
                }

I added in my relationship creation code:

//modification time
                //Let's add our relationship code here.
                //This is a two step process:  1. create the relationship and 2. update the feed item.
                
$sql = '';
                
                
$relData = array(
                    
'rel_parent_id' => $entry_id,
                    
'rel_child_id' => $this->parent_weblog_id,
                    
'rel_type' => 'blog'    
                
);
                
                
$sql = $DB->insert_string('exp_relationships', $relData);
                
$relID = '';
                
                if (
$this->runsql ) {
                    $DB
->query($sql);
                    
$relID = $DB->insert_id;
                
} else {
                    
if ( $this->debug ) { print $sql . "<br/>\n"; }
                }
                
                
//Apparently, our relationship thing was successful, so let's update the weblog data now.
                
if ($relID != '') {
                    $relData
= array(
                        
'feed_id' => $relID
                    
);
                    
                    
$relDataSwap = $this->customFieldSwap($relData, 3);
                    
                    
$sql = $DB->update_string('exp_weblog_data', $relDataSwap, 'entry_id = ' . $entry_id);
                
                    if (
$this->runsql ) {
                    $relID
= $DB->query($sql);
                    
} else {
                        
if ( $this->debug ) { print $sql . "<br/>\n"; }
                    }
                    
                
                }

Andrew, this whole thing is your creation.  If me posting any mod code to it bothers you, just let me know and I’ll yank it.  You did some awesome work with this whole thing and I don’t want to piss you off.  :)

Profile
 
 
Posted: 21 May 2009 05:37 AM   [ Ignore ]   [ # 224 ]  
Summer Student
Total Posts:  20
Joined  05-28-2008

I have used XMLGrab to import content from an old CMS to EE, and the entries are showing in EE fine - everything’s there that should be, and I have been able to go in and edit an ‘old’ entry to make sure all the information we now want is there.

However, then on creating a template to attempt to display entries, I am getting the following errors:

Warning: Template::require_once(./admin/modules/weblog/mod.weblog.php) [function.Template-require-once]: failed to open stream: No such file or directory in /home/[...]/admin/core/core.template.php on line 991

Fatal error
: Template::require_once() [function.require]: Failed opening required './admin/modules/weblog/mod.weblog.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/[...]/admin/core/core.template.php on line 991

It doesn’t matter which weblog I am trying to show, and even if I have {title} as the only tag between the weblog tags - this is what I get.

I’ve read through this thread seeing if anyone else has had similar issues, but it appears not - EEK!

Profile
 
 
Posted: 26 May 2009 02:20 PM   [ Ignore ]   [ # 225 ]  
Grad Student
Rank
Total Posts:  48
Joined  09-29-2008

can anyone help me get feed grab working?  my blog host seems to have changed something with the Atom feed and now i get “array” populated in my “Content” custom weblog.

here is the link to the feed:
http://www.chicagonow.com/blogs/whats-it-worth/atom.xml


here is my import template:

{exp:feedgrab url="http://www.chicagonow.com/blogs/whats-it-worth/atom.xml"
                          
weblog="22"
                          
title="title"
                          
unique=“guid”
                          date
="published"
                          
use="link|content"
                          
fields="ext_link|content" }
<h1>Feed import finished</h1>
Profile
 
 
Posted: 09 June 2009 08:21 AM   [ Ignore ]   [ # 226 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  509
Joined  05-14-2004

This plugin is great.  It is 99% or what I needed.  I am pulling my tweets into my blog as entries so people can possibly comment and I have a record of my daily life.

The biggest issue I have is actually related to the twitter feed itself and I was hoping that FeedGrab had some kind of workaround and or feature request.

title=“title”  This parameter reads the title of the RSS feed and creates a new title for the entry, which is fine except that twitter places the entire tweet in the title and description.  Is there a way to put a custom title in that field?  ie title=“Tweeted on pubDate” so my titles make sense when the entries are created?  Possibly allow for EE style date formatting.

The other minor issue I have is that the feed publishes the author automatically, ie ffpblog:tweet.  Is there a way to get it to erase everything up to the first : since that seems to be how Twitter formats the feed?

I am currently using twitter timeline to display tweets, but it would be so much better to have them created as entries within the blog so I can capture them for posterity.

thanks.

 Signature 

Far From Perfect
Broadcasting live from Iraq via High-speed Bytes/Second Internet!

Profile
 
 
Posted: 13 June 2009 06:22 PM   [ Ignore ]   [ # 227 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  485
Joined  10-10-2004

I have the same problem as below - pulling in RSS from a twitter feed. Looks fine in CP. Doesn’t output anything. I tried adding site=”” parameter just in case but no luck. This is not a multisite setup.

dreas - 29 April 2009 09:06 AM

sorry for cross post to “how to”
10 days and no response so..

I’ve got a Feedgrab issue where a twitter feedgrab import is not working as it should when output to weblog:entries

I have the “FeedGrab” plugin pulling in data from twitter.

it looks more or less like this:

{exp:feedgrab url="http://twitter.com/statuses/user_timeline/11111111.rss"
weblog="21"
title="title"
date="pubDate"
use="link|description"
fields="twtlink|twtbody"
unique="twtlink" }

twtlink and twtbody are custom fields within a field group dedicated to twitter feeds.

twtbody is a text input field of 161 characters w text formatting set to none
twtlink is a text input field of 256 characters w text formatting set to xhtml


the import appears to succeed pulling in 20 or so items, generating a response in the following format:

Fetching: http://twitter.com/statuses/user_timeline/111111.rss
Checking: twitterusername: twitter post data
Found new entry: twitterusername: twitter post data

the data is visible and editable in the CP.
new posts may be entered directly in the CP

the issue comes when I try to use it in an entry form as follows:

{exp:weblog:entries weblog="tweet_feed" orderby="date" limit="10" dynamic="off" show_future_entries="yes"}

{title}
<br/>
{twtbody}<br/>
{twtlink}
{
/exp:weblog:entries}

entries entered directly in the CP render fine.
entries pulled in via FeedGrab
- while they look fine on import and in the CP -
do not render…

any bright ideas as to what is happening and how to solve it?

thanks!

Profile
 
 
Posted: 13 June 2009 06:58 PM   [ Ignore ]   [ # 228 ]  
Grad Student
Rank
Total Posts:  50
Joined  04-16-2008

code example?
charset?

Rob Quigley - 13 June 2009 06:22 PM

I have the same problem as below - pulling in RSS from a twitter feed. Looks fine in CP. Doesn’t output anything. I tried adding site=”” parameter just in case but no luck. This is not a multisite setup.

dreas - 29 April 2009 09:06 AM

sorry for cross post to “how to”
10 days and no response so..

I’ve got a Feedgrab issue where a twitter feedgrab import is not working as it should when output to weblog:entries
.......

Profile
 
 
Posted: 24 June 2009 04:49 AM   [ Ignore ]   [ # 229 ]  
Grad Student
Avatar
Rank
Total Posts:  33
Joined  03-08-2008

Hi bafadam

Are you able to post your template code and the exact settings you made to the two weblogs?

I’ve tried inserting your changes into pi.feedgrab.php but I don’t think I’m quite there yet. Feedgrab was working fine and inserting entries but I can’t get them to associate themselves with their feeds.

I have made custom field in my posts weblog called ‘feed_id’ which is a relationship field pointing at the feeds weblog

I have:

{exp:feedgrab url="{resource_rss_url}"
                          
weblog="9"
                          
title="title"
                          
feed_id="{entry_id}"
                          
post_date="pubDate"
                          
use="link|description"
                          
fields="post_url|post_description"}

...but I’m getting an SQL error

(Description: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘WHERE entry_id = 187’ at line 1
Query: UPDATE `exp_weblog_data` SET WHERE entry_id = 187)

Any help really appreciated!

 Signature 

We’ve just launched myGengo | Simple human translation

Moresided | Creative things with new technology, in Japan and the UK.

Profile
 
 
Posted: 24 June 2009 08:39 AM   [ Ignore ]   [ # 230 ]  
Summer Student
Total Posts:  15
Joined  01-08-2009
{exp:feedgrab url="http://themancast.spookyouthouse.com/feed"
                  
weblog="3"
                  
title="title"
                  
pubdate="pubdate"
                  
use="link|description"
                  
fields="feed_link|feed_description"
                  
parent_weblog_id="2"}

I think you’re missing the “parent_weblog_id” parameter there, chief.  I could export my weblog settings for you, but that’s a little more work than I think I need to do.  Try the parent_weblog_id and let me know if that works for you.

Profile
 
 
Posted: 24 June 2009 11:15 AM   [ Ignore ]   [ # 231 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  827
Joined  12-01-2002

Is parent_weblog_id equal to site_id? I’ve never heard of it before.

Profile
 
 
Posted: 24 June 2009 11:18 AM   [ Ignore ]   [ # 232 ]  
Summer Student
Total Posts:  15
Joined  01-08-2009
allgood2 - 24 June 2009 11:15 AM

Is parent_weblog_id equal to site_id? I’ve never heard of it before.

He’s using the mod that I posted, not the standard.  It was just an example on how you could relate one weblog to another via the code that was already posted.

Profile
 
 
Posted: 24 June 2009 06:42 PM   [ Ignore ]   [ # 233 ]  
Grad Student
Avatar
Rank
Total Posts:  33
Joined  03-08-2008

Thanks for replying.
Have added parent_weblog_id but I’m still getting the same error.
Also noticed in my code I have this parameter | feed_id=”{entry_id}” | whereas you don’t.
Thinking either my custom fields are wrong or something else I don’t know.

The mySQL error is in the ‘SET WHERE’ bit which I think happens here:

$sql = $DB->update_string('exp_weblog_data', $relDataSwap, 'entry_id = ' . $entry_id);

Hmm…

 Signature 

We’ve just launched myGengo | Simple human translation

Moresided | Creative things with new technology, in Japan and the UK.

Profile
 
 
Posted: 29 June 2009 11:18 PM   [ Ignore ]   [ # 234 ]  
Grad Student
Avatar
Rank
Total Posts:  88
Joined  07-18-2007

Is there a way to specify particular keywords, and have feedgrab only grab articles that include those keywords?

Thanks for a great plugin!

Profile
 
 
   
13 of 15
13
 
‹‹ form_helper 1.9 bugs      Widon’t ››
Post Marker Legend
New Topic New posts Hot Topic Hot Topic with new posts New Poll New Poll Moved Topic Moved Topic Sticky Topic Sticky topic
Old Topic No new posts Hot Old Topic Hot Topic with no new posts Old Poll Old Poll Closed Topic Closed Topic Announcement Announcements
Theme
Change Theme
Visitor Statistics
The most visitors ever was 1743, on December 02, 2009 03:47 PM
Total Registered Members: 120007 Total Logged-in Users: 98
Total Topics: 126139 Total Anonymous Users: 70
Total Replies: 663535 Total Guests: 470
Total Posts: 789674    
Members ( View Memberlist )