9 of 14
9
Plugin: CSVGrab
Posted: 28 June 2008 09:50 AM   [ Ignore ]   [ # 145 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  187
Joined  08-03-2007

Question:  Will the plugin allow for “updating” an original imported list?  Or, do entries have to be deleted and re-imported to update information? (if, for example, we had multiple changes to multiple entries)

 Signature 

Paul Burton | 16toads Design

Profile
 
 
Posted: 28 June 2008 11:05 AM   [ Ignore ]   [ # 146 ]  
Grad Student
Rank
Total Posts:  70
Joined  10-18-2002

CSVGrab will write over an existing entry [WRONG: see reply #151] if the “unique” field you specify matches the same field in the existing entry. In your case, this is “official-name”. So you can reimport multiple times.

You may find it helpful to set trace=“TRUE” (actually trace=anything). This makes CSVgrab print out some info about what it is doing during the import.

Profile
 
 
Posted: 30 June 2008 06:06 PM   [ Ignore ]   [ # 147 ]  
Lab Assistant
RankRank
Total Posts:  152
Joined  09-28-2002

I had problems importing a CSV file with about 450 rows.

Sample CSV file:

Name,Last Name,Email Address,Phone,ID,Contact ID,Owner ID,Date,In Directory?
"John Doe","Doe","john.doe@example.com","800-555-1212","","246","","2003-07-15","Y"
"Jane Doe"
,"Doe","jane.doe@example.com","800-555-1212","","98","75","2001-12-05","N"

CSVGrab template:

{exp:csvgrab url="http://www.example.com/import/azlist_contacts.csv"
    
delimiter=","
    
encloser="QUOTE"
    
trace="TRUE"
    
site_id="1"
    
weblog="54"
    
author="1"
    
title="1"
    
date="8"
    
skip="1"
    
use="2|3|4|5|6|7|9"
    
fields="azlist_contact_last_name|azlist_contact_email_address|azlist_contact_phone_number|azlist_contact_ucid|azlist_contact_contact_id|azlist_contact_owner_id|azlist_contact_in_directory"
}

Import template results:

TRACE: Array ( [title] => Array ( [is_custom] => 0 [field] => 1 ) [date] => Array ( [is_custom] => 0 [field] => 8 ) [azlist_contact_last_name] => Array ( [is_custom] => 1 [field] => 2 [id] => 208 [format] => none ) [azlist_contact_email_address] => Array ( [is_custom] => 1 [field] => 3 [id] => 202 [format] => none ) [azlist_contact_phone_number] => Array ( [is_custom] => 1 [field] => 4 [id] => 203 [format] => none ) [azlist_contact_ucid] => Array ( [is_custom] => 1 [field] => 5 [id] => 206 [format] => none ) [azlist_contact_contact_id] => Array ( [is_custom] => 1 [field] => 6 [id] => 213 [format] => none ) [azlist_contact_owner_id] => Array ( [is_custom] => 1 [field] => 7 [id] => 214 [format] => none ) [azlist_contact_in_directory] => Array ( [is_custom] => 1 [field] => 9 [id] => 215 [format] => none ) )

TRACE:

TRACE: 2 -

No records were imported.

Profile
 
 
Posted: 30 June 2008 07:33 PM   [ Ignore ]   [ # 148 ]  
Grad Student
Rank
Total Posts:  70
Joined  10-18-2002

Instead of this:

TRACE:

TRACE: 2 -

you should see something like this:

TRACE: Resource id #47

TRACE: 1 - Array ( [0] => John Doe [1] => Doe ...)

The “Resource id” trace is a handle to the input file, and the second trace shows you the contents of the csv line it just read.

So it looks like it is not reading your CSV file at all. Test the URL to make sure it’s right, and assuming it works, you can instead try pointing to a local file instead of using a URL. The path is relative to the your top-level web directory (e.g. public_html/ or www/), or, use an absolute path (/home/youraccount/public_html/foo.csv) or whatever is appropriate for your webhosting account.

Profile
 
 
Posted: 01 July 2008 10:13 AM   [ Ignore ]   [ # 149 ]  
Lab Assistant
RankRank
Total Posts:  152
Joined  09-28-2002

I changed it to an absolute path and it loaded the data into the template, but it didn’t import the data into the database.

Profile
 
 
Posted: 01 July 2008 11:04 AM   [ Ignore ]   [ # 150 ]  
Grad Student
Rank
Total Posts:  70
Joined  10-18-2002

Can you include some more of what the template showed in its debugging output? How about the TRACE’s? Were there any lines that said ‘Checking” or “Found new entry: ...”? Maybe try a shorter version of the data (only a few rows) and include the complete output.

Profile
 
 
Posted: 04 July 2008 03:10 PM   [ Ignore ]   [ # 151 ]  
Grad Student
Rank
Total Posts:  98
Joined  06-04-2006

Dan, your answer about using CSVGrab to update existing entries, above:

>>
CSVGrab will write over an existing entry if the “unique” field you specify matches the same field in the existing entry. In your case, this is “official-name”. So you can reimport multiple times.
<<

I don’t think this is correct. The documentation states the opposite, actually:

“The unique field determines which fields are checked to see if the current feed element has already been imported e.g.: unique=“title,date” will not import any items with the same title and date as an existing element in the selected weblog. Note, the field names are the fields set up within your own weblog (ie, title, date, and any custom fields) not the names of the fields in the feed.”

I’ve tried this, and it works like the documentation says. So, if you modify your CSV file, and re-import, you’ll get duplicates, or nothing, depending on what’s set as “unique”.

If Andrew (or someone) would modify the plugin or make a similar one called something like “CSVUpdate” so it COULD be used for updates, I would be jumping for joy.

 Signature 

Debra Weiss | drw Design

Profile
 
 
Posted: 04 July 2008 09:01 PM   [ Ignore ]   [ # 152 ]  
Grad Student
Rank
Total Posts:  70
Joined  10-18-2002

Debra, thanks for pointing this out. I have edited my previous entry. Sorry - I was importing the same data file over and over and failed to notice it was just skipping those entries.

Profile
 
 
Posted: 09 July 2008 09:08 AM   [ Ignore ]   [ # 153 ]  
Lab Assistant
RankRank
Total Posts:  172
Joined  06-18-2007

Hi

I’m having trouble importing entries. It looks like the plugin needs an encloser. Excel does not save CSV’s with enclosers, so I’m a bit stuck!

Richard

Profile
 
 
Posted: 09 July 2008 09:13 AM   [ Ignore ]   [ # 154 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

Try adding an encloser=“QUOTE” or encloser=”’”. The parameter is now required by the PHP function the plugin uses, but I think it will ignore it if the file does not need it. (I think…)

Andrew

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 09 July 2008 09:35 AM   [ Ignore ]   [ # 155 ]  
Grad Student
Rank
Total Posts:  70
Joined  10-18-2002

I just ran a little test with the PHP function fgetcsv(), which is what CSVGrab uses. I tried two .csv files, one with quotes, and one without, and set the enclosure to ‘“’. It did parse and read both files correctly, so Andrew’s surmise is correct. (At least for PHP 5.2.6, which is what I’m using.)

Interestingly, if I set the enclosure to ‘’ (empty string), I get PHP warnings: “enclosure must be a character”

Profile
 
 
Posted: 09 July 2008 11:12 AM   [ Ignore ]   [ # 156 ]  
Lab Assistant
RankRank
Total Posts:  172
Joined  06-18-2007

Thanks for the responses. I am having some success using encloser=“QUOTE”, but not with nothing in it.

Andrew, was that a typo above, or did you really mean:

encloser=”‘“

Profile
 
 
Posted: 10 July 2008 07:17 AM   [ Ignore ]   [ # 157 ]  
Lab Assistant
RankRank
Total Posts:  161
Joined  11-28-2007

We need to build a product that allows members to upload csv files into a weblog. They will be uploading more than 150 files weekly.

Can csvgrab be used with cron so we don’t have to take the manual step of going to the page to call the function? I asked a similar question on the FeedGrab board to use with a different product. I tried the EE cron plugin on FeedGrab, but so far I haven’t been able to make it work.

 Signature 

Promise Media

Profile
 
 
Posted: 10 July 2008 07:25 AM   [ Ignore ]   [ # 158 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

Yes, I use cron to call the plugin. Set up the template as you would normally, and then get the cronjob to access the web page using something like wget or lynx, eg,

wget http://yoursite.com/index.php/your-template-group/your-import-timeplate

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 10 July 2008 07:32 AM   [ Ignore ]   [ # 159 ]  
Lab Assistant
RankRank
Total Posts:  161
Joined  11-28-2007

Thanks, Andrew. And thanks for those plugins. I think we will be able to use them for some interesting projects.

 Signature 

Promise Media

Profile
 
 
Posted: 10 July 2008 02:36 PM   [ Ignore ]   [ # 160 ]  
Summer Student
Avatar
Total Posts:  9
Joined  12-04-2007

Quick question,
Does the revised plugin support the auto increment feature when it detects a duplicate title being imported.

Basically i’m trying to import 3673 entries, but due to some of them having the same title, it is importing 1763.

Is there a way for the current version of the plugin to import the same title, but modify the url_title.

I noticed there is now a ‘unique’ parameter option, but I do not think it applies to this case…  I could be mistaken..

any thoughts?

—- andrew thanks so much for developing this plugin, and your continued work into it..—-

Profile
 
 
Posted: 10 July 2008 02:42 PM   [ Ignore ]   [ # 161 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

The unique parameter does tell the plugin what to look at to determine if an entry is a duplicate.

It defaults to using the title and date, so try setting it to a custom field that will always be different.

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 10 July 2008 05:56 PM   [ Ignore ]   [ # 162 ]  
Summer Student
Avatar
Total Posts:  9
Joined  12-04-2007

worked great andrew!  thanks again…

Profile
 
 
   
9 of 14
9
 
Post Marker Legend
New Topic New posts Hot Topic Hot Topic with new posts New Poll New Poll Moved Topic Moved Topic Sticky Topic Sticky topic
Old Topic No new posts Hot Old Topic Hot Topic with no new posts Old Poll Old Poll Closed Topic Closed Topic Announcement Announcements
Theme
Change Theme
Visitor Statistics
The most visitors ever was 1149, on July 16, 2007 09:33 AM
Total Registered Members: 65095 Total Logged-in Users: 66
Total Topics: 82241 Total Anonymous Users: 34
Total Replies: 442007 Total Guests: 249
Total Posts: 524248    
Members ( View Memberlist )