20 of 20
20
Plugin: CSVGrab
Posted: 15 December 2009 09:19 PM   [ Ignore ]   [ # 343 ]  
Lab Assistant
RankRank
Total Posts:  135
Joined  11-12-2008
Mark Terpstra - 25 August 2009 07:54 PM

EDIT

I finally realized that the problem was that although I assigned Gypsy fields to the target weblog, I had not assigned a Field Group to the weblog I was trying to import into. Once I did that, I successfully imported some test entries.

It turns out that when I try an import into a weblog that is using Gypsy fields, the data is imported fine into the exp_weblog_titles table but NOT in the exp_weblog_data table.

I just tried a test import on a different weblog on this same site that is NOT using Gypsy fields and it worked fine.

Has anyone else had a problem importing into Gypsy fields? Is there a workaround?

I am also searching to see how to get this to work with Gypsy fields. Suggestions?

 Signature 

Blog |
Twitter

Profile
 
 
Posted: 19 January 2010 01:07 PM   [ Ignore ]   [ # 344 ]  
Research Assistant
RankRankRank
Total Posts:  464
Joined  09-06-2006

I cannot get this to work: using csvgrab 0.2.3 and EE ver 1.6.8

{exp:csvgrab url="/home/user/public_html/find.txt" 
weblog="2" 
title="5"
delimiter="TAB"
trace="true"
enclosure="QUOTE"
stagger="3600"
offset="21600"
use="4|5|6|7|8|9|10|11|12" 
fields="find-a-physio-name|find-a-physio-company|find-a-physio-address1|find-a-physio-address2|find-a-physio-city|find-a-physio-postal-code|find-a-physio-telephone|find-a-physio-email|find-a-physio-website" 


Here is my TAB file saved as find.txt and uploaded in text mode via ftp:

"name"    "company"    "address1"    "address2"    "Corner Brook"    "bbb222"    "(709) 111 1111"    "me@myemail.com"    "website" 

and I keep getting this error:
Warning: fgetcsv() [function.fgetcsv]: enclosure must be a character in /home/user/public_html/admin/plugins/pi.csvgrab.php on line 202


Here is the trace:
TRACE: Array ( [title] => Array ( [is_custom] => 0 [field] => 5 ) [date] => Array ( [is_custom] => 0 [field] => ) [find-a-physio-name] => Array ( [is_custom] => 1 [field] => 4 [id] => 4 [format] => none ) [find-a-physio-company] => Array ( [is_custom] => 1 [field] => 5 [id] => 5 [format] => none ) [find-a-physio-address1] => Array ( [is_custom] => 1 [field] => 6 [id] => 6 [format] => none ) [find-a-physio-address2] => Array ( [is_custom] => 1 [field] => 7 [id] => 7 [format] => none ) [find-a-physio-city] => Array ( [is_custom] => 1 [field] => 8 [id] => 8 [format] => none ) [find-a-physio-postal-code] => Array ( [is_custom] => 1 [field] => 9 [id] => 9 [format] => none ) [find-a-physio-telephone] => Array ( [is_custom] => 1 [field] => 10 [id] => 10 [format] => none ) [find-a-physio-email] => Array ( [is_custom] => 1 [field] => 11 [id] => 11 [format] => none ) [find-a-physio-website] => Array ( [is_custom] => 1 [field] => 12 [id] => 12 [format] => none ) )

TRACE: Resource id #48

 Signature 

EastwoodDesign.ca

Profile
 
 
Posted: 19 January 2010 01:29 PM   [ Ignore ]   [ # 345 ]  
Lab Assistant
RankRank
Total Posts:  113
Joined  10-18-2002

Hi, you have

enclosure="QUOTE" 
. Try
encloser="QUOTE" 

.

Profile
 
 
Posted: 19 January 2010 01:36 PM   [ Ignore ]   [ # 346 ]  
Research Assistant
RankRankRank
Total Posts:  464
Joined  09-06-2006
Dan Halbert - 19 January 2010 06:29 PM

Hi, you have

enclosure="QUOTE" 
. Try
encloser="QUOTE" 
.

Thanks a bunch.  That was it.

 Signature 

EastwoodDesign.ca

Profile
 
 
Posted: 25 January 2010 05:46 PM   [ Ignore ]   [ # 347 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  168
Joined  03-03-2009

I am importing data from an old website into a new EE version.  We have 10 years worth of monthly magazines we are publishing on line.  Each month has a monthly feature, editorial and a topic for each day.  I am using 2 weblogs:  monthly and daily.  I am able to import the data into them just fine (did a limited test).  However, I am not sure how to handle the relationship field.

Each daily is related to the month’s issue it was published in.  How would I import the daily data so that it automatically is related to a particular month?  I would assume I would have a column that I input the related field data…but what data is that?  The entry id for the month?

Found this and it is the same kind of issue, but they never did say how they got around it:
http://expressionengine.com/forums/viewthread/43857/P18/

Profile
 
 
Posted: 07 February 2010 04:19 PM   [ Ignore ]   [ # 348 ]  
Summer Student
Total Posts:  15
Joined  10-07-2009

Hi,

I’ve been using CSVGrab few times and had no problem. I just tried running a small import for 50 lines of CSV file and it is not working.

I turned on the “TRACE” option and it returns “TRACE: Resource id #45” error.

My CSV file is using UTF8 encoding and Unix (\n) line break option.

Any help would be appreciated.

Regards
Rudy

 Signature 

Rudy Affandi

Profile
 
 
Posted: 07 February 2010 06:58 PM   [ Ignore ]   [ # 349 ]  
Lab Assistant
RankRank
Total Posts:  113
Joined  10-18-2002

Hi - the “TRACE: Resource id #45” is not an error: it indicates that the file is being opened. However, if you see nothing after that then CSVgrab is having trouble parsing the file or matching it up with your template, or there’s an error in the template.

Iff the “use” and “fields” value in your template are not in one-to-one correspondence, then CSVgrab will fail silently.

If you post your template and some sample data lines, that could help.

Dan

Profile
 
 
Posted: 15 February 2010 02:00 AM   [ Ignore ]   [ # 350 ]  
Grad Student
Rank
Total Posts:  44
Joined  05-02-2008

I need some help here.

I tried to use CSVgrab to add entries with multiple categories using the category IDs rather than the category name.
The entries were added to the weblog, but the categories were not assigned to the entries.

I’m using CSVGrab v.0.2.1

And here’s the template I’m using:

{exp:csvgrab url="http://www.domain.com/import/sample_feb14.csv"
   
delimiter=","
   
encloser="QUOTE"
   
skip="1"
   
date="8"
   
weblog="3"
   
title="6"
   
category_field="3"
   
use="4|7"
   
fields="submitted_url|submitted_summary"
   
unique="title,submitted_url"

And here’s a sample row of csv data:

36,"test","26,58,60","http://www.url.com","nickname","title","summary","2010-01-23 18:00:00" 

The category field contains the category IDs of three existing categories.
Can anyone tell me why the categories were not assigned to the entries?

And there’s another issue: based on the output of the template, it seemed that CSVgrab tried to create three new categories named 25, 58, and 60.  And when I looked at the exp_categories table, sure enough, there are three new categories with names (not IDs) 25, 58 and 60. And the group ID for these unwanted categories is 0. Also, these three categories and the category group zero are not visible anywhere in “Admin › Weblog Administration › Category Groups” in the control panel.
Can I safely delete these three records from the exp_categories table?

Profile
 
 
Posted: 16 February 2010 06:55 AM   [ Ignore ]   [ # 351 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  188
Joined  12-21-2005

I think you’ll need to use the category names, rather than the ids in you CSV file.

There is a category_group= parameter to choose which group new categories are created in. eg, category_group=“1”

Andrew

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 17 February 2010 01:50 PM   [ Ignore ]   [ # 352 ]  
Grad Student
Rank
Total Posts:  44
Joined  05-02-2008

Thanks for your reply.

PHP is not my strong point, but looking at the code of the plugin, it looks like we should be able to used category IDs.

Lines 384 - 391:

if (isset($this->cats) && $this->cats != ''
    
{            
         $cats 
explode(',',$this->cats);
         
$cats array_unique($cats);

         
$results $DB->query("SELECT cat_id, parent_id FROM exp_categories 
        WHERE (cat_id IN ('"
.implode("','",$cats)."') OR cat_name IN ('".implode("','",$cats)."'))
        AND group_id = '"
.$query->row['cat_group']."'"); 

And for those unwanted categories in the exp_categories table, can I delete them manually without damaging anything?

Profile
 
 
Posted: 22 February 2010 11:51 AM   [ Ignore ]   [ # 353 ]  
Research Assistant
RankRankRank
Total Posts:  303
Joined  11-28-2007

I need to import old entry dates and keep them that way. When I use date= and the number of the field, it seems to work fine. But when I look at the entry in EE, I see today’s date as the entry date and not the old date.

Am I misunderstanding how the date parameter works? Is there a way of keeping an old entry date?

The date format appears to be consistent with how it is described on the csvgrab Web site.

 Signature 

Caribbean Vacations - Online Marketing

Profile
 
 
Posted: 23 March 2010 05:17 AM   [ Ignore ]   [ # 354 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  188
Joined  12-21-2005

Hi all,

I am currently testing a new module for EE 2.0 to replace the FeedGrab, CSVGrab and XMLGrab plugins.

For more details, and if you’d like to help test it, please visit:

DataGrab for ExpressionEngine 2.0

Thanks,
Andrew

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 27 May 2010 02:39 PM   [ Ignore ]   [ # 355 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  281
Joined  12-13-2007

BTW - If anyone’s trying to get this to work with gypsy fields, change the query to this:

$customfieldsArray $DB->query("SELECT exp_weblog_fields.field_id, 
                                exp_weblog_fields.field_name, 
                                exp_weblog_fields.field_label, exp_weblog_fields.field_fmt
                                 FROM exp_weblogs, exp_weblog_fields
                                 WHERE exp_weblogs.field_group = exp_weblog_fields.group_id
                                 OR field_is_gypsy = 'y'
                AND gypsy_weblogs
                LIKE '%"
.$this->feed_weblog_id."%' OR exp_weblogs.weblog_id = '".$this->feed_weblog_id."'"); 
 Signature 

Fusionary | Geniuscar

Profile
 
 
Posted: 27 May 2010 03:20 PM   [ Ignore ]   [ # 356 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  830
Joined  02-25-2008

I love CSV Grab and have used it a number of times to quickly and easily import loads of data.

Is there a way to append the content that has already been uploaded through CSV Grab?

For example, if I import 80 rows of data with 10 cells per row and then a week later the content of the .csv file is updated, modifying some content and adding new, is it possible to re-import that .csv file and have it append the entries on the site? I tried to do it and the way I currently have it configured it simply creates duplicate entries.

Current configuration:

{exp:csvgrab url="http://www.mysite.edu/projects/list.csv"
         
weblog="18"
         
title="1"
         
skip="2"
         
delimiter=","
         
author="21"
         
encloser="QUOTE"
         
use="1|2|3|4|5|6|7|8|9"
         
fields="research-project-title|research-topic|research-faculty-member|research-project-sponsor|research-project-start-date|research-project-end-date|research-location|research-project-partners|research-project-description" 


Thanks in advance for the assistance on this.

Forrest

 Signature 

Rouviere Media — I Build Expression Engine Sites. sans-borders.

Profile
 
 
Posted: 01 June 2010 05:19 AM   [ Ignore ]   [ # 357 ]  
Lab Assistant
RankRank
Total Posts:  173
Joined  02-23-2009

Am trying to work out if CSVGrab will do what I need for an upcoming project:

1) Can I import automatically, say, every 24 hours? I’m going to have a new CSV file uploaded nightly to the server so need to just update the appropriate weblog each night.

2) Can this process overwrite all the old weblog entries, or will it cause duplicates? (Only some of the data is going to change each night but I’ll only have the full CSV to work with each time.)

3) I understand that CSVGrab can set the category for entries from the CSV. How does this work with sub-categories? For, say, areas of the world, if I want an entry to be categorised as both “Asia” and “Japan”, can the CSV just have “Asia|Japan” for that entry and I take the categories from that? Will EE understand that Japan is a sub-category of Asia? How would I create new sub-categories?

Thanks loads for your help

Profile
 
 
Posted: 08 July 2010 05:41 AM   [ Ignore ]   [ # 358 ]  
Lab Assistant
RankRank
Total Posts:  115
Joined  11-13-2008

Hi,

I just thought I would post my experience here in using the csvgrab.

My primary intention was to get the csvgrab to import and setup a large category group list for me. It worked perfectly !!! grin

One major hiccup I had which is worth nothing is that I was working offline while developing my site, and this made the csvgrab useless, so you will need to turn your System to ‘On’ online to get the csvgrab process to work.

And another finicky thing that caused me problems, believe it or not, was the curly quotes used within the {exp:csvgrab} parameters. I was copying directly from the plugin documentation within my EE Control Panel, and I noticed (after almost screaming) that the curly quotes were different when I cut and paste from various sources, so please make sure that you physically type the double quotes, or make sure they are consistent within the {exp:csvgrab} tags and parameters.

Basically then, I followed Andrew Weaver’s csvgrab plugin instruction on setting up a ‘dummy weblog’ etc with custom_fields and my ultimate desired Category Group.

I then set up an excel / csv file with 4 columns and only one Row Entry.

So in the first column cell, I placed all my categories separated by a ‘comma’. This is the crucial part here. You don’t put the categories in individual rows or cells. Put them all in one cell as the csvgrab process then reads all the categories associated with this one entry.

In the second column cell I placed a dummy title

In the third column cell I placed my dummy custom_field entry

In the fourth column cell I placed my dummy custom_field entry

I then followed Andrew Weaver’s instructions in setting up the ‘go’ template AND basically it worked well by inserting one entry and then inserting all the categories in my Category Group.

You just need to be thinking clearly on whether your categories have spaces within one category name, and then the delimiters you use. Note here below that I used the semi-colon as the csv column delimiter and then the comma as the actual category delimiter which defined my individual categories.

{exp:csvgrab url="http://www.website.com/mycsvfile.csv"
    
delimiter=";"
    
encloser="QUOTE"
    
weblog="13"
    
category_group="19"
    
category_field="1"
    
category_delimiter=","
    
title="2"
    
use="3|4"
    
fields="custom_field|custom_field"

Hope this helps someone else who has struggled with the notion of having to hand-type hundreds of categories. grin

Thank you Andrew for producing a brilliant plugin.

Profile
 
 
Posted: 08 July 2010 12:13 PM   [ Ignore ]   [ # 359 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  128
Joined  02-05-2008

okay, that’s awesome. i’ll file that in the back of my tiny head for future ref.

 Signature 

rebecca :: littlered media :: expressionengine pro net :: super mom :: i’m in pradipta’s rolodex

Profile
 
 
   
20 of 20
20