5 of 14
5
Plugin: CSVGrab
Posted: 07 December 2007 11:11 AM   [ Ignore ]   [ # 73 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  420
Joined  10-10-2004

Good to hear that further development on this plugin will continue. Typically there’s a couple different ways to accomplish something with EE but I found that when I needed this plugin there was no other option to accomplish the task at hand and that it was the difference between success and failure. So, thanks for posting that you’ll keep this going forward!

Profile
 
 
Posted: 27 December 2007 02:14 PM   [ Ignore ]   [ # 74 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  716
Joined  05-14-2004

I’m not clear on how this functions, so I thought I would ask. smile  I’m having a challenge getting my entries to import without it spitting out big fat php errors and importing a bunch of rubbish.

{exp:csvgrab url="http://domain.com/myfile.csv"
                          
weblog="5"
                          
title="1"
                          
skip="6"
                          
delimiter="TAB"
                          
encloser="QUOTE"
                          
use="2|3"
                          
fields="body|extended"}

Obviously, the link to the file is pointing to the right place, I’ve just changed it for this posting.

I think I’m just not totally sure what each of the parameters does.  I figured out that weblog=“5” is the weblog I want to import into, not the one I want to grab from, which is great, though I’d love to be able to specify from which weblog I wish to pull the entries that live in the .csv file.

Furthermore, I’m not sure why I’m skipping 6 lines or using “TAB” or what the encloser is or what the use parameter does.  Does the fields parameter pull from the .csv or import those fields to the new weblog?

I did read any docs I could find and this entire thread, along with any other information I could find on it and I’m still having a tough time. I know in the first post, the plug-in author gave a very brief “key” to the parameters, but something a little more specific would be great.

Any assistance would be appreciated. Thanks!  grin

 Signature 

Moxie Design Studios™ • Top Shelf Blog & Website Design by The Moxie Girls™
EE ProNetwork • (EE 1.6.5, build 20081024)

Profile
 
 
Posted: 28 December 2007 04:56 AM   [ Ignore ]   [ # 75 ]  
Summer Student
Avatar
Total Posts:  23
Joined  01-04-2006

The parameter skip stands for the number of lines in your document you want to skip.
The delimiter parameter stands for the separation of values on a single line. like SPACE TAP or , or ;
The encloser is about the enclosure of values. Example: “value” or ‘value’
Use works together with the fields parameter. With use you point to the column in de csv file. And with fields you select a custom weblog field.

i hoop this will help

Profile
 
 
Posted: 28 December 2007 09:08 AM   [ Ignore ]   [ # 76 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  716
Joined  05-14-2004

Thanks, Maix.  I appreciate you response.

I get that “use” works together with the fields parameter, but as I asked before, does that mean the fields you’re pulling or the fields you’re importing into?  For the delimiter, I don’t know why I would choose TAB or COMMA or SPACE TAP, etc. over another.  Which one is most common place? Why one over the other?  And why would I want to “skip” such n’ such amount of lines? How do I know if I want single or double quotes for my enclosure values?

See what I mean? smile I mostly handle front-end stuff, but would love to learn more about database whatnot so I don’t have to ask so many questions. I think a practical, everyday example in the docs would be really helpful to those who aren’t well-versed in database tables and columns. (i.e., your Average Joe who wants to just move basic blog entries from one EE blog to another. Nothing fancy, just your garden variety type thing.)

Thanks again for your help, Maix!  Any additional input would be greatly appreciated.

Thanks!

 Signature 

Moxie Design Studios™ • Top Shelf Blog & Website Design by The Moxie Girls™
EE ProNetwork • (EE 1.6.5, build 20081024)

Profile
 
 
Posted: 28 December 2007 11:21 AM   [ Ignore ]   [ # 77 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  140
Joined  05-13-2006

Joelle,

I’ll do my best to answer your questions.

I get that “use” works together with the fields parameter, but as I asked before, does that mean the fields you’re pulling or the fields you’re importing into?

The “use” and “fields” work together, as you’ve stated, and it tells the plug-in what data to enter where. The “use” digits refer to existing columns in your database, from the CSV file. The “fields” are what you’ve set up in EE as weblog custom fields. For example, if you have a field called “article-summary” and the CSV data to match is in column “6”, you would enter use=“6” and fields=“article-summary”. More than one field to fill up with data? Just use the pipe character, “|”, as instructed.

For the delimiter, I don’t know why I would choose TAB or COMMA or SPACE TAP, etc. over another.  Which one is most common place? Why one over the other?

In my experience, it doesn’t matter, just how your CSV file is set up, though I usually pick comma. If you’re exporting from Microsoft Excel, you can choose all sorts of different delimiters. Same with phpMyAdmin or other web database managers. The delimiter simply structures the database into unique columns that the plug-in can interpret.

How do I know if I want single or double quotes for my enclosure values?

Again, it’s how your CSV file is configured. I don’t think it really matters, as I’ve used both without a hitch.

 Signature 

Subvert Marketing Inc. - Whitehorse, Yukon, Canada

Profile
 
 
Posted: 28 December 2007 01:05 PM   [ Ignore ]   [ # 78 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  716
Joined  05-14-2004

Thanks, Geof!  That helps a lot. It’s nice to know why I’m doing such n’ such so I don’t make mistakes in the future. Though, those are inevitable. haha!

I’ll see if I can figure out this beast. Thanks for taking the time to explain it to me. cheese

 Signature 

Moxie Design Studios™ • Top Shelf Blog & Website Design by The Moxie Girls™
EE ProNetwork • (EE 1.6.5, build 20081024)

Profile
 
 
Posted: 28 December 2007 01:08 PM   [ Ignore ]   [ # 79 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  716
Joined  05-14-2004

Oh, one more quick question: the skipping of lines.  How do I know I want to skip lines and how does one determine that?  If, let’s say, I was using the default weblog fields and I’ve exported the .csv file from phpMyAdmin.  I want to import those entries into a new weblog using similar fields.  You’ve shared how to tell what columns to populate which fields, but do I need to worry about skipping lines?

Thanks again!

 Signature 

Moxie Design Studios™ • Top Shelf Blog & Website Design by The Moxie Girls™
EE ProNetwork • (EE 1.6.5, build 20081024)

Profile
 
 
Posted: 28 December 2007 01:18 PM   [ Ignore ]   [ # 80 ]  
Summer Student
Avatar
Total Posts:  23
Joined  01-04-2006

Do not worry about skipping lines.

It can be useful when for example the first 10 lines of your csv are already in ExpressionEngine.

Profile
 
 
Posted: 28 December 2007 02:03 PM   [ Ignore ]   [ # 81 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  716
Joined  05-14-2004

Thanks, Maix.  How do you mean?  I exported the database from one EE installation on another server and I’m importing the entries into an EE weblog on a new server. So do I not need to skip anything?

Thanks! smile

 Signature 

Moxie Design Studios™ • Top Shelf Blog & Website Design by The Moxie Girls™
EE ProNetwork • (EE 1.6.5, build 20081024)

Profile
 
 
Posted: 28 December 2007 02:30 PM   [ Ignore ]   [ # 82 ]  
Summer Student
Avatar
Total Posts:  23
Joined  01-04-2006

In that case you have nothing to skip.

In some other cases the first line of the csv file is used for the name/description of the data columns then you want to skip them. Or when you have a product list were the first 100 lines/items are already in ExpressionEngine. You can skip them with skip=“100”

Profile
 
 
Posted: 28 December 2007 03:36 PM   [ Ignore ]   [ # 83 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  716
Joined  05-14-2004

So then do I need the “use” information? I’m not sure what “2|3” even are.  lol.

I just exported the entire database from the old site and now I want to pull entries from “X” EE weblog in the old database (.csv file) into “Y” EE weblog on a new installation of EE.  Would I put something like this?

{exp:csvgrab url="http://domain.com/myfile.csv"
                          
weblog="5"
                          
title="1"
                          
delimiter=";"
                          
encloser="QUOTE"
                          
use="2|3"
                          
fields="body|extended"}

The weblog I want to import into is 5 and according to phpMyAdmin the delimiter is a semicolon (“Fields terminated by ;”). It also says that “Fields enclosed by quote”.  Like I said, I’m not sure what use=“2|3” is exactly.  How do I know what columns I want to use? Where does that information live?

Does this look right? I tried using this and it just pulled in a bunch of jumbled rubbish entries and spit out php errors.

Thanks again! grin

 Signature 

Moxie Design Studios™ • Top Shelf Blog & Website Design by The Moxie Girls™
EE ProNetwork • (EE 1.6.5, build 20081024)

Profile
 
 
Posted: 28 December 2007 03:53 PM   [ Ignore ]   [ # 84 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  140
Joined  05-13-2006

Joelle,

2|3 are the columns in your CSV database. For example, this is a snippet from my CSV file representing 3 columns separated by a comma:

"Dawson City residents choose their world's Seven Wonders","2007-10-22","4",

If I wanted to import the title “Dawson City residents choose their world’s Seven Wonders” and category “4” into EE, I would use 1|3.

As for your errors, use a debugging method. To do this, create a simplified duplicate of your CSV with only a few entries (2 or 3 at most) and then import that with the CSVGrab plug-in. Start with a single use/field and if it works, add another. Keep doing this until you produce an error, then let us know what the error message is.

 Signature 

Subvert Marketing Inc. - Whitehorse, Yukon, Canada

Profile
 
 
Posted: 21 January 2008 02:21 PM   [ Ignore ]   [ # 85 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  328
Joined  04-02-2007

Does anyone know if this will allow multiple categories?  Right now I see this option:

category_field="9"

but can we do something like:

category_field="9|14|87"

Profile
 
 
Posted: 22 January 2008 07:40 AM   [ Ignore ]   [ # 86 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

Just to clarify, the category_field parameter selects which column of the CSV file to use for categories.

There is another parameter called category which selects a default EE category to put the entry in.

Unfortunately, neither allow you to use the “9|14|87” format that you asked about, but I can see how it would be useful (in the category=“9|14|87” case at least).

I’m currently working on an updated version of this plugin, but email or PM me if you need this functionality urgently.

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 30 January 2008 08:25 AM   [ Ignore ]   [ # 87 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  112
Joined  03-21-2007

Hi,

I am having problems with importing a date field with csvGrab (version 0.11). The template I am using is:

{exp:csvgrab url="http://localhost/EE/new10-a.csv"
                          
weblog="1"
                          
title="1"
                          
skip="2"
                          
delimiter="TAB"
                          
encloser="QUOTE"
                          
use="2|3|5"
                          
fields="summary|body|date" }


and this imports the data fine except for the date field all of which are set to January 1 1970.

The date field in my csv is formatted like “1998-10-22”

The error that is generated is

Notice: Undefined index: in /var/www/EE/system/plugins/pi.csvgrab.php on line 186

Any ideas on how to get csvGrab to grab the correct date?

SOLUTION:

Add date"5” to code above ie:

{exp:csvgrab url="http://localhost/EE/new10-a.csv"
                          
weblog="1"
                          
title="1"
                          
skip="2"
                          
delimiter="TAB"
                          
encloser="QUOTE"
                          
date="5"
                          
use="2|3"
                          
fields="summary|body" }

Profile
 
 
Posted: 30 January 2008 09:06 AM   [ Ignore ]   [ # 88 ]  
Lab Assistant
RankRank
Total Posts:  165
Joined  12-21-2005

I’ve a new(-er) version of CSVGrab that you can try. Its much the same on the surface, but is cleaned up a bit underneath. It handles dates a little better than previously.

To get dates working, add a date parameter to indicate which column to use for the date, eg, date=“5”. It currently only looks for dates in YYYY-MM-DD format.


Edit: Please see the CSVGrab page to download current version

 Signature 

Andrew Weaver — brandnewbox.co.uk


Member of EE Professional Network · More plugins, extensions and modules · Documentation

Profile
 
 
Posted: 30 January 2008 09:17 AM   [ Ignore ]   [ # 89 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  112
Joined  03-21-2007

Thanks Andrew

Profile
 
 
Posted: 11 February 2008 12:13 AM   [ Ignore ]   [ # 90 ]  
Grad Student
Avatar
Rank
Total Posts:  77
Joined  05-10-2006

Hello Andrew.

This looks outstanding! Just one question though… Is there any way to fix this so that we can have multiple categories in the field (eg. 3|4|7|9)?  I’ve seen other related questions, but don’t know if your latest release has this functionality yet?

I assume it would involve the use of 2 different delimiters in the CSV though, right?  Something like:

TITLE;DATE;AUTHOR;SUMMARY;BODY;3|4|5|6|7;LINK


...where the 3|4|5|6|7 are the categories that the entry is assigned to.

Can this be implemented?

Thank you again!

Profile
 
 
   
5 of 14
5
 
Post Marker Legend
New Topic New posts Hot Topic Hot Topic with new posts New Poll New Poll Moved Topic Moved Topic Sticky Topic Sticky topic
Old Topic No new posts Hot Old Topic Hot Topic with no new posts Old Poll Old Poll Closed Topic Closed Topic Announcement Announcements
Theme
Change Theme
Visitor Statistics
The most visitors ever was 1149, on July 16, 2007 09:33 AM
Total Registered Members: 64944 Total Logged-in Users: 65
Total Topics: 81925 Total Anonymous Users: 41
Total Replies: 440354 Total Guests: 308
Total Posts: 522279    
Members ( View Memberlist )