CSUS - 21 May 2009 04:54 PM

Absolutely fantastic plugin,

Will truly save me tens of hours of entry, but one odd issue I’ve run into is dates are being entered a whole day behind the date listed in the csv file.  I’ve run the import with trace on, and it appears they’re initially entering properly, but showing up in the entry wrong. Thanks for any help you can provide.

you’re not alone. i’m running into this right now - i have my date formatted yyyy-mm-dd as it should be, and trace shows it correctly as well. WEIRD.

i looked in the plugin file, there’s a lot of date formatting going on (localization, etc.) and there is an offset field that is an input param. i tried setting that, no luck. i put it as 1 and -1, really, i was just trying to affect some change! i can’t figure this out. i turned off daylight savings in the global preferences, no change.

does someone shed some light on the offset field, or the time_offset field, as this is clearly a little issue with this plugin! help.

UPDATE: set offset=”-86400” in your import template (that’s a full day in seconds) and you’ll get the right day—at least, that worked for me!!

I can’t get this to work! I have been trying for a few hours… A little help please?

You can check the file format at the link above and the custom field names are correct.

Thanks,
David

dpdexter - 07 June 2009 01:39 AM

url="http://208.43.169.39/2009_calendar.txt"

HI David - I looked at the file above, and it does not have proper line endings. The lines end in carriage returns (\r) only. They should be newlines (\n). Assuming you are on a Mac, make sure you upload the file in text mode, which will fix the line endings on upload. Or edit the file and fix the line endings before uploading.

You can check the file on a Mac by starting a Terminal and doing:

I believe curl exists on OS X: I don’t have one to check. If not, download the file in a browser and save it to a file using the browser (don’t cut and paste). Firefox, at least on Windows, does not change the line endings when saving to a file.

The od will do a character dump of your file and you’ll be able to see the line endings.

If you have SSH access to your webhost, and it’s Linux, you could also run od -c there.

Thank you Dan. Just needed to re-encode and update in text format and it worked perfectly!

Has anyone been able to import multiple fields (columns) into multiple category groups with this plugin? It was discussed some time ago as a feature request in this thread, but it seems it was never added.

I really need a way to import around 700 records that have TWO categorizations, and therefore need to be classified via TWO category groups—one for ‘product type’ and one for ‘product application’. I can use CSVgrab as-is, and just import the first category group, but it means I’m going to have to go through each and every record and manually assign categories for the second category group—a major pain!

If anyone knows a workaround, I would LOVE to hear it!! 

Nick

Do you HAVE to use a space for a category delimiter?

I’m trying to use a pipe ( | )  but it isn’t working.  If I put category_delimiter=”|” it doesn’t import the categories. 

Here is the code I’m trying:

It was a while ago, but I think I had trouble with “|” as well, due to it being treated as part of a regexp, because CSVgrab was using PHP split(). See post #184. That post contains a modified CSVgrab that fixes that problem and a few other ones.

Another gotcha you may know already: The category names are the actual names, not the url titles.

One final thing: you are using field 2 both as a category field and a regular input field. Do you mean to store field 2 in a custom field? You might want to remove 2 from the regular fields:

Fantabulous!

Thank you so much, I need to try this out.

You know, now that you point out the redundancy of importing the second column into a separate weblog (originally to add some keywords to the page as ‘other categories this business is listed in) I now realize that a simple call for the cats would work.  der….

Thanks again, off to try your modified CSVGrab (excellent plugin BTW and I hope it graduates to EE2.x)

Kevin

————-

Brilliant!  This worked perfectly, thank you and Andrew for the work on this amazing plugin.

@Dan

Are you saying we *can* import categories into multiple category groups with your update? Great, if yes! I’m not clear on what the separator is… if pipe ‘|’ doesn’t work, what should be used? What’s the purpose of “Category_delimiter…”?

Cheers, Nick

Nick Foxall - 24 July 2009 09:43 PM

Are you saying we *can* import categories into multiple category groups with your update? Great, if yes! I’m not clear on what the separator is… if pipe ‘|’ doesn’t work, what should be used? What’s the purpose of “Category_delimiter…”?

Nick - No, my fix does not do multiple category groups. The original CSVgrab can put a single item into multiple categories, as long as they are all in the same group. To separate the multiple categories, you use the delimiter, which defaults to “,”. You can specify a different delimiter, but due to the way the original CSVgrab split up the list of categories, “|” did not work (because it’s a meta-character in regular expressions). I just used a different splitting mechanism (explode() instead of split() ).

Andrew,

do you think you could add a little note about the multibyte thing in the documentation (like the tip you have for the URLs in the control panel).  I got bitten by it today and 18 pages is harder to pour through than a single page.  It seems to be common enough.

Thanks.

I also got bitten by the new line, \n, mac thing. Might also be worthy of a note.

Is there a way to run CSVgrab templates (and perhaps other EE scripts) through the command line using PHP instead of cURL?

I’m using CSVgrab to manage a very large amount of data. The cURL is timing out and using too much CPU for mySQL. Our host said that using PHP to run the commands would expedite things and make them more efficient simply by removing it from the web environment.

Has anyone here ever needed to make adjustments like these or noticed any thread talking about anything like this?

Thanks so much.ccb.

I am having a torrid time getting the plugin to just read my file?? Pretty mystified.

Debug Checklist:

• Permissions are all good, file is readable by all
• fopen is switched on in php.ini
• I think my line endings are fine - I’m on a mac, but I exported from phpMyAdmin in Windows Xcel csv format (how do I confirm this?)
• Here’s my csv file

I have the following in my template:

Any help would be awesome, thanks!

By the way, I should mention the error I’m getting is as follows:

Warning: fopen(http://bestafricatravel.com/data/accom_province.csv) [function.fopen]: failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in /[doc_root]/[system_folder]/plugins/pi.csvgrab.php on line 192

Also, I opened my .csv up with the Terminal in VI and the line endings are all good for Unix consumption.

Ok, figured this out at long last.
For some reason the plugin (or fopen?) only works when you refer to the full www. URL (or use the IP address). I was trying to open the file without the www.

Anyone know how I can assign status with CSVgrab? If this is not currently doable can I put in a feature request?

I’m using CSV grab to bulk-import names of people in many countries, particularly in France.

Every time there’s a ‘é’, or a ‘ç’, the import translates this to gobeltygook and that’s how it arrives in the database.

I’ve modified my regex.core file (this is actually an exterior one to the system - so that these grabs can be called with PHP) to skip ISO encoding - my files are in UTF-8 and the ISO was throwing an error. Now that its processing in UTF-8 automatically, the error isn’t thrown, it just records incorrect info.

Is anyone able to tell me which settings to check, or point me to resources to learn more about this utterly confusing issue?

Thanks much,
.ccb.

Just FYI and not to change the subject—I just imported over 19,000 entries into EE and everything went smooth (had to restart a couple times due to PHP timing out, but the unique=”” helped with that)

thanks!