I’ve succesfully imported more than 3,000 entries using the following code:

However when I go to view entries in a template I get the following:

Bob SMITH
{member-number} {member-type} {MAddress1} {MAddress2} {MAddress3} {MAddress4} {MSubRegion}

None of my custom fields seem to be rendering. I have checked all associations and groups and they are correct. If I manually enter data then the entry displays correctly:

test name
10010 Member test test test test test

The template code i am using is:

Any ideas on how to fix this greatly appreciated.

Nardog, the plugin is not site aware. There isn’t any code in the current version of the plugin to actually set the right site_id.

Edit one of these entries in the back end to use site_id = 4 instead of site_id = 1 and see if it fixes that particular one.

Disclaimer: Unless Andrew modified the plugin in the last few days..

Hi Sue,

Thanks for replying. I understood from Andrew’s site that site_id= does work.

Step 5: Set up the import template and plugin

Create a new template called go and add the call to the plugin:

{exp:csvgrab url=“http://ee.bnb-dev.co.uk/grab/golf.csv”
  delimiter=”,”
  encloser=“QUOTE”
  site_id=“2”
  weblog=“4”
  category_group=“2”
  title=“2”
  category_field=“3”
  use=“7|8|9|10|11”
  fields=“golf_addr|golf_city|golf_state|golf_zip|golf_phone”
  unique=“golf_phone”
}

In my case the import worked fine but the problem is with the display of the custom fields. When I check a random entry in the CP edit tab all the fields are populated correctly. Yet when I view that entry through the template the custom fields do not display although they are present in the entry.

Good, he’s changed it then.

Is that all you have in the template? What happens if you reduce the template to just the exp:entries tag (hard coding the weblog field name).

Hi Sue,

That is all I have in the template and I have tried hard coding the the weblog field name. What’s wierd is that the content is all there in the CP yet the template does not pick up the custom fields unless the entry was manually entered rather than imported with csvgrab.

This is weird. All of a sudden CSVGrab only processes around 1400 entries at a time before quitting on me. Each time, it ranges from successfully processing anywhere from 1390 to 1410 or so. Any clues on that one?!

Running out of memory? Turn on error reporting, and/or check the logs.

From the error log:

Edit: Also, I had the PHP mem_limit set to 128M

So, why is it only kicking in after 1400 entries? Are those imported fine? The “connot modify header inforamtion” error is quite often caused by additional whitespace at the beginning of your plugin or code, you might want to check that.

That’s what’s so strange. Not sure where to start looking in the code as it’s the CSVGrab plugin. Maybe I’ll reupload it and see if that does anything…?

Can’t hurt. What I meant was making doubly sure that <?php is in the first line of the plugin, and not perhaps a linebreak or so. It’s happened to me. If that does not help, yes, re-uploading a fresh version certainly would not hurt.

Thanks Ingmar, will give that a go. I’m also re-exporting my CSV file as well just to cover all the bases…

Hitting the sack for now. No errors in the plugin it seems, and this load crapped out at around 11,000 entries. This is the pattern: it gets to 10k-ish entries imported then tanks. Then each subsequent attempt after that only gets to around that 1500 number before dying. I’m wondering if it’s a mysql issue, or even perhaps something more specific to EE.

...and back. After changing further things, still getting the exact same result. Thinking there’s a memory leak somewhere, either in the plugin or in EE itself, that manifests after hitting that 10,000 entry marker…

I had the same issue that it would drop out at about the 10,000 record mark, the only thing I did was chop my csv file into 10,000 record chunks. I’m assuming it’s a timeout problem but as I was in a rush I didn’t have time to play about with Andrew’s code. I did successfully import 560,000 records over the course of weekend though while managing to keep the kids amused at the same time!

Another alternative is to store the data external to the EE tables and then use PHPMyAdmin with the LOAD DATA command. This is a lot faster although I had to reconfigure PHP to load up 10MB CSV files. I loaded up 720,000 records in about 20 mins.

Hi collateit,
I did the same thing, splitting the csv down into 5,000 row files. Stil died, but got down to 1400 records at a time. Sheesh. I am/was on Media Temple’s dedicated virtual server package, but their support is slow and offered little in the way of new knowledge. I couldn’t do any db loads from the command line either, as apparently we’re not given those permissions. Ugh. I basically elected to change providers and move to a fully dedicated box now

Update: This is a PHP configuration issue I have since addressed. Feel free to ignore this post.
The adventure continues… I have changed hosting providers and am now running on PHP 5 instead of 4. Now I get the following error when I try to run the script:

As a new EE user, I’m still confused by the conventions for posting plug-ins in threads.

Some authors update the first post to include a link when a newer version becomes available. Others post the new versions throughout.

This thread, which has many pages, has the newer versions sprinkled throughout.

If there isn’t a convention, could I vote for updating the first post with the current version?