Thread

This question may be related to a resolved thread.

Hi,

i updated from 2.1.3 (20110411) to 2.4.0 (20120123).
Now files i upload via ftp don’t show up.
I need to always synchronize in the file manager that they show up :-(

Hello Paradise,

Thank you for your question.

Are you saying that before the update you did not have to sync files that were uploaded via FTP? That’s strange, as far as I know that has never been automated. You have to sync the files or ExpressionEngine has no way of knowing about them being added. This is the expected behavior.

I hope this is helpful and if I can clarify anything, please let me know.

Is there anything else I can help you with Paradise?

Thanks,

Before the update there was only thumbnails right? Now there is also “a list”. It took a while until i recognized that i can switch at the bottom of the dialog.

So after a ftp upload and than publishing an article i clicked on “add file” and switched to the directory (what was on the left before) and i saw first a white gif with a loading circle in it instead of my uploaded images. I always had to switch to another directory and than back before i saw the new images (thought a bug) but it worked.

You tell me that what i have now is the normal behavior, but why was there a loading gif?

I waaaaaaaant that back :-(

Oh and if i remember right - i never ever did synchronize in EE until this update.

Hey Paradise,

I think we are using terms differently.

When you say ftp, I think you mean using an FTP client to FTP the files to the server.

It sounds like when you use the word FTP you mean that you use the “Add File” button. If this is the case, then you should not have to sync files. By uploading the image using the ExpressionEngine interface, ExpressionEngine is aware of the file. In the other case that I was talking about, you would have to sync.

So now on to the issue. You are saying that you upload the file with the ExpressionEngine file manager and you see the preview gif, but the link does not show up in the published post until you sync files. That’s not expected behavior and something is a miss.

After the update, did you clear your cache? In the Control Panel go to Tools > Data > Clear Caching. Select All Caches and click Submit.

Sorry about the confusion, hopefully we can get this sorted quickly.

Cheers,

Sorry Shane, i talk about real ftp. I never ever used the EE upload manager.
I always upload with Dreamweaver and never had to sync.

Sure cleared the cache 😊

Hi Paradise,

Okay, so you are using FTP to get these files up to the server.

Are you using Dreamweaver and it’s uploading the template changes to the ftp server? I am sorry, I have never used Dreamweaver. I am not familiar with it.

Are you saying that image files are needing to be synced after upload? If so, this is the expected behavior.

You said that you have never used the EE upload manager but then you said “Before the update there was only thumbnails right? Now there is also “a list”. It took a while until i recognized that i can switch at the bottom of the dialog.” Where are you seeing this?

Sorry about all the confusion, we will get this.

Cheers,

Ok, attachment 2 is EE 2.4.0 with the new “add file” dialog that i get if i click “add file” when i write an entry.
Add the bottom is “Show files as” where i can switch between “a list” and “thumbnails”.
The old dialog in EE 2.1.3 did not have this and was always thumbnails (attachment 1).

Forget Dreamweaver - see it as ftp client like Filezilla.

Before the update i uploaded my images for an entry with Filezilla or another ftp client. Then i went right away to “Content>Publish” and clicked on “Add File” and the dialog from attachment 1 showed up. The images i uploaded before where there right away without any synchronization.

Hi guys, we’re experiencing the same issue since upgrading to 2.4.0 (from 2.2.2).

We have a cron task that pulls in news articles from an XML feed and adds them to ExpressionEngine using the Channel Entries API. Similar to Paradise, this task FTPs image files up to our web server and populates a custom file field for each news entry. In 2.2.2 this worked fine but since upgrading we need to manually synchronise the image directory before EE recognises that the entries have a file associated to them.

Some support on this would be greatly appreciated. I’m guessing that in 2.2.2 the Channel Entries API added the file to the database automatically - is there a new method that we need to call to make this happen in 2.4.0?

Thanks

Hi everyone,

I’m sorry the File Manager is no longer behaving as you expect. Syncing files is a process required for getting the relevant file metadata into exp_files.

You may be able to trigger the sync operation as part of a cron job, or you can submit a Feature Request to have the process improved.

Let me know if there is anything else I can do to help.

Cheers,

Thanks Dan, I’ll submit a feature request.

Edit: feature request is here http://ellislab.com/forums/viewthread/214182/

Edit: Just to add on to this (I’m Senior dev at Chroma, didn’t originally post in this thread, that was one of our other devs). Frankly, the answer to submit a feature request isn’t really good enough.

With the latest update, something that used to work using the Channel entries API (a cron job to import news with images) is now broken because of changes to how you handle uploaded files. Surely the syncing of uploaded files via the channels API should either be handled automatically, or hooks given so we can add these newly added images to the files database table. The answer “You may be able to trigger the sync operation as part of a cron job” isn’t very clear in pointing our dev team in the right direction. Can you not provide more detailed tech support to questions? We’re happy to do the Cron job ourselves, but an indication at where we should trigger this would be a better response from you. Thanks.

A feature request for a feature that was already in there and working and you took out?

Hi all,

We just upgraded and had this problem too. I wrote a script to migrate all the uploaded files that are not already in the database. Place the PHP file in your root uploads directory and change the database variables at top to match your setup. Access the page to run the code. Afterward, make sure to clear your EE cache by going to Tools > Data > Clear Caching.

Hope this helps someone else.

Cheers,
Mike

<?php

// Database config variables; change as needed
$database_path = "localhost";
$database_login = "user";
$database_password = "123456";
$database_name = "EE_database_name";

$debug = 1;
$lines_per_query = 100;

function printlog($message=""){
  global $debug;
  if($debug == 1){
    echo $message."\n";
  }
}

function build_insert_fields($data) {
  $fields = array();
 foreach ($data as $field=>$value) {
  array_push($fields,'`' . $field . '`');
 }
 $field_list = join(',', $fields);
 return "(" . $field_list . ")";
}

function build_insert_values($data) {
  $values = array();
 foreach ($data as $field=>$value) {
  array_push($values, "'" . mysql_real_escape_string($value) . "'");
 }
 $value_list = join(', ', $values);
 return "(" . $value_list . ")";
}


function build_list($dir=false){
  global $exp_files, $upload_locations;
  
  if(!$dir) $dir = getcwd();
  
  $return = array();
  $scan = scandir($dir);
  
  foreach($scan as $i => $item){
    $path = $dir."/".$item;
    if($item != "." & $item != ".." & $item != "_thumbs" & !in_array($path,$exp_files)){
      
      // follow directories  
      if(is_dir($path)){
        printlog("Following directory ".$path);
        $return = array_merge($return, build_list($path));
      } else {
        
        // get all the file data
        $data = array();
        $data['file_id'] = null;
        $data['site_id'] = 1;
        $data['title'] = $item;
        $data['upload_location_id'] = isset($upload_locations[$dir."/"]) ? $upload_locations[$dir."/"] : 0;
        $data['rel_path'] = $path;
        $data['mime_type'] = mime_content_type($path);
        $data['file_name'] = $item;
        $data['file_size'] = filesize($path);
        $data['uploaded_by_member_id'] = 72; // arbetrary value
        $data['upload_date'] = filectime($path);
        $data['modified_by_member_id'] = 72; //arbetrary value
        $data['modified_date'] = filemtime($path);
        if(strpos($data['mime_type'],"image") === 0){
          $image_size = getimagesize($path);
          $data['file_hw_original'] = $image_size[1]." ".$image_size[0];
        } else {
          $data['file_hw_original'] = "";
        }
        $return[$path] = $data;
      }
    }
  }
  
  return $return;
  
}


if (!$db = mysql_connect($database_path,$database_login,$database_password)) {
  die('Could not connect: ' . mysql_error());
}
mysql_select_db($database_name, $db);

  // get files already in DB
  $exp_files = array();
  $result = mysql_query("SELECT rel_path FROM exp_files");
  while($row = mysql_fetch_array($result)){
    array_push($exp_files,$row['rel_path']);
  }
  
  // get upload locations
  $upload_locations = array();
  $result = mysql_query("SELECT id, server_path FROM exp_upload_prefs");
  while($row = mysql_fetch_array($result)) {
    $upload_locations[$row['server_path']] = $row['id'];
  }
  
  // build list
  $file_list = array_values(build_list());
  
  
  // make the SQL insert statements
  printlog("Building SQL statements");
  
  // fields
  $fields = build_insert_fields($file_list[0]);
  
  
  // values
  $values_array = array();
  foreach($file_list as $path => $data){
    array_push($values_array, build_insert_values($data));
  }
  
  $i=0;
  while(count($values_array) > $i){
    
    // make changes to the db
    printlog("Running queries $i to ".($i+$lines_per_query));
    
    $SQL = "INSERT INTO `exp_files` ".$fields." VALUES ".implode(", ", array_slice($values_array, $i, $lines_per_query));
    
    
    $result = mysql_query($SQL);
    if (!$result) {
      $message  = 'Invalid query: ' . mysql_error() . "\n";
      $message .= 'Whole query: ' . $query;
      die($message);
    }
    
    
    $i = $i + $lines_per_query;
  }
  // make changes to the db
  printlog("Success");
  
  
mysql_close($db);


?>

Hey BioLogos,

That’s awesome! This is what makes the community as awesome as it is!

This is much appreciated.

Cheers,

Hi guys,

I found issues with BioLogos script, as my server doesn’t recognize mime_content_type() or finfo_file() (PHP 5.2.6)

So I was searching other ways, and in my case, I had relative paths in (../images) and I guess this doesn’t work anymore. After putting full server path, now I can sync normal.

Maybe this will be useful for someone updating from 213.

Hey Davor,

Thanks for the tip!!

I am going to close of this thread.

If you need anything else, please just let us know by opening a new thread.

Cheers,