Join us for EEConf 2023, October 4-5 in Orlando. Speakers. Networking. Community powered. Awesome.

Join us for EEConf 2023, October 4-5 in Orlando.

Register Now!

We use cookies to improve your experience. No personal information is gathered and we don't serve ads. Cookies Policy.

ExpressionEngine Logo ExpressionEngine
Features Pricing Support Find A Developer
Partners Upgrades
Blog Add-Ons Learn
Docs Forums University
Log In or Sign Up
Log In Sign Up
ExpressionEngine Logo
Features Pro new Support Find A Developer
Partners Upgrades
Blog Add-Ons Learn
Docs Forums University Blog
  • Home
  • Forums

Cloud Files using S3 bucket won't sync sub directories in the bucket

Development and Programming

Torch's avatar
Torch
14 posts
2 months ago
Torch's avatar Torch

Everything else works if I have files in the root of my S3 bucket. If I add a folder to the bucket with more files inside it, it doesn’t show or sync. The sync count does recognize the file count, but syncing does nothing. All my policies are set to not deny any access so I’m not sure what the problem is. It’d be nice if there was some documentation on getting this set up, it’s been an absolute nightmare to even get this far.

       
Torch's avatar
Torch
14 posts
2 months ago
Torch's avatar Torch

Confirming that if I create a folder from within EE it does create it in the s3 bucket. But if I create the folder in the s3 bucket first and then try to sync in EE it just won’t show.

       
Tom Jaeger's avatar
Tom Jaeger
358 posts
2 months ago
Tom Jaeger's avatar Tom Jaeger

Hi Torch,

Thanks for posting this. We took a quick peek at things this AM, and it looks like there may be a logic issue. We’re diving into it currently.

I’m also sorry hear hear it’s been a nightmare thus far for you. We’re also doing a large overhaul on the docs for cloud files currently.

We’ll post an update here shortly.

Cheers,

-Tom Jaeger

       
Torch's avatar
Torch
14 posts
2 months ago
Torch's avatar Torch

Thanks Tom, the nightmare is more on the AWS side 😊, a lot has changed with policies and ACL requirements to get things running since I last used it. Good to know there may be a fix in place though for cloud files to recognize the folder prefixes.

       
Bryan's avatar
Bryan
7 posts
2 months ago
Bryan's avatar Bryan

Hi Torch,

Thanks again for bringing this issue to our attention. We have released a fix in version 1.0.2 which is now available for download here https://expressionengine.com/add-ons/cloud-files

We are also in the process of improving the documentation and making the bundled docs folder accessible through the Add-on store.

Please let us know if you are still experiencing this issue with the new version.

       
Torch's avatar
Torch
14 posts
2 months ago
Torch's avatar Torch

Thanks for that! I installed the new version but now getting errors when i go to …admin.php?/cp/files: I’ve tried in both PHP 7.4.33 and 8.1.13 The directories do seem to be syncing up though, so that’s improved. Just getting the PHP errors.

Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/S3/RegionalEndpoint/ConfigurationProvider.php, line 74

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/DefaultsMode/ConfigurationProvider.php, line 76

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/Endpoint/UseFipsEndpoint/ConfigurationProvider.php, line 73

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/Endpoint/UseDualstackEndpoint/ConfigurationProvider.php, line 74

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/EndpointDiscovery/ConfigurationProvider.php, line 77

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/Retry/ConfigurationProvider.php, line 79

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/ClientSideMonitoring/ConfigurationProvider.php, line 81

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/S3/UseArnRegion/ConfigurationProvider.php, line 74

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/S3/RegionalEndpoint/ConfigurationProvider.php, line 74

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/DefaultsMode/ConfigurationProvider.php, line 76

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/Endpoint/UseFipsEndpoint/ConfigurationProvider.php, line 73

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/Endpoint/UseDualstackEndpoint/ConfigurationProvider.php, line 74

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/EndpointDiscovery/ConfigurationProvider.php, line 77

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/Retry/ConfigurationProvider.php, line 79

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/ClientSideMonitoring/ConfigurationProvider.php, line 81

Severity: E_DEPRECATED Deprecated Use of “self” in callables is deprecated user/addons/cloud_files/vendor-build/aws/aws-sdk-php/src/S3/UseArnRegion/ConfigurationProvider.php, line 74

Severity: E_DEPRECATED Deprecated Creation of dynamic property ExpressionEngine\Dependency\GuzzleHttp\Handler\CurlMultiHandler::$_mh is deprecated user/addons/cloud_files/vendor-build/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php, line 90

Severity: E_DEPRECATED Warning Cannot modify header information - headers already sent by (output started at ee/legacy/core/Exceptions.php:120) ee/ExpressionEngine/Boot/boot.common.php, line 478

Severity: E_WARNING

       
Bryan's avatar
Bryan
7 posts
2 months ago
Bryan's avatar Bryan

Awesome, glad the syncing is fixed!

As far as those deprecation warnings go are you sure you’re not running PHP 8.2? We have a patch in the works that resolves this issue on PHP 8.2 but that’s the only version where they started to deprecate the use of “self” in callables as your warnings describe. It could be that you’re changing the CLI version of PHP but the web server version is stuck on 8.2 (you could verify this by visiting the PHP info utility in the control panel).

But either way we should have an update out soon that addresses this deprecation.

       
Torch's avatar
Torch
14 posts
2 months ago
Torch's avatar Torch

Sorry about that, I thought i switched versions but it was still hung up in what i thought was 8.1.13. Anyway, confirmed running fine in 7.4.33. Thank you.

πŸ’― 1
       
Torch's avatar
Torch
14 posts
2 months ago
Torch's avatar Torch

Hi, just curious if there’s anything to do to improve performance of my template pages that are loading files that i’m displaying using the File Grid field and files from AWS.

I have some entries that i add about 3-5 pdfs to using the File Grid, the file grid is used to selected pdfs that are in a AWS S3 bucket. Everything works fine, but entries that have at least 1 pdf associated with that field takes considerably more time to load on the front end than an entry that doesn’t have any files associated. Odd because the template tag is only referencing the URL to the pdf on AWS, I’m not loading any images or anything heavy.

Very simple template tag I’m using:

{tech_manuals}
            {tech_manuals:file}
            <li>
              <a href="http://{url}" target="_blank" rel="noopener">{title}</a>
            </li>
            {/tech_manuals:file}
          {/tech_manuals}

Same template used, but an entry displaying nothing from the file grid load in milliseconds, an entry with even 1 file in the File Grid takes up to 3-5 seconds.

       
Bryan's avatar
Bryan
7 posts
2 months ago
Bryan's avatar Bryan

Hi Torch, thanks for letting us know about this issue. I think it is related to another performance issue with remote filesystems that we are looking into currently. I don’t think there is any short-term fix I can give you at the moment but we’re hoping to have a new minor release out in the next week that should also address the performance problem. I will try to set up a similar test case for your situation and make sure it improves the template load time as well.

πŸ”₯ 1 πŸ‘ 1
       

Reply

Sign In To Reply

ExpressionEngine Home Features Pro Contact Version Support
Learn Docs University Forums
Resources Support Add-Ons Partners Blog
Privacy Terms Trademark Use License

Packet Tide owns and develops ExpressionEngine. © Packet Tide, All Rights Reserved.