Jump to content

AWS S3 Storage HTTP 400 issue [ v4.6.10 ]

AutoMinded

By AutoMinded
in Technical Problems

 Share
Go to solution Solved by Marc Stridgen,

Recommended Posts

AutoMinded Apprentice

AutoMinded

Posted
Posted

I wanted to open a ticket but that is no longer possible.

Running latest version and I want to move the uploads to AWS S3. I have created a bucket following the very outdated guide (please update it)
https://invisioncommunity.com/4guides/advanced-options/server-management/configuring-amazon-s3-r230/

Adding the storage method results in error 400.

Quote

There appears to be a problem with your Amazon (xxxxxx) file storage settings which can cause problems with uploads.
After attempting to upload a file to the directory, the URL to the file is returning a HTTP 400 error. Update your settings and then check and see if the problem has been resolved

As I was sure everything is properly configured on AWS I have installed the AWS CLI tool. Used it on the same Linux server and on a Windows PC to double check. On both machines I'm able to list, download and upload files to the bucket. I'm also able to view uploaded images via the s3 public url. So that error 400 is not related to my bucket but to the IPB software.
https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

I'm happy to share my bucket details and access keys in a private message as I can de-activate them afterwards.

Has anything changed in your software or where can I find more information about the error please? The System & Error log are empty. Running PHP v7.4.27

Search results for similar error
https://invisioncommunity.com/forums/topic/464674-amazon-s3-file-storage-http-400-error/

 

 

IPB_StorageSettings.jpg

Link to comment
Share on other sites
AutoMinded Apprentice

AutoMinded

Posted
  • Author
Posted

Thanks and I have sent you a PM with the details.

And yes that's what I thought as well but it's working from the command line with AWS CLI from the same server 😞 

Do you know what IPB tries to do in the background please? Will maybe help to debug better.

Link to comment
Share on other sites
AutoMinded Apprentice

AutoMinded

Posted
  • Author
Posted

It's a shared server so I need to prep some things to grant support ssh access.

Can you confirm it's working on a test forum please? I have enabled logging on the bucket.

So via AWS CLI I can do anything without issues from the same server. Using IPB still gives error 400.

I digged in the code and is there a reason why you first await the error before using the correct s3 endpoint?

Anyway, on the bucket I have this error from IPB. Sensitive info has been replaced by xxx

Quote

[08/Feb/2022:09:22:20 +0000] xxx.xxx.xxx.xxx - 7G712202WDZ7D1Y3 REST.PUT.OBJECT test/16afc91dcdfbe741d633b7206e99596d.ips.txt "PUT /test/16afc91dcdfbe741d633b7206e99596d.ips.txt HTTP/1.1" 400 AuthorizationHeaderMalformed 371 - 2 - "-" "Invision Community 4" - VQF83+LNqxvUnYI27PUMKtpDn4Ng1L5W247CbeCNlUwx62kc51Yh3cE8WTdYxLACfKbyxEiZxhI= SigV4 ECDHE-RSA-AES128-GCM-SHA256 AuthHeader xxxxx.s3.eu-central-1.amazonaws.com TLSv1.2 -

Does that ring a bell please?

Link to comment
Share on other sites
AutoMinded Apprentice

AutoMinded

Posted
  • Author
Posted

My issue is resolved but it's not thanks to support. My entered information was correct and I had followed the outdated guide on your website. We pay for this software, the guides are outdated, everything works via AWS CLI and yet it was my fault?

This post is to help anyone else who runs into the same issue. Amazon advises to stop using ACLs and set security via IAM Policies. But that's not working for IPB as they test certain ACL permissions.

Debugging deeper.

Quote

[08/Feb/2022:10:13:55 +0000] xxx.xxx.xxx.xxx arn:aws:iam::856116516470:user/ipbfiles 4VJM8237XV8MK49T REST.PUT.OBJECT test/37a7b62f6cf9b5ebb2a0cec92949f69e.ips.txt "PUT /test/37a7b62f6cf9b5ebb2a0cec92949f69e.ips.txt HTTP/1.1" 400 AccessControlListNotSupported 277 - 17 - "-" "Invision Community 4" - ULGLz7YnlTSEFGYCeen7IHplGrZWu5gyM2FYTqYjJwPx4AaO/J0ls9wU/A6TjsbvRFy9DkKTEdE= SigV4 ECDHE-RSA-AES128-GCM-SHA256 AuthHeader xxx.s3.eu-central-1.amazonaws.com TLSv1.2 -

Go to your AWS Bucket --> Permissions --> Object Ownership --> Edit --> Enable ACLs

Now I can add a new Storage Profile via IPB Admin.

This step is not mentioned in the outdated guide.
https://invisioncommunity.com/4guides/advanced-options/server-management/configuring-amazon-s3-r230/

image.png.40193dd7570da2326d008464fa9b538e.png

Link to comment
Share on other sites
Marc Stridgen Grand Master

Marc Stridgen

Posted
Posted
8 minutes ago, AutoMinded said:

We pay for this software, the guides are outdated, everything works via AWS CLI and yet it was my fault?

Just to point out on this one, we provide guides on how to set up our side, and attempt to provide the AWS side. However you dont actually pay us to provide guides on how to set up AWS. I can certainly take a look at that, but I do need to clarify this point. 

Link to comment
Share on other sites
dragonfly411 Apprentice

dragonfly411

Posted
Posted
17 minutes ago, Marc Stridgen said:

Just to point out on this one, we provide guides on how to set up our side, and attempt to provide the AWS side. However you dont actually pay us to provide guides on how to set up AWS. I can certainly take a look at that, but I do need to clarify this point. 

That is correct but if IPB code is not following the latest AWS recommendation this should be mentioned. Not everybody knows how to debug issues and read PHP code 🙂

I have given plenty of information about what I had tried and shared my access key with you via PM. Since it was working with AWS CLI on the same server it had to be related to IPB code.

Solved for now and it might help other people in the future.

Link to comment
Share on other sites
Marc Stridgen Grand Master

Marc Stridgen

Posted
Posted
40 minutes ago, dragonfly411 said:

That is correct but if IPB code is not following the latest AWS recommendation this should be mentioned. Not everybody knows how to debug issues and read PHP code 🙂

I have given plenty of information about what I had tried and shared my access key with you via PM. Since it was working with AWS CLI on the same server it had to be related to IPB code.

Solved for now and it might help other people in the future.

Sorry to hear you feel this was not dealt with correctly, however you were not asked to debug anything there. In fact to the contrary. We had requested access so we could take a look. 

With regard the AWS item not following recommendations, I will bring this up internally. Please however be aware that items will change at places out of our control, and these can take time to implement on our end. The S3 documentation has also been noted internally to be looked at. There is certainly no intention in any way to be difficult.

I'm glad to see it resolved your issue in any case

Link to comment
Share on other sites
Marc Stridgen Grand Master

Marc Stridgen

Posted
Posted

I have just confirmed this with my colleagues. ALCs are used because some objects are purposely not public, while others are. So for the purposes of our software that is indeed what is required. I have made a note to update our documentation to include this. Previously this selection was not present when the documentation was done, which is why it is not there at the moment.

Hope that clarifies

Link to comment
Share on other sites
dragonfly411 Apprentice

dragonfly411

Posted
Posted

Just a suggestion for feature improvements to save potential bandwidth costs 🙂

S3 Outgoing bandwidth costs money and we host video uploads as well. So I had given public access via a policy and set a Referer condition to limit bandwidth leaching.

This works for own uploads but not for forum uploads as IPB explicitly grants public LIST access.

Policy on the bucket 

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::MyBucketName/*",
            "Condition": {
                "StringLike": {
                    "aws:Referer": [
                        "https://www.domain.com/*",
                        "https://domain.com/*"
                    ]
                }
            }
        }
    ]
}

Permissions set by IPB

bucket_list.jpg.ce1452f12c1a28c4f8ffc0de15a55089.jpg

 

Link to comment
Share on other sites
Marc Stridgen Grand Master

Marc Stridgen

Posted
Posted
10 minutes ago, dragonfly411 said:

Just a suggestion for feature improvements to save potential bandwidth costs 🙂

S3 Outgoing bandwidth costs money and we host video uploads as well. So I had given public access via a policy and set a Referer condition to limit bandwidth leaching.

This works for own uploads but not for forum uploads as IPB explicitly grants public LIST access.

Policy on the bucket 

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::MyBucketName/*",
            "Condition": {
                "StringLike": {
                    "aws:Referer": [
                        "https://www.domain.com/*",
                        "https://domain.com/*"
                    ]
                }
            }
        }
    ]
}

Permissions set by IPB

bucket_list.jpg.ce1452f12c1a28c4f8ffc0de15a55089.jpg

 

Any suggestions you would need to post up within our suggestions forum on the community. Adding them to a support request would end up with it being buried, which Im sure isnt your intention there 🙂 

Link to comment
Share on other sites
Mark Spieker Explorer

Mark Spieker

Posted
Posted
On 2/8/2022 at 5:18 AM, AutoMinded said:

My issue is resolved but it's not thanks to support. My entered information was correct and I had followed the outdated guide on your website. We pay for this software, the guides are outdated, everything works via AWS CLI and yet it was my fault?

This post is to help anyone else who runs into the same issue. Amazon advises to stop using ACLs and set security via IAM Policies. But that's not working for IPB as they test certain ACL permissions.

Debugging deeper.

Go to your AWS Bucket --> Permissions --> Object Ownership --> Edit --> Enable ACLs

Now I can add a new Storage Profile via IPB Admin.

This step is not mentioned in the outdated guide.
https://invisioncommunity.com/4guides/advanced-options/server-management/configuring-amazon-s3-r230/

image.png.40193dd7570da2326d008464fa9b538e.png

Thanks Dude! Saved my bacon, I would have NEVER EVER figured this one out, and had already invested hours of doing it over and over again hoping for a different result.

Link to comment
Share on other sites
  • 2 months later...
  • 1 year later...
AutoMinded Apprentice

AutoMinded

Posted
  • Author
Posted
On 6/10/2023 at 7:03 PM, Square Wheels said:

I don't think this is up to date still.  I tried to follow it.

 

It's a shame people are still struggling with this when the information was shared +1 year ago. Just look higher for the correct info.

Link to comment
Share on other sites
Marc Stridgen Grand Master

Marc Stridgen

Posted
Posted
2 minutes ago, AutoMinded said:

It's a shame people are still struggling with this when the information was shared +1 year ago. Just look higher for the correct info.

Have to admit to being a little confused by your response there. The guide was updated after what you had stated above, and the user is struggling after following the guide

Link to comment
Share on other sites
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
Invision Community © 2024 IPS, Inc.
×
×
  • Create New...