Invision Community 4: SEO, prepare for v5 and dormant account notifications Matt November 11, 2024Nov 11
Posted February 4Feb 4 Hi,I recently added a new file storage configuration using Amazon S3 in my Invision Community (v4.7.20). The migration process started successfully, and files from the uploads/* folder were being moved to S3. However, I encountered two issues:Some files were not moved, even though the migration process has been marked as completed. Is there a way to re-run the synchronization to ensure all files are correctly transferred?Some files exist on Amazon S3 but are missing from the local uploads folder. However, when attempting to access them, the forum still generates URLs pointing to uploads/ instead of the S3 bucket. It seems like the migration process moved these files, but their storage path wasn't updated correctly.Is there a way to validate and fix this?If necessary, where can I manually update the file storage paths for these files?Any guidance would be greatly appreciated!Thanks!
February 5Feb 5 When a file is moved to S3, the local copy is deleted from the /uploads folder on your server at the end of the move. If you still have files there, they were not in the file storage you moved to S3.As for files still pointing to /uploads, those links were most likely manually copied/pasted into the editor/posts. Their URLs won't automatically update to the new location. You'd need to manually edit the posts or run a custom script to fix the links. Edited February 5Feb 5 by teraßyte
February 5Feb 5 Author 4 hours ago, teraßyte said:When a file is moved to S3, the local copy is deleted from the /uploads folder on your server at the end of the move. If you still have files there, they were not in the file storage you moved to S3.As for files still pointing to /uploads, those links were most likely manually copied/pasted into the editor/posts. Their URLs won't automatically update to the new location. You'd need to manually edit the posts or run a custom script to fix the links.Hi, thanks for your response!Could you clarify what you mean by "manually copied/pasted"? I noticed that most of the broken images are found in user signatures and have no attach_thumb_location in the core_attachments table, while they do have an attach_location that is stored in S3. Is this related to how they were inserted, or is there another reason?Also, regarding the first part of your response, do you know if there’s any additional way to verify that all files were successfully moved to S3? Or is it possible that the process was interrupted at some point, leaving some files in /uploads without being copied to S3?Also, I see that the core_file_storage table contains two methods: FileSystem and Amazon. However, I don’t see a table that explicitly indicates which files should use Amazon storage.Could you clarify how IPB determines which storage method to use for specific files? Is there another table or setting that links files to a specific storage method?Thanks again for your help!
February 5Feb 5 What terabyte means there, is someone manually copying a link to an item. We would need an example to see in order to advise further there. In order to answer the questions however11 hours ago, goldmorphin said:Some files were not moved, even though the migration process has been marked as completed. Is there a way to re-run the synchronization to ensure all files are correctly transferred?There is no way in which to re-run. If you have a serious issue with these, it would be best to revert to your backup taken before this started11 hours ago, goldmorphin said:Some files exist on Amazon S3 but are missing from the local uploads folder. However, when attempting to access them, the forum still generates URLs pointing to uploads/ instead of the S3 bucket. It seems like the migration process moved these files, but their storage path wasn't updated correctly.Is there a way to validate and fix this?If necessary, where can I manually update the file storage paths for these files?That sounds like someone has manually added links to posts rather than attachments on the posts themselves.
February 5Feb 5 Author 1 hour ago, Marc said:What terabyte means there, is someone manually copying a link to an item. We would need an example to see in order to advise further there. In order to answer the questions howeverThere is no way in which to re-run. If you have a serious issue with these, it would be best to revert to your backup taken before this startedThat sounds like someone has manually added links to posts rather than attachments on the posts themselves.Hi @Marc ,Thanks for your reply!Unfortunately, reverting to a backup is not an option in this case, so I’ll need to find a way to fix the issue with the current setup.Regarding the situation with links, I’ve attached two screenshots showing a member’s signature.In the thread, the image URL appears as:https://my_forum_address/uploads/monthly_2019_02/209050206_.png.051f14ac59663ff197c50460183ef21f.pngHowever, in the signature edit menu, the URL is:<my_s3_bucket_address>/<bucket_name>/monthly_2019_02/209050206_.png.051f14ac59663ff197c50460183ef21f.pngThe file does exist in the S3 bucket and can be accessed without any issues.Here’s how the file appears in the database:mysql> select * from core_attachments where attach_location like "monthly_2019_02/209050206_.png.051f14ac59663ff197c50460183ef21f.png" limit 1 \G *************************** 1. row *************************** attach_id: 24741 attach_ext: png attach_file: пламя.png attach_location: monthly_2019_02/209050206_.png.051f14ac59663ff197c50460183ef21f.png attach_thumb_location: attach_thumb_width: 0 attach_thumb_height: 0 attach_is_image: 1 attach_hits: 1 attach_date: 1550040599 attach_post_key: attach_member_id: 9670 attach_filesize: 32922 attach_img_width: 341 attach_img_height: 192 attach_is_archived: 0 attach_security_key: NULL attach_labels: NULL attach_moderation_status: skipped attach_img_rotate: 0 1 row in set (0.07 sec)Do you have any idea why this is happening? Is this expected behavior?I’m happy to write a script to process cases like this, but I’m not sure why the forum is pulling the file path from the root directory in one case and from the S3 bucket in another.Any insights or suggestions would be greatly appreciated!Thanks again!
February 5Feb 5 We would need to look further into this for you, however the access details on file appear to be incorrect or missing. Could you please update these details by visiting your client area, selecting the relevant purchase, then clicking "Review/Update Access Information" under the "Stored Access Information" section. We look forward to further assisting you.
February 5Feb 5 Author 2 minutes ago, Marc said:We would need to look further into this for you, however the access details on file appear to be incorrect or missing. Could you please update these details by visiting your client area, selecting the relevant purchase, then clicking "Review/Update Access Information" under the "Stored Access Information" section. We look forward to further assisting you. Thank you for your response.Currently, my forum is restricted by a whitelist as part of the process of resolving this issue. Would it be possible for you to provide a list of IP addresses that your team will be using to access the site? That way, I can add them to the whitelist and ensure you have the necessary access.Looking forward to your reply.
February 6Feb 6 Author Hi @Marc,I've updated the access details as requested and also added your IP. Please let me know if you need anything else to proceed with the investigation.Looking forward to your response.
February 6Feb 6 Specific links to a couple of posts with the issue if you can please. I will then get a ticket created for you
February 6Feb 6 Author 1 hour ago, Marc said:Specific links to a couple of posts with the issue if you can please. I will then get a ticket created for youAt the moment, I have found this topic with a comment where the member's signature is missing:🔗 https://[my_forum_domain]/topic/23034-rpg-zerus/?&_rid=1#findComment-106021For now, I would like to focus on this specific case as I believe the issue might be more widespread. Please replace the domain accordingly when accessing the link.Let me know if you need anything else. Thanks!