Upgrading the software

[Graeme S.]

Hi,

We have followed the steps to manually upgrade the files using sFTP, as the automatic one only suggested SQL that was irrelevant and had no effect, and couldn't move on.
Yes, everything is compatible, we are on PHP 8.0. All the system checks etc run.

We are upgrading from v4.5.4.2 to 4.7.14.

1. Snapshotting the server
2. Upload all the files via sFTP (took 2 hours)
3. Going to /admin/upgrade = says no application to upgrade
Fine, but we still get error messages about that we should upgrade to 4.7.14, so the files are not applied properly
4. I go to the support tab, and enter info in Step 1, so I land on Step 2
BOOM. The server CPU goes up to 200% and the server stops responding.

I think it's because, the support system says "Sometimes badly cached data can cause problems. We have cleared all cached data."

If it did clear some cache somewhere, well that created a huge load. It did this before, and I waited at that time 2 hours. Still not working. So I had to revert the server, with data loss as a result.

How can I safely upgrade from v4.5.4.2. to 4.7.14 without the entire thing blowing up?

 

[Graeme S.]

This url: https://example.com/admin/?app=core&module=support&controller=support&next=1

Visiting this URL will make it blow up, probably because of the auto cache clearing?

[Miss_B]

21 minutes ago, Graeme S. said:

Upload all the files via sFTP (took 2 hours)

Did you upload the files inside the folder that contains your Ipb forum files overwritting them?

Additionally have a look at the Upgrade section at the blow link:

 

[Graeme S.]

Hi, yes I overwrote all the files via sFTP.
Yes I have followed the steps in the Install and upgrade instructions, which says the same.

[Jim M]

1 hour ago, Graeme S. said:

3. Going to /admin/upgrade = says no application to upgrade

This would indicate the files were not uploaded to the correct spot if they were successfully uploaded at all. Please ensure that you are uploading the contents of the "ips_****" folder to your server where you have Invision Community installed, overwriting what is there.

[Graeme S.]

Yes that is done.
 

[Jim M]

18 minutes ago, Graeme S. said:

Yes that is done.
 

You will want to double check that was actually done properly. As multiple checks in the software would indicate that it has not been. Looking on your server, the last time files were touched were back in 2020 so this would indicate that this has not been done properly.

[Miss_B]

3 hours ago, Graeme S. said:

Yes that is done.
 

You should really make sure to upload the files from the latest Ipb package inside the folder that contains your current Ipb forum files and don't forget to overwrite them.

If you haven't done already a backup of the database, it's best to do it before running the upgrader.

[Graeme S.]

Thank you for your help, but this is all done.

It's really a simple instruction - upload the contents of the latest version and place and overwrite the existing files with these new files. That's it.

I think there are some things missing in the installation jumps in the code that you are not aware of. I will try a local full installation and see if that can make it work instead. It will take much longer time, but I think it's the only way forward as the instructions provided doesn't go in to any troubleshooting or help beyond the obvious.

[Marc]

4 hours ago, Graeme S. said:

Thank you for your help, but this is all done.

It's really a simple instruction - upload the contents of the latest version and place and overwrite the existing files with these new files. That's it.

I think there are some things missing in the installation jumps in the code that you are not aware of. I will try a local full installation and see if that can make it work instead. It will take much longer time, but I think it's the only way forward as the instructions provided doesn't go in to any troubleshooting or help beyond the obvious.

The reality here, is the only way it can see old files (think there is nothing to upgrade) are for the following reasons

1. The latest files are not present
2. Your server is for some reason loading old files (perhaps something like opcache)

As you overwrite any files that exist, it would be overwriting the file which the system reads to see what version it is. So its reading that file and not seeing any updated version. No matter what else is done to troubleshoot there, this is how the system works.

[Graeme S.]

I reverted it, and it's back to what is was before.
I can see that the "Upgrader" behaves the same way before with showing "There are no applications to upgrade". Perhaps we need to fix something before we can move forward?
Maybe something "core" is missing in the installation, that makes it not work later either.
Do you know where this logic comes from?

In the same installation I get the message that we should upgrade to 4.7.14 (which is not working, as it get stuck at database fixes that are irrelevant/not able to fix, even manually, and it gets stuck)

(It shows PHP 7 because it got downgraded on the server restore, I will change it before moving forward)

[Jim M]

12 minutes ago, Graeme S. said:

I reverted it, and it's back to what is was before.
I can see that the "Upgrader" behaves the same way before with showing "There are no applications to upgrade". Perhaps we need to fix something before we can move forward?
Maybe something "core" is missing in the installation, that makes it not work later either.
Do you know where this logic comes from?

You would need to confirm that you're uploading the files in the correct spot and your server is seeing those changes. There is nothing more which the software does in this area to start the upgrade process.

Have you inspected the dates as I mentioned in my previous post? This would indicate that the files are not uploading/overwriting the files on your server.

[Graeme S.]

Hi, yes, but this issue comes before I even start to upload the files, so it's unrelated to any upgrade.
I want to make sure the current installation is working first, before I try to upgrade.

But to answer the question yes I did try another upload of the files, and that time it just crashed completely, with error 500. Im 1000% sure I replaced the right folders and files, and it was more than 8000 files. I had to revert.

The next step is to do a fresh installation locally and see what's being done, we cant risk more downtime.
If anyone is a developer and can answer/have some tips, perfect. The default setup is not working as intended, all manuals has been followed.

[teraßyte]

A few points:

• Until you upload the new 4.7.14 files in the correct location, the upgrade script will tell you there's nothing to upgrade.
• After uploading the new files you get a 500 error because the last 4.7.14 version requires PHP 8 and you still need to run the upgrader from /admin/upgrade.
• If you want to be sure the upgrade works correctly, you could try a test upgrade on a copy of the forum on another folder/server first.

Edited November 20, 2023 by teraßyte

[Jim M]

17 minutes ago, Graeme S. said:

Hi, yes, but this issue comes before I even start to upload the files, so it's unrelated to any upgrade.
I want to make sure the current installation is working first, before I try to upgrade.

There is a bug on the version you're on so you will not be able to use the auto-upgrade functionality which applies the files for. If you go to /admin/upgrade prior to uploading files, you will get the "No applications to upgrade" message because you haven't uploaded the files yet.

17 minutes ago, Graeme S. said:

But to answer the question yes I did try another upload of the files, and that time it just crashed completely, with error 500. Im 1000% sure I replaced the right folders and files, and it was more than 8000 files. I had to revert.

PHP 8 is required and with any PHP upgrade, please ensure you've installed all the correct modules and any third party applications/plugins are compatible with that PHP version. As you are coming from an outdated release, odds are they are not compatible with PHP 8 but you'll need to check those 500 Internal Server Errors in your server error log. I'd personally, disable all third party applications/plugins or uninstall them prior to upgrading.

Use this compatibility checker to check compatibility:

17 minutes ago, Graeme S. said:

The next step is to do a fresh installation locally and see what's being done, we cant risk more downtime.
If anyone is a developer and can answer/have some tips, perfect. The default setup is not working as intended, all manuals has been followed.

You can utilize a test instance to test the upgrade. Keep in mind that Marc and myself are official support from Invision Community. If you did the above and still encountering issues, we would need to look at it with the issue in place.

[Graeme S.]

Hi,

I'm comparing the databases and so far I can see many differences there. Ie the table ibf_core_members have a different structure, many things has changed name or are added.
Can you provide any manual scripts to upgrade the database manually?
I have a local setup to compare now, and this is just what I've seen so far.

[Jim M]

2 minutes ago, Graeme S. said:

Can you provide any manual scripts to upgrade the database manually?

The only way to upgrade your community would be through the upgrade process.

You may wish to hire a third-party provider to perform your upgrade for you if this is not something you are comfortable performing. Alternatively, looking at our Cloud may be a good option as you will never run into this on our Cloud.

[Graeme S.]

Ok thanks for your answers.

So you are saying that if the files are uploaded properly, then the upgrader will work?

[teraßyte]

17 minutes ago, Graeme S. said:

Ok thanks for your answers.

So you are saying that if the files are uploaded properly, then the upgrader will work?

Yes. But remember that together with the files, you also need the correct PHP version, though.

 

You're likely using PHP 7.4 or an earlier version because 4.5 doesn't support PHP 8. (Support for PHP 8.0 was added in 4.6 if I remember correctly.)

[Graeme S.]

Hi, yes I have upgraded to PHP 8.1 prior to uploading the files.

[Jim M]

14 minutes ago, Graeme S. said:

Hi, yes I have upgraded to PHP 8.1 prior to uploading the files.

 

2 hours ago, Jim M said:

PHP 8 is required and with any PHP upgrade, please ensure you've installed all the correct modules and any third party applications/plugins are compatible with that PHP version. As you are coming from an outdated release, odds are they are not compatible with PHP 8 but you'll need to check those 500 Internal Server Errors in your server error log. I'd personally, disable all third party applications/plugins or uninstall them prior to upgrading.

Use this compatibility checker to check compatibility:

You would want to read my post above explaining what could cause that. Keep in mind that the version you’re on does not support PHP8 so you can ignore any errors outside of /admin/upgrade

[Graeme S.]

Ok I have resolved this. I would like to thank you for your attempts to support, but as you point out the Self-hosting option is much more technical, and not something you support. 

So, the solution for my problem was the following:
1. Snapshot the server, 
2. Dump the db locally
3. Download the new files from the client area
4. Run the new files locally (localhost, on a local php server that is) with the old db and see what's going on, tons of suggestions and errors. First of all the engine needed to change to InnoDB to allow for a different way of indexing, plus tons of other db fixes and clean ups.
I did write a 10 page manual for our own usage until next time. No place for that here. Also, add the -TESTINSTALL flag when setting it up, but that's all in the manual: 

5. Once done, it doesn't work with just uploading the files as they had been changed after the local upgrade
6. Go in to the server and manually delete all folders (except Images, Notes, Upload - your users will thank you) Do not delete via FileZilla etc. It will take forever. I went to the server and used "rm".
6. Upload "fresh" files that you did not use to upgrade your local, this is around 8000 files and will take 2-3 hours to upload.
7. After upload now you need to make a new Nginx configuration by "rebuilding the web app conf" (if you use Runcloud). Something needs to be refreshed and a server restart didn't help.
8. Now, once it's back again, you can run the /admin/ and login, and then use the upgrade feature which works surprisingly well, but takes a while, around 30-40 min.
9. Once done you are good to go. Snap shot again.

Bonus points for turning off your forum before doing all this (inside the admin panel), so they users doesnt try to write to your files while you are doing all this.

Expect down time of at least 5 hours when doing all this, not counting any mishaps or issues. 

There you go!

[teraßyte]

35 minutes ago, Graeme S. said:

6. Upload "fresh" files that you did not use to upgrade your local, this is around 8000 files and will take 2-3 hours to upload.

Small trick: Compress all the files you have to upload into a single ZIP file, upload it to the server, and then uncompress it through SSH. It saves a lot of time. 🙂

Edited November 23, 2023 by teraßyte