AtariAge Posted October 2, 2023 Posted October 2, 2023 31 minutes ago, UncrownedGuard said: Between the IPS Cloud outages, the glances that I have seen look very good! I hope this project takes off as a central point for 3rd party tools was what made the marketplace great! Yeah, my forum has mostly been down the last hour, so I'm not very keen on the IPS Cloud service right now. ..Al UncrownedGuard and David N. 2
Alex Duffy Posted October 2, 2023 Posted October 2, 2023 I just got these errors on both my site and this forum. Both are very slow to load now.
David N. Posted October 2, 2023 Author Posted October 2, 2023 22 minutes ago, Alex Duffy said: I just got these errors on both my site and this forum. Both are very slow to load now. Same here, again intermittently down/slow with 504 errors, this time I confirmed the down time with "Down For Everyone or Just Me". I'm still experiencing those errors now intermittently.
AGambler Posted October 2, 2023 Posted October 2, 2023 Same here, I also noticed that https://nvisioncommunity.com/ is consistently experiencing a 504 error. When will this be fixed? It's really annoying. I've been seeing this error for 2 days now.
Alex Duffy Posted October 2, 2023 Posted October 2, 2023 Both are back to full speed now. It was only this morning that I noticed this.
Marc Posted October 2, 2023 Posted October 2, 2023 Sorry to see you were having problems. These have indeed now been resolved, and sites should be back to normal
Marc Posted October 2, 2023 Posted October 2, 2023 Sorry to hear you have been having issues. We have now resolved these and sites should be back up and running AGambler 1
David N. Posted October 2, 2023 Author Posted October 2, 2023 3 minutes ago, Marc Stridgen said: Sorry to hear you have been having issues. We have now resolved these and sites should be back up and running Thank you Marc, I appreciate it, however the general idea was that this is happening more frequently than it should, and a long term solution should be implemented so this does not occur again. Here's a quote from @Matt from March 22: On 3/22/2023 at 1:25 PM, Matt said: As Charles said, it is very unusual for us to experience this, so we are incredibly frustrated. Our uptime for 2022 was 100%, and so far this year including this event, it's 99.68% so it really is a very rare occurrence. It doesn't mean we're being complacent, but it illustrates that this isn't the usual experience. Unfortunately, this experience is still too frequent.
Management Matt Posted October 2, 2023 Management Posted October 2, 2023 We're still gathering data, but we experienced a significant network attack between 11:00pm and 3:15am EDT which affected roughly 15% of our client accounts who experienced degraded performance and short pockets of intermittent downtime. It has since been resolved but we'll need to do a closer look at the data during US office hours for a more complete and accurate picture. David N. and Marc 2
David N. Posted October 2, 2023 Author Posted October 2, 2023 7 hours ago, UncrownedGuard said: Between the IPS Cloud outages, 6 hours ago, AtariAge said: Yeah, my forum has mostly been down the last hour, so I'm not very keen on the IPS Cloud service right now. I've been complaining about the lack of reliability of IPS Cloud hosting services for the past few months. Please do voice your experience/opinion:
Management Matt Posted October 2, 2023 Management Posted October 2, 2023 Hi David, Any period of degraded performance is frustrating for us all. We have put a lot of energy and cost into ensuring our systems stay online and weather even the most persistent online storms but very rarely some do get through and we need to tweak our protection. Looking back at our independently monitored uptime, we have a 99.9% uptime record with very few periods of significant downtime since Jan 1st 2023. What happened in the early hours of the morning was inconsistent with our long standing performance records and we do not take it lightly. We continue to improve our systems and will keep you updated of any changes. Thank you again for your patience. You, or anyone else reading is very welcome to submit a support ticket if they have any questions or concerns. You are welcome to bookmark and review our independent status page: https://status.invisioncommunity.com P.S. I'm going to move this to the existing topic.
UncrownedGuard Posted October 2, 2023 Posted October 2, 2023 1 hour ago, David N. said: I've been complaining about the lack of reliability of IPS Cloud hosting services for the past few months. Please do voice your experience/opinion: Honestly, the experience for me as been pretty great. They had that major downtime earlier this year and now this, but overwise I run at 100% up time every month. I use updown.io as it is one of the cheaper solutions to track my uptime and I'd recommend a service like that for your site to give a more reportable data set. Matt 1
Management Matt Posted October 2, 2023 Management Posted October 2, 2023 Looking through David's ticket history, I can see there have been some difficult to track issues that impact his community only and aren't indicative of the greater experience, these seem more around database optimisation for his community or slow outbound connections. It's something our support team are working with David on.
Max Posted October 2, 2023 Posted October 2, 2023 7 hours ago, Matt said: Looking through David's ticket history, I can see there have been some difficult to track issues that impact his community only and aren't indicative of the greater experience, these seem more around database optimisation for his community or slow outbound connections. It's something our support team are working with David on. We have a lot of users complaining about the same thing.
David N. Posted October 3, 2023 Author Posted October 3, 2023 (edited) 17 hours ago, Matt said: Looking through David's ticket history, I can see there have been some difficult to track issues that impact his community only and aren't indicative of the greater experience These are unrelated to my complaining about the Cloud hosting service being intermittently down/slow for everyone, and for periods of times ranging from a few minutes to 6 hours (earlier this year). When this happens, I check that: My internet connection is working fast as usual. I can access any other site. My site is down. Invisioncommunity.com is down. Both my site and Invisioncommunity.com are confirmed as being down on https://downforeveryoneorjustme.com/ So far I've experienced this happening: October 1, September 25, September 8, april 25, march 21 ... which is much more often than I was experiencing when I was self hosting. Edited October 3, 2023 by David N.
Management Matt Posted October 3, 2023 Management Posted October 3, 2023 When we are hit by a network attack (which happens daily), our automation launches new instances to meet demand. On some occasions (a few times this year) there may be some degraded performance for between 1 and 5 minutes while scaling launches. Earlier in the year we had an outage with our upstream provider which was incredibly frustrating. No one likes degraded performance or downtime, let alone our team who are woken by automated alarms at 2am to come and deal with it, but it's the nature of the internet. When you compile a list it may look significant, but it's still just tiny blips in an otherwise robust service. We really do our best to minimise disruption and we'll continue to revise our alarms, scaling and database instances. Max, David N. and SeNioR- 2 1
David N. Posted October 3, 2023 Author Posted October 3, 2023 31 minutes ago, Matt said: We really do our best to minimise disruption and we'll continue to revise our alarms, scaling and database instances. I understand that on your end, you're doing your best to deal with technical challenges. I have complete faith in the Invision Community team. I've witnessed the competence and reactivity of your team both on this forum and privately when using support tickets. I've invested a good chunk of my budget and a lot of my time into your platform which is now at the core of my business plan, and I intend to continue doing so in the long term. On my end, as a service user, I can't help but compare the service I'm getting. I use two points of comparison: My experience with my previous hosting provider. My visitor experience with websites that I visit frequently. So far this year I've found IC Cloud to be less reliable than my previous host or my experience visiting those other sites, which is concerning to me - hence this thread. I have a question regarding https://status.invisioncommunity.com/ - how come when I click History & Incidents, I see 100% uptime for Invision Community Platform (US) for the past 30 days even though there was some down time? georgebkk 1
Management Matt Posted October 3, 2023 Management Posted October 3, 2023 I'm honestly unsure. I think it's two factors, the first being that the downtime was intermittent with degraded performance (1-2 mins down, then 5-6 up) - but pages were being returned and that it only affected about 15% of our client sites. I guess Uptime rounds up or down and it wasn't statistically significant when balanced with the entire network and 24 hour period. If you look at our site which was in the affected 15%, then you can see the short periods of true downtime between slower page loads - but it was a few minutes at a time. David N. and georgebkk 2
AGambler Posted October 4, 2023 Posted October 4, 2023 I still have a problem. The forum was down again. When will this be fixed?
Jim M Posted October 4, 2023 Posted October 4, 2023 1 hour ago, AGambler said: I still have a problem. The forum was down again. When will this be fixed? There was a slight attack at 1pm EDT but the system scaled and is working again. Could you please clarify if you're still having an issue? I am able to access your community. You can watch our status page for information on this too.
Max Posted October 5, 2023 Posted October 5, 2023 On 10/3/2023 at 5:05 AM, David N. said: I understand that on your end, you're doing your best to deal with technical challenges. I have complete faith in the Invision Community team. I've witnessed the competence and reactivity of your team both on this forum and privately when using support tickets. I've invested a good chunk of my budget and a lot of my time into your platform which is now at the core of my business plan, and I intend to continue doing so in the long term. On my end, as a service user, I can't help but compare the service I'm getting. I use two points of comparison: My experience with my previous hosting provider. My visitor experience with websites that I visit frequently. So far this year I've found IC Cloud to be less reliable than my previous host or my experience visiting those other sites, which is concerning to me - hence this thread. I have a question regarding https://status.invisioncommunity.com/ - how come when I click History & Incidents, I see 100% uptime for Invision Community Platform (US) for the past 30 days even though there was some down time? This is the exact same experience for us. We never had issues on our dedicated server. I love the idea of being able to scale with cloud hosting for our bigger traffic days. So far that hasn't been the experience and I am hearing it from the community. David N. and WebCMS 1 1
David N. Posted November 7, 2023 Author Posted November 7, 2023 This is happening again right now. My site and this site are both super slow or completely down.
InfinityRazz Posted November 7, 2023 Posted November 7, 2023 10 minutes ago, David N. said: This is happening again right now. My site and this site are both super slow or completely down. https://status.invisioncommunity.com/ Tbf, does say systems under maintenance..
David N. Posted November 7, 2023 Author Posted November 7, 2023 (edited) Ok so it appears as if things are back to normal. This incident lasted from 8:23 until 8:38 GMT. I had just sent an email to our mailing list and done social network promotion to direct people to a major announcement on our site, which was mostly down or unusable for 15mn. This is very, very frustrating. Edited November 7, 2023 by David N.
Dll Posted November 7, 2023 Posted November 7, 2023 We're seeing the same things in the EU data centre. It's so frequent at the moment, and very frustrating, particularly as support replies essentially amount to 'it's not happening now, so I can't investigate'. We'll be heading back to self-hosted if it continues, sadly. asd937, David N., WebCMS and 1 other 1 1 2
Recommended Posts