Jump to content

IPS QA TEAM ?


najaru_1

Recommended Posts

Does the now-official QA team mean the end of publicly available (to active customers) beta tests?

Gallery 4.2, for example, was not made generally available in beta form as far as I am aware - at least not for any sensible length of time. Judging by the bug tracker, that appears to have been a bad decision (there are 4 confirmed "critical" status bugs for 4.2.0 on the first page alone). As a result I personally am treating 4.2.0 as a beta and have no intention of installing it on my live site.

Link to comment
Share on other sites


Does the now-official QA team mean the end of publicly available (to active customers) beta tests?



Gallery 4.2, for example, was not made generally available in beta form as far as I am aware - at least not for any sensible length of time. Judging by the bug tracker, that appears to have been a bad decision (there are 4 confirmed "critical" status bugs for 4.2.0 on the first page alone). As a result I personally am treating 4.2.0 as a beta and have no intention of installing it on my live site.




They will upgrade to the latest versions on the forums before releasing so that it can also be tested out here but with more uses then a single person can give in a test of the product.
Link to comment
Share on other sites


They will upgrade to the latest versions on the forums before releasing so that it can also be tested out here but with more uses then a single person can give in a test of the product.



Does that mean the answer to the question in my first sentence is "yes" then?

Upgrading this site doesn't help with public testing of anything that requires ACP access or non-standard permissions. As ever with testing, the more people that have the opportunity to install the software, the more likely that bugs will be found prior to release.
Link to comment
Share on other sites


Does the now-official QA team mean the end of publicly available (to active customers) beta tests?



Gallery 4.2, for example, was not made generally available in beta form as far as I am aware - at least not for any sensible length of time. Judging by the bug tracker, that appears to have been a bad decision (there are 4 confirmed "critical" status bugs for 4.2.0 on the first page alone). As a result I personally am treating 4.2.0 as a beta and have no intention of installing it on my live site.



Nothing has changed. We've always had a QA team - the only thing that's new is the badges, and we recruited a few more people.
Link to comment
Share on other sites

OK, so it was available to a slightly larger restricted group than I thought... it's still not the same as proper public beta testing is it? :)

The current state of the Gallery bug tracker should be all the evidence you need to show that the testing process for this particular release was inadequate.

Link to comment
Share on other sites


OK, so it was available to a slightly larger restricted group than I thought... it's still not the same as proper public beta testing is it? :smile:



The current state of the Gallery bug tracker should be all the evidence you need to show that the testing process for this particular release was inadequate.




We are very well aware of the issues with gallery, we have all been working very hard on it as well, both to improve the process, and resolve any outstanding issues. We will have an update very soon.
Link to comment
Share on other sites


We are very well aware of the issues with gallery, we have all been working very hard on it as well, both to improve the process, and resolve any outstanding issues. We will have an update very soon.




I'm glad to hear you are trying to improve the process, I look forward to seeing what you come up with.

Nobody expects entirely bug-free releases because (a) that is impossible, and (b) testing can never replicate large-scale real-world deployment. However my experience as a customer over the last 18 months has been that IPS has a poor record when it comes to critical bugs in release versions.

I would also question how effective your version control is. Time and again I see bugs in the tracker that are marked as fixed, but are actually not fixed when the next release comes around, or bugs that were fixed previously but then recur in subsequent versions.

You should also be able to work on and issue bug-fix releases while simultaneously and separately developing new features in another code branch. This is something I have never seen happen at IPS. Once you've decided to work on a new major version, all bug-fixing point releases for the current version cease. That means customers can be faced with many months of waiting for the new features to be ready before outstanding bugs get fixed. I'd be interested to know why you take this approach.
Link to comment
Share on other sites


I would also question how effective your version control is. Time and again I see bugs in the tracker that are marked as fixed, but are actually not fixed when the next release comes around, or bugs that were fixed previously but then recur in subsequent versions.





... have to say something here.... do you code?
If you did, you would realize that many times, fixing one bug can cause the re-emergence of another.
A fix that looks to be kosher and proper to fix one bug may have rippling effects throughout the code.
Link to comment
Share on other sites


... have to say something here.... do you code?


If you did, you would realize that many times, fixing one bug can cause the re-emergence of another.




I am not a developer myself but work closely with developers in my day job, so am familiar with the problem of regression. I sympathise with developers, but the fact that reoccurring bugs are understandable does not make them acceptable. There are various practices that can help to prevent this from occurring, effective use of version control being one.

I obviously don't see behind-the-scenes at IPS so can only give my impression as a user, and that is that old bugs reoccur or "fixed" bugs are not fixed too frequently for IPS's regression testing and version control to be sufficient.
Link to comment
Share on other sites

Actually, version control does virtually nothing to prevent an old bug from resurfacing. Old bugs do not resurface because we reverted a version in SVN - they recur because of numerous other reasons, usually because fixing the original bug causes a new bug, and when that new bug is fixed, it is usually by changing the code that was implemented when fixing the original bug... e.g., human mistake. No amount of version control (which yes, we employ), stops the human factor in software development.

Link to comment
Share on other sites


Does the now-official QA team mean the end of publicly available (to active customers) beta tests?



Gallery 4.2, for example, was not made generally available in beta form as far as I am aware - at least not for any sensible length of time. Judging by the bug tracker, that appears to have been a bad decision (there are 4 confirmed "critical" status bugs for 4.2.0 on the first page alone). As a result I personally am treating 4.2.0 as a beta and have no intention of installing it on my live site.




If you are interested I have just put a 4.2.1 version into a special QA forum available to all clients who have an active IP.Gallery licence.

So if you would like to test the version before we release it as final, you are more than welcome.

Link to comment
Share on other sites


I am not a developer myself but work closely with developers in my day job, so am familiar with the problem of regression. I sympathise with developers, but the fact that reoccurring bugs are understandable does not make them acceptable. There are various practices that can help to prevent this from occurring, effective use of version control being one.



I obviously don't see behind-the-scenes at IPS so can only give my impression as a user, and that is that old bugs reoccur or "fixed" bugs are not fixed too frequently for IPS's regression testing and version control to be sufficient.




Hence the reason for Q&A before the public beta release.
Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...