IPB, BBCode, and concerns about it's future

[grinler]

I have recently upgraded from IPB 3.1.4 to 3.4.3 and I have to say that experience has been not good. For the past few weeks since I performed this upgrade I have had to devote almost all of my time to fixing bugs and resolving complaints from my members. The vast majority of these issues, for my site at least, have been the mangling of BBCode in IPB. I know there are plenty of concerns about the RTE, but if BBCode worked as it should in the standard editor it would not hurt as much.

While researching all of the problems, I have stumbled on some comments that I find concerning. These are comments where IPS downplays our concerns regarding the RTE and the diminishing level of BBCode support by saying that 95% of the IPB users don't even use BBCode. That may be the case, but I could make a good bet that the vast majority of these same users aren't formatting their posts at all. I could even go one step further and say that those who use BBCode are the ones who are generating the majority of the content on the site, not the casual users who probably wont use any formatting at all. It is important for IPS to remember that the forum owners are your customers and not their visitors. If your product does not work for us then regardless of who visits our site, we are going to have to find a solution that does work.

Unfortunately for me and my site, it appears that there is a segment of your customer base that you seem to have forgotten. These are support sites like mine, BleepingComputer.com, that have used and continue to extensively use BBCode in their posts. On support sites a common method is to create databases of pre-made BBCode formatted posts called canned speeches. These can be used as necessary when providing support to a member. We have 1000s of these canned posts on my site and there are many more written by our helpers that they use on the various sites they visit. These types of pre-made posts are commonly shared among other support sites to be used by their staff, essentially creating a global canned speech database. By taking away BBCode you are going to essentially make years of work created on a variety of sites useless.

Just because BBCode is antiquated, doesn't make it a bad thing. The advantage of BBCode is that it is a standard that almost all forum software vendors currently use some form of. It works well, and is easily saved, pasted into posts, and portable between a variety of different forum software. The fact that it is so portable allows a helper to use the same canned speech at a variety of forums where they may help, whether that forum is running IPB or vBulletin.

Now you say there is no difference between using BBCode and using HTML. I agree, if you can learn BBCode, you can learn the basic HTML codes as well. What is not being mentioned is that the HTML that IPS may allow may be different than what another forum software allows. This makes HTML much less portable than BBCode. On the other hand, BBCode is a standard that is widely recognized on almost all forum software. Using [ b ] means bold. It doesn't matter if it's converted to <b> on one site, <strong> on another, or <span style="font-weight: bold"> on another. For us, the user of the BBCode, all we care is that if we use the bold BBCode on any site, it will make that text bold. By switching to pure HTML, you make it much harder for support sites to coexist with other ones and thus ultimately make it harder for the owners and their power users, who generate the most posts, to work with your software.

Personally, I think if IPS could do some form of the following it would make those sites that still rely on BBCode to continue to use IPB, while still allowing you to pursue your holy grail of the RTE editor:

• Fix BBCode. It's horribly broken in 3.4.x. Codeboxes are useless and consistently cause problems if you use them for anything but the most basic things, tags are broken, and mistakes in terminating BBCode are unforgiving and can cause fatal PHP errors.
• Stop trying to remove BBCode from IPB and maybe instead just separate BBCode, and the standard editor, from the RTE. Don't try to make them interchangeable. If you want to use and edit BBCode, use the standard editor. If you want to use WYSIWYG, then only use the RTE.
• Add BBCode controls back to the standard editor. Why were those even removed?
• Add some consistency to spacing. Why is text after a quote right on top of the quote, while there is decent spacing after the codebox? Here is an example of newlines being stripped from BBCode posts for the sake of the RTE. This one little thing has single-handedly made 1000s of posts at my site and other look like unprofessional and sloppy.
• Stop changing standard BBCodes. I must be missing something, but I just don't understand the reasoning behind changing the img and list tags?
• Stop trying to auto-convert BBCode to these new formats. Most of the time the autoconversion just does not work or does something unintended. I would rather the BBCode just not parse at all then you try and fix it for us.

As a customer I need to know from IPS what your roadmap is regarding these issues so that I can plan accordingly. All we hear is little comments here and there describing your master plan and to be honest it scares the hell out of me. I have been using IPB for over 8 years, just bought a few more licenses, and the last thing I want to do is switch to another forum vendor. With that said, can you please provide us, your clients, some specific information on what you are planning for the near future?

[broni]

Superb post!

If I may add just one suggestion/question...

Why does IPS insist on working with CKEditor since it has NOT been working properly from the very first version? To the contrary. It's been actually getting worse with every new version.

Why not to implement some editor which actually works?

I see all kind of other editors on non-IPB sites and they DO work.

[Inu]

I am losing members because of the BBcode headaches, they can't even get a simple

 box working. This is sad. Many of the reports in the bug tracker deal with bbcode.

[TSP]

Thank you for your post grinler, you bring up many points I've tried to get across since they beta tested 3.4 on this community forum. I have used many hours testing, submitting bug reports and providing feedback on the editor. What I really would like to do is to devote more time on other tasks in our community and the communities we host and develop for.

The sad thing about it is that I feel 3.4 improved in all the other areas, except the editor. There are reasons I want to upgrade to 3.4, but I feel I can't with the amount of bugs and usability issues I'm looking at with the editor and parsing.

We currently host 4 norwegian forums and are bringing another one into the fold in April. All of the three largest forums have 2000-3000 visitors during peak hours. I'm responsible for and do the majority of the developing and technical tasks for all these forums.

— 3.2.3 forum with 20+ million posts and 7 500 unique posting users a month. Here I am the community manager.

— 3.3.4 forum with 12+ million posts and 3 000 unique posting users a month. (Here they have guest posting and many use my anonymous hook to post, so actual numbers are larger)

— 3.3.4 forum with 11+ million posts and 3 000 unique posting users a month. (Here they also use the anonymous hook a lot, no guest posting)

— 3.4.3 forum. This is a staff forum to the first forum mentioned where we've been testing the 3.4 release

— A forum with 3+ million posts that will be converted to IPB 3.4.3 in April

Originally I was chasing an 3.3 upgrade for our 3.2-forum but when it was a short time until 3.4 I decided to chase 3.4 instead. So far that chase have resulted in no upgrades for our 3.2 forum or 3.3 forums. Instead it gave me headaches and massive concerns about the 3.4-editor and the future of the editor in IPB.

I get told in some support tickets that I should upgrade to resolve some of the issues we're having on 3.2 and 3.3. But I simply have to reply it's not an option for us at this time.

The problem is that they decided to change so much about the editor with their 3.4-release. In my opinion they should've kept it off the 3.X-series. The changes are just so massive.

My biggest problems:

— New lines when switching between RTE / STD (really, using them in STD at all..) is a mess.

— Bugs with code-tag

— Bugs with quote-tag

— Portability between forums

— My trust in the editor is at an all time low (I regularly get posts with <br> tags in them and no lines between paragraphs for instance. Even when I use no bbcode at all. I have also reported issues where content / text in posts would disappear in thin air. This is unacceptable.)

I really can't elaborate much more on this at the moment, but really most of my 300 latest posts and bug tracker comments are probarbly about the editor....

[Matt]

Thanks for taking the time to post that, it's definitely given me a lot to think about.

However, our plans are not to remove BBCode in 4, but merely shift all processing to one area.

We continue to fix the BBCode issues that are reported to us. Please take a moment to write up any bugs you come across and we'll make sure they're taken care of.

[Emsi]

Unfortunately our experience has been very similar and we wholeheartedly share Grinler's concerns. Like BleepingComputer we are a technical support community mostly to provide assistance with malware removal and our products. We switched from a highly customized .NET based solution to IPB around the time 3.1 was released. Our main reason was that the software itself was commonly used by other similar communities as ours (GeeksToGo, Malwarebytes, Neowin, BleepingComputer, SpywareInfo, WhatTheTech etc.) and because it was easy to use for our customers. Version 3.1.4 was pretty much perfect for both our and our users' needs but unfortunately it went straight downhill from there.

Usability went down the drain when 3.2 removed the option to disable quick replies and introduced the new editor. Like many technical support communities we are working with log files a lot. These can get huge so we usually ask user's to attach them. There is no obvious way to do that using the quick reply. The attachment options are hidden behind the "More Reply Options" button. In 3.1 that wasn't a big deal. Just disable the quick reply function and be done with it. But this is no longer possible now. As a result the most commonly asked question we get and the most common problem our forum users face is how to attach a log to a reply. Something as simple as attaching a file or uploading an image really shouldn't be as much hassle as it is right now.

Grinler already explained all the BBCode issues that the switch to CKEditor introduced. They aren't any different for us. Our staff as well as our volunteers had to rewrite their canned speeches multiple times over the past releases since 3.2. Now with 3.4 they even have to maintain different sets of canned speeches: One for IPB and one for literally everything else, as IPB introduced incompatible changes to elemental BBCode constructs like lists ([*] [/*]anyone?). While our employees don't mind, they are getting paid for doing this kind of work anyways, our volunteers do. Volunteers are the life blood of pretty much every support community (this one included I might add). You don't want to make their lives harder than they need to be by regularly breaking something as elemental as BBCode.

The new editor has been introduced over 2 years ago. It is still a horrible mess as can be seen by everyone by just looking at the sheer amount of editor related bug reports. And although hundreds of bugs are being fixed with each new version, dozens of new ones show up every time. The fact that you now started to deliberately break BBCode just to get one more bug report closed for the editor is just a testament of how sad the entire editor situation has become.

We talked to quite a few other communities what they are planning to do if IPB continues to become less and less suitable for online support and so far the answer has been either to stay at version 3.1.4 or switch to a different software entirely. Both answers aren't in Invision's interest. So we sincerely hope IPB 4.0 will be a step into the right direction.

[broni]

Matt

I'd really appreciate if you could comment on my post:

Why does IPS insist on working with CKEditor since it has NOT been working properly from the very first version? To the contrary. It's been actually getting worse with every new version.

Why not to implement some editor which actually works?

I see all kind of other editors on non-IPB sites and they DO work.

[bfarber]

Usability went down the drain when 3.2 removed the option to disable quick replies and introduced the new editor. Like many technical support communities we are working with log files a lot. These can get huge so we usually ask user's to attach them. There is no obvious way to do that using the quick reply. The attachment options are hidden behind the "More Reply Options" button. In 3.1 that wasn't a big deal. Just disable the quick reply function and be done with it. But this is no longer possible now. As a result the most commonly asked question we get and the most common problem our forum users face is how to attach a log to a reply. Something as simple as attaching a file or uploading an image really shouldn't be as much hassle as it is right now.

Improving the attachment experience for users, and the overall posting and sharing experience suite-wide, is definitely a goal on our plate for 4.0. What you are describing isn't truly a change or regression in the software - attachments have never been accepted on the fast reply area. This is something we believe we can improve on in the next major release, and I am positive the changes we will make will meet your needs in this area.

Grinler already explained all the BBCode issues that the switch to CKEditor introduced. They aren't any different for us. Our staff as well as our volunteers had to rewrite their canned speeches multiple times over the past releases since 3.2. Now with 3.4 they even have to maintain different sets of canned speeches: One for IPB and one for literally everything else, as IPB introduced incompatible changes to elemental BBCode constructs like lists ([*] [/*]anyone?). While our employees don't mind, they are getting paid for doing this kind of work anyways, our volunteers do. Volunteers are the life blood of pretty much every support community (this one included I might add). You don't want to make their lives harder than they need to be by regularly breaking something as elemental as BBCode.

The new editor has been introduced over 2 years ago. It is still a horrible mess as can be seen by everyone by just looking at the sheer amount of editor related bug reports. And although hundreds of bugs are being fixed with each new version, dozens of new ones show up every time. The fact that you now started to deliberately break BBCode just to get one more bug report closed for the editor is just a testament of how sad the entire editor situation has become.

We talked to quite a few other communities what they are planning to do if IPB continues to become less and less suitable for online support and so far the answer has been either to stay at version 3.1.4 or switch to a different software entirely. Both answers aren't in Invision's interest. So we sincerely hope IPB 4.0 will be a step into the right direction.

I apologize for your frustrations, and assure you we are working towards a more stable editor (and overall software lineup) with each release. The overall number of bug reports open is around 60, with editor bug reports comprising less than half of those. As you have indicated, each release resolves many bug reports (sometimes "hundreds" may in fact be true, although this number would include all bugs and not just editor-related bugs), and we certainly do not consider our job done yet.

Let me assure you that we do not deliberately break BBCode, and BBCode is still supported. Bugs that are reported which relate to the use of BBCode are still being investigated and resolved for upcoming maintenance releases. If you are aware of any that are not yet resolved, please do be sure to let us know.

Matt

I'd really appreciate if you could comment on my post:

CKEditor itself is not unstable. The problems we face have to do with maintaining bbcode support (primarily), which is not something a WYSIWYG like CKEditor is really designed to do. The technicalities are irrelevant, however. We maintain committed to delivering a stable editor and a positive experience when posting in the software, regardless of what software powers that editor.

For what it's worth, however - several of our competitors do indeed use CKEditor for their RTE uses. There are only a handful of suitable, stable, enterprise-class WYSIWYG solutions and CKEditor is often considered the leader of the pack (I realize opinions vary).

[broni]

If the above comment regarding CKEditor are true then I have two questions:

1. Why CKEditor seems to have more bugs (instead of less) with evry new edition? There is no question in my mind that CKEditor from IPB version 3.2.3 (which I still use mainly due to CKEditor bugs) has less bugs then in IPB version 3.4.x

2. Let's take very first board which I frequent, http://www.techspot.com/community/forums. They use mceEditor and it works perfectly fine. The very same applies to some other boards I use.

Overall I have to say CKEditor is by far the worst editor I deal with.

[Ronald Epstein]

Hello Everyone!

I am new here, just having our community brought over to IPS from Huddler.

I am not completely aware of all the differences between BBcode and HTML but I can tell you that, coming from the Huddler platform, the editor on IPS is absolutely atrocious.

I started another thread in this forum area talking about it before I even saw this thread was in progress.

Our site relies on posting press releases from software companies and movie studios. That involves cutting and pasting feature-rich text from email and word documents directly into the editor and hoping it gets preserved.

On Huddler, it worked perfectly. On IPS, it's a complete mess. Fonts are destroyed and don't even think about trying to post anything that contains vertical columns of information as all of that gets collapsed into one giant mess once posted.

I was cursing up a storm this evening after I had to edit a post and found that the IPS editor introduced a handful of HTML characters afterwards.

I sent a letter to one of the higher-ups on IPS, and from the response I received, it didn't seem like the company was that interested in doing anything to improve the editor.

Listen, Huddler didn't get a lot of things right on their platform. In most ways, the IPS experience exceeds the place we came from. However, to find ourselves working with an editor system that is so ass-backwards is very discouraging. This is something that needs to be put on top of IPS list of items to fix/enhance. Even the editor on vBulletin is far better than what IPS has implemented here.

I hope IPS is listening.

[TracyIsland]

@Ronald Epstein: I will be the first to admit I know next to nothing about the editor or BBC but have you tried using the clipboard symbol with Microsoft Word symbol? That preserves all of the markup you have in a Word document.

Just trying to help.

Brian

[Ronald Epstein]

Brian,

You *may* have pointed out something I was not aware of.

Let me wait until I get another press release sent to me and I will try that option.

Thank you so much for pointing that out. I'll be back to let everyone know if it works.

[stoo2000]

Brian,

You *may* have pointed out something I was not aware of.

Let me wait until I get another press release sent to me and I will try that option.

Thank you so much for pointing that out. I'll be back to let everyone know if it works.

You may also want to enable HTML posting for your user, tables etc are not supported within BBCODE.

[broni]

Forgive me but we seem to be drifting away from original issues.

We don't want to use RTE with workarounds but some RTE which works.

[AtariAge]

Grinler, great post. I also run into problems with the editor often on my forum, as I have many programmers trying to post code bits and the formatting often gets mangled. I also have many "High Score" clubs on my site, where lists of video game high score contests are maintained, usually with fancy formatting that ends up being a huge pain in the neck to maintain.

I'm running 3.3.4 at the moment, but I will be converting to 3.4.3 this week. I was hoping the editor situation had improved, but after reading posts in this thread (as well as monitoring the Bug Tracker for the past few versions) I'm a bit concerned that while some improvements have been made, new issues will supplant those improvements.

I can understand Invision's rationale for using a third-party editor, and it makes sense on paper. However, given how much time Invision has had to spend fixing editor-related bugs over the past few releases, I'm curious if Invision would have made a different decision if they could go back in time.

The editor plays a supremely important role in forum software, since the editor is the means through which nearly all content is generated on the forum. If users get frustrated using the editor, they may abandon your site and go find another forum with similar subject matter. If forum admins see that their users are getting stymied by the editor, they may start looking at other forum products (and there's certainly plenty of competition).

I look forward to a future post from Invision detailing what they intend to do to address this area in IP.Board 4.0.

..Al

[Rheddy]

I would also like to see implementation of bbcode and HTML to be much easier. The bbcode function is enabled by default, you don't need to enable anything in the ACP. However, as I recently discovered, I had jump through all kinds of hoops and processes in order to enable HTML to work via the post editor box. HTML posting won;t even work when you're using quick reply, which I've found a lot of members on my site love the "quick posting" feature. However, when using the quick reply, HTML won't be enabled.

I would like to see both features fixed for Community Suite version 4. When you start typing in the posting editor, all of the various formatting features appear for use. How about enabling it so that when someone wants to posts, they can simply select which editor they want to choose (HTML or bbcode). IPB already includes a drop down selector box with the bbcodes for formatting, I would suggest also including one for HTML. Sort of have a toggle switch that allows a user to switch between bbcode and HTML, like how users can enable or disable the WYSIWYG feature.

[grinler]

I'm running 3.3.4 at the moment, but I will be converting to 3.4.3 this week. I was hoping the editor situation had improved, but after reading posts in this thread (as well as monitoring the Bug Tracker for the past few versions) I'm a bit concerned that while some improvements have been made, new issues will supplant those improvements.

Al, I would seriously consider holding off to 3.4.4 if you can. A lot of the BBCode issues will hopefully be resolved in that version.

[AtariAge]

Al, I would seriously consider holding off to 3.4.4 if you can. A lot of the BBCode issues will hopefully be resolved in that version.

Thanks, I'll take a look through the Bug Tracker to see what BBCode issues have been fixed. I'm not hell bent to get onto 3.4.x, but I already did a complete migration in a test environment (it's a big forum with 2.7 million posts), merged custom code changes, and updated my custom skin to 3.4.3, so I'm ready to go. However, if 3.4.4 is right around the corner, I'd rather wait than have to upgrade again--even minor point releases can be a fair bit of work for me.

..Al

[khalwat]

I'm glad to hear I'm not the only one who is having major issues with the editor screwing up formatting when switching between BBCode and the "rich text" editor. It's been the source of many, many hours of pain for me.

[Matt]

The CKEditor component is actually fine in terms of stability. If you read through the posts here, the majority of complaints are to do with the BBCode conversion, which we agree hasn't been as smooth as we'd hoped. It's actually incredibly complex because BBCode itself can be very complex and doesn't always tally up nicely with HTML tags.

As Brandon has said, we are working to make sure this part of the editor is improved for 3.4.4 along with the code and quote box issues.

I'm not suggesting that we are or should, but if we removed BBCode completely, 95% of the complaints and bug reports would vanish as CKEditor would be doing what it does best - cleaning and formatting rich text as HTML.

[bfarber]

We just upgraded our company forums to 3.4.4 yesterday, which includes many editor and bbcode bug fixes. Please try clearing your browser cache and testing in our Test Posting forum. If you find any remaining bugs, be sure to post bug reports. We want to work together to ensure the 3.4.4 release resolves all remaining (known) issues with the editor. :)

[Samourai]

Okay so I've read everything, and I unfortunately agree with almost all the posts... Lots of bugs on my community are due to the editor, and my members are starting to find it unbearable.
BTW, I also posted two small suggestions :

[broni]

Cleared cache and tested.

One of the main issues still there.

Copying from code box with IE (Firefox works fine)

Posted three lines, wrapped in "code".

Copied three lines from the code box using IE and it copied as one line.

This particular issue is present since version 3.2.3

http://community.invisionpower.com/topic/381424-test/#entry2386884

[Wolfie]

I'm not suggesting that we are or should, but if we removed BBCode completely, 95% of the complaints and bug reports would vanish as CKEditor would be doing what it does best - cleaning and formatting rich text as HTML.

That 95% of the complaints would still be there, just in the form of "bring back bbCode" vs the problems with the editor.

One of the main issues still there.

Submit a bug report on it in the bug tracker. It's the best thing to do.

[broni]

I think it was reported right after 3.2.3 came out and.....impossible to solve I guess.