PurplePixel

I am testing conversion from vBulletin 3.8.11 but BB Code does not seem to convert. Am I missing any settings or this is the normal behaviour?

Thank you @CodingJungle, looks really good. I will test it. It's been frustrating not having a proper way of debugging code and if the codebase would have used some class in order to eval the code instead direct eval() that would be easy to hook up to and do this.

@Stuart Silvester It probably avoids only for the themes. But not for plugin and application hooks as they are generated on-the-fly. An it's not the only code that is generated on the fly. Search for eval( in the source code so you can see how many times you will find this.

Debugging eval() issues (hooks, etc.) are very hard since the code is loaded in the memory, so I wrote a class that can write in the /tmp/ directory the php eval'd code so one can use xdebug on the code. Moreover, it does search for the namespace and the first class name and sets the file names based on that (for the ones that do not have a class insides the file name will be sha1($code)). I have shared this patch in hope you can implement something similar for a better debugging mechanism. It can be imported with git so you can verify it. It would be great if the core would use a class like EvalDebug::eval($code) instead of eval($code) which is impossible to modify to write to disk (and believe me I really tried to do that). Even if the EvalDebug class does not contain anything else except eval($code) inside. Like this: class _EvalDebug { public static function eval(string $code) { return eval($code) } } This way, developers can manipulate the mechanism of eval(). Let me know what you think, thanks! eval_disk_writer.patch

Will do. By default it's just mkdir() and that's what php.net says: So everywhere in the code it's actually 0777 set since the $mode parameter is not specified 😄 I will try and see if it worth the effort. I think during conversion some image manipulation is happening actually. Thanks, will do that.

Thanks @Ryan Ashbrook and @Paul E. for advices will check what fits best for us and get back to you if I need more information. 🙂

I have found a few bugs which I have fixed through hooks. It would be great if you fix them in the forum directly. Here's a list: #1 // File: system/Theme/Advanced/Theme.php // From: ... protected static function _writeThemePathDirectory( $app, $container, $path ) { ... if ( ! @mkdir( $dirToWrite ) ) ... } ... // To: ... protected static function _writeThemePathDirectory( $app, $container, $path ) { ... // This fixes recursive directory creation as some applications seem to need. if ( ! @mkdir( $dirToWrite, 0777, true ) ) ... } ... #2 // File: system/Image/Image.php // From: ... public static function create( $contents ) { ... $signatures = array( 'gif' => array( '47' . '49' . '46' . '38' . '37' . '61', '47' . '49' . '46' . '38' . '39' . '61' ), 'jpeg' => array( 'ff' . 'd8' . 'ff' ), 'png' => array( '89' . '50' . '4e' . '47' . '0d' . '0a' . '1a' . '0a' ) ); ... } ... // To: ... public static function create( $contents ) { ... $signatures = array( 'gif' => array( '47' . '49' . '46' . '38' . '37' . '61', '47' . '49' . '46' . '38' . '39' . '61' ), 'jpeg' => array( 'ff' . 'd8' . 'ff' ), 'png' => array( '89' . '50' . '4e' . '47' . '0d' . '0a' . '1a' . '0a' ), // Some vBulletin images have this signature. 'bmp' => array( '42' . '4d' . '36' . '4f' . '00' . '00' . '00' . '00', ) ); ... } ...

I managed successfully on setting up the conversion data and start the conversion but I cannot see any of the imported forum in the front-end. I tried to figure it out and discovered that forums do not have permissions set, so that is why. Is there any way to automate permissions to be set to visible on all imported data? Or at least a fast way to mass-edit the forums? We have a few hundreds...

Are there tests suites available for the core Invision code and functionality? We would like to know if our plugins and app potentially break key functionality in an automated way.

Thanks @Paul E. and @Stuart Silvester for the great feedback you both shared! I can see you both are very experienced in migration and all the advices you wrote will be added to our todo before migration list. It's like war: expect for the worst and hope for the best. I think I wasn't very clear about this part. I was trying to say that this is my local database that I am iterating with such sample so I can see what fails and how to improve fast. When the sample works as desired, we can start testing the conversion on the whole database, then see what will fail, benchmark it, profile it and improve it. After we are happy with this part, I think we can create on a temporary server with the real data for beta testing. Server where our team and forum members can test the solution. That's a great piece of advice. We have to instruct our members to report anything, not only things that are clearly off. That is what I had in mind, either /dev/null, or a fake smtp catcher so we can also see if the communication is working and looking as desired. I think you are talking about the EMAIL_DEBUG_PATH constant, which can write the outgoing e-mails to disk instead of sending them. Sounds like a good approach since we can use /dev/null while testing and write to disk on a later stage. Yes, I am pretty sure there will be a lot of members that will be reluctant to changes and will probably complain a lot but for sure we will respect our community since it's the heart of the project. That sounds like a great improvement! I suppose this tweaks exist in Invision Community 4.5.0 beta 9, right? We did not upgrade to latest yet, but will upgrade in the next few weeks. @Stuart Silvester is there a unit testing suite for the whole application? We were thinking to create some unit testing for the plugins we are developing and I was wondering if we can integrate in something that already exists or we should create our own system. I have to say, I am still thinking to adapt the code to a command line interface and run the forum migration from there. If feels much safer and more controllable than doing this through requests to a web service. And some existing unit tests would be really great so I can make sure the migration tool works as desired. @Paul E. did you and your team implement an automated unit tests suite or all the tests were manually done? Thank you all for the great feedback you provided! It really helps to see that others did conversion with millions of posts and they worked (although not perfect, but with a bit of sweat, it was done). All of this input really helps!

Thank you so much for this detailed feedback @Paul E.! I am pretty sure there are going to be some hard working days until we get the conversion right. I hope we won't miss any parts, but you never know on such complex migrations until it's too late. And then it's impossible to revert and very hard to migrate the missing data. Anyway, your feedback really helps! As for migration taking 4-5 days, that's a lot. All this time, the forum was down or could be partially used somehow? I also plan a profiling session to see what I can optimise in the migration scripts. Also we have to see how to plan to migrate the assets apart from database. Our plan is to start with a small sample of the old solution (1500 members / posts / etc.) and iterate the conversion process until we are happy with it. After that, we will make a clone of the live solution which we will convert (and optimise the process even further if possible), then get 50 members from the live forum to test the data integrity apart from our team. I hope that will be enough to ensure the data integrity. Any insights of issues with the migration process really help to not repeat them again 🙂

I wasn't planning to create an application, but wasn't sure what was the purpose of the two buttons in converter view. I guess I'll just ignore them 😄. So I already created a plugin that hooks to the software() method, that works well as I see and I think that is the cleanest approach. Thanks! I see. So I should trust ACP converter even for millions of posts as you say. One question though: do you have any benchmarks on how much time should I expect for conversion for 1 million posts?

Other than online doc that is available, are there any other resources for plugin and app developers? Can we get official workshops/training sessions? I think we are happy to pay for a session as well.

Can we get access to Invision's repo and make PRs for fixes/tidying? Eg: Wrong description from my point of view, because it is thrown when a dataset is empty as well.

I guess you are right @Paul E.. I'm a perfectionist and try to do this as clean as possible, but after all this will only happen once. The forum has a few million posts forum and I just want to make sure I am in control when this will be done. Also I am trying to do this using command line since ACP does not feel right. Any ideas if anyone did this before? I tried to scrap the developers forum but did not find any answers. Also @bfarber is there any way to contribute to the IPB code? I have a few ideas that may help everyone. Thanks!