Birched

Hi all,
I am at the early stages of migrating/converting a big old forum. To give an idea of numbers, if I can get our archives back in nicely, it currently sits at around 15k forums, 150k topics, 8 mill posts. We currently use many of our forums a lot like clubs are used here, so I'm planning to shift a bunch of those forums to clubs (I'm guessing 2-3k clubs, with avg 4-6 forums inside each one.)
During the conversion process, I've already run into a couple of snags that make it clear that most IPS sites are not dealing with this volume (for example, assigning group permissions to forums appears to be designed for a couple of dozen forums.... Also, IPS support says that they consider more than 100 forums a lot.) So far, I've been able to troubleshoot and push through (albeit slowly), but I'm wondering: are any of you running boards of this size, in particular with this many forums and/or clubs, and if so, have you got any pointers? I know that for vBulletin I ended up going as far as profiling the code and making some point changes for efficiency; I am hoping that isn't necessary here, especially if some kind and industrious soul has already done some optimisation.
Thank you for any ideas or suggestions!

Now now, let's not hosting-shame.
This is something that could have tripped any of us up, regardless of whether the site were running on a repurposed Sega Genesis in the garage or had a full dedicated datacenter, as it involved multiple moving parts across multiple providers, and the SSL cert would not have been visible from the browser.
Being methodological in troubleshooting, not allowing confirmation biases to impede our attempts to identify the root source, and generally keeping our disordered personality traits in check go a long way to getting things resolved as quickly and calmly as possible.

I'm not sure that the number of topics and posts is your issue. The challenge you'll face is with the number of forums and clubs. I'm not sure that interface things like selecting which forum to move a thread into works well with 15,000 entries in it.
As you encounter hundreds of entries in various configurations, you'll find that this hasn't been priority focus in development, especially with third-party additions. As you deviate from the "norm," it's likely effort will be required.
Could tagging be an alternative to any of this?

That shouldn't be the case, we'll have a look into this 🙂 thank you for letting us know.

Welcome to IPS!
Some notes for you:  
1.  For site migrations, there are several good developers who specialize in migrations either from other software or from older versions.  You can check out the list of Providers here:
https://invisioncommunity.com/third-party/providers/
(Or you can migrate yourself)
2. Private sections in posts - I haven't heard of anything either.  Sounds interesting though.   You can post in Customization Request:
https://invisioncommunity.com/forums/forum/506-customization-requests/
3.  Private topics - I haven't heard of anything either.  Not sure of the mods posted by @Miss_B will do what you really want.  
If you can group the users by membergroups, then this can be done through the core membergroup permissions.  If you can group the users manually, then you might be able to use Clubs (but you would need to create a club for every group).  
4.  You may want to talk to @Morrigan, who can probably provide some guidance on RPG-related communities.  

I'm trying to spend some time every day lurking in these forums and playing on a test site to learn and get myself calibrated to what will be required for the migration I am planning. It looks like a big task and I am pretty sure I will have to hire some help for some of it, but for now: a couple of questions.
I've been reading today about how bbcode is on its way out here (and custom bbcode is already not supported.) Although that's awesome news for mobile users, this strikes fear into my bones! We have lots of custom (and sometimes somewhat complicated) bbcode. I'm not going to go into some long list here; instead, I want to ask about two particular additions to our site that I have not been able to find. There is a reasonable chance that it's because I didn't know where to look, or was looking for them with the wrong words!
1. Private sections in posts: We have a bbcode 'secret' that allows anyone to specify a list of usernames who will be able to read the contents of the tags (moderators can read them all.) E.g. [secret="someusername"]Only someusername and moderators can see this.[/secret] Does this functionality exist in a plugin (or should I be calling it an "app"?) perhaps to add a button to the post editor? I was unable to find it in the marketplace, but I also found it was difficult to search for button extensions to post editing there, perhaps I was missing some terminology.
2. Private threads: We have a functionality at thread creation where any thread creator in certain subforums can specify a list of users who can (or, alternatively, can't) read the thread. Again, moderators are excluded. Does this functionality exist in the IPB core or somewhere in the marketplace? I feel like this one is more likely to already exist, but I failed to find it.
Thank you!

For your first request, I have not seen such a plugin/app myself. You can request one to be made for you at the request forum.
https://invisioncommunity.com/forums/forum/506-customization-requests/
For your second request, it does not exist in the core, but I have seen a couple of apps that doe that. One by @Adriano Faria and one by @DawPi.
 
 
 

The only way to search archives would be through google.
So like site: https://mysite.com my search terms
They will not be able to be searched in the IPS search function anymore. They are removed from the search index.

Yes. Indexed and still available, but it's fine if it takes longer for archived material, or for a search that includes the archives.
Basically, we are sitting on almost 20 years of material that people still go back to at times. One of the attractive things about IPB for me is that there is built-in logic to serve our most recent content faster, while still maintaining access to our older ("archived") content.

Yes, archiving content (via the archiving function in the ACP) removes them from the built-in site search. If the content is reachable by web spiders (i.e. viewable to guests), you'll still have that content indexed in public search engines, like Google.
We don't use clubs this way, but we did convert with blogs that were so numerous that IPS was unprepared to deal with it from the ACP. I'd certainly recommend testing your conversion and seeing what happens when the content is moved over from your existing community.

You may want to connect with @Morrigan, who runs a similar community.