large Invision DBs?

[LiquidFractal]

I have a business idea, which involves using a database which could conceivably grow to hundreds of thousands (and who knows - maybe millions down the line!) of entries (which would be only text and simple markup, e.g. bold, italics, etc).

Is there anyone out there who uses the native Invision DB for large amounts of data?  If so, do you / at what point have you experience(d) slower site operation and DB access?

I'm currently debating whether to embed my business project in my existing Invision framework/DB or to begin with an independent mySQL database/website and go from there.

Thanks in advance to anyone who can offer their experiences!

 

[Makoto]

Could you possibly elaborate on what you mean? It sounds like you're talking about a NoSQL database? Something like this unfortunately would not work with IPS.

But for what it's worth, I currently run a forum with around 1.8m posts and a database size of 5-6GB, and the average MySQL response time on my server is only 20-30ms.

This is running the latest stable release of MariaDB and tuned for performance, but still, MySQL only accounts for about 1/5th of the script execution time on my server, the rest of it is just PHP.

As long as you have enough memory to store your database in-memory performance is relatively good with MariaDB.

[LiquidFractal]

@Makoto Yeah, sorry - probably wasn't clear enough.  By "Invision DB" I meant creating a database using the database functionality afforded by the Pages app.  Embedding my project in the Invision platform would allow me to use all the sweet stuff for membership, subscriptions etc. that Invision has, but if the DB and # of entries gets to be really large and I experience lag or reduced service as a result (thinking waaaaay into the hypothetical future here!) I'd want to know I could port the database out of Invision and into, say, mySQL or a noSQL framework.

It would be a niche site so it wouldn't get loads of traffic at a time that I can imagine, but I'm trying to think open-ended and whether or not a Pages DB would be adequate in the long run (would also need some custom pages written as well).

[Makoto]

Ah, I see.

I believe you may be a bit confused, however, The Pages database itself does actually run off MySQL like the rest of IPS. It's not a custom database engine or anything of the sort; everything is still actually stored in MySQL.

Pages is already pretty performant when it comes to databases and things of the like, and you can even implement caching on pages to further reduce overhead, so I don't think it's something you'll need to be very concerned about.

Naturally, as your site grows and you get more traffic, you may inevitably need to look into a more powerful hosting solution, but this is a reality no matter what kind of platform you are using.

tl;dr Pages DB is fine, if it's a good solution for you, use it! You can worry about scaling as you get larger and actually need it.

[bfarber]

I know of a client using Pages that has around 7M records in a Pages database off the top of my head.

I think you'll be fine.

[LiquidFractal]

Ahh, so.  This makes this much clearer for me - thanks @Makoto and @bfarber.  I thought Pages DBs were some sort of Invision-specific DB framework which sat inside the Invision platform which sat on top of mySQL, so to speak.

So I would assume that it's theoretically possible to take a huge Pages DB and convert it to mySQL (or Oracle, etc. etc.) should the need arise (although it seems that perhaps it may not)?

[Joel R]

5 hours ago, liquidfractal said:

I thought Pages DBs were some sort of Invision-specific DB framework which sat inside the Invision platform which sat on top of mySQL, so to speak.

It's a content management system for articles, recipes, bios, tutorials, help guides, etc.  All of these are built with Pages:

• Guides & Support:  https://invisioncommunity.com/4guides
• News:  https://invisioncommunity.com/news/
• Release Notes:  https://invisioncommunity.com/release-notes/

 

[evandixon]

18 hours ago, liquidfractal said:

I thought Pages DBs were some sort of Invision-specific DB framework which sat inside the Invision platform which sat on top of mySQL, so to speak.

So I would assume that it's theoretically possible to take a huge Pages DB and convert it to mySQL (or Oracle, etc. etc.) should the need arise (although it seems that perhaps it may not)?

Each database gets its own mySQL table, so you could write your own program to read that data if you needed.

[LiquidFractal]

14 hours ago, evandixon said:

Each database gets its own mySQL table, so you could write your own program to read that data if you needed.

Cool!  That's more or less what I wanted to know, since it addresses the question of whether or not a Pages DB could be ported out into its own mySQL database if need be.

Thanks for this!  Now I just need to inquire about some custom coding...

[Makoto]

11 minutes ago, liquidfractal said:

Cool!  That's more or less what I wanted to know, since it addresses the question of whether or not a Pages DB could be ported out into its own mySQL database if need be.

Thanks for this!  Now I just need to inquire about some custom coding...

If you're in need of custom coding, I'm more than happy to offer my services to you 🙂

Please feel free to PM me for more details if you're interested!

[LiquidFractal]

3 minutes ago, Makoto said:

If you're in need of custom coding, I'm more than happy to offer my services to you 🙂

Please feel free to PM me for more details if you're interested!

Thanks!  I know you're one of the coders here so will keep you in mind, and may post in the Dev forums (or whichever forum is appropriate for inquiring about coding projects).  But I may well shoot you a PM beforehand. 🙂