Document based storage

[DiskusjonNO]

Hello,
Performance is everything! So I think for larger sites it should be an option to use document based storage like CouchDB, mongodb and such to store data.
Maybe a converter from SQL to a document based server would be nice?


Kind regards

[Mat Barrie]

For larger sites, you can either cluster, or convert to MSSQL. Contrary to popular belief, these "NoSQL" type platforms do not have any real benefit over a traditional RDBMS in virtually all situations. Also, for a relational-data driven application like IP.Board, something that doesn't, well, store relational data would highly likely be a performance degradation.

[bfarber]

We've found on many servers that experience performance problems that the bottlenecks end up relating to disk i/o. Processors and memory is so fast these days, while disk access is much much slower (in computer terms). I don't think storing everything as flatfiles and having to open/read/close and open/write/close these files repeatedly would be good at all.

[Ikadon]

For larger sites, you can either cluster, or convert to MSSQL. Contrary to popular belief, these "NoSQL" type platforms do not have any real benefit over a traditional RDBMS in virtually all situations. Also, for a relational-data driven application like IP.Board, something that doesn't, well, store relational data would highly likely be a performance degradation.

Where are you taking this information from and how comes that big companies, not known to spend money on nonsens, invest billions into this new kind of horizontally scalable databases?
The major difference is a different approach (e.g. no "Joins") and that there is no SQL-API, which leads to both, many pros and some cons. Generally speaking a NoSQL-database is always faster than a RDBMS, that alone comes from not following ACID ;)

(I'm not speaking for document stores, but more for NoSQL-support in general)

[bfarber]

1) We distribute an application that any user can purchase, download, and install on shared hosting. To that end, we support the most common server setups our customers will be using - namely PHP5 + MySQL. I don't think we will be investing into a database solution that 5 people MIGHT find a use for.

We have supported Oracle and to a lesser extent, PostgreSQL (through some in-house development and through our community developers program) and neither of those was widely popular, and I would argue that either of those engines are more popular than any no-sql solution by a long shot.

2) MySQL is generally going to be much much faster at storing and retrieving data than a flat-file based database solution. I would disagree with you wholly when you suggest that NoSQL would always be faster than a RDBMS.

[Ikadon]

Yup, I thought so, still I'd love the possibility to use a NoSQL-database. Maybe I'll play around with IP.Products when I have more time and try wrapping some MySQL-parts *g*

I would disagree with you wholly when you suggest that NoSQL would always be faster than a RDBMS.

You certainly can, but in this case you would only need to dive into this topic and read scientific-articles and would be disabused. We are talking about the same level of skill here, not a RDBMS-expert versus a NoSQL-nooby, right? Well, then NoSQL wins in any case (unstructured and even structured data), just read scientific papers.

Also, I was not talking about document based storages alone, but about NoSQL in general. I was about to open a discussion in off-topic but then I found this thread and posted in here, not paying attention to the fact it was placed in feedback-section :(

[bfarber]

FWIW you are free to create any database driver you like. The database drivers are extendable.

[Ikadon]

FWIW you are free to create any database driver you like. The database drivers are extendable.

Thank you.

As I'm currently working on a term paper about NoSQL, its possibilities and a comparison between NoSQL and RDBMS I'll maybe give it a try to have something more appealing for the audience :)

[bfarber]

I would be interested in your findings if you do.

All you do is change the $INFO['sql_driver'] variable in conf_global.php, then create classDb(Driver).php in ips_kernel.

[Ikadon]

Yup, can't guarantee anything though, many other things to do besides this paper *sighs* and apart from that I've never worked with anything else than MySQL on a "higher software-level".
The thing is that these scientific papers are really interesting and I would love to share them, but I'm not allowed as it's only free for students of universities that pay :(

Before I dived into this topic I didn't think that NoSQL-databases would outrun RDBMS with structured data as well, but they do oO. Well, it depends on the software, but they definetely have potential, besides Web 2.0 too I guess (games and other software)

[Klaus Mouse]

MongoDB is actually far supieror to MySQL. I was a MySQL man for about 8 years before switching over to MongoDB and found that when working on relations - MongoDB does a very very nice job.

1. Mongoid is the gem that is used for MongoDB while developing in a Ruby on Rails environment.
2. Mongoid has relational class methods that allow you to reference (currently only in mysql) or embed. Embedding would be something similar to "Susie's" member object while you're embedding class instances of "session" into her "document." This way if Susie accesses the site 10,000,000,000 times then her performance on the site may be slower than Johny who is normal and only have say 2,504 sessions embedded into his User document.
3. Other relationship items can be:

has_one
has_many
has_and_belongs_to_many
embedded_in
embeds_many
embeds_one

4. MongoID (noSQL ORM) has other features such as:

Automatic Timestamps (create, update)
Paranoia (hide on delete instead of destroy on delete)
Chaching
Version Control (only works with flat document classes - no relational versioning)

So, like MySQL - MongoDB a NoSQL engine has its advantages and it all depends on how you want to use them. A piece of software like Invision Power Board would be a pefect fit for a NoSQL engine where Posts are documents and threads are collections of embedded posts. The advantage to this would be in MySQL you have a "posts" table that can reach enormous sizes whereas in MongoDB the size of the "thread" document is as large as their are posts thus you have less "internal searching around" to do.

One last note - MongoDB has one major advantage that MySQL could never possibly have. Simultaneous writes. In MySQL you're allowed one write instance where the table gets locked during that write and it must complete or fail to unlock. In MongoDB by default you have one write however with Sharding you can increase the number of simulatenous writes to your database. Example:

1 MySQL cluster with 10 MySQL servers = 1 write
1 MongoDB "cluster" wth 10 MongoDB shards = 10 writes

The switch from MySQL to MongoDB would increase your database interactivity capacity by 10 times. For high demend applications (maybe not a forum) this could be an awesome feature that is worth the lack of standard SQL syntax.

Just my two cents.

Cheers,

Mike Trimm