Disk READ or disk WRITE? Which is better?

[SJ77]

Hi Just wondering,

Does anyone know if IPB benefits more from disk read speed or does it benefit more from disk write speed?

(assuming I had to pick one)

Thank you :)

[Rhett]

1 hour ago, SJ77 said:

Hi Just wondering,

Does anyone know if IPB benefits more from disk read speed or does it benefit more from disk write speed?

(assuming I had to pick one)

Thank you :)

It would depend on your site, let's step back and see why you are asking this though? there seems to be more to the story here that would affect the answer. 

What problem are you trying to solve? 

 

[Makoto]

Yeah, it's impossible to provide a general answer here.

What setting / performance tweak / server configuration are you considering that has to do with read/write speeds to begin with?

[SJ77]

Well I have to move from a server that has the database on a 2X SSD raid 1 configuration to a server that will have the database on a 6X HDD Raid 1 configuration.

Seems like 6X read speed but 1X writes.. I am a bit worried.

[Rhett]

12 minutes ago, SJ77 said:

Well I have to move from a server that has the database on a 2X SSD raid 1 configuration to a server that will have the database on a 6X HDD Raid 1 configuration.

Seems like 6X read speed but 1X writes.. I am a bit worried.

I wouldn’t move to anything but SSD, doing so would be like moving back in time 10 years! 

[SJ77]

Just now, Rhett said:

I wouldn’t move to anything but SSD, doing so would be like moving back in time 10 years! 

I don't have much of a choice. I am out of disk space. All the big space servers have hdd.
I already know ssd is ideal but will hdd get the job done? or is my site going to suck? I guess that is where I am at.

I do have quite a good bit of traffic. mostly lurkers. That's why if READS are important for IPS.. I might be fine. 6X read speed.

But if WRITES are where the magic is at... I am doomed. Bottleneck city. I do have a good amount of memory on server so maybe it can just load the whole database into memory and be fine.. I don't know

[Rhett]

What about noving your files to an S3 and staying in the sad server?

You can’t have it all without paying for it though. 

P.s I think you or your host is confused on 6x vs 2x and read/write speeds for what it’s worth. What you are saying doesn’t make sense. 

[SJ77]

I am the host! 6 disks in RAID 1 configuration get boost in reads. However, because it's mirroring same data to all drives writes have no benefit.

2X in this case simply was meant to mean, I have 2 SSD's in RAID 1 on old sever. 

S3 will cost enough to drive me out of business.

[Makoto]

A 6x1TB RAID 1 configuration?

That's a little odd. You're paying for 6 drives but only getting the capacity of one and way more redundancy than is necessary.

Really, it's not READ/WRITE speeds that are the biggest factor here. The reason you want to host your site on a SSD (particularly your database server), is because of the extremely low latency SSD's provide.

It doesn't really how much you boost your read speeds by arraying together multiple HDD's, your latency is still going to suck with spinning disks, and that is going to hurt performance in the end.

Why don't you do what I do for my server configurations and have the best of both worlds?

I run my OS, database server, and other frequently accessed static resources off a 2x512GB RAID 1 SSD configuration, and have a 3x2TB hardware RAID 5 configuration for storage (IP.Downloads and attachment uploads).

RAID 5 offers you increased read performance for serving large, downloadable files and some redundancy.

I serve over 40TB of bandwidth per month on the above configuration without much issue. Keeping in mind, that's with all the optimizations I've applied to the server to keep it performing so well under these loads, but it can give you an idea of what is possible.

[SJ77]

13 hours ago, Makoto said:

A 6x1TB RAID 1 configuration?

That's a little odd. You're paying for 6 drives but only getting the capacity of one and way more redundancy than is necessary.

Just to explain a bit more the old server had 2 ssd and 2 hdd. (4 disks total) the new server has 6 disks. I needed much more storage space. All 6 disks on new server are large hdd (got a great Black Friday deal)

During setup I was able to partition the 6 drives. For the “/“ partition, it only lets me do raid 1 and for the “/home” partition I was able to do raid 10

i will also need to serve 40TB of bandwidth mostly because I host video files using IP Downloads app. 

 

[ASTRAPI]

From my experience most of the forums use around 70% read and 30% write but as Rhett already mansion it depends from your forum and the way that you use it....

For me an Ssd disk is a must and you can get easily on dedicated server disks of 1Tb so i think it should be enough.

For two ssd's of 1Tb you will use Raid 1 and for three disks Raid 5 and for 4 disks Raid 10.

Also separated partitions in my opinion is not needed with Ssd's anymore....

I know that you got the server on black friday and you got a good price but i don't think that you can get maximum performance for your task.

[SJ77]

Thanks for the info ASTRAPI.. The issue is 1TB is not going to even be close to enough. I am adding ~80GB per month. and have already used up nearly 1TB. I wanted to have 12TB+ so that I can have the same setup for several years.

Most of the servers with that kind of space won't include SSD's.

My back up plan is to keep my old server (which has ssd) and also keep the new server (hdd).. I can host site on old server and use the new server for ONLY video files used by IP Downloads app.  Most of the space needs are for video files used by downloads app. I think in ACP I can host those on another server.  It might work.  I think it's an FTP connection to another server. ???

 

[Morgin]

10 hours ago, SJ77 said:

Thanks for the info ASTRAPI.. The issue is 1TB is not going to even be close to enough. I am adding ~80GB per month. and have already used up nearly 1TB. I wanted to have 12TB+ so that I can have the same setup for several years.

Most of the servers with that kind of space won't include SSD's.

My back up plan is to keep my old server (which has ssd) and also keep the new server (hdd).. I can host site on old server and use the new server for ONLY video files used by IP Downloads app.  Most of the space needs are for video files used by downloads app. I think in ACP I can host those on another server.  It might work.  I think it's an FTP connection to another server. ???

 

The model has largely moved from self hosting data to using object storage. It’s scalable and more cost efficient. You’d have to run your own estimates based on your traffic https://calculator.s3.amazonaws.com/index.html but I would be shocked if you can self host at a cheaper price point than using Amazon S3 or any one of a number of compatible object storage systems. 

[Makoto]

5 minutes ago, Morgin said:

The model has largely moved from self hosting data to using object storage. It’s scalable and more cost efficient. You’d have to run your own estimates based on your traffic https://calculator.s3.amazonaws.com/index.html but I would be shocked if you can self host at a cheaper price point than using Amazon S3 or any one of a number of compatible object storage systems. 

Unfortunately S3 storage is still way more expensive than most self-hosted solutions you can get. S3 does have advantages like unrivaled redundancy and potentially better performance if you're willing to pay for it, but consider that 2TB of HDD storage on S3 alone runs $90/mo on S3, and that doesn't even account for bandwidth costs.

S3 works as a solution for some people, notably enterprises that can spend the money and where rapid scalability is a major priority, but it's not an affordable or practical solution for everyone.

[Morgin]

35 minutes ago, Makoto said:

Unfortunately S3 storage is still way more expensive than most self-hosted solutions you can get. S3 does have advantages like unrivaled redundancy and potentially better performance if you're willing to pay for it, but consider that 2TB of HDD storage on S3 alone runs $90/mo on S3, and that doesn't even account for bandwidth costs.

S3 works as a solution for some people, notably enterprises that can spend the money and where rapid scalability is a major priority, but it's not an affordable or practical solution for everyone.

I calculated it at $43/month but I guess depends which server you are using. 

Digital Ocean Spaces (s3 compatible) is $40 per month w/ 1TB outbound transfer. 

Either way, if you are serving >2TB of data, you should be concerned about redundancy and performance. As I noted in another thread, you could use an open source s3 compatible service like minio, but again on a dollar for dollar basis I’d be surprised if self hosting ended up being much cheaper. Maybe I’m just not familiar with reliable hosts that have gobs of storage and cheap bandwidth 

[Makoto]

12 minutes ago, Morgin said:

I calculated it at $43/month but I guess depends which server you are using. 

Digital Ocean Spaces (s3 compatible) is $40 per month w/ 1TB outbound transfer. 

Either way, if you are serving >2TB of data, you should be concerned about redundancy and performance. As I noted in another thread, you could use an open source s3 compatible service like minio, but again on a dollar for dollar basis I’d be surprised if self hosting ended up being much cheaper. Maybe I’m just not familiar with reliable hosts that have gobs of storage and cheap bandwidth 

Yes, it just depends on your needs and your specific scenario. What works for you will not necessarily work for everyone else. Amazon S3 is a useful service, but it is not for everyone.

As mentioned above, I run a server serving over 40TB/month on a custom built raid configuration. I also maintain off-site backups myself, and still have a reasonable amount of on-site redundancy.

I'm on a 3-year lease with my datacenter right now, and the server configuration I mentioned above (with 3x2TB HDD's and 2x480GB SSD's), runs me $190/month. That's including the cost of the server itself and with 50TB of monthly bandwidth allowance on a dedicated 1Gbps line.

Again, I'm not saying self-hosting is better than Amazon S3, just saying it's not a magical, universally superior in every situation and always cheaper solution for everyone. It's not.

Everyone has unique needs, and sometimes self-hosting is the better solution for a select number of people with very specific requirements and a limited budget to work with.

[Morgin]

16 minutes ago, Makoto said:

Yes, it just depends on your needs and your specific scenario. What works for you will not necessarily work for everyone else. Amazon S3 is a useful service, but it is not for everyone.

As mentioned above, I run a server serving over 40TB/month on a custom built raid configuration. I also maintain off-site backups myself, and still have a reasonable amount of on-site redundancy.

I'm on a 3-year lease with my datacenter right now, and the server configuration I mentioned above (with 3x2TB HDD's and 2x480GB SSD's), runs me $190/month. That's including the cost of the server itself and with 50TB of monthly bandwidth allowance on a dedicated 1Gbps line.

Again, I'm not saying self-hosting is better than Amazon S3, just saying it's not a magical, universally superior in every situation and always cheaper solution for everyone. It's not.

Everyone has unique needs, and sometimes self-hosting is the better solution for a select number of people with very specific requirements and a limited budget to work with.

100%. 

If you fit within the parameters of most object storage services it’s great value. I’d wager most clients of IPS who need some storage are going to fall into that bucket  

Your data requirements and transfer is pretty significant though so yeah, lots of benefits for you to self-manage and host. 

Going by @SJ77‘s other posts re storage, I think third party object storage should be considered for their use case. 

[Makoto]

Yeah, I agree it's a good solution for a majority of clients.

The main concern I have with it in @SJ77's scenario is bandwidth. iirc, Amazon's bandwidth pricing is $0.155/GB, which for 40TB of monthly bandwidth usage, would total $6,200/month. Naturally, that is way more bandwidth than 99.9% of clients consume, so it's not a major concern for most people.

The storage pricing is reasonable. It's the bandwidth pricing that can get you.

But if that pricing is off for what you're referring to, please correct me!

[Martin A.]

https://wasabi.com/pricing/
100% Amazon S3 compatible. Doesn't charge for bandwidth usage. There are also providers out there that don't charge for it if it goes through CloudFlare. One of them is BackBlaze. They are not an S3 service though, but you can use it as one by using an extra integration layer such as minio.io.

Currently in the process of moving one of my clients over to this service.

[Makoto]

5 minutes ago, Martin A. said:

https://wasabi.com/pricing/
100% Amazon S3 compatible. Doesn't charge for bandwidth usage. There are also providers out there that don't charge for it if it goes through CloudFlare. One of them is BackBlaze. They are not an S3 service though, but using an extra integration layer such as minio.io

Currently in the process of moving one of my clients over to this service.

That looks incredibly interesting, though their FAQ is a little concerning:

Quote

Wasabi’s hot cloud storage service is designed for use cases where you store your data in our public cloud storage service and you access this data at a reasonable rate.      If your use case creates an unreasonable burden on our infrastructure, we reserve the right to limit your egress traffic and/or ask you to switch to our Legacy pricing plan.  Wasabi’s hot cloud storage service is not designed to be used to serve up (for example) web pages at a rate where the downloaded data far exceeds the stored data or any other use case where a small amount of data is served up a large amount of times.    Please note that we often have excess egress bandwidth available for our customers and while we don’t expect to have to limit a customer’s egress traffic, if this condition occurs we will inform you of the condition via a service notification.

Seems like it might be a bit too good to be true for users pushing more than 10x their storage in bandwidth per month, but they don't clarify a clear limit here.

Blackblaze's pricing is the closest to reasonable that I've seen (though still ~10x the price I pay for bandwidth from my datacenter).

[Martin A.]

That basically mean that you can't host 500GB of data and have more than that in downloads per month. 

I was not aware of that limitation, but I don't believe this will affect us as the majority of our usage if attachments and images, which will be served through CloudFlare.

So for the kind of sites you and OP runs, Wasabi may not be a service you can use.

[Makoto]

It does look like a nice service otherwise though, thanks for sharing!