Jump to content

How much bw and transfer/month does your community demand?

ørret

Recommended Posts

ørret Enthusiast

ørret

Posted
Posted

Hello! First of all, I just wonder, is IPS official forum linked up to a 1 GBit-Port or a 100 MBit-port? (or something else?), and second, how much transfer pr month is this forum requiring? I'm not running a big board but I am just very interested, also I'm asking because I see I can buy a box at 132 mbit port (5tb transfer included) and upgrade to 1gbit port (10 tb transfer included) if I need to. Further on I would like to know about other "big" sites and what they require.

It's obvious that a site running IP.Downloads/IP.Gallery like this will have more transfer pr month, but I would like to know how much (if it's not a company secret then ^^) it requires running this site including these apps and its downloads, and also how much you would "think" such site would require without IP.Downloads/IP.Gallery.

Take your own board as example (if it's "big"/very active). Thanks in advance.

Link to comment
Share on other sites
Gaffney Collaborator

Gaffney

Posted
Posted

not really. You see surinp3's board is not optimized.



1- The board's cachable files (images, css, js) are missing cache control headers.


2- JS and CSS are not compressed.



I have 80-100 users online and I only use about 11GB per month.



I use roughly 20GB a month for my forums and 20GB for wordpress. But depending on the activity, some months forums can take up to 50GB.
Link to comment
Share on other sites
Bono Explorer

Bono

Posted
Posted

Hello! First of all, I just wonder, is IPS official forum linked up to a 1 GBit-Port or a 100 MBit-port? (or something else?), and second, how much transfer pr month is this forum requiring? I'm not running a big board but I am just very interested, also I'm asking because I see I can buy a box at 132 mbit port (5tb transfer included) and upgrade to 1gbit port (10 tb transfer included) if I need to. Further on I would like to know about other "big" sites and what they require.



It's obvious that a site running IP.Downloads/IP.Gallery like this will have more transfer pr month, but I would like to know how much (if it's not a company secret then ^^) it requires running this site including these apps and its downloads, and also how much you would "think" such site would require without IP.Downloads/IP.Gallery.



Take your own board as example (if it's "big"/very active). Thanks in advance.




For forum you don't need more than 100mbit unless you are hosting big files. I got on average 2000 unique and around 30-35k pageviews and my forum uses 35GB of traffic per month. So traffic is not an issue here, for most sites 100GB would be more than enough for everything.

Jump from 10mbit to 100mbit makes big difference and jump from 100mbit to 1gbit will not make any difference for forum. It is just waste of money and nothing more.
Link to comment
Share on other sites
Bono Explorer

Bono

Posted
Posted

Interesting indeed. Speed and loading time is mostly the issue? I think I heard someone said that cpu didn't "matter" that much but memory is vital. Btw what's your site?




For me none of this variables is the issue, maybe only latency to my server in USA because I live in EU. I'm just saying 1Gbit is not cost effective, on my server I got site Joomla site that has millions of visits and my network peak was maxed out at 25mbit while using around 800GB per month of traffic.
Cpu and memory matters but it depends how much load have you put on your box, if you have only one site on your dedicated box and you are using 10-20% of resources that is not important.

I have forum in local language but I guess you can check how fast or slow is for you.

http://forum.hondaclub.hr

My server average speed per month is around 3.50Mbit/s so for 99% 100mbit is more than enough.
Link to comment
Share on other sites
  • 2 weeks later...
cthree Explorer

cthree

Posted
Posted

I don't know exactly how many GB a month I use. I know I consume between 20-30 Mbps constantly with ~10M page views/month.

I guess that works out to somewhere around 250GB/day or about 7-8TB/month.

100K members, 18M posts, 4M PMs and 450K photos.

Link to comment
Share on other sites
Bono Explorer

Bono

Posted
Posted

cthree it would be interesting to know with what specs you are pushing this amount of traffic, size of db and how many qps your database is pushing.

You are right there is no point in staffing server with too much ram, ram requirements depends how big is database, but it is always good to have more ram so you can cache most things in database, when it hits hdd it becomes slow even if they are SSD.

These are my stats but i use single SAS drive for database:

top - 22:18:47 up 314 days, 19:42, 1 user, load average: 1.50, 1.23, 1.02
Tasks: 199 total, 2 running, 197 sleeping, 0 stopped, 0 zombie
Cpu(s): 6.2%us, 6.2%sy, 5.2%ni, 81.1%id, 1.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 4151104k total, 3856404k used, 294700k free, 161056k buffers
Swap: 2096440k total, 823784k used, 1272656k free, 1945272k cached

Avg. iowait is 1.48 which is very good.

Link to comment
Share on other sites
Bono Explorer

Bono

Posted
Posted

@orret: [url="http://www.s2ki.com/forums"]http://www.s2ki.com/forums[/url]





1.48% is excellent. It means disk IO isn't a problem for you and you'll gain little from trying to improve it.



I suggest you nice -15 your mysqld process. That will de-prioritize most of the other crap running on the machine and give highest priority to mysqld.



in my.cnf:



[mysqld_safe]


nice = -15



I'm not sure what stats you refer to. My mysqld does ~400 commands/s on average. My database is about 45GB in size. It's a Dell 1950 with one 4 core cpu, 16GB of ram, PERC5/i controller w/2x500GB 7.2K RPM SATA hard drives (RAID1) for the OS and an 80GB Fusion ioDrive card. Software is CENTOS 5 with Percona Server mysql RPMs.



[url="http://www.percona.com/"]http://www.percona.com/[/url]


[url="http://www.fusionio.com/products/iodrive/"]http://www.fusionio....oducts/iodrive/[/url]



I posted my board stats in an earlier reply.



I see you are also running Honda forum specialized in S2k. :)

My nice is already set to "5" and I just wondered how much QPS is pushing board big as yours. I got on avg. 220 QPS which is also pretty decent. Since my db is only 200MB big I don't fast IO and single SAS drive is more than enough.

What are your thoughts about Percona did you see some major improvements over mysql? In my upgrade road map I got Percona but for now Mysql is not giving me any problems. And I also noticed that you are first who first had Apache then went to lighttpd and then moved back to apache. Very strange, usually people who move away from apache never come back.
Link to comment
Share on other sites
  • 2 weeks later...
  • 2 weeks later...
cthree Explorer

cthree

Posted
Posted

do You have any experience with regular mysql servers?


Do You recommend percona?




Percona Server is regular MySQL. It just has a bunch of InnoDB patches applied. It is significantly faster and more efficient than the plain MySQL distribution when using InnoDB. The Percona guys have documented the differences on their web site.


I see you are also running Honda forum specialized in S2k. :)



My nice is already set to "5" and I just wondered how much QPS is pushing board big as yours. I got on avg. 220 QPS which is also pretty decent. Since my db is only 200MB big I don't fast IO and single SAS drive is more than enough.



What are your thoughts about Percona did you see some major improvements over mysql? In my upgrade road map I got Percona but for now Mysql is not giving me any problems. And I also noticed that you are first who first had Apache then went to lighttpd and then moved back to apache. Very strange, usually people who move away from apache never come back.




Times change. Back then, hardware wasn't anywhere near as powerful as it is now and it was no where near as cheap. I used lighttpd to support proxy http and fastCGI requests with load balancing to 4 backend application servers. Those 5 machines were replaced by one Dell 2950. The load balanced configuration was too brittle and too high maintenance to stay with considering how cheap and powerful server hardware became 3-4 years ago. The release of mod_rails/passenger for Apache 2.2 also meant I didn't have to support CGI/fastCGI for my ruby and rails apps and could run them alongside my PHP apps.

Have you considered adding a second SAS drive as a RAID1 mirror of the first? I would at a minimum. Save you a ton of grief if that drive should fail.

Percona has a free InnoDB hot backup utility which allows point in time backup of your innodb tables without locking and without taking the server offline. That's a bonus. It uses binary logs to track the changes to the tables while the backup is in progress. It makes nightly backups a breeze. I run a small MySQL slave VM on one of my other machines to maintain a continuous near-line backup of the database. It's also handy if you want to make a quick copy. Stop the slave, flush the logs, shutdown MySQL, copy the database, restart MySQL and let the slave catch up again. Very useful.

QPS averages ~290 qps but that's not all IPB. There are a number of apps and sites that share the server.


top - 06:39:21 up 3 days, 23:08,  1 user,  load average: 0.18, 0.13, 0.09

Tasks: 119 total,   1 running, 118 sleeping,   0 stopped,   0 zombie

Cpu(s):  2.0%us,  0.7%sy,  0.0%ni, 96.7%id,  0.4%wa,  0.0%hi,  0.1%si,  0.0%st

Mem:  16432000k total, 15718252k used,   713748k free,   135008k buffers

Swap: 18481144k total,   155288k used, 18325856k free,  1478576k cached


  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                           

 3646 mysql      0 -15 13.6g  13g 5172 S  3.9 83.7 522:23.46 mysqld



[quote name='

Link to comment
Share on other sites
Gary. Explorer

Gary.

Posted
Posted

top - 06:39:21 up 3 days, 23:08, 1 user, load average: 0.18, 0.13, 0.09


Tasks: 119 total, 1 running, 118 sleeping, 0 stopped, 0 zombie


Cpu(s): 2.0%us, 0.7%sy, 0.0%ni, 96.7%id, 0.4%wa, 0.0%hi, 0.1%si, 0.0%st


Mem: 16432000k total, 15718252k used, 713748k free, 135008k buffers


Swap: 18481144k total, 155288k used, 18325856k free, 1478576k cached



PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND


3646 mysql 0 -15 13.6g 13g 5172 S 3.9 [color="#FF0000"]83.7[/color] 522:23.46 mysqld



some memory ur mysql is using :o
Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
Invision Community © 2024 IPS, Inc.
×
×
  • Create New...