Invision Community 4: SEO, prepare for v5 and dormant account notifications By Matt Monday at 02:04 PM
Tripp★ Posted September 26, 2014 Posted September 26, 2014 I'm trying to speed up my site, and yet keep it somewhat secure. The site is 100% SSL/HTTPS with a legit certificate... I've tried using Nginx, Pagespeed, Caches (such as Varnish, APC) a few years ago and go absoultely no where with them (As they slowed down my site, not sped it up). So over the last months I've been trying other methods, optimising as many images (With kraken.io) as possible and managed to reduce my home page from 8.99MB to a sweet 1-2MB, and I enabled HTTP-Keep Alive which has seen some positive results. I even started trying to minifying ips.js (And other .js files), and it's so far working but I have no idea if that's just a cache somewhere or if I'm lucky and other people with other browsers or something are going to run into problems. In public/js/ is it actually safe to minify ips.js? I used a number of Minify sites in the past and tried this - but it always messed up the site. Since then I've put in caching (to visitors hard drives) and the like in. Using www.jsmini.com/ I was able to get the .js file from ~130KB to ~78.9KB which is around a 39.31% reduction. My hopes are that I can minify all js files in the /public/js/ folder this way as any other attempt to minify the script (Much like the HTML) from within the ACP or (I believe I tried htaccess to minify before) has failed - or didn't work at all - or produced negative consequences, just like Async'ing all JS. 39% might not sound like a lot, but my theory is if I could reduce it I could save scrape up some wasted time. At the moment my site's response time is ghastly: Whilst I've got things like Pagespeed installed (And appearing in HTTP Response Headers) that's not exactly working and I don't understand it at all. So my options are limited (especially with all this new stuff that I have no idea what any of it means). I've always been in shared hosting, I thought it was as easy as uploading a site and getting to work - until recently (moved to VPS) where you need to do a lot more to get it running smoothly, and the learning curve for someone who has been Windows for all his life, learning Linux and SSH has been... Mountainous. Things are very different on the internet today to when I first started using forums and websites back in 2003. Oh and the thread on how to setup Pagespeed (which is written from Nginx point of view, tried using that before... That back fired too), bares absolutely no resemblance to my pagespeed.conf what-so-ever. So I pretty much gave up on that, as I said it's installed but that's it. Whilst I will give extra information to those interested in helping, I cannot exactly give my URL out publicly as it is an adult website and I've always taken extra measures around this site and this community to keep my site and any links to my site "Quiet". Lastly because my site is non-profit and I'm monetarily challenged (Poor), I don't have enough money per month to shell out on a 128GB RAM dedicated server and I certainly don't have the member base to justify it either. What's really kicking me up the USB Port, is the fact that my site's TTFB is shockingly appauling, and I have tried everything to get it lower. My VPS Specs are: OpenVZ (Cent OS 6.2, or higher, I installed it with 6.2 on it, but I am unsure as to how to find the exact version) Intel® Xeon® CPU E3-1230 V2 @ limited to: 825.000 MHz (4 Cores) 1GB RAM and 1GB VSwap With: Apache 2.4.10 PHP 5.5.17 MySQL 5.6.17 EasyApache Rebuilt with: Apache 2.4 Deflate Expires Fileprotect Headers MPM Worker Mod FastCGI v2.3.9 Mod SuPHP 0.7.2 Proxy UniqueId Version PHP 5.5 Bcmath CGI Calendar CurlSSL FTP GD Iconv Imap MailHeaders Mbregex Mbstring MySQL "Improved" extension. Mysql Mysql of the system Openssl Pear Phar SQLite3 Silence Deprecated Patch Sockets TTF (FreeType) Zip Zlib Additional OptMods Mod Security 2.8.0 Xcache 3.1.0 for PHP (Yeah, never got this working either) mod_pagespeed I use a heavily modified (Hooks and Apps) IPB3.4.6 with Gallery, IP.Content and an Arrowchat Chatbar, there is a separate area of the site on the same server running IPS4. I'd appreciate any constructive advise, guides (I think it's best if I started from the foundations, honestly), and any protips, and stuff from the community. I've never really asked for help like this before; so I'm like a deer in headlights. Thanks in advance. :)
RevengeFNF Posted September 26, 2014 Posted September 26, 2014 Do you have Zend Opcache enabled? It comes with php 5.5.
Tripp★ Posted September 26, 2014 Author Posted September 26, 2014 Just checked - No. Is there an easy setting in WHM that I can flick "Enabled" to do it, or does this require rooting around in PuTTY?
RevengeFNF Posted September 26, 2014 Posted September 26, 2014 You need to enter with ssh on your server, and enable it in php.ini
Tripp★ Posted September 26, 2014 Author Posted September 26, 2014 Alright perhaps tomorrow - on on Monday ("Work hours" - Gives time for others to reply too) I'll definitely look into this, and I'll let you know on my next post, RevengeFNF.
Pete T Posted September 26, 2014 Posted September 26, 2014 Alright perhaps tomorrow - on on Monday ("Work hours" - Gives time for others to reply too) I'll definitely look into this, and I'll let you know on my next post, RevengeFNF. you can install Zend in WHM don't need ssh ! I don't have WHM any more but there script in one of the options allows you build Zend any many other extras from WHM.
Tripp★ Posted September 26, 2014 Author Posted September 26, 2014 I will look into that first then, before considering SSH involvement - It's a load of hoops and hurdles to get into, so thanks for the protip there, Pete.
Pete T Posted September 26, 2014 Posted September 26, 2014 I will look into that first then, before considering SSH involvement - It's a load of hoops and hurdles to get into, so thanks for the protip there, Pete. its called Easy Apache should be software area believe.
Grumpy Posted September 26, 2014 Posted September 26, 2014 You can configure php.ini file in full depth from whm. Don't remember the steps off head... Though, personally I find just doing it via ssh faster (since there's no drill-down like in whm)... I believe zend opcache is not supported by cpanel yet, so, be aware if you ever happen to go to cpanel for support. You'll need to disable your xcache entirely or just xcache's php opcache portion if you want to enable zend opcache. They will break each other. You should also keep in mind that most in-depth optimization guides are written for stand-alone systems without a control panel. As such, just following other's guides won't work unless specifically for whm/cpanel environment or specific ipb settings that is not system setup dependent. If you had failed attempts at something that you gave up, you need remove it entirely as there are many things that conflict with one another. --------------- You mentioned that you tried many different minify of js. Has the IPB's built in minifier somehow failed you? It's your best option. Lowering filesize isn't what's important, it's clumping all your js into one. And IPB's built in minifier will do that. Minifing one js at a time (which it sounds like) doesn't make a significant difference. For example, at a response rate of 1s (like you mention), loading 10 js with 1KB will take 10 seconds. Loading 1 js with 100KB will still take 1 second despite being 10x larger in file size total. So primary importance of minifying is not with reduction of file size, but reduction of file count. --------------- You will need to post your output from "top". You can see sticky for more info. It'll give us a glance at how your system is running and why it's slow in general. Be sure to post all of what you see, not just a portion of it. --------------- You may want to elaborate as to why solutions such as APC, nginx, etc. didn't work for you. Or what you tried in more depth. A badly configured/installed solution of anything will only result in worse performance as you are adding more things to the mix. It is important to do it right rather than toss a bunch of solutions and hope that they work.
ASTRAPI Posted September 27, 2014 Posted September 27, 2014 If you are on a vps (how much you pay as you may get a dedicated from 15-20 euro) is time to use Nginx php 5.5 or 5.6 FPM and MariaDB... with opcache and Cloudflare in front With a nice optimization it will fly
RevengeFNF Posted September 27, 2014 Posted September 27, 2014 For me the best is with Nginx in the front end and Apache in the back end. In his case, because he only have 1Gb of Ram, a Nginx only solution might be better.
Tripp★ Posted September 27, 2014 Author Posted September 27, 2014 its called Easy Apache should be software area believe. Yes sir, I am very familiar with WHM and Easy Apache. Well, very familiar when compared to Linux and SSH. You can configure php.ini file in full depth from whm. Don't remember the steps off head... Though, personally I find just doing it via ssh faster (since there's no drill-down like in whm)... I believe zend opcache is not supported by cpanel yet, so, be aware if you ever happen to go to cpanel for support. You'll need to disable your xcache entirely or just xcache's php opcache portion if you want to enable zend opcache. They will break each other. You should also keep in mind that most in-depth optimization guides are written for stand-alone systems without a control panel. As such, just following other's guides won't work unless specifically for whm/cpanel environment or specific ipb settings that is not system setup dependent. If you had failed attempts at something that you gave up, you need remove it entirely as there are many things that conflict with one another. Wow, thanks for the in detail post, Grumpy; I will break it down and respond to each bit. I will check this out on Google to be sure, but I suspect your right, kinda like running two Anti-Viruses, I can't see it working out too well. So if what you suspect is correct I will rebuild Apache without XCache which should uninstall it completely. I agree with what you're saying towards the Guides I'd treat them like Guides - not instruction manuals, but consider someone who's never used Linux, or SSH got dropped on an unmanaged VPS server - when I first got it I thought I was way over my head, then I started learning, I didn't give up. Now I think I'm way, way, way, way, WAY over my head... But I'm not giving up. I want to learn. I feel you could be right however about the failed attempts that I gave up. Some of the stuff was very hard to install, and I had to do things, and other things just to make it install and thus they're not so easily removed, it's possible I may still have some kind of remnant to Nginx on my server and I don't know about it or something. This has happened before actually - on a test server (not my live one, of course) I played around with it, and because I had not kept track of what I had done, and ultimately I needed to reinstall my test VPS but that was a couple of years back. You mentioned that you tried many different minify of js. Has the IPB's built in minifier somehow failed you? It's your best option. Lowering filesize isn't what's important, it's clumping all your js into one. And IPB's built in minifier will do that. Minifing one js at a time (which it sounds like) doesn't make a significant difference. For example, at a response rate of 1s (like you mention), loading 10 js with 1KB will take 10 seconds. Loading 1 js with 100KB will still take 1 second despite being 10x larger in file size total. So primary importance of minifying is not with reduction of file size, but reduction of file count. Yes indeed, I have tried different minify's, and yes IPB's built in Minifier is to me a failure - it minifies (at least for me) absolutely nothing. Back when I was on Cloudflare I had to disable it in favour of their version because it worked marginally better than IPB's built in one. I realise I might be saying treachery and heresy right now, but that was my experence of it. If I cannot combine all the .js files I am left with one option. To minify them. When minified my logic pointed out; that they would take a lot less time to download (per file), thus the loading of the site can be increased. Now of course, if there was a way or if IPB's version does this, and makes the files that need to load - less at the same time that would be awesome, however it doesn't. At least not for me. You will need to post your output from "top". You can see sticky for more info. It'll give us a glance at how your system is running and why it's slow in general. Be sure to post all of what you see, not just a portion of it. Right. Fair enough. top - 15:45:20 up 21:37, 0 users, load average: 0.12, 0.17, 0.08 Tasks: 44 total, 1 running, 43 sleeping, 0 stopped, 0 zombie Cpu(s): 0.7%us, 0.1%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1048576k total, 1001464k used, 47112k free, 0k buffers Swap: 1048576k total, 502572k used, 546004k free, 471156k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 27738 SITE 20 0 290m 27m 8428 S 2.7 2.7 0:05.04 php 26046 nobody 20 0 2296m 41m 12m S 0.3 4.1 0:01.90 httpd 1 root 20 0 19236 972 844 S 0.0 0.1 0:00.05 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd/169 3 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khelper/169 140 root 16 -4 10644 256 252 S 0.0 0.0 0:00.00 udevd 560 root 20 0 243m 1176 808 S 0.0 0.1 0:00.46 rsyslogd 609 root 20 0 66620 344 340 S 0.0 0.0 0:00.00 sshd 625 root 20 0 105m 768 764 S 0.0 0.1 0:00.00 mysqld_safe 784 mysql 20 0 2791m 227m 6408 S 0.0 22.2 3:23.32 mysqld 819 root 20 0 19612 960 820 S 0.0 0.1 0:00.10 dovecot 829 mailnull 20 0 70108 844 708 S 0.0 0.1 0:00.01 exim 832 dovenull 20 0 42356 2376 2372 S 0.0 0.2 0:00.00 pop3-login 833 dovenull 20 0 42464 2676 2448 S 0.0 0.3 0:00.13 imap-login 834 dovecot 20 0 13316 1028 936 S 0.0 0.1 0:00.02 anvil 835 root 20 0 13448 1108 972 S 0.0 0.1 0:00.05 log 837 dovenull 20 0 42356 2376 2372 S 0.0 0.2 0:00.00 pop3-login 839 dovenull 20 0 42360 2504 2448 S 0.0 0.2 0:00.00 imap-login 841 root 20 0 16432 1696 1108 S 0.0 0.2 0:00.10 config 842 dovecot 20 0 29452 1412 1240 S 0.0 0.1 0:00.19 auth 852 root 20 0 139m 24m 22m S 0.0 2.4 0:03.36 httpd 858 root 20 0 135m 1368 1296 S 0.0 0.1 0:00.07 pure-ftpd 860 root 20 0 133m 1348 1300 S 0.0 0.1 0:00.05 pure-authd 867 root 20 0 114m 724 624 S 0.0 0.1 0:00.52 crond 877 root 20 0 21544 384 368 S 0.0 0.0 0:00.00 atd 930 root 20 0 34940 1740 1384 S 0.0 0.2 0:00.11 cPhulkd - proce 1086 root 20 0 111m 3460 1004 S 0.0 0.3 0:01.70 cpsrvd-ssl 1096 root 20 0 103m 780 504 S 0.0 0.1 0:00.06 cpdavd - accept 1107 root 20 0 38848 2424 1448 S 0.0 0.2 0:01.91 queueprocd - wa 1155 root 20 0 83708 4708 1572 S 0.0 0.4 0:10.08 tailwatchd 1206 root 38 18 31024 1260 764 S 0.0 0.1 0:00.13 cpanellogd - sl 10473 root 20 0 148m 5568 1380 S 0.0 0.5 0:13.86 lfd - sleeping 26044 root 20 0 79720 9.9m 2900 S 0.0 1.0 0:00.05 leechprotect 26045 nobody 20 0 91540 2896 532 S 0.0 0.3 0:00.04 httpd 26047 nobody 20 0 2296m 41m 13m S 0.0 4.1 0:01.91 httpd 26048 nobody 20 0 2298m 40m 13m S 0.0 4.0 0:01.88 httpd 26049 nobody 20 0 2298m 39m 13m S 0.0 3.8 0:01.74 httpd 26050 nobody 20 0 2370m 40m 14m S 0.0 3.9 0:02.08 httpd 27553 SITE 20 0 289m 27m 8276 S 0.0 2.7 0:15.71 php 27795 SITE 20 0 288m 26m 8152 S 0.0 2.6 0:04.15 php 27907 cpanelro 20 0 111m 10m 2444 S 0.0 1.0 0:00.04 cpsrvd-ssl 28013 root 20 0 27504 576 376 S 0.0 0.1 0:00.00 vzctl 28014 root 20 0 105m 1844 1500 S 0.0 0.2 0:00.00 bash 28039 root 20 0 14900 1240 992 R 0.0 0.1 0:00.02 top That was the complete listing at the time of copying. My site's cPanel username has been altered to SITE for security and anonymity purposes. You may want to elaborate as to why solutions such as APC, nginx, etc. didn't work for you. Or what you tried in more depth. A badly configured/installed solution of anything will only result in worse performance as you are adding more things to the mix. It is important to do it right rather than toss a bunch of solutions and hope that they work. With APC I can't remember, all I do remember is that it was never caching, even when I set up IP.Board to use it, this was 2011/2012 so I could have ran into a bug or something that has since been patched, all I remember is it didn't work and I didn't try since because of the hoops I had to jump through to get it to work. It was on my test VPS account which is long gone now. But I did "toss a bunch of solutions and hoped that they work"ed. Today I realise that was naive (I realised that after it died in 2011/2012) hence why I've tried to be-careful this time. I certainly do not think it will kill me to make a nice big backup of my community and then wipe the server clean and start fresh - ultimately this is most likely what's going to happen as I said on my current VPS I've tried Nginx already and various other things - getting Google's Pagespeed module installed was a pain in the butt, but I have finally gotten it installed and it's doing nothing. Then again I've not really played with any of the config for it, because as I stated in my first post I have no idea if it's actually doing anything or what it's really doing. I've even considered pooling as much money together and asking someone to help me because it's out of my depth - but I dunno. Wouldn't that be admitting defeat? I'm not ready to throw in the towel, just yet. If you are on a vps (how much you pay as you may get a dedicated from 15-20 euro) is time to use Nginx php 5.5 or 5.6 FPM and MariaDB... with opcache and Cloudflare in front With a nice optimization it will fly Right... Some problems there, for my VPS I pay around $20-$30 a month and I've never seen a dedicated server for 15-20 euros and I would be dubious of such claims. I'm not sure why so many people are swearing by Nginx, that is better suited (I believe) to static content such as images, not dynamic content such as PHP (Please correct me if I am wrong), as for Cloudflare... I've tried them before I've been with them for years infact, until recently. Look I pay a small amount for my SSL certificate, and if I want to continue using Cloudflare it is my understanding that I either have to pay an extra $20 a month to use a shared Cloudflare Certificate or $200 a month if an extra $20 a month is out of my price range an extra $200 a month is out of the question. I'm committed to sticking with SSL. I've never even heard of MariaDB and if that's anything like MySQL I'm still learning and coming to grips with that. But hey, at least I'm trying to understand and learn - and that's the point, right? For me the best is with Nginx in the front end and Apache in the back end. In his case, because he only have 1Gb of Ram, a Nginx only solution might be better. Again, I'm not so sure. How would I go about this Front-End Back-End stuff? I was under the understanding that you cannot run Apache and Nginx on the same server, it's like running two Anti-Viruses, they clash and make a mess. Nginx DID get me results, when my site wasn't SSL. My Response Time (from Pingdom) dropped to 600-900ms. However as soon as I made SSL enabled on the whole site this shot up to 1800-2100ms and after "uninstalling Nginx" (I hope) and using Apache with an MPM Module (Like my host recommended) it's gone back down to 1,100-1,300. But this is still unacceptable for me, and I'm losing engagement from my users because the "Site is Sluggish" problem is I have a strong connection and I've never noticed this "Sluggishness" but I can see that it's there, through Pingdom and other site testing tools. Another thing that has been suggested to me is switching to Xen Hosting rather than OpenVZ. It's a couple of dollars more expensive but I'm not "entirely" sure if this is worth it - to me it seems I have underlying issues I need to resolve first. Edit: My webhost as generously offered to update me from OpenVZ to Xen, 4GB RAM (and 4GB Swap, I believe), 50Mbit instead of 25Mbit and so on at a discounted price. So I will be starting on a fresh server - which means no more bits and pieces of half-arsed installed cloggage. A chance to start "Fresh".
RevengeFNF Posted September 27, 2014 Posted September 27, 2014 But you can I use VestaCP Control panel. Its free, very simple, and it will install Apache + Nginx on your vps without any problems. Like i said in the past here, why have only one when you can have the best of both? Another thing, i think you should look for another VPS. 20$ for a vps with only 1Gb of Ram, is too much. And thats your bottleneck. I Pay only 9,99€ for a VPS with 4Gb of Ram, witch is enough for me. 1 [####* 5.7%] Tasks: 63, 38 thr; 1 running 2 [##* 3.3%] Load average: 0.54 0.31 0.17 3 [# 1.0%] Uptime: 3 days, 04:31:51 Mem[|||||||||||||||||||||||||||||||||||********** 2077/4096MB] Swp[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||127/128MB]
Tripp★ Posted September 27, 2014 Author Posted September 27, 2014 But you can I use VestaCP Control panel. Its free, very simple, and it will install Apache + Nginx on your vps without any problems. Like i said in the past here, why have only one when you can have the best of both? Another thing, i think you should look for another VPS. 20$ for a vps with only 1Gb of Ram, is too much. And thats your bottleneck. I Pay only 9,99€ for a VPS with 4Gb of Ram, witch is enough for me. Okay, I will have to look into this because practically everywhere I've looked I've read that, that will not work. And leaving my webhost isn't going to happen. Unlike some of the ones I've been with these people have been nothing but nice, and I'm loyal to them. Their service has been without fault, their customer service is astonishingly amazing, and they're reliable- absolutely reliable. They treat me and my needs individually - I'm not another number to them. Besides they just got me a very good deal, and are looking at stuff they can do on the hardware side of it to further increase my speed output. Thanks but that's one idea I will not entertain. I will look into this Apache+Nginx thing however. Has anyone else tried this?
Dmacleo Posted September 27, 2014 Posted September 27, 2014 heh soon you will be looking into unmanaged dedicated and really dealing with some crap on your own. and you'll be glad you did I bet :)
Tripp★ Posted September 27, 2014 Author Posted September 27, 2014 heh soon you will be looking into unmanaged dedicated and really dealing with some crap on your own. and you'll be glad you did I bet I'm afraid I don't quite follow, I'm not sure if it's something hopeful to look forward to or some kind of sarcastic remark. All I really asked is what do you think about speed improvements, done on a software level like configs and such (stuff I need to consider, stuff I need to look out for, hear what other IPB customers say about the configs they've done (hopefully) and figure out this wonderful world of Server Optimisation). If you're referring to "Be glad you did" to my willingness to learn then I completely agree to that - absolutely.I just never expected someone to turn around and say I should move hosts - that's not very constructive, nor is it what I asked for feedback on. Please don't take that as me being ungrateful or rude. I mean it with respect and I am very grateful to the feedbacks I've been getting. Edit: My webhost as generously offered to update me from OpenVZ to Xen, 4GB RAM (and 4GB Swap, I believe), 50Mbit instead of 25Mbit and so on at a discounted price. So I will be starting on a fresh server - which means no more bits and pieces of half-arsed installed cloggage. A chance to start "Fresh". Oh and just to make clear - I haven't accepted or declined yet. I'm honestly amazed by all the support from everyone here and my hosts.
Grumpy Posted September 27, 2014 Posted September 27, 2014 Since you are saying you're getting a new server, not sure if it's worth bothering to find the issues on the existing solution. But your top is very interesting... Seems like your system is having a breeze handling your content, despite being slow. Though, it is possible that because it's a vps, it can't tell the difference between %idle/wa/si and thus misrepresenting the actual issue. (Common in vpses as they don't get full data collection) The swap usage is significant, but I feel like that's just residual swap usage and not active since there's plenty of space in cached. This is very preliminary, but I'd guess in the order of likelihood... MySQL | You may be creating a lot of temporary table on disk due to a poor configuration. Thus driving up disk usage unnecessarily and slowing down the site significantly. But this static won't show because it's a vps. You can post your mysql config and/or output of mysqltuner (see sticky). Insufficient apache/php child process and/or zombie issue. You can post apache config (just the base one. httpd.conf, probably won't need vhosts') as well as output of "ps aux | grep httpd" and "ps aux | grep php". Also, "apachectl fullstatus" is very helpful. See sticky on how to get that. Alternatively, you can get apache full status from WHM's apache status. It's long. If you can't get apache status, you can also run "netstat -anp | grep :80 | awk '{print $6 "t" $7}'" (note apostrophe inside quotes). I intentionally selected only column 6 & 7 since you seem like you want to hide ips. Residual xcache/apc configuration in your forums. This can cause unnecessary lookups by your forum to a non-existence source, causing time wasting. You can post your conf-global.php for it obviously with your username/password/dbname/host removed. ================== If you get a brand new system. I'd also throw this option for you. If you only have one site with this server, consider using no control panel at all. It's a bit like throwing you into the deep waters, but if you manage to survive long enough to not to drown, you will get to love it as you have the full depth of freedom to do everything, from optimizing to tweaking. Also, a great thing about community is that you can often find free labor. Not just from us, but from your forum. You may be able to find an enthusiast to assist you with your system management. Given they can be trusted... ================== Edit: Probably hard to google. Zombie is when a resource isn't dying when it should. If a process completes, it should clear old stuff and start anew to handle new results. But if it fails to do the cycle and is stuck at a state of not dying, it's called a zombie.
Dmacleo Posted September 27, 2014 Posted September 27, 2014 what I mean is if you are already on vps looking for speed and load handling it won't be long before you go dedicated and, even though its a steep learning curve, you'll be glad you did due to the benefits.
RevengeFNF Posted September 28, 2014 Posted September 28, 2014 what I mean is if you are already on vps looking for speed and load handling it won't be long before you go dedicated and, even though its a steep learning curve, you'll be glad you did due to the benefits. It depends on the revenue he can generate with the site. Because dedicated servers are a lot more expensive. He can check for a cheaper solution, that is a VPS with more Ram and a SSD Hard Drive. Its not a dedicated server, but it can serve a normal site.
Tripp★ Posted October 6, 2014 Author Posted October 6, 2014 Since you are saying you're getting a new server, not sure if it's worth bothering to find the issues on the existing solution. But your top is very interesting... Seems like your system is having a breeze handling your content, despite being slow. Though, it is possible that because it's a vps, it can't tell the difference between %idle/wa/si and thus misrepresenting the actual issue. (Common in vpses as they don't get full data collection) The swap usage is significant, but I feel like that's just residual swap usage and not active since there's plenty of space in cached. This is very preliminary, but I'd guess in the order of likelihood... MySQL | You may be creating a lot of temporary table on disk due to a poor configuration. Thus driving up disk usage unnecessarily and slowing down the site significantly. But this static won't show because it's a vps. You can post your mysql config and/or output of mysqltuner (see sticky). Insufficient apache/php child process and/or zombie issue. You can post apache config (just the base one. httpd.conf, probably won't need vhosts') as well as output of "ps aux | grep httpd" and "ps aux | grep php". Also, "apachectl fullstatus" is very helpful. See sticky on how to get that. Alternatively, you can get apache full status from WHM's apache status. It's long. If you can't get apache status, you can also run "netstat -anp | grep :80 | awk '{print $6 "t" $7}'" (note apostrophe inside quotes). I intentionally selected only column 6 & 7 since you seem like you want to hide ips. Residual xcache/apc configuration in your forums. This can cause unnecessary lookups by your forum to a non-existence source, causing time wasting. You can post your conf-global.php for it obviously with your username/password/dbname/host removed. ================== If you get a brand new system. I'd also throw this option for you. If you only have one site with this server, consider using no control panel at all. It's a bit like throwing you into the deep waters, but if you manage to survive long enough to not to drown, you will get to love it as you have the full depth of freedom to do everything, from optimizing to tweaking. Also, a great thing about community is that you can often find free labor. Not just from us, but from your forum. You may be able to find an enthusiast to assist you with your system management. Given they can be trusted... ================== Edit: Probably hard to google. Zombie is when a resource isn't dying when it should. If a process completes, it should clear old stuff and start anew to handle new results. But if it fails to do the cycle and is stuck at a state of not dying, it's called a zombie. Okay I will run through those today and see what I can get. I have ran MySQL tuner before (recently in fact) here was the results: root@server [/]# perl mysqltuner.pl >> MySQLTuner 1.3.0 - Major Hayden <major@mhtx.net> >> Bug reports, feature requests, and downloads at http://mysqltuner.com/ >> Run with '--help' for additional options and output filtering [OK] Currently running supported MySQL version 5.6.17 [OK] Operating on 64-bit architecture -------- Storage Engine Statistics ------------------------------------------- [--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MRG_MYISAM [--] Data in MyISAM tables: 547M (Tables: 2031) [--] Data in InnoDB tables: 13M (Tables: 178) [--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 52) [!!] Total fragmented tables: 62 -------- Security Recommendations ------------------------------------------- [OK] All database users have passwords assigned -------- Performance Metrics ------------------------------------------------- [--] Up for: 4d 0h 53m 27s (1M q [4.943 qps], 157K conn, TX: 102B, RX: 320M) [--] Reads / Writes: 87% / 13% [--] Total buffers: 169.0M global + 1.1M per thread (151 max threads) [OK] Maximum possible memory usage: 338.9M (33% of installed RAM) [OK] Slow queries: 0% (22/1M) [OK] Highest usage of available connections: 31% (47/151) [OK] Key buffer size / total MyISAM indexes: 8.0M/137.1M [OK] Key buffer hit rate: 99.9% (27M cached / 39K reads) [!!] Query cache is disabled [OK] Sorts requiring temporary tables: 0% (14 temp sorts / 35K sorts) [!!] Joins performed without indexes: 8111 [!!] Temporary tables created on disk: 33% (29K on disk / 89K total) [OK] Thread cache hit rate: 99% (48 created / 157K connections) [!!] Table cache hit rate: 3% (2K open / 63K opened) [OK] Open file limit used: 84% (3K/4K) [OK] Table locks acquired immediately: 99% (1M immediate / 1M locks) [OK] InnoDB buffer pool / data size: 128.0M/13.3M [OK] InnoDB log waits: 0 -------- Recommendations ----------------------------------------------------- General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance Adjust your join queries to always utilize indexes When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries without LIMIT clauses Increase table_open_cache gradually to avoid file descriptor limits Read this before increasing table_open_cache over 64: http://bit.ly/1mi7c4C Variables to adjust: query_cache_type (=1) join_buffer_size (> 256.0K, or always use indexes with joins) tmp_table_size (> 16M) max_heap_table_size (> 16M) table_open_cache (> 2000) As for APC that was a long time ago on another server (a test server) completely. This server was apache, I switched to Nginx, that worked (666ms or so) until I activated SSL then it went up sky high. Go back to Apache (and the current configuration) and I got down to the first post on this topic. By updating my VPS's resource allowances; I got it down further but there is clearly room for more improvement. It depends on the revenue he can generate with the site. Because dedicated servers are a lot more expensive. He can check for a cheaper solution, that is a VPS with more Ram and a SSD Hard Drive. Its not a dedicated server, but it can serve a normal site. The revenue? None- as in absolutely zero. Not one cent. Well... Advertising pays for me to put advertising up of my own. As I said we're still a very small site. ====== I upgraded the server, it's still OpenVZ (In the end, as I couldn't afford the ongoing expenditure of a XenVPS). I get 50Mb/sec (roughly) and 2GB RAM + 2GB "VSWAP"), I haven't had time to do much else other than changing my DNS from Cloudflare and setting my nameservers to by own domain. When I upgraded I noticed a speed increase, and again when I put my nameservers on my own domain other than ns.kate.cloudflare.com whatever (From when I tried Cloudflare) so right now; and no other settings have been changed I've got it down to: Which in my small world is a major achievement, getting it down from four digits to three. I will at some point be looking at removing XCache and putting on "Memcached" instead - but I am still a little vague on it. I think (one of many of) my Achilles heel here is that there is a configuration for these cache things or a Admin Control Panel which I can't seem to find or access, I remember there being one for one of them that I've used before - but it never cached anything (the cache was always empty). Anyway I will be looking into it today and I wanted to thank everyone for putting up with me and helping so far.
Grumpy Posted October 7, 2014 Posted October 7, 2014 in mysql config, try: query_cache_size = 64M join_buffer_size = 4M tmp_table_size = 32M max_heap_table_size = 32M add if not already present. change if already present in configs. /etc/my.cnf And restart mysql. Wait 24hrs, post tuner & top again. Also, try running ioping. https://code.google.com/p/ioping/ Run: ioping . Hit Ctrl + C if you think you got enough samples. Please don't run other ioping features unless you know what you're doing.
Tripp★ Posted October 9, 2014 Author Posted October 9, 2014 Alright, Grumpy. I did as you suggested and the MySQL Config file was exactly where you said it was (I was previously never able to find it), and here are the results, 1 day and 15 hours later. mysqltuner results: >> MySQLTuner 1.3.0 - Major Hayden >> Bug reports, feature requests, and downloads at >> Run with '--help' for additional options and output filtering [OK] Currently running supported MySQL version 5.6.17 [OK] Operating on 64-bit architecture -------- Storage Engine Statistics ------------------------------------------- [--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MRG_MYISAM [--] Data in MyISAM tables: 543M (Tables: 2031) [--] Data in InnoDB tables: 13M (Tables: 178) [--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 52) [!!] Total fragmented tables: 54 -------- Security Recommendations ------------------------------------------- [OK] All database users have passwords assigned -------- Performance Metrics ------------------------------------------------- [--] Up for: 1d 15h 25m 22s (907K q [6.394 qps], 90K conn, TX: 56B, RX: 168M) [--] Reads / Writes: 87% / 13% [--] Total buffers: 248.0M global + 4.9M per thread (151 max threads) [OK] Maximum possible memory usage: 984.1M (48% of installed RAM) [OK] Slow queries: 0% (1/907K) [OK] Highest usage of available connections: 4% (7/151) [OK] Key buffer size / total MyISAM indexes: 8.0M/135.3M [OK] Key buffer hit rate: 99.8% (14M cached / 24K reads) [!!] Query cache is disabled [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 21K sorts) [!!] Joins performed without indexes: 4532 [OK] Temporary tables created on disk: 25% (4K on disk / 15K total) [OK] Thread cache hit rate: 99% (7 created / 90K connections) [!!] Table cache hit rate: 6% (1K open / 30K opened) [!!] Open file limit used: 86% (3K/4K) [OK] Table locks acquired immediately: 99% (848K immediate / 848K locks) [OK] InnoDB buffer pool / data size: 128.0M/13.3M [OK] InnoDB log waits: 0 -------- Recommendations ----------------------------------------------------- General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance Adjust your join queries to always utilize indexes Increase table_open_cache gradually to avoid file descriptor limits Read this before increasing table_open_cache over 64: Variables to adjust: query_cache_type (=1) join_buffer_size (> 4.0M, or always use indexes with joins) table_open_cache (> 2000) open_files_limit (> 4161) root@server [/]# perl mysqltuner.plhttp://mysqltuner.com/http://bit.ly/1mi7c4C ioping results: 4096 bytes from . (simfs /vz/private/169): request=1 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=2 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=3 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=4 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=5 time=0.2 ms 4096 bytes from . (simfs /vz/private/169): request=6 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=7 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=8 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=9 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=10 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=11 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=12 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=13 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=14 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=15 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=16 time=0.8 ms 4096 bytes from . (simfs /vz/private/169): request=17 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=18 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=19 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=20 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=21 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=22 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=23 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=24 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=25 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=26 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=27 time=0.0 ms 4096 bytes from . (simfs /vz/private/169): request=28 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=29 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=30 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=31 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=32 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=33 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=34 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=35 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=36 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=37 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=38 time=60.7 ms 4096 bytes from . (simfs /vz/private/169): request=39 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=40 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=41 time=0.0 ms 4096 bytes from . (simfs /vz/private/169): request=42 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=43 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=44 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=45 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=46 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=47 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=48 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=49 time=11.5 ms 4096 bytes from . (simfs /vz/private/169): request=50 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=51 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=52 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=53 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=54 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=55 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=56 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=57 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=58 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=59 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=60 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=61 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=62 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=63 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=64 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=65 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=66 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=67 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=68 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=69 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=70 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=71 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=72 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=73 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=74 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=75 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=76 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=77 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=78 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=79 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=80 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=81 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=82 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=83 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=84 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=85 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=86 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=87 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=88 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=89 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=90 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=91 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=92 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=93 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=94 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=95 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=96 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=97 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=98 time=21.4 ms 4096 bytes from . (simfs /vz/private/169): request=99 time=0.1 ms 4096 bytes from . (simfs /vz/private/169): request=100 time=0.1 ms ^C --- . (simfs /vz/private/169) ioping statistics --- 100 requests completed in 99561.7 ms, 985 iops, 3.8 mb/s min/avg/max/mdev = 0.0/1.0/60.7/6.5 ms root@server [/ioping-0.6]# ioping .
Grumpy Posted October 9, 2014 Posted October 9, 2014 dang. Didn't see that you were on mysql 5.6... So, forgot to also tell you to set: query_cache_type = 1 Tidbit: from 5.6, query cache is disabled by default, so even setting query size alone doesn't enable query cache, must also enable by setting type (default of 1 in <=5.5 and default of 0 on >= 5.6). ---------------- open_files_limit If this is not mentioned anywhere in your mysql config, it's fine. If it's set to 0, it's fine. 0 / default is auto. If it is manually set to 4161, please set to 0 or 50000. ------------------------------ table_open_cache Mysql tuner will keep asking for more. but due to how IPB works, raising won't help. Just a side note. ------------------------ [!!] Joins performed without indexes: 4532 Did you install any addons / modules / etc to your site that might cause this? It basically means it's pulling data inefficiently. And to correct it, you must either stop requesting that, adjust the request so that it hits the indexes or add indexes on what is applicable. Its very existence isn't an issue, but if it becomes a sizeable chunk, it'll become a big issue. As far as I'm aware, default IPB yields near or exactly zero in this. Currently ~0.44% for you (4k / 907k).
Tripp★ Posted October 10, 2014 Author Posted October 10, 2014 Yeah I missed the "top" command too, whoops. Here's my current my.cnf file in it's entirety: [mysqld] query_cache_type = 1 innodb_file_per_table=1 default-storage-engine=MyISAM local-infile=0 open_files_limit=50000 max_allowed_packet=268435456 query_cache_size = 64M join_buffer_size = 4M tmp_table_size = 32M max_heap_table_size = 32M innodb_buffer_pool_size=134217728 In over 24 hours time, I will post the results including top again. Thanks Grumpy. :) As for what's causing joins without indexes; I don't know. Is there anyway I can find out? There was a mod that's on here, that was causing "Slow Queries", I uninstalled that though. (>Board Index Face mod, I believe). I'm not sure if it's related to the the joins without indexes though.
ASTRAPI Posted October 10, 2014 Posted October 10, 2014 You need a mysql optimization for sure and i don't know what settings do you use for Nginx and all the rest services...... Remotely by posting is not the easiest way to do it but anyway :) I was send you a PM :)
Recommended Posts
Archived
This topic is now archived and is closed to further replies.