Gabriel Torres Posted January 12, 2014 Share Posted January 12, 2014 Hi, As per suggestions made here, I decided to move from Apache to nginx. Performance is about the same, so I imagine a few tweaks might be necessary. Information about our hardware specs, traffic, and forum stats are already posted here: http://community.invisionpower.com/topic/396624-ipb-aggressive-caching/ Please evaluate as below and let me know what I could be adjusting. Thanks. My current config: nginx.conf: worker_processes 1; worker_connections 2048; keepalive_timeout 5; Top: top - 15:11:45 up 312 days, 17:34, 1 user, load average: 2,19, 1,91, 2,34 Tasks: 186 total, 3 running, 183 sleeping, 0 stopped, 0 zombie %Cpu(s): 12,2 us, 0,2 sy, 0,0 ni, 87,5 id, 0,0 wa, 0,0 hi, 0,1 si, 0,0 st KiB Mem: 32970636 total, 26539488 used, 6431148 free, 1045108 buffers KiB Swap: 29630460 total, 7768 used, 29622692 free, 24106580 cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16039 www-data 20 0 365m 41m 19m R 88 0,1 1:04.94 php-fpm 16011 www-data 20 0 367m 54m 30m R 76 0,2 1:13.39 php-fpm 16038 www-data 20 0 367m 51m 28m S 52 0,2 0:56.72 php-fpm 16060 www-data 20 0 360m 38m 19m S 46 0,1 0:05.14 php-fpm 16040 www-data 20 0 367m 53m 30m S 37 0,2 0:32.52 php-fpm 15988 nobody 20 0 28708 2136 832 S 2 0,0 0:02.93 nginx Link to comment Share on other sites More sharing options...
GreenLinks Posted January 12, 2014 Share Posted January 12, 2014 First question , did you read Nginx articles,examples etc.. ? I honestly doubt so because otherwise you'd not configure nginx like that. I would not post this if you were not a big board. Link to comment Share on other sites More sharing options...
Gabriel Torres Posted January 12, 2014 Author Share Posted January 12, 2014 Hi GreenLinks, I am new to nginx, and that is why I am asking for help. If you could post some links to good articles on configurartion/optimization I'd appreciate, because there are many "tutorials" out there made by people running low-traffic websites, and it is hard for me with no experience with nginx to separate the good articles from the bad ones. Thanks, Gabriel. Link to comment Share on other sites More sharing options...
Dmacleo Posted January 12, 2014 Share Posted January 12, 2014 from the reading I have done general rule of thumb is worker_processes is number of processors you have. the only controversy I have seen over it is with hyperthreading, some say include that while others say only count each core as 1. eg quad core with HT called 4 and not 8. really don't know which is better. Link to comment Share on other sites More sharing options...
Makoto Posted January 12, 2014 Share Posted January 12, 2014 I'll copy and paste an old guide I had published on here. It still needs to be updated (for example, DotDeb is not needed on Debain Wheezy to install PHP-FPM, though you obviously already have it installed and running), but maybe it will be of some use to you.Setting up a secure IP.Board installation with Nginx This is a guide for those interested in setting up an IP.Board installation using Nginx as their primary web server. This guide will mostly just cover the basics. In the future, I plan to write extensions to this guide for more advanced configuration techniques, such as global SSL/HTTPS support, Google SPDY and converting to MariaDB + XtraDB.What will be covered in this guide:Installing the latest stable release of Nginx on Debian or CentOSBasic Nginx configurationSetting up rewrite rules for IP.Board through NginxInstalling and configuring PHP-FPMSetting up proper, secure permissions for your IP.Board powered website If you have to ask this, you might not need or want to. Apache is the easier, more popular route people tend to take. Nginx is a small but very powerful and highly efficient web server. Apache with mod_php can be inefficient and needlessly waste system resources. Nginx can offer a potential gain in performance with the added benefit of not hogging your servers memory pool. Nginx is also simply a personal preference of mine, even though I'm not resource constrained on my box. Why should I use Nginx over Apache?Nginx Installation instructionsDebian file,Add the following repository to your /etc/apt/sources.list # Nginx Official Debian Repository deb http://nginx.org/packages/debian/ squeeze Nginx deb-src http://nginx.org/packages/debian/ squeeze Nginx Now import the PGP key, wget http://nginx.org/keys/Nginx_signing.key apt-key add Nginx_signing.key rm Nginx_signing.key Update and install, apt-get update apt-get install Nginx Nginx should automatically start after installation. If you have Apache installed and running however, this will fail. So please ensure you stop (and optionally uninstall) Apache before proceeding further. If you don't uninstall Apache, just know the two will be fighting for ports and IP's to bind every time you restart the server. You can confirm Nginx is running by using ./etc/init.d/Nginx status CentOS ,Add the following repository to /etc/yum.repos.d/Nginx.repo [Nginx] name=Nginx repo baseurl=http://nginx.org/packages/centos/$releasever/$basearch/ gpgcheck=1 enabled=1 Now import the PGP key, wget http://nginx.org/keys/Nginx_signing.key rpm --import Nginx_signing.key rm Nginx_signing.key Install and start, yum install Nginx /etc/init.d/Nginx start If you have Apache installed and running, Nginx will fail to start, so please ensure you stop Apache before proceeding further. You can confirm Nginx is running by using . You can also have Nginx start automatically on boot by running ./etc/init.d/Nginx statuschkconfig --level 345 Nginx on PHP-FPM Installation Instructions Debian file,Add the following repository to your /etc/apt/sources.list # DotDeb Debian Repository deb http://packages.dotdeb.org squeeze all deb-src http://packages.dotdeb.org squeeze all Now import the GPG key, wget http://www.dotdeb.org/dotdeb.gpg apt-key add dotdeb.gpg rm dotdeb.gpg Update and install, apt-get update apt-get install php5 php5-apc php5-cli php5-common php5-curl php5-fpm php5-gd php5-imagick php5-imap php5-mysql CentOS Nothing complicated here. Just install, yum install php-cli php-curl php-fpm php-gd php-imap php-mysql php-xml php-pecl-apcYou can also have PHP-FPM start automatically on boot by running .chkconfig --level 345 php-fpm on Nginx Configuration Now that you have both Nginx and PHP-FPM up and running, we can move on to configuring the web server. First, let's make some small adjustments to /etc/nginx/nginx.conf, user nginx; worker_processes 4; error_log /var/log/nginx/error.log error; pid /var/run/Nginx.pid; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; server_tokens off; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; #tcp_nopush on; keepalive_timeout 30; #gzip on; include conf.d/*.conf; } There's not much to say here. Increase worker_processes from the default of 1 to however many processor cores your server has. For example, if your server has a single quad core processor, set this value to 4. I've also changed the error_log directive to only log errors. Now let's move on to configuring your IP.Board website. Use this as your base template for /etc/nginx/conf.d/ipboard.conf, server { listen 80; server_name yourdomain.com www.yourdomain.com; root /srv/http/yourdomain.com/root; # Basic web server configuration. index index.php #access_log off; client_max_body_size 1G; # GZIP static content not processed by IPB. gzip on; gzip_static on; gzip_http_version 1.1; gzip_vary on; gzip_comp_level 6; gzip_proxied any; gzip_types text/plain text/css application/json application/x-javascript application/xml application/xml+rss text/javascript application/javascript text/x-js; gzip_buffers 16 8k; gzip_disable "MSIE [1-6].(?!.*SV1)"; # Set up rewrite rules. location / { try_files $uri $uri/ /index.php; } # Stub and FPM Status location /server_status { stub_status on; allow 127.0.0.1; deny all; } location /fpm_status { allow 127.0.0.1; deny all; fastcgi_pass unix:/var/run/php-fpm/ipboard.sock; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include /etc/nginx/fastcgi_params; } # Deny access to hidden files location ~ /. { deny all; } # Mask fake admin directory location ~^/admin/(.*)$ { deny all; } # Secure real admin directory location ~^(/nimda/).*(.php) { #allow 127.0.0.1; #deny all; #auth_basic "Restricted Area"; #auth_basic_user_file $document_root/nimda/.htpasswd; fastcgi_pass unix:/var/run/php-fpm/ipboard.sock; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include /etc/nginx/fastcgi_params; } # IP.Board PHP/CGI Protection location ~^(/uploads/).*(.php)$ { deny all; } location ~^(/hooks/).*(.php)$ { deny all; } location ~^(/cache/).*(.php)$ { deny all; } location ~^(/screenshots/).*(.php)$ { deny all; } location ~^(/downloads/).*(.php)$ { deny all; } location ~^(/blog/).*(.php)$ { deny all; } location ~^(/public/style_).*(.php)$ { deny all; } # Caching directives for static files. location ~^(/uploads/profile/).*.(jpg|jpeg|gif|png)$ { access_log off; expires 1d; } location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|xml|htm|txt|swf|cur)$ { access_log off; expires 1w; } # Pass PHP scripts to php-fpm location ~ .php$ { fastcgi_pass unix:/var/run/php-fpm/ipboard.sock; fastcgi_index index.php; fastcgi_buffers 16 8k; fastcgi_buffer_size 16k; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include /etc/nginx/fastcgi_params; } } There are a lot of things to cover here. First things first, replace yourdomain.com www.yourdomain.com in the server_name directive with your forums domain name. Include both the www and non-www version of the domain as above, regardless of which you actually use. You'll also need to replace yourdomain.com in the root and error_log directives. Next, let's cover where we're going to be installing the forum to. You'll notice I'm not using a /home/someuser heirachy as you're likely used to. There are two main reasons for this. Neatness and security. If you have your own server, you should keep your personal user account seperated from the actual web server. This is simply good practice in general. /srv/ on the Linux filesystem is for site-specific data which is served by the system. On all of my servers, I use a /srv/http/{host}/{subhost} hierarchy. "root" referrences the domains root. If you ever host another service on your forum (such as a wiki at wiki.yourdomain.com), you could store it in /srv/http/yourdomain.com/wiki Let's go ahead and set this up for your IP.Board website now. Remember to replace yourdomain.com! mkdir -p /srv/http/yourdomain.com/root useradd --system ipboard groupadd --system http gpasswd -a nginx http chown -R ipboard:http /srv/http/yourdomain.com If you want to disable access logging on your site to reduce disk load, just uncomment the access_log directive under the basic web server configuration section. Either way, we'll be setting Nginx up to not waste log entries for static resources further down. Now adjust client_max_body_size to the maximum allowed file size users can upload to your site. So if your highest allowed upload size for any of your members anywhere on your site is 750 MegaBytes, set this to 750M. Next, we get into configuring rewrite rules. There's nothing you really need to do here. All we have to do with Nginx is pass a try_files directive, which is better than relying on the rewrite method required with Apache. The Stub Status module allows you to see how many active connections your server has by visiting yourdomain.com/server_status. If you have a static IP address, you can set it up to ensure that only you will have access to this page, though leaving it public isn't really a vulnerability. Next, we want a secure installation, right? So that means we're not going to be using /admin anymore. Come up with something creative that you can easily remember to replace /admin. The example above uses "nimda" (which is just "admin" backwards), so replace that with whatever you come up with. Going a step further, there are two primary ways you can further secure your ACP. The most secure way would be to only allow connections from your IP. Though this will obviously not be feasible if you or another one of your administrators have an IP that constantly changes. If you do have a static IP that you can use, however, uncomment the "allow" and "deny" lines while adding an allow directive for your IP address bellow allow 127.0.0.1. The second option is to use .htpasswd authentication. IP.Board has a built in function to set this up for you in the security center. You can configure this after installing your forum. After creating the .htpasswd file in your ACP, all you will need to do is uncomment the auth_basic directives and restart Nginx. The caching directives are split for user avatars and everything else not processed by IP.Board. I have avatars set to expire after one day. For everything else it's one week. Feel free to adjust this to your own personal prefference.PHP-FPM Configuration On to PHP-FPM! First, let's go ahead and get rid of the default/example configuration we don't need. This is /etc/php5/fpm/pool.d/www.conf on Debian and /etc/php-fpm.d/www.conf on CentOS. Now, create a new file replacing www.conf with ipboard.conf and using this as the base template: [ipboard] ; Set the prefix directory and the user/group to run under prefix = /var/run/php-fpm user = php-fpm group = http ; Configure listen(2) directives listen = ipboard.sock listen.backlog = 4096 listen.owner = php-fpm listen.group = http listen.mode = 0660 ; Set up the process manager pm = static pm.max_children = 10 pm.max_requests = 250 pm.status_path = /fpm_status ; The timeout for serving a single request. Prevents runaway scripts. request_terminate_timeout = 5m ; Only execute .php scripts. chdir = /srv/http/yourdomain.com/root security.limit_extensions = .php ; Environment variables. ;env[HOSTNAME] = $HOSTNAME ;env[TMP] = /tmp ;env[TMPDIR] = /tmp ;env[TEMP] = /tmp env[DOCUMENT_ROOT] = /srv/http/yourdomain.com/root ; PHP flags and security directives for just this site php_flag[display_errors] = off php_admin_value[open_basedir] = /srv/http/yourdomain.com/root:/tmp:/usr/bin php_admin_value[disable_functions] = escapeshellarg,escapeshellcmd,exec,ini_alter,parse_ini_file,passthru,pcntl_exec,popen,proc_close,proc_get_status,proc_nice,proc_open,proc_terminate,show_source,shell_exec,symlink php_admin_value[upload_max_filesize] = 1G php_admin_value[post_max_size] = 1G First, replace all instances of "yourdomain.com" accordingly again. Next, let's go ahead and set up everything we need for PHP-FPM to run properly, mkdir /var/run/php-fpm useradd --system php-fpm chown php-fpm:root /var/run/php-fpm You can skip creating the directory on CentOS, as it will have already been created, but you will still need to apply the new permissions. Finding the optimal setting for pm.max_children is a bit of trial and error. I have it set to 10 as a default. You may need to increase or lower this setting depending on how constrained your server is and how much traffic your site receives. IP.Board tends to use an average of about ~52MB per process for me. So if you have 2GB of memory to spare on your server after accounting for MySQL and any other services you run, you could probably safely support up to 35 static processes, though you may not need nearly this many. Setting this too high may result in your server running out of memory, so be careful. At this point, let's go ahead and test to make sure everything works. (Use /etc/init.d/php5-fpm restart for Debian) /etc/init.d/nginx restart /etc/init.d/php-fpm restart No problems? Good! Problems? Double check your work and check your error logs. Let's create /srv/http/yourdomain.com/root/test.php as a test file with a simple phpinfo() call, <?php phpinfo(); ?> Now visit yourdomain.com/test.php. You should see a PHPINFO page containing the open_basedir and other directives we set. If you don't, you did something wrong. Go back and review. If all is well, delete the test.php file. (Really, delete it. Security audit tools scan for files like this, and you don't want to needlessly expose your server information to malicious parties.)Setting Up IP.Board If you're setting up a new IP.Board website, you'll need to install mysql-server, set up a new database and so on. There are plenty of resources available for learning how to do this if you don't know how already. After you copy your entire forum directory into /srv/http/yourdomain.com/root, we just need to set up proper permissions, chown -R ipboard:http /srv/http/yourdomain.com httproot=/srv/http/yourdomain.com/root find $httproot -type d -exec chmod 0750 {} ; find $httproot -type f -exec chmod 0640 {} ; find $httproot/{uploads,cache,downloads,hooks,screenshots,blog,public/style_*} -type d -exec chmod 0770 {} ; find $httproot/{uploads,cache,downloads,hooks,screenshots,blog,public/style_*} -type f -exec chmod 0660 {} ; If you haven't already, rename your /admin directory to whatever you chose earlier. Afterwards, create the following constants.php file in your forums root, replacing "nimda" appropriately, <?php define( 'CP_DIRECTORY', 'nimda' ); ?> Update the permissions, chmod 0640 constants.php chown ipboard:http constants.php And that's it! IP.Board should be up and running. If you're having trouble, review all of the above steps and check your error log (/var/log/nginx/error.log). Link to comment Share on other sites More sharing options...
Makoto Posted January 12, 2014 Share Posted January 12, 2014 from the reading I have done general rule of thumb is worker_processes is number of processors you have. the only controversy I have seen over it is with hyperthreading, some say include that while others say only count each core as 1. eg quad core with HT called 4 and not 8. really don't know which is better. It can be recommended to up this value if your load is more disk than CPU bound (eg. a file server). Also, from what I can see Gabriel, your forum is plenty fast on my end. Pages are fully loaded and rednered within 0.5-1.2 seconds. If it seems slow-ish for you, it may just be because you're in a different continent, like I said in your other thread :tongue: There's little to nothing you can do about that, other than find a good DC in Brazil. I wouldn't have any recommendations for you, and it may in fact be true that they all suck. But cross continental latency is something that's going to hit you if you host in the U.S. and live in Brazil. In these regards, I may consider recommending you look into technologies such as Google's SPDY and even Pagespeed to help optimize and reduce some latency troubles, but this won't offer you any guaranteed results and involves a decent bit of work and effort to configure properly. Edit: I also have AdBlock enabled like another user. It takes around 2-3 seconds for the page to fully render with it disabled. It's going to be even slower for you in Brazil because of the latency issues, but your website itself doesn't seem to have load time issues. Link to comment Share on other sites More sharing options...
Grumpy Posted January 13, 2014 Share Posted January 13, 2014 from the reading I have done general rule of thumb is worker_processes is number of processors you have. the only controversy I have seen over it is with hyperthreading, some say include that while others say only count each core as 1. eg quad core with HT called 4 and not 8. really don't know which is better. I suppose it depends on the writer's feelings of HT. HT does make things more efficient at multitasking, but it didn't add any true power to the system. But depending largely on HT as if it's a real core tends to cause problems when optimizing and yield more erratic behavior. Like, it will say the load is "okay" and then suddenly jump to ridiculous numbers after just a small increase whereas a system with HT turned off won't have such behavior as often. But at the end of the day for this specific case, it won't matter. Because... OP isn't going to make an nginx only server. So, even trying to maximize cores for nginx is pointless. Nginx cpu/mem consumption is so low (compared to php-fpm), it's not going to be a notable issue, even if set too high. Even with assignment of 1/2 of cores avl, OP is not going to run short of nginx workers. Link to comment Share on other sites More sharing options...
Gabriel Torres Posted January 14, 2014 Author Share Posted January 14, 2014 First of all, thanks for the very complete guide on nginx. As for the latency, I will post my recent finding on the other topic about this issue, so we stay in topic here... ;) Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.