Jump to content

nginx optimization

Recommended Posts


As per suggestions made here, I decided to move from Apache to nginx. Performance is about the same, so I imagine a few tweaks might be necessary.

Information about our hardware specs, traffic, and forum stats are already posted here: http://community.invisionpower.com/topic/396624-ipb-aggressive-caching/

Please evaluate as below and let me know what I could be adjusting. Thanks.

My current config:

worker_processes 1;
worker_connections 2048;
keepalive_timeout 5;
top - 15:11:45 up 312 days, 17:34,  1 user,  load average: 2,19, 1,91, 2,34
Tasks: 186 total,   3 running, 183 sleeping,   0 stopped,   0 zombie
%Cpu(s): 12,2 us,  0,2 sy,  0,0 ni, 87,5 id,  0,0 wa,  0,0 hi,  0,1 si,  0,0 st
KiB Mem:  32970636 total, 26539488 used,  6431148 free,  1045108 buffers
KiB Swap: 29630460 total,     7768 used, 29622692 free, 24106580 cached
16039 www-data  20   0  365m  41m  19m R    88  0,1   1:04.94 php-fpm
16011 www-data  20   0  367m  54m  30m R    76  0,2   1:13.39 php-fpm
16038 www-data  20   0  367m  51m  28m S    52  0,2   0:56.72 php-fpm
16060 www-data  20   0  360m  38m  19m S    46  0,1   0:05.14 php-fpm
16040 www-data  20   0  367m  53m  30m S    37  0,2   0:32.52 php-fpm
15988 nobody    20   0 28708 2136  832 S     2  0,0   0:02.93 nginx
Link to comment
Share on other sites

Hi GreenLinks,

I am new to nginx, and that is why I am asking for help. If you could post some links to good articles on configurartion/optimization I'd appreciate, because there are many "tutorials" out there made by people running low-traffic websites, and it is hard for me with no experience with nginx to separate the good articles from the bad ones.



Link to comment
Share on other sites

from the reading I have done general rule of thumb is worker_processes is number of processors you have. the only controversy I have seen over it is with hyperthreading, some say include that while others say only count each core as 1.

eg quad core with HT called 4 and not 8.

really don't know which is better.

Link to comment
Share on other sites

I'll copy and paste an old guide I had published on here. It still needs to be updated (for example, DotDeb is not needed on Debain Wheezy to install PHP-FPM, though you obviously already have it installed and running), but maybe it will be of some use to you.

Setting up a secure IP.Board installation with Nginx

This is a guide for those interested in setting up an IP.Board installation using Nginx as their primary web server. This guide will mostly just cover the basics. In the future, I plan to write extensions to this guide for more advanced configuration techniques, such as global SSL/HTTPS support, Google SPDY and converting to MariaDB + XtraDB.

What will be covered in this guide:Installing the latest stable release of Nginx on Debian or CentOSBasic Nginx configurationSetting up rewrite rules for IP.Board through NginxInstalling and configuring PHP-FPMSetting up proper, secure permissions for your IP.Board powered website

If you have to ask this, you might not need or want to. Apache is the easier, more popular route people tend to take. Nginx is a small but very powerful and highly efficient web server. Apache with mod_php can be inefficient and needlessly waste system resources. Nginx can offer a potential gain in performance with the added benefit of not hogging your servers memory pool. Nginx is also simply a personal preference of mine, even though I'm not resource constrained on my box.

Why should I use Nginx over Apache?

Nginx Installation instructions



Add the following repository to your

# Nginx Official Debian Repository
deb http://nginx.org/packages/debian/ squeeze Nginx
deb-src http://nginx.org/packages/debian/ squeeze Nginx

Now import the PGP key,

wget http://nginx.org/keys/Nginx_signing.key
apt-key add Nginx_signing.key
rm Nginx_signing.key

Update and install,

apt-get update
apt-get install Nginx

Nginx should automatically start after installation. If you have Apache installed and running however, this will fail. So please ensure you stop (and optionally uninstall) Apache before proceeding further. If you don't uninstall Apache, just know the two will be fighting for ports and IP's to bind every time you restart the server. You can confirm Nginx is running by using .

/etc/init.d/Nginx status



Add the following repository to

name=Nginx repo

Now import the PGP key,

wget http://nginx.org/keys/Nginx_signing.key
rpm --import Nginx_signing.key
rm Nginx_signing.key

Install and start,

yum install Nginx
/etc/init.d/Nginx start

If you have Apache installed and running, Nginx will fail to start, so please ensure you stop Apache before proceeding further. You can confirm Nginx is running by using . You can also have Nginx start automatically on boot by running .

/etc/init.d/Nginx status

chkconfig --level 345 Nginx on

PHP-FPM Installation Instructions



Add the following repository to your

# DotDeb Debian Repository
deb http://packages.dotdeb.org squeeze all
deb-src http://packages.dotdeb.org squeeze all

Now import the GPG key,

wget http://www.dotdeb.org/dotdeb.gpg
apt-key add dotdeb.gpg
rm dotdeb.gpg

Update and install,

apt-get update
apt-get install php5 php5-apc php5-cli php5-common php5-curl php5-fpm php5-gd php5-imagick php5-imap php5-mysql


Nothing complicated here. Just install,

yum install php-cli php-curl php-fpm php-gd php-imap php-mysql php-xml php-pecl-apc

You can also have PHP-FPM start automatically on boot by running .

chkconfig --level 345 php-fpm on

Nginx Configuration

Now that you have both Nginx and PHP-FPM up and running, we can move on to configuring the web server. First, let's make some small adjustments to /etc/nginx/nginx.conf,

user  nginx;
worker_processes  4;
error_log  /var/log/nginx/error.log error;
pid        /var/run/Nginx.pid;
events {
    worker_connections  1024;
http {
    include       mime.types;
    default_type  application/octet-stream;
    server_tokens off;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    keepalive_timeout  30;
    #gzip  on;
    include conf.d/*.conf;

There's not much to say here. Increase worker_processes from the default of 1 to however many processor cores your server has. For example, if your server has a single quad core processor, set this value to 4. I've also changed the error_log directive to only log errors.

Now let's move on to configuring your IP.Board website. Use this as your base template for /etc/nginx/conf.d/ipboard.conf,

server {
    listen       80;
    server_name  yourdomain.com www.yourdomain.com;
    root         /srv/http/yourdomain.com/root;
    # Basic web server configuration.
    index        index.php
    #access_log   off;
    client_max_body_size  1G;
    # GZIP static content not processed by IPB.
    gzip  on;
    gzip_static on;
    gzip_http_version 1.1;
    gzip_vary on;
    gzip_comp_level 6;
    gzip_proxied any;
    gzip_types text/plain text/css application/json application/x-javascript application/xml application/xml+rss text/javascript application/javascript text/x-js;
    gzip_buffers 16 8k;
    gzip_disable "MSIE [1-6].(?!.*SV1)";
    # Set up rewrite rules.
    location / {
        try_files  $uri $uri/ /index.php;
    # Stub and FPM Status
    location /server_status {
        stub_status on;
        deny all;
    location /fpm_status {
        deny all;
        fastcgi_pass   unix:/var/run/php-fpm/ipboard.sock;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        /etc/nginx/fastcgi_params;
    # Deny access to hidden files
    location ~ /. {
        deny  all;
    # Mask fake admin directory
    location ~^/admin/(.*)$ {
        deny     all;
    # Secure real admin directory
    location ~^(/nimda/).*(.php) {
        #deny          all;
        #auth_basic    "Restricted Area";
        #auth_basic_user_file $document_root/nimda/.htpasswd;
        fastcgi_pass   unix:/var/run/php-fpm/ipboard.sock;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        /etc/nginx/fastcgi_params;
    # IP.Board PHP/CGI Protection
    location ~^(/uploads/).*(.php)$ {
        deny     all;
    location ~^(/hooks/).*(.php)$ {
        deny     all;
    location ~^(/cache/).*(.php)$ {
        deny     all;
    location ~^(/screenshots/).*(.php)$ {
        deny     all;
    location ~^(/downloads/).*(.php)$ {
        deny     all;
    location ~^(/blog/).*(.php)$ {
        deny     all;
    location ~^(/public/style_).*(.php)$ {
        deny     all;
    # Caching directives for static files.
    location ~^(/uploads/profile/).*.(jpg|jpeg|gif|png)$ {
        access_log off;
        expires    1d;
    location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|xml|htm|txt|swf|cur)$ {
        access_log off;
        expires    1w;
    # Pass PHP scripts to php-fpm
    location ~ .php$ {
        fastcgi_pass   unix:/var/run/php-fpm/ipboard.sock;
        fastcgi_index  index.php;
        fastcgi_buffers 16 8k;
        fastcgi_buffer_size 16k;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        /etc/nginx/fastcgi_params;

There are a lot of things to cover here. First things first, replace yourdomain.com www.yourdomain.com in the server_name directive with your forums domain name. Include both the www and non-www version of the domain as above, regardless of which you actually use. You'll also need to replace yourdomain.com in the root and error_log directives.

Next, let's cover where we're going to be installing the forum to. You'll notice I'm not using a /home/someuser heirachy as you're likely used to. There are two main reasons for this. Neatness and security. If you have your own server, you should keep your personal user account seperated from the actual web server. This is simply good practice in general. /srv/ on the Linux filesystem is for site-specific data which is served by the system. On all of my servers, I use a /srv/http/{host}/{subhost} hierarchy. "root" referrences the domains root. If you ever host another service on your forum (such as a wiki at wiki.yourdomain.com), you could store it in /srv/http/yourdomain.com/wiki

Let's go ahead and set this up for your IP.Board website now. Remember to replace yourdomain.com!

mkdir -p /srv/http/yourdomain.com/root
useradd --system ipboard
groupadd --system http
gpasswd -a nginx http
chown -R ipboard:http /srv/http/yourdomain.com

If you want to disable access logging on your site to reduce disk load, just uncomment the access_log directive under the basic web server configuration section. Either way, we'll be setting Nginx up to not waste log entries for static resources further down.

Now adjust client_max_body_size to the maximum allowed file size users can upload to your site. So if your highest allowed upload size for any of your members anywhere on your site is 750 MegaBytes, set this to 750M.

Next, we get into configuring rewrite rules. There's nothing you really need to do here. All we have to do with Nginx is pass a try_files directive, which is better than relying on the rewrite method required with Apache.

The Stub Status module allows you to see how many active connections your server has by visiting yourdomain.com/server_status. If you have a static IP address, you can set it up to ensure that only you will have access to this page, though leaving it public isn't really a vulnerability.

Next, we want a secure installation, right? So that means we're not going to be using /admin anymore. Come up with something creative that you can easily remember to replace /admin. The example above uses "nimda" (which is just "admin" backwards), so replace that with whatever you come up with.

Going a step further, there are two primary ways you can further secure your ACP. The most secure way would be to only allow connections from your IP. Though this will obviously not be feasible if you or another one of your administrators have an IP that constantly changes. If you do have a static IP that you can use, however, uncomment the "allow" and "deny" lines while adding an allow directive for your IP address bellow allow The second option is to use .htpasswd authentication. IP.Board has a built in function to set this up for you in the security center. You can configure this after installing your forum. After creating the .htpasswd file in your ACP, all you will need to do is uncomment the auth_basic directives and restart Nginx.

The caching directives are split for user avatars and everything else not processed by IP.Board. I have avatars set to expire after one day. For everything else it's one week. Feel free to adjust this to your own personal prefference.

PHP-FPM Configuration

On to PHP-FPM! First, let's go ahead and get rid of the default/example configuration we don't need. This is /etc/php5/fpm/pool.d/www.conf on Debian and /etc/php-fpm.d/www.conf on CentOS. Now, create a new file replacing www.conf with ipboard.conf and using this as the base template:

; Set the prefix directory and the user/group to run under
prefix = /var/run/php-fpm
user = php-fpm
group = http
; Configure listen(2) directives
listen = ipboard.sock
listen.backlog = 4096
listen.owner = php-fpm
listen.group = http
listen.mode = 0660
; Set up the process manager
pm = static
pm.max_children = 10
pm.max_requests = 250
pm.status_path = /fpm_status
; The timeout for serving a single request. Prevents runaway scripts.
request_terminate_timeout = 5m
; Only execute .php scripts.
chdir = /srv/http/yourdomain.com/root
security.limit_extensions = .php
; Environment variables.
;env[TMP] = /tmp
;env[TMPDIR] = /tmp
;env[TEMP] = /tmp
env[DOCUMENT_ROOT] = /srv/http/yourdomain.com/root
; PHP flags and security directives for just this site
php_flag[display_errors] = off
php_admin_value[open_basedir] = /srv/http/yourdomain.com/root:/tmp:/usr/bin
php_admin_value[disable_functions] = escapeshellarg,escapeshellcmd,exec,ini_alter,parse_ini_file,passthru,pcntl_exec,popen,proc_close,proc_get_status,proc_nice,proc_open,proc_terminate,show_source,shell_exec,symlink
php_admin_value[upload_max_filesize] = 1G
php_admin_value[post_max_size] = 1G

First, replace all instances of "yourdomain.com" accordingly again. Next, let's go ahead and set up everything we need for PHP-FPM to run properly,

mkdir /var/run/php-fpm
useradd --system php-fpm
chown php-fpm:root /var/run/php-fpm

You can skip creating the directory on CentOS, as it will have already been created, but you will still need to apply the new permissions.

Finding the optimal setting for pm.max_children is a bit of trial and error. I have it set to 10 as a default. You may need to increase or lower this setting depending on how constrained your server is and how much traffic your site receives. IP.Board tends to use an average of about ~52MB per process for me. So if you have 2GB of memory to spare on your server after accounting for MySQL and any other services you run, you could probably safely support up to 35 static processes, though you may not need nearly this many. Setting this too high may result in your server running out of memory, so be careful.

At this point, let's go ahead and test to make sure everything works. (Use /etc/init.d/php5-fpm restart for Debian)

/etc/init.d/nginx restart
/etc/init.d/php-fpm restart

No problems? Good! Problems? Double check your work and check your error logs.

Let's create /srv/http/yourdomain.com/root/test.php as a test file with a simple phpinfo() call,


Now visit yourdomain.com/test.php. You should see a PHPINFO page containing the open_basedir and other directives we set. If you don't, you did something wrong. Go back and review. If all is well, delete the test.php file. (Really, delete it. Security audit tools scan for files like this, and you don't want to needlessly expose your server information to malicious parties.)

Setting Up IP.Board

If you're setting up a new IP.Board website, you'll need to install mysql-server, set up a new database and so on. There are plenty of resources available for learning how to do this if you don't know how already. After you copy your entire forum directory into /srv/http/yourdomain.com/root, we just need to set up proper permissions,

chown -R ipboard:http /srv/http/yourdomain.com
find $httproot -type d -exec chmod 0750 {} ;
find $httproot -type f -exec chmod 0640 {} ;
find $httproot/{uploads,cache,downloads,hooks,screenshots,blog,public/style_*} -type d -exec chmod 0770 {} ;
find $httproot/{uploads,cache,downloads,hooks,screenshots,blog,public/style_*} -type f -exec chmod 0660 {} ;

If you haven't already, rename your /admin directory to whatever you chose earlier. Afterwards, create the following constants.php file in your forums root, replacing "nimda" appropriately,

    define( 'CP_DIRECTORY', 'nimda' );

Update the permissions,

chmod 0640 constants.php
chown ipboard:http constants.php

And that's it! IP.Board should be up and running. If you're having trouble, review all of the above steps and check your error log (/var/log/nginx/error.log).

Link to comment
Share on other sites

from the reading I have done general rule of thumb is worker_processes is number of processors you have. the only controversy I have seen over it is with hyperthreading, some say include that while others say only count each core as 1.

eg quad core with HT called 4 and not 8.

really don't know which is better.

It can be recommended to up this value if your load is more disk than CPU bound (eg. a file server).

Also, from what I can see Gabriel, your forum is plenty fast on my end. Pages are fully loaded and rednered within 0.5-1.2 seconds. If it seems slow-ish for you, it may just be because you're in a different continent, like I said in your other thread :tongue:

There's little to nothing you can do about that, other than find a good DC in Brazil. I wouldn't have any recommendations for you, and it may in fact be true that they all suck. But cross continental latency is something that's going to hit you if you host in the U.S. and live in Brazil.

In these regards, I may consider recommending you look into technologies such as Google's SPDY and even Pagespeed to help optimize and reduce some latency troubles, but this won't offer you any guaranteed results and involves a decent bit of work and effort to configure properly.

Edit: I also have AdBlock enabled like another user. It takes around 2-3 seconds for the page to fully render with it disabled. It's going to be even slower for you in Brazil because of the latency issues, but your website itself doesn't seem to have load time issues.

Link to comment
Share on other sites

from the reading I have done general rule of thumb is worker_processes is number of processors you have. the only controversy I have seen over it is with hyperthreading, some say include that while others say only count each core as 1.

eg quad core with HT called 4 and not 8.

really don't know which is better.

I suppose it depends on the writer's feelings of HT. HT does make things more efficient at multitasking, but it didn't add any true power to the system. But depending largely on HT as if it's a real core tends to cause problems when optimizing and yield more erratic behavior. Like, it will say the load is "okay" and then suddenly jump to ridiculous numbers after just a small increase whereas a system with HT turned off won't have such behavior as often.

But at the end of the day for this specific case, it won't matter. Because...

  1. OP isn't going to make an nginx only server. So, even trying to maximize cores for nginx is pointless.
  2. Nginx cpu/mem consumption is so low (compared to php-fpm), it's not going to be a notable issue, even if set too high.
  3. Even with assignment of 1/2 of cores avl, OP is not going to run short of nginx workers.
Link to comment
Share on other sites


This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...