⇓ Sendfile support

[Makoto]

Sendfile is an IPS plugin that implements server X-Sendfile support into the software suite. Use this to improve server performance on large communities that handle a lot of attachment and/or IDM file downloads.

What is X-Sendfile?

In short, it gives PHP a way to tell your web server to deliver files to the client after proper permission checks and other actions have been performed by IPS. This way, you are not relying on PHP itself to deliver the files to the client.

The reason you don't want PHP to handle file transfers is simple: PHP is really inefficient at it. This is particularly important if you run IPS in a self-hosted environment and your community makes heavy use of attachments or IP.Downloads file submissions.

When you have a lot of users trying to download files concurrently, PHP will rapidly consume your servers available memory pool and, assuming your server makes use PHP-FPM or fastCGI, it will lock up your available PHP processes that are used to serve pages to your other users. This means basic topics and other pages will not load until another users download has completed.

By utilizing X-Sendfile, you are able to pass the workload back off to the web-server. This is immensely more memory efficient, faster, and will keep your PHP processes free. This overall makes IPS significantly more scalable, especially in a self-hosted environment.

Configuration

Assuming you were able to run the test script provided in the plugin description without error, all you will have to do after installing this plugin is enable it and ensure you have the correct web server selected. After that, that's it! The plugin will hook into all local file download requests processed by IPS and pass them off directly to the web server using X-Sendfile.

 

 

1. Plugins

Sendfile can be installed and configured directly from the Plugins page in your AdminCP.

2. Settings

To configure Sendfile, just click the pencil icon in the plugins list.

3. Enable

Make sure the "Enable Sendfile" option is toggled

4. Web server selection

Sendfile will attempt to automatically detect the web-server you are using and populate this field accordingly. All you need to do is make sure it is correct and click "Save"! If you are not sure, just ask your hosting provider.

 

---

That's it! If you have any issues, please let me know here, or submit a bug to my Issues tracker.

 

☕ Found this useful? Want more free plugins and guides like this one? Show your support by buying me a coffee!
- https://ko-fi.com/makoto_dev

Edited December 23, 2018 by Makoto

[ExiledVip3r]

This plugin worked fine for me on IPB 4.4, however after updating to 4.5 it's started to cause all downloads to error out. 

Nginx error logs show the following:

Quote

2021/01/09 19:14:12 [error] 795161#795161: *1188 rewrite or internal redirection cycle while internally redirecting to "/index.php", client: <snip>, server: www.<snip>.com, request: "GET /files/file/13459-<snip>/?do=download&r=816033&confirm=1&t=1&csrfKey=10a1f5ccf53d7fab9ed2f2e39fa9c35f HTTP/1.1", upstream: "fastcgi://unix:/run/php/php7.4-fpm.sock", host: "www.<snip>.com"

 

[Makoto]

Just now, ExiledVip3r said:

This plugin worked fine for me on IPB 4.4, however after updating to 4.5 it's started to cause all downloads to error out. 

Nginx error logs show the following:

Can you review the plugins configuration and make sure everything is still set up properly?

Are you sure nothing was changed server configuration wise prior to the upgrade? No permission changes or anything like that?

[ExiledVip3r]

Plugin settings are correct. It's set to enabled and Nginx for webserver.

Using the sendfile_test works fine as well.

It seemed to work fine when I was using an older version of the plugin on IPB 4.5. Errors started when I updated the plugin to the current version.

Nothings changed in the server config that I can remember changing. But here's my nginx config file for it:

server {
        listen 80;
        listen [::]:80;
        server_name <snip>.com;
        rewrite ^ http://www.<snip>.com$request_uri permanent;
}

server {
        listen 80;
        listen [::]:80;

        listen 443 ssl http2;
        listen [::]:443 ssl http2;

        ssl_certificate /etc/nginx/ssl/<snip>.2020.chained.crt;
        ssl_certificate_key /etc/nginx/ssl/<snip>.2020.key;
        ssl_dhparam  /etc/nginx/ssl/dhparam.pem;

        server_name www.<snip>.com static.<snip>.com dl.<snip>.com srv1.<snip>.com;

        root /home/<snip>/www/public;

        index index.php index.html;

        client_max_body_size 1024M;

        access_log /var/log/nginx/<snip>.access.log main buffer=32k flush=5m;
        error_log /var/log/nginx/<snip>.error.log;
        #access_log off;
        #error_log on;

        rewrite ^/privacypolicy$ /privacy permanent;

        location ~ ^/uploads/screenshots/(.*){ return 301 https://www.<snip>.com/screenshots/$1; }


        location ~* \.(?:ico|js|css|png|jpeg|jpg|gif|xml|bmp)$ {
                expires max;
                add_header Pragma public;
                add_header Cache-Control "public, must-revalidate, proxy-revalidate";
                log_not_found off;
                access_log off;
                #limit_req zone=two burst=10;
        }

        #location ~* (?:/public/style_|/cache/|/hooks/|/uploads/).*\.(?:php|cgi|pl|php3|php4|php5|php6|phtml|shtml)$ { deny all; }
        location ~ /\.ht { access_log off; log_not_found off; deny all; }
        location = /robots.txt  { access_log off; log_not_found off; }
        location = /favicon.ico { access_log off; log_not_found off; }

        location / {
                #include admin;
                #try_files $uri $uri/ /index.php?$is_args$args;
                try_files $uri $uri/ /index.php?$query_string;
                #return 503;
        }

        error_page 503 @maintenance;
        location @maintenance {
                rewrite ^(.*)$ /error503.html break;
        }

        location ~ ^/(uploads|datastore|system|plugins)/.*\.(?:php\d*|phtml)$ {
                allow 127.0.0.1;
                deny all;
        }

        location ~ \.php$ {
                include fastcgi_params;
                try_files $uri =404;

                fastcgi_send_timeout 60;
                fastcgi_read_timeout 60;
                fastcgi_buffers 128 8k;
                fastcgi_pass unix:/run/php/php7.4-fpm.sock;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_param PHP_VALUE open_basedir="/home/<snip>/www:/usr/share:/home/<snip>/tmp:/tmp";

        }

}

 

Edited January 10, 2021 by ExiledVip3r

[Makoto]

Ah, the plugin changed a bit with 4.5 if you're using Nginx.

Quote

If you are using this with Nginx, you will need to reconfigure your internal redirects in your storage settings.

You'll need to go to your storage settings in the AdminCP and reconfigure the internal path for your storage configurations there,

I see you're not actually using any internal routes in your Nginx configuration, which isn't a huge deal, you'll just need to configure this to point to /uploads or wherever else your web server is configured to deliver files from.

[ExiledVip3r]

Ah! Working perfectly now. Thanks for the help!

Suppose I should've actually read the changelog. 

[Makoto]

Just now, ExiledVip3r said:

Ah! Working perfectly now. Thanks for the help!

Suppose I should've actually read the changelog. 

No worries! Glad it's working now. It was changed up a bit with 4.5 to make it better suited for more complex configurations, as now you can configure everything individually if you have multiple storage configurations.

Let me know if you run into any other issues!

[2002faq]

I sendfile only useful when assets are stored on the local system? Any affect if they are in S3?

Thank you

[Makoto]

10 minutes ago, misfit76 said:

I sendfile only useful when assets are stored on the local system? Any affect if they are in S3?

Thank you

SendFile only functions for files stored on the local filesystem, it will not do anything on Amazon S3 or other cloud storage configurations.