Clover13 Posted November 28, 2023 Posted November 28, 2023 (edited) Members are running into upload failures in some cases, not sure why this is happening as there is quite a bit of disk space quota available (40GB) at the time of these failures. Files are being left behind (2.3GB each on avg) because of the failures and eventually completely capping the disk space quota. Would a single image take up to 40GB of free disk space when it can't do it all in memory? ImagickException::445 unable to write pixel cache '/tmp/magick-ZvfCYR5vTGWaIyfii_ZaZEePhJOiAI69': Disk quota exceeded @ error/cache.c/WritePixelCachePixels/5852 #0 /home/xxxxxxxxxxxx/public_html/system/Image/Imagemagick.php(237): Imagick->rotateImage() #1 /home/xxxxxxxxxxxx/public_html/system/Image/Image.php(169): IPS\Image\_Imagemagick->rotate() #2 /home/xxxxxxxxxxxx/public_html/system/File/File.php(326): IPS\_Image::create() #3 /home/xxxxxxxxxxxx/public_html/system/File/File.php(436): IPS\_File::create() #4 /home/xxxxxxxxxxxx/public_html/system/Helpers/Form/Upload.php(755): IPS\_File::createFromUploads() #5 /home/xxxxxxxxxxxx/public_html/system/Helpers/Form/Upload.php(309): IPS\Helpers\Form\_Upload->processUploads() #6 /home/xxxxxxxxxxxx/public_html/init.php(931) : eval()'d code(17): IPS\Helpers\Form\_Upload->__construct() #7 /home/xxxxxxxxxxxx/public_html/system/Helpers/Form/Editor.php(766): IPS\Helpers\Form\hook607->__construct() #8 /home/xxxxxxxxxxxx/public_html/system/Helpers/Form/Editor.php(185): IPS\Helpers\Form\_Editor->getUploader() #9 /home/xxxxxxxxxxxx/public_html/system/Content/Item.php(5092): IPS\Helpers\Form\_Editor->__construct() #10 /home/xxxxxxxxxxxx/public_html/system/Content/Item.php(4769): IPS\Content\_Item->commentFormElements() #11 /home/xxxxxxxxxxxx/public_html/uploads/template_cache/template_2_7a734b11232065691bc1176040cb26ad_topics.php(5425): IPS\Content\_Item->commentForm() #12 /home/xxxxxxxxxxxx/public_html/system/Theme/SandboxedTemplate.php(61): IPS\Theme\Cache\class_forums_front_topics->topic() #13 /home/xxxxxxxxxxxx/public_html/applications/forums/modules/front/forums/topic.php(611): IPS\Theme\_SandboxedTemplate->__call() #14 /home/xxxxxxxxxxxx/public_html/system/Dispatcher/Controller.php(118): IPS\forums\modules\front\forums\_topic->manage() #15 /home/xxxxxxxxxxxx/public_html/system/Content/Controller.php(50): IPS\Dispatcher\_Controller->execute() #16 /home/xxxxxxxxxxxx/public_html/applications/forums/modules/front/forums/topic.php(39): IPS\Content\_Controller->execute() #17 /home/xxxxxxxxxxxx/public_html/init.php(931) : eval()'d code(15): IPS\forums\modules\front\forums\_topic->execute() #18 /home/xxxxxxxxxxxx/public_html/init.php(931) : eval()'d code(21): IPS\forums\modules\front\forums\ignoretopics_hook_topicController->execute() #19 /home/xxxxxxxxxxxx/public_html/init.php(931) : eval()'d code(21): IPS\forums\modules\front\forums\raffles_hook_loadCss->execute() #20 /home/xxxxxxxxxxxx/public_html/system/Dispatcher/Dispatcher.php(153): IPS\forums\modules\front\forums\tthumb_hook_forums_module_topic->execute() #21 /home/xxxxxxxxxxxx/public_html/index.php(13): IPS\_Dispatcher->run() #22 {main} Edited November 28, 2023 by Clover13
Driven 2 Services Posted November 28, 2023 Posted November 28, 2023 The /tmp directory is often a partition set to a limited site, or a limited size RAM disk.
Clover13 Posted November 28, 2023 Author Posted November 28, 2023 5 minutes ago, Driven 2 Services said: The /tmp directory is often a partition set to a limited site, or a limited size RAM disk. It's part of the home directory for the site, but it's possible that specific tmp directory could be partitioned on its own. However adjusting disk quota in WHM does alleviate the problem, so that leads me to believe it isn't partitioned separately or in a limited way but is tied to the disk quota defined in WHM. I guess what's odd is the shear size of what's required to run these images through Imagemagick. The member sent me a sample file that failed and it was 20MB, but the temp file remaining after the upload failure is 2.3GB and unclear how much space Imagemagick took up on disk before it failed, but I'm assuming upwards of 20GB+ for a single upload/conversion to hit a failure point since that much space was available at the time of failure. Since increasing the disk quota in WHM and cleaning up the tmp files, the uploads have worked fine, but I shouldn't need 100GB+ of headroom to upload a 20MB file, so something is definitely wonky.
DawPi Posted November 28, 2023 Posted November 28, 2023 47 minutes ago, Clover13 said: Disk quota exceeded @ error/cache.c/WritePixelCachePixels/5852 Hmm, out of space? Just guessing..
Ryan Ashbrook Posted November 28, 2023 Posted November 28, 2023 Working with images, particularly large ones, is actually a pretty intensive process. For instance, if I open Affinity Designer, t immediately uses about 750-800MB of memory, then opening a single 1.6 MB file makes memory usage jump to 1 GB. If the files are going to /tmp then it's likely because ImageMagick is running out of memory available to it so it's forced to use disk instead, which has larger space to work with, but is much slower. It may be worth contacting ImageMagick with your configuration to see if any improvements can be made.
Clover13 Posted November 28, 2023 Author Posted November 28, 2023 (edited) 24 minutes ago, DawPi said: Hmm, out of space? Just guessing.. Yes, that is what happened. The question is why. 20GB of free space available, 20MB file upload and it runs out of space? 1 minute ago, Ryan Ashbrook said: Working with images, particularly large ones, is actually a pretty intensive process. For instance, if I open Affinity Designer, t immediately uses about 750-800MB of memory, then opening a single 1.6 MB file makes memory usage jump to 1 GB. If the files are going to /tmp then it's likely because ImageMagick is running out of memory available to it so it's forced to use disk instead, which has larger space to work with, but is much slower. It may be worth contacting ImageMagick with your configuration to see if any improvements can be made. Thanks Ryan! Yeah, that's how I understood it. Once memory runs out, it goes to disk. Just mind blowing the amount of space, but I know image processing is intensive, just wasn't thinking that intensive. Good idea to contact them, will look into that. Edited November 28, 2023 by Clover13
Randy Calvert Posted November 28, 2023 Posted November 28, 2023 Also most servers /tmp is much smaller by default. Maybe also work with host to up the size of tmp. It won’t fix the underlying issue but could reduce impact. Or write a script to clean tmp occasionally.
Clover13 Posted November 28, 2023 Author Posted November 28, 2023 Just now, Randy Calvert said: Also most servers /tmp is much smaller by default. Maybe also work with host to up the size of tmp. It won’t fix the underlying issue but could reduce impact. Or write a script to clean tmp occasionally. If a tmp file exists, it means the upload failed. Cleanup helps reducing impact to other uploads that would otherwise succeed if those tmp files wouldn't exist however. Ultimately, we don't want any failures, so it's about trying to figure out what exactly is required for these file uploads. FWIW, some of this is related to the ImageMagick policy defined, looking at my server resource limits for it, I see the following which seems quite large, but also tells me that a 20MB image in this case exceeds the 64GB max memory required and goes to disk and that one file eventually fills up the quota. I'm not exactly sure what the cap is however, but increasing it to 100GB with about 90GB free has thus far eliminated these issues. 20GB of additional free space was insufficient. Resource limits: Width: 1.1529215EP Height: 1.1529215EP List length: unlimited Area: 265.58GP Memory: 61.8342GiB Map: 123.668GiB Disk: unlimited File: 786432 Thread: 24 Throttle: 0 Time: unlimited
Recommended Posts