Least expensive way to host LARGE downloads files?

[Mastric]

I'm not sure what you're current host is, but dedicated solutions can cover the TB/mo you'll just need to add appropriate drives in the configuration. https://www.hivelocity.net/dedicated-servers/#products

 

[The Old Man]

Hi @superj707

I thought I was in a similar position and started looking at the costs for Amazon S3 and learning more about this week after reading this topic.
My VPS was showing 83% of the 75GB storage as used up (approx 62GB, leaving just 13GB free).

I found the Amazon S3 estimator difficult to understand, I'm in the UK but my server is on the East coast of the US where most of my members are, so I picked East Coast for the location with some guestimate transfer amount from the UK and the other top 5 countries according to the stats to help it estimate. Although I found some of the figures it requested by the stats programs in CPanel. I had no idea what to enter for the number of Get and Post requests and some of the other questions. I spent hours Googling for guides on translating what it was referring to so that I could enter as much info as possible to get a better idea, and I got pretty much no where.

Whilst Googling, I thought it just didn't sound right, my sites are surely not that large, so I did some searching to see what was using up all of the space via CPanel, and it turned out there were 100's of error_log and core.xxxxxx files clogging up many subfolders within the public_html folder on the drive, which were put there by the server due to some issue with error logging and core dumping I had a few months back. Most were a minimum of 40MB but there were loads of them, and I found some files at 1.5GB and some even larger than 2.5GB. All completely useless to me. Why is it so complex to work out and find the the stats it asks for?! Grrr..

I've just removed all of them using Search Remote Files feature in Filezilla (which took a while to run!). I carefully deleted them all in one go after sorting the search results, and now I'm showing storage space as 7.4GB used (5.5%) with 67GB free. This means I've freed up 54GB of space and can hold off on the Amazon S3 service for now, although I'd love to find a guide to help me use the estimator to work out what my site's files and IPS uploads would cost.

It may be worth checking your server, just in case.

 

[CP]

33 minutes ago, The Old Man said:

Whilst Googling, I thought it just didn't sound right, my sites are surely not that large, so I did some searching to see what was using up all of the space via CPanel, and it turned out there were 100's of error_log and core.xxxxxx files clogging up many subfolders within the public_html folder on the drive, which were put there by the server due to some issue with error logging and core dumping I had a few months back. Most were a minimum of 40MB but there were loads of them, and I found some files at 1.5GB and some even larger than 2.5GB. All completely useless to me. Why is it so complex to work out and find the the stats it asks for?! Grrr..

Are these the core files you are referring too? Also, were these files found in other locations other then just main public_html folder? 

[The Old Man]

Hi @CP

Yes, those were the core. files. Some folders had about 30 or more inside and all were 40MB+ each. Yes, for instance they were found in my member's blog folders forums/blog/member name/core.xxxxxx

Quote

Hello Graham,

System administration has determined that you have a script in your account that is "core dumping" on the server.  The core dumps were removed, but were location in your account at the following location:

/home/xxxxxx/public_html/content

Core dumps occur when a process is loaded into memory and terminates itself prematurely, causing the memory segment to be written to the disk as a file. You can find more information about them as well as how to read them here: http://www.inmotionhosting.com/support/website/what-is/what-are-core-dumps

While we are not sure exactly what is causing this behavior, we attempted to extract some data from the core dump to assist in locating the source:

unparseable coredump file

The core dumps being caused by the above command is related to a scripting issue, or the process that generated it may have crashed during execution. If this is a recurring issue, we highly recommend reviewing the file that generates the core dump to prevent further issues.

Feel free to contact us with any further questions.

 

Quote

We are happy to assist you with this issue. We are sorry to hear that you are having a problem with your 500 error on your account. I have renamed your .htaccess file to .htaccess_old this temporarily resolves this you would have to fix what is wrong in the .htaccess file. to resolve the issue by way of a developer who is good with reg EX coding.

I think this was part of the issue....

Quote

Fatal error: Call to a member function language() on null in /home/xxxxxxx/public_html/content/system/Output/Output.php on line 535

Fatal error: Uncaught exception 'IPS\Db\Exception' with message 'Prepared statement needs to be re-prepared' in /home/xxxxxxx/public_html/content/system/Db/Db.php:542 Stack trace: #0 /home/xxxxxx/public_html/content/system/Db/Db.php(839): IPS\_Db->preparedQuery('REPLACE INTO `c...', Array) #1 /home/xxxxxx/public_html/content/system/Session/Front.php(299): IPS\_Db->replace('core_sessions', Array, true) #2 [internal function]: IPS\Session\_Front->write('277ba10b6edad72...', '') #3 {main} thrown in /home/xxxxxx/public_html/content/system/Db/Db.php on line 542

Despite them advising the core dumps had been removed, they were but only in this one add-on domain. My other folders were choked with them.

[Daddy]

On 4/26/2017 at 10:24 PM, Rhett said:

My first instinct after seeing OVH is to "Run Fast" the other direction. 

I average 1TB a month with 5 million page views and I'm running an OVH dedicated server just fine. @superj707 there's plenty of providers who offer unlimited bandwidth and storage, however, they'll cap you if you become a problem and I think your mere file base right from the start would scare any potential provider away. I would highly recommend deleting old and outdated content as well as incorporating more compression in your systems algorithm.

PS: https://www.ovh.com/us/dedicated-servers/fs/160fs2.xml is well more than enough, in fact, it's a bit overkill. My OVH allowance is 500gb but I always go over and they don't say anything about it.

[CP]

Thanks for the reply @The Old Man I appreciate it. Looks like I will be busy.  

[The Old Man]

It'll be worth @CP, you have 362MB of storage used up in those 4 examples, alone. 

I found using FileZilla to search for them in my public_html folder and sub folders much easier than manually searching for them without missing any. 

[SJ77]

Hi Old Man

I agree with you about the Amazon S3 price estimator. There are so MANY options , loaded with jargon.  Their estimator tool is essential useless as far as I am concerned. These tech guys that design these things sometimes have no idea how to make something useful for a regular person trying to use their product.

 

[Il_Picasso]

Go with OVH