Automatic Daily Offsite Backup Of My IPB/Server

[WestTac1]

Can anyone recommend a solution that backs up my server, more specifically my forums, automatically nightly to a cloud or another offsite option?

I looked at some online services, and they are ridiculously expensive.

Thanks for any help

Seth

[ASTRAPI]

You can get a vps if your forum is not huge and do a local backup and then transfer it there

Very easy very cheap and fast

Just create a simple bash script or look at then net for a few samples.

[Tracy Perry]

Better than "transferring it" is to (if your forum is on a VPS itself which I'm going to assume since you say "your server") configure rsync on it and another VPS. 

Or NFS share restricted to the IP of the server so that others cannot connect to it.

Or you could look at https://backupsy.com/.  I know a few that are using that and having good luck with it.

[BomAleold]

an example of bash script https://github.com/Flynsarmy/cloud-db-backup?files=1

[Paul.F]

I use Plesk on my VPS, with the dropbox extension. This backs up my Website to my dropbox on a schedule. This also allows you to roll back to a backup from 1 day ago, or a week ago. The extension is free for Plesk. Plesk is $4 USD per month. Much cheaper then Cpanel at approx $14 USD per month.

[Valthos]

www.codeguard.com is a good service for web and database backups.

[Sly_Ripper]

I use R1Soft with a licence/hosted from r1softlicenses.com, setup could be challenging if you're not comfortable with Linux. I run incremental daily backups, it even logs into MySQL and flushes the databases so there's no issue if you need to perform a restore. 

I've restored a whole server from it which suffered from a hardware failure and it booted up right where the backup left off.

It could be a bit overkill for what you need but well worth it if you have a large amount of GBs to backup.

[Tracy Perry]

5 hours ago, Sly_Ripper said:

It could be a bit overkill for what you need but well worth it if you have a large amount of GBs to backup.

It's a nice ability -but for a lot the minimum $140 USD a month buy-in will be cost prohibitive.  I know for me, it's easier just to use rsync to a remote server for the vital files/data.  I can burn a new instance out and be back up running in a few hours that way.  But my sites are not "life & death" as I do not depend on them for any income.

[Makoto]

Please don't use a budget VPS to store/handle your backups. I would discourage you from using a VPS for backups period, there's no reason to when there are so many affordable and vastly superior options out there.

Amazon S3 can provide an excellent storage and backup solution, and best of all you only pay for what you use. Infrequent access storage (which is great for backups) runs only $0.0125/GB. That's cheap enough to store 100GB worth of backups for only $1.25/month. You can automate backups using S3 as well, you just need to transfer your backups to S3 instead of to another server with rsync.

There's also Google Cloud Storage, which runs $0.02/GB for infrequent access storage.

Both are much, much more suitable for handling backup storage than a VPS. Not only does it end up being much cheaper, using a real reputable storage service is also going to be astronomically more reliable than any VPS you'll ever find, which is the primary reason I recommend using one.

[Tracy Perry]

19 minutes ago, Makoto said:

Please don't use a budget VPS to store/handle your backups. I would discourage you from using a VPS for backups period, there's no reason to when there are so many affordable and vastly superior options out there.

Again, depends on your redundancy.  I used to store the backups across 3 different VPS (rsync), then rsynced to my desktop and also rsynced to my two servers (each RAID 10 , 1 with 2TB storage and the other 4TB) at the house - at no additional cost since I was using those VPS's for other things also.  The odds of 6 different locations going TU at the same time are astronomical - especially considering the VPS's were not from the same provider.  Since all I need is the actual file(s) for the sites and a few config files - I could regen the OS instance easily.

Now, can/does everyone have  that ability?  Nope, and I realize that.  But not everyone wants to deal with Amazon S3 either (some people I know have a LOT of the Amazon IP space blocked at the firewall) and others have an aversion of anything Google period.

And  honestly - Amazon can suffer from the same problem that anyone else does on storage.  It really depends on what one needs and how familiar they are setting it up.

[Makoto]

Offline backups are important no matter what, you should always keep some bakups saved locally somewhere, though they don't have to be as frequent.

But (again) no server configuration you will ever build will come close to competing with the level of redundancy a storage solution such as Amazon S3 or Google Cloud Storage can provide, ever. As strong and as redundant as your above setup may be, it still would not come close to competing.

See:

• https://aws.amazon.com/s3/faqs/#data-protection
• http://docs.aws.amazon.com/AmazonS3/latest/dev/DataDurability.html

A setup like you're describing above is also notably more complex and substantially more expensive of an investment. Amazon S3 is not that difficult to set up, at the very least it's not much more difficult than setting up a VPS to rsync with. You're just paying a lot more for significantly less with a VPS.

[Tracy Perry]

23 hours ago, Makoto said:

A setup like you're describing above is also notably more complex and substantially more expensive of an investment. Amazon S3 is not that difficult to set up, at the very least it's not much more difficult than setting up a VPS to rsync with. You're just paying a lot more for significantly less with a VPS.

I think you may have missed the point that it was costing me no additional monies. 

It was utilizing infrastructure that was already in place and being used for other purposes. 

For those that don't mind dealing/using Amazon's features, it would be fine.  But to just dismiss outright the use of a VPS (or several) for that function is being a tad shortsighted.

[Makoto]

40 minutes ago, Tracy Perry said:

I think you may have missed the point that it was costing me no additional monies. 

I didn't miss it, and I didn't say it was costing you anything. I said the setup you are describing is complex and substantially more of an investment, and I meant that in a general sense. It's not something that pretty much anyone else will have on hand. It's awesome that you have a setup like that and can make use of it, but almost no one else will have that kind of hardware on hand. So while that works for you, it's not relevant to anyone else here.

40 minutes ago, Tracy Perry said:

For those that don't mind dealing/using Amazon's features, it would be fine.

Yes, so you've stated above as well. The firewall issue is anecdotal and unlikely to be a problem for most, especially for those that are managing their own dedicated servers. Outside of that, I don't know what you mean by "for those that don't mind dealing with Amazon", because using S3 to handle backups is about as simple as using rsync. There's nothing really you have to "deal" with. Just about everything can be fairly easily automated.

40 minutes ago, Tracy Perry said:

For those that don't mind dealing/using Amazon's features, it would be fine.  But to just dismiss outright the use of a VPS (or several) for that function is being a tad shortsighted.

It's not really being shortsighted by saying VPS' are a poor choice just for managing backups, since they factually are. You're comparing hosting your backups on a relatively minuscule RAID configuration that is shared and heavily abused by various other clients running god knows what on their servers with Amazon's massive datacenters, where you're data is heavily replicated across storage configurations so large and so complex it would blow the average persons mind. Then on top of that, your data is again replicated across multiple other datacenters that do the exact same thing but in completely different physical locations.

All of this is being done at a fraction of the cost of what you will pay for a VPS, so I'm not sure why you would ever think that a VPS is the better choice when faced with an option between the two.

[Tracy Perry]

2 minutes ago, Makoto said:

IOutside of that, I don't know what you mean by "for those that don't mind dealing with Amazon", because using S3 to handle backups is about as simple as using rsync. There's nothing really you have to "deal" with. Just about everything can be fairly easily automated.

Personally, I don't really care to give them any of my funds due to the mount of hassle I have from blocking their sites that are PITA's in their actions.  I get a TON of spammer attempts from their IP ranges.

 

4 minutes ago, Makoto said:

All of this is being done at a fraction of the cost of a VPS, so I'm not sure why you would ever think that a VPS is the better choice when faced with the option

I get it.. you don't like the thought of VPS's for backups.. but many others do... and have been very successful with them long before Amazon came on the scene.  I personally (and multiple others I know) prefer to have as much control over our environment as possible... kind of like how some prefer more features/choices in scripts.

[Makoto]

VPS' lose data all the time. It's not at all a rare occurrence for data loss to occur with a VPS. You're sharing a server with many people that heavily abuse shared hardware. VPS' are not made for redundant storage, they're made primarily for hosting web and other types of servers. You deploy applications on them and use them for their processing power, they are not meant to be used for backups.

So the reason I don't like the thought of using VPS' for backups is because VPS' are plain and simply comparatively unreliable. While you have a better chance of consecutively winning the powerball than ever losing any data you host with S3, almost everyone who has been in this industry long enough has dealt with at least one host that has lost their data before.

So, in summary,

VPS'

• Data loss is not exceptionally rare
• Expensive when it comes to storage
• No data replication
• Data is generally split across single RAID configurations
• Easy setup with rsync

Amazon S3 / Google Cloud Storage

• Better chance of winning the powerball consecutively than ever losing a single byte of data
• Extremely affordable and infinitely scalable when it comes to storage
• Data is replicated across multiple datacenters. Two of the datacenters your data is stored in can consecutively vanish off the face of the earth and you still won't suffer any data loss
• Data is stored across a multitude of gigantic storage configurations spread throughout massive datacenters
• Easy setup with any of the many quality CLI utilities available for S3

I really can't explain the above in any greater detail. If having complete control over virtualized hardware is important enough to you that you would disregard everything above and still use a VPS to handle your backups, I'm not going to try and convince you to change your ways. You can only lead a horse to water, you can't make him drink.

Speaking of horses, I think this one is probably dead. Probably because he wouldn't drink the water.

[NoGi]

I currently backup to my prod server and then rsync to my SSD based VPS which happens to also be my test server so it's handy having prod backups on that same box. Offline backups are also taken.

Interested in checking out what S3 also offers as a 3rd online backup location as I wouldn't be dumping my VPS. Do you just rsync to it? 

[Makoto]

You would need to use one of the available CLI utilities for communicating with S3. I'll write up a guide on setting up automated backups with S3 sometime in the future.

[NoGi]

Thanks that would be great. Using cli isn't an issue as that's all I get with my VPS lol

[Luis Manson]

what do you people use?

right now i just make mysqldumps from a mysql slave.

Im looking at:

• offsite backups of www-root to my home storage or google nearline.

i have some ideas, but i do not know any tools to do this:

dnotify backup, to only backup changes without scanning the whole site (mainly uploads/) 

google nearline script to do daily backup/incremental

i plan to do mysqldump's upload to my home storage, but i wonder how to secure backups in case the server with the ssh keys get compromised

 

 

 

[ASTRAPI]

Just use Attic backups as you can encrypt them

[Mopar1973Man]

Heck I've been using "rsync".

Neat part is I'm a Linux user at home. so I build a shell script and placed it in the "anacron" folder so its executed in the time frame by the folder. Daily, weekly, monly, etc. Doesn't matter if your PC is off or not. Next time you fire up Linux will run it the script if missed. So then my script runs rsync to backup my home folder. My VPS already covers the other stuff. So I'm double backed up. Home machines are Ubuntu 14.04.3 LTS and server is CentOS Linux so works well.

[NoGi]

On 1/6/2016 at 9:06 AM, Makoto said:

You would need to use one of the available CLI utilities for communicating with S3. I'll write up a guide on setting up automated backups with S3 sometime in the future.

@Makoto, I've got it all working now with s3cmd sync via cli to S3 and it's working really well.

BTW, how are you going with that PM I sent you regarding the watermarks? Haven't heard back in a while