Customized remote backup

Factual Questions
1

There are two home offices, located on opposite sides of the country. One has a network attached storage device, the other has a regular internal drive. Both have dynamic IP addresses allocated from their respective ISPs. I have a hosted Web site that can run scripts as necessary.

I want a program that runs in the background to back up the NAT device from one office to the other. In other words, it would run similar to normal backup software (i.e., automatically making backups of files that have been added or changed), but it would send the files over the Internet to the other office.

I’m posting here because there are two things I haven’t found. First, the mainstream backup software I’ve looked at so far doesn’t seem to work with an NAT (it’s a Linksys NAS200 mini-Linux box, if that makes a difference). The ones that I’ve found that do seem to be geared towards large-enterprises and run well over a grand(!).

Second, while a lot of backup software has FTP capability, they seem to assume sending to a static IP/location.

Have I overlooked a program that already does this?

If there is nothing available, how hard would it be/what would be entailed by writing (or, rather, having written) a set of scripts to link the two offices together? That is, something that would sit on both computers, communicate with the Web site to locate each other, then use that information to make the connection. Aren’t the elements already in place—isn’t this basically how Napster et al work? I wouldn’t necessarily need the Web site to manage the throughput of the data (is that a cogent use of the term?), just establish the basic P2P connection.

I’ve used Rentacoder in the past with great success, but I don’t have any idea what something like this would require programming-wise. That is, is this a small project (i.e., simple modifications to a set of pre-existing scripts) or something larger and more involved?

Any advice/direction you have would be greatly appreciated – even if you point me to a more focused forum where this type of question would be more at home.

Thanks!

Rhythm

2

Unless you have significant bandwidth (both upload and download) this is not a good idea because it will simply take too long. Look at your daily incremental backups and calculate how long it would take to transfer it.

At one site I was at they had a dedicated T1 (1.5 Mbps) link and wanted to do backups from the other end until I produced stats which shows that the initial backup would max the link for over a week and the daily backup would max the link for more than 24 hours.

3

To expand on what Quartz said, don’t forget that your upload speed is usually much slower (sometimes by an order of magnitude) than your download speed.

That said, just off the top of my head, I think you could accomplish this pretty easily with cron (free), rsync (free), and a dynamic DNS service (free or cheap, depending on what you need to do).

I think writing the required scripts would be pretty quick. As a WAG, maybe a day’s work?

4

Thanks so far!
As for bandwidth, I was thinking of starting local then shipping. That is, the initial backup would take place locally, so while it may take a few hours to image an entire drive, it won’t be hampered by slow upload speeds. Once the first backup is done, I’d box it up and overnight it to the remote location (for the two or three days of transition, I could work in a separate space so the initial image is clean). Once it’s there, then incremental backups, while non-trivial, woluld be capable of transmitting in a few overnight hours. Looking at the type of files we work on, we’d rarely exceed fifty MB, with the ultra-rare exception of creeping up to a hundred. Looking at an upload calculator and putting in basic values from a recent bandwidth test (~500kbps) says that even a hundred meg may only take about an hour.

Aren’t cron and rsync Unix commands? I don’t have any real access to the NAT other than their browser-based interface. We do have a Mac in the system, but not at both ends—everything else are Winboxes.

5

Hmm. What about DeltaCopy. Looks like it’s rsync for Windows, and it integrates with the Windows Task Scheduler, which would be your cron analogue. I’ve never tried it, but it looks interesting.

I have used DynDNS and DNS Made Easy for dynamic DNS hosting. I know you can set up a DynDNS account for free, but you will have to click a link in an email once a month to keep it alive. Or, you can pay something like $30/year or thereabouts and you won’t have to do anything like that. I think that’s what I pay DNS Made Easy. You will need some kind of script or service to keep their DNS servers updated if your IP address changes, but you can usually download one from their site.

6

Hmm, I just read your other thread, and now I’m confused. Can you access the files on your NAS from Windows (the files you need to back up)? Via a drive letter mapping or UNC path or something?

P.S. In my experience, NAT generally stands for Network Address Translation, NAS is Network Attached Storage.

7

How often are you going to update the base backup? And how are you going to do it?

Don’t eyeball the amount of data that you’ll be backing up, quantify it.

And I’d suggest differential backups rather than incremental ones: it makes recovery so much simpler.

8

NAS. NAS. NAS. Oops. :smack:

The RAID(1) in the NAS should protect us from hard drive failure, and the ability to read one of those drives in a different enclosure (point of the other thread) protects us from the NAS failing. But what if we’re robbed? What if the house catches on fire or is destroyed by a herd of rabid giraffes? What if some quirk in the NAS (it’s running a software RAID) simultaneously corrupts both disks? It’s awfully irresponsible of us to not do regular backups and keep them somewhere else.
RE: the connection
Given the prevalence and age of P2P or VPN connection technology, I’m a bit baffled why there isn’t a readymade commercial solution. Isn’t this almost exactly how P2P works? Two computers connect to a central server, it coordinates the exchange of information about their IP address and ports, then leaves it to them to form, maintain, and close the connection. I’m just looking to have this information passed on to a backup utility.

Given the amount of pre-written scripts and routines out there, does anyone have an idea of what it would take a “real” programmer to put something like that together? That is, if I end up going to Rentacoder with this, what kind of range could I be looking at?

RE: Incremental backup
This is looking a bit harder to get around. I’ve seen customized non-Linksys firmware, but I’m not sure I wouldn’t be trading one potential source of problems for another. I guess it makes sense—I may be able to keep track of what programs I save from the desktop, but an incremental backup program would need to keep track from the server’s vantage point.

I have lots of old hardware around to play with, and it looks like Linux isn’t very demanding anyway. How hard would it be to set one a Linux file server to mimic what the NAS is doing now? It looks like Ubuntu Server Edition has about everything I can think of—or am missing something big?

Thanks!

Rhythm

9

Just a note on this: many of the clients that you can use to update DynDNS when your IP address changes can also be configured to periodically update your account’s status, and eliminate the need to manually visit a URL to keep it alive. I use ddclient with DynDNS, and it works great. So far, it’s been set it up, and forget about it.