LinuxSA Mailing list archives

  From: Michael Neuling <michael.neuling@keyworks.com.au>
  To  : Richard Russell <richard@yellowgoanna.com>
  Date: Fri, 01 Feb 2002 15:36:28 +1030

Re: Union File Systems...

I had exacly the same problem but you don't need any special file
system, just use hard links.  If a file is the same then just hard
link it to the last version else create a new copy of the file.  The
directory strucure is recreated the same as the working directory
(links don't replace directories).

This, of course, doesn't work well if you have a lot of large files
with small changes.  We use it here at work and there is no problems
with efficiency, and of course backup access time is as good as it
gets.  

Restoring is easy as cp -a.  

One cool feature of using hard links is that you can delete any
incremental backup (with a simple rm -rf command) and the rest of the
backups remain consistant.  So if your backup disk runs out of space
you can just remove older incrementals or make them less regular by
deleting a few of them (say you keep daily backups for a month then
after that you only keep weeklies, then all you need to do is go back
a month and delete 6 out of every 7 incremental backups).

I have a few simple scripts which do all of this for you.  I've been
meaning to rewrite them in C (or as a option to rsync) but I've not
had the time (plus they run quick enough as is).  I run them each
night as our onsite backups.  Works a treat.  You can have them if you
like (or you can pay me for them :-))

In message <002d01c1aad8$554f9020$0c00a8c0@albert> you wrote:
> Hi all...
> 
> I was just discussing with a friend some techniques for keeping large
> backups online, and remembered Union File Systems... I believe FreeBSD
> does these nicely, but, for reasons involving other software, Linux is
> required in this situation -- does anyone know of how close union fs's
> are for Linux? Is there a core project out there? Is anyone willing to
> pay for it to be done?
> 
> I've done some brief searching around, and have come across something
> called IFS (Inherited File System) and TFS (Translucent File System --
> Sun) ... But not real sure what the status is on these...
> 
> Also, in this specific instance, the idea that I have running around in
> my head is pretty specific... I'd want:
> 
> Say four complete backups, that look like live filesystems, but are
> mounted read-only for users...
> 
> Eg:
> 
> /thisweek
> /lastweek
> /theweekbefore
> /theweekbeforethat
> 
> Then within each of these, there would be a directory for the backup of
> each day:
> 
> /weekend (full, done on sat or sun)
> /monday
> /tuesday
> /wednesday
> /thursday
> /friday
> 
> Monday through to Friday would be effectively incremental backups, but
> mounted on top of weekend (or on top of all the previous days), so they
> look like full FS's.
> 
> The idea would be that once per week, you could do a dump/restore (or
> tar c/tar x or etc) to create the weekly backup, and then every night,
> use rsync, dump, or whatever, to create the difference filesystems...
> After this, they could all be read-only, as you specifically wouldn't
> want to change anything anyway...
> 
> So it doesn't really need to be a proper live union FS, just one that I
> can create, then have read-only...
> 
> Of course, one option that I thought of was Katie, but I don't know if
> it's production ready yet, and it may be too slow to use on hundred MB
> plus fs's...
> 
> Comments?
> 
> rr
> 
> 
> -- 
> Richard Russell
> Yellow Goanna Pty Ltd
> e: richard@yellowgoanna.com
> m: +61 412 827 805
> f: +61 8 8462 2362
>  
> 
> -- 
> LinuxSA WWW: http://www.linuxsa.org.au/  IRC: #linuxsa on irc.linux.org.au
> To unsubscribe from the LinuxSA list:
>   mail linuxsa-request@linuxsa.org.au with "unsubscribe" as the subject
> 


Regards,
Michael
--
  "I don't have to be careful.  I got a gun."
    Homer J. Simpson

-- 
LinuxSA WWW: http://www.linuxsa.org.au/  IRC: #linuxsa on irc.linux.org.au
To unsubscribe from the LinuxSA list:
  mail linuxsa-request@linuxsa.org.au with "unsubscribe" as the subject