Hello there! Here’s the thing: I got some old HDD for my Debian home server, and now that I have plenty of disk space I want to keep a backup of the OS, so that if something accidentally breaks (either SW or HW) I can quickly fix it.
now the question is: which directory should I include and which should I exclude from the backup? I use docker a lot, is there any docker-specific directory that I should back up?
I think most options have been covered here, but I’d like to consider some other bits…
User accounts & file permissions:- if you have >1 account, note that the UserID is internally numbered (starting from 1000, so Bob=1000, Sue=1001) and your file system is probably setup using the numerical UserID… so re-creating the users in a different order would give Sue access to Bob’s files and vice versa.
Similarly, backing up /etc /var etc… you should check if any applications (ie databases) need specific chmod and chown settings
Rsync, tar, etc can cover some of this, you just need to check you rebuild users in the correct order.
Maybe Ansible is another approach? So your disaster recovery would be:
- Install plain OS on new drive
- Get Ansible access to it (ie basic netwroking)
- Rebuild OS and instsll applicstions automatically with Ansible
- Restore application & home folders (again with Ansible)
When you get this working, it’s amazing to watch an entire system being rebuilt
It is a question I’ve spent a lot of time trying to work out. Can’t speak to docker.
Some of the specifics of Keeps and Dontkeeps depend on details of your system. You have to find out where the distro, DM and other apps keep the following:
Dontkeeps:
- trashes
- temp files
- file indexes … IMHO these dont backup properly if you leave them in and will prevent you from completing the task
- device files
Keeps:
- list of installed packages — explicit and deps separate if possible
- config files: /etc, ~/.config, ~/.* on a case by case basis… I say remove the obvious large temp dirs and keep the rest by default for simplicity
- for the system configs I’ve had a tool called etckeeper running for a while because it was highly recommended but I’ve never actually restored from it…
- personal documents and other files such as typically kept in the home directory
- /root occasionally has something you need
Ways to investigate:
- use a disk usage utility to find out where your storage is being used up … It’ll help you find large Dontkeeps
- watch for recently modified files
- dirs and files that are modified all the time are usually temp dirs. But sometimes they have something useful like your firefox profile.
Most backup solutions are ONE of the following:
- User files
- System files
Don’t spend too much time crying about needing two solutions. Just make your backup today and reach perfection later.
Remember: sync isn’t backup. Test your backup if you can (but its not as easy as it sounds). Off site your most precious files.
I back up all user data, for me that’s some folders in
/var
, all of/etc
(don’t need everything, but it’s small enough), mysqldumps and pgsqldumps of my databases and the output ofdpkg --get-selections
. And the backup script itself.Everything is rsynced to a server in another location.
I want to switch to Borg backup but haven’t found the time yet.
I want to switch to Borg backup but haven’t found the time yet.
Since you haven’t switched yet, perhaps consider restic too. I looked into Borg when I was trying to decide which backup software to use, but ended up deciding on restic instead and using resticprofile to ease implementation.
everything. download clonezilla, perform a bare-metal backup and if the worst happens, you can reflash your new disk to exactly how the old one was without skipping a beat.
*most of the time. And test backups to make sure they really work or you have no true backup.
clonezilla tests it as part of its backup process.
I usually only keep documents and media. Programs can be redownloaded and reinstalled (and it might be better to reinstall them in case you move to a new OS anyway to ensure compatibility).
For docker specifically, only keep stuff that’s specific for your instance; which you normally setup as an external volume anyway. Docker is designed such that you should be able to nuke the container, and all persistent data is restored via an external volume on the host. If you’re not doing that, you should immediately go and set that up now (to get the data out safely, setup a volume connection such that the container path is new - that way you don’t accidentally destroy what’s there, copy the stuff you need out, then readjust the path so it’s correct)
If you need to restore it, back it up.
How do you plan to restore if the whole drive dies?
thanks for the reply!
How do you plan to restore if the whole drive dies?
good point: I guess if the problem is that I messed something up I may just run the backup script in reverse? if the problem is HW then I’d have to reinstall and then restore the backupped directory probably is there a “standard” way?