Suggestions on backing-up VM filesystems

[Whoever]

I have a couple of KVM VMs with the images stored as qcow2 image files that I want to backup to another machine (and ultimately off-site). I want to automate this to run on a weekly basis.

I don't want to stop the VMs if possible. I realize that this could mean slightly corrupted backups if the qemu volume is being written to at the time that I make the backup.

Any suggestions on how to do this?

[MartinR]

Amanda?

[Whoever]

Amanda?

The issue isn't backing up the files.

The issue is getting a full snapshot of the VM's filesystem, without stopping the VM for the time it takes to make the snapshot.

[MartinR]

Sorry, but you'll have to be a bit clearer. Amanda records the filesystem as part of its backup. After all /var/log/messages gives you a pretty broad hint that you need the directories /var and /var/log!

[pjsr2]

What you need to consider carefully is:
- which applications are running on the VM and whether these applications hold data changes (cached) in memory that have not been (completely) written to disk?
- copying an image of a live system is not an atomic operation: files may be changing while you are copying the image.

For example, you may need to let a running database server create a complete data dump just before you take your backup. Thus you can always restart the database server from that dump. The database server may not be restarting properly when the image is copied while the VM is running, because the copy from the data file is taken at a different point in time from the moment the transaction log is copied, so the contents of the data and the transaction log no longer match.

[MartinR]

Yep, just the same as a bare metal system.

[jlehtone]

Luckily, VM can reboot way faster than bare metal; no minutes of hardware POST.

I would say that reasonable options are
(1) controlled backup of essential files from live system, or
(2) shutdown, copy image, and restart.

[MartinR]

Both reasonable suggestions, but do be aware that option 2 will be doing an image backup of the container file. If you have a fair amount of free space on your virtual disks you will have backups full of empty space. Every time you do it the whole disk is backed up, whether changes have occurred or not.

Most database systems have the option to perform a dump of some description. Dumping the database to a file before the backup will ensure that you have a recoverable position.

[KernelOops]

I've been in the same position, how to reliably backup virtualbox VMs.

In my case, they are a mix of centos VMs and windows 10.

For windows 10 VMs, since they run nothing important (who would do serious work on windows 10 anyway... hah), I decided to create systemd unit files for each VM, then send a "savestate", which immediately shuts down the VMs and allows me to quickly back up whole images. Then quickly restart them once the backup procedure is complete.

For the linux VMs, I decided to backup their files via samba shares with zero down time. If a VM dies, I can just boot a new VM in its place, reconfigure/setup via my ansible playbook and load the samba shares.