Suggestions on backing-up VM filesystems

General support questions
Post Reply
Whoever
Posts: 1122
Joined: 2013/09/06 03:12:10

Suggestions on backing-up VM filesystems

Post by Whoever » 2020/06/21 20:34:26

I have a couple of KVM VMs with the images stored as qcow2 image files that I want to backup to another machine (and ultimately off-site). I want to automate this to run on a weekly basis.

I don't want to stop the VMs if possible. I realize that this could mean slightly corrupted backups if the qemu volume is being written to at the time that I make the backup.

Any suggestions on how to do this?
Last edited by Whoever on 2020/06/22 03:00:54, edited 1 time in total.

MartinR
Posts: 577
Joined: 2015/05/11 07:53:27
Location: UK

Re: Suggestions on backing-up VM filesystems

Post by MartinR » 2020/06/21 22:15:05

Amanda?

Whoever
Posts: 1122
Joined: 2013/09/06 03:12:10

Re: Suggestions on backing-up VM filesystems

Post by Whoever » 2020/06/21 22:24:04

MartinR wrote:
2020/06/21 22:15:05
Amanda?
The issue isn't backing up the files.

The issue is getting a full snapshot of the VM's filesystem, without stopping the VM for the time it takes to make the snapshot.

MartinR
Posts: 577
Joined: 2015/05/11 07:53:27
Location: UK

Re: Suggestions on backing-up VM filesystems

Post by MartinR » 2020/06/22 07:52:55

Sorry, but you'll have to be a bit clearer. Amanda records the filesystem as part of its backup. After all /var/log/messages gives you a pretty broad hint that you need the directories /var and /var/log!

pjsr2
Posts: 476
Joined: 2014/03/27 20:11:07

Re: Suggestions on backing-up VM filesystems

Post by pjsr2 » 2020/06/22 09:54:21

What you need to consider carefully is:
- which applications are running on the VM and whether these applications hold data changes (cached) in memory that have not been (completely) written to disk?
- copying an image of a live system is not an atomic operation: files may be changing while you are copying the image.

For example, you may need to let a running database server create a complete data dump just before you take your backup. Thus you can always restart the database server from that dump. The database server may not be restarting properly when the image is copied while the VM is running, because the copy from the data file is taken at a different point in time from the moment the transaction log is copied, so the contents of the data and the transaction log no longer match.

MartinR
Posts: 577
Joined: 2015/05/11 07:53:27
Location: UK

Re: Suggestions on backing-up VM filesystems

Post by MartinR » 2020/06/22 11:11:23

Yep, just the same as a bare metal system.

User avatar
jlehtone
Posts: 2866
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Suggestions on backing-up VM filesystems

Post by jlehtone » 2020/06/22 12:16:01

Luckily, VM can reboot way faster than bare metal; no minutes of hardware POST.

I would say that reasonable options are
(1) controlled backup of essential files from live system, or
(2) shutdown, copy image, and restart.

MartinR
Posts: 577
Joined: 2015/05/11 07:53:27
Location: UK

Re: Suggestions on backing-up VM filesystems

Post by MartinR » 2020/06/22 13:42:50

Both reasonable suggestions, but do be aware that option 2 will be doing an image backup of the container file. If you have a fair amount of free space on your virtual disks you will have backups full of empty space. Every time you do it the whole disk is backed up, whether changes have occurred or not.

Most database systems have the option to perform a dump of some description. Dumping the database to a file before the backup will ensure that you have a recoverable position.

User avatar
KernelOops
Posts: 263
Joined: 2013/12/18 15:04:03
Location: xfs file system

Re: Suggestions on backing-up VM filesystems

Post by KernelOops » 2020/06/23 05:45:32

I've been in the same position, how to reliably backup virtualbox VMs.

In my case, they are a mix of centos VMs and windows 10.

For windows 10 VMs, since they run nothing important (who would do serious work on windows 10 anyway... hah), I decided to create systemd unit files for each VM, then send a "savestate", which immediately shuts down the VMs and allows me to quickly back up whole images. Then quickly restart them once the backup procedure is complete.

For the linux VMs, I decided to backup their files via samba shares with zero down time. If a VM dies, I can just boot a new VM in its place, reconfigure/setup via my ansible playbook and load the samba shares.
--
I love my computer - all my friends live there.
--

Post Reply

Return to “CentOS 7 - General Support”