operation failed: failed to format image: 'Failed to lock byte 101'

[larsser]

I get the error in the title when doing a auto backup with this command: /Backup/virtnbdbackup/virtnbdbackup -d ZimbraServer -l auto -z -o "/Backup/ZimbeaServer/$(date +'Week %V %Y')"
I've just pulled the latest version (which did not fix it) so any ideas whats causing it and how to fix it?

Full log:

[2023-05-29 12:34:00] INFO root virtnbdbackup - main [MainThread]: Backup level: [auto]
[2023-05-29 12:34:00] INFO root virtnbdbackup - main [MainThread]: Compression enabled, level [2]
[2023-05-29 12:34:00] INFO root virtnbdbackup - main [MainThread]: Backup mode auto, target folder is empty: executing full backup.
[2023-05-29 12:34:00] INFO root virtnbdbackup - main [MainThread]: Libvirt library version: [8010000]
[2023-05-29 12:34:00] INFO root disktype - Optical [MainThread]: Skipping attached [cdrom] device: [hda].
[2023-05-29 12:34:00] INFO root virtnbdbackup - main [MainThread]: Backup will save [1] attached disks.
[2023-05-29 12:34:00] INFO root virtnbdbackup - main [MainThread]: Concurrent backup processes: [1]
[2023-05-29 12:34:00] INFO root checkpoint - redefine [MainThread]: Loading checkpoint list from: [/tmp/Week 22 2023/checkpoints]
[2023-05-29 12:34:00] INFO root checkpoint - create [MainThread]: Checkpoint handling.
[2023-05-29 12:34:00] INFO root checkpoint - create [MainThread]: Using checkpoint name: [virtnbdbackup.0].
[2023-05-29 12:34:00] INFO root virtnbdbackup - main [MainThread]: Local NDB Endpoint socket: [/var/tmp/virtnbdbackup.1041165]
[2023-05-29 12:34:00] INFO root virtnbdbackup - main [MainThread]: Temporary scratch file target directory: [/var/tmp]
[2023-05-29 12:34:00] INFO root virtnbdbackup - startBackupJob [MainThread]: Starting backup job.
[2023-05-29 12:34:00] WARNING fs fs - freeze [MainThread]: argument unsupported: QEMU guest agent is not configured
[2023-05-29 12:34:00] ERROR root virtnbdbackup - startBackupJob [MainThread]: Failed to start backup: [operation mislykkedes: failed to format image: 'Failed to lock byte 101']```

[abbbi]

during the backup process a dirty bitmap is created by the libvirt daemon. To keep things consistant, it creates a so called "scratch file" that holds the data that changes while the bitmap creation is in progress. Its sort of the same situation as if a snapshot is created.

By default virtnbdbackup tells libvirtd to create the scratch file in target directory /var/tmp/. It seems it fails to create the scratch file (which is basically just a regular qcow image file). You need to check qemu or libvirt logs for the cause or maybe try setting an alternative path via --scratchdir option.

What filesystem is /var/tmp/ in your case? Maybe: https://bugzilla.redhat.com/show_bug.cgi?id=1891673

The cause for the issue is likely on libvirt/qemu side, not virtnbdbackup.

[larsser]

I've tried with --scratchdir but I just ran into another error so I've rebooted to a live Linux and did a fsck on both of my RAID arrays and it didn't find anything major.
After the machine booted normal again the issue is now gone.

I didn't know about /var/tmp/ only /tmp/ and I've overlooked it in the logs.
There is these file in /var/tmp/ can i remove them if no backup is running?:

virtnbdbackup.2286268.vda.pid
virtnbdbackup.2286268.vdb.pid
virtnbdbackup.23037.vda.pid
virtnbdbackup.23037.vdb.pid
virtnbdbackup.3508085
virtnbdbackup.37512.vda.pid

Thank you for your input!

[abbbi]

yes, these files can be removed if no backup is active, seems these are leftover files of prior (probably failed) backup attempts.