virtnbdmap fails to map data: Unexpected block range size.

[Zoltan-LS]

Hello,

I am trying to do this : https://github.com/abbbi/virtnbdbackup?tab=readme-ov-file#single-file-restore-and-instant-recovery

virtnbdmap -f  /backup/1738594502/vda.full.data

.
.
.
[2025-02-04 10:12:33] INFO virtnbdmap - main [MainThread]: Started nbdkit process pid: [1190321], Logfile: [/tmp/nbdkit2nmwwgt7.log]
[2025-02-04 10:12:33] INFO virtnbdmap - main [MainThread]: Done mapping backup image to [/dev/nbd0]
[2025-02-04 10:12:33] INFO virtnbdmap - main [MainThread]: Press CTRL+C to disconnec

The nbd devices only give me proper answers if the file system is ext.

[root@kvm1-dev ~]# file -s  /dev/nbd0p1
/dev/nbd0p1: Linux rev 1.0 ext4 filesystem data, UUID=2027b160-50b5-42e2-9060-1342b5f6bd7e, volume name "BOOT" (extents) (64bit) (large files) (huge files)
[root@kvm1-dev ~]# file -s  /dev/nbd0p3
/dev/nbd0p3: ERROR: cannot read `/dev/nbd0p3' (Input/output error)
[root@kvm1-dev ~]#

Any ideas why this can happen?

Regards,
Zoltan

[abbbi]

Might also be that xfs UUID clashes or missing xfs utils cause the problem. What does "dmesg" or "journalctl" show if you try to access the XFS based partitions?

fdisk -l /dev/nbd0

shows?

Another issue might be that sometimes the calculation that re-maps the offsets in the backup files to the real block offsets is done wrong (and i suspect this to be the case here). Hard to catch all those possible situations for testing, so the mapping functionality should be considered "beta". In the case of wrong offset calculations i think the logfile /tmp/nbdkit2nmwwgt7.log will contain some exceptions.

[abbbi]

its a total shoot into the dark, but could you try with this branch, too?

https://github.com/abbbi/virtnbdbackup/tree/nbdmap

patched version available at:

https://github.com/abbbi/virtnbdbackup/actions/runs/13144766618

(artifacts packages.zip)

[Zoltan-LS]

Doesn't seem to have made a difference.

You can download such a backup to test from here : https://zoltan.man-1.vm.plesk-server.com/vda.full.data

It's a simple, basic Alma image

[abbbi]

You can download such a backup to test from here : https://zoltan.man-1.vm.plesk-server.com/vda.full.data

unfortunately that 404 for me ..
I tried with several virtual machines i have here and was unable to reproduce the issue :(
so there must be something special about your vm disk / block geometry.

[Zoltan-LS]

Sorry, created a proper subdomain for download https://download.zoltan.man-1.vm.plesk-server.com/vda.full.data

[abbbi]

Doesn't seem to have made a difference.
It's a simple, basic Alma image

thanks, i was able to reproduce. I just pushed a few minor fixes to the branch again an iam now able to mount
this image correctly on my test system:

./virtnbdmap -f ../Downloads/vda.full.data -d /dev/nbd4
root@cefix:/home/abi/Downloads# mount /dev/nbd4p3 /mnt

I however get some journalctl messages about corrupted xfs metadata:

Feb 05 10:10:11 cefix kernel: XFS (nbd4p3): Unmount and run xfs_repair
Feb 05 10:10:11 cefix kernel: XFS (nbd4p3): First 128 bytes of corrupted metadata buffer:

not sure if this is because the backup was done without filesystem freeze (no qemu agent) or if theres still an issue
left with the mapping.

[Zoltan-LS]

That is ok. Repair_xfs is need to be ran on the restored disk as well. I'll check this and get back to you .

[Zoltan-LS]

This is perfect so far.

I'll run a few tests , with bigger disks and incrementals and let you know.

The base idea here was to reduce restore time on big disks. I am expecting some memory/cpu penalties here with the mapper.

You have a really good piece of software here :) Stable , reliable , top notch , really impressive!

[abbbi]

thanks, ive merged the changes to the master branch: any enhancement here is an actually better than the state before ;)
There might still be issues, but they're really backup-dependant.