[vbox-dev] vhd resize corruption

[Aaron Brice]

So the 40GB .vhd file that contained a Windows 7 VM was running out of
room, and I wanted to resize it.

With VBoxManage 4.3.10 on Ubuntu I ran:

VBoxManage modifyhd --resize 61440

Anyway, long story short it won't boot, gparted live ISO doesn't find a
partition table..

So I started investigating the .vhd file format. There is 512 bytes in
the beginning consisting of a copy of the footer, then 1024 bytes of the
header, then a variably sized BAT (Block Allocation Table), then the
blocks of data. When the max size of the disk changes the BAT has to be
resized to accommodate potential new blocks. It looks like modifyhd
resizes the BAT, but did not shift the BAT pointers or move the data, it
just expanded the BAT into the first data block, wiping out the MBR. I
don't have the original for comparison, but now the first entry in the
BAT is 0xa3 (163). This used to point to the sector just after the BAT,
but now the BAT has expanded (from 20480 4-byte entries to 30720) and
sector 163 is entry 20481 in the table, and the MBR table is all 0xff's..

Relevant bug: https://www.virtualbox.org/ticket/11960

Thanks,
Aaron

[Alexander Eichner]

Hi Aaron,

sorry for the late answer (was on vacation last week). I?m not able to reproduce the
resize corruption you report (I was never able to looking at the mentioned ticket).
Which tool did you use to create the original VHD image?

Regards,
Alexander Eichner

Post by Aaron Brice
So the 40GB .vhd file that contained a Windows 7 VM was running out of
room, and I wanted to resize it.
VBoxManage modifyhd --resize 61440
Anyway, long story short it won't boot, gparted live ISO doesn't find a
partition table..
So I started investigating the .vhd file format. There is 512 bytes in
the beginning consisting of a copy of the footer, then 1024 bytes of the
header, then a variably sized BAT (Block Allocation Table), then the
blocks of data. When the max size of the disk changes the BAT has to be
resized to accommodate potential new blocks. It looks like modifyhd
resizes the BAT, but did not shift the BAT pointers or move the data, it
just expanded the BAT into the first data block, wiping out the MBR. I
don't have the original for comparison, but now the first entry in the
BAT is 0xa3 (163). This used to point to the sector just after the BAT,
but now the BAT has expanded (from 20480 4-byte entries to 30720) and
sector 163 is entry 20481 in the table, and the MBR table is all 0xff's..
Relevant bug: https://www.virtualbox.org/ticket/11960
Thanks,
Aaron
_______________________________________________
vbox-dev mailing list
vbox-dev at virtualbox.org
https://www.virtualbox.org/mailman/listinfo/vbox-dev

[Alexander Eichner]

Hi Aaron,

(please keep the topic on the list)

I tested the code by writing random data to it, resizing the image and verifying that the disk content didn?t changed by reading and comparing the data.

Resizing is a quick operation because VBox just relocates the data blocks which would be overwritten by the extended BAT. The data blocks are appended to the image file and
in most cases only one block needs to be relocated. If you still ave the original and the broken VHD image file around can you please mail me the first 5MB of each image so
I can have a look at them?

Regards,
Alexander

VirtualBox 4.2 was used to create the .vhd, according to the vhd header.
What do you mean by not able to reproduce, just that it still boots or did you verify the BAT table doesn't overwrite a data block and that the offsets were recalculated correctly?
When I ran the resize, it completed very quickly (a few seconds) to resize a 40GB file to 60GB capacity. The BAT table is towards the beginning of the file, I'm not sure how you can increase the size of the BAT table so quickly without overwriting data, I would think you'd have to create a new file? It takes several minutes to copy the file..
Aaron

Post by Alexander Eichner
Hi Aaron,
sorry for the late answer (was on vacation last week). I?m not able to reproduce the
resize corruption you report (I was never able to looking at the mentioned ticket).
Which tool did you use to create the original VHD image?
Regards,
Alexander Eichner

Post by Aaron Brice
So the 40GB .vhd file that contained a Windows 7 VM was running out of
room, and I wanted to resize it.
VBoxManage modifyhd --resize 61440
Anyway, long story short it won't boot, gparted live ISO doesn't find a
partition table..
So I started investigating the .vhd file format. There is 512 bytes in
the beginning consisting of a copy of the footer, then 1024 bytes of the
header, then a variably sized BAT (Block Allocation Table), then the
blocks of data. When the max size of the disk changes the BAT has to be
resized to accommodate potential new blocks. It looks like modifyhd
resizes the BAT, but did not shift the BAT pointers or move the data, it
just expanded the BAT into the first data block, wiping out the MBR. I
don't have the original for comparison, but now the first entry in the
BAT is 0xa3 (163). This used to point to the sector just after the BAT,
but now the BAT has expanded (from 20480 4-byte entries to 30720) and
sector 163 is entry 20481 in the table, and the MBR table is all 0xff's..
Relevant bug: https://www.virtualbox.org/ticket/11960
Thanks,
Aaron
_______________________________________________
vbox-dev mailing list
vbox-dev at virtualbox.org
https://www.virtualbox.org/mailman/listinfo/vbox-dev

[Alexander Eichner]

Hi Aaron,

finally found the bug. The corruption was caused by to a missing cast and resulting in an integer overflow later on.
The fix will be part of the next maintenance release.

Regards,
Alexander Eichner

Post by Alexander Eichner
Hi Aaron,
thanks for the images, I?ll have a look at them.
We are definitely interested in data corruption issues but because we are a small team with lots of work
it might take a while to get a response sometimes as we are working on other issues. Besides, I was on vacation last week and didn?t watch
mails closely.
I?ll see what I can find out from your images.
Regards,
Alexander Eichner

Post by Alexander Eichner
Hi Aaron,
(please keep the topic on the list)

Sorry, I had unsubscribed from the list when it didn't seem like anyone was interested in data corruption issues..

Post by Alexander Eichner
I tested the code by writing random data to it, resizing the image and verifying that the disk content didn?t changed by reading and comparing the data.
Resizing is a quick operation because VBox just relocates the data blocks which would be overwritten by the extended BAT. The data blocks are appended to the image file and
in most cases only one block needs to be relocated. If you still ave the original and the broken VHD image file around can you please mail me the first 5MB of each image so
I can have a look at them?

I didn't backup the original. However, I wrote this script to resize my broken .vhd back down to the original size: https://gist.github.com/ambrice/c36b46237fcc979d80c9
I'm not 100% sure after running that script that it was back to the original state, for example I didn't adjust the drive geometry fields. However after running that, "vhd-util check" from the blktap-utils package says it's valid, whereas before it was complaining that "block 0 (offset 0xa3) clobbers headers". So I then cloned it to a .vdi and resized the .vdi and that worked (after running the Windows 7 repair to restore the partition table and boot manager).
So I have a valid 40GB .vhd which may not match the original 100%, but I just tried to resize it to 60GB again and ran into the same problem. I ran: VBoxManage modifyhd AaronVM.vhd --resize 61440
It got quickly to 100% with no errors, and the file size didn't change at all (40766087168 bytes before and after). Attached are the first 5MB of the file before and after the resize.
Thanks,
Aaron
<after.vhd.xz><before.vhd.xz>