User:CWood/Disk Image Management

From OSDev Wiki
Jump to navigation Jump to search

Inevitably, in the early days of operating system development, you will use a virtual machine. Indeed, you may need to use several. Disk images are, thus, an inevitability, and the only viable option for using virtual machines (see below). This tutorial is meant more for people who are not so familiar with the terminal, and the Linux kernel internals (some of these will definitely work in BSD etc, others I have no idea, as I'm not familiar with other *NIXen)

Comparison of Alternatives

USB Flash Drives

Many of these commands can be avoided, by using 'tricks' such as using a bootable USB drive as your disk image(s). But, this comes with its own set of issues, which are much more difficult to address. Namely:

  • Loss of portability - If you plan on releasing your OS, it will entail more work for people to build your OS. This is because not everybody will have a USB flash drive present, and many will be unwilling to look for one.
  • Loss of convenience - If you are developing away from your own dev box, it is not enough to have the source files, and prerequisite toolchain any more; you will need to remember to have the correct flash drive present. This gets even more unwieldy if you use the flash drive for other things as well, especially if you take it with you to places other than your dev box(en).
  • Lack of size - Many flash drives, especially the cheaper ones, are limited in capacity. While it may be unlikely, it is entirely possible that your OS, especially as it matures, will outgrow your chosen flash drive, meaning a new one, with more capacity, must be obtained, and set up.
  • Difficulties with version control - As stated above, it is possible you may need more than one bootable image. There are two ways to go about this with USB drives:
    • Multiple drives. This will get unwieldy. Fast.
    • Multiple kernels on one drive. This requires use of a multiboot bootloader, or similar. However, if one kernel runs rogue and starts modifying things on the disk, there is nothing to prevent this. Furthermore, file management will get messy as more versions are added.
    • Multiple partitions on one drive. This still requires a multiboot loader, and still has write protection issues. However, file management is simplified, at the expense of (possibly) having to store duplicate data.

That is not to say, however, that bootable USB flash memory is not without its benefits.

  • When you get to the point wherein you are ready to test on real hardware, there is no work involved in setting up a hard disk/floppy disk for boot. Simply boot straight from your development drive of choice. This is also convenient, as it is easier to test on a multitude of hardware using this method.

Issues with Disk Images

Partition Management

Mounting Partitions in a Clean, Portable Way

One of the main issues with disk images, is partition management. If you wish to have a partitioned image (which is most people, especially if HDD support is needed), you will run into issues at some point, the main one being mounting in the host OS.

If your host OS is a *NIX, you may find yourself doing the following (or something similar) to mount a particular partition on an image. 

losetup /dev/loop0 disk.img -o 1048576

Clearly, this uses a magic number, that is completely devoid of context. For reference, the magic number refers to the offset, in bytes, to the start of the partition. There is, however, a much better way.

If you have used fdisk to edit disk images, you will have seen it refer to things like /dev/loop0p1. A common source of confusion is why fdisk refers to this, but the loop module doesn't implement them. Well, as it turns out, it does, at least in Linux. It's probably just turned off in your kernel, however. To activate this behaviour, run: 

sudo rmmod loop
sudo modprobe loop max_part=63
losetup /dev/loop0 disk.img
mount /dev/loop0p1 /mnt

No magic numbers, and a much neater command history. This is much easier to remember, but the ramifications go much deeper than that. Portability is hugely improved by this, as the shell script/makefile is no longer dependent on the size of the partitions in your disk image. Indeed, by adding a parameter to the configuration of your makefile (through whatever means this is achieved in your OS), it can be made to be independent of partition order as well.

Finding the Correct Loopback Device

This one is fairly easy, and will be somewhat obvious to people who have used Bash for any length of time. But, its a time saver nonetheless.

Setting up your loopback device is easy, using: 

sudo losetup /dev/loop0 disk.img

However, this is not portable. If /dev/loop0 is being used by something else on the system, we cannot claim it for ourselves, and there is no guarantee that it is not being used by something else. Therefore, it is better to use: 

sudo losetup -f disk.img

However, after running this, we no longer know where disk.img is mounted. So, to find this, we can run: 

losetup -a | grep disk.img | cut -d: -f1

What this does is list all of the loopback devices, grep for our image, and chop out which specific device this belongs to. This can be put inside a mount command, like so (Makefile): 

mount $(shell losetup -a | grep disk.img | cut -d: -f1)p1 /mnt

(Shell script): 

mount `losetup -a | grep disk.img | cut -d: -f1`p1 /mnt

This will find our loopback device, and mount it to /mnt, so that we can perform file operations from the host OS. The p1 is explained in the previous section, Mounting Partitions in a Clean, Portable Way.

Retrieved from "https://wiki.osdev.org/index.php?title=User:CWood/Disk_Image_Management&oldid=15448"