Testing
Developing an operating system is an almost completely useless exercise unless there is a way to test new versions of the OS or kernel as they are being coded. There are many different ways to set up a testing environment. Broadly, they can be divided into two methods:
- A physical machine
- A virtual machine (VM)
Things to look for in a successful testing environment
- Speed. You want to be able to reboot your (possibly virtual) testing machine and have it ready to go with the new kernel before you've finished typing the commands. You also want the code to execute as quickly as possible, so you can get back to coding instead of waiting for GRUB to load. Physical machines can win here, but it depends on hardware speed, of course (floppy drives are especially slow). A Core 2 Duo emulating a 486 will probably be faster than the 486 itself.
- Ease of use. Twenty different commands just to build and run a new kernel isn't going to be fun to do, so you'll probably do it less. And testing less is a great way for bugs to slip through. Emulators usually win here, because you can set up your makefile to build a disk image, ready to run in the emulator. Diskless Booting is an option for real machines.
- Closeness to the real thing. It's considered to be a good thing if your testing machine is as close to real as you can possibly manage. Of course, a physical machine is best for this, followed by Bochs.
Using a physical machine for testing
The main advantage of using a real machine for testing is that your operating system, if it runs on your testbed, is fairly likely to run on any other computer. Whereas, if you test using an emulator, it might not work on the real thing. The downsides are mainly that you need the space and money to maintain a second computer, or suffer from time spent rebooting your development machine. With the cheap hardware available today, it is recommended that you have a second physical computer. You can spend $3000 on a brand new testing computer, or you can spend $20 on a recycled 486. There are a few components, however, that it might be a good idea to have as a minimum:
- Floppy drive (for booting and testing FAT12 drivers).
- A VGA-compatible video card.
- Serial port.
- Hard disk (500MB is more than enough).
- PCI/ISA bus.
- Network card such as an NE2000 or a 3c90x.
This sort of machine makes a good base. From this, anything else you add is just an excuse to write more drivers. To perform a test, you can boot your kernel from a portable medium (Floppy or USB), or download it from your development PC using Diskless Booting. Booting from CD-ROM is another option, although it will probably require a lot of blank media (or CD-RWs). Another great technique is using a pen drive: just copy your floppy image on it, and the BIOS will emulate a floppy disk for you, as long as you use the BIOS routines to read it.
Brendan wrote a very good guide to purchasing and managing testing machines in this forum topic.
Using a virtual machine for testing
The advantages of using a virtual machine for testing are that it is a very simple process to compile and then run your new kernel (a shell script can make this a one command process), and that if you choose your emulator wisely, you can get a lot of free help in terms of debugging messages. There is a list of emulators here. Perhaps the two most recommended for OSDev are Bochs and QEMU. More information on virtual machine testing is available from the respective emulator's pages.
One downside of using a virtual machine is that choice of hardware is limited. For instance, both Bochs and QEMU have a limited selection of VGA and networking devices. Those devices that they do emulate may (though it is unlikely) have bugs, and not operate exactly like real devices (emulators sometimes zero memory, whereas real machines usually do not).
Advanced matters
Hard disk access testing
Testing hard disk access poses extra problems over the usual testing problems, because it means that it is no longer possible to use your main development platform as a testbed (you could, but it is generally a bad idea). However, if you really need to (because of a lack of emulator or second machine), here are some guidelines:
- Your kernel should be very stable, so that it is ensured that it will not write to random places in memory, not follow NULL pointers and other such things. Being in protected mode is an advantage here, because you cannot accidentally call BIOS code that might write to disk.
- Give your file system code a try on a ramdisk first. The ramdisk content could be constructed from scratch or generated from data's loaded by your bootloader (GRUB's module loading facility will be your friend here) Make sure you have the ability to check what has been written in the ramdisk without having to write the ramdisk to a real disk (otherwise you may as well have tried it on the real disk).
- When developing a component that will perform disk accesses, make sure you test it on an emulator first and that it behaves as expected. Enforce boundary checks in the disk functions so that you'll not getting out of a partition or things alike. You may even hardcode boundaries in your low-level driver and report a fatal condition if the upper component requires it to go out of the 'safe' array on the disk.
- Try to test drives that are on a separate controller (e.g. different IDE cable) from your main disk(s)), so that your code will access I/O ports that couldn't harm your main disk. If all else fails, test it on a dedicated second computer.
Finding specific hardware
If you need to write drivers for a specific piece of hardware, there are several options. Older hardware devices may be available on eBay (and other similar places) if you do not already own them. You might also like to try this forum thread, where many users have listed some of their computers for testing other user's operating systems. The vast majority of hardware listed in the hardware category is fairly common, and would be a good choice to develop for.
Unit Testing
- Main article: Unit Testing
The above article discusses a method of testing operating system components individually, and putting in place a test driver that makes sure the components always work as expected (for instance, fixing one bug does not introduce another).