Getting Started

First of all, developing an operating system is probably one of the most challenging things you can do on a computer (next to killing the final boss in Doom on Nightmare difficulty level). Composing an operating system requires a lot of knowledge about several complex areas within computer science. You need to understand how hardware works and be able to read and write the complex assembly language as well as a higher-level language (such as C, C++, or Pascal). Your mind has to be able to wrap itself around abstract theory and hold a myriad of thoughts. Feel discouraged yet? Don't fear! Because all of these things are also the things that makes OS programming fun and entertaining.

There is nothing like the feeling of accomplishment when you, finally, after hours of struggling, solve the problem. And after some time you are able to look back and see all of the things you've created from scratch. Your handwritten system is able to boot, performs magic against the hardware, and gives the user a user interface and programs to play with.

There is no absolute path you have to take when creating an OS. Once you get your initial system up and running (and you do this by finding appropriate tutorials), you choose the path you want to take next. Your OS is exactly that--yours. You have ultimate control, and the sky's the limit!

The Hard Truth

Hopefully the basic fact that operating system development is a complicated and ongoing process does not discourage you. The truth is, operating system development is truly unparalleled since it requires the utmost amount of patience and careful code design, and it returns very little to no "instant gratification" you get from the development of things like games and web-based scripting.

You have been fairly warned of the hard work ahead, but if you are still interested then proceed forward into the realm of the operating system programmer. Prepare yourself for occasional bouts of confusion, discouragement, and for some of us...temporary insanity. In time, and with enough dedication, you will find yourself among the elite few who have contributed to a working operating system. If you do get discouraged along the way, refresh yourself with the content of this book. Hopefully it will remind you why you started such an insane journey in the first place.

At this stage, it would also pay to read the Beginner Mistakes page. Users on the forum have noticed a lot of these mistakes getting repeated over time, and avoiding them is a great way to not make a fool of yourself.

Responsibility

People tend to claim that it is OK to write inefficient software, stating that computer systems are so fast these days, that you won't see the impact. This type of mentality is dangerous in operating system design. It might be OK to write sloppy code when making a simple application, but when it comes to critical code that may get called thousands of times per second, you need to take out all the overhead you can. The operating system should supply the computer as a basic resource to the running applications, with as little complication, abstraction, and overhead as possible.

People who design operating systems in this day and age tend to have the "everything but the kitchen sink" mentality. They take it upon themselves to account for everything, which of course is good, but it shouldn't be done at the expense of allowing poorly-written programs to flourish. There are many things that go on "under the hood" when program errors occur. Poorly-written programs cost precious execution time and involve task switches that are expensive in both memory and frequency. We encourage you to discourage poorly-written software.

Required Knowledge

Main article: Required Knowledge

If you think you can skip this, it's just for you.

This section has been moved to a separate page because it is referred to so often in forum discussions.

Organize your plans

Before proceeding, consider what it is you want to get out of writing an operating system. What are your motivations in taking on this project? There are many possible reasons for taking on a hobby OS project, and most os-devers have more than one. Even just saying, "I just want to" can be enough, though the more you consider and clarify your goals and motives, the more you can focus on what you really want.

Be honest with yourself, as well. There's no shame in having larger ambitions for your project, even (or especially) if they aren't the primary objective. Try to acknowledge all of your goals, not just the one you think is your main purpose.

Try to settle on which aspects of OS design you are most interested in or see a need to work on. Most of what goes into OS dev, especially early on, is kernel design and development, but the kernel itself is only a small part of most operating systems; if your primary interest is in UX, or networking, or driver programming, you should think about whether you really need (now or in the future) to write your own OS at all or if you would be just as satisfied developing those things on an existing kernel. More than a few people have gone into OS dev when they really wanted to design a desktop environment, so this is a very important question to ask yourself.

Try to think of any non-OS projects you might want to take on first, or at the same time, especially ones which might serve as practice or preparation for the OS project. There's usually no need to work on the OS project right now, and the more you have prepared ahead of time, the better off you'll be (up to a point, at least--preparation is one thing, procrastination is something else).

Similarly, if you mean to work on forking an existing design to experiment with, or to modify for some specific purpose, focus on that rather than general development issues. Consider what part of the existing code base you will need, and which ones you want to change.

Try to work out some of your specific project goals, and be prepared to plan out separate projects if it helps to do so. If you are simply intending to putter around and see where it takes you, that's fine; if your intent is to overthrow Microsoft, that's fine (if probably unrealistic), too. Once you know what you want to do, you can break down the details of it into specific objectives, and work out what it would take to reach them. Don't try to force too many divergent objectives into one project--if you have different things you want to try with contradictory goals, split them into different projects.

It may help if you write out an overview of your planned OS design, with any specific requirements or details you feel are notable or which could clarify what you need help with, and add it to your public repository if you can. This not only will make it easier for others to help you, it will help organize and stabilize your plans, much like writing an outline for a story or paper. Be prepared to maintain it as your goals and plans change, but keep a copy of older versions (or better still, keep the document under version control) so you can see how your work develops over time.

Finally, review the time and resources which the project will require, and decide if they are feasible. If you know that you only have a certain amount of time to devote to the project, take that into account, and whatever you do, don't commit to an outside deadline even if you are certain you can reach it. OS dev takes time--a lot of time--and trying to finish a full OS project in a semester isn't realistic.

Choosing your development environment

You need a platform to develop your new system on. Following the trends of general computing, the most popular is GNU/Linux, but many use Windows too. Developers using a GNU/Linux system have a slight advantage in availability of tools, but this can be solved on Windows using a system such as Cygwin or MinGW.

• Binutils: Fundamental tools for manipulation of object files.
• GCC: The GNU Compiler Collection. GCC contains compilers for C, C++, Fortran, and Ada, amongst others.
• Make: For automating the build process, which becomes really helpful once you have more than a handful of files.
• Grep and sed: For doing more powerful searches and search-and-replaces (helpful when filling out tables with data).
• Diffutils: Incredibly useful for showing the differences between two files.
• Perl or Python: One of these two scripting languages should be installed. Useful for string manipulation, amongst other things. Perl used to be the recommendation, but Python is now quite mature and is possibly easier to learn. Both have hundreds of packages/modules available for doing various tasks.
• An Assembler: For example NASM or GAS. This varies depending on your target CPU architecture.
• An editor: For writing your Assembly, C, and other (code) files.

You might not use all of these tools, but it is best to have them on hand "just in case," and know how to use them, even at a basic level. But if you decided to use another language then the tooling is mostly up to you and maybe the list above just won't help you in any way. Below is the information mostly related to the C/C++ or Assembly developers.

GNU/Linux

The most recommended system for OS development is GNU/Linux. When using GNU/Linux, most of the GNU development tools are probably already present. If not, use your distribution's package management tools (APT, RPM, Portage, Pacman, Apk, etc.) to install them as needed. Again, making a cross-compiler is required, so as not to link in the development system's runtime files.

Common editors are Vim, Emacs, KDevelop, Komodo Edit, etc. Some prefer lightweight editors instead of an IDE, such as gedit, Geany and SciTE. Many like Midnight Commander which has a Text UI and a built-in editor (mcedit) and therefore extremely lightweight and lightning fast.

About which distributions you should use, consult the list of Linux distributions. They come in all shapes and sizes, but as long as they're relatively general-purpose, they should be fine.

If you are unsure, try Ubuntu, Fedora or Linux Mint.

Windows

In order to get the tools necessary, you should install the Cygwin environment. MinGW or DJGPP are alternatives, but MSYS2 is strongly suggested as it is the most complete and compatible environment, and also includes a package manager to install libraries and tools.

Microsoft has recently (as of writing) released the Windows Subsystem for Linux as an optional feature for Windows 10. It is basically a real Ubuntu command line distribution running on top of Windows WITHOUT the use of a VM. The latest GCC and Binutils (6.1.0 and 2.27 as of writing) compile and work correctly in this environment. Using the Bash shell, you can access your Windows hard disks through /mnt/<drive letter>. The advantage of this solution is that you can work with whichever Windows or Linux tools that you require, without having to find out if they work in Cygwin. Many of the needed tools can be installed using "apt-get".

For all of the above, it is strongly suggested to build a cross-compiler, not only because the default compilers target different executable formats, but because it's generally a good idea. Check the GCC Cross-Compiler page for details and instructions.

You will also need an editor. Using Notepad will work, but it's easier if you have a more complete editor. For example, Notepad++ or Notepad2 are used by many people. If you are comfortable with Unix editors, you can pick one from the choice Cygwin provides (which includes e.g. Vim and Emacs, which take some getting-used-to but are very powerful).

It is also possible to use Visual Studio, or the freely downloadable Visual C++ Express Edition, to write and compile your operating system. You will require a special configuration file, and you will certainly be in the minority, but it does work quite well. You can even install the Windows SDK on top, enabling 64 bit development. The only pitfall is this doesn't support Inline Assembly.

Other tools such as Watcom or Borland can be used, too, but they each have specific requirements of their own, and are not widely used for this kind of work.

Another consideration is that you will probably have as a goal for your OS to be self-hosting, that is, you can compile your operating system using your operating system. If your OS is written in C, your minimal requirements will therefore be a C compiler and C library. If you intend for your OS to be a Windows clone rather than another POSIX-compliant OS, you will need a C library that does Windows calls instead of POSIX calls, and you will need a C compiler that uses just the C library instead of doing POSIX calls. GCCWIN + PDPCLIB fits this bill.

MacOS

Because under the hood it uses FreeBSD's userland, it is fully POSIX compatible. All the usual tools are available (vi, bash, dd, cat, sed, tar, cpio, etc.) Almost every tutorial works out-of-the-box. The missing tools are mostly file system related: no loopback device, no fdisk, no mkfs.vfat nor mtools. But you can use diskutil for these purposes, or use brew or macports to install those missing tools.

To get gcc, you used to have an mpkg on the 2nd Installation DVD for the older versions. Newer MacOS versions (10.13 and up) can install command line XCode (not the IDE, just the toolchain) by running "xcode-select --install" from a Terminal. This will install gcc, binutils and make. This gcc is actually a masquaraded CLang, but featurefull enough to build your own cross-compiler without problems. It is preferred to use the official compiler for bootstraping gcc than to install one from brew or macports.

Testing your operating system

Main article: Testing

The above article goes into a lot of depth about choosing how to test your operating system and how to integrate that with your development process. Both physical and emulated testing environments are discussed.

Protecting your code

During your code building you will write hundreds, even thousands, of lines of code. You'll spend an unmentionable number of hours, and sit up late at night coding when you really should go to bed. The last thing you need is a disk crash or a poorly written 'rm' or 'format' command throwing all your work away.

What you need is a version control system. CVS has been used for a number of years, but has gotten a lot of competition from Subversion, Bazaar, Mercurial, and Git lately. If you can, you should set up a remote computer or server as a version control server, but if you do not have such a machine available you can also host the version control system on your local development computer. Just remember to backup your code to CD or FTP once in a while.

We cannot stress this point strongly enough: if you are not using source control already, you should start doing so immediately. You only need make a serious mistake in your code once to realize the importance of having your code securely versioned and easily retrievable. While it may seem like overkill for a small, private hobby project, once you get into the habit of using revision control, you'll wonder how you ever did without it.

For Git you can create your project on GitHub or Bitbucket. Both come with free, private repositories.

An additional benefit of using version control on a network-accessible repository is that it makes it a lot easier to collaborate with and get help from others. This can be quite useful, especially in the forums, as it can avoid the need for constantly posting updated versions of your code to a message thread--you simply point the conversation towards your repository, and the others in the thread will have direct access to your most current changes. It is also crucial if, as the project grows, you begin to work with other developers on the project (just don't expect that to happen overnight).

Common starting points

The easiest way to get a "Hello World" 64-bit higher half kernel going is the Limine Bare Bones tutorial. A different approach would be to learn how the computer itself starts up, on the Boot Sequence page.

There are many other Tutorials available, too.

Obtaining further knowledge

There is an amazing amount of knowledge about operating system development available on the Internet today. It's just a matter of finding it. First of all, there is this wiki itself. Amongst others we have lots of Tutorials. Since you're here, you've probably already found it. Also on this site is the forum, where many developers hang out and can help you (but make sure you read How To Ask Questions first). Quite a few books on operating system development have been written. A number of these are featured on our Books page, and more over at osdever.net as well.

See Also

Articles

• Bare bones tutorials
• Resources
• What order should I make things in?

External Links

• Bona Fide OS Development contains several tutorials and documentation.
• Implementing an Operating System by Andrew S. Tanenbaum.