Why do I need a Cross Compiler?
- Notice: This page is specific to GCC. If you use another compiler, you should research how cross-compilation is normally done with that compiler and do it that way. GCC is quite tightly bound to its native target system, many other compilers are not. Some compilers don't even have a native target, they are always cross-compilers.
You need to use a cross-compiler unless you are developing on your own operating system. The compiler must know the correct target platform (CPU, operating system), otherwise you will run into trouble. You may be able to use the compiler that comes with your system if you pass a number of options to beat it into submission, but this will create a lot of completely unnecessary problems.
It is possible ask your compiler what target platform it is currently using by calling the command:
gcc -dumpmachine
If you are developing on 64-bit Linux, then you will get a response such as 'x86_64-unknown-linux-gnu'. This means that the compiler thinks it is creating code for Linux. If you use this GCC to build your kernel, it will use your system libraries, headers, the Linux libgcc, and it will make a lot of problematic Linux assumptions. If you use a cross-compiler such as i686-elf-gcc, then you get a response back such as 'i686-elf' that means the compiler knows it is doing something else and you can avoid a lot of problems easily and properly.
How to build a Cross-Compiler
- Main article: GCC Cross Compiler
It is easy and takes a few moments to build a cross-compiler that targets your operating system. It may take a while to build it on slower computers, but you only need to do it once, and you save all the time you would otherwise spend on "fixing" the completely unnecessary problems you would encounter otherwise. Later on, when you start building a user-space for your operating system, it is worth creating an OS Specific Toolchain for absolute control of the compiler and to easy compiling user-space programs.
Transitioning to a Cross-Compiler
Perhaps you have not been using a cross-compiler until now, in which case you are likely doing a lot of things wrong. Unfortunately, a lot of kernel tutorials suggest passing certain options and doing things in a manner that potentially causes a lot of trouble. This section documents some of the things you should watch out for. Please read this section carefully and point others to it if you see them using troublesome options.
Linking with your compiler rather than ld
You shouldn't be invoking ld directly. Your cross-compiler is able to work as a linker and using it as the linker allows it control at the linking stage. This control includes expanding the -lgcc to the full path of libgcc that only the compiler knows about. If you get weird errors during compilation, use your cross-compiler for linking and it may go away. If you do need ld, be sure to use the cross-linker (i686-elf-ld) rather than the system linker.
Using cross-tools
You get a lot of useful programs when you build your cross-binutils. For instance, you get i686-elf-readelf, i686-elf-as, i686-elf-objdump, i686-elf-objcopy, and more. These programs know about your operating system and handle everything correctly. You can use some of the programs that come with your local operating system instead (readelf, objcopy, objdump) if they know about the file format of your operating system, but it is in general best to use your cross tools instead. These tools all consistently have the prefix 'i686-elf-' if the platform of your OS is i686-elf.
Options that you should pass to your Compiler
You need to pass some special options to your compiler to tell it it isn't building user-space programs.
-ffreestanding
This is important as it lets the compiler know it is building a kernel rather than user-space program. The documentation for GCC says you are required to implement the functions memset, memcpy, memcmp and memmove yourself in freestanding mode.
-mno-red-zone (x86_64 only)
You need to pass this on x86_64 or interrupts will corrupt the stack. The red zone is a x86_64 ABI feature that means that signals happen 128 bytes further down the stack. Functions that use less than that amount of memory is allowed to not increment the stack pointer. This means that CPU interrupts in the kernel will corrupt the stack. Be sure to pass enable this for all x86_64 kernel code.
-fno-exceptions, -fno-rtti (C++)
It is wise to disable C++ features that don't work out-of-the-box in kernels. You need to supply a C++ support library to the kernel (in addition to libgcc) to make all C++ features work. If you don't use these C++ features, it should be sufficient to pass these options.
Options you should link with
These options only make sense when linking (not when compiling) and you should use them. You should pass the compilation options as well when linking, as some compilation options (such as -mno-red-zone) control the ABI and this needs to be known at link time as well.
-nostdlib (same as both -nostartfiles -nodefaultlibs)
The -nostdlib option is the same as passing both the -nostartfiles -nodefaultlibs options. You don't want the start files (crt0.o, crti.o, crtn.o) in the kernel as they only used for user-space programs. You don't want the default libraries such as libc, because the user-space versions are not suitable for kernel use. You should only pass -nostdlib, as it is the same as passing the two latter options.
-lgcc
You disable the important libgcc library when you pass -nodefaultlibs (implied by -nostdlib). The compiler needs this library for many operations that it cannot do itself or that is more efficient to put into a shared function. You must pass this library at the end of the link line, after all the other object files and libraries, or the linker won't use it and you get strange linker errors. This is due to the classic static linking model where an object file from a static library is only pulled in if it is used by a previous object file. Linking with libgcc must come after all the object files that might use it.
Options that you shouldn't pass to your Compiler
There is a number of options you normally shouldn't pass to your cross-compler when building a kernel. Unfortunately, a lot of kernel tutorials suggest you use these. Please do not pass a option without understanding why it is needed and don't suggest to people that they use them. Often, these options are used by those that don't use cross-compilers to cover up other problems.
-m32, -m64 (compiler)
If you build a cross-compiler such as i686-elf-gcc, then you don't need to tell it to make a 32-bit executable. Likewise, you don't need to pass -m64 to x86_64-elf-gcc. This will make your Makefiles much simpler as you can simply select the correct compiler and things will work. You can use x86_64-elf-gcc to build a 32-bit kernel, but it's much easier to just build two cross-compilers and use them. In addition, using a cross-compiler for every CPU you target will make it easy to port third-party software without tricking them into passing -m32 as well.
-melf_i386, -melf_x86_64 (linker)
You don't need to pass these for the same reason as -m32 and -m64. Additionally, these options are for ld, and you shouldn't be invoking ld directly in the first place, but rather linking with your cross-compiler.
-32, -64 (assembler)
The cross-assembler (i686-elf-as) defaults to the platform you specified when building binutils, and so you don't need to repeat the choice here. You can use the cross-compiler as an assembler, but it is okay to call the assembler directly.
-nostdinc
You shouldn't pass this option as it disables the standard header include directories. However, you do want to use these headers as they contain many useful declarations. The cross-compiler comes with a bunch of useful headers such as stddef.h, stdint.h, stdarg.h, and more.
If you don't use a cross-compiler, you get the headers for your host platform (such as Linux) which are unsuitable for your operating system. For that reason, most people that don't use a cross-compiler use this option and then have to reimplement stddef.h, stdint.h, stdarg.h and more themselves. People often implement those files incorrectly as you need compiler magic to implement features such as stdarg.h.
-fno-builtin
This option is implied by -ffreestanding and there is no reason to pass it yourself. The compiler defaults to -fbuiltin that enables builtins, but -fno-builtin disables them. Builtins mean that the compiler knows about standard features and can optimize their use. If the compiler sees a function called 'strlen', it normally assumes it is the C standard 'strlen' function and it is able to optimize the expression strlen("foo") into 3 at compile time, instead of calling the function. This option has value if you are creating some really non-standard environment in which common C functions don't have their usual semantics. It is possible to enable builtins again with -fbuiltin following -ffreestanding but this can lead to surprising problems down the road, such as the implementation of calloc (malloc + memset) being optimized into a call to calloc itself.
-fno-stack-protector
The Stack Smashing Protector is a feature that stores a random value on the stack of selected functions and verifies the value is intact upon return. This statistically prevents stack buffer overflows overwriting the return pointer on the stack, which would subvert control flow. Adversaries are often able to exploit such faults, and this feature requires the adversary to correctly guess a 32-bit value (32-bit systems) or a 64-bit value (64-bit systems). This security feature requires runtime support. Compilers from many operating system vendors enable this feature by having -fstack-protector be the default. This breaks kernels that don't use a cross-compiler, if they don't have the runtime support. Cross-compilers such as the *-elf targets have the stack protector disabled by default and there's no reason to disable it yourself. You may want to change the default to enabling it when you add support for it to your kernel (and user-space), which would make it automatically used by your kernel because you didn't pass this option.
Problems that occur without a Cross-Compiler
You need to overcome a lot of problems to use your system gcc to build your kernel. You don't need to deal with these problems if you use a cross-compiler.
More complicated compilation commands
The compiler assumes it is targetting your local system, so you need a lot of options to make it behave. A trimmed down command sequence for compiling a kernel without a cross-compiler could look like this:
as -32 boot.s -o boot.o
gcc -m32 kernel.c -o kernel.o -ffreestanding -nostdinc
gcc -m32 my-libgcc-reimplemenation.c -o my-libgcc-reimplemenation.o -ffreestanding
gcc -m32 -T link.ld boot.o kernel.o my-libgcc-reimplemenation.o -o kernel.bin -nostdlib -ffreestanding
Actually, the average case is worse. People tend to add many more problematic or redundant options. With a real cross-compiler, the command sequence could look this this:
i686-elf-as boot.s -o boot.o
i686-elf-gcc kernel.c -o kernel.o -ffreestanding
i686-elf-gcc -T link.ld boot.o kernel.o -o kernel.bin -nostdlib -ffreestanding -lgcc
Reimplementing libgcc
You cannot use the host libgcc when building a kernel. The Linux libgcc has some nasty dependencies last timed I checked. The common case newbies run into is 64-bit integer division on 32-bit systems, but the compiler may generate such calls in many cases. You will often end up rewriting libgcc when you should have been using the real thing in the first place.
Rewriting freestanding headers (often incorrectly)
If you don't pass -nostdinc you get the target system headers (which is your local system if not using a cross-compiler), and that will cause a lot of problems in the non-cross-compiler case. You will end up rewriting the standard freestanding headers such as stdarg.h, stddef.h, stdint.h. and more. Unfortunately, as mentioned above, these headers need a bit of compiler magic to get just right. If you use a cross-compiler, all these freestanding headers can be used out of the box with no effort.
Complicated compiling user-space programs
You need to pass even more options to the command lines that build programs for your operating systems. You need a -Ipath/to/myos/include and -Lpath/to/myos/lib to use the C library, and more. If you set up an OS Specific Toolchain, you just need
i686-myos-gcc hello.c -o hello
to cross-compile the hello world program to your operating system.
Compiler releases break your OS
Not everyone is using the same gcc as you are, which means that people on other operating systems (even versions, or compiler releases) will have trouble building your operating system correctly. If you use a cross-compiler, then everyone is using the same compiler version and assumptions about the host system won't make it into your operating system.
Support
You will have a much easier time getting support from the operating system development community. Properly using a cross-compiler shows that you have followed instructions and are at the same level as everyone else, and that your local system compiler isn't causing trouble.
And so on
As the project grows in size, it becomes much more complicated to maintain your operating system without a real cross-compiler. Even if your ABI is very much like Linux, your operating system isn't Linux. Porting third party software is near impossible without a cross-compiler. If you set up a real OS Specific Toolchain and a sysroot of your OS, you can compile software just by giving --host=i686-myos to ./configure. With a cross-compiler you can port software in the standard manner.
Background information
Where did the idea of cross compiling come from?
With GNU software, since most of it has a long history of being ported from one UNIX implementation to another, they came up with a simple set of rules: You have a build machine, a host machine, and a target machine.
What are the basics of cross compiling?
The "build" machine is the machine you're compiling the software on. This software being compiled may be compiled to run on some other type of machine. See, you may be building on an x86-based machine, and wishing for the software to run on a SPARC based machine. The build machine is implicit and will usually be auto-detected by the configure script for the software. Its only real purpose is so that, if the software being compiled chooses to keep the configure arguments used to configure it somewhere in the built package, the people to whom the package is distributed will know what machine the package was built on. The name of the build machine may be used to configure the package to use workarounds as well if the build machine is universally known to have certain problems building that software.
The "host" machine is the machine on which the software must run. So in the previous example, the "build" machine is an i686-elf-yourBuildOs machine, and the host is a sparc32-elf-unix4 machine.
In the example, you must then, have a sparc32-elf-unix4 cross compiler, which can run on the i686-elf-yourBuildOs machine, and spit out (target) your host machine. A cross compiler is usually named after the host it targets, and not after the host it runs on, so by looking at the name of a compiler, you can usually tell what machine it targets.
The "target" only matters when compiling software that is used to build other software. That is, when compiling compilers, linkers, assemblers and the like. They need to be told what machine they will themselves target. When compiling something like a movie player, or other non-building software, the "target" does not matter. In other words, "target" is used to tell a compiler or other development software being compiled what host it, the compiler being compiled, should target.
For most software, like a text editor, if you're compiling it, you only need to specify host (and usually not even that). Specifying a host causes the software to be built using a compiler on the build machine which is a cross compiler that targets that host. This way, since the software was compiled using that host's targeting cross compiler, the software will be able to run on that host, even though it was built on a (potentially) different built machine.
An example to concrete things
Your distribution comes equipped with a compiler that targets yourMachineArch-distributionNativeExecFormat-distribution, and that runs on the same yourMachineArch-distributionNativeExecFormat-distribution machine. It was built on some machine (could have been built on m68k-macho-osx or some other colourful machine like that), with a cross compiler that targeted yourMachineArch-distributionNativeExecFormat-distribution, and while it was being built, its own "target" was set to yourMachineArch-distributionNativeExecFormat-distribution, so that it now both runs on, and spits out executables for yourMachineArch-distributionNativeExecFormat-distribution.
A cross compiler is useful for completely unambiguously targeting a specific machine. When you begin developing for a machine separate from your native distribution's stock compiler, you'll need a cross compiler. Actually, you already *do* target a machine that's different from your native machine from the very first time you begin trying to produce completely standalone executables (that is, when you're doing OSDev).
Your native compiler targets yourMachineArch-distributionNativeExecFormat-distribution. But you want to target yourOsTargetArch-yourChosenFormat-none; where "none" as the OS-name represents a machine with no system libraries. When you compile your kernel, you want to build with a compiler that does not know of any system libraries for your target machine. The kernel must be standalone completely. Everything the kernel needs must be in its source tree so that no implicit system libraries are needed (or linked in without your knowledge).
Later on, when you have a userspace and a set of system libraries for your programs to link to, you'll then build a compiler that targets: yourOsTargetArch-yourChosenFormat-yourOs while keeping the compiler that targets yourOsTargetArch-yourChosenFormat-none. The difference between the two is that the "none" OS targeting compiler does not have any libraries in its /lib directory, and has no system libraries to link in. It is therefore impossible for any software you compile using this "bare metal" targeting compiler to have unknown dependencies. The "yourOS" targeting cross compiler is where you would place your own system libraries, (in its /lib directory), such that when it builds software, it will link them against those system libraries, thus linking them against your kernel API. You would continue to use the "bare bones" targeting compiler to build the kernel, and whenever you build programs on your development machine which are expected to run on your kernel's target machine, you would use the yourOsTargetArch-yourChosenFormat-yourOs compiler, which has a /include and /lib directories full of your own OS's native system includes and libraries.
Later still, when you're confident that you can both develop and use your kernel, from within your kernel, you'll want to stop building all your programs on a separate "development" machine. More specifically, at this point, you want to have your "build" machine be the same as your test machine such that you build programs that run on your OS, from within your OS. At this point, it's time to take the final step, and, from your separate "development" machine, build a compiler that will run on your OS (host = yourOsTargetArch-yourChosenFormat-yourOs), and target your OS (target = yourOsTargetArch-yourChosenFormat-yourOs). This cross compiler, the canadian cross, will allow you to natively compile other programs while running on your own OS's userspace. It is essentially a "distribution native compiler", like the one that comes with your distro. From that point on, having a compiler that runs on and targets your OS, you can essentially more freely port programs, and allow people to do the same while running your OS, assuming you package that native compiler with your OS.
When do I not need a cross-compiler?
If you create a real operating system and manage to port gcc to it, that gcc will produce the very same code as i686-myos-gcc. That means that you don't need a cross-compiler on your own operating system, because the gcc there will already do the right thing. This is why the Linux kernel is built with the Linux gcc, instead of a Linux cross-compiler.
A concrete example
- gcc -v* from my Ubuntu machine gives:
gravaera@gravaera-laptop:/void/cygwin/osdev/zbz$ gcc -v Using built-in specs. Target: i486-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.4.3-4ubuntu5' \ --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ \ --prefix=/usr --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib \ --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 \ --program-suffix=-4.4 --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-plugin --enable-objc-gc \ --enable-targets=all --disable-werror --with-arch-32=i486 --with-tune=generic --enable-checking=release \ --build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu Thread model: posix gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
The person who built the compiler that runs on my machine built it on a machine just like mine, an 'i486-linux-gnu' (build), and intended for it to run on a machine like his/mine: the same i486-linux-gnu (host), and he meant for this compiler he was building for me, to emit executables that targeted my very machine, so that when I compile programs, they will be able to run on a host that is i486-linux-gnu. Therefore he made the compiler target i486-linux-gnu.