Libgcc without red zone
- Main article: Libgcc
This article discuses how to build libgcc without the need to support a red-zone in your kernel. If you're not targeting x86-64 you don't need this as i*86 has no such requirements.
What is the 'red-zone'?
The red-zone is a feature described in the x86-64 ABI.
It is a 128 byte long region located directly below the stack pointer. This region is free-for-use for the compiler without the requirement to notify the application / the os or any running interrupt handler.
For user applications there is no issue as interrupts and other kernel related code won't interfere with the user stack. In your kernel however things can get ugly, especially so if you have nested interrupts and no red-zone support. Imagine running inside your interrupt handler, gcc puts some data inside the red zone, a nested interrupt occurs and clobbers the red-zone or vice versa.
To get around this the red-zone can be disabled by passing -mno-red-zone to GCC.
x86_64-elf-gcc $CFLAGS -mno-red-zone ...
Why modify libgcc?
If you link against libgcc (as you should) there is one problem: libgcc is build with red-zone enabled.
So while your kernel works just fine the methods in libgcc may mess things up by accident. The solution is simple - rebuild libgcc with -mno-red-zone. Fortunately GCC supports this in a straight forward way by providing multilib support inside it's source tree.
Preparations
- Main article: GCC Cross-Compiler
Extract and prepare the GCC sources as described when building your GCC Cross-Compiler but don't run configure just yet.
Create the following file and save it as t-x86_64-elf inside gcc/config/i386/ under your GCC sources.
# Add libgcc multilib variant without red-zone requirement
MULTILIB_OPTIONS += mno-red-zone
MULTILIB_DIRNAMES += no-red-zone
By default this new configuration will not be used by GCC unless it's explicitly told to. Open gcc/config.gcc in your favorite editor and search for case block like this:
x86_64-*-elf*)
tm_file="${tm_file} i386/unix.h i386/att.h elfos.h newlib-stdint.h i386/i386elf.h i386/x86-64.h"
;;
This is the target configuration used when creating a GCC Cross-Compiler for x86_64-elf. Modify it to include the new multilib configuration:
x86_64-*-elf*)
tmake_file="${tmake_file} i386/t-x86_64-elf" # include the new multilib configuration
tm_file="${tm_file} i386/unix.h i386/att.h elfos.h newlib-stdint.h i386/i386elf.h i386/x86-64.h"
;;
Building libgcc
Run configure and then invoke the all-target-libgcc and install-target-libgcc as usual and GCC will build libgcc in two versions - one with red-zone enabled and one without. You can check the successful build by checking the installed libgcc.a archives:
find $TOOLCHAIN_PREFIX/lib -name 'libgcc.a'
If all went well you should see an additional libgcc installed in the no-red-zone multilib directory:
./gcc/x86_64-pc-elf/4.9.1/libgcc.a
./gcc/x86_64-pc-elf/4.9.1/no-red-zone/libgcc.a
Linking against the correct multilib version
Assuming you're using GCC to link your kernel you're probably fine. All that's needed is to make sure -mno-red-zone is in your LDFLAGS when doing the final linker call.
x86_64-elf-gcc $LDFLAGS -mno-red-zone -o kernel $SOURCES
If you're unsure which libgcc version is going to be used you can check by passing -mno-red-zone and -print-libgcc-file-name to GCC:
x86_64-elf-gcc -mno-red-zone -print-libgcc-file-name
lib/gcc/x86_64-pc-elf/4.9.1/no-red-zone/libgcc.a