ELF (Executable and Linkable Format) was designed by Unix System Laboratories while working with Sun Microsystems on SVR4 (UNIX System V Release 4.0). Consequently, ELF first appeared in Solaris 2.0 (aka SunOS 5.0), which is based on SVR4. The format is specified in the System V ABI.
A very versatile file format, it was later picked up by many other operating systems for use as both executable files and as shared library files. It does distinguish between TEXT, DATA and BSS.
Today, ELF is considered the standard format on Unix-alike systems. While it has some drawbacks (e.g., using up one of the scarce general purpose registers of the IA-32 when using position-independent code), it is well supported and documented.
ELF is a format for storing programs or fragments of programs on disk, created as a result of compiling and linking. An ELF file is divided into sections. For an executable program, these are the text section for the code, the data section for global variables and the rodata section that usually contains constant strings. The ELF file contains headers that describe how these sections should be stored in memory.
Note that depending on whether your file is a linkable or an executable file, the headers in the ELF file won't be the same: process.o, result of gcc -c process.c $SOME_FLAGS
C32/kernel/bin/.process.o architecture: i386, flags 0x00000011: HAS_RELOC, HAS_SYMS start address 0x00000000 Sections: Idx Name Size VMA LMA File off Algn 0 .text 00000333 00000000 00000000 00000040 2**4 CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE 1 .data 00000050 00000000 00000000 00000380 2**5 CONTENTS, ALLOC, LOAD, DATA 2 .bss 00000000 00000000 00000000 000003d0 2**2 ALLOC 3 .note 00000014 00000000 00000000 000003d0 2**0 CONTENTS, READONLY 4 .stab 000020e8 00000000 00000000 000003e4 2**2 CONTENTS, RELOC, READONLY, DEBUGGING 5 .stabstr 00008f17 00000000 00000000 000024cc 2**0 CONTENTS, READONLY, DEBUGGING 6 .rodata 000001e4 00000000 00000000 0000b400 2**5 CONTENTS, ALLOC, LOAD, READONLY, DATA 7 .comment 00000023 00000000 00000000 0000b5e4 2**0 CONTENTS, READONLY
The 'flags' will tell you what's actually available in the ELF file. Here, we have symbol tables and relocation: all that we need to link the file against another, but virtually no information about how to load the file in memory (even if that could be guessed). We don't have the program entry point, for instance, and we have a sections table rather than a program header.
|.text||where code stands, as said above. objdump -drS .process.o will show you that|
|.data||where global tables, variables, etc. stand. objdump -s -j .data .process.o will hexdump it.|
|.bss||don't look for bits of .bss in your file: there's none. That's where your uninitialized arrays and variable are, and the loader 'knows' they should be filled with zeroes ... there's no point storing more zeroes on your disk than there already are, is it?|
|.rodata||that's where your strings go, usually the things you forgot when linking and that cause your kernel not to work. objdump -s -j .rodata .process.o will hexdump it. Note that depending on the compiler, you may have more sections like this.|
|.comment & .note||just comments put there by the compiler/linker toolchain|
|.stab & .stabstr||debugging symbols & similar information.|
/bin/bash, a real executable file
/bin/bash: file format elf32-i386 /bin/bash architecture: i386, flags 0x00000112: EXEC_P, HAS_SYMS, D_PAGED start address 0x08056c40 Program Header: PHDR off 0x00000034 vaddr 0x08048034 paddr 0x08048034 align 2**2 filesz 0x000000e0 memsz 0x000000e0 flags r-x
The program header itself... taking 224 bytes, and starting at offset 0x34 in the file
INTERP off 0x00000114 vaddr 0x08048114 paddr 0x08048114 align 2**0 filesz 0x00000013 memsz 0x00000013 flags r--
The program that should be used to 'execute' the binary. Here, it reads as '/lib/ld-linux.so.2', which means some dynamic libraries linking will be required before we run the program.
LOAD off 0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12 filesz 0x0007411c memsz 0x0007411c flags r-x
Now we're requested to read 7411c bytes, starting at file's start (?) and being 7411c bytes large (that's virtually the whole file!), which will be read-only but executable. They'll be to appear starting at virtual address 0x08048000 for the program to work properly.
LOAD off 0x00074120 vaddr 0x080bd120 paddr 0x080bd120 align 2**12 filesz 0x000022ac memsz 0x000082d0 flags rw-
More bits to load, (likely to be .data section). Notice that the 'filesize' and 'memsize' differ, which means the .bss section will actually be allocated through this statement, but left as zeroes while 'real' data only occupy first 0x22ac bytes starting at virtual address 0x80bd120.
DYNAMIC off 0x00075f4c vaddr 0x080bef4c paddr 0x080bef4c align 2**2 filesz 0x000000e8 memsz 0x000000e8 flags rw-
The dynamic sections are used to store information used in the dynamic linking process, such as required libraries and relocation entries.
NOTE off 0x00000128 vaddr 0x08048128 paddr 0x08048128 align 2**2 filesz 0x00000020 memsz 0x00000020 flags r--
NOTE sections contain information left by either the programmer or the linker, for most programs linked using the GNU 'ld' linker it just says 'GNU'
EH_FRAME off 0x000740f0 vaddr 0x080bc0f0 paddr 0x080bc0f0 align 2**2 filesz 0x0000002c memsz 0x0000002c flags r--
that's for Exception Handler information, in case we should link against some C++ binaries at execution (afaik).
/bin/bash, loaded (as in /proc/xxxx/maps) 08048000-080bd000 r-xp 00000000 03:06 30574 /bin/bash 080bd000-080c0000 rw-p 00074000 03:06 30574 /bin/bash 080c0000-08103000 rwxp 00000000 00:00 0 40000000-40014000 r-xp 00000000 03:06 27304 /lib/ld-2.3.2.so 40014000-40015000 rw-p 00013000 03:06 27304 /lib/ld-2.3.2.so
We can recognize our 'code bits' and 'data bits', by stating that the second one should be loaded at 0x080bd*120* and that it starts in file at 0x00074*120*, we actually preserved page-to-disk blocks mapping (e.g. if page 0x80bc000 is missing, just fetch file blocks from 0x75000). That means, however, that a part of the code is mapped twice, but with different permissions. I suggest you do give them different physical pages too if you don't want to end up with modifiable code.
Loading ELF Binaries
The ELF file format is described in the ELF Specification. The most relevant sections for this project are 1.1 to 1.4 and 2.1 to 2.7.
The steps involved in identifying the sections of the ELF file are:
- Read the ELF Header. The ELF header will always be at the very beginning of an ELF file. The ELF header contains information about how the rest of the file is laid out. You are interested only in the program headers.
- Find the Program Headers, which specify where in the file to find the text and data sections and where they should end up in the executable image.
There are a few simplifying assumptions you can make about the types and location of program headers. In the files you will be working with, there will always be one text header and one data header. The text header will be the first program header and the data header will be the second program header. This is not generally true of ELF files, but it will be true of the programs you will be responsible for.
The file geekos/include/geekos/elf.h provides data types for structures which match the format of the ELF and program headers.
This is a rough guideline for what Parse_ELF_Executable() has to do:
- Check that exeFileData is non-null and exeFileLength is large enough to accommodate the ELF headers and phnum program headers.
- Check that the file starts with the ELF magic number (4 bytes) as described in figure 1-4 (and subsequent table) on page 11 in the ELF specification.
- Check that the ELF file has no more than EXE_MAX_SEGMENTS program headers (phnum field of the elfHeader).
- Fill in numSegments and entryAddr fields of the exeFormat output variable.
- For each program header k in turn, fill in the corresponding segmentList[k] array element of exeFormat with offsetInFile, lengthInFile, startAddress, sizeInMemory, protFlags with information from that program header k. See figure 2-1 on page 33 in the ELF specification.
Relocation becomes handy when you need to load, for example, modules or drivers. It's possible to use the "-r" option to ld to permit you to have multiple object files linked into one big one, which means easier coding and faster testing.
The basic outline of things you need to do for relocation:
- Check the object file header (it has to be ELF, not PE, for example)
- Get a load address (eg. all drivers start at 0xA0000000, need some method of keeping track of driver locations)
- Allocate enough space for all program sections (ST_PROGBITS)
- Copy from the image in RAM to the allocated space
- Go through all sections resolving external references against the kernel symbol table
- If all succeeded, you can use the "e_entry" field of the header as the offset from the load address to call the entry point (if one was specified), or do a symbol lookup, or just return a success error code.
Once you can relocate ELF objects you'll be able to have drivers loaded when needed instead of at startup - which is always a Good Thing (tm).
The header is found at the start of the ELF file.
|Position (32 bit)||Position (64 bit)||Value|
|0-3||0-3||Magic number - 0x7F, then 'ELF' in ASCII|
|4||4||1 = 32 bit, 2 = 64 bit|
|5||5||1 = little endian, 2 = big endian|
|7||7||OS ABI - usually 0 for System V|
|16-17||16-17||1 = relocatable, 2 = executable, 3 = shared, 4 = core|
|18-19||18-19||Instruction set - see table below|
|24-27||24-31||Program entry position|
|28-31||32-39||Program header table position|
|32-35||40-47||Section header table position|
|36-39||48-51||Flags - architecture dependent; see note below|
|42-43||54-55||Size of an entry in the program header table|
|44-45||56-57||Number of entries in the program header table|
|46-47||58-59||Size of an entry in the section header table|
|48-49||60-61||Number of entries in the section header table|
|50-51||62-63||Index in section header table with the section names|
The flags entry can probably be ignored for x86 ELFs, as no flags are actually defined.
Instruction Set Architectures:
The most common architectures are in bold.
This is an array of N (given in the main header) entries in the following format. Make sure to use the correct version depending on whether the file is 32 bit or 64 bit as the tables are quite different.
32 bit version:
|0-3||Type of segment (see below)|
|4-7||The offset in the file that the data for this segment can be found (p_offset)|
|8-11||Where you should start to put this segment in virtual memory (p_vaddr)|
|12-15||Undefined for the System V ABI|
|16-19||Size of the segment in the file (p_filesz)|
|20-23||Size of the segment in memory (p_memsz)|
|24-27||Flags (see below)|
|28-31||The required alignment for this section (must be a power of 2)|
64 bit version:
|0-3||Type of segment (see below)|
|4-7||Flags (see below)|
|8-15||The offset in the file that the data for this segment can be found (p_offset)|
|16-23||Where you should start to put this segment in virtual memory (p_vaddr)|
|24-31||Undefined for the System V ABI|
|32-39||Size of the segment in the file (p_filesz)|
|40-47||Size of the segment in memory (p_memsz)|
|48-55||The required alignment for this section (must be a power of 2)|
Segment types: 0 = null - ignore the entry; 1 = load - clear p_memsz bytes at p_vaddr to 0, then copy p_filesz bytes from p_offset to p_vaddr; 2 = dynamic - requires dynamic linking; 3 = interp - contains a file path to an executable to use as an interpreter for the following segment; 4 = note section. There are more values, but mostly contain architecture/environment specific information, which is probably not required for the majority of ELF files.
Flags: 1 = executable, 2 = writable, 4 = readable.
Using the PIC with programs
The logic that will allow an ELF program to run (which is quite simple once you have a scheduler) is this: *IRQ fires*->Scheduler->ELF Program on Queue->Run ELF Program until an exit() is called (usually in crt0)->Take process off the Queue
Dynamic Linking is when the OS gives a program shared libraries if it needs them. Meaning, the libraries are found in the system and then "bind" to the program that needs them while the program is running, versus static linking, which links the libraries before the program is run. The main advantages are that programs take up less memory, and are smaller in file size. The main disadvantage, however, is that the program becomes less portable because the program depends on many different shared libraries.
In order to implement this, you need to have proper scheduling in place, a library, and a program to use that library. You can create a library with GCC:
myos-gcc -c -fPIC -o oneobject.o oneobject.c myos-gcc -c -fPIC -o anotherobject.o anotherobject.c myos-gcc -shared -fPIC -Wl,-soname,nameofmylib oneobject.o anotherobject.o -o mylib.so
This library should be treated as a file, which is loaded when the OS detects its attempted usage. You will need to implement this "Dynamic Linker" into a certain classification of code such as in your memory management or your task management section. When the ELF program is run, the system should attach the shared object data to a malloc() region of memory, where the function calls to the libraries redirect to that malloc() region of memory. Once the program is finished, the region can be given up back to the OS with a call to free().
That should be a good starting point to writing a dynamic linker.
- The ELF file format in detail
- ELF Format Specifications Detailed and up-to-date ELF information (including SPARC in depth) by Oracle.
- System V ABI about ELF
- LSB specifications
See (generic or platform-specific) 'Core' specifications for additional ELF information.
- Executable and Linkable Format on Wikipedia,which contains a detail of elf references
- The ELF file format(64-bit) ELF 64-Bit, General extension to ELF32.
- x86-64 ABI Documented x86-64 specific extensions with ELF64.
- Manually Creating an ELF Executable Detailed guide on how to create ELF binaries from scratch.