Object Files

Object files basically consist of compiled and assembled code, data, and all the additional information necessary to make their content usable. In the process of building an operating system, you will use a lot of object files. While for common development tasks you do not need to know their exact details, when you want to create or use one with various specifics, the details can be very important.

Note: The term 'object file' has no relation to the high-level concept of 'object-oriented programming'. Object files predate the earliest forms of OOP (the 'Actors model', circa 1966) by over ten years, and the term was in use at IBM by 1958 or earlier.

Core Concepts

An object file is one of three types of files which contain object code, a modified form of machine code in which additional information that allows for linking and relocation of the final loaded executable is included.

For most purposes, a compiler or assembler will produce object code as its final result, rather than a true executable binary. While most assemblers, and some compilers, have an option to produce a raw binary image, this is usually only applied to boot loaders, Read-Only Memory chips, and other special-purpose executables. As a practical matter, virtually all systems today use both object files and relocatable executables. Even the simplest file format in regular use today, the MS-DOS .COM format, is not a pure binary executable image; the MS-DOS loader uses the first 0x100 bytes of the segment for the Program Segment Prefix, so that portion of the segment image is excluded in the file.

The majority of systems today have significantly more complex object formats, in which the address information is replaced with a stub or symbol of some kind, and which contain information regarding the relative location of externally visible functions, variables, and so forth. This facilitates the processes of

• linking, in which two or more object and/or library files are combined to form an executable file, and
• loading, where the address stubs are replaced by the loader with the actual memory locations that the code will reside within the process' memory.

Object files, executable files and library files

Whereas Wikipedia considers executables to be a subset of object files, on the basis that both contain object code rather than a binary image, there are significant differences. In some systems, they are a completely different format (COFF vs PE), or they have different fields (ELF program/section headers). The key difference is that in executables, the full object code of the program is present (save for whatever may be in shared libraries, as explained below), while object files are only the object code for the specific module that they were generated from. This means that non-executable files do not contain loadable code.

As stated earlier, this does not necessarily mean that the 'executable' file is the actual binary image that is executed; in most modern systems, that is produced in the loading step. In many cases, executable files still contain object code, not pure machine code, and the address locations may not be resolved until loading, but they do include all of the statically-linked code for the working program. Some linkers (such as ld, the Unix/Linux linker, which is invoked implicitly by GCC when generating an executable) have the option of - or even default to, as ld does - resolving the addresses at link time, but even in this case, the executable files generated usually contain additional information to facilitate the loading process - e.g., a separate read-only data section, a definition of the writable data area (sometimes called the .bss section), a section defining the stack area, etc. - and may have linkage information for using shared libraries.

A third type of object code file is a library file, a file that contains elements used by several programs, and made available for general use. Most functions, variables, and other elements used by the majority of programs are held in libraries. Libraries differ from regular object files mainly in that (on most systems) they are arranged so that independent elements of the library can be extracted from the file by the linker, so that only the elements used by the program are included in the executable generated from them.

On most systems today, libraries come in two types, static libraries, which are directly linked into into the executable file at linkage time, and shared libraries (also called dynamic-link libraries or DLLs in the Windows world), which are loaded and linked at run time to the programs which use them. The main difference is that shared libraries, as the name implies, can be shared by several programs at once, lowering memory usage. However, this comes at the cost of having to load the shared library in addition to the executable file when the first use of an element in them is made, and then linking it to the program(s) using them at run-time. Shared libraries are generally cached, to reduce the loading overhead, and usually are not loaded until the elements in them are actually used, meaning that if the part of the program using the shared library is not invoked, the library need not be loaded at all. Still, the trade-offs are such that code which isn't likely to be shared by several programs at once generally is linked as a static library instead, with only very common elements (e.g., the standard C and C++ libraries) being dynamically linked.

Relocations

A good part of the object file contains the code and its associated data. In source, code contains references to other functions and storage of data. In the object file such references are converted to instructions and relocation pairs, as the compiler can't tell in advance where the code will end up. As an example a function call on an x86 will look like this (in an object file):

 14:   e8 fc ff ff ff          call   15 <sprintf+0x15>
                       15: R_386_PC32  vsnprintf 

The disassembly contains the opcode for call (e8) plus the offset -4 (fc ff ff ff). If this were to be executed it would make a call to address 15, which looks like halfway through the instruction. The second line (the relocation entry) lists that the address at position 15 (the -4) should be fixed with a displacement to the address of vsnprintf. That means it should get the address of the called function minus the address of the relocation. However blankly entering the difference would not work, as the call address is relative to the next instruction, not the start of the offset bytes halfway into an opcode. This is where the -4 comes in: the result of the relocation is added to the field being padded. By subtracting 4 (adding -4) from the address, the displacement becomes relative to the end of the instruction, and the call ends up where it should go. In the executable file:

 804a1d4:       e8 07 00 00 00          call   804a1e0 <vsnprintf>
                        804a1d5: R_386_PC32     vsnprintf
 804a1d9:       c9                      leave

 (...)
0804a1e0 <vsnprintf>:

The displacement needed for the call is the address of vsnprintf minus the address of the next instruction, i.e. 0x804a1e0 - 0x804a1d9 = 0x7, which is the value seen in the call bytes (07 00 00 00). This is equivalent to the address of the target minus the address of the relocation plus the value stored: 0x804a1e0 - 0x804a1d5 + -4 = 0x7.

Relocating code

When an executable is created, it will be set to use a specific address by default. This can be a problem when you need several object files in the same address space and they may overlap, or you want to perform address space randomization, you might find relocating an executable an option.

Since relocations are only needed to build an executable, but not when you run it, they normally aren't present in a linked file. Instead you need to specifically tell the linker to emit relocations when necessary. For the GCC Cross-Compiler, this can be done with the -q switch. Note that the -i and -r switches have a similar description, but cause the linker to yield an object file rather than an executable.

Relocating is of itself fairly straightforward by finding the differences. Start with loading the sections to the location of your choice, then for each relocation entry:

• compute the original address where the relocation was applied
• compute the address where the relocation applies now (its moved by the same amount you moved the original section from its original location)
• do the same for the destination of the relocation
• compute what the relocation value is - the destination for absolute relocations, and the destination minus the origin for relative relocations.
• compute what the relocation value was using the original location.
• subtract the old value from the new value
• add the result to the original relocation value in memory.

If the sections are moved relatively to each other, then relocating can become as simple as only adding the displacement to the absolute relocations. The relative locations do not get changed as both the source and the target are moved by the same amount.

Common errors

• Passing -i or -r to ld. It does not work except for some limited cases, as it generates a file where relocations have not been applied at all.
• Assuming code and data are continuous. A pitfall when trying to make a PE file multiboot-compatible. A section is generally page-aligned (4k), but a PE file is sector-aligned (512b). So if a section is not multiple of 4k in size, relative addresses to the data section will be off by a multiple of 512 bytes as the gap has been removed from the binary. Worse, it is perfectly valid to have metadata sections between the various loadable sections, which can put addresses off.
• Loading as a flat binary. All executables that aren't flat binaries have a header up front. Blatantly loading a file and starting at the start will execute the header instead of your code. Again, there is a tutorial that tries to get away with this.
• Assuming the entry point is at the start. The linker has a certain amount of freedom in what order it loads the object files, and so does the compiler. That means that main doesn't need to be at the start of the code section.