PE

From OSDev Wiki
(Redirected from PE Binaries)
Jump to navigation Jump to search
Executable Formats
Microsoft

16 bit:
COM
MZ
NE
Mixed (16/32 bit):
LE
32/64 bit:
PE
COFF

*nix
Apple

With Windows 9x/NT, a new executable file type was required. Thus was born the "PE" Portable Executable, which is still in use. Unlike its predecessors, WIN-PE is a true 32bit file format, supporting relocatable code. It does distinguish between TEXT, DATA, and BSS. It is, in fact, a bastardized version of the COFF format.

If you did set up a Cygwin environment on your Windows machine, "PE" is the target format for your Cygwin GCC toolchain, which causes the unaware some headache when trying to link parts build under Cygwin with parts build under Linux or BSD (which use the ELF target by default). (Hint: You have to build a GCC Cross-Compiler)

The PE format is used by Windows 95 and higher, Windows NT 3.1 and higher, the Mobius, ReactOS and UEFI. The PE format is also used as a container for .NET assemblies by both the Microsoft .Net CLI and Mono, in which case it doesn't actually store executable data but .Net metadata with IL attached to it.

Inside the PE file

Below will attempt to explain the various concepts and parts that make up a PE file rather than the exact data structures inside of them since that would certainly take up too much space. The reasoning is that most resources on PE files tend to just dump a bunch of data structures in your face without fully explaining what they are for. So by reading the following and knowing what a PE file consists of you will have a better understanding of what to expect and how to utilize PE file resources.

Overview

The PE file consists of several parts. They are briefly sumarized below, then more detail is entered on each part. Here are the definitions for the field types. PE files are stored in little-endian order, the same byte order as an x86.

An overview of the format

DOS Stub

The PE format begins with a MS-DOS stub (a header plus executable code) which makes it a valid MS-DOS executable. The MS-DOS header begins with the magic code 0x5A4D and is 64 bytes long, followed by real-mode executable code. The standard stub used almost universally is 128-bytes long (including header and executable code) and simply outputs "This program cannot be run in DOS mode." Despite many utilities that with PE files are hard coded to expect the PE header to start at exactly 128 bytes in, this is incorrect since in some linkers, including Microsoft's own Link, it is possible to replace the MS-DOS stub with one of your own choosing, and many older programs did this to allow the developer to bundle a MS-DOS and Windows version into a single file. The correct way is to read a formerly reserved 4-byte address inside the MS-DOS header located at 0x3C (field commonly known as e_lfanew) which contains the address at which PE file signature is found, and PE file header follows immediately. Usually this is a pretty standard value (most of the time this field is set to 0xE8 by the default link.exe stub). Microsoft seemingly recommends aligning the PE header on an 8 byte boundary (https://web.archive.org/web/20160609191558/http://msdn.microsoft.com/en-us/gg463119.aspx, page 10, figure 1).

PE header

The PE header contains information that concerns the entire file rather than individual pieces that will be coming up later. The bare minimum header contains a 4-byte signature (0x00004550), the machine type/architecture of the executable code inside, a time stamp, a pointer to symbols, as well as various flags (is the file an executable, DLL, can the application handle addresses above 2GB, does the file needed be copy to the swap file if ran from a removable device, etc). Unless you're using a really stripped down statically linked PE file to save memory with a hard coded entry point and no resources, then the PE header alone isn't enough. 

// 1 byte aligned
struct PeHeader {
	uint32_t mMagic; // PE\0\0 or 0x00004550
	uint16_t mMachine;
	uint16_t mNumberOfSections;
	uint32_t mTimeDateStamp;
	uint32_t mPointerToSymbolTable;
	uint32_t mNumberOfSymbols;
	uint16_t mSizeOfOptionalHeader;
	uint16_t mCharacteristics;
};

Optional header

The optional PE header follows directly after the standard PE header. It's size is specified in the PE header which you can also use to tell if the optional header exists. The optional PE header begins with a 2-byte magic code representing the architecture (0x010B for PE32, 0x020B for PE64, 0x0107 ROM). This can be used in conjunction with the machine type to see in the PE header to detect if the PE file is running on a compatible system. There are a few other useful memory-related variables including the size and virtual base of the code and data, as well as the application's version number (completely user specified, some update utilities use this to detect if a newer version is available), entry point, and how many directories there are (see below).

Part of the optional header is NT-specific. This include the subsystem (console, driver, or GUI application), how much stack and heap space to reserve, and the minimum required Operating System, subsystem and Windows version. You can use your own values for all of these depending on the needs of your OS. 

// 1 byte aligned
struct Pe32OptionalHeader {
	uint16_t mMagic; // 0x010b - PE32, 0x020b - PE32+ (64 bit)
	uint8_t  mMajorLinkerVersion;
	uint8_t  mMinorLinkerVersion;
	uint32_t mSizeOfCode;
	uint32_t mSizeOfInitializedData;
	uint32_t mSizeOfUninitializedData;
	uint32_t mAddressOfEntryPoint;
	uint32_t mBaseOfCode;
	uint32_t mBaseOfData;
	uint32_t mImageBase;
	uint32_t mSectionAlignment;
	uint32_t mFileAlignment;
	uint16_t mMajorOperatingSystemVersion;
	uint16_t mMinorOperatingSystemVersion;
	uint16_t mMajorImageVersion;
	uint16_t mMinorImageVersion;
	uint16_t mMajorSubsystemVersion;
	uint16_t mMinorSubsystemVersion;
	uint32_t mWin32VersionValue;
	uint32_t mSizeOfImage;
	uint32_t mSizeOfHeaders;
	uint32_t mCheckSum;
	uint16_t mSubsystem;
	uint16_t mDllCharacteristics;
	uint32_t mSizeOfStackReserve;
	uint32_t mSizeOfStackCommit;
	uint32_t mSizeOfHeapReserve;
	uint32_t mSizeOfHeapCommit;
	uint32_t mLoaderFlags;
	uint32_t mNumberOfRvaAndSizes;
};

Data Directories

While technically part of the optional header and follows directly after it is a list of entries pointing to data directories (only in executable images and DLLs). Because the optional header can vary in size, you only need to pay attention to the directories that exist and you expect, since it is likely that new data directories will be added to the PE specification in the future (.Net is an example of one that was recently added). Each data directory is referenced as an 8-byte entry in the optional header. The first 4 bytes is the Relative Virtual Address, or RVA (see Sections below), of the directory, and the last 4 bytes is the size of the directory.

Each data directory that the entries point to have their own format. Data directories are used to describe import tables for dynamic linking, a table of resources that are embedded inside of the PE file, debug information (line numbers and break points), the CLI .Net header.

Position (PE/PE32+) Section
96/112 The export table address and size. Same format as .edata
104/120 The import table address and size. Same format as .idata
112/128 The resource table address and size. Same format as .rsc
120/136 The exception table address and size. Same format as .pdata
128/144 The attribute certificate table offset (not RVA) and size. See Signed PE below
136/152 The base relocation table address and size. Same format as .reloc
144/160 The debug data starting address and size. Same format as .debug
152/168 Architecture, reserved MBZ
160/176 Global Ptr, the RVA of the value to be stored in the global pointer register. The size member of this structure must be set to zero.
168/184 The thread local storage (TLS) table address and size. Same format as .tls

Sections

A PE file is made up of sections which consist of a name, offset within the file, virtual address to copy to, as well as the size of the section in the file and in virtual memory (which may differ, in which case the difference should be cleared 0s), and associated flags. The sections usually follow universal naming (".text", ".rsrc", etc), but this can also vary between linker and in some cases can be user-defined, so it is better to depend on the flags to tell if a section is executable or writable. However in saying that, if you have custom data that you wish to embed inside of the executable, then placing it inside of a section and identifying it by the section's name can be a good idea since you won't be changing the PE format and your executable will remain compatible with PE tools.

The Relative Virtual Base is a phrase that comes up a lot in PE documentation. The RVA is the address at where something exists once it's loaded into memory, rather than an offset into the file. To calculate the file's address from an RVA without actually loading the sections into memory, you can use the table of section entries. By using the virtual address and size of each section you can find which section the RVA belongs to, then subtract the difference between the section's virtual address and file offset.

Section header

Each section has an entry in section header table. 

struct IMAGE_SECTION_HEADER { // size 40 bytes
	char[8]  mName;
	uint32_t mVirtualSize;
	uint32_t mVirtualAddress;
	uint32_t mSizeOfRawData;
	uint32_t mPointerToRawData;
	uint32_t mPointerToRelocations;
	uint32_t mPointerToLinenumbers;
	uint16_t mNumberOfRelocations;
	uint16_t mNumberOfLinenumbers;
	uint32_t mCharacteristics;
};

In asm linkage

if in nasm you declare a block of code like this: 

segment .code
aAsmFunction:
;Do whatever
mov BYTE[aData], 0
ret
segment .data
aData: db 0xFF

The segments will apear as sections. Using this it is possible to keep C and Asm seperate, as a linker will not automatically merge .code and .text, which is the normal output by C compilers.

Position Independent Code

If each section specifies which virtual address to load it in to, you may be wondering how multiple DLLs can exist in one virtual address space without conflict. It is true that most code you'll find in a PE file (DLL or otherwise) is position dependent and linked to a specific address. However to resolve this issue there exist a structure called a Relocation Table that is attached to each section entry. The table is basically a HUGE long list of every address stored in that section so you can offset it to the location where you loaded the section.

Because addresses can point across section borders, relocations should be done after each section is loaded into memory. Then reiterate over each section, iterate through each address in the Relocation Table, find out what section that RVA exists in and add/subtract the offset between that section's linked virtual address and the section's virtual address you loaded it into.

Signed PE with Attribute Certificate Table

Many PE executable (most notably all Microsoft updates) are signed with a certificate. These information is stored in the Attribute Certificate Table, pointed by the Data Directory's 5th entry. It is important that for the Attribute Certificate Table no RVA is stored, rather a simple file offset. The format is concatenated signatures, each with the following structure:

Offset Size Field Description
0 4 dwLength Specifies the length of the attribute certificate entry.
4 2 wRevision Contains the certificate version number, magic 0x0200 (WIN_CERT_REVISION_2_0)
6 2 wCertificateType Specifies the type of content in bCertificate, magic 0x0002 (WIN_CERT_TYPE_PKCS_SIGNED_DATA)
8 x bCertificate Contains a PKCS#7 SignedData structure

For Secure Boot under EFI such a signature is a must. It worth nothing that the PE format allows multiple certificates to be embedded in a single PE file, but UEFI firmware implementations usually only allow one, which must be signed by the Microsoft KEK. If the firmware allows installing more KEK (not typical), then you can use other certificates as well.

The bCertificate data is a PKCS#7 signature with certificate, encoded in ASN.1 format. Microsoft uses signtool.exe to create these signature entries, but an Open Source solution exists, called sbsigntool (also available on github with debian packaging).

CLI / .Net

CLI works alongside the PE format. Rather than being an extension to the format, it really exists as its own format inside of a format with a completely different way of storing tables and values. All the .Net data and headers exist inside of sections that are loaded into memory (they are loaded into memory since CLI involves heavy language reflection requiring the metadata without thrashing the disk). The second reason that the .Net metadata exists inside of the sections rather than the PE headers is because the PE loader actually has no concept of .Net at all. (Exception: There is a data directory entry pointing to the RVA of the CLI header so tools can easily access the .Net data without loading it into virtual memory.) In fact, I highly doubt the Windows kernel has any sort of concept of .Net at all. The way .Net works is by dynamically linking against the .Net runtime (mscoree.dll), and setting the entry point to a symbol (_CorExeMain) that resolves to a location inside of mscoree.dll instead of the local executable. This means that Windows CE, WINE, and ReactOS can all load .Net assemblies once the .Net framework can be installed without any specific code.

Loading a PE file

To load a PE file is quite simple;

1. Extract from the header the entry point, heap and stack sizes.

2. Iterate through each section and copy it from the file into virtual memory (although not required, it is good to clear the difference between the section size in memory and in the file to 0).

3. Find the address of the entry point by finding the correct entry in the symbol table.

4. Create a new thread at that address and begin executing!

To load a PE file that requires a dynamic DLL you can do the same, but check the Import Table (referred to by the data directory) to find what symbols and PE files are required, the Export Table (also referred to by the data directory) inside of that PE file to see where those symbols are and match them up once you've loaded that PE's sections into memory (and relocated them!) And lastly, beware that you'll have to recursively resolve each DLL's Import Tables as well, and some DLLs can use tricks to reference a symbol in the DLL loading it so make sure you don't get your loader stuck in a loop! Registering symbols loaded and making them global might be a good solution.

It may also be a good idea to check the Machine and Magic fields for validity, not just the PE signature. This way your loader won't try loading a 64 bit binary into 32 bit mode (this would be certain to cause an exception).

64 bit PE

64 bit PE's are extremely similar to normal PE's, but the machine type, if AMD64, is 0x8664, not 0x14c. This field is directly after the PE signature. The magic number also changes from 0x10b to 0x20b. The magic field is at the beginning of the optional header. Plus, the BaseOfData member of the optional header is non existent. This is because the ImageBase member is expanded to 64 bits. The BaseOfData is removed to make room for it

See Also

Retrieved from "https://wiki.osdev.org/index.php?title=PE&oldid=28535"