Lowcore and Prefixing

From OSDev Wiki
Jump to navigation Jump to search

The Lowcore (also called the prefix area or prefix page) is the 8 KiB block of storage at the bottom of each CPU's real address space that the hardware itself uses for interruption handling, IPL, and status storing. The Prefixing mechanism is what lets every CPU in a multiprocessor system have its own private Lowcore, even though all of them architecturally "see" it at real address 0x0000.

Understanding both is a prerequisite for handling any interruption on z/Architecture (s390x), as the CPU stores and fetches PSWs from fixed Lowcore locations.

At a glance
Size (z/Architecture) 8 KiB (2 × 4 KiB frames)
Alignment 8 KiB, in absolute storage
Real address 0x0000–0x1FFF
Set with SPX / SIGP (Set Prefix)
Read with STPX
Size (ESA/390) 4 KiB

Note: This page describes the architectural Lowcore as defined by the IBM z/Architecture Principles of Operation (SA22-7832, tested against the z17 edition), plus a short overview of how the ZXFoundation kernel uses the software-defined portion. It is NOT the IHAPSA (Prefixed Save Area) mapping used by z/OS. Field names, offsets, and software-use areas will not match the z/OS macro, and should not be assumed to be interchangeable.

Terminology

Term Meaning
Absolute address An address into physical main storage, after prefixing has been applied.
Real address The address seen by the program with DAT off (or the result of DAT), before prefixing is applied.
Prefix register A per-CPU register that holds the absolute address of that CPU's Lowcore.
Lowcore / Prefix area The 8 KiB region designated by the prefix register.
Absolute zero page Absolute storage 0x0000–0x1FFF. Only reachable through the prefix swap (see below) or with absolute-addressing instructions.

The Prefixing Mechanism

How it works

Prefixing is a transformation applied by the hardware to a real address to produce an absolute address. Let P be the (8 KiB aligned) value in the prefix register. For a real address A:

Real address range Resulting absolute address
0x0000 ≤ A < 0x2000 A + P
P ≤ A < P + 0x2000 A − P
Anything else A (unchanged)

In other words, the two 8 KiB areas are swapped: a CPU whose prefix is non-zero sees its own Lowcore at real 0x0, and sees the original absolute zero page at real address P. Everything else is identity-mapped.

        Real address space              Absolute (physical) storage
      (as seen by CPU n, P = 0x40000)
     +-----------------------+        +-----------------------+
0x0  |  CPU n Lowcore (8K)   | ---->  |  P:  CPU n Lowcore    |
     +-----------------------+        +-----------------------+
     |                       |        |                       |
     |      unchanged        | ---->  |      unchanged        |
     |                       |        |                       |
P    |  Absolute zero page   | ---->  |  0x0: Absolute zero   |
     +-----------------------+        +-----------------------+

Where it sits in the translation pipeline

Prefixing is applied after DAT and only to real addresses. The order is therefore:

  1. Logical (virtual) address → DAT (if PSW bit 5 is set) → real address
  2. Real address → prefixing → absolute address
  3. Absolute address → main storage

Consequences worth remembering:

  • Interruption handling always operates on the real addresses 0x0000–0x1FFF, so the hardware always lands in the current CPU's Lowcore, regardless of DAT state.
  • Absolute addressing (e.g. in some diagnostic and dump paths) bypasses prefixing entirely.

The Prefix Register

  • The prefix is an 8 KiB-aligned address; in z/Architecture it is held in bits 33–50 of the register, which limits the Lowcore location to the first 2 GiB of absolute storage.
  • After a CPU reset (initial or clear) the prefix is zero, so every CPU starts out sharing absolute zero as its Lowcore. Software is expected to give each CPU its own area as early as possible.
  • In ESA/390 mode the prefix area is 4 KiB; z/Architecture uses 8 KiB.
Instruction Mnemonic Description
B210 SPX Set Prefix on the current CPU (privileged).
B211 STPX Store Prefix of the current CPU (privileged).
AE SIGP The Set Prefix order sets the prefix of a target CPU, which must be in the stopped state.

Changing the prefix safely

Before issuing SPX, the new Lowcore page should already contain valid New PSW fields, since an interruption may arrive immediately after the swap. A recommended sequence for the boot CPU:

  1. Allocate an 8 KiB-aligned, 8 KiB block in absolute storage below the 2 GiB boundary.
  2. Zero it, then populate at least the New PSWs and any stack pointers the handlers will rely on.
  3. Issue SPX.
  4. From this point, real 0x0 refers to the new area. Absolute zero is now reachable only at real address P.
#include <stdint.h>

static inline void set_prefix(uint32_t addr)
{
    __asm__ volatile ("spx %0" : : "Q" (addr) : "memory");
}

static inline uint32_t store_prefix(void)
{
    uint32_t addr;
    __asm__ volatile ("stpx %0" : "=Q" (addr) : : "memory");
    return addr;
}

Lowcore Layout

Offsets below are in hexadecimal (and decimal), measured from the start of the Lowcore. All multi-byte fields are big-endian. Fields marked Arch are defined by the Principles of Operation and written by hardware. Fields marked SW are in storage the architecture leaves for the operating system, and are implementation-defined.

Architecturally defined fields

Offset (hex) Offset (dec) Size Name Type
0x0000 0 8 IPL PSW (ESA/390 or short PSW view) Arch
0x0008 8 8 IPL CCW 1 Arch
0x0010 16 8 IPL CCW 2 (also loader size / sysparm block address) Arch
0x0080 128 4 External-interruption parameter Arch
0x0084 132 2 CPU address Arch
0x0086 134 2 External-interruption code Arch
0x0088 136 4 SVC interruption identification (ILC, code) Arch
0x008C 140 4 Program-interruption identification (ILC, code) Arch
0x0090 144 4 Exception code (vector / data exception) Arch
0x0094 148 2 Monitor-class number Arch
0x0096 150 2 PER code Arch
0x0098 152 8 PER address Arch
0x00A0 160 1 Exception access identification (EAID) Arch
0x00A1 161 1 PER access identification Arch
0x00A2 162 1 Operation access identification Arch
0x00A3 163 1 Architectural-mode identification Arch
0x00A8 168 8 Translation-exception identification (TEID) Arch
0x00B0 176 8 Monitor code Arch
0x00B8 184 12 I/O-interruption code Arch
0x00C8 200 4 Facility list (STFL) Arch
0x00E8 232 8 Machine-check interruption code Arch
0x00F4 244 4 External-damage code Arch
0x00F8 248 8 Failing-storage address Arch
0x0100 256 8 Monitor-counter array origin Arch
0x0108 264 4 Monitor-counter array size Arch
0x010C 268 4 Monitor-exception count Arch
0x0110 272 8 Breaking-event address Arch

Interruption PSW slots

Each interruption class has a 16-byte Old PSW (written by hardware) and a 16-byte New PSW (loaded by hardware). See PSW for the format.

Class Old PSW New PSW
Restart 0x0120 0x01A0
External 0x0130 0x01B0
Supervisor Call 0x0140 0x01C0
Program 0x0150 0x01D0
Machine Check 0x0160 0x01E0
I/O 0x0170 0x01F0
constexpr uint64_t LC_EXT_OLD_PSW  = 0x0130;
constexpr uint64_t LC_SVC_OLD_PSW  = 0x0140;
constexpr uint64_t LC_PGM_OLD_PSW  = 0x0150;
constexpr uint64_t LC_MCCK_OLD_PSW = 0x0160;
constexpr uint64_t LC_IO_OLD_PSW   = 0x0170;

Status-save areas (stored by hardware on request)

These areas are written by the CPU on STORE STATUS and by the machine-check facility. They are architecturally defined, and are useful for building dump and crash-recovery paths.

Offset (hex) Size Name Type
0x11B0 8 Machine-check extended save area designation Arch
0x1200 128 Floating-point registers (16 × 8) Arch
0x1280 128 General registers (16 × 8) Arch
0x1300 16 PSW save area (overlaps the fixed logout area) Arch
0x1318 4 Prefix register Arch
0x131C 4 Floating-point control register Arch
0x1324 4 TOD programmable register Arch
0x1328 8 CPU timer Arch
0x1330 8 Clock comparator Arch
0x1338 8 Last breaking-event address Arch
0x1340 64 Access registers (16 × 4) Arch
0x1380 128 Control registers (16 × 8) Arch
0x1500 8 Crypto-counter designation Arch
0x1508 8 Adapter-interruption-controls (AIC) designation Arch
0x1800 256 Transaction diagnostic block (TDB) Arch

The Lowcore therefore spans roughly 6400 bytes (0x1900) of defined content, even though it always occupies a full 8 KiB prefix area.

Software-use areas

The remaining space is available to the operating system. The architecture assigns it no meaning, and hardware never touches it. ZXFoundation uses it for the following categories, the exact layout of which is considered kernel-internal and may change between releases:

Region Purpose
0x0200–0x027F Synchronous-entry register save area, stack protection value, and temporary return PSW storage.
0x0280–0x02AF Restart save slot, per-CPU back-pointer, and return PSWs.
0x02B0–0x032F Last-break copy, entry/exit accounting timers, interruption clock snapshots, and a clock-comparator shadow (Linux-style CPU-time accounting model).
0x0330–0x036F Control-register snapshots, current execution-domain pointer, and the per-context stack pointers (kernel, asynchronous, DAT-off, restart, machine-check).
0x0370–0x0397 Restart-interruption hand-off (callback, cookie, source, flags) and address-space origin values.
0x0398–0x03B7 Current context identifier, logical CPU identity, and spinlock bookkeeping values.
0x0400 Per-CPU data pointer. Fixed offset, relied on by early assembly entry code.
0x1400–0x14FF 256-byte kernel crash-record area (ZXFoundation specific).

Note: The Linux-derived fields use the same ideas as Linux/s390 (stack-per-context, entry/exit timers, saved return PSW), but offsets and widths are not guaranteed to match struct lowcore in the Linux kernel.

Practical Considerations

Alignment and allocation

The prefix must be 8 KiB aligned. Allocate Lowcores from absolute storage below 2 GiB, as the prefix register cannot designate anything higher.

The AP (secondary CPU) bring-up

Application processors start with a prefix of zero. A typical flow from the boot CPU is:

  1. Allocate and initialize the AP's Lowcore at a unique 8 KiB-aligned location, filling in at least the restart New PSW and a restart stack.
  2. Issue SIGP Set Prefix to the AP (which must be stopped).
  3. Issue SIGP Restart. The AP loads the Restart New PSW from its own Lowcore, and starts running with DAT off.
  4. The AP code enables DAT and joins the kernel.

Since the AP's restart PSW runs with DAT off, its entry point and initial stack must be given as real/physical addresses.

Accessing another CPU's Lowcore

The BSP cannot rely on real 0x0 to look at another CPU's Lowcore, because that address resolves to its own. You need the other CPU's prefix value, which is then accessed through a normal mapping of that physical page (e.g. a direct map).

Accessing absolute zero

Once the prefix is non-zero, absolute storage 0x0–0x1FFF is visible at real address P. This is relevant for dump tooling, and for retrieving data left there by the IPL process.

Common pitfalls

  • Interrupts with an uninitialised New PSW: if an interruption occurs while the relevant New PSW is zero, the CPU will typically enter a loop of program-checks or load a disabled-wait PSW.
  • Missing New PSW setup before SPX: the new Lowcore must be fully prepared first.
  • Confusing real and absolute addresses when building page tables or storing a "physical" Lowcore pointer: always record the absolute address.
  • Sharing a Lowcore between CPUs: this is possible architecturally, but defeats the purpose of per-CPU stacks and save areas, and will corrupt state on the first simultaneous interruption.
  • Assuming ESA/390 offsets: the 8-byte PSW slots at 0x18–0x78 are not used for interruptions in z/Architecture mode.

See Also