User:Klange/QEMU virt

These are my personal notes on the QEMU aarch64 -machine virt target.

Kernel Link Address

The virt machine places RAM at 0x4000_0000 and will happily load your kernel there, but you don't want to do this as that's also where it puts the device tree. If your kernel is too low in RAM, QEMU will quietly skip loading the device tree. Recommendation from geist is 0x4010_0000.

PCIe

With the latest virt target, expect PCIe to only be available from the high memory location provided through the device tree. To get the hardcoded low address for the ECAM, use -machine virt-2.12 instead.

To find the ECAM offset for a device, the controller QEMU emulates helpfully simplifies things:

(bus << 20) | (slot << 15) | (function << 12) | (field)

Framebuffer

The virt target comes with PCIe and you can happily attach a Bochs graphics adapter with -device bochs-display. It will provide the MMIO interface, but you'll have to actually assign BARs for it (and the framebuffer) as there's no firmware setting up PCIe for you. You don't need to do anything with the bridge, though.

Note that the Bochs adapter appears to have issues on some accelerators, like HVF; it seems to be trapping on every right, so it is incredible slow. The QEMU docs recommend the virtio-gpu-pci which uses regular guest memory and has flush commands.

fw-cfg

QEMU's fw-cfg interface is available and you can use it to stuff arbitrary data and files into your VM, such as a ramdisk that I can't figure out how to get from the -initrd option despite that not printing any warnings...

To communicate with fw-cfg on ARM64, locate the MMIO address through the DTB (it has a name like fw-cfg@XXXXXX) and then treat it as three MMIO ports, something like this:

struct fw_cfg {
  union {
    volatile uint64_t _64;
    volatile uint32_t _32;
    volatile uint8_t  _8;
  } data; /* data is "string-preserving", so if you want a 32-bit integer, it'll be big-endian, but all of it will still be in the first four bytes */
  volatile uint16_t selector;
  uint16_t padding_1; /* data, selector, and dma_addr are 64-bit aligned... not sure why they didn't stick selector at the end to avoid padding confusion since it's always 16-bit... */
  uint32_t padding_2;
  volatile uint64_t dma_addr;
};

Annoyingly, everything here is big-endian and you need to be careful with your access widths as QEMU will give you as much as you ask for.

To confirm that you are talking to QEMU (or rather, to confirm the fw-cfg device is actually working, obviously you're talking to QEMU if you find one at all...), write 0 to the selector (make sure you write two bytes) and then read at least 4 bytes from the data response (you can do that as a read of a uint32_t or four uint8_ts or even just read a uint64_t as the other bytes will be 0) and confirm you get "QEMU" out of it.

To read the list of selectors, write 0x0019 in big-endian layout to the selector field and then read a 32-bit count (also big-endian) from the data port, and then a series of structs:

struct fw_cfg_file {
  uint32_t size;
  uint16_t selector;
  uint16_t padding;
  char name[56];
};

Read the structs as bytes and then endian-swap the size. For the purpose of display while debugging, you may also want to endian-swap the selector, but just remember it needs to be big-endian again when you use it.

Read files by writing the (big-endian) selector you got from the list to the selector field in the MMIO, and then read the bytes up to the size however you want.

You can also use the DMA interface, which is a lot faster. Set up a struct like this:

struct fw_cfg_dma {
  volatile uint32_t control;
  volatile uint32_t length;
  volatile uint64_t address;
};

control is a combination of a bitfield and the selector. Again, all of these are big-endian.

struct fw_cfg_dma dma;
dma.control = swizzle((selector << 16) | (1 << 3 /* read */) | (1 << 1 /* from selector */));
dma.length  = swizzle(size);
dma.address = swizzle64(physical_address_to_read_into);

Once you've set up this struct, write its address (64-bit and big-endian!) to the dma_addr field of the MMIO. QEMU docs say that the write to the latter 4 bytes is what triggers the transaction and that a single 64-bit write should do the write thing, so this should immediately trigger the DMA.

You can then re-read the values in control to check for errors; it should have been 0'd by QEMU on success. QEMU also says that DMA transactions are synchronous so it should either be done already or have errored.

With all of that done, you now have an easy way to get arbitrary blobs and strings into your kernel at runtime.

RTC

QEMU provides a real-time clock. It has a 32-bit counter that gives Unix timestamps in seconds. It will rollover in 2038, so maybe add a check to catch that.

You just need to read 0x09010000 as a little-endian 32-bit unsigned integer.

Presumably this can move and DTB will tell you where it actually is...

Serial

Write bytes to 0x09000000. Weirdly the QEMU docs seem to suggest the MMIO access is 32-bit, but it's little-endian so ints or chars or whatever, doesn't matter. Just spew to that.

Make sure you also enable the serial output (-serial mon,stdio or something) if you actually want to see it...