Using qemu-user-static to help the Debian e500 bootstrap

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port"

I was trying to figure out an easy way to do basic testing without directly having access to the hardware and I discovered that QEMU supports several MPC85xx processor variants for its user-mode emulation.  The standard "qemu-user-static" package even allows you to trivially run non-native programs (either in a chroot or on your normal system) without much complexity.

Unfortunately, I could not figure out any way to convince QEMU to pick the right CPU type by default, especially since there might not be any reasonable config files in the chroot.  The solution ended up being to add a second binary alongside the normal "/usr/bin/qemu-ppc-static" file.  I wrote the following short little C program:

  #include <stdlib.h>
  #include <unistd.h>


  #define QEMU_BIN "/usr/bin/qemu-ppc-static"
  #define CPU_TYPE "MPC8548E_v21"


  int main(int argc, char **argv, char **envp)
  {
          int i;


          char **newargv = malloc(sizeof(*newargv) * (argc + 3));
          if (newargv) {
                  newargv[0] = QEMU_BIN;
                  newargv[1] = "-cpu";
                  newargv[2] = CPU_TYPE;
                  for (i = 1; i < argc; i++)
                          newargv[i + 2] = argv[i];
                  execve(QEMU_BIN, newargv, envp);
          }
          exit(255);
  }


Then I compiled it like this and put it in /usr/local/bin: (UPDATE: Fixed options for static compile)
  $ gcc -static -Wall -Wextra -Werror -ggdb3 -o qemu-e500v2-static qemu-e500v2-static.c
  $ sudo install qemu-e500v2-static /usr/local/bin/


Next I installed the Debian package "qemu-user-static" on my dev system and disabled the previously-loaded PPC support: (UPDATE: Diverted somewhere the init script won't find it)
  $ sudo aptitude install qemu-user-static binfmt-support
  $ sudo update-binfmts --package qemu-user-static \
            --remove qemu-ppc /usr/bin/qemu-ppc-static
  $ sudo dpkg-divert --divert /usr/share/binfmt.diverted.qemu-ppc \
            --local --rename --add /usr/share/binfmts/qemu-ppc


Then I needed to modify the existing PPC handler for e500v2 (and my custom C program)
  $ cp /usr/share/binfmt.diverted.qemu-ppc ~/qemu-e500v2.binfmt
  $ vim ~/qemu-e500v2.binfmt

The resulting file should look somewhat like this (the flags/offset/magic/mask are unmodified from the original):

  package <local>

  interpreter /usr/local/bin/qemu-e500v2-static

  flags: OC

  offset: 0

  magic \x7fELF......stuff....

  mask \xff\xff\xff\xff.........stuff.....

Then you just need to load your new version and off you go:
  $ sudo update-binfmts --import ~/qemu-e500v2.binfmt
  $ LD_LIBRARY_PATH=/usr/powerpc-linux-gnuspe/lib \
            /usr/powerpc-linux-gnuspe/lib/ld.so.1
  Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]
  [...]


With the rest of the multiarch support slowly going into Debian, this seems extremely promising to me for being able to avoid a lot of the classic cross-build hassle by doing pretend-native builds with a cross-compiler and a bunch of native headers and other tools.  Eventually you will also be able to have a native-architecture-only chroot with only those two binaries (the qemu-e500v2-static and the qemu-ppc-static) in it.

Unfortunately not enough of multiarch is working yet to really make it 100% feasible, but it should at least allow me to work around many of the egregious cross-compiling bugs by actually being able to run the compiled binaries locally.

On the other hand, if my goal was instead to actually identify and fix such cross-compiling bugs, then I would probably need to disable the qemu binfmt lest I get falsely successful builds.

The more likely use-case is that I will use this to perform some basic testing of my freshly built packages when I don't have direct hardware access.

Cheers,
Kyle Moffett