Debian PowerPC e500v2 port, part 3

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port"

Before we can actually start cross-compiling packages, we need to create a quick set of symlinks so that they can find our compiler.  The "gcc-4.6" packages we built in the previous steps created a set of tools named like this:

powerpc-linux-gnuspe-gcc-4.6
powerpc-linux-gnuspe-g++-4.6

Unfortunately, by default autoconf won't actually find those, and the "gcc-defaults" package does not have a cross-compiler mode (although it can be cross-"compiled", as we will need to do later).  So symlink the relevant binaries to versions without the "-4.6" at the end:

$ cd /usr/bin
$ for i in powerpc-linux-gnuspe-*-4.6; do sudo ln -s "$i" "${i%-4.6}"; done

So the next step is to make sure I have all of the Essential and Build-Essential packages up-to-date for installation into a usable root filesystem for building everything else.  That means I need to figure out a list of those packages first.

To make this easier, we can use the "grep-dctrl" tool to look through the APT package cache on our favorite Debian box.  MAKE SURE YOU USE A BOX WITH DEBIAN TESTING/UNSTABLE.  If you don't then you won't get the right package lists for your new port.

Let's first start with the "Essential: yes" packages:

$ sudo apt-get install grep-dctrl
$ grep-available -Xn -s Package -FEssential 'yes' >new-pkgs.txt

Now add "build-essential" to the list and get ready to list dependencies:

$ echo 'build-essential' >>new-pkgs.txt
$ : >all-pkgs.txt

Then we need to recursively follow dependencies.  It's all basically done with the following scriptlet:

$ while [ -s new-pkgs.txt ]; do \
      cat new-pkgs.txt >>all-pkgs.txt; \
      for i in $(cat new-pkgs.txt); do \
          grep-available -PXn -s Depends,PreDepends "$i"; \
      done | sed -e 's/([^)]\+)//g' -e 's/, /\n/g' \
          -e 's/ *|[^\n]*//g' | grep . | sort | uniq >dep-pkgs.txt; \
      cat dep-pkgs.txt all-pkgs.txt all-pkgs.txt \
          | sort | uniq -u >new-pkgs.txt; \
  done

Once that's done running (it should take ~30 seconds), you will have the complete binary package list in all-pkgs.txt.  Now we need to find all the source packages necessary to build those binary packages:

$ for i in $(cat all-pkgs.txt); do \
      grep-available -PXn -sSource:Package "$i"; \
  done | sed -e 's/ (.*//' | sort | uniq >all-srcs.txt

Go ahead and use the resulting all-srcs.txt file to make yourself a checklist, because the rest of this is going to be a painfully long and manual process.  I've included my list below, wrapped onto a single line to minimize wasted space:

base-files base-passwd bash binutils build-essential bzip2 coreutils dash db debconf debianutils diffutils dpkg e2fsprogs eglibc findutils gcc-4.6 gcc-defaults gdbm gmp grep gzip hostname insserv libselinux libsepol libtimedate-perl linux-2.6 lsb make-dfsg mpclib mpfr4 ncurses patch perl sed sensible-utils shadow sysvinit tar tzdata util-linux xz-utils zlib

Not too unreasonable, right?  What you will rapidly realize is that build-dependencies are the key, and we have not actually tried to list those out yet.  The biggest problem tends to be docs-generation tools (EG: LaTeX, Doxygen, etc) which are entirely unnecessary for bootstrap purposes but have lots of dependencies of their own (EG: xorg, qt4, etc).

So let's pick one of these and get started.  I've included the precise versions I used below, but you can just leave off the "=1.4-1" bit to get the latest unstable version assuming your APT is set up correctly.

$ mkdir gzip && cd gzip && apt-get source gzip=1.4-1

First you should check the "Build-Depends" field in the "debian/control" file, to make sure you have all of the necessary dependencies.  Note that some dependencies (automake, autoconf, texinfo, etc) should be satisfied by your build-system architecture (EG: amd64), while others (libc6, etc) should be satisfied by "-cross" packages for your target architecture.

$ grep Build-Depends: gzip-1.4/debian/rules
Build-Depends: debhelper (>= 5), texinfo, autoconf, automake, autotools-dev

 Ok, everything looks good there.  Now check the "debian/rules" file to check for the following few issues:

1. The rules should reference DEB_HOST_GNU_TYPE to talk about the target system.  If it seems to use DEB_BUILD_GNU_TYPE for that then it's probably broken.
2. Any testsuites should be disabled when DEB_HOST_GNU_TYPE != DEB_BUILD_GNU_TYPE, otherwise the build will likely fail.  You may be able to work around these kinds of issues by setting DEB_BUILD_OPTIONS=nocheck.

There are certainly lots of other ways that packages can fail to cross-compile correctly, but if it uses autoconf/automake and either cdbs or debhelper then it should be mostly OK.  The next step is to try a cross-compile:

$ ( cd gzip-1.4 && dpkg-buildpackage -a"${MYARCH}" -us -uc -B ) 2>&1 | tee build-gzip.log

Even if the build finishes successfully, you should use "lintian" and "dpkg-deb -c" to verify that the package files look correct.  If you have any errors or unexpected package contents then you have some serious work to do to figure out why and fix the package.  In the "gzip" case I identified a minor bug (Debian bug #644785) in the packaging which resulted in one particular setting coming from the build-system instead of from the host.

NOTE: If you have to make any changes at all, make sure you generate a nice patch and submit it back to Debian with the "reportbug" tool.

Once you have the package built, you should check to see if it resulted in any library packages (lib* and lib*-dev).  If so, you should try to get those installed with "dpkg-cross" the same way that the eglibc ones were installed previously, as you may need them to satisfy other cross-build dependencies later on.

That's it for tonight!

Cheers,
Kyle Moffett