Debian powerpcspe progress, new sbuild installed!

(UPDATE: added "omitdebsrc=true" to the "BD-*" multistrap config entries)

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port".

I have great news to report today!  I finally managed to get the finicky "gcc-4.6", "gcj-4.6", and "gcc-defaults" packages to all build and install, so now I have an actual working system with enough stuff to build most of the archive, including "build-essential" and "sbuild".

Simon McVittie wrote an excellent tutorial on this particular bit of setup on his blog: Space-efficient reproducible builds using schroot, sbuild and LVM.  Unfortunately, things are a bit more complicated for powerpcspe for a few reasons:

• The core system packages will come from both the "unstable" and "unreleased" Debian-Ports repositories, so we need to use both.
• Right now all of my hand-built packages have just been stuffed into my local package repository using "dput" and "mini-dinstall", which also needs to be included in the core packages list.

Once the archive is mostly up-to-date and the relevant patches merged into the official packaging, you will be able to use the same "cdebootstrap" commands from Simon's tutorial, but I will be using a variant of my old "multistrap" configuration for now.

NOTE: You will need an unused partition with at bare minimum 40G of free space in order to make sbuild work properly with LVM snapshots, but preferably 80G or more.  When you are installing your root filesystem with multistrap or cdebootstrap then make sure you leave enough room.  In particular, the "schroot" logical volume needs to have enough free space to be able to install all the development packages needed for any given package build (including stuff like TeX, etc), typically around 6-8GB, but only for one system at a time due to independent LVM snapshots used for each build.  Additionally, each snapshot taken will use 6-8GB depending on your LVM config, so leave some free space in the volume group.

Once you have a working bootable image, you will need to install a few packages to get this going:
  $ aptitude install build-essential sbuild lvm

Now, set up your spare partition as an LVM volume group.  Since I am booting from NFS, I'm using an entire disk ("/dev/sda") and my volume group is called "nfs1u".  The "tune2fs" command can be omitted, I just pasted it in here because those are the defaults I generally tend to use for most servers.

WARNING: MAKE SURE YOU USE THE EMPTY VOLUME HERE OR YOU WILL DESTROY YOUR DATA!!!  If you do manage to erase your root filesystem or something, please feel free to give up here as it's only going to get more complicated from this point on.  Also, please send me a note so I can laugh at you mercilessly for 5 minutes before I accidentally do the same thing to myself.

Creating the volume group:
  # pvcreate -M 2 --dataalignment 4M /dev/sda
  # vgcreate -M 2 nfs1u /dev/sda

Creating the chroot volume; minimum size for a few chroots is 16GB, I used 32GB:
  # lvcreate -n schroot -L 32G nfs1u
  # mke2fs -t ext4 -L nfs1u:schroot /dev/mapper/nfs1u-schroot
  # tune2fs -e remount-ro -c 1 -i 0 -o user_xattr,acl,journal_data_ordered \
      /dev/mapper/nfs1u-schroot

Creating the build scratch-space volume; you need up to 12GB to build some packages, but I used 32GB again to leave room just in case:
  # lvcreate -n build -L 32G nfs1u
  # mke2fs -t ext4 -L nfs1u:build /dev/mapper/nfs1u-build
  # tune2fs -e remount-ro -c 1 -i 0 -o user_xattr,acl,journal_data_ordered \
      /dev/mapper/nfs1u-build

Mounting both volumes (you probably want to put them in "/etc/fstab" too): [EDIT: Changed "/srv/build" to "/var/lib/sbuild/build" and fixed permissions as per sbuild docs]
  # mkdir     /srv/schroot
  # chmod 000 /srv/schroot
  # mount /dev/mapper/nfs1u-schroot /srv/schroot
  # chmod 000 /srv/schroot/lost+found
  # chattr +i /srv/schroot/lost+found
  #   # chmod 000 /var/lib/sbuild/build
  # mount /dev/mapper/nfs1u-build /var/lib/sbuild/build
  # chown sbuild /var/lib/sbuild/build
  # chgrp sbuild /var/lib/sbuild/build
  # chmod 2770   /var/lib/sbuild/build
  # chmod 000    /var/lib/sbuild/build/lost+found
  # chattr +i    /var/lib/sbuild/build/lost+found

Then you will need to set up the "schroot" tool to work with the new logical volume setup, so create the file "/etc/schroot/chroot.d/sid-powerpcspe-sbuild":
  [sid-powerpcspe-sbuild]
  description=Debian unstable ("sid") for PowerPCSPE
  aliases=unstable
  
  root-groups=sbuild
  source-root-users=root
  
  type=lvm-snapshot
  device=/dev/nfs1u/schroot
  location=/sid-powerpcspe-sbuild
  lvm-snapshot-options=-L 6G
  
  script-config=sbuild/config

NOTE: For my system I also temporarily added this line to "/etc/schroot/sbuild/fstab":
  /srv/local-pkgmirror /srv/local-pkgmirror none ro,bind 0 0

Now you need to reinstall a totally fresh clean system into the chroot.  If the archive is in good shape the following command should Just Work™:
  # cdebootstrap --flavour=build sid sid http://DEBIAN-PORTS-MIRROR

On the other hand, I didn't have the luxury of a working archive, so what I did was create a "multistrap" configuration file at "/srv/schroot/sid-powerpcspe-sbuild.conf":
  [General]
  arch=powerpcspe
  directory=/srv/schroot/sid-powerpcspe-sbuild/
  
  noauth=true
  cleanup=true
  unpack=true
  
  aptsources=DP-unstable DP-unreleased Local BD-unstable BD-unreleased
  bootstrap=DP-unstable DP-unreleased Local-Bootstrap
  
  [DP-unstable]
  packages=build-essential fakeroot
  source=http://opensource.exmeritus.com/mirror/debian-ports/
  suite=unstable
  keyring=debian-ports-archive-keyring
  omitdebsrc=true
  
  [DP-unreleased]
  packages=
  source=http://opensource.exmeritus.com/mirror/debian-ports/
  suite=unreleased
  keyring=
  
  [BD-unstable]
  packages=
  source=http://incoming.debian-ports.org/buildd/
  suite=unstable
  keyring=
  omitdebsrc=true
  
  [BD-unreleased]
  packages=
  source=http://incoming.debian-ports.org/buildd/
  suite=unreleased
  keyring=
  omitdebsrc=true
  
  [Local-Bootstrap]
  packages=
  source=copy:///srv/local-pkgmirror/ unstable/
  keyring=
  
  [Local]
  packages=
  source=file:///srv/local-pkgmirror/ unstable/
  keyring=

I installed the base system:
  # multistrap -f /srv/schroot/sid-powerpcspe-sbuild.conf

Then I forcibly disabled package init-scripts (to prevent daemons from starting in sbuild chroots).  Note that if you use the "sbuild" flavor of "cdebootstrap" then this should be done for you already.
  # cat >/srv/schroot/sid-powerpcspe-sbuild/usr/sbin/policy-rc.d <<'EOF'
  > #!/bin/sh
  > echo "****************************************" >&2
  > echo "All rc.d operations are denied by policy" >&2
  > echo "****************************************" >&2
  > exit 101
  > EOF
  # chmod 755 "/srv/schroot/sid-powerpcspe-sbuild/usr/sbin/policy-rc.d"

Go ahead and add a user account to the "sbuild" group so they can build packages:
  # adduser kmoffett sbuild

You can test that your new chroot works like this:
  kmoffett$ schroot -c sid-powerpcspe-sbuild echo "Hello world"

I'll try walk through how to configure sbuild itself tomorrow, and then hopefully the buildd daemon some time next week.

Cheers,
Kyle Moffett

Debian PowerPC e500v2 port, part 7

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port"

So last time I managed to get the old Debian archive to install onto a new NFS root filesystem, but I had not yet booted or configured that NFS root.  To do that, you need to be able to boot your board (with kernel) and get that filesystem mounted as "/" somehow; whether using busybox or some other mechanism.

At this point "/etc/inittab" doesn't exist yet, so you will need to boot with "init=/bin/bash" in the kernel arguments.  Since you're probably booting with a serial console, you also want "console=ttyS0,115200n1".  Additionally, you probably need some root filesystem options like "root=/dev/hda1 rw" or "root=192.168.1.42:/srv/nfsroot/my-board rw" or something.

Once the board boots and displays a shell, you're halfway there!  Before you can get packages to configure, though, you need to workaround a "libc6" postinst bug.  Fake the existence of an actual "init" program by running these command:
  $ mkfifo /dev/initctl
  $ cat </dev/initctl >/dev/null &


Next work around a "dash" postinst bug with these commands:
  $ dpkg-divert --package dash --divert /bin/sh.old --add /bin/sh
  $ dpkg-divert --package dash --divert /usr/share/man/man1/sh.1.gz.old \
            /usr/share/man/man1/sh.1.gz

Finally you can configure all the packages.  Run this command and reply to the interactive prompts as necessary:
  $ dpkg --configure -a


Once that finishes, you just have 2 steps remaining before you can reboot.  Set up a serial console and change root's password:
  $ echo 'T0:2345:respawn:/sbin/getty -L ttyS0 115200 xterm-color' \
            >>/etc/inittab
  $ passwd root
  Enter new UNIX password:
  Retype new UNIX password:
  passwd: password updated successfully


Ok, you're done!  Sync and reboot!  This time you don't need to pass any special options.
  $ sync && reboot -f


Unfortunately most of the last few days has been dealing with other problems like merging our HWW-1U-1A U-Boot board support upstream and dealing with other internal projects, so this is it for today.

Cheers,
Kyle Moffett

Debian PowerPC e500v2 port, part 5

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port"

I've been continuing working on cross-compiling a fresh base system, continuing from Part 4.  Since I got stuck on a GCC 4.6 bug I don't know how to fix at the moment, I'm working off as many other packages as I can for the moment.

Additional packages (with notes and Debian bugs) listed below:

bzip2           - 1.0.5-7               - OK
xz-utils        - 5.1.1alpha+20110809-2 - DEB_BUILD_OPTIONS=nocheck
pcre3           - 8.12-4                - OK
grep            - 2.9-2                 - OK
gdbm            - 1.8.3-10              - OK
libsepol        - 2.1.0-1               - #638018
libselinux      - 2.1.0-1.1             - #645121 (and DEB_STAGE=stage1)

hostname        - 3.06                  - OK

make-dfsg       - 3.81-8.1              - OK

debconf         - 1.5.41                - N/A (Architecture: all)

build-essential - 11.5                  - OK

findutils       - 4.5.10-1              - OK (from experimental, see #645274)

patch           - 2.6.1-2               - OK (DEB_BUILD_OPTIONS=nocheck)

gpm             - 1.20.4-4              - #645278

ncurses         - 5.9-2                 - OK

lsb             - 3.2-28                - BAD (needs python-all-dev)

I'm slowly inching closer to being able to install a new root filesystem!

Cheers,
Kyle Moffett

Using "dput" and "mini-dinstall" to create a local Debian package repository

While working on bootstrapping a Debian port, I needed a place to keep a bunch of packages for later cross or native installation.  Even though I happen to have access to the debian-ports mirror for powerpcspe, I still need a place for the cross-compiled or bootstrap packages that are not appropriate for that repository.

So I created a local repository using the "mini-dinstall" command.  Configuration and setup was pretty easy, I just created a "~/.mini-dinstall.conf" file with the following contents.
  [DEFAULT]
  archivedir = /srv/stuff/local-pkgmirror
  mail_to =
  verify_sigs = false
  architectures = all, powerpcspe, amd64
  archive_style = flat
  generate_release = true
  mail_on_success = false
  release_codename = Local
  release_description = Local Packages
  release_label = kmoffett
  release_origin = kmoffett
  release_suite = local


Please note that "/srv/stuff" is just a convenient local partition with lots of free space, you can put the repository anywhere you would like.  I then created a "~/.dput.cf" file (make sure the repository location matches):
  [local]
  fqdn = localhost
  method = local
  incoming = /srv/stuff/local-pkgmirror/mini-dinstall/incoming
  run_dinstall = 0
  post_upload_command = mini-dinstall -b

To get everything created and ready, I ran "mini-dinstall -b" once first, then to "upload" packages into the repository I just run "dput -u local $PACKAGE.changes".  Note that if that command fails you may need to delete the "$PACKAGE.local.upload" file in order to try again.

Cheers,
Kyle Moffett

Debian PowerPC e500v2 port, part 4

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port"

So in a previous posting, I was discussing how to cross-compile packages.  For the moment, much of the existing powerpcspe archive is still perfectly usable, but several of the toolchain packages are so badly out of date it is easier to just re-cross from scratch.

Last time I built the "gzip" package as a demo, now I am trying to get the target build-dependencies for gcc-4.6 set up so I can build a new GCC for my target environment.  For the moment I have the QEMU setup disabled to prevent any accidents, but I may reenable it if I run into any stubborn packages.

As before, to satisfy build-dependencies you will need some mix of native packages (EG: "doxygen") and cross-compiled "dpkg-cross" packages (EG: "libelfg0-dev-powerpcspe-cross").  There's no really automatic way to tell which is which, although if it begins with "lib" you probably need to cross-compile it.

Since most of these so far have been pretty uneventful, I'll just list the packages I completed and any notes or Debian bugs related to the crossbuilding:

gzip        - 1.4-1              - #644785

zlib        - 1:1.2.3.4.dfsg-3   - OK

gmp         - 2:5.0.2+dfsg-1     - OK
mpfr4       - 3.1.0-2            - OK

ppl         - 0.11.2-4           - #645003 (and DEB_STAGE=stage1)

cloog-ppl   - 0.15.9-3           - OK
libelf      - 0.8.13-3           - OK
libmpc      - 0.9-4              - DEB_BUILD_OPTIONS=nocheck
base-files  - 6.5                - OK
base-passwd - 3.5.23             - OK

binutils    - 2.21.90.20111004-2 - OK

gcc-4.6     - 4.6.1-15           - #645018 #645021 STUCK HERE

After doing this now have a much larger list of "-X" options that need to be passed to dpkg-cross when installing headers and libraries for cross-building:


-X libc-bin -X libc-dev-bin -X multiarch-support -X dpkg -X install-info -X ncurses-bin -X texinfo -X make -X python


Cheers,
Kyle Moffett

Using qemu-user-static to help the Debian e500 bootstrap

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port"

I was trying to figure out an easy way to do basic testing without directly having access to the hardware and I discovered that QEMU supports several MPC85xx processor variants for its user-mode emulation.  The standard "qemu-user-static" package even allows you to trivially run non-native programs (either in a chroot or on your normal system) without much complexity.

Unfortunately, I could not figure out any way to convince QEMU to pick the right CPU type by default, especially since there might not be any reasonable config files in the chroot.  The solution ended up being to add a second binary alongside the normal "/usr/bin/qemu-ppc-static" file.  I wrote the following short little C program:

  #include <stdlib.h>
  #include <unistd.h>


  #define QEMU_BIN "/usr/bin/qemu-ppc-static"
  #define CPU_TYPE "MPC8548E_v21"


  int main(int argc, char **argv, char **envp)
  {
          int i;


          char **newargv = malloc(sizeof(*newargv) * (argc + 3));
          if (newargv) {
                  newargv[0] = QEMU_BIN;
                  newargv[1] = "-cpu";
                  newargv[2] = CPU_TYPE;
                  for (i = 1; i < argc; i++)
                          newargv[i + 2] = argv[i];
                  execve(QEMU_BIN, newargv, envp);
          }
          exit(255);
  }


Then I compiled it like this and put it in /usr/local/bin: (UPDATE: Fixed options for static compile)
  $ gcc -static -Wall -Wextra -Werror -ggdb3 -o qemu-e500v2-static qemu-e500v2-static.c
  $ sudo install qemu-e500v2-static /usr/local/bin/


Next I installed the Debian package "qemu-user-static" on my dev system and disabled the previously-loaded PPC support: (UPDATE: Diverted somewhere the init script won't find it)
  $ sudo aptitude install qemu-user-static binfmt-support
  $ sudo update-binfmts --package qemu-user-static \
            --remove qemu-ppc /usr/bin/qemu-ppc-static
  $ sudo dpkg-divert --divert /usr/share/binfmt.diverted.qemu-ppc \
            --local --rename --add /usr/share/binfmts/qemu-ppc


Then I needed to modify the existing PPC handler for e500v2 (and my custom C program)
  $ cp /usr/share/binfmt.diverted.qemu-ppc ~/qemu-e500v2.binfmt
  $ vim ~/qemu-e500v2.binfmt

The resulting file should look somewhat like this (the flags/offset/magic/mask are unmodified from the original):

  package <local>

  interpreter /usr/local/bin/qemu-e500v2-static

  flags: OC

  offset: 0

  magic \x7fELF......stuff....

  mask \xff\xff\xff\xff.........stuff.....

Then you just need to load your new version and off you go:
  $ sudo update-binfmts --import ~/qemu-e500v2.binfmt
  $ LD_LIBRARY_PATH=/usr/powerpc-linux-gnuspe/lib \
            /usr/powerpc-linux-gnuspe/lib/ld.so.1
  Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]
  [...]


With the rest of the multiarch support slowly going into Debian, this seems extremely promising to me for being able to avoid a lot of the classic cross-build hassle by doing pretend-native builds with a cross-compiler and a bunch of native headers and other tools.  Eventually you will also be able to have a native-architecture-only chroot with only those two binaries (the qemu-e500v2-static and the qemu-ppc-static) in it.

Unfortunately not enough of multiarch is working yet to really make it 100% feasible, but it should at least allow me to work around many of the egregious cross-compiling bugs by actually being able to run the compiled binaries locally.

On the other hand, if my goal was instead to actually identify and fix such cross-compiling bugs, then I would probably need to disable the qemu binfmt lest I get falsely successful builds.

The more likely use-case is that I will use this to perform some basic testing of my freshly built packages when I don't have direct hardware access.

Cheers,
Kyle Moffett

Debian PowerPC e500v2 port, part 3

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port"

Before we can actually start cross-compiling packages, we need to create a quick set of symlinks so that they can find our compiler.  The "gcc-4.6" packages we built in the previous steps created a set of tools named like this:

powerpc-linux-gnuspe-gcc-4.6
powerpc-linux-gnuspe-g++-4.6

Unfortunately, by default autoconf won't actually find those, and the "gcc-defaults" package does not have a cross-compiler mode (although it can be cross-"compiled", as we will need to do later).  So symlink the relevant binaries to versions without the "-4.6" at the end:

$ cd /usr/bin
$ for i in powerpc-linux-gnuspe-*-4.6; do sudo ln -s "$i" "${i%-4.6}"; done

So the next step is to make sure I have all of the Essential and Build-Essential packages up-to-date for installation into a usable root filesystem for building everything else.  That means I need to figure out a list of those packages first.

To make this easier, we can use the "grep-dctrl" tool to look through the APT package cache on our favorite Debian box.  MAKE SURE YOU USE A BOX WITH DEBIAN TESTING/UNSTABLE.  If you don't then you won't get the right package lists for your new port.

Let's first start with the "Essential: yes" packages:

$ sudo apt-get install grep-dctrl
$ grep-available -Xn -s Package -FEssential 'yes' >new-pkgs.txt

Now add "build-essential" to the list and get ready to list dependencies:

$ echo 'build-essential' >>new-pkgs.txt
$ : >all-pkgs.txt

Then we need to recursively follow dependencies.  It's all basically done with the following scriptlet:

$ while [ -s new-pkgs.txt ]; do \
      cat new-pkgs.txt >>all-pkgs.txt; \
      for i in $(cat new-pkgs.txt); do \
          grep-available -PXn -s Depends,PreDepends "$i"; \
      done | sed -e 's/([^)]\+)//g' -e 's/, /\n/g' \
          -e 's/ *|[^\n]*//g' | grep . | sort | uniq >dep-pkgs.txt; \
      cat dep-pkgs.txt all-pkgs.txt all-pkgs.txt \
          | sort | uniq -u >new-pkgs.txt; \
  done

Once that's done running (it should take ~30 seconds), you will have the complete binary package list in all-pkgs.txt.  Now we need to find all the source packages necessary to build those binary packages:

$ for i in $(cat all-pkgs.txt); do \
      grep-available -PXn -sSource:Package "$i"; \
  done | sed -e 's/ (.*//' | sort | uniq >all-srcs.txt

Go ahead and use the resulting all-srcs.txt file to make yourself a checklist, because the rest of this is going to be a painfully long and manual process.  I've included my list below, wrapped onto a single line to minimize wasted space:

base-files base-passwd bash binutils build-essential bzip2 coreutils dash db debconf debianutils diffutils dpkg e2fsprogs eglibc findutils gcc-4.6 gcc-defaults gdbm gmp grep gzip hostname insserv libselinux libsepol libtimedate-perl linux-2.6 lsb make-dfsg mpclib mpfr4 ncurses patch perl sed sensible-utils shadow sysvinit tar tzdata util-linux xz-utils zlib

Not too unreasonable, right?  What you will rapidly realize is that build-dependencies are the key, and we have not actually tried to list those out yet.  The biggest problem tends to be docs-generation tools (EG: LaTeX, Doxygen, etc) which are entirely unnecessary for bootstrap purposes but have lots of dependencies of their own (EG: xorg, qt4, etc).

So let's pick one of these and get started.  I've included the precise versions I used below, but you can just leave off the "=1.4-1" bit to get the latest unstable version assuming your APT is set up correctly.

$ mkdir gzip && cd gzip && apt-get source gzip=1.4-1

First you should check the "Build-Depends" field in the "debian/control" file, to make sure you have all of the necessary dependencies.  Note that some dependencies (automake, autoconf, texinfo, etc) should be satisfied by your build-system architecture (EG: amd64), while others (libc6, etc) should be satisfied by "-cross" packages for your target architecture.

$ grep Build-Depends: gzip-1.4/debian/rules
Build-Depends: debhelper (>= 5), texinfo, autoconf, automake, autotools-dev

 Ok, everything looks good there.  Now check the "debian/rules" file to check for the following few issues:

1. The rules should reference DEB_HOST_GNU_TYPE to talk about the target system.  If it seems to use DEB_BUILD_GNU_TYPE for that then it's probably broken.
2. Any testsuites should be disabled when DEB_HOST_GNU_TYPE != DEB_BUILD_GNU_TYPE, otherwise the build will likely fail.  You may be able to work around these kinds of issues by setting DEB_BUILD_OPTIONS=nocheck.

There are certainly lots of other ways that packages can fail to cross-compile correctly, but if it uses autoconf/automake and either cdbs or debhelper then it should be mostly OK.  The next step is to try a cross-compile:

$ ( cd gzip-1.4 && dpkg-buildpackage -a"${MYARCH}" -us -uc -B ) 2>&1 | tee build-gzip.log

Even if the build finishes successfully, you should use "lintian" and "dpkg-deb -c" to verify that the package files look correct.  If you have any errors or unexpected package contents then you have some serious work to do to figure out why and fix the package.  In the "gzip" case I identified a minor bug (Debian bug #644785) in the packaging which resulted in one particular setting coming from the build-system instead of from the host.

NOTE: If you have to make any changes at all, make sure you generate a nice patch and submit it back to Debian with the "reportbug" tool.

Once you have the package built, you should check to see if it resulted in any library packages (lib* and lib*-dev).  If so, you should try to get those installed with "dpkg-cross" the same way that the eglibc ones were installed previously, as you may need them to satisfy other cross-build dependencies later on.

That's it for tonight!

Cheers,
Kyle Moffett

Debian PowerPC e500v2 port, part 2

Well, I've been working again on the Debian powerpcspe port and there is a lot more progress to be had! (This series starts at: "How to bootstrap a new Debian port")

I left off last time after EGLIBC failed to build with errors about undefined references to "__stack_chk_guard".  It turns out the problem is that EGLIBC does not correctly detect the presence of support for the "-fstack-protector" GCC option, as per Debian bug #644662.  The fix is to download and apply this patch, which fixes the configure script test.

The second issue I ran into is that dpkg-gensymbols is not very good at finding all the relevant multi-arch libraries while trying to build packages, as per Debian bug #595144.  The fix there is an existing patch to the "dpkg" source package by Steve Langasek; it changes the "libdpkg-perl" binary package to fix the dpkg-gensymbols tool.

As a side note, it turns out that my previous gcc-4.6-cross-stage1-dep-fix-v2.patch broke non-bootstrap-stage builds of GCC.  As a result, I've posted a new version to Debian bug #644439; this one should handle native, cross, and cross-bootstrap builds fine although it's still undergoing compile tests here.

The final issue with the EGLIBC build is that it tries to build the "locales-all" package by executing the built binaries, which of course fails for cross-compiled EGLIBC (see Debian bug #644771).  The fix is trivial, simply don't build "locales-all" when cross-compiling, found in this patch.

So before we can retry building EGLIBC, we need to build an updated libdpkg-perl for our build host:

Step 6.1 - Building and installing a patched libdpkg-perl:

$ sudo apt-get build-dep dpkg=1.16.1
$ sudo apt-get source dpkg=1.16.1
$ curl -o 'Dpkg-Shlibs.pm-multiarch-search-paths.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595144;msg=37;att=1'
$ patch -d dpkg-1.16.1 -p1 <Dpkg-Shlibs.pm-multiarch-search-paths.patch
$ ( cd dpkg-1.16.1 && dpkg-buildpackage -b -us -uc ) 2>&1 | tee dpkg.log
$ sudo dpkg -i libdpkg-perl_1.16.1_all.deb

Going back to the EGLIBC commands I listed previously for step 6, here is an updated version:

Step 6.2 - Commands to build a stage2 EGLIBC:

$ sudo apt-get build-dep eglibc=2.13-21
$ apt-get source eglibc=2.13-21
$ curl -o 'eglibc-2.13-cross-stage1-support.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644546;msg=5;att=1'
$ curl -o 'eglibc-fix-ssp-detection.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644662;msg=5;att=1'
$ curl -o 'eglibc-disable-cross-locales-all.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644771;msg=5;att=1'
$ patch -d eglibc-2.13 -p1 <eglibc-2.13-cross-stage1-support.patch
$ patch -d eglibc-2.13 -p1 <eglibc-fix-ssp-detection.patch
$ patch -d eglibc-2.13 -p1 <eglibc-disable-cross-locales-all.patch... MAKE ANY OTHER CHANGES TO EGLIBC HERE ...
$ ( cd eglibc-2.13 && DEB_GCC_VERSION=-4.6 \
      dpkg-buildpackage -a"${MYARCH}" -B -us -uc ) 2>&1 \
      | tee eglibc-stage2.log

Once the build finishes, it will have produced a bunch of deb files, but you only actually need the libc6 and libc6-dev packages.  As before, we will eventually be able to install foreign architecture packages but for now we have to forcibly dpkg-cross them (with a few ignored dependencies).

Step 6.3 - Commands to install the stage2 EGLIBC:

$ sudo dpkg-cross -M -a "${MYARCH}" -X libc-bin -X libc-dev-bin -i \
      "libc6_2.13-21_${MYARCH}.deb" 
"libc6-dev_2.13-21_${MYARCH}.deb"

After this completes we have a fully usable C library for our target architecture, and it's time for the final GCC cross-compiler build with all features enabled.

Step 7.1 - Commands to build a final full-featured GCC:

$ sudo apt-get build-dep gcc-4.6=4.6.1-13
$ apt-get source gcc-4.6=4.6.1-13
... MAKE ANY CHANGES TO GCC HERE ...
$ ( cd gcc-4.6-4.6.1 && DEB_GCC_TARGET="${MYARCH}" DEB_STAGE=stage2 \
  dpkg-buildpackage -b -us -uc ) 2>&1 | tee gcc-4.6-stage2.log

The final GCC build leaves about 23 packages lying around in the directory, and you should probably install all of them.

Step 7.2 - Command to install the final GCC:

$ sudo dpkg -i $(dcmd --deb gcc-4.6_4.6.1-13_*.changes)

NOTE: DO NOT UPLOAD ANY OF THE PACKAGES YOU BUILT!!!  They intentionally deviate from standard in various ways due to the cross-building process and you should wait until you can rebuild them natively before uploading.

Cheers,
Kyle Moffett

How to bootstrap a new Debian port

Ok... This is is kind of a funny way to start a blog, but it's the first really serious content that I felt like posting, so here goes!

I've been working on a Debian port for the e500v2 architecture "powerpcspe", which has a rather odd way of doing floating point compared to normal PowerPC systems.

Just recently I've submitted a bunch of Debian bugs for different pieces of the bootstrap process that were rather tedious, in the hope that someone doing a new architecture port in the future might benefit. To have a central place for the entire procedure (ideally one that Google can find easily), I'm posting it all here.


The first step is to figure out your architecture triplet and get that added to dpkg as a new target.  For example, the "powerpcspe" architecture uses the target triplet "powerpc-linux-gnuspe".

Temporarily you can modify the following files to get a local system to recognize a new architecture, but before you ship you should get the dpkg maintainers to add it upstream:

/usr/share/dpkg/archtable
/usr/share/dpkg/cputable
/usr/share/dpkg/triplettable

So let's set these shell variables:

$ unset MYARCH MYTRIPLET TARGET
$ MYARCH=powerpcspe
$ MYTRIPLET="$(dpkg-architecture -a"$MYARCH" -qDEB_HOST_GNU_TYPE)"

NOTE: You will see a warning that your GNU system type does not match your GCC system type.  This is OK, and part of cross-compiling.

Then you need to bootstrap a cross-compiler toolchain with that triplet:

1. Build binutils (assembler, linker, etc)
2. Perform a headers-only Linux Kernel build
3. Build a minimal C-only GCC using just the target's kernel headers
4. Build headers and run-time objects using the minimal GCC
5. Build a secondary C-only GCC using the target's EGLIBC headers
6. Build a full EGLIBC using the secondary C-only GCC
7. Build a final GCC using the full EGLIBC

Later, if you need to rebuild the tools you can skip steps 3-5, as you will
already have a usable target GLIBC and target headers to start from.

Step 1 - Commands to build and install a cross-binutils:

$ sudo apt-get build-dep binutils=2.21.90.20111004-1
$ apt-get source binutils=2.21.90.20111004-1
... MAKE ANY CHANGES TO BINUTILS HERE ...
$ ( cd binutils-2.21.90.20111004 && TARGET="${MYTRIPLET}" \
          dpkg-buildpackage -b -us -uc ) 2>&1 | tee binutils-build.log
$ sudo dpkg -i "binutils-${MYTRIPLET}_2.21.90.20111004-1_amd64.deb"

Step 2 - Commands to download and modify Linux Kernel headers:

$ sudo apt-get build-dep linux-2.6=3.0.0-3
$ apt-get source linux-2.6=3.0.0-3
$ pushd linux-2.6-3.0.0
$ vim debian/config/defines ## Add "${MYARCH}" to list
$ cp -a debian/config/{powerpc,"${MYARCH}"}
$ vim debian/config/"${MYARCH}"/* ## Make it work
... MAKE ANY OTHER CHANGES TO THE LINUX KERNEL HERE ...
## THE NEXT COMMAND FAILS WITH A WARNING MESSAGE, THAT IS OK!!!
$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -T"debian/control-real"

Step 2.1 - You will need to edit the "debian/rules
" Makefile in the Debianized linux-2.6 sources and append the following text at the end:

.PHONY: FORCE
binary-%: FORCE
dh_testdir
$(MAKE) -f debian/rules.gen binary-$*_$(DEB_HOST_ARCH)

Step 2.2 - Commands to actually build Linux Kernel headers:

$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -Tsource
$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -Tbinary-libc-dev \
          --as-root -rfakeroot
$ popd

FIXME: Right now 2011/10/05 dpkg in testing/unstable does not support installing packages from foreign architectures, so we have to run the following command to forcibly convert the linux-libc-dev package with dpkg-cross.  Eventually "foreign-architecture ${MYARCH}" in your /etc/dpkg/dpkg.cfg file will be enough to make "dpkg -i" just work.

Step 2.3 - Command to install the Linux Kernel Headers:

$ sudo dpkg-cross -M -a "${MYARCH}" -i \
        "linux-libc-dev_3.0.0-3_${MYARCH}.deb"

Alternate Step 2.3 (UNTESTED) - Eventually (with full multiarch dpkg, already in some Ubuntu) you will just use this command:

$ sudo dpkg -i "linux-libc-dev_3.0.0-3_${MYARCH}.deb"

FIXME: The GCC stage1 build has a bug right now (Debian Bug #644439) which causes the gcc-4.6-powerpc-linux-gnuspe package to falsely depend on a libgcc-powerpcspe-cross package which is not part of a stage1 build.

Until that bug is fixed upstream, you will need to apply this patch to the GCC sources or it won't install (included in commands below)  (UPDATE!)  The old patch breaks native builds, use this one instead.

Step 3 - Commands to build a minimal stage1 GCC:

$ sudo apt-get build-dep gcc-4.6=4.6.1-13
$ apt-get source gcc-4.6=4.6.1-13
$ curl -o 'gcc-4.6-cross-stage1-dep-fix-v2.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644439;msg=30;att=1'
$ patch -d gcc-4.6-4.6.1 -p1 <gcc-4.6-cross-stage1-dep-fix-v3.patch
... MAKE ANY OTHER CHANGES TO GCC HERE ...
$ ( cd gcc-4.6-4.6.1 && DEB_GCC_TARGET="${MYARCH}" DEB_STAGE=stage1 \
    dpkg-buildpackage -b -us -uc ) 2>&1 | tee gcc-4.6-stage1.log
$ sudo dpkg -i "cpp-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"

FIXME: The EGLIBC debian package does not actually have a way to build this as a part of a minimal package file (see Debian Bug #644546), so we need to apply this patch that lets us build stage1 packages.

Step 4 - Commands to build EGLIBC headers and run-time objects

$ sudo apt-get build-dep eglibc=2.13-21
$ apt-get source eglibc=2.13-21
$ curl -o 'eglibc-2.13-cross-stage1-support.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644546;msg=5;att=1'
$ patch -d eglibc-2.13 -p1 <eglibc-2.13-cross-stage1-support.patch
... MAKE ANY OTHER CHANGES TO GLIBC HERE ...
$ ( cd eglibc-2.13 && DEB_GCC_VERSION=-4.6 DEB_STAGE=stage1 \
    dpkg-buildpackage -a"${MYARCH}" -b -us -uc ) 2>&1 \
    | tee eglibc-stage1.log

FIXME: Right now 2011/10/05 dpkg in testing/unstable does not support installing packages from foreign architectures, so we have to run the following command to forcibly convert the libc6-dev package with dpkg-cross.  Eventually "foreign-architecture ${MYARCH}" in your /etc/dpkg/dpkg.cfg file will be enough to make "dpkg -i" just work.

FIXME2: Additionally, this package currently has excess dependencies on nonexistent libc6 and libc-dev-bin packages.  Hopefully when the Debian EGLIBC maintainers merge the stage1-support patch, they will also fix the dependencies for stage1 packages.  Until then, the dpkg-cross command includes "-X libc6 -X libc-dev-bin".

Step 4.1 - Command to install the EGLIBC headers and run-time objects:

$ sudo dpkg-cross -M -a "${MYARCH}" -X libc6 -X libc-dev-bin \
-i "libc6-dev_2.13-21_${MYARCH}.deb"

Alternate Step 4.1 (UNTESTED, BROKEN) - Eventually (with full multiarch dpkg, and the dependencies fixed) you will just use this command:

$ sudo dpkg -i "libc6-dev_2.13-21_${MYARCH}.deb"

Step 5 - Commands to build and install a stage2 GCC:

$ sudo apt-get build-dep gcc-4.6=4.6.1-13
$ apt-get source gcc-4.6=4.6.1-13
... MAKE ANY CHANGES TO GCC HERE ...
$ ( cd gcc-4.6-4.6.1 && DEB_GCC_TARGET="${MYARCH}" DEB_STAGE=stage2 \
  dpkg-buildpackage -b -us -uc ) 2>&1 | tee gcc-4.6-stage2.log
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}-base_4.6.1-13_amd64.deb"
$ sudo dpkg -i "libgcc1-${MYARCH}-cross_4.6.1-13_all.deb"
$ sudo dpkg -i "libgcc1-dbg-${MYARCH}-cross_4.6.1-13_all.deb"
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"
$ sudo dpkg -i "cpp-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"

FIXME: The same EGLIBC patch as is used above still needs to be applied or the DEB_GCC_VERSION variable will not work (see Debian Bug #644546).

Step 6 - Commands to build a final EGLIBC (UPDATE: Known to be broken):

$ sudo apt-get build-dep eglibc=2.13-21
$ apt-get source eglibc=2.13-21
$ curl -o 'eglibc-2.13-cross-stage1-support.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644546;msg=5;att=1'
$ patch -d eglibc-2.13 -p1 <eglibc-2.13-cross-stage1-support.patch
... MAKE ANY OTHER CHANGES TO GLIBC HERE ...
$ ( cd eglibc-2.13 && DEB_GCC_VERSION=-4.6 \
      dpkg-buildpackage -a"${MYARCH}" -b -us -uc ) 2>&1 \
      | tee eglibc-final.log

FIXME:  Unfortunately I'm stuck here for today...  This EGLIBC build fails with errors about undefined references to "__stack_chk_guard", presumably because it's being built with a stripped-down GCC that does not have those features enabled.  I can only assume that either the stage2 GCC needs to have some additional features turned on or there needs to be an extra round of EGLIBC build after the full GCC is working.

(UPDATE: See "Debian PowerPC e500v2 port, part 2" for further progress.)

Cheers,

Kyle Moffett