2011-10-06

How to bootstrap a new Debian port

Ok... This is is kind of a funny way to start a blog, but it's the first really serious content that I felt like posting, so here goes!

I've been working on a Debian port for the e500v2 architecture "powerpcspe", which has a rather odd way of doing floating point compared to normal PowerPC systems.

Just recently I've submitted a bunch of Debian bugs for different pieces of the bootstrap process that were rather tedious, in the hope that someone doing a new architecture port in the future might benefit. To have a central place for the entire procedure (ideally one that Google can find easily), I'm posting it all here.


The first step is to figure out your architecture triplet and get that added to dpkg as a new target.  For example, the "powerpcspe" architecture uses the target triplet "powerpc-linux-gnuspe".

Temporarily you can modify the following files to get a local system to recognize a new architecture, but before you ship you should get the dpkg maintainers to add it upstream:
/usr/share/dpkg/archtable
/usr/share/dpkg/cputable
/usr/share/dpkg/triplettable
So let's set these shell variables:
$ unset MYARCH MYTRIPLET TARGET
$ MYARCH=powerpcspe
$ MYTRIPLET="$(dpkg-architecture -a"$MYARCH" -qDEB_HOST_GNU_TYPE)"
NOTE: You will see a warning that your GNU system type does not match your GCC system type.  This is OK, and part of cross-compiling.

Then you need to bootstrap a cross-compiler toolchain with that triplet:

  1. Build binutils (assembler, linker, etc)
  2. Perform a headers-only Linux Kernel build
  3. Build a minimal C-only GCC using just the target's kernel headers
  4. Build headers and run-time objects using the minimal GCC
  5. Build a secondary C-only GCC using the target's EGLIBC headers
  6. Build a full EGLIBC using the secondary C-only GCC
  7. Build a final GCC using the full EGLIBC

Later, if you need to rebuild the tools you can skip steps 3-5, as you will
already have a usable target GLIBC and target headers to start from.

Step 1 - Commands to build and install a cross-binutils:
$ sudo apt-get build-dep binutils=2.21.90.20111004-1
$ apt-get source binutils=2.21.90.20111004-1
... MAKE ANY CHANGES TO BINUTILS HERE ...
$ ( cd binutils-2.21.90.20111004 && TARGET="${MYTRIPLET}" \
          dpkg-buildpackage -b -us -uc ) 2>&1 | tee binutils-build.log
$ sudo dpkg -i "binutils-${MYTRIPLET}_2.21.90.20111004-1_amd64.deb"
Step 2 - Commands to download and modify Linux Kernel headers:
$ sudo apt-get build-dep linux-2.6=3.0.0-3
$ apt-get source linux-2.6=3.0.0-3
$ pushd linux-2.6-3.0.0
$ vim debian/config/defines ## Add "${MYARCH}" to list
$ cp -a debian/config/{powerpc,"${MYARCH}"}
$ vim debian/config/"${MYARCH}"/* ## Make it work
... MAKE ANY OTHER CHANGES TO THE LINUX KERNEL HERE ...
## THE NEXT COMMAND FAILS WITH A WARNING MESSAGE, THAT IS OK!!!
$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -T"debian/control-real"
Step 2.1 - You will need to edit the "debian/rules
" Makefile in the Debianized linux-2.6 sources and append the following text at the end:

.PHONY: FORCE
binary-%: FORCE
dh_testdir
$(MAKE) -f debian/rules.gen binary-$*_$(DEB_HOST_ARCH)
Step 2.2 - Commands to actually build Linux Kernel headers:
$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -Tsource
$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -Tbinary-libc-dev \
          --as-root -rfakeroot
$ popd
FIXME: Right now 2011/10/05 dpkg in testing/unstable does not support installing packages from foreign architectures, so we have to run the following command to forcibly convert the linux-libc-dev package with dpkg-cross.  Eventually "foreign-architecture ${MYARCH}" in your /etc/dpkg/dpkg.cfg file will be enough to make "dpkg -i" just work.

Step 2.3 - Command to install the Linux Kernel Headers:
$ sudo dpkg-cross -M -a "${MYARCH}" -i \
        "linux-libc-dev_3.0.0-3_${MYARCH}.deb"
Alternate Step 2.3 (UNTESTED) - Eventually (with full multiarch dpkg, already in some Ubuntu) you will just use this command:
$ sudo dpkg -i "linux-libc-dev_3.0.0-3_${MYARCH}.deb"
FIXME: The GCC stage1 build has a bug right now (Debian Bug #644439) which causes the gcc-4.6-powerpc-linux-gnuspe package to falsely depend on a libgcc-powerpcspe-cross package which is not part of a stage1 build.

Until that bug is fixed upstream, you will need to apply this patch to the GCC sources or it won't install (included in commands below)  (UPDATE!)  The old patch breaks native builds, use this one instead.

Step 3 - Commands to build a minimal stage1 GCC:
$ sudo apt-get build-dep gcc-4.6=4.6.1-13
$ apt-get source gcc-4.6=4.6.1-13
$ curl -o 'gcc-4.6-cross-stage1-dep-fix-v2.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644439;msg=30;att=1'
$ patch -d gcc-4.6-4.6.1 -p1 <gcc-4.6-cross-stage1-dep-fix-v3.patch
... MAKE ANY OTHER CHANGES TO GCC HERE ...
$ ( cd gcc-4.6-4.6.1 && DEB_GCC_TARGET="${MYARCH}" DEB_STAGE=stage1 \
    dpkg-buildpackage -b -us -uc ) 2>&1 | tee gcc-4.6-stage1.log
$ sudo dpkg -i "cpp-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"
FIXME: The EGLIBC debian package does not actually have a way to build this as a part of a minimal package file (see Debian Bug #644546), so we need to apply this patch that lets us build stage1 packages.

Step 4 - Commands to build EGLIBC headers and run-time objects
$ sudo apt-get build-dep eglibc=2.13-21
$ apt-get source eglibc=2.13-21
$ curl -o 'eglibc-2.13-cross-stage1-support.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644546;msg=5;att=1'
$ patch -d eglibc-2.13 -p1 <eglibc-2.13-cross-stage1-support.patch
... MAKE ANY OTHER CHANGES TO GLIBC HERE ...
$ ( cd eglibc-2.13 && DEB_GCC_VERSION=-4.6 DEB_STAGE=stage1 \
    dpkg-buildpackage -a"${MYARCH}" -b -us -uc ) 2>&1 \
    | tee eglibc-stage1.log
FIXME: Right now 2011/10/05 dpkg in testing/unstable does not support installing packages from foreign architectures, so we have to run the following command to forcibly convert the libc6-dev package with dpkg-cross.  Eventually "foreign-architecture ${MYARCH}" in your /etc/dpkg/dpkg.cfg file will be enough to make "dpkg -i" just work.

FIXME2: Additionally, this package currently has excess dependencies on nonexistent libc6 and libc-dev-bin packages.  Hopefully when the Debian EGLIBC maintainers merge the stage1-support patch, they will also fix the dependencies for stage1 packages.  Until then, the dpkg-cross command includes "-X libc6 -X libc-dev-bin".

Step 4.1 - Command to install the EGLIBC headers and run-time objects:
$ sudo dpkg-cross -M -a "${MYARCH}" -X libc6 -X libc-dev-bin \
-i "libc6-dev_2.13-21_${MYARCH}.deb"
Alternate Step 4.1 (UNTESTED, BROKEN) - Eventually (with full multiarch dpkg, and the dependencies fixed) you will just use this command:
$ sudo dpkg -i "libc6-dev_2.13-21_${MYARCH}.deb"
Step 5 - Commands to build and install a stage2 GCC:
$ sudo apt-get build-dep gcc-4.6=4.6.1-13
$ apt-get source gcc-4.6=4.6.1-13
... MAKE ANY CHANGES TO GCC HERE ...
$ ( cd gcc-4.6-4.6.1 && DEB_GCC_TARGET="${MYARCH}" DEB_STAGE=stage2 \
  dpkg-buildpackage -b -us -uc ) 2>&1 | tee gcc-4.6-stage2.log
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}-base_4.6.1-13_amd64.deb"
$ sudo dpkg -i "libgcc1-${MYARCH}-cross_4.6.1-13_all.deb"
$ sudo dpkg -i "libgcc1-dbg-${MYARCH}-cross_4.6.1-13_all.deb"
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"
$ sudo dpkg -i "cpp-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"

FIXME: The same EGLIBC patch as is used above still needs to be applied or the DEB_GCC_VERSION variable will not work (see Debian Bug #644546).

Step 6 - Commands to build a final EGLIBC (UPDATE: Known to be broken):
$ sudo apt-get build-dep eglibc=2.13-21
$ apt-get source eglibc=2.13-21
$ curl -o 'eglibc-2.13-cross-stage1-support.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644546;msg=5;att=1'
$ patch -d eglibc-2.13 -p1 <eglibc-2.13-cross-stage1-support.patch
... MAKE ANY OTHER CHANGES TO GLIBC HERE ...
$ ( cd eglibc-2.13 && DEB_GCC_VERSION=-4.6 \
      dpkg-buildpackage -a"${MYARCH}" -b -us -uc ) 2>&1 \
      | tee eglibc-final.log
FIXME:  Unfortunately I'm stuck here for today...  This EGLIBC build fails with errors about undefined references to "__stack_chk_guard", presumably because it's being built with a stripped-down GCC that does not have those features enabled.  I can only assume that either the stage2 GCC needs to have some additional features turned on or there needs to be an extra round of EGLIBC build after the full GCC is working.

(UPDATE: See "Debian PowerPC e500v2 port, part 2" for further progress.)

Cheers,
Kyle Moffett

No comments:

Post a Comment