Debian PowerPC e500v2 port, part 8

This is part of the Debian PowerPC e500 porting effort, the series on my blog starts at: "How to bootstrap a new Debian port"

Sorry that it's been so long since the last post... I was able to get my board booted from NFS, but after that things got complicated.

I spent quite a while trying to re-cross-bootstrap more packages; but GCC 4.6 refused to build in a reverse-cross configuration (IE: build a compiler on amd64 that runs on powerpcspe and targets powerpcspe).

Unfortunately, there's a bit of a nasty dependency loop that prevented me from doing a native build either.  In order to build a new GCC 4.6 with multiarch support, I needed to install a new multiarch libc6... except that multiarch libc6 "Breaks" my old GCC 4.4 (which doesn't have multiarch).

I ended up doing a native build of a new multiarch libc6 on my NFS-booted system and forcibly installing that (which leaves my GCC 4.4 broken).  Thankfully libc6 creates a file in "/etc/ld.so.conf.d/" that GCC uses to help find libraries and I could work around the runtime crt*.o objects like this:

  # cd /usr/lib
  # for i in powerpc-linux-gnuspe/*; do ln -s "$i"; done

With that I was able to build a new GCC 4.6.  Unfortunately, I still had an old gcc-defaults package (depending on gcc-4.4) and the new gcc-defaults package would not build unless I could install gcc-4.6 and gcj-jdk and a half-dozen other things I don't have.

Thankfully, with a couple more quick compatibility symlinks I could just remove "gcc" and "g++" and leave "build-essential" technically broken but have a working build environment:

  # cd /usr/bin
  # for i in *g{cc,++}-4.6; do ln -s "$i" "${i%-4.6}"; done

After that, I had a new libc6 installed, along with "multiarch-support"!  Finally!  Woohoo!!!

I upgraded dpkg, apt, aptitude, and about a half-dozen other libraries and tools that all need multiarch these days...

Now I need to figure out what I need to install/build/rebuild in order to make sbuild work again, then I should be able to get a new "buildd" server going.

And of course once I have one going I have enough spare hardware to start about 18 of them.

I'll hopefully post again later today with more progress.

Cheers,
Kyle Moffett

Debian PowerPC e500v2 port, part 2

Well, I've been working again on the Debian powerpcspe port and there is a lot more progress to be had! (This series starts at: "How to bootstrap a new Debian port")

I left off last time after EGLIBC failed to build with errors about undefined references to "__stack_chk_guard".  It turns out the problem is that EGLIBC does not correctly detect the presence of support for the "-fstack-protector" GCC option, as per Debian bug #644662.  The fix is to download and apply this patch, which fixes the configure script test.

The second issue I ran into is that dpkg-gensymbols is not very good at finding all the relevant multi-arch libraries while trying to build packages, as per Debian bug #595144.  The fix there is an existing patch to the "dpkg" source package by Steve Langasek; it changes the "libdpkg-perl" binary package to fix the dpkg-gensymbols tool.

As a side note, it turns out that my previous gcc-4.6-cross-stage1-dep-fix-v2.patch broke non-bootstrap-stage builds of GCC.  As a result, I've posted a new version to Debian bug #644439; this one should handle native, cross, and cross-bootstrap builds fine although it's still undergoing compile tests here.

The final issue with the EGLIBC build is that it tries to build the "locales-all" package by executing the built binaries, which of course fails for cross-compiled EGLIBC (see Debian bug #644771).  The fix is trivial, simply don't build "locales-all" when cross-compiling, found in this patch.

So before we can retry building EGLIBC, we need to build an updated libdpkg-perl for our build host:

Step 6.1 - Building and installing a patched libdpkg-perl:

$ sudo apt-get build-dep dpkg=1.16.1
$ sudo apt-get source dpkg=1.16.1
$ curl -o 'Dpkg-Shlibs.pm-multiarch-search-paths.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595144;msg=37;att=1'
$ patch -d dpkg-1.16.1 -p1 <Dpkg-Shlibs.pm-multiarch-search-paths.patch
$ ( cd dpkg-1.16.1 && dpkg-buildpackage -b -us -uc ) 2>&1 | tee dpkg.log
$ sudo dpkg -i libdpkg-perl_1.16.1_all.deb

Going back to the EGLIBC commands I listed previously for step 6, here is an updated version:

Step 6.2 - Commands to build a stage2 EGLIBC:

$ sudo apt-get build-dep eglibc=2.13-21
$ apt-get source eglibc=2.13-21
$ curl -o 'eglibc-2.13-cross-stage1-support.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644546;msg=5;att=1'
$ curl -o 'eglibc-fix-ssp-detection.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644662;msg=5;att=1'
$ curl -o 'eglibc-disable-cross-locales-all.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644771;msg=5;att=1'
$ patch -d eglibc-2.13 -p1 <eglibc-2.13-cross-stage1-support.patch
$ patch -d eglibc-2.13 -p1 <eglibc-fix-ssp-detection.patch
$ patch -d eglibc-2.13 -p1 <eglibc-disable-cross-locales-all.patch... MAKE ANY OTHER CHANGES TO EGLIBC HERE ...
$ ( cd eglibc-2.13 && DEB_GCC_VERSION=-4.6 \
      dpkg-buildpackage -a"${MYARCH}" -B -us -uc ) 2>&1 \
      | tee eglibc-stage2.log

Once the build finishes, it will have produced a bunch of deb files, but you only actually need the libc6 and libc6-dev packages.  As before, we will eventually be able to install foreign architecture packages but for now we have to forcibly dpkg-cross them (with a few ignored dependencies).

Step 6.3 - Commands to install the stage2 EGLIBC:

$ sudo dpkg-cross -M -a "${MYARCH}" -X libc-bin -X libc-dev-bin -i \
      "libc6_2.13-21_${MYARCH}.deb" 
"libc6-dev_2.13-21_${MYARCH}.deb"

After this completes we have a fully usable C library for our target architecture, and it's time for the final GCC cross-compiler build with all features enabled.

Step 7.1 - Commands to build a final full-featured GCC:

$ sudo apt-get build-dep gcc-4.6=4.6.1-13
$ apt-get source gcc-4.6=4.6.1-13
... MAKE ANY CHANGES TO GCC HERE ...
$ ( cd gcc-4.6-4.6.1 && DEB_GCC_TARGET="${MYARCH}" DEB_STAGE=stage2 \
  dpkg-buildpackage -b -us -uc ) 2>&1 | tee gcc-4.6-stage2.log

The final GCC build leaves about 23 packages lying around in the directory, and you should probably install all of them.

Step 7.2 - Command to install the final GCC:

$ sudo dpkg -i $(dcmd --deb gcc-4.6_4.6.1-13_*.changes)

NOTE: DO NOT UPLOAD ANY OF THE PACKAGES YOU BUILT!!!  They intentionally deviate from standard in various ways due to the cross-building process and you should wait until you can rebuild them natively before uploading.

Cheers,
Kyle Moffett

How to bootstrap a new Debian port

Ok... This is is kind of a funny way to start a blog, but it's the first really serious content that I felt like posting, so here goes!

I've been working on a Debian port for the e500v2 architecture "powerpcspe", which has a rather odd way of doing floating point compared to normal PowerPC systems.

Just recently I've submitted a bunch of Debian bugs for different pieces of the bootstrap process that were rather tedious, in the hope that someone doing a new architecture port in the future might benefit. To have a central place for the entire procedure (ideally one that Google can find easily), I'm posting it all here.


The first step is to figure out your architecture triplet and get that added to dpkg as a new target.  For example, the "powerpcspe" architecture uses the target triplet "powerpc-linux-gnuspe".

Temporarily you can modify the following files to get a local system to recognize a new architecture, but before you ship you should get the dpkg maintainers to add it upstream:

/usr/share/dpkg/archtable
/usr/share/dpkg/cputable
/usr/share/dpkg/triplettable

So let's set these shell variables:

$ unset MYARCH MYTRIPLET TARGET
$ MYARCH=powerpcspe
$ MYTRIPLET="$(dpkg-architecture -a"$MYARCH" -qDEB_HOST_GNU_TYPE)"

NOTE: You will see a warning that your GNU system type does not match your GCC system type.  This is OK, and part of cross-compiling.

Then you need to bootstrap a cross-compiler toolchain with that triplet:

1. Build binutils (assembler, linker, etc)
2. Perform a headers-only Linux Kernel build
3. Build a minimal C-only GCC using just the target's kernel headers
4. Build headers and run-time objects using the minimal GCC
5. Build a secondary C-only GCC using the target's EGLIBC headers
6. Build a full EGLIBC using the secondary C-only GCC
7. Build a final GCC using the full EGLIBC

Later, if you need to rebuild the tools you can skip steps 3-5, as you will
already have a usable target GLIBC and target headers to start from.

Step 1 - Commands to build and install a cross-binutils:

$ sudo apt-get build-dep binutils=2.21.90.20111004-1
$ apt-get source binutils=2.21.90.20111004-1
... MAKE ANY CHANGES TO BINUTILS HERE ...
$ ( cd binutils-2.21.90.20111004 && TARGET="${MYTRIPLET}" \
          dpkg-buildpackage -b -us -uc ) 2>&1 | tee binutils-build.log
$ sudo dpkg -i "binutils-${MYTRIPLET}_2.21.90.20111004-1_amd64.deb"

Step 2 - Commands to download and modify Linux Kernel headers:

$ sudo apt-get build-dep linux-2.6=3.0.0-3
$ apt-get source linux-2.6=3.0.0-3
$ pushd linux-2.6-3.0.0
$ vim debian/config/defines ## Add "${MYARCH}" to list
$ cp -a debian/config/{powerpc,"${MYARCH}"}
$ vim debian/config/"${MYARCH}"/* ## Make it work
... MAKE ANY OTHER CHANGES TO THE LINUX KERNEL HERE ...
## THE NEXT COMMAND FAILS WITH A WARNING MESSAGE, THAT IS OK!!!
$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -T"debian/control-real"

Step 2.1 - You will need to edit the "debian/rules
" Makefile in the Debianized linux-2.6 sources and append the following text at the end:

.PHONY: FORCE
binary-%: FORCE
dh_testdir
$(MAKE) -f debian/rules.gen binary-$*_$(DEB_HOST_ARCH)

Step 2.2 - Commands to actually build Linux Kernel headers:

$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -Tsource
$ dpkg-buildpackage -a"${MYARCH}" -b -us -uc -Tbinary-libc-dev \
          --as-root -rfakeroot
$ popd

FIXME: Right now 2011/10/05 dpkg in testing/unstable does not support installing packages from foreign architectures, so we have to run the following command to forcibly convert the linux-libc-dev package with dpkg-cross.  Eventually "foreign-architecture ${MYARCH}" in your /etc/dpkg/dpkg.cfg file will be enough to make "dpkg -i" just work.

Step 2.3 - Command to install the Linux Kernel Headers:

$ sudo dpkg-cross -M -a "${MYARCH}" -i \
        "linux-libc-dev_3.0.0-3_${MYARCH}.deb"

Alternate Step 2.3 (UNTESTED) - Eventually (with full multiarch dpkg, already in some Ubuntu) you will just use this command:

$ sudo dpkg -i "linux-libc-dev_3.0.0-3_${MYARCH}.deb"

FIXME: The GCC stage1 build has a bug right now (Debian Bug #644439) which causes the gcc-4.6-powerpc-linux-gnuspe package to falsely depend on a libgcc-powerpcspe-cross package which is not part of a stage1 build.

Until that bug is fixed upstream, you will need to apply this patch to the GCC sources or it won't install (included in commands below)  (UPDATE!)  The old patch breaks native builds, use this one instead.

Step 3 - Commands to build a minimal stage1 GCC:

$ sudo apt-get build-dep gcc-4.6=4.6.1-13
$ apt-get source gcc-4.6=4.6.1-13
$ curl -o 'gcc-4.6-cross-stage1-dep-fix-v2.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644439;msg=30;att=1'
$ patch -d gcc-4.6-4.6.1 -p1 <gcc-4.6-cross-stage1-dep-fix-v3.patch
... MAKE ANY OTHER CHANGES TO GCC HERE ...
$ ( cd gcc-4.6-4.6.1 && DEB_GCC_TARGET="${MYARCH}" DEB_STAGE=stage1 \
    dpkg-buildpackage -b -us -uc ) 2>&1 | tee gcc-4.6-stage1.log
$ sudo dpkg -i "cpp-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"

FIXME: The EGLIBC debian package does not actually have a way to build this as a part of a minimal package file (see Debian Bug #644546), so we need to apply this patch that lets us build stage1 packages.

Step 4 - Commands to build EGLIBC headers and run-time objects

$ sudo apt-get build-dep eglibc=2.13-21
$ apt-get source eglibc=2.13-21
$ curl -o 'eglibc-2.13-cross-stage1-support.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644546;msg=5;att=1'
$ patch -d eglibc-2.13 -p1 <eglibc-2.13-cross-stage1-support.patch
... MAKE ANY OTHER CHANGES TO GLIBC HERE ...
$ ( cd eglibc-2.13 && DEB_GCC_VERSION=-4.6 DEB_STAGE=stage1 \
    dpkg-buildpackage -a"${MYARCH}" -b -us -uc ) 2>&1 \
    | tee eglibc-stage1.log

FIXME: Right now 2011/10/05 dpkg in testing/unstable does not support installing packages from foreign architectures, so we have to run the following command to forcibly convert the libc6-dev package with dpkg-cross.  Eventually "foreign-architecture ${MYARCH}" in your /etc/dpkg/dpkg.cfg file will be enough to make "dpkg -i" just work.

FIXME2: Additionally, this package currently has excess dependencies on nonexistent libc6 and libc-dev-bin packages.  Hopefully when the Debian EGLIBC maintainers merge the stage1-support patch, they will also fix the dependencies for stage1 packages.  Until then, the dpkg-cross command includes "-X libc6 -X libc-dev-bin".

Step 4.1 - Command to install the EGLIBC headers and run-time objects:

$ sudo dpkg-cross -M -a "${MYARCH}" -X libc6 -X libc-dev-bin \
-i "libc6-dev_2.13-21_${MYARCH}.deb"

Alternate Step 4.1 (UNTESTED, BROKEN) - Eventually (with full multiarch dpkg, and the dependencies fixed) you will just use this command:

$ sudo dpkg -i "libc6-dev_2.13-21_${MYARCH}.deb"

Step 5 - Commands to build and install a stage2 GCC:

$ sudo apt-get build-dep gcc-4.6=4.6.1-13
$ apt-get source gcc-4.6=4.6.1-13
... MAKE ANY CHANGES TO GCC HERE ...
$ ( cd gcc-4.6-4.6.1 && DEB_GCC_TARGET="${MYARCH}" DEB_STAGE=stage2 \
  dpkg-buildpackage -b -us -uc ) 2>&1 | tee gcc-4.6-stage2.log
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}-base_4.6.1-13_amd64.deb"
$ sudo dpkg -i "libgcc1-${MYARCH}-cross_4.6.1-13_all.deb"
$ sudo dpkg -i "libgcc1-dbg-${MYARCH}-cross_4.6.1-13_all.deb"
$ sudo dpkg -i "gcc-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"
$ sudo dpkg -i "cpp-4.6-${MYTRIPLET}_4.6.1-13_amd64.deb"

FIXME: The same EGLIBC patch as is used above still needs to be applied or the DEB_GCC_VERSION variable will not work (see Debian Bug #644546).

Step 6 - Commands to build a final EGLIBC (UPDATE: Known to be broken):

$ sudo apt-get build-dep eglibc=2.13-21
$ apt-get source eglibc=2.13-21
$ curl -o 'eglibc-2.13-cross-stage1-support.patch' \
  'http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644546;msg=5;att=1'
$ patch -d eglibc-2.13 -p1 <eglibc-2.13-cross-stage1-support.patch
... MAKE ANY OTHER CHANGES TO GLIBC HERE ...
$ ( cd eglibc-2.13 && DEB_GCC_VERSION=-4.6 \
      dpkg-buildpackage -a"${MYARCH}" -b -us -uc ) 2>&1 \
      | tee eglibc-final.log

FIXME:  Unfortunately I'm stuck here for today...  This EGLIBC build fails with errors about undefined references to "__stack_chk_guard", presumably because it's being built with a stripped-down GCC that does not have those features enabled.  I can only assume that either the stage2 GCC needs to have some additional features turned on or there needs to be an extra round of EGLIBC build after the full GCC is working.

(UPDATE: See "Debian PowerPC e500v2 port, part 2" for further progress.)

Cheers,

Kyle Moffett