I need an ARMv6 cross compiler for my B and B+ pis, primarily for building the kernel since this requires many hours on the pi itself and sometimes requires trial and error to get right.
I would also appreciate some explanation of how to use the final product to compile a kernel for the pi, since cross-compiling is a new topic for me.
The host platform is also linux.
[This was written before the Pi 2 came out, which has an ARMv7 processor, and most linux distributions have an ARMv7 cross-compiler package available. That will save you going through all this. But ARMv6 cross-compilers, needed for the previous A/B/+ versions, are much harder to find and that's what this is about.]
Building such a toolchain from scratch yourself is a daunting task, but you can get a good combination of flexibility and convenience with the assistance of crosstool-NG. "Toolchain" here refers to the compiler, linker, libraries, and other bits and pieces necessary to the task. Two other fundamental terms involved in cross-compiling are target, referring to the platform you are compiling for (in this case, the raspberry pi) and host, referring to the platform you are compiling on (in this case, linux x86 or x86_64).
Crosstool-NG itself (the tool used to build the toolchain) is not available as a binary package as far as I am aware, but you may want to check your distro's repositories first just in case. It is easy to build anyway; first download the latest version of the source from the front page of the crosstool-NG site (at the time of this writing, 1.19.0 [Later: Also works w/ 1.20.0 to build 3.18.x kernel]) and unpack it somewhere:
> tar -xjf crosstool-ng-1.19.0.tar.bz2
> cd crosstool-ng-1.19.0
There are not any particularly interesting configuration options; you can have a look via ./configure --help. The default installation root is /usr/local; this can be set with --prefix=.
> ./configure
Configure will bail out with an informative error if you do not have the necessary pre-requisites installed. These include:
-dev or -devel package)All of which are available for any normal GNU/Linux distribution; if you can't find them via your package manager, you can ask for help on Unix & Linux Exchange.
> make
> make install
Crosstool-NG is now ready to use.
Presuming you installed in-path (/usr/local should be, you can check with echo $PATH), you will now have the ct-ng command available. This is a multiple function command which takes a second parameter; for a list try ct-ng with no parameters.
Create a directory to work in. You will probably want to keep this around, since while it will not be necessary in order to use the toolchain (which ends up somewhere else), it will save you a lot of time if you want to re-build it for whatever reason. cd into this directory and begin with ct-ng menuconfig. The interface below should appear.
This is very similar to the menuconfig interface available for configuring a linux kernel build, and like that, it also leaves behind (and initially loads, if it exists) a .config file which will be used for building the toolchain. Fortunately, there are not nearly so many options to go through here as there are with the kernel, and most of the defaults are suitable. I won't go over all of them, just those that are essential. The <Help> button in the bottom bar (navigate this with Tab) will give you information about each highlighted option.
You don't have to change anything in the Paths and misc options directory, but there are a few things worth noting. For example, it is a good idea to uncheck Strip down all toolchain executables in case there is a problem and you want to run them in a debugger with symbols available. You can also specify a Local tarballs directory; this is where crosstool-NG will look for toolchain component source tarballs. Anything you don't have will be downloaded; the first time you use it you can let it download everything and won't have to think about this.
Inside the second menu, Target options, the default Target Architecture is arm, which is what we want. Beneath that you can add a descriptive suffix used for labelling the build tools, v6j is a good idea because it corresponds to what we are going to set under the Architecture Level option below that, which is armv6j. It is important to get that correct, as is the Use specific FPU option which should be vfp.
The other defaults are fine, but make sure Floating point is (hardware(FPU)) and Use the MMU is ticked.
There's nothing that needs doing under Toolchain options, but you might want to select Enable nls at the bottom and set Tuple's vendor string to something like "rpi" instead of the default "unknown".
Under Operating System change Target OS to linux. This affects whether or not some important subsequent options are presented, so do it now if you intend to follow along while reading.
Since one of the goals here is to build the kernel, it might seem that bare-metal makes more sense, but this is not the case (one reason for that can be found in the help for the bare-metal option). Once you select that, more options will appear, but you don't need to change any of them.
There is a long list of possibilities under Linux kernel version. Note that you do not need an exact match for either the kernel you are using or the one you intend to compile (as close as possible should be fine), but at the bottom of the list there is a custom tarball or directory option. If you don't pick that, crosstool-NG will download a vanilla source tree tarball from kernel.org and use that. Again, everything (?) will still work, but if you do intent to subsequently build a kernel this is a bit pointless since you won't be able to use that source -- you need the raspberry pi version from github. So it is worthwhile to get that first and choose the custom option. You will then find a Path to custom source, tarball or directory option which you can fill in with the path to your git cloned source tree directory. If this is not correct, the build will fail.
Nothing in Binary utilities needs to be changed. Under C compiler, you should untick Optimize gcc libs for size; this is only significant if you are doing, e.g., static builds for tiny embedded platforms (which the rpi is not), and it may negatively impact performance if you do want to make static binaries. You also have the opportunity to include a C++ cross-compiler in this menu (you want one of those, don't you?), and a Java cross-compiler, but beware this is not the openjdk javac that's standard on linux, it's GCC's gjc.
When picking a gcc version, do not use 4.8.0 - 4.8.2. These have a bug which may lead to filesystem corruption if you use this to build a kernel.
The next menu is C-library. There are three choices, you probably want either glibc or eglibc -- it should not make much difference. Debian derived linux distros such as raspbian use the latter, but this does not mean you must with your cross-compiler. It is worth noting that eglibc is now being merged back into glibc; 2.19 will be the final release (see "EGLIBC 2.19 Branch Created" under News).
For the record, before writing this I built two versions of the toolchain which differed in their choice of C library and linux kernel:
One used vanilla kernel 3.10.2 and glibc 2.17.
The other used custom directory (the rpi github 3.13.6 kernel) and the 'trunk' eglibc option. The latter is available at the bottom of the C-library->eglibc version list. This ended up downloading version 2.19. If you want to do this, you'll need subversion available (all normal linux distros will have a package for this) and to either open your firewall for tcp and udp connections to port 3690, or else tick C-library->use http:// instead of svn://.
Neither of these had a problem cross-compiling a working kernel for the pi from the github 3.13.6 branch. However I have had problems with some kernels built using eglibc and now always favor glibc, so that's what I recommend.
One possible minor concern with regard to the C library is the library version (note that eglibc versions, being "based on" glibc, have parallel numbers) and ABI compatibility. At the time of this writing Raspbian is still 2.13 based, and the crosstool-NG options range from 2.13 to 2.17. If your primary goal is building the kernel (or other statically linked, standalone binaries), you probably want the highest version since there are no binary compatibility concerns there (and you can now skip reading down to the horizontal line). With regard to dynamically linked userland executables, (e)glibc strives for backward compatibility, meaning if you cross-compile against an older version than the one used on the target platform, there should not be a problem. However, the same is not necessarily true for executables compiled against a newer version than on the target, since forward compatibility cannot be a serious goal in the development of the library. Backward compatibility is not guaranteed perfect, but glibc does report statistics regarding this. Those statistics focus on the API (i.e. compatibility of source code, specifically the use of GNU extensions, not standard C code -- the significance of which depends on what you are building), but there are details regarding binary compatibility there too. Presumably these are consequences of API incompatibility.
Meaning realistically there are no worries in the backward compatibility department. If you are paranoid about forward compatibility because you are building bleeding edge GNU C optimized code and aren't sure what (e)glibc version is running on your pi, it's the same as ldd --version.
Nothing in Debug facilities or Companion Libraries is a requirement. When you are done perusing options, Exit and save the configuration.
Crosstool-NG needs to download the source for everything -- or everything you don't already have, as already noted. This could be as much as 200 MB and will take a few minutes. To begin:
> ct-ng build
In the same directory as the .config file left behind by ct-ng menuconfig. Including the download time (~15 minutes), my first build took half an hour on a quadcore 3.4 Ghz machine. For the second build, I filled in Paths and misc options->Local tarballs directory so that the only thing required to download was the eglibc trunk. That build took 20 minutes.
The build leaves behind a .build directory containing the unpacked sources and compiled tools required to build the toolchain. It does not by default contain the toolchain -- that will be in $HOME/x-tools (you can change this via Paths and misc options->Prefix directory). The .build directory is no longer needed, but you may want to hang onto it anyway -- it also contains the downloaded tarballs, which can be reused.
Cross-compiling the kernel is actually easy, because it does not require you to build any prerequisites to parallel the rpi's userland. Source packages which use make -- including the kernel -- make use of a few environment variables. The easiest way to exploit these is to define them in a file. This will be sourced by a shell, so we don't need a shebang:
export ARCH=arm
# Replace this with your target tuple.
# If you aren't sure, look in $HOME/x-tools.
export TARGET=armv6j-raspbian-linux-gnueabi
# Replace this with the path to your x-tools directory.
export TOOL_DIR=/home/goldilocks/x-tools/$TARGET
export SYSROOT=$TOOL_DIR/$TARGET/sysroot/
export CROSS_COMPILE=$TOOL_DIR/bin/$TARGET-
export INSTALL_MOD_PATH=$SYSROOT
When you fill this in, don't leave out the pattern in the reuse of variables, e.g., your $TOOL_DIR should end with $TARGET, and so on. Then source it:
> source make.env
Don't execute it directly because the variables won't be set in your current shell. make.env should be whatever you called that file. Check:
> echo $ARCH
arm
That's critical when you run make menuconfig for the kernel. Even if you are using a pre-defined .config, menuconfig will pre-process it assuming the target is your host architecture and make modifications accordingly. There is a clue about this at the top of the interface.
If that says "Linux/x86" instead, you haven't set $ARCH. Exit without saving and try again. You can do this without sourcing the make.env script by using:
make ARCH=arm menuconfig
However, when you do the build, you will need $CROSS_COMPILE and $INSTALL_MOD_PATH set appropriately. Without the former, make will use your system's compiler and the result will not be usable.
When you subsequently run make modules_install, you will find them in $INSTALL_MOD_PATH/lib/modules.
For more background on configuring, compiling, and installing the kernel, see here. Once you have the source from github, you can skip steps 1, 4, and 6 as they do not apply on the raspberry pi.
