Now that multi-core workstations are the norm, many developers use make's parallel-build feature to reduce compile times. Adding a -j2 flag to a make command divides the build into two separate processes -- allowing each process to execute on a different core more or less -- cutting the compile time almost in half. For example, cross compiling a kernel for an Atmel AT91SAM9260 on a 1.8 Ghz Athlon X2 laptop took 00:06:08. Using two parallel processes cut that time to 00:03:16.

Distcc is a utility that sends parallel-build processes to other distcc hosts. The distributed processes run at the lowest priority so you can use your co-workers' computers as build cluster nodes without grinding them to a halt. Most desktop distributions have a distcc package or you can get the source code from distcc.samba.org. On Ubuntu it is as simple as:

• apt-get install distcc
  
• Edit /etc/default/distcc to enable the daemon, configure the allowed networks and listening interface
  
• /etc/init.d/distcc start
  

To use distcc with the native toolchain set CC="distcc" in your make command. To cross compile add the path to the target architecture's gcc executable. Note that each distcc host must have the same cross compiler installed. To use the Timesys toolchain to build an armv5l kernel uImage with distcc:

make -j4 ARCH=arm CROSS_COMPILE=armv5l-linux- \
CC="distcc /opt/timesys/toolchains/armv5l-linux/bin/armv5l-linux-gcc" \
DISTCC_HOSTS="localhost 192.168.3.159" uImage

This example used two dual core machines to build the kernel in 00:02:32.

Stepping through the make options:

• -j4 use four parallel processes
  
• ARCH=arm processor architecture of build target
  
• CROSS_COMPILE=armv5l-linux- prepend armv5l-linux- to local build commands. For example use armv5l-linux-ld instead of the native toolchain's ld
  
• CC="distcc /opt/timesys/toolchains/armv5l-linux/bin/armv5l-linux-gcc" use the armv5l compiler with distcc
  
• DISTCC_HOSTS="localhost 192.168.3.159" use these hosts as distcc targets
  
• uImage build target
  

Using two processes per core can provide better results in some cases. Keep in mind that as you add processes the main build machine can become bogged down by juggling what it should compile versus what it should send out. At one point in this experiment there were six dual core hosts available (including localhost) and the build was divided into 24 processes. The result was a 00:01:22 build time. By removing localhost from DISTCC_HOSTS and using 20 processes the build time improved to 00:01:06.

Experiment with the amount of distcc hosts and parallel processes to find the best build time for your project.