Distcc and ccache

Dec 16, 2013

4 min read

Jun 30, 2023 16:33 EEST

I separate the compiling target with the portnumber.

Overview for ports using crosscompile and distcc

CPU	System	Port
x86	i686-pc-linux-gnu	3632 (standard)
AMD64	x86_64-pc-linux-gnu	3633

Distcc with AMD64

We use port 3633 for AMD64 compiling, so we need to adapt the configuration:

emerge distcc

Now set the right options in the file /etc/conf.d/distccd:

# /etc/conf.d/distccd: config file for /etc/init.d/distccd

DISTCCD_OPTS=""

# this is the distccd executable 
DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file
DISTCCD_PIDFILE="/var/run/distccd/distccd.pid"

# set this option to run distccd with extra parameters
# Default port is 3632.  For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3633"

# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL  [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical

DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"

# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --listen option
# for increased security. You can specify an IP to permit connections 
# from or a CIDR mask
# --listen accepts only a single IP
# --allow is now mandatory as of distcc-2.18.
# example:  --allow 192.168.0.0/24
# example:  --allow 192.168.0.5 --allow 192.168.0.150
# example:  --listen 192.168.0.2
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/16"
#DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.2"

# set this for niceness
# Default is 15
DISTCCD_NICE="15"

Edit /etc/distcc/hosts:

localhost:3633
or
distcc-config --set-hosts localhost:3633

Then start distcc with:

/etc/init.d/distccd restart

Distcc with X86

I ran here a 64-bit gentoo coompiled for AMD64 (x86_64) so we need to do some configuration to compile for x86. At first we need to generate a port overlay, edit /etc/make.conf:

# To get crossdev running
PORTDIR_OVERLAY="/usr/local/portage"

Create the new directory:

mkdir /usr/local/portage

Install cross-compile environment:

emerge crossdev
crossdev -t i686-pc-linux-gnu

Adapt the configuration:

cd /etc/init.d
cp distccd distccd-x86

Edit /etc/init.d/distccd-x86:

#!/sbin/runscript
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.18/init,v 1.1 2004/10/12 17:21:43 lisa Exp $

depend() {
       need net
       use ypbind
}

start() {
       [ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null

       ebegin "Starting distccd"
       chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null
       TMPDIR="${TMPDIR}" \
       PATH="/usr/i686-pc-linux-gnu/gcc-bin/4.1.1:${PATH}" \
       /sbin/start-stop-daemon --start --quiet --startas ${DISTCCD_EXEC} \
       --pidfile ${DISTCCD_PIDFILE} -- \
       --pid-file ${DISTCCD_PIDFILE} -N ${DISTCCD_NICE} --user distcc \
       ${DISTCCD_OPTS}

       eend $?
}

stop() {
       ebegin "Stopping distccd"
       start-stop-daemon --stop --quiet --pidfile "${DISTCCD_PIDFILE}"
       rm -f "${DISTCCD_PIDFILE}"
       eend $?
}
cd /etc/conf.d
cp distccd distccd-x86

Now set the right options in the file /etc/conf.d/distccd-x86:

# /etc/conf.d/distccd: config file for /etc/init.d/distccd

DISTCCD_OPTS=""

# this is the distccd executable 
DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file
DISTCCD_PIDFILE="/var/run/distccd/distccd-x86.pid"

# set this option to run distccd with extra parameters
# Default port is 3632.  For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3632"

# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL  [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical

DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"

# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --listen option
# for increased security. You can specify an IP to permit connections 
# from or a CIDR mask
# --listen accepts only a single IP
# --allow is now mandatory as of distcc-2.18.
# example:  --allow 192.168.0.0/24
# example:  --allow 192.168.0.5 --allow 192.168.0.150
# example:  --listen 192.168.0.2
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/16"
#DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.2"

# set this for niceness
# Default is 15
DISTCCD_NICE="15"

To configure an i686 computer to use our AMD64 for compiling edit the /etc/distcc/hosts on the i686 computer:

192.168.0.151
or
distcc-config --set-hosts localhost 192.168.0.151

Use localhost only if localhost is fast enough elsewhere omit it.

Start distcc-x86 now with:

/etc/init.d/distccd-x86 restart

To start it at computer restart enter:

rc-update add distccd-x86 default

Get status

View which computer compile on our:

DISTCC_DIR="/var/tmp/portage/.distcc" distccmon-text 1
DISTCC_DIR="/var/tmp/portage/.distcc" distccmon-gui

Enable distcc with Gentoo

Edit /etc/make.conf to enable distcc and ccache for portage:

# for 7 CPUs or PCs (CPU*2 +1)
MAKEOPTS="-j15"
FEATURES="distcc ccache"
CCACHE_SIZE="2G"

Dec 16	Atmel	2 min read
Dec 16	Compiling a kernel	1 min read
Dec 16	LVM2	3 min read
Dec 16	Layman	1 min read
Dec 16	Portage	1 min read