aboriginal: www/documentation.html annotate

annotate www/documentation.html @ 526:ad14395e1f0c

Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite...

author	Rob Landley <rob@landley.net>
date	Mon, 08 Dec 2008 00:13:49 -0600
parents	16adbffb58b0
children	4828f4597ede

rev	line source
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1 <!--#include file="header.html" -->
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	3 <h1>Documentation for Firmware Linux</h1>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	4
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	5 <p>Note, this documentation is currently under construction. This is three
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	6 files concatenated together, with their contents in the process of being
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	7 reorganized and rewritten. Some of it's up to date, some isn't.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	8
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	9 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	10 <li><a href="#what_is_it">What is Firmware Linux?</a></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	11 <li><a href="#how_system_image">How do I use system images?</a></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	12 <li><a href="#how_build_source">How do I build my own customized system images from source code?</a></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	13 <li><a href="#how_implemented">How is Firmware Linux implemented?</a></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	14 <li><a href="#why">Why do things this way?</a></li>
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	15 <li><a href="#new_platform">Adding a new target platform</a></li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	16 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	17
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	18 <hr />
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	19 <a name="what_is_it"><h1>What is Firmware Linux?</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	20
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	21 <p>Firmware Linux is an embedded Linux build system, designed to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	22 replace cross compiling with native compiling under emulation. It provides
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	23 an easy way to get started with embedded development, building your own code
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	24 against uClibc, and testing it on non-x86 hardware platforms.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	25
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	26 <p>This documentation uses the name "Firmware Linux' (or abbreviation "FWL") to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	27 refer to the <a href=downloads>build system</a>, and calls the output of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	28 the build a "<a href=downloads/binaries/system-image>system image</a>".
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	29 The build system is implemented as a series of bash scripts and configuration
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	30 files which compile a Linux development system for the specified target and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	31 package it into a bootable binary image.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	32
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	33 <p>These system images provide a simple native Linux development
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	34 environment for a target, built from seven source packages: busybox, uClibc,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	35 gcc, binutils, make, bash, and the Linux kernel. This is the smallest
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	36 environment that can rebuild itself entirely from source code, and thus the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	37 minimum a host system must cross compile in order to create a fully independent
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	38 native development environment for a target.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	39
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	40 <p>Booting a development system image under an emulator such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	41 <a href=http://bellard.org/qemu/>QEMU</a> allows fully
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	42 native builds for supported target platforms to be performed on cheap and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	43 powerful commodity PC hardware. Building and installing additional packages
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	44 (zlib, bison, openssl...) within a system image can provide an arbitrarily
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	45 complex native development environment, without resorting to any additional
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	46 cross compiling.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	47
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	48 <p>FWL currently includes full support for arm, mips, powerpc, x86 and x86-64
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	49 targets, and partial support for sh4, mips, and sparc. The goal for the FWL
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	50 1.0 release is to support every target QEMU can emulate in "system" mode.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	51
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	52 <p>Firmware Linux is licensed under GPL version 2. Its component packages are
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	53 licensed under their respective licenses (mostly GPL and LGPL).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	54
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	55 <h2>Optional extra complexity</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	56
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	57 <p>Intermediate stages of the build (such as the cross compiler and the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	58 unpackaged root filesystem directory) may also be useful to Linux developers,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	59 so tarballs of them are saved during the build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	60
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	61 <p>By default the build cross-compiles some optional extra packages (toybox,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	62 distcc, uClibc++) and preinstalls them into the target filesystem. This is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	63 just a convenience; these packages build and install natively within the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	64 minimal development system image just fine.<!-- TODO: experimentally confirm
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	65 that, make it configurable, add genext2fs and strace to the list? -->)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	66
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	67 <hr />
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	68 <a name="how_system_image"><h1>Using system images</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	69
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	70 <p>If you want to jump straight to building your own software natively for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	71 embedded targets, you can <a href=downloads/binaries>download a prebuilt
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	72 binary image</a> instead of running the build scripts to produce your own.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	73
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	74 <p>Here are the different types of output produced by the build:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	75
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	76 <h2>system-image-*.tar.bz2</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	77
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	78 <p>System images boot a complete linux system under an emulator. Each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	79 system-image tarball contains an ext2 root filesystem image, a Linux kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	80 configured to run under the emulator <a href=http://bellard.org/qemu/>QEMU</a>,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	81 and a run-emulator.sh script.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	82
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	83 <p>The steps to test boot a system image under qemu 0.9.1 are:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	84 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	85 <li>install QEMU 0.9.1 or later</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	86 <li>download the appropriate <a href=downloads/image>prebuilt binary tarball</a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	87 for the target you're interested in</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	88 <li>extract it: <b>tar -xvjf system-image-$TARGET.tar.bz2</b></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	89 <li>cd into it: <b>cd system-image-$TARGET</b></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	90 <li>execute it: <b>./run-emulator.sh</b></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	91 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	92
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	93 <p>This boots the system image under the appropriate emulator, with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	94 the emulated Linux's /dev/console hooked to stdin and stdout of the emulator
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	95 process. (I.E. the shell prompt the script gives you after the boot messages
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	96 scroll past is for a shell running inside the emulator. This lets you pipe
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	97 the output of other programs into the emulator, and capture the emulator's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	98 output.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	99
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	100 <p>Type "cat /proc/cpuinfo" to confirm you're running in the emulator, then
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	101 play around and have fun. Type "exit" when done.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	102
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	103 <p>Inside a system image, you generally wget source code from some URL and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	104 compile it. (For example, you can wget the FWL build, extract it, and run it
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	105 inside one of its own system images to trivially prove it can rebuild itself.)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	106 If you run a web server on your host's loopback interface, you an access it
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	107 inside QEMU using the special address "10.0.2.2". Example build scripts
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	108 are available in the /usr/src directory.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	109
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	110 <h3>Extra space and speed</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	111
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	112 <p>The system images by themselves are fairly small (64 megabytes), and don't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	113 have a lot of scratch space for building or installing other packages. If a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	114 file named "<b>hdb.img</b>" exists in the current directory, run-emulator.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	115 will automatically designate it as a second virtual hard drive and attempt to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	116 mount the whole unpartitioned device on <b>/home</b> inside the emulator.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	117
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	118 <p>Some optional command line arguments to run-emulator.sh provide extra
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	119 space and extra speed for compiling more software:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	120
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	121 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	122 <li><p><b>--make-hdb $MEGABYTES</b> - if the hard drive image to mount on /home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	123 doesn't already exist, create a sparse file of the indicated size and format
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	124 it ext3.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	125
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	126 <li><p><b>--with-hdb $FILENAME</b> - use specified $FILENAME from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	127 the host as the hard drive image to mount on the emulated system's /home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	128 (instead of the default "hdb.img"). Fail if it doesn't exist, unless
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	129 --make-hdb was also specified.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	130
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	131 <li><p><b>--with-distcc $CC_PATH</b> - enable the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	132 <a href="#distcc_trick">distcc accelerator trick</a>. This option provides
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	133 the path to an appropriate cross compiler directory, so run-emulator.sh can
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	134 launch a distcc daemon on the host's loopback device configured to call that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	135 cross compiler, and configure the emulated system to call out to that cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	136 compiler through distcc.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	137 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	138
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	139 <p>Running an armv4l system image with the cross compiler
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	140 installed in the user's home directory, using a hard drive image in the user's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	141 home directory (to be created with a size of 2 gigabytes if it doesn't already
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	142 exist) might look like:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	143
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	144 <blockquote><pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	145 <b>./run-emulator.sh --make-hdb 2048 --with-hdb ~/blah.img --with-distcc ~/cross-compiler-armv4l</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	146 </pre></blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	147
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	148 <h2>mini-native-*.tar.bz2</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	149
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	150 <p>These <a href=downloads/mini-native>tarballs</a> contain the same root
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	151 filesystem as the corresponding system images, just in an archive
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	152 instead of packaged into a filesystem image.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	153
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	154 <p>If you want to boot your own system image on real hardware instead of an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	155 emulator, the appropriate mini-native tarball is a good starting point. If
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	156 all you want is a native uClibc development environment for your host, try:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	157
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	158 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	159 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	160 <b>chroot mini-native-x86_64 /usr/chroot-setup.sh</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	161 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	162 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	163
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	164 <p>The boot script /usr/qemu-setup.sh or /usr/chroot-setup.sh performs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	165 minimal setup for the appropriate environment, mounting /proc and /sys and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	166 such. It starts a single shell prompt, and automatically cleans up when that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	167 process exits.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	168
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	169 <p>If you're interested in building a more complex development environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	170 within this one (adding zlib and perl and such before building more complicated
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	171 packages), the best way to learn how is to read
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	172 <a href=http://www.linuxfromscratch.org/lfs/view/6.4/>Linux
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	173 From Scratch</a>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	174
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	175 <p>Note that mini-native is just one potential filesystem layout; the FWL
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	176 build scripts have several other configurations available when you build from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	177 source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	178
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	179 <h2>cross-compiler-*.tar.bz2</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	180
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	181 <p>The cross compilers created during the FWL build are relocatable C compilers
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	182 for each target platform. The primary reason for offering each cross compiler
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	183 as a downloadable binary is to implement the <a href="#distcc_trick">distcc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	184 accelerator trick</a>. Using them to cross compile additional software is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	185 supported, but not recommended.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	186
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	187 <p>If you'd like to use one for something other than distcc, this documentation
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	188 mostly assumes you already know how. Briefly:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	189
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	190 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	191 <li>download the appropriate cross-compiler-$TARGET.tar.bz2</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	192 <li>extract it somewhere (doesn't matter where)</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	193 <li>add the resulting cross-compiler-$TARGET/bin subdirectory to your $PATH</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	194 <li>either use $TARGET-gcc as your compiler, or set your $CROSS_COMPILE prefix
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	195 to "$TARGET-" with a trailing dash.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	196 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	197
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	198 <p>Also, stock up on asprin and clear a space to beat your head against; you'll
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	199 need both. See <a href="#why_cross_compiling_sucks">why cross compiling
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	200 sucks</a> for more details.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	201
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	202 <p>Note that although this cross compiler has g++, it doesn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	203 have uClibc++ in its lib or include subdirectories, which is required to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	204 build most c++ programs. If you need extra libraries, it's up to you to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	205 cross-compile and install them into those directories.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	206
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	207 <a name="how_build_source"><h1>How do I build my own customized system images
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	208 from source code?</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	209
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	210 <p>To build your own root filesystem and system images from source code,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	211 download and run <a href=downloads>the FWL build scripts</a>. You'll
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	212 probably want to start with the most recent <a href=downloads>release
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	213 version</a>, although once you've got the hang of it you might want to follow
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	214 the <a href=/hg/firmware>development version</a>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	215
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	216 <p>For a quick start, download the tarball, extract it, cd into it, and run
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	217 "<b>./build.sh</b>". This script takes one argument, which is the target to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	218 build for. Run it with no arguments to list available targets.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	219
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	220 <p>This should produce all the tarballs listed in the previous section in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	221 the "build" directory. To perform a clean build, "rm -rf build" and re-run
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	222 build.sh.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	223
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	224 <h2>How building from source works</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	225
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	226 <p>The build system is a series of shell scripts which download, compile,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	227 install, and use the appropriate source packages to generate a system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	228 image. These shell scripts are designed to be easily read and modified,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	229 acting both as tools to perform a build and as documentation on how to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	230 build these packages.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	231
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	232 <p>The <b>build.sh</b> script is a simple wrapper which calls the following
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	233 other scripts in sequence:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	234 <ol>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	235 <li>download.sh</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	236 <li>host-tools.sh</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	237 <li>cross-compiler.sh $TARGET</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	238 <li>mini-native.sh $TARGET</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	239 <li>package-mini-native.sh $TARGET</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	240 </ol>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	241
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	242 <p>In theory, the stages are othogonal. If you have an existing cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	243 compiler, you can add it to the $PATH and skip cross-compiler.sh. Or you
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	244 can use _just_ cross-compiler.sh to create a cross compiler, and then go build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	245 something else with it. The host-tools.sh stage can often be skipped
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	246 entirely.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	247
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	248 <h3>Build stages</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	249
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	250 <p>The following files control the individual stages of the build. Each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	251 may be called individually, from in the top level directory of FWL:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	252
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	253 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	254 <li><p><b>download.sh</b> - Download source packages from the web.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	255
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	256 <p>This script does not take an arguments. It's a series of calls to a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	257 download function (defined in sources/include.sh) that checks if an existing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	258 copy of the tarball matching a defined $SHA1 sum exists in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	259 <b>sources/packages</b> directory, and if not uses wget to fetch it from the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	260 $URL (or else from a series of fallback mirrors). A blank value for $SHA1
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	261 will accept any file as correct, ignoring its contents.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	262
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	263 <p>After downloading all tarballs, the function <b>cleanup_oldfiles</b> deletes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	264 any unused files from sources/packages (generally previous versions left over
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	265 after a package upgrade while using the development version of the FWL
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	266 build scripts).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	267
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	268 <p>Running this stage with the argument "--extract-all" will extract all
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	269 the tarballs at once, to populate the cache used by setupfor. (This is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	270 primarily used to avoid race conditions when building multiple architectures
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	271 in parallel with build-all-targets.sh. This is an esoteric internal
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	272 detail you can safely ignore if you're not doing that.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	273 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	274
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	275 <li><p><b>host-tools.sh</b> - Set up a known environment on the host</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	276
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	277 <p>This script does not take an arguments. In theory this is an optional step,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	278 and may be omitted, as the binaries produced by this script are not included in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	279 any of the output tarballs.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	280
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	281 <p>This script populates the <b>build/host</b> directory with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	282 host versions of the busybox and toybox command line tools (the same ones
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	283 that the target's eventual root filesystem will contain), plus symlinks to the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	284 host's compiler toolchain.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	285
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	286 <p>This allows the calling scripts to trim the $PATH to point to just this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	287 one directory, which serves several purposes:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	288
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	289 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	290
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	291 <li><p><b>Isolation</b> - This prevents the ./configure stages of the source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	292 packages from finding and including unexpected dependencies on random things
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	293 installed on the host.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	294
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	295 <li><p><b>Portability</b> - Using a known set of command line utilities
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	296 insulates the build from variations in the host's Linux distribtion (such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	297 Ubuntu's /bin/echo lacking suport for the -e option).</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	298
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	299 <li><p><b>Testing</b> - It ensures the resulting system can rebuild itself
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	300 under itself, since the initial build was done with the same tools we
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	301 install into the target's root filesystem. The initial build acts as a smoke
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	302 test of most of the packages used to create the resulting system, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	303 restricting $PATH ensures that no necessary commands are missing. (Variation
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	304 can still show up between x86/arm/powerpc versions, of course.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	305
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	306 <p>It also moves most failures to the beginning. If anything's going to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	307 break, it's usually the host-tools build. After that runs, we're mostly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	308 in a known and tested state.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	309
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	310 <li><p><b>Dependency tracking</b> - If we don't explicitly know everything
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	311 we need to build ourselves in the first place, we can't be sure we added
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	312 it to the final system to get a self-hosting environment.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	313 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	314
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	315 <p>A secondary purpose of host-tools.sh is to build packages (such as distcc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	316 and genext2fs) which might not be installed on the host system. Some of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	317 these aren't needed by build.sh but may be used by later run-emulator.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	318 invocations (such as the ./run-build-image.sh script).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	319
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	320 <p>Note that this script does not attempt to build qemu, due to the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	321 unreasonable requirement of installing gcc 3.x on the host. The FWL build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	322 scripts do not use qemu (except as an optional test at the end of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	323 cross-compiler.sh which is skipped if qemu is not available). You will need
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	324 to install qemu (or another emulator, or find real hardware) to use the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	325 resulting system images, but they should build just fine without it.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	326
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	327 <p>This stage is optional. You don't need to run this stage if you don't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	328 want to. If the build/host directory doesn't exist (or doesn't contain
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	329 a "busybox" executable), the build will use the host's original $PATH.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	330 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	331
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	332 <li><p><b>cross-compiler.sh</b> - Build a cross compiler for the target, for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	333 use by mini-native.sh and the distcc accelerator.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	334
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	335 <p>In order to build binaries for the target, the build must first create
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	336 a cross compiler to build those target binaries with. This script creates
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	337 that cross compiler. If you already have a cross compiler, you can
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	338 supply it here (the easy way is to create a build/cross-compiler-$TARGET/bin
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	339 directory and put "$TARGET-gcc" style symlinks in it) and skip this step.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	340
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	341 <p>This script takes one argument: the architecture to build for. It
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	342 produces a cross compiler that runs on the host system and produces binaries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	343 that run on the target system. This cross compiler is created using the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	344 source packages binutils, gcc, uClibc, the Linux kernel headers, and a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	345 compiler wrapper to make the compiler relocatable.<</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	346
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	347 <p>The reason for the compiler wrapper is that by default, gcc hardwires
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	348 lots of absolute paths into itself, and thus only runs properly in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	349 directory it was built in. The compiler wrapper rewrites its command line
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	350 to prevent gcc from using its built-in (broken) path logic.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	351
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	352 <p>The build requires a cross-compiler even if the host and target system use
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	353 the same processor because the host and target may use different C libraries.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	354 If the host has glibc and the target uses uClibc, then the (dynamically linked)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	355 target binaries the compiler produces won't run on the host. (Target binaries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	356 that won't run on the host are what distinguishes cross-compiling from native
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	357 compiling. Different processors are just one reason for it: glibc vs uClibc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	358 is another, ELF vs binflat or a.out executable format is a third...)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	359
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	360 <p>This script produces produces a working cross compiler in the build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	361 directory, and saves a tarball of it as "cross-compiler-$TARGET.tar.bz2"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	362 for use outside the build system. This cross compiler is fully relocatable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	363 (because of the compiler wrapper), so any normal user can extract it into their
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	364 home directory, add cross-compiler-$TARGET/bin to their $PATH, and run
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	365 $TARGET-gcc to create target binaries.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	366 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	367
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	368 <li><p><b>mini-native.sh</b> - Use the cross compiler to create
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	369 a minimal native build environment for the target platfrom.<p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	370
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	371 <p>This script takes one argument: the architecture to build for.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	372
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	373 <p>This script uses the cross compiler found at
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	374 build/cross-compiler-$ARCH/bin (with $ARCH- prefixes) to build a root
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	375 filesystem for the target, as well as a target Linux kernel configured for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	376 use with qemu. A usable cross compiler is left in the build directory by
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	377 the cross-compiler.sh script, or you can install your own.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	378
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	379 <p>The basic root filesystem consists of busybox and uClibc. If the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	380 configuration variable NATIVE_TOOLCHAIN is set (this is enabled by
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	381 default), this script adds a native compiler to the target, consisting of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	382 linux kernel headers, gcc, binutils, make, and bash. It also adds distcc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	383 to potentially distribute work to cross compilers living outside the emulator.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	384 This provides a minimal native development environment, which may be expanded
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	385 by building and installing more packages under the existing root filesystem.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	386 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	387
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	388 <li><p><b>package-mini-native.sh</b> - Create an ext2 filesystem image
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	389 of the native root filesystem.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	390
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	391 <p>This script takes one argument: the architecture to package.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	392
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	393 <p>This uses genext2fs to create an ext2 filesystem image from the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	394 build/mini-native-$ARCH directory left by running mini-native.sh, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	395 creates a system-image-tarball containing the result. It first compiles
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	396 genext2fs and adds it to build/host if the host system hasn't already got a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	397 copy.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	398
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	399 <p>This script also generates a run-emulator.sh script to call the appropriate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	400 emulator, using the architecture's configuration information.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	401 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	402
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	403 <li><p><b>run-from-build.sh</b> - Runs a system image you compiled from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	404 source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	405
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	406 <p>Calls run-emulator.sh in the appropriate build/system-image-$TARGET
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	407 directory, with a 2 gigabyte <b>hdb.img</b> for /home and distcc connected
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	408 to build/cross-compiler/$TARGET. Between runs it calls e2fsck on the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	409 system image's root filesystem.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	410
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	411 <p>This is not technically a build stage, as it isn't called from build.sh,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	412 but it's offered as a convenience for users. It uses the existing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	413 cross-compiler and system-image directories in build/ and doesn't mess
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	414 with the tarballs that were created from them.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	415 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	416 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	417
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	418 <p>The following generally aren't called directly, but are used by the rest of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	419 the build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	420
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	421 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	422 <li><p><b>config</b> - User definable configuration variables</li></p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	423
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	424 <p>This file contains environment variables which you can set to customize
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	425 the FWL build process. Setting any of these variables to a nonblank value
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	426 changes the build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	427
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	428 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	429 <li><p><b>NATIVE_TOOLCHAIN</b> - This tells mini-native.sh to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	430 include a compiler toolchain (binutils, gcc, bash, make, and distcc).
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	431 Without this, it builds a small uClibc/busybox system. This is the only
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	432 variable enabled by default in config.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	433
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	434 <p>Setting NATIVE_TOOLCHAIN="headers" will leave the libc and kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	435 header files in the appropriate include directory, for use by some other native
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	436 compiler. Building and installing additional tools (such as "make", or a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	437 compiler such as pcc, llvm/clang, or tinycc) then becomes your problem.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	438 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	439
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	440 <li><p><b>NATIVE_TOOLSDIR</b> - This tells mini-native.sh to change the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	441 directory layout to conform to a Linux From Scratch "intermediate" system, with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	442 everything under a /tools directory. (This provides a cleaner environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	443 for creating a new completely customized system at the root level.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	444 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	445
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	446 <li><p><b>RECORD_COMMANDS</b> - Records all command lines used to build each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	447 package.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	448
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	449 <p>This inserts a logging wrapper in the $PATH which logs the command lines
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	450 used by the build. Afterwards, the script
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	451 "sources/toys/report_recorded_commands.sh" can generate a big report on which
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	452 commands were used to build each package for each architecture. To get a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	453 single list of the command names used by everything, do:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	454
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	455 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	456 <p>echo $(find build -name "cmdlines.*" \| xargs awk '{print $1}' \| sort -u)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	457 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	458
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	459 <p>(Note: this will miss things which which call executables at absolute values
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	460 instead of checking $PATH, but the only interesting ones so far are the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	461 #!/bin/bash type lines at the start of shell scripts.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	462 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	463
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	464 <li><p><b>CROSS_BUILD_STATIC</b> - Tells cross-compiler.sh to statically link
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	465 all binaries in the cross compiler toolchain it creates.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	466
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	467 <p>The prebuilt binary versions in the download directory are statically linked
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	468 against uClibc, by building a mini-native environment and re-running the build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	469 under that with CROSS_BUILD_STATIC=1. The sources/build-all-targets.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	470 script can do this automatically with the "--use-static-host $TARGET"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	471 argument. (Requires QEMU installed.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	472
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	473 <li><p><b>PREFERRED_MIRROR</b> - Tells download.sh to try to download
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	474 packages from this URL first, before falling back to the normal mirror list.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	475 For example, "PREFERRED_MIRROR=http://impactlinux.com/fwl/mirror".</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	476
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	477 <li><p><b>USE_TOYBOX</b> - Tells the host-tools.sh and mini-native.sh to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	478 install the <a href=http://impactlinux.com/code/toybox>toybox</a> implementation
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	479 of commands (where available) instead of the busybox versions. This is an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	480 alternate (simpler) implementation of many commands.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	481
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	482 <p>Currently FWL always uses the toybox "patch" command, because the busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	483 version can't apply patches at offsets.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	484 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	485
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	486 <li><p><b>USE_UNSTABLE</b> - Lists packages to build alternate "unstable"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	487 versions for.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	488
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	489 <p>The value of this config entry is a comma separate list of packages.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	490
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	491 <p>Many packages in download.sh have an UNSTABLE= tag providing a URL to an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	492 alternate version. Generally these link to newer versions, often unstable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	493 development versions, for testing purposes.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	494
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	495 <p>In addition to changing the download location, using alternate versions of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	496 packages prepends an "alt-" in front of the package name in various places
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	497 (such as the patches from the sources/patches directory and the configuration
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	498 files used from sources/targets). It changes the behavior of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	499 "download" and "setupfor" shell functions.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	500 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	501
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	502 <li><p><b>USE_COLOR</b> - Color code the various build stages.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	503
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	504 <p>Enabling this provides a quick visual indicator of which build stage is in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	505 progress.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	506
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	507 <p>This is disabled by default both because its utility is a matter of taste,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	508 and because finding a half-dozen different colors that work on both white and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	509 black backgrounds is hard, and gnome-terminal can't produce an actual black
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	510 background. (In its default palette, "black" is a fairly light grey.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	511 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	512 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	513 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	514
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	515 <li><p><b>sources/build-all-targets.sh</b> - Build all supported targets
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	516 at once.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	517
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	518 <p>This performs a similar function to build.sh, but for all targets instead
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	519 of just one. It can build targets in parallel with the --fork option, logs the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	520 output of the various build stages, and generates a README.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	521
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	522 <p>This script populates a second output directory, buildall, with its output.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	523 This is probably only of interest to FWL's developers.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	524 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	525 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	526
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	527 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	528 <a name="how_implemented"><h1>How is Firmware Linux implemented?</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	529
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	530 <h2>Directory layout</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	531
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	532 <p>The top level directory of FWL contains the user interface of FWL (scripts
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	533 the user is expected to call or edit), and the "sources" directory containing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	534 code that isn't expected to be directly called by end users.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	535 Important directories under sources include:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	536
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	537 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	538 <li><p><b>sources/targets</b> - Configuration information for each target.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	539
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	540 <p>Adding a new target to FWL involves creating a new directory under here
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	541 (which determines the name of the target), and adding two miniconfig files
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	542 (for linux and uClibc), and a "details" file defining environment variables.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	543 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	544
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	545 <li><p><b>sources/packages</b> - Source tarballs for the packages to be built.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	546 This directory starts empty, and is populated by download.sh.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	547
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	548 <li><p><b>sources/native</b> - This directory hierarchy is copied into the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	549 target verbatim (under /usr). It contains the boot script and some sample
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	550 source code.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	551
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	552 <li><p><b>sources/toys</b> - Build utilities, mostly original code written for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	553 FWL. Not necessarily specific to this project, but can't be downloaded from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	554 somewhere else.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	555 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	556
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	557 <p>Output files from running the build scripts, and all temporary files, go
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	558 in the "build" subdirectory. This entire directory can be deleted between
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	559 builds.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	560
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	561 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	562 <li><p><b>build/sources</b> - cached copies of the extracted source tarballs,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	563 so setupfor can "cp -lfR" instead of having to re-extract and re-patch the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	564 source each time.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	565
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	566 <li><p><b>build/host</b> - Output of host-tools.sh. If this directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	567 exists and contains a "busybox" executable, include.sh will set the $PATH to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	568 point only to this directory.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	569
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	570 <li><p><b>build/temp-$TARGET</b> - Temporary directory for building each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	571 target. Feel free to delete this between runs, it should be empty unless
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	572 a build broke, in which case it has the source tree that failed to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	573 build. (The temporary directory for host-tools.sh is "host-temp", in case
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	574 someday somebody creates a $TARGET named "host".)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	575
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	576 <li><p><b>build/cross-compiler-$TARGET</b> - Output directory for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	577 cross-compiler.sh. The corresponding cross-compiler tarball is just an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	578 archive of this directory. Used by mini-native.sh.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	579
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	580 <li><p><b>build/mini-native-$TARGET</b> - Output directory for mini-native.sh.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	581 The corresponding mini-native tarball is just an archive of this directory.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	582 Used by package-mini-native.sh.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	583
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	584 <li><p><b>build/system-image-$TARGET</b> - Output directory for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	585 package-mini-native.sh. The corresponding system-image tarball is just an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	586 archive of this directory. Used by run-from-build.sh.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	587 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	588
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	589 <h2>Shared infrastructure</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	590
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	591 <p>The top level file for the behind-the-scenes plumbing is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	592 <b>sources/include.sh</b>. This script is not run directly, but is instead
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	593 included from the other scripts. It does a bunch of things:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	594
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	595 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	596 <li><p>Its command line argument is the architecture to build.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	597 If run with no arguments, it outputs all available architectures by
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	598 listing the subdirectories under sources/targets. (The special environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	599 variable $NO_ARCH tells it to skip that part; this is used by download.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	600 and host-tools.sh which are architecture independent.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	601 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	602
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	603 <li><p>It parses the "config" file at the top directory, reading in the user
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	604 defined configuration variables. (You can also supply these as environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	605 variables, if you want to specify them for just one run.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	606
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	607 <li><p>It sets several other environment variables, specifying things like the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	608 $SOURCE and $BUILD directories, and detecting the number of $CPUS.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	609
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	610 <li><p>It adjusts the $PATH. If build/host exists and contains a busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	611 executable (meaning host-tools.sh did its thing already), $PATH is set to just
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	612 that directory.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	613
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	614 <li><p>If host-tools.sh ran with $RECORD_COMMANDS, it sets the $PATH to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	615 point to the logging wrapper directory. ($WRAPPY_LOGPATH specifies where
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	616 the logging wrapper should write its log file, and $WRAPPY_REALPATH says where
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	617 to find the actual commands the logging wrapper hands off to.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	618 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	619
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	620 <p>It also reads <b>sources/functions.sh</b>, which provides shell functions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	621 used by the rest of the build, including:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	622
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	623 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	624 <li><p><b>download</b> - Used by download.sh. Calls wget if necessary, uses
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	625 sha1sum to verify the files. Treat as a fancy call to "wget".</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	626
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	627 <li><p><b>dienow</b> - abort the current script, exiting with an error message.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	628 (Can even exit from nested shell functions.) Treat as a fancy "exit".</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	629
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	630 <li><p><b>setupfor</b> - extract a source package (named in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	631 first argument) into a temporary directory, and change the current directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	632 to there. Treat as a fancy "tar -xvjf" followed by cd.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	633
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	634 <p>Source code is cached, meaning each package's source tarball is only
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	635 actually extracted and patched once (into build/sources) and the temporary
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	636 copies are directories full of hard links to the cached source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	637 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	638
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	639 <li><p><b>cleanup</b> - delete temporary copy of source code after build.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	640 Treat as a fancy "rm -rf". (If the exit code of the last command was nonzero,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	641 it calls dienow instead of deleting the source code that didn't build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	642 properly, to preserve the evidence of what went wrong.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	643 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	644
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	645 <p>Most of what include.sh does is optional. A lot of it's there to speed up
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	646 and simplify the rest of the build. (You don't really need to call "make -j"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	647 for parallel bilds, and can re-extract and re-patch source code each time
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	648 you need it rather than cacheing the extracted version.) Most of the rest is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	649 error checking, from "dienow" to the sha1sum checking in "download".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	650
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	651 <p>None of this should be important to understanding how to build or install
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	652 any of the actual packages. It just abstracts away repetitive, uninteresting
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	653 bits.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	654
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	655 <h2>Downloading source code</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	656
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	657 <p>The FWL source distribution does not include any third party source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	658 tarballs. Instead, these are downloaded by running download.sh, which calls
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	659 the shell function <b>download</b>, which calls wget as necessary. The
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	660 download.sh script is just a series of calls to the download function.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	661
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	662 <p>The first thing download.sh does is check for the --extract option,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	663 and if present set the environment variable EXTRACT_ALL, which tells
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	664 each call to the download function to call the extract shell function on
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	665 the appropriate tarball to prepopulate the source cache. (See "Extracting
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	666 source code", below.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	667
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	668 <p>Each call to the download function is controlled by the following
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	669 <b>environment variables</b>:<p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	670
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	671 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	672 <li><p><b>URL</b> - The URL from which to download this source package
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	673 into the sources/packages directory.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	674
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	675 <p>In addition to specifying a web location, this URL specifies the name of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	676 source package to fetch. If this source tarball cannot be fetched from this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	677 location, download tries to download the file from a series of fallback mirrors,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	678 most importantly http://impactlinux.com/firmware/mirror which should have every
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	679 source tarball used by the build.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	680
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	681 <p>The package name is the filename at the end of URL minus any version
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	682 information and file type extensions, so "bash-2.04b.tar.bz2" becomes "bash".
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	683 The shell function "basename" uses a rather complicated regex to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	684 extract the package name from a URL. The package name is used by things
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	685 like setupfor, allowing the build scripts to mostly ignore the versions of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	686 the packages they build.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	687
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	688 <li><p><b>SHA1</b> - The sha1sum of the source tarball to fetch.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	689
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	690 <p>If this value is blank, the sha1sum calculated from the file will be
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	691 displayed but not verified. This means any file will be accepted as correct
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	692 as long as it exists with the right name, but the build won't be able to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	693 detect corrupted or truncated files.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	694
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	695 <p>When updating to a new version of a package, a common trick is to update
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	696 the URL and blank the SHA1, run ./download.sh to fetch the new file, cut and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	697 paste the SHA1 value displayed after the download to set the SHA1 variable,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	698 and then re-run ./download.sh to confirm they match.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	699
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	700 <li><p><b>PREFERRED_MIRROR</b></p> - This contains the URL of a mirror site
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	701 to be checked _before_ downloading from the actual $URL specified in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	702 download.sh.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	703
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	704 <p>This allows download.sh to fetch some or all of its packages from a local
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	705 mirror of the files, instead of going out to the net. Any files not found
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	706 in this mirror will be fetched from the standard URL, and the fallback mirrors
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	707 as necessary.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	708
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	709 <p>(Note: inside qemu the special address 10.0.2.2 passes through connections
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	710 to 127.0.0.1 on the host, so if you run a web server on hour host's loopback
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	711 address you can easily pass source code into the emulator without going out
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	712 to an external network.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	713
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	714 <li><p><b>UNSTABLE</b> - URL to an alternate version of the file, for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	715 testing purposes.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	716
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	717 <p>This version is only downloaded when USE_UNSTABLE contains the name of this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	718 package in its package list. It doesn't fall back to check the mirror list,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	719 and is not affected by PREFERRED_MIRROR.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	720
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	721 <p>Unstable packages are saved as a tarball called "alt-$PACKAGE-0" plus the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	722 file type extension, so the name to save is based on the filename in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	723 normal $URL rather than on what the $UNSTABLE address points to. (So even if
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	724 your UNSTABLE address ends with "snapshot.tgz" or "tip.tar.bz2", it will
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	725 still wind up somewhere the rest of the build can find it.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	726 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	727
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	728 <p>At the end of download.sh is a call to the shell function cleanup_oldfiles,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	729 which deletes unused files. The include.sh snapshots the current time in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	730 the variable $START_TIME, and download calls "touch" to update the timestamp on
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	731 each file it verifies the sha1sum of. Then cleanup_oldfiles deletes every
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	732 file from sources/packages with a date older than $START_TIME.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	733
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	734 <p>Note that download updates the timestamp on stable packages when
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	735 downloading corresponding unstable stable packages, so cleanup_oldfiles
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	736 won't delete them. In this special case they're not considered "unused
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	737 files", but it won't verify their integrity or fetch them if they're not
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	738 already there. If a package is not in the USE_UNSTABLE list, download.sh won't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	739 update the timestamp on unstable source tarballs, leaving them marked as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	740 unused and thus deleted by cleanup_oldfiles.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	741
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	742 <h2>Extracting source code</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	743
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	744 <p>The function "setupfor" extracts <b>sources/packages/$PACKAGENAME-*</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	745 tarballs. (If $PACKAGENAME is found in the comma separated $USE_UNSTABLE list,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	746 the build adds an "alt-" prefix to the package name.) This populates a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	747 corresponding directory under build/sources, and applies all the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	748 <b>sources/patches/$PACKAGENAME-*.patch</b> files in alphabetical order.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	749 (So if a package has multiple patches that need to be applied in a specific
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	750 order, name them something like "bash-001-dothingy.patch",
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	751 "bash-002-next.patch" to control this.) The trailing "-" before filename
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	752 wildcards prevents collisions between things like "uClibc" and "uClibc++".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	753
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	754 <p>FWL implements source cacheing. The first call to setupfor extracts the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	755 package into build/sources, and then creates a directory of hard links in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	756 current target's build/temp-$TARGET directory with cp -lfR. Later setupfor
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	757 calls just create the directory of hard links from the existing source tree.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	758 (This is a hybrid approach between building "out of tree" and building
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	759 in-tree.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	760
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	761 <p>The <b>./download.sh --extract</b> option prepopulates the source cache,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	762 extracting and patching each source tarball. This is useful for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	763 scripts such as sources/build-all-targets.sh which perform multiple builds in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	764 parallel.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	765
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	766 <p>The reason for keeping extracted source tarballs around is that extracting
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	767 and patching tarballs is a fairly expensive operation, which uses a significant
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	768 amount of disk space and doesn't parallelize well. (It tends to be disk
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	769 limited as much as CPU limited, so trying for more parallelism wouldn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	770 necessarily help.) In addition, the same packages are repeatedly extracted:
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	771 the cross-compiler and mini-native stages use many of the same packages, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	772 some packages (such as the Linux kernel) are extracted and removed repeatedly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	773 to grab things like kernel headers separately from actually building a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	774 bootable kernel. (Also, different architectures build the exact same
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	775 packages, with the same set of patches. Even patches to fix a bug on a single
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	776 architecture are applied for all architectures; if this causes a problem, it's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	777 not a mergeable patch capable of going upstream.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	778
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	779 <h2>Building host tools</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	780
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	781 <p>The host-tools.sh script sets up the host environment. Usually the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	782 host environment is already in a usable state, but this script explicitly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	783 enumerates exactly what we need to build, and provides our own (known)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	784 versions of everything except the host compiler toolchain in the directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	785 <b>build/host</b>. Once we've finished, the $PATH can be set to just that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	786 directory.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	787
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	788 <p>The build calls seven commands from the host compiler toolchain: ar, as,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	789 nm, cc, gcc, make, and ld. All those have to be in the $PATH, so host-tools.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	790 creates symlinks to those from the original $PATH.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	791
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	792 <p>Next host-tools.sh builds toybox for the "patch" command, because busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	793 patch can't simple handle offsets and is thus extremely brittle in the face of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	794 new package versions. (This is different from "fuzz factor", which removes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	795 context lines to find a place to insert a patch, and tends to break a lot.)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	796 If USE_TOYBOX is enabled, a defconfig toybox is used and all commands are
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	797 installed.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	798
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	799 <p>Next host-tools builds a "defconfig" busybox and installs it into
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	800 build/host. This provides all the other commands the build needs.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	801
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	802 <h3>What's the minimum the build actually needs?</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	803
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	804 <p>When building a new system, environmental dependencies are a big issue.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	805 Figuring out what package needs what, and what order to build things in,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	806 is the hardest part of putting together a system.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	807
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	808 <p>Running the build without build/host calls lots of extra commands, including
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	809 perl, pod2man, flex, bison, info, m4, and so on. This is because the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	810 ./configure stages of the various packages detect optional functionality,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	811 and use it. One big reason to limit the build environment is to consistently
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	812 produce the same output files, no matter what's installed on the host.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	813
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	814 <p>The minimal list of commands needed to build a working system image is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	815 1) a working toolchain (ar, as, nm, cc, gcc, make, ld), 2) /bin/bash (and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	816 a symlink /bin/sh pointing to it), 3) the following command line utilities
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	817 in the $PATH:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	818
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	819 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	820 <p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	821 awk basename bzip2 cat chmod chown cmp cp cut date dd diff
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	822 dirname echo egrep env expr find grep gzip hostname id install ln ls mkdir
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	823 mktemp mv od patch pwd readlink rm rmdir sed sha1sum sleep sort tail tar
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	824 touch tr true uname uniq wc which whoami xargs yes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	825 </p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	826 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	827
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	828 <p>These commands are supplied by current versions of busybox.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	829
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	830 <p>Bash has been the standard Linux shell since before the 0.0.1 release in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	831 1991, and is installed by default on all Linux systems. (Ubuntu broke its
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	832 /bin/sh symlink to point to the Defective Annoying SHell, so many scripts call
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	833 #!/bin/bash explicitly now rather than relying on a broken symlink.) We
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	834 can't stop the build from relying on the host version of this tool; editing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	835 $PATH has no effect on the #!/bin/bash lines of shell scripts.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	836
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	837 <p>The minimal set of commands necessary to build a system image was
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	838 determined experimentally, by running a build with $RECORD_COMMANDS and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	839 then removing commands from the list and checking the effect this had on
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	840 the build. (Note that the minimal set varies slightly from target to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	841 target.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	842
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	843 <p><b>$RECORD_COMMANDS</b> tells host-tools.sh to set up a logging wrapper
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	844 that intercepts each command line in the build and writes it to a log file, so
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	845 you can see what the build actually uses. (Note that when host-tools.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	846 sets up build/wrapper, it doesn't set up build/host, so the build still uses
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	847 the host system's original command line utilities instead of building busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	848 versions. If you'd like to record the build using build/host commands,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	849 run host-too.sh without $RECORD_COMMANDS set and then run it again with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	850 $RECORD_COMMANDS to set up the logging wrapper pointing to the busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	851 tools.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	852
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	853 <p>The way $RECORD_COMMANDS works is by building a logging wrapper
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	854 (sources/toys/wrappy.c) and populating a directory (build/wrapper) with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	855 symlinks to that logging wrapper for each command name in $PATH. When later
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	856 build stages run commands, the wraper appends the command line to the log file
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	857 (specified in the environment variable $WRAPPY_LOGPATH, host-tools.sh sets
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	858 this to "$BUILD/cmdlines.$STAGE_NAME.$PACKAGE_NAME"), recording
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	859 each command run. The logging wrapper then searches $WRAPPY_REALPATH to find
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	860 the actual command to hand its command line off to.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	861
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	862 <h2>Building a cross compiler</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	863
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	864 <p>We cross compile so you don't have to. The point of this project is to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	865 make cross compiling go away, but you need to do some to get past it. So
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	866 let's get it over with.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	867
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	868 <p>The <b>cross-compiler.sh</b> script builds a cross compiler. Its output
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	869 goes into <b>build/cross-compiler-$TARGET</b> directory, which is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	870 deleted at the start of the build if it already exists, so re-running this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	871 script always does a clean build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	872
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	873 <p>Creating a cross compiler is a five step process:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	874
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	875 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	876 <li><p><b>binutils</b> - Build assembler and linker for the target platform.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	877
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	878 <p>This package has no interesting dependencies, and thus can be the first
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	879 thing you build for a target.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	880 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	881
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	882 <li><p><b>gcc</b> - Build C/C++ compiler for the target platform.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	883
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	884 <p>This package needs binutils, and must be built after that. It does not
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	885 need a C library, so can be built before that.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	886
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	887 <p>The mini-native build doesn't require C++ support, but the build adds
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	888 gcc-g++ to the basic gcc-core and enables C++ support so the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	889 <a href="#distcc_trick">distcc accelerator trick</a> can speed up C++
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	890 builds.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	891
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	892 <p>We create an "xgcc" symlink pointing to the host compiler to force gcc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	893 not to attempt to rebuild itself with itself. (It needed to be able to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	894 build xgcc with the host compiler, but doesn't trust the host compiler to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	895 build an actual binary to deploy. Note that this xgcc builds _host_ binaries,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	896 not target binaries.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	897 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	898
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	899 <li><p><b>compiler wrapper</b> - Install a wrapper around gcc to enforce sane
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	900 path logic.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	901
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	902 <p>This builds a wrapper for gcc from "sources/toys/gcc-uClibc.c". This
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	903 compiler wrapper rewrites the gcc command line to start with --nostdinc and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	904 --nostdlib, and then explicitly adds the correct header and library search
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	905 paths, and when linking adds the correct object files and libraries.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	906
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	907 <p>It needs to do this because gcc's path logic has been consistently broken
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	908 for about two decades now. (See <a href="#why_cross_compiling_sucks">why
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	909 cross compiling sucks</a> for more details.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	910
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	911 <p>The compiler hands off the new command line to $ARCH-rawgcc, so the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	912 old $ARCH-gcc gets renamed to that and the wrapper gets the old name.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	913
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	914 <p>To allow the compiler wrapper to easily find the headers and libraries,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	915 the build moves them to known locations. The system headers and libraries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	916 go into "include" and "lib" directories at the same level as the "bin"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	917 directory containing the wrapper script, and gcc's own headers and libraries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	918 go into "gcc/include" and "gcc/lib". The wrapper then finds itself (using
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	919 argv[0] and if necessary searching the $PATH it inherits), and backs up one
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	920 level to find the headers and libraries it needs to add to the gcc path.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	921 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	922
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	923 <li><p><b>linux</b> - kernel headers.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	924
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	925 <p>This package doesn't have any prerequisites, but C libraries need it
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	926 to build themselves. (Kernel headers define the system call API for the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	927 Linux kernel.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	928 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	929
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	930 <li><p><b>uClibc</b> - uClibc (micro C library).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	931
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	932 <p>This package is target code that needs to be built with a cross compiler
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	933 (gcc and binutils), and also needs kernel headers. It requires all three of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	934 the other packages, and thus must be built last.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	935
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	936 <p>Note that we only build a standard C library. We don't build/install
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	937 a standard C++ library (uClibc++), because distcc doesn't need headers
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	938 or libraries in the cross compiler. Thus the cross compiler has enough
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	939 C++ support to be used from the native environment via distcc, but not
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	940 enough to cross compile C++ code on its own.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	941
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	942 <p>The compiler wrapper actually uses links to "libc.so", which is a linker
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	943 script pointing to libuClibc.so.0. We patch uClibc so it doesn't put absolute
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	944 paths into its libc.so; without them the linker searches the supplied library
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	945 search paths, and thus the compiler may be installed in an arbitrary
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	946 location.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	947 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	948 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	949
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	950 <p>Afterwards the build strips some of the binaries, tars up the result,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	951 and performs some quick sanity tests (building dynamic and static versions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	952 of hello world. If the target configuration lists a version of QEMU to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	953 test individual binaries under on the host, it runs the static version
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	954 to make sure it outputs "Hello world".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	955 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	956
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	957 <h2>Building a minimal native development environment for the target system</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	958
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	959 <p>The <b>mini-native.sh</b> script uses the cross compiler from the previous
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	960 step to build a kernel and root filesystem for the target. The resulting
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	961 system should boot and run under an emulator, or on real target hardware.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	962
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	963 <p>Because cross-compiling is persnickety and difficult, we do as little of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	964 it as possible. This script should perform all the cross compiling anyone ever
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	965 needs to do. It uses the cross-compiler to generate the simplest possible
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	966 native build environment for the target which is capable of rebuilding itself
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	967 under itself.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	968
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	969 <p>Anything else that needs to be built for the target can then be built
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	970 natively, by running this kernel and root filesystem under an emulator and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	971 building new packages there, bootstrapping up to a full system if necessary.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	972 The emulator we use for this is QEMU. Producing a minimal build environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	973 powerful enough to boot and compile a complete Linux system requires seven
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	974 packages: the Linux kernel, binutils, gcc, uClibc, BusyBox, make, and bash.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	975 We build a few more than that, but those are optional extras.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	976
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	977 <p>This root filesystem can also be packaged using the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	978 <a href=http://www.linuxfromscratch.org/lfs>Linux From Scratch</a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	979 /tools directory approach, staying out of the way so the minimal build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	980 environment doesn't get mixed into the final system, by setting the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	981 $NATIVE_TOOLSDIR environment variable. If you don't know why you'd want
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	982 to do that, you probably don't want to.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	983
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	984 <p>In either configuration, the main target directory the build installs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	985 files into is held in the environment variable "$TOOLS". If $NATIVE_TOOLSDIR
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	986 is set this will be "/tools" in the new root filesystem, otherwise it'll
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	987 be "/usr".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	988
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	989 <p>The steps the script goes through are:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	990
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	991 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	992 <li><p><b>directory setup</b> - Create empty directories for the basic
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	993 filesystem layout.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	994
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	995 <p>If $NATIVE_TOOLSDIR is set, build script will create a Linux From Scratch
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	996 style intermediate system by moving the filesystem layout under /tools,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	997 which means skipping the top level directories and installing most files
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	998 into /tools instead of /usr. This also sets the variable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	999 $UCLIBC_DYNAMIC_LINKER to tell the compiler wrapper to create binaries that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1000 depend on shared libraries in /tools rather than the default
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1001 "/lib/ld-uClibc.so.0". (With the /tools layout, the qemu-setup.sh script can
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1002 recreate most of the top level directories at runtime, often as symlinks
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1003 into /tools.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1004 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1005
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1006 <li><p><b>Copy sources/native</b> - The most important thing here is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1007 the qemu-setup.sh script, but there's also example source code in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1008 src directory.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1009
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1010 <li><p><b>Linux kernel</b> - Build a kernel that can boot under QEMU.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1011
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1012 <p>We need kernel headers to build uClibc, so install those while we've got the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1013 kernel tarball extracted. (We could grab these files directly from the cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1014 compiler, but we rebuild from source to keep the layers cleanly separated.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1015
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1016 <p>The kernel build uses <b>sources/targets/$ARCH/miniconfig-linux</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1017 to configure the kernel for the appropriate QEMU target, and the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1018 $KERNEL_PATH variable to figure out which kernel image file to use.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1019
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1020 <li><p><b>uClibc</b> - Build standard C library.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1021
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1022 <p>The binaries in the target system are dynamically linked, so we need
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1023 shared libraries installed. Again, we could grab these files out of the cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1024 compiler, but we rebuild from source to keep the layers cleanly separated.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1025
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1026 <p>We unconditionally install the development files (headers and static
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1027 libraries), and delete them later if $NATIVE_TOOLCHAIN isn't set.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1028 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1029
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1030 <li><p><b>toybox</b> - Build optional command line utilities.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1031
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1032 <p>This isn't strictly required. If $USE_TOYBOX isn't set, this only
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1033 installs symlinks for the "patch" command and "oneit" commands. (The
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1034 oneit command is similar to init=/bin/sh, except it allows terminal control
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1035 to work and shuts the system down cleanly on exit. It's used by
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1036 qemu-setup.sh to provide a more forgiving command line.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1037
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1038 <p>If $USE_TOYBOX is set, this installs toybox versions of many commands
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1039 instead of the busybox versions. These tend to be simpler, more
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1040 straightforward implementations than the busybox versions. (Note: your author
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1041 is biased here.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1042 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1043
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1044 <li><p><b>busybox</b> - Build busybox command line utilities.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1045
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1046 <p>This provides the bulk of the command line utlities for the new
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1047 system.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1048
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1049 <p>Once upon a time, "make defconfig" provided the largest sane configuration
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1050 in busybox, enabling every working command and feature that didn't have
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1051 undesirable side effects (such as debugging options) or require special
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1052 configuration to use (such as SELINUX). Unfortunately, over time this goal
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1053 was lost and make defconfig bit-rotted into a fairly random configuration.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1054
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1055 <p>To recapture the original "largest sane configuration" goal, the build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1056 starts with "<b>make allyesconfig</b>" and applies
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1057 <b>sources/trimconfig-busybox</b> to remove features that would otherwise
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1058 cause problems. The trimconfig file has comments in it if you're wondering
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1059 why specific features are disabled.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1060 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1061
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1062 <li><p><b>Check $NATIVE_TOOLCHAIN</b> - Build a native development toolchain
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1063 only if $NATIVE_TOOLCHAIN is set.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1064
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1065 <p>$NATIVE_TOOLCHAIN is the only configuration option set by default. You
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1066 can disable it in "config" if you want to build skeletal target system and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1067 add your own software to it by hand.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1068
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1069 <p>If it is enabled, the following happens:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1070 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1071 <li><p><b>binutils</b> - Build a native assembler and linker.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1072
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1073 <li><p><b>gcc and libsupc++</b> - Build a native C and C++ compiler.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1074
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1075 <p>This process is still a bit tangled. The fundamental reason for this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1076 is that the gcc build process is pathologically misdesigned. (See
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1077 <a href="#gcc_sucks">what the hell is wrong with gcc</a> for a long
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1078 digression into the details.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1079
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1080 <p>The secondary reason is that libstdc++ is built into gcc, which makes as much
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1081 sense as building glibc into gcc. GCC's C++ support is not cleanly separated
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1082 into layers, so replacing their built-in libstdc++ with the much smaller
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1083 uClibc++ requires performing additional surgery on the gcc build process
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1084 to get it to stop being actively stupid. (For simplicity we punted on this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1085 while building the cross compiler, but now we need to make it work.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1086
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1087 <p>So after beating gcc over the head with almost a dozen different environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1088 variables and a bunch of ./configure options to get it to cross compile like
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1089 a normal program, we then have to chdir into the <b>libsupc++</b> subdirectory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1090 to build a static library which uClibc++ needs in order to interface properly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1091 with the compiler. (It defines things like stack unwinding and the current
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1092 exception model, which the C++ library needs to know but which gcc doesn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1093 cleanly export for external use.) Logically this step belongs with the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1094 uClibc++ build, but we have to export this information from the gcc source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1095 directory because that's where it lives.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1096
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1097 <p>We also clean up after a bug where gcc uses multilib directories (such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1098 /lib64) on some systems even when we explicitly told it we didn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1099 want multilib. (This package isn't very good at taking "no" for an answer.)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1100 And we create a "cc" symlink to gcc, because some packages use that as their
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1101 compiler and SUSv3 says we should have one.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1102 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1103
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1104 <li><p><b>compiler wrapper</b> - Wrap gcc, to control path logic.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1105
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1106 <p>The native build still installs the compiler wrapper
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1107 (from sources/toys/gcc-uClibc.c) to rewrite gcc's command line arguments
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1108 and bypass its built-in path logic. In theory native
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1109 compiling is less tricky and the final location we're installing the compiler
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1110 at is known at compile time, so we could just patch the compiler's source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1111 code to check the right paths. But going there rapidly turns into a nightmare
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1112 of tangled historical scar tissue, and breaks in new and exciting ways
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1113 with each new gcc release. The only way to get gcc to use sane paths is to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1114 take path decisions out of its hands entirely.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1115
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1116 <p>This does the same header/library shuffling and symlink creation as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1117 the cross compiler did, but without a prefix on the symlink names this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1118 time.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1119 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1120
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1121 <li><p><b>uClibc++</b> - Build micro standard c++ library.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1122
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1123 <p>C++ has its own standard library, and its own standard header files,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1124 without which the overloaded bit shift operators can't even perform I/O.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1125 The package uClibc++ provides much smaller and simpler versions of these
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1126 than the libstdc++-v3 metastasized through gcc-g++.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1127
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1128 <p>This package mostly builds out of the box, assuming the cross compiler has
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1129 minimal c++ support and you have the right pliers to extract libsupc++ from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1130 the gcc build. We start with the defconfig, switch off TLS and LONG_DOUBLE
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1131 support that uClibc doesn't currently provide, and blank the RUNTIME_PREFIX
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1132 so it installs where we tell it to. Then we shuffle the libraries around
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1133 so the compiler wrapper can find them and make symlinks from the generic
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1134 "libstdc++.{so,a}" names to the corresponding libuClibc++ files.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1135 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1136
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1137 <li><p><b>make</b> - Build make</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1138
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1139 <p>A toolchain doesn't do you much good without the "make" command. Fairly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1140 straightforward to build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1141 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1142
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1143 <li><p><b>bash</b> - The standard Linux command shell.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1144
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1145 <p>Bash has been the standard Linux command shell since 1991, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1146 lots of scripts explicitly say #!/bin/bash. In addition, bash extensions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1147 like curly brace filename expansion are in common use.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1148
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1149 <p>Someday, busybox might provide a decent replacement for bash, but since
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1150 busybox has four different shells (lash, hush, msh, ash) which don't share a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1151 lot of code, development is fragmented and proceeds slowly. A bash replacement
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1152 will need to be callable as "#!/bin/bash" since debian pointed #!/bin/sh at the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1153 Defective Annoying SHell and greatly discouraged use of that symlink.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1154
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1155 <p>We intentionally build an older version of bash (2.04b) which is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1156 sufficient for our purposes, and much smaller and simpler than the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1157 current bash 3.x monsters. We have to hardwire a few things ./configure
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1158 entries because this version doesn't like cross compiling, and we do so
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1159 by supplying a config.cache file with the appropriate entries. It also doesn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1160 work if you try to build in parallel, so we don't supply -j.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1161 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1162
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1163 <li><p><b>distcc</b> - command to distribute compiles across a network
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1164 cluster.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1165
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1166 <p>We install this for <a href="#distcc_trick">the distc accelerator
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1167 trick</a>. It's entirely optional.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1168
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1169 <p>We create a $TOOLS/distcc directory full of symlinks to distcc with the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1170 names of gcc, cc, g++, and c++. Inserting that directory at the start of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1171 $PATH makes the build use distcc in place of the normal native compiler.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1172 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1173 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1174
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1175 <p>That's everything in the $NATIVE_TOOLCHAIN. The rest is minor cleanup
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1176 and packaging.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1177
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1178 <li><p><b>Build static and dynamic hello world binaries</b></p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1179
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1180 <p>These are installed into $TOOLS/bin as hello-dynamic and hello-static.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1181 These are debugging tools: If you can't boot the system to a shell prompt,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1182 try running hello-static as init to see if it runs and gives you output.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1183 If that works try hello-dynamic to see if shared libraries are loading.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1184
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1185 <li><p><b>Strip some binaries</b> to save space.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1186
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1187 <li><p><b>Create the mini-native tarball</b> - we're done.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1188 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1189
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1190 <p>In theory, you can add more packages to mini-native.sh, or run another
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1191 similar script to use the cross compiler to produce output into the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1192 mini-native directory. In practice, this is not recommended. Cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1193 compiling is an endless sinkhole of frustration, and the easiest way to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1194 deal with it is not to go there.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1195
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1196 <h2>Packaging up a system image to run under emulation</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1197
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1198 <p>The <b>package-mini-native.sh</b> script packages a system image for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1199 use by QEMU. Its output goes into <b>build/system-image-$TARGET</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1200 directory, which is deleted at the start of the build if it already exists,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1201 so re-running this script always does a clean build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1202
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1203 <p>The steps here are:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1204
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1205 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1206 <li><p><b>use genext2fs</b> to package the output of mini-native.sh as an 64
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1207 megabyte ext2 image.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1208
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1209 <li><p><b>create run-emulator.sh</b> by appending an emulator invocation
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1210 command line to </b>sources/toys/run-emulator.sh</b>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1211
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1212 <p>This calls a shell function "emulator_command" from the target architecture
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1213 definition, passing in the name of the ext2 image containing the root filesystem
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1214 and the kernel image to boot. A shell function "qemu_defaults" is defined
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1215 to let emulator_command grab logs of common boilerplate, such as kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1216 command line options. (In theory run_emulator is free to use a different
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1217 emulator, or even output a command to send the files to real hardware
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1218 through a network connection or jtag or some such.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1219
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1220 <p>The path for some or the run-emulator.sh kernel command line
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1221 arguments is also adjusted based on $NATIVE_TOOLSDIR.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1222 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1223
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1224 <li><p>For the powerpc architecture, ppc_rom.bin is copied from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1225 sources/toys. (This architecture needs a custom boot rom for qemu
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1226 to be able to boot a bzImage via -kernel.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1227
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1228 <li><p>Tar up the result</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1229 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1230
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1231 <h2>Running on real hardware</a></h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1232
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1233 <p>To run a system on real hardware (not just under an emulator), you need to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1234 do several things. Dealing with myriad individual devices is beyond the scope
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1235 of this project, but the general theory is:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1236
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1237 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1238 <li><p>Figure out how to flash your device (often a jtag with openocd)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1239
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1240 <li><p>Configure and install a bootloader (uboot, apex, etc.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1241
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1242 <li><p>Build and install a kernel targeted to your hardware (in the kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1243 source, see arch/$ARCH/configs for default .config files for various
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1244 boards)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1245
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1246 <li><p>Package and install the root filesystem appropriately for your system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1247 (ext2, initramfs, jffs2).</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1248 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1249
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1250 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1251
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1252 <a name="why"><h1>Why do things this way</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1253
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1254 <p>This section has some historical information. It describes the evolution
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1255 of the firmware Linux build process, but hasn't been brought up to date
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1256 in all the particulars.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1257
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1258 <h2>The basic theory</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1259
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1260 <p>The Linux From Scratch approach is to build a minimal intermediate system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1261 with just enough packages to be able to compile stuff, chroot into that, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1262 build the final system from there. This isolates the host from the target,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1263 which means you should be able to build under a wide variety of distributions.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1264 It also means the final system is built with a known set of tools, so you get
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1265 a consistent result.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1266
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1267 <p>A minimal build environment consists of a C library, a compiler, and BusyBox.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1268 So in theory you just need three packages:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1269
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1270 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1271 <li>A C library (uClibc)</li>
526 ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1272 <li>A toolchain (pcc, or llvm/clang)</li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1273 <li>BusyBox</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1274 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1275
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1276 <p>Unfortunately, that doesn't work yet.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1277
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1278 <h2>Some differences between theory and reality.</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1279
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1280 <h3>Environmental dependencies.</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1281
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1282 <p>Environmental dependencies are things that need to be installed before you
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1283 can build or run a given package. Lots of packages depend on things like zlib,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1284 SDL, texinfo, and all sorts of other strange things. (The GnuCash project
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1285 stalled years ago after it released a version with so many environmental
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1286 dependencies it was impossible to build or install. Environmental dependencies
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1287 have a complexity cost, and are thus something to be minimized.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1288
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1289 <p>A good build system will scan its environment to figure out what it has
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1290 available, and disable functionality that depends on stuff that isn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1291 available. (This is generally done with autoconf, which is disgusting but
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1292 suffers from a lack of alternatives.) That way, the complexity cost is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1293 optional: you can build a minimal version of the package if that's all you
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1294 need.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1295
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1296 <p>A really good build system can be told that the environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1297 it's building in and the environment the result will run in are different,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1298 so just because it finds zlib on the build system doesn't mean that the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1299 target system will have zlib installed on it. (And even if it does, it may not
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1300 be the same version. This is one of the big things that makes cross-compiling
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1301 such a pain. One big reason for statically linking programs is to eliminate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1302 this kind of environmental dependency.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1303
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1304 <p>The Firmware Linux build process is structured the way it is to eliminate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1305 as many environmental dependencies as possible. Some are unavoidable (such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1306 C libraries needing kernel headers or gcc needing binutils), but the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1307 intermediate system is the minimal fully functional Linux development
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1308 environment I currently know how to build, and then we switch into that and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1309 work our way back up from there by building more packages in the new
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1310 environment.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1311
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1312 <h3>Resolving environmental dependencies.</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1313
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1314 <p><b>To build uClibc you need kernel headers</b> identifying the syscalls and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1315 such it can make to the OS. Way back when you could use the kernel headers
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1316 straight out of the Linux kernel 2.4 tarball and they'd work fine, but sometime
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1317 during 2.5 the kernel developers decided that exporting a sane API to userspace
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1318 wasn't the kernel's job, and stopped doing it.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1319
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1320 <p>The 0.8x series of Firmware Linux used
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1321 <a href=http://ep09.pld-linux.org/~mmazur/linux-libc-headers/>kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1322 headers manually cleaned up by Mariusz Mazur</a>, but after the 2.6.12 kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1323 he had an attack of real life and fell too far behind to catch up again.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1324
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1325 <p>The current practice is to use the Linux kernel's "make headers_install"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1326 target, created by David Woodhouse. This runs various scripts against the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1327 kernel headers to sanitize them for use by userspace. This was merged in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1328 2.6.18-rc1, and was more or less debugged by 2.6.19. So can use the Linux
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1329 Kernel tarball as a source of headers again.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1330
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1331 <p>Another problem is that the busybox shell situation is a mess with four
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1332 implementations that share little or no code (depending on how they're
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1333 configured). The first question when trying to fix them is "which of the four
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1334 do you fix?", and I'm just not going there. So until bbsh goes in we
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1335 <b>substitute bash</b>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1336
526 ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1337 <p>Finally, <b>most packages expect gcc</b>. None of the other compilers under
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1338 development are a drop-in replacement for gcc yet, and none of them include
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1339 a "make" program. The tcc project once showed great promise, but
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1340 development stalled because Fabrice Bellard's other major project
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1341 (qemu) is taking up all his time these days, and the developers he handed
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1342 off to have chosen to stick with a 20 year old CVS repository format
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1343 which hinders new development. Back in 2004 Fabrice
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1344 <a href=http://bellard.org/tcc/tccboot.html>built a modified Linux
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1345 kernel with tcc</a>, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1346 <a href=http://fabrice.bellard.free.fr/tcc/tccboot_readme.html>listed</a>
526 ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1347 what needed to be upgraded to build an unmodified kernel, but sometime
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1348 around 2005 the project essentially died. Since then, the BSD guys have
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1349 made a serious effort at reviving pcc, and Apple has sponsored LLVM/clang.</p>
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1350
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1351 <p>At some point, either busybox or toybox will probably grow a "make"
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1352 implementation (if for no other reason that I have vague plans to write
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1353 one), but that's not very interesting until there's a viable alternative to
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1354 the gnu toolchain. In the meantime the only open source compiler that can
ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1355 build a complete Linux system is still GCC.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1356
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1357 <p>The gnu compiler actually consists of three packages <b>(binutils, gcc, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1358 make)</b>, which is why it's generally called the gnu "toolchain". (The split
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1359 between binutils and gcc is for purely historical reasons, and you have
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1360 to match the right versions with each other or things break.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1361
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1362 <p>This means that to compile a minimal build environment, you need seven
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1363 packages, and to actually run the result we use an eighth package (QEMU).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1364
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1365 <p>This can actually be made to work. The next question is how?</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1366
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1367 <h2>Additional complications</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1368
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1369 <h3>Cross-compiling and avoiding root access</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1370
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1371 <p>The first problem is that we're cross-compiling. We can't help it.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1372 You're cross-compiling any time you create target binaries that won't run on
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1373 the host system. Even when both the host and target are on the same processor,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1374 if they're sufficiently different that one can't run the other's binaries, then
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1375 you're cross-compiling. In our case, the host is usually running both a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1376 different C library and an older kernel version than the target, even when
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1377 it's the same processor.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1378
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1379 <p>The second problem is that we want to avoid requiring root access to build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1380 Firmware Linux. If the build can run as a normal user, it's a lot more
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1381 portable and a lot less likely to muck up the host system if something goes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1382 wrong. This means we can't modify the host's / directory (making anything
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1383 that requires absolute paths problematic). We also can't mknod, chown, chgrp,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1384 mount (for --bind, loopback, tmpfs)...</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1385
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1386 <p>In addition, the gnu toolchain (gcc/binutils) is chock-full of hardwired
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1387 assumptions, such as what C library it's linking binaries against, where to look
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1388 for #included headers, where to look for libraries, the absolute path the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1389 compiler is installed at... Silliest of all, it assumes that if the host and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1390 target use the same processor, you're not cross-compiling (even if they have
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1391 a different C library and a different kernel, and even if you ./configure it
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1392 for cross-compiling it switches that back off because it knows better than
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1393 you do). This makes it very brittle, and it also tends to leak its assumptions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1394 into the programs it builds. New versions may someday fix this, but for now we
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1395 have to hit it on the head repeatedly with a metal bar to get anything remotely
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1396 useful out of it, and run it in a separate filesystem (chroot environment) so
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1397 it can't reach out and grab the wrong headers or wrong libraries despite
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1398 everything we've told it.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1399
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1400 <p>The absolute paths problem affects target binaries because all dynamically
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1401 linked apps expect their shared library loader to live at an absolute path
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1402 (in this case /lib/ld-uClibc.so.0). This directory is only writeable by root,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1403 and even if we could install it there polluting the host like that is just
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1404 ugly.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1405
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1406 <p>The Firmware Linux build has to assume it's cross-compiling because the host
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1407 is generally running glibc, and the target is running uClibc, so the libraries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1408 the target binaries need aren't installed on the host. Even if they're
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1409 statically linked (which also mitigates the absolute paths problem somewhat),
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1410 the target often has a newer kernel than the host, so the set of syscalls
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1411 uClibc makes (thinking it's talking to the new kernel, since that's what the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1412 ABI the kernel headers it was built against describe) may not be entirely
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1413 understood by the old kernel, leading to segfaults. (One of the reasons glibc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1414 is larger than uClibc is it checks the kernel to see if it supports things
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1415 like long filenames or 32-bit device nodes before trying to use them. uClibc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1416 should always work on a newer kernel than the one it was built to expect, but
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1417 not necessarily an older one.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1418
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1419 <h2>Ways to make it all work</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1420
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1421 <h3>Cross compiling vs native compiling under emulation</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1422
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1423 <p>Cross compiling is a pain. There are a lot of ways to get it to sort of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1424 kinda work for certain versions of certain packages built on certain versions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1425 of certain distributions. But making it reliable or generally applicable is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1426 hard to do.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1427
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1428 <p>I wrote an <a href=/writing/docs/cross-compiling.html>introduction
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1429 to cross-compiling</a> which explains the terminology, plusses and minuses,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1430 and why you might want to do it. Keep in mind that I wrote that for a company
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1431 that specializes in cross-compiling. Personally, I consider cross-compiling
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1432 a necessary evil to be minimized, and that's how Firmware Linux is designed.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1433 We cross-compile just enough stuff to get a working native build environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1434 for the new platform, which we then run under emulation.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1435
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1436 <h3>Which emulator?</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1437
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1438 <p>The emulator Firmware Linux 0.8x used was User Mode Linux (here's a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1439 <a href=http://www.landley.net/writing/docs/UML.html>UML mini-howto</a> I wrote
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1440 while getting this to work). Since we already need the linux-kernel source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1441 tarball anyway, building User Mode Linux from it was convenient and minimized
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1442 the number of packages we needed to build the minimal system.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1443
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1444 <p>The first stage of the build compiled a UML kernel and ran the rest of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1445 build under that, using UML's hostfs to mount the parent's root filesystem as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1446 the root filesystem for the new UML kernel. This solved both the kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1447 version and the root access problems. The UML kernel was the new version, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1448 supported all the new syscalls and ioctls and such that the uClibc was built to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1449 expect, translating them to calls to the host system's C library as necessary.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1450 Processes running under User Mode Linux had root access (at least as far as UML
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1451 was concerned), and although they couldn't write to the hostfs mounted root
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1452 partition, they could create an ext2 image file, loopback mount it, --bind
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1453 mount in directories from the hostfs partition to get the apps they needed,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1454 and chroot into it. Which is what the build did.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1455
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1456 <p>Current Firmware Linux has switched to a different emulator, QEMU, because
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1457 as long as we're we're cross-compiling anyway we might as well have the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1458 ability to cross-compile for non-x86 targets. We still build a new kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1459 to run the uClibc binaries with the new kernel ABI, we just build a bootable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1460 kernel and run it under QEMU.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1461
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1462 <p>The main difference with QEMU is a sharper dividing line between the host
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1463 system and the emulated target. Under UML we could switch to the emulated
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1464 system early and still run host binaries (via the hostfs mount). This meant
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1465 we could be much more relaxed about cross compiling, because we had one
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1466 environment that ran both types of binaries. But this doesn't work if we're
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1467 building an ARM, PPC, or x86-64 system on an x86 host.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1468
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1469 <p>Instead, we need to sequence more carefully. We build a cross-compiler,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1470 use that to cross-compile a minimal intermediate system from the seven packages
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1471 listed earlier, and build a kernel and QEMU. Then we run the kernel under QEMU
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1472 with the new intermediate system, and have it build the rest natively.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1473
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1474 <p>It's possible to use other emulators instead of QEMU, and I have a todo
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1475 item to look at armulator from uClinux. (I looked at another nommu system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1476 simulator at Ottawa Linux Symposium, but after resolving the third unnecessary
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1477 environmental dependency and still not being able to get it to finish compiling
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1478 yet, I gave up. Armulator may be a patch against an obsolete version of gdb,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1479 but I could at least get it to build.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1480
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1481 <h1>Packaging</h1>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1482
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1483 <p>The single file packaging combines a linux kernel, initramfs, squashfs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1484 partition, and cryptographic signature.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1485
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1486 <p>In Linux 2.6, the kernel and initramfs are already combined into a single
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1487 file. At the start of this file is either the obsolete floppy boot sector
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1488 (just a stub in 2.6), or an ELF header which has 12 used bytes followed by 8
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1489 unused bytes. Either way, we can generally use the 4 bytes starting at offset
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1490 12 to store the original length of the kernel image, then append a squashfs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1491 root partition to the file, followed by a whole-file cryptographic
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1492 signature.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1493
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1494 <p>Loading an ELF kernel (such as User Mode Linux or a non-x86 ELF kernel)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1495 is controlled by the ELF segments, so the appended data is ignored.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1496 (Note: don't strip the file or the appended data will be lost.) Loading an x86
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1497 bzImage kernel requires a modified boot loader that can be told the original
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1498 size of the kernel, rather than querying the current file length (which would
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1499 be too long). Hence the patch to Lilo allowing a "length=xxx" argument in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1500 config file.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1501
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1502 <p>Upon boot, the kernel runs the initramfs code which finds the firmware
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1503 file. In the case of User Mode Linux, the symlink /proc/self/exe points
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1504 to the path of the file. A bootable kernel needs a command line argument
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1505 of the form firmware=device:/path/to/file (it can lookup the device in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1506 /sys/block and create a temporary device node to mount it with; this is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1507 in expectation of dynamic major/minor happening sooner or later).
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1508 Once the file is found, /dev/loop0 is bound to it with an offset (losetup -o,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1509 with a value extracted from the 4 bytes stored at offset 12 in the file), and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1510 the resulting squashfs is used as the new root partition.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1511
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1512 <p>The cryptographic signature can be verified on boot, but more importantly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1513 it can be verified when upgrading the firmware. New firmware images can
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1514 be installed beside old firmware, and LILO can be updated with boot options
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1515 for both firmware, with a default pointing to the _old_ firmware. The
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1516 lilo -R option sets the command line for the next boot only, and that can
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1517 be used to boot into the new firmware. The new firmware can run whatever
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1518 self-diagnostic is desired before permanently changing the default. If the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1519 new firmware doesn't boot (or fails its diagnostic), power cycle the machine
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1520 and the old firmware comes up. (Note that grub does not have an equivalent
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1521 for LILO's -R option; which would mean that if the new firmware doesn't run,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1522 you have a brick.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1523
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1524 <h2>Filesystem Layout</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1525
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1526 <p>Firmware Linux's directory hierarchy is a bit idiosyncratic: some redundant
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1527 directories have been merged, with symlinks from the standard positions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1528 pointing to their new positions. On the bright side, this makes it easy to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1529 make the root partition read-only.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1530
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1531 <h3>Simplifying the $PATH.</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1532
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1533 <p>The set "bin->usr/bin, sbin->usr/sbin, lib->usr/lib" all serve to consolidate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1534 all the executables under /usr. This has a bunch of nice effects: making a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1535 a read-only run-from-CD filesystem easier to do, allowing du /usr to show
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1536 the whole system size, allowing everything outside of there to be mounted
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1537 noexec, and of course having just one place to look for everything. (Normal
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1538 executables are in /usr/bin. Root only executables are in /usr/sbin.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1539 Libraries are in /usr/lib.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1540
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1541 <p>For those of you wondering why /bin and /usr/sbin were split in the first
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1542 place, the answer is it's because Ken Thompson and Dennis Ritchie ran out
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1543 of space on the original 2.5 megabyte RK-05 disk pack their root partition
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1544 lived on in 1971, and leaked the OS into their second RK-05 disk pack where
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1545 the user home directories lived. When they got more disk space, they created
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1546 a new direct (/home) and moved all the user home directories there.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1547
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1548 <p>The real reason we kept it is tradition. The execuse is that the root
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1549 partition contains early boot stuff and /usr may get mounted later, but these
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1550 days we use initial ramdisks (initrd and initramfs) to handle that sort of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1551 thing. The version skew issues of actually trying to mix and match different
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1552 versions of /lib/libc.so.* living on a local hard drive with a /usr/bin/*
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1553 from the network mount are not pretty.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1554
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1555 <p>I.E. The seperation is just a historical relic, and I've consolidated it in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1556 the name of simplicity.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1557
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1558 <p>On a related note, there's no reason for "/opt". After the original Unix
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1559 leaked into /usr, Unix shipped out into the world in semi-standardized forms
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1560 (Version 7, System III, the Berkeley Software Distribution...) and sites that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1561 installed these wanted places to add their own packages to the system without
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1562 mixing their additions in with the base system. So they created "/usr/local"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1563 and created a third instance of bin/sbin/lib and so on under there. Then
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1564 Linux distributors wanted a place to install optional packages, and they had
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1565 /bin, /usr/bin, and /usr/local/bin to choose from, but the problem with each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1566 of those is that they were already in use and thus might be cluttered by who
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1567 knows what. So a new directory was created, /opt, for "optional" packages
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1568 like firefox or open office.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1569
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1570 <p>It's only a matter of time before somebody suggests /opt/local, and I'm
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1571 not humoring this. Executables for everybody go in /usr/bin, ones usable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1572 only by root go in /usr/sbin. There's no /usr/local or /opt. /bin and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1573 /sbin are symlinks to the corresponding /usr directories, but there's no
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1574 reason to put them in the $PATH.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1575
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1576 <h3>Consolidating writeable directories.</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1577
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1578 <p>All the editable stuff has been moved under "var", starting with symlinking
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1579 tmp->var/tmp. Although /tmp is much less useful these days than it used to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1580 be, some things (like X) still love to stick things like named pipes in there.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1581 Long ago in the days of little hard drive space and even less ram, people made
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1582 extensive use of temporary files and they threw them in /tmp because ~home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1583 had an ironclad quota. These days, putting anything in /tmp with a predictable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1584 filename is a security issue (symlink attacks, you can be made to overwrite
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1585 any arbitrary file you have access to). Most temporary files for things
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1586 like the printer or email migrated to /var/spool (where there are
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1587 persistent subdirectories with known ownership and permissions) or in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1588 user's home directory under something like "~/.kde".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1589
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1590 <p>The theoretical difference between /tmp and /var/tmp is that the contents
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1591 of /tmp should be deleted by the system init scripts on every
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1592 reboot, but the contents of /var/tmp may be preserved across reboots. Except
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1593 there's no guarantee that the contents of any temp directory won't be deleted.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1594 So any program that actually depends on the contents of /var/tmp being
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1595 preserved across a reboot is obviously broken, and there's no reason not to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1596 just symlink them together.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1597
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1598 <p>(I case it hasn't become apparent yet, there's 30 years of accumulated cruft
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1599 in the standards, convering a lot of cases that don't apply outside of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1600 supercomputing centers where 500 people share accounts on a mainframe that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1601 has a dedicated support staff. They serve no purpose on a laptop, let alone
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1602 an embedded system.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1603
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1604 <p>The corner case is /etc, which can be writeable (we symlink it to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1605 var/etc) or a read-only part of the / partition. It's really a question of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1606 whether you want to update configuration information and user accounts in a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1607 running system, or whether that stuff should be fixed before deploying.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1608 We're doing some cleanup, but leaving /etc writeable (as a symlink to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1609 /var/etc). Firmware Linux symlinks /etc/mtab->/proc/mounts, which
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1610 is required by modern stuff like shared subtrees. If you want a read-only
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1611 /etc, use "find /etc -type f \| xargs ls -lt" to see what gets updated on the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1612 live system. Some specific cases are that /etc/adjtime was moved to /var
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1613 by LSB and /etc/resolv.conf should be a symlink somewhere writeable.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1614
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1615 <h3>The resulting mount points</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1616
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1617 <p>The result of all this is that a running system can have / be mounted read
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1618 only (with /usr living under that), /var can be ramfs or tmpfs with a tarball
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1619 extracted to initialize it on boot, /dev can be ramfs/tmpfs managed by udev or
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1620 mdev (with /dev/pts as devpts under that: note that /dev/shm naturally inherits
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1621 /dev's tmpfs and some things like User Mode Linux get upset if /dev/shm is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1622 mounted noexec), /proc can be procfs, /sys can bs sysfs. Optionally, /home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1623 can be be an actual writeable filesystem on a hard drive or the network.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1624
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1625 <p>Remember to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1626 put root's home directory somewhere writeable (I.E. /root should move to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1627 either /var/root or /home/root, change the passwd entry to do this), and life
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1628 is good.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1629
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1630
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1631 <p>Firmware Linux is an embedded Linux distribution builder, which creates a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1632 bootable single file Linux system based on uClibc and BusyBox/toybox. It's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1633 basically a shell script that builds a complete Linux system from source code
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1634 for an arbitrary target hardware platform.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1635
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1636 <p>The FWL script starts by building a cross-compiler for the appropriate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1637 target. Then it cross-compiles a small Linux system for the target, which
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1638 is capable of acting as a native development environment when run on the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1639 appropriate hardware (or under an emulator such as QEMU). Finally the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1640 build script creates an ext2 root filesystem image, and packages it with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1641 a kernel configured to boot under QEMU and shell scripts to invoke qemu
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1642 appropriately.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1643
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1644
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1645 <p>The FWL boot script for qemu (/tools/bin/qemu-setup.sh) populates /dev
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1646 from sysfs, sets up an emulated (masquerading) network (so you can wget
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1647 source packages or talk to <a href="#distcc_trick">distcc</a>), and creates
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1648 a few symlinks needed to test build normal software packages (such as making
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1649 /lib point to /tools/lib). It also mounts /dev/hdb (or /dev/sdb) on /home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1650 if a second emulated drive is present.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1651
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1652 <p>For most platforms, exiting the command shell will exit the emulator.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1653 (Some, such as powerpc, don't support this yet. For those you have to kill
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1654 qemu from another window, or exit the xterm. I'm working on it.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1655
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1656 <p>To use this emulated system as a native build environment, see
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1657 <a href="#native_compiling">native compiling</a>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1658
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1659
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1660 <a name="new_platform"><h1>Adding a new target platform</h1></a>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1661
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1662 <p>The differences between platforms are confined to a single directory,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1663 sources/targets. Each subdirectory under that contains all the configuration
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1664 information for a specific target platform FWL can produce system images
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1665 for. The same scripts build the same packages for each platform, differing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1666 only in which configuration directory they pull data from.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1667
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1668 <p>Each target configuration directory has three interesting files:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1669
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1670 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1671 <li><p><b>details</b> - sets a bunch of environment variables</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1672 <li><p><b>miniconfig-uClibc</b> - configuration for uClibc.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1673 <li><p><b>miniconfig-linux</b> - configuration for the Linux kernel</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1674 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1675
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1676 <p>These configuration files are read and processed by the script
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1677 <b>include.sh</b>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1678
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1679 <h2>Target name.</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1680
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1681 <p>The name of the target directory is saved in the variable "$ARCH", and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1682 used to form a "tuple" for gcc and binutils by appending "-unknown-linux" to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1683 the directory name. So the first thing to do is find out what platform name
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1684 gcc and binutils want for your target platform, and name your target directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1685 appropriately.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1686
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1687 <p>(Note: if your platform really can't use an "${ARCH}-unknown-linux" style
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1688 tuple, and instead needs a tuple like "bfin-elf", you can set the variable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1689 CROSS_TARGET in the "details" file to override the default value and feed
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1690 some other --target to gcc and binutils. You really shouldn't have to do
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1691 this unless gcc doesn't yet fully support Linux on your platform. Try the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1692 default first, and fix it if necessary.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1693
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1694 <p>The name of the target directory is also used in the name of the various
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1695 directories generated during the build (temp-$ARCH, cross-compiler-$ARCH,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1696 and mini-native-$ARCH, all in the build/ directory), and as the prefix of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1697 cross compiler binaries ($ARCH-gcc and friends).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1698
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1699 <h2>$ARCH/details</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1700
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1701 <p>The following environment variables may be set in the "<b>details</b>"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1702 file:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1703
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1704 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1705 <li><p><b>CROSS_TARGET</b> - By default this is set to "${ARCH}-unknown-linux".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1706
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1707 <p>This is used by binutils and gcc. If your target really can't use that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1708 tuple name (perhaps needing a tuple like "bfin-elf" instead), you can set the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1709 variable CROSS_TARGET in the "details" file to override the default value and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1710 feed some other --target to gcc and binutils.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1711
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1712 <p>You usually shouldn't have to set this yourself unless gcc doesn't yet fully
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1713 support Linux on your platform. Try the default first, and fix it if
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1714 necessary.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1715 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1716
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1717 <li><p><b>KARCH</b> - architecture value for the Linux kernel (ARCH=$KARCH).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1718
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1719 <p>The Linux kernel uses different names for architectures than gcc or binutils
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1720 do. To see all your options, list the "arch" directory of the linux kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1721 source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1722 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1723
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1724 <li><p><b>KERNEL_PATH</b> - Path in the linux kernel source tree where the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1725 bootable kernel image is generated.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1726
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1727 <p>This is the file saved out of the kernel build, to be fed to qemu's -kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1728 option. Usually "arch/${KARCH}/boot/zImage", but
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1729 sometimes bzImage or image in that directory, sometimes vmlinux in the top
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1730 level directory...</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1731 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1732
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1733 <li><p><b>GCC_FLAGS</b> - Any extra flags needed by gcc.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1734
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1735 <p>Usually blank, but sometimes used to specify a floating point coprocessor
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1736 or ABI.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1737 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1738
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1739 <li><p><b>BINUTILS_FLAGS</b> - Any extra flags needed by binutils.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1740
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1741 <p>Usually blank.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1742 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1743
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1744 <li><p><b>QEMU_TEST</b> - Optional emulator for sanity test.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1745
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1746 <p>At the end of the cross compiler build, a quick sanity
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1747 test builds static and dynamic "Hello world!" executables with the new cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1748 compiler. If QEMU_TEST isn't blank and a file qemu-$QEMU_TEST
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1749 exists in the $PATH, the cross compiler build script will then run qemu's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1750 application emulation against the static version of "hello world" as an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1751 additional sanity test, to make sure it runs on the target processor and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1752 outputs "Hello world!".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1753
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1754 <p>Leave it blank to skip this test.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1755 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1756
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1757 <li><p><b>emulator_command</b> - Shell function run to generate the actual
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1758 emulator invocation at the end of the run-$ARCH.sh shell script in the system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1759 image tarball.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1760
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1761 <p>This is actually a shell function, not an environment variable. It's
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1762 called from package-mini-native.sh to output an emulator command line to
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1763 stdout (generally using "echo").<p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1764
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1765 <p>The function receives two arguments: $1 is the name of the ext2 image
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1766 containing the root filesystem, and $2 is the name of the kernel image.
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1767 The function can also call another shell function, <b>qemu_defaults</b>,
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1768 which is defined in package-mini-native.sh and which provides most of
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1769 the qemu command line. (If you use a different emulator, you don't have to
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1770 call this function, but if you use qemu it makes things a lot easier and
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1771 more consistent.) The qemu_command function outputs $ROOT and $CONSOLE
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1772 variables for its root= and console= kernel command line arguments, so
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1773 set those before calling it.</p>
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1774 </li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1775
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1776 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1777
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1778 <a name="miniconfig"><h2>Miniconfig files</h2></a>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1779
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1780 <p>The expanded .config files used to build both Linux and uClibc are copied
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1781 into the /usr/src directory of mini-native filesystems during the build,
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1782 and kept for future reference.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1783
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1784 <p>The Linux kernel and uClibc each need a configuration file to build.
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1785 Firmware Linux uses the "miniconfig" file format, which contains only the configuration
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1786 symbols a user would have to switch on in menuconfig if they started from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1787 allnoconfig.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1788
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1789 <p>To generate a miniconfig, first configure your kernel with menuconfig,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1790 then copy the resulting .config file to a temporary filename (such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1791 "tempfile"). Then run the miniconfig.sh script in the sources/toys directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1792 with the temporary file name as your argument and with the environment variable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1793 ARCH set to the $KARCH value in your new config file (and exported if
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1794 necessary). This should produce a new file, "mini.config", which is your
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1795 .config file converted to miniconfig format.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1796
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1797 <p>For example, to produce a miniconfig for a given platform:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1798 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1799 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1800 make ARCH=$KARCH menuconfig
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1801 mv .config tempfile
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1802 ARCH=$KARCH miniconfig.sh tempfile
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1803 ls -l mini.config
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1804 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1805 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1806
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1807 <p>To expand a mini.config back into a full .config file (to build a kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1808 by hand, or for further editing with menuconfig), you can go:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1809
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1810 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1811 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1812 make ARCH=$KARCH allnoconfig KCONFIG_ALLCONFIG=mini.config
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1813 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1814 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1815
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1816 <p>Remember to supply an actual value for $KARCH.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1817
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1818 <h2>$ARCH/miniconfig-linux</h2>
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1819 <p>This is the miniconfig file to build a Linux kernel for the appropriate
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1820 target. This is usually aimed at booting under QEMU, but if you'd like
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1821 to come up with your own configuration for actual target hardware, feel
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1822 free.</p>
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1823
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1824 <h2>$ARCH/miniconfig-uClibc</h2>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1825
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1826 <p>Just like the Linux kernel, uClibc needs a .config file to build, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1827 so the Firmware Linux configuration file supplies a miniconfig. Note that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1828 uClibc doesn't require an ARCH= value, because all its architecture information
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1829 is stored in the config file. Otherwise the procedure for creating and using
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1830 it is the same as for the Linux kernel, just with a different filename and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1831 contents.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1832
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1833 <p>Most of each miniconfig-uClibc is identical from platform to platform.
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1834 At some point in the future this boilerplate might be factored out into
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1835 a common file, but so far removing the duplication hasn't been worth the
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1836 extra complexity.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1837
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1838 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1839
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1840 <!--
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1841
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1842 <p>This root filesystem acts as a minimal native build environment for the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1843 target platform. This means it contains a compiler and associated build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1844 tools capable of building a complete new Linux system under itself. If you're
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1845 interested in building a more complex development environment within this one,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1846 see the <a href=http://www.linuxfromscratch.org/lfs/view/6.4/>Linux From
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1847 Scratch</a> project for ideas on how to bootstrap your way up (adding
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1848 zlip and perl and such before building more complicated packages).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1849
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1850 <p>Note that FWL can build a LFS "temporary system", but that the packaged
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1851 mini-native tarballs and system images are not configured that way.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1852
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1853 <p>The vast majority of the space taken up by this filesystem is the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1854 development toolchain and associated support files (mostly header files and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1855 libraries).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1856
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1857 <p>If you're doing anything fancy, you'll probably want to rebuild it from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1858 source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1859
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1860 GCC is bad at cross compiling.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1861
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1862 <p>A cross compiler reads input files and writes output files. So does a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1863 docbook to PDF converter; this is nothing special. A program can take both
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1864 explicit and implicit input. Explicit input is listed on the command line,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1865 or perhaps piped into stdin. Implicit input in the case of the docbook->pdf
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1866 converter would include fonts and stylesheets, which might live at some
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1867 common path on the host, or be handed out by a server.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1868
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1869 <p>Compilers take implicit input from five places:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1870
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1871 compiler #includes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1872 system library #includes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1873 compiler libs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1874 system libs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1875
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1876
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1877 <p>In theory, someday busybox may provide a decent /bin/bash replacement,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1878 but unfortunately busybox shell development is terminally fragmented (between
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1879 lash, hush, msh, and ash, which do not share significant amounts of code),
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1880 so don't hold your breath.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1881
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1882 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1883 variables for invoking download
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1884 mirrors
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1885 --extract, setupfor, cacheing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1886 why hard links instead of symlinks
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1887 Packages that modify distributed source files without breaking links: bad.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1888
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1889 CPUS, make -j (As much parallelism as possible; SMP increasing.)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1890
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1891 Minimal native build environment, seven packages.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1892 Why bash?
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1893 Busybox ash buggy. Busybox has four shells, not one scalable shell.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1894 Older version of bash, smaller and simpler.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1895
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1896 sources/native
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1897 src
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1898 qemu-setup.sh/chroot-setup.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1899 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1900
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1901 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1902
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1903 <h2><a name="native_compiling">Native compiling under emulation</a></h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1904
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1905 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1906 Why do this?
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1907 the distcc trick
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1908 run-with-distcc.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1909 hdb for working space.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1910 run-with-home.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1911 Building on nfs sucks rocks.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1912 Building out of tree with cp -rs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1913
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1914 /tools (Linux From Scratch chapter 5).
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1915 qemu-setup.sh and the /lib symlink.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1916
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1917 Deficiencies in the current mini-native filesystem:
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1918 host-tools.sh, bzip2, coreutils, diffutils.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1919
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1920 Building glibc is your problem.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1921 It requires perl. Statically linking "hello world" is 400k. It's evil.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1922 Still, building it natively sucks less than trying to cross compile it.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1923 Pretty much follow the non-cross Linux From Scratch procedures.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1924
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1925 Building a distro:
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1926 Linux From Scratch.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1927 Gentoo embedded.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1928 Debian/ubuntu.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1929
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1930 anatomy_of_include
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1931 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1932
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1933 <h2><a name="customize">Customizing mini-native</a></h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1934
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1935 Adding packages (to the script, without the script)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1936
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1937 build/temp-$ARCH
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1938 build/cross-compiler-$ARCH
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1939 build/mini-native-$ARCH
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1940 build/qemu-image-*.tar
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1941
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1942
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1943 <a name="gcc_sucks><h2>What the hell is wrong with GCC?</h2></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1944
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1945 <p>First of all, gcc wants to build itself with itself. When you build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1946 gcc it wants to compile a temporary version of itself, and then build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1947 itself again with that temporary compiler, and then build itself a _third_
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1948 time with the second compiler.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1949
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1950 -->
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1951
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1952
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1953
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1954
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1955 <!--#include file="footer.html" -->

Mercurial > hg > aboriginal

annotate www/documentation.html @ 526:ad14395e1f0c