aboriginal: www/documentation.html annotate

annotate www/documentation.html @ 1229:313c702a0984

Remove toybox.

author	Rob Landley <rob@landley.net>
date	Tue, 24 Aug 2010 03:08:47 -0500
parents	333c8f799302
children	4c0a2018502c

rev	line source
881 0da87d1ef528 Update the web page. Put the left-side nav bar back, move about page to "about.html" and make news.html the default index. Add architectures page. Note that IRC discussion is now on #edev. Rob Landley <rob@landley.net> parents: 830 diff changeset	1 <!--#include file="header.html" -->
0da87d1ef528 Update the web page. Put the left-side nav bar back, move about page to "about.html" and make news.html the default index. Add architectures page. Note that IRC discussion is now on #edev. Rob Landley <rob@landley.net> parents: 830 diff changeset	2
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	3 <h1>Documentation for Firmware Linux</h1>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	4
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	5 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	6 <li><a href="#what_is_it">What is Firmware Linux?</a></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	7 <li><a href="#how_system_image">How do I use system images?</a></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	8 <li><a href="#how_build_source">How do I build my own customized system images from source code?</a></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	9 <li><a href="#how_implemented">How is Firmware Linux implemented?</a></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	10 <li><a href="#why">Why do things this way?</a></li>
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	11 <li><a href="#new_platform">Adding a new target platform</a></li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	12 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	13
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	14 <hr />
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	15 <a name="what_is_it"><h1>What is Firmware Linux?</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	16
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	17 <p>Firmware Linux is a toolkit for building custom virtual machines.
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	18 It lets you boot virtual PowerPC, ARM, MIPS and other exotic systems
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	19 on your x86 laptop, and do development in them.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	20
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	21 <p>The name "Firmware Linux" reflects FWL's origins as a tool for
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	22 embedded development. It provides an easy way to get started with
1050 333c8f799302 Update lots and lots of web pages for the project name change to Aboriginal Linux. Rob Landley <rob@landley.net> parents: 1040 diff changeset	23 that, building your own code against uClibc and testing it on various
333c8f799302 Update lots and lots of web pages for the project name change to Aboriginal Linux. Rob Landley <rob@landley.net> parents: 1040 diff changeset	24 hardware platforms. But it has other uses as well, including
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	25 cross-platform regression testing, portability auditing, and toolchain
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	26 debugging.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	27
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	28 <p>This documentation uses the name "Firmware Linux" (or abbreviation
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	29 "FWL") to refer to the <a href=downloads>build system</a>, and calls
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	30 the output of the build a "<a
1050 333c8f799302 Update lots and lots of web pages for the project name change to Aboriginal Linux. Rob Landley <rob@landley.net> parents: 1040 diff changeset	31 href=downloads/binaries>system image</a>". The build
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	32 system is implemented as a series of bash scripts and configuration
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	33 files which compile a Linux development environment for the specified
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	34 target system and package it into a bootable binary image.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	35
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	36 <p>The base development environment is built from seven source
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	37 packages: busybox, uClibc, gcc, binutils, make, bash, and the Linux
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	38 kernel. This is the smallest environment that can rebuild itself
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	39 entirely from source code, and thus the minimum a host system must
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	40 cross compile in order to create a fully independent native
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	41 development environment for a target.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	42
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	43 <p>Booting a development system image under an emulator such as <a
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	44 href=http://bellard.org/qemu/>QEMU</a> allows fully native builds for
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	45 supported target platforms to be performed on cheap and powerful
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	46 commodity PC hardware. You can then build and install additional
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	47 packages (zlib, bison, openssl...) within the virtual machine's native
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	48 development environment, without having to do any additional cross
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	49 compiling.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	50
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	51 <p>FWL currently includes full support for arm, mips, powerpc, x86,
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	52 x86-64 targets, and several other more exotic platforms; see the <a
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	53 href=screenshots>screenshots page</a> for a complete list. The goal
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	54 for the FWL 1.0 release is to support every target QEMU can emulate in
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	55 "system" mode.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	56
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	57 <p>Firmware Linux is licensed under GPL version 2. Its component packages are
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	58 redistributed under their respective licenses (mostly GPL and LGPL).</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	59
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	60 <h2>Optional extras</h2>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	61
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	62 <p>Intermediate stages of the build (such as the cross compiler and the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	63 unpackaged root filesystem directory) may also be useful to Linux developers,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	64 so tarballs of them are saved during the build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	65
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	66 <p>By default the build cross-compiles some optional extra packages (toybox,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	67 distcc, uClibc++) and preinstalls them into the target filesystem. This is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	68 just a convenience; these packages build and install natively within the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	69 minimal development system image just fine.<!-- TODO: experimentally confirm
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	70 that, make it configurable, add genext2fs and strace to the list? -->)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	71
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	72 <hr />
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	73 <a name="how_system_image"><h1>Using system images</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	74
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	75 <p>If you want to jump straight to building your own software natively for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	76 embedded targets, you can <a href=downloads/binaries>download a prebuilt
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	77 binary image</a> instead of running the build scripts to produce your own.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	78
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	79 <p>Here are the different types of output produced by the build:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	80
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	81 <h2>system-image-*.tar.bz2</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	82
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	83 <p>System images boot a complete linux system under an emulator. Each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	84 system-image tarball contains an ext2 root filesystem image, a Linux kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	85 configured to run under the emulator <a href=http://bellard.org/qemu/>QEMU</a>,
1050 333c8f799302 Update lots and lots of web pages for the project name change to Aboriginal Linux. Rob Landley <rob@landley.net> parents: 1040 diff changeset	86 and scripts launch the virtual system under the emulator in various
333c8f799302 Update lots and lots of web pages for the project name change to Aboriginal Linux. Rob Landley <rob@landley.net> parents: 1040 diff changeset	87 configurations.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	88
1050 333c8f799302 Update lots and lots of web pages for the project name change to Aboriginal Linux. Rob Landley <rob@landley.net> parents: 1040 diff changeset	89 <p>The steps to test boot a system image under QEMU are:</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	90 <ul>
1050 333c8f799302 Update lots and lots of web pages for the project name change to Aboriginal Linux. Rob Landley <rob@landley.net> parents: 1040 diff changeset	91 <li>install QEMU 0.12.4 or later</li>
333c8f799302 Update lots and lots of web pages for the project name change to Aboriginal Linux. Rob Landley <rob@landley.net> parents: 1040 diff changeset	92 <li>download the appropriate <a href=downloads/binaries>prebuilt binary tarball</a>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	93 for the target you're interested in</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	94 <li>extract it: <b>tar -xvjf system-image-$TARGET.tar.bz2</b></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	95 <li>cd into it: <b>cd system-image-$TARGET</b></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	96 <li>execute it: <b>./run-emulator.sh</b></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	97 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	98
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	99 <p>This boots the system image under the appropriate emulator, with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	100 the emulated Linux's /dev/console hooked to stdin and stdout of the emulator
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	101 process. (I.E. the shell prompt the script gives you after the boot messages
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	102 scroll past is for a shell running inside the emulator. This lets you pipe
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	103 the output of other programs into the emulator, and capture the emulator's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	104 output.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	105
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	106 <p>Type "cat /proc/cpuinfo" to confirm you're running in the emulator, then
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	107 play around and have fun. Type "exit" when done.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	108
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	109 <p>Inside a system image, you generally wget source code from some URL and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	110 compile it. (For example, you can wget the FWL build, extract it, and run it
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	111 inside one of its own system images to trivially prove it can rebuild itself.)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	112 If you run a web server on your host's loopback interface, you an access it
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	113 inside QEMU using the special address "10.0.2.2". Example build scripts
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	114 are available in the /usr/src directory.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	115
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	116 <h3>Extra space and speed</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	117
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	118 <p>The system images by themselves are fairly small (64 megabytes), and don't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	119 have a lot of scratch space for building or installing other packages. If a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	120 file named "<b>hdb.img</b>" exists in the current directory, run-emulator.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	121 will automatically designate it as a second virtual hard drive and attempt to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	122 mount the whole unpartitioned device on <b>/home</b> inside the emulator.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	123
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	124 <p>Some optional command line arguments to run-emulator.sh provide extra
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	125 space and extra speed for compiling more software:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	126
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	127 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	128 <li><p><b>--make-hdb $MEGABYTES</b> - if the hard drive image to mount on /home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	129 doesn't already exist, create a sparse file of the indicated size and format
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	130 it ext3.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	131
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	132 <li><p><b>--with-hdb $FILENAME</b> - use specified $FILENAME from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	133 the host as the hard drive image to mount on the emulated system's /home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	134 (instead of the default "hdb.img"). Fail if it doesn't exist, unless
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	135 --make-hdb was also specified.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	136
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	137 <li><p><b>--with-distcc $CC_PATH</b> - enable the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	138 <a href="#distcc_trick">distcc accelerator trick</a>. This option provides
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	139 the path to an appropriate cross compiler directory, so run-emulator.sh can
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	140 launch a distcc daemon on the host's loopback device configured to call that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	141 cross compiler, and configure the emulated system to call out to that cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	142 compiler through distcc.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	143 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	144
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	145 <p>Running an armv4l system image with the cross compiler
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	146 installed in the user's home directory, using a hard drive image in the user's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	147 home directory (to be created with a size of 2 gigabytes if it doesn't already
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	148 exist) might look like:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	149
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	150 <blockquote><pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	151 <b>./run-emulator.sh --make-hdb 2048 --with-hdb ~/blah.img --with-distcc ~/cross-compiler-armv4l</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	152 </pre></blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	153
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	154 <h2>mini-native-*.tar.bz2</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	155
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	156 <p>These <a href=downloads/mini-native>tarballs</a> contain the same root
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	157 filesystem as the corresponding system images, just in an archive
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	158 instead of packaged into a filesystem image.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	159
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	160 <p>If you want to boot your own system image on real hardware instead of an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	161 emulator, the appropriate mini-native tarball is a good starting point. If
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	162 all you want is a native uClibc development environment for your host, try:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	163
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	164 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	165 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	166 <b>chroot mini-native-x86_64 /usr/chroot-setup.sh</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	167 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	168 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	169
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	170 <p>The boot script /usr/qemu-setup.sh or /usr/chroot-setup.sh performs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	171 minimal setup for the appropriate environment, mounting /proc and /sys and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	172 such. It starts a single shell prompt, and automatically cleans up when that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	173 process exits.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	174
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	175 <p>If you're interested in building a more complex development environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	176 within this one (adding zlib and perl and such before building more complicated
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	177 packages), the best way to learn how is to read
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	178 <a href=http://www.linuxfromscratch.org/lfs/view/6.4/>Linux
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	179 From Scratch</a>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	180
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	181 <p>Note that mini-native is just one potential filesystem layout; the FWL
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	182 build scripts have several other configurations available when you build from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	183 source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	184
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	185 <h2>cross-compiler-*.tar.bz2</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	186
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	187 <p>The cross compilers created during the FWL build are relocatable C compilers
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	188 for each target platform. The primary reason for offering each cross compiler
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	189 as a downloadable binary is to implement the <a href="#distcc_trick">distcc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	190 accelerator trick</a>. Using them to cross compile additional software is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	191 supported, but not recommended.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	192
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	193 <p>If you'd like to use one for something other than distcc, this documentation
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	194 mostly assumes you already know how. Briefly:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	195
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	196 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	197 <li>download the appropriate cross-compiler-$TARGET.tar.bz2</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	198 <li>extract it somewhere (doesn't matter where)</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	199 <li>add the resulting cross-compiler-$TARGET/bin subdirectory to your $PATH</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	200 <li>either use $TARGET-gcc as your compiler, or set your $CROSS_COMPILE prefix
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	201 to "$TARGET-" with a trailing dash.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	202 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	203
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	204 <p>Also, stock up on asprin and clear a space to beat your head against; you'll
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	205 need both. See <a href="#why_cross_compiling_sucks">why cross compiling
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	206 sucks</a> for more details.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	207
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	208 <p>Note that although this cross compiler has g++, it doesn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	209 have uClibc++ in its lib or include subdirectories, which is required to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	210 build most c++ programs. If you need extra libraries, it's up to you to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	211 cross-compile and install them into those directories.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	212
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	213 <a name="how_build_source"><h1>How do I build my own customized system images
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	214 from source code?</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	215
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	216 <p>To build your own root filesystem and system images from source code,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	217 download and run <a href=downloads>the FWL build scripts</a>. You'll
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	218 probably want to start with the most recent <a href=downloads>release
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	219 version</a>, although once you've got the hang of it you might want to follow
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	220 the <a href=/hg/firmware>development version</a>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	221
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	222 <p>For a quick start, download the tarball, extract it, cd into it, and run
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	223 "<b>./build.sh</b>". This script takes one argument, which is the target to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	224 build for. Run it with no arguments to list available targets.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	225
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	226 <p>This should produce all the tarballs listed in the previous section in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	227 the "build" directory. To perform a clean build, "rm -rf build" and re-run
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	228 build.sh.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	229
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	230 <h2>How building from source works</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	231
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	232 <p>The build system is a series of shell scripts which download, compile,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	233 install, and use the appropriate source packages to generate a system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	234 image. These shell scripts are designed to be easily read and modified,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	235 acting both as tools to perform a build and as documentation on how to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	236 build these packages.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	237
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	238 <p>The <b>build.sh</b> script is a simple wrapper which calls the following
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	239 other scripts in sequence:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	240 <ol>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	241 <li>download.sh</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	242 <li>host-tools.sh</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	243 <li>cross-compiler.sh $TARGET</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	244 <li>mini-native.sh $TARGET</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	245 <li>package-mini-native.sh $TARGET</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	246 </ol>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	247
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	248 <p>In theory, the stages are othogonal. If you have an existing cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	249 compiler, you can add it to the $PATH and skip cross-compiler.sh. Or you
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	250 can use _just_ cross-compiler.sh to create a cross compiler, and then go build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	251 something else with it. The host-tools.sh stage can often be skipped
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	252 entirely.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	253
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	254 <h3>Build stages</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	255
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	256 <p>The following files control the individual stages of the build. Each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	257 may be called individually, from in the top level directory of FWL:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	258
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	259 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	260 <li><p><b>download.sh</b> - Download source packages from the web.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	261
655 62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	262 <p>This script does not take any arguments. It's a series of calls to a
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	263 download function (defined in sources/include.sh) that checks if an existing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	264 copy of the tarball matching a defined $SHA1 sum exists in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	265 <b>sources/packages</b> directory, and if not uses wget to fetch it from the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	266 $URL (or else from a series of fallback mirrors). A blank value for $SHA1
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	267 will accept any file as correct, ignoring its contents.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	268
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	269 <p>After downloading all tarballs, the function <b>cleanup_oldfiles</b> deletes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	270 any unused files from sources/packages (generally previous versions left over
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	271 after a package upgrade while using the development version of the FWL
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	272 build scripts).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	273
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	274 <p>Running this stage with the argument "--extract-all" will extract all
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	275 the tarballs at once, to populate the cache used by setupfor. (This is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	276 primarily used to avoid race conditions when building multiple architectures
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	277 in parallel with build-all-targets.sh. This is an esoteric internal
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	278 detail you can safely ignore if you're not doing that.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	279 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	280
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	281 <li><p><b>host-tools.sh</b> - Set up a known environment on the host</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	282
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	283 <p>This script does not take an arguments. In theory this is an optional step,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	284 and may be omitted, as the binaries produced by this script are not included in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	285 any of the output tarballs.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	286
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	287 <p>This script populates the <b>build/host</b> directory with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	288 host versions of the busybox and toybox command line tools (the same ones
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	289 that the target's eventual root filesystem will contain), plus symlinks to the
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	290 host's compiler toolchain (I.E. compiler, linker, assembler, and so on).</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	291
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	292 <p>This allows the calling scripts to trim the $PATH to point to just this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	293 one directory, which serves several purposes:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	294
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	295 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	296
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	297 <li><p><b>Isolation</b> - This prevents the ./configure stages of the source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	298 packages from finding and including unexpected dependencies on random things
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	299 installed on the host.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	300
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	301 <li><p><b>Portability</b> - Using a known set of command line utilities
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	302 insulates the build from variations in the host's Linux distribtion (such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	303 Ubuntu's /bin/echo lacking suport for the -e option).</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	304
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	305 <li><p><b>Testing</b> - It ensures the resulting system can rebuild itself
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	306 under itself, since the initial build was done with the same tools we
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	307 install into the target's root filesystem. The initial build acts as a smoke
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	308 test of most of the packages used to create the resulting system, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	309 restricting $PATH ensures that no necessary commands are missing. (Variation
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	310 can still show up between x86/arm/powerpc versions, of course.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	311
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	312 <p>It also moves most failures to the beginning. If anything's going to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	313 break, it's usually the host-tools build. After that runs, we're mostly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	314 in a known and tested state.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	315
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	316 <li><p><b>Dependency tracking</b> - If we don't explicitly know everything
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	317 we need to build ourselves in the first place, we can't be sure we added
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	318 it to the final system to get a self-hosting environment.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	319 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	320
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	321 <p>A secondary purpose of host-tools.sh is to build packages (such as distcc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	322 and genext2fs) which might not be installed on the host system. Some of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	323 these aren't needed by build.sh but may be used by later run-emulator.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	324 invocations (such as the ./run-build-image.sh script).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	325
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	326 <p>Note that this script does not attempt to build qemu, due to the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	327 unreasonable requirement of installing gcc 3.x on the host. The FWL build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	328 scripts do not use qemu (except as an optional test at the end of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	329 cross-compiler.sh which is skipped if qemu is not available). You will need
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	330 to install qemu (or another emulator, or find real hardware) to use the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	331 resulting system images, but they should build just fine without it.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	332
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	333 <p>This stage is optional. You don't need to run this stage if you don't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	334 want to. If the build/host directory doesn't exist (or doesn't contain
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	335 a "busybox" executable), the build will use the host's original $PATH.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	336 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	337
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	338 <li><p><b>cross-compiler.sh</b> - Build a cross compiler for the target, for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	339 use by mini-native.sh and the distcc accelerator.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	340
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	341 <p>In order to build binaries for the target, the build must first create
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	342 a cross compiler to build those target binaries with. This script creates
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	343 that cross compiler. If you already have a cross compiler, you can
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	344 supply it here (the easy way is to create a build/cross-compiler-$TARGET/bin
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	345 directory and put "$TARGET-gcc" style symlinks in it) and skip this step.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	346
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	347 <p>This script takes one argument: the architecture to build for. It
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	348 produces a cross compiler that runs on the host system and produces binaries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	349 that run on the target system. This cross compiler is created using the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	350 source packages binutils, gcc, uClibc, the Linux kernel headers, and a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	351 compiler wrapper to make the compiler relocatable.<</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	352
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	353 <p>The reason for the compiler wrapper is that by default, gcc hardwires
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	354 lots of absolute paths into itself, and thus only runs properly in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	355 directory it was built in. The compiler wrapper rewrites its command line
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	356 to prevent gcc from using its built-in (broken) path logic.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	357
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	358 <p>The build requires a cross-compiler even if the host and target system use
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	359 the same processor because the host and target may use different C libraries.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	360 If the host has glibc and the target uses uClibc, then the (dynamically linked)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	361 target binaries the compiler produces won't run on the host. (Target binaries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	362 that won't run on the host are what distinguishes cross-compiling from native
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	363 compiling. Different processors are just one reason for it: glibc vs uClibc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	364 is another, ELF vs binflat or a.out executable format is a third...)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	365
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	366 <p>This script produces produces a working cross compiler in the build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	367 directory, and saves a tarball of it as "cross-compiler-$TARGET.tar.bz2"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	368 for use outside the build system. This cross compiler is fully relocatable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	369 (because of the compiler wrapper), so any normal user can extract it into their
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	370 home directory, add cross-compiler-$TARGET/bin to their $PATH, and run
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	371 $TARGET-gcc to create target binaries.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	372 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	373
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	374 <li><p><b>mini-native.sh</b> - Use the cross compiler to create
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	375 a minimal native build environment for the target platfrom.<p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	376
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	377 <p>This script takes one argument: the architecture to build for.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	378
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	379 <p>This script uses the cross compiler found at
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	380 build/cross-compiler-$ARCH/bin (with $ARCH- prefixes) to build a root
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	381 filesystem for the target, as well as a target Linux kernel configured for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	382 use with qemu. A usable cross compiler is left in the build directory by
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	383 the cross-compiler.sh script, or you can install your own.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	384
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	385 <p>The basic root filesystem consists of busybox and uClibc. If the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	386 configuration variable NATIVE_TOOLCHAIN is set (this is enabled by
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	387 default), this script adds a native compiler to the target, consisting of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	388 linux kernel headers, gcc, binutils, make, and bash. It also adds distcc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	389 to potentially distribute work to cross compilers living outside the emulator.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	390 This provides a minimal native development environment, which may be expanded
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	391 by building and installing more packages under the existing root filesystem.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	392 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	393
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	394 <li><p><b>package-mini-native.sh</b> - Create an ext2 filesystem image
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	395 of the native root filesystem.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	396
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	397 <p>This script takes one argument: the architecture to package.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	398
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	399 <p>This uses genext2fs to create an ext2 filesystem image from the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	400 build/mini-native-$ARCH directory left by running mini-native.sh, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	401 creates a system-image-tarball containing the result. It first compiles
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	402 genext2fs and adds it to build/host if the host system hasn't already got a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	403 copy.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	404
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	405 <p>This script also generates a run-emulator.sh script to call the appropriate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	406 emulator, using the architecture's configuration information.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	407 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	408
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	409 <li><p><b>run-from-build.sh</b> - Runs a system image you compiled from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	410 source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	411
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	412 <p>Calls run-emulator.sh in the appropriate build/system-image-$TARGET
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	413 directory, with a 2 gigabyte <b>hdb.img</b> for /home and distcc connected
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	414 to build/cross-compiler/$TARGET. Between runs it calls e2fsck on the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	415 system image's root filesystem.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	416
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	417 <p>This is not technically a build stage, as it isn't called from build.sh,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	418 but it's offered as a convenience for users. It uses the existing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	419 cross-compiler and system-image directories in build/ and doesn't mess
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	420 with the tarballs that were created from them.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	421 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	422 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	423
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	424 <p>The following generally aren't called directly, but are used by the rest of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	425 the build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	426
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	427 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	428 <li><p><b>config</b> - User definable configuration variables</li></p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	429
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	430 <p>This file contains environment variables which you can set to customize
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	431 the FWL build process. Setting any of these variables to a nonblank value
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	432 changes the build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	433
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	434 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	435 <li><p><b>NATIVE_TOOLCHAIN</b> - This tells mini-native.sh to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	436 include a compiler toolchain (binutils, gcc, bash, make, and distcc).
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	437 Without this, it builds a small uClibc/busybox system. This is the only
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	438 variable enabled by default in config.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	439
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	440 <p>Setting NATIVE_TOOLCHAIN="headers" will leave the libc and kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	441 header files in the appropriate include directory, for use by some other native
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	442 compiler. Building and installing additional tools (such as "make", or a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	443 compiler such as pcc, llvm/clang, or tinycc) then becomes your problem.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	444 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	445
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	446 <li><p><b>NATIVE_TOOLSDIR</b> - This tells mini-native.sh to change the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	447 directory layout to conform to a Linux From Scratch "intermediate" system, with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	448 everything under a /tools directory. (This provides a cleaner environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	449 for creating a new completely customized system at the root level.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	450 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	451
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	452 <li><p><b>RECORD_COMMANDS</b> - Records all command lines used to build each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	453 package.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	454
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	455 <p>This inserts a logging wrapper in the $PATH which logs the command lines
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	456 used by the build. Afterwards, the script
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	457 "sources/toys/report_recorded_commands.sh" can generate a big report on which
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	458 commands were used to build each package for each architecture. To get a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	459 single list of the command names used by everything, do:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	460
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	461 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	462 <p>echo $(find build -name "cmdlines.*" \| xargs awk '{print $1}' \| sort -u)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	463 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	464
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	465 <p>(Note: this will miss things which which call executables at absolute values
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	466 instead of checking $PATH, but the only interesting ones so far are the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	467 #!/bin/bash type lines at the start of shell scripts.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	468 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	469
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	470 <li><p><b>CROSS_BUILD_STATIC</b> - Tells cross-compiler.sh to statically link
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	471 all binaries in the cross compiler toolchain it creates.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	472
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	473 <p>The prebuilt binary versions in the download directory are statically linked
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	474 against uClibc, by building a mini-native environment and re-running the build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	475 under that with CROSS_BUILD_STATIC=1. The sources/build-all-targets.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	476 script can do this automatically with the "--use-static-host $TARGET"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	477 argument. (Requires QEMU installed.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	478
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	479 <li><p><b>PREFERRED_MIRROR</b> - Tells download.sh to try to download
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	480 packages from this URL first, before falling back to the normal mirror list.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	481 For example, "PREFERRED_MIRROR=http://impactlinux.com/fwl/mirror".</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	482
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	483 <li><p><b>USE_TOYBOX</b> - Tells the host-tools.sh and mini-native.sh to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	484 install the <a href=http://impactlinux.com/code/toybox>toybox</a> implementation
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	485 of commands (where available) instead of the busybox versions. This is an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	486 alternate (simpler) implementation of many commands.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	487
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	488 <p>Currently FWL always uses the toybox "patch" command, because the busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	489 version can't apply patches at offsets.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	490 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	491
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	492 <li><p><b>USE_UNSTABLE</b> - Lists packages to build alternate "unstable"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	493 versions for.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	494
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	495 <p>The value of this config entry is a comma separate list of packages.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	496
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	497 <p>Many packages in download.sh have an UNSTABLE= tag providing a URL to an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	498 alternate version. Generally these link to newer versions, often unstable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	499 development versions, for testing purposes.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	500
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	501 <p>In addition to changing the download location, using alternate versions of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	502 packages prepends an "alt-" in front of the package name in various places
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	503 (such as the patches from the sources/patches directory and the configuration
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	504 files used from sources/targets). It changes the behavior of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	505 "download" and "setupfor" shell functions.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	506 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	507
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	508 <li><p><b>USE_COLOR</b> - Color code the various build stages.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	509
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	510 <p>Enabling this provides a quick visual indicator of which build stage is in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	511 progress.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	512
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	513 <p>This is disabled by default both because its utility is a matter of taste,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	514 and because finding a half-dozen different colors that work on both white and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	515 black backgrounds is hard, and gnome-terminal can't produce an actual black
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	516 background. (In its default palette, "black" is a fairly light grey.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	517 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	518 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	519 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	520
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	521 <li><p><b>sources/build-all-targets.sh</b> - Build all supported targets
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	522 at once.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	523
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	524 <p>This performs a similar function to build.sh, but for all targets instead
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	525 of just one. It can build targets in parallel with the --fork option, logs the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	526 output of the various build stages, and generates a README.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	527
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	528 <p>This script populates a second output directory, buildall, with its output.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	529 This is probably only of interest to FWL's developers.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	530 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	531 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	532
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	533 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	534 <a name="how_implemented"><h1>How is Firmware Linux implemented?</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	535
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	536 <h2>Directory layout</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	537
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	538 <p>The top level directory of FWL contains the user interface of FWL (scripts
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	539 the user is expected to call or edit), and the "sources" directory containing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	540 code that isn't expected to be directly called by end users.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	541 Important directories under sources include:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	542
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	543 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	544 <li><p><b>sources/targets</b> - Configuration information for each target.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	545
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	546 <p>Adding a new target to FWL involves creating a new directory under here
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	547 (which determines the name of the target), and adding two miniconfig files
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	548 (for linux and uClibc), and a "details" file defining environment variables.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	549 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	550
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	551 <li><p><b>sources/packages</b> - Source tarballs for the packages to be built.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	552 This directory starts empty, and is populated by download.sh.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	553
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	554 <li><p><b>sources/native</b> - This directory hierarchy is copied into the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	555 target verbatim (under /usr). It contains the boot script and some sample
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	556 source code.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	557
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	558 <li><p><b>sources/toys</b> - Build utilities, mostly original code written for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	559 FWL. Not necessarily specific to this project, but can't be downloaded from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	560 somewhere else.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	561 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	562
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	563 <p>Output files from running the build scripts, and all temporary files, go
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	564 in the "build" subdirectory. This entire directory can be deleted between
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	565 builds.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	566
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	567 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	568 <li><p><b>build/sources</b> - cached copies of the extracted source tarballs,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	569 so setupfor can "cp -lfR" instead of having to re-extract and re-patch the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	570 source each time.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	571
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	572 <li><p><b>build/host</b> - Output of host-tools.sh. If this directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	573 exists and contains a "busybox" executable, include.sh will set the $PATH to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	574 point only to this directory.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	575
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	576 <li><p><b>build/temp-$TARGET</b> - Temporary directory for building each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	577 target. Feel free to delete this between runs, it should be empty unless
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	578 a build broke, in which case it has the source tree that failed to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	579 build. (The temporary directory for host-tools.sh is "host-temp", in case
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	580 someday somebody creates a $TARGET named "host".)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	581
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	582 <li><p><b>build/cross-compiler-$TARGET</b> - Output directory for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	583 cross-compiler.sh. The corresponding cross-compiler tarball is just an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	584 archive of this directory. Used by mini-native.sh.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	585
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	586 <li><p><b>build/mini-native-$TARGET</b> - Output directory for mini-native.sh.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	587 The corresponding mini-native tarball is just an archive of this directory.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	588 Used by package-mini-native.sh.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	589
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	590 <li><p><b>build/system-image-$TARGET</b> - Output directory for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	591 package-mini-native.sh. The corresponding system-image tarball is just an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	592 archive of this directory. Used by run-from-build.sh.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	593 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	594
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	595 <h2>Shared infrastructure</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	596
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	597 <p>The top level file for the behind-the-scenes plumbing is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	598 <b>sources/include.sh</b>. This script is not run directly, but is instead
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	599 included from the other scripts. It does a bunch of things:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	600
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	601 <li><p>It parses the "config" file at the top directory, reading in the user
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	602 defined configuration variables. (You can also supply these as environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	603 variables, if you want to specify them for just one run.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	604
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	605 <li><p>It sets several other environment variables, specifying things like the
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	606 $SOURCE and $BUILD directories, and detecting the number of $CPUS. Many
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	607 of these are set using the <b>export_if_blank</b> function, which keeps
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	608 any existing value of the variable, allowing them to be externally
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	609 overridden.</p></li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	610
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	611 <li><p>It adjusts the $PATH. If build/host exists and contains a busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	612 executable (meaning host-tools.sh did its thing already), $PATH is set to just
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	613 that directory. If build/wrapdir exists, that's used instead for command
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	614 line logging via sources/more/record-commands.sh.</p></li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	615
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	616 <li><p>If host-tools.sh ran after record-commands.sh, it sets the $PATH to
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	617 point to the logging wrapper directory. ($WRAPPY_LOGPATH specifies where
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	618 the logging wrapper should write its log file, and $WRAPPY_REALPATH says where
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	619 to find the actual commands the logging wrapper hands off to.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	620 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	621
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	622 <p>It also reads <b>sources/functions.sh</b>, which provides shell functions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	623 used by the rest of the build, including:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	624
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	625 <ul>
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	626 <ul>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	627 <li><p><b>read_arch_dir</b> - parses the appropriate sources/targets
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	628 directory to read architecture information and set lots of environment
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	629 variables. It takes one argument, the architecture name to build. If
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	630 run with no arguments, it outputs all available architectures by listing
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	631 the subdirectories under sources/targets.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	632
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	633 <p>All the build stages except download.sh and host-tools.sh call read_arch_dir
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	634 with their first command line argument.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	635 </li>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	636
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	637 <li><p><b>download</b> - used by download.sh. Calls wget if necessary, uses
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	638 sha1sum to verify the files. Saves the results in the directory pointed to
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	639 by $SRCDIR (set to "packages" by sources/include.sh). Treat as a fancy call
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	640 to "wget".</p></li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	641
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	642 <li><p><b>dienow</b> - abort the current script, exiting with an error message.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	643 (Can even exit from nested shell functions.) Treat as a fancy "exit".</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	644
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	645 <li><p><b>setupfor</b> - extract a source package (named in the first argument)
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	646 into a temporary directory (under $WORK), and change the current directory to
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	647 there. Treat as a fancy "tar -xvjf" followed by cd.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	648
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	649 <p>Source code is cached, meaning each package's source tarball is only
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	650 actually extracted and patched once (into build/packages) and the temporary
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	651 copies are directories full of hard links (or optionally symlinks) to the
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	652 cached source.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	653 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	654
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	655 <li><p><b>cleanup</b> - delete temporary copy of source code after build.
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	656 Treat as a fancy "rm -rf" (except that it remembers the working directory
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	657 from the last setupfor call, rather than reqiring it to be specified on
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	658 the command line).</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	659
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	660 <p>If the exit code of the last command was nonzero,
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	661 it calls dienow instead of deleting the source code that didn't build
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	662 properly, to preserve the evidence of what went wrong.</p></li>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	663
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	664 <li><p><b>maybe_fork</b> - If the environment variable <b>FORK</b> is set, run
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	665 the first argument in the background. Otherwise, run it in the foreground.
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	666 (This is used with the <b>wait</b> command, which blocks until all background
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	667 jobs belonging to this shell have finished. If this shell has no background
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	668 processes wait returns immediately.)</p></li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	669 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	670
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	671 <p>Most of what these shell functions do is optional. Most of it's there to
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	672 speed up and simplify the rest of the build, and perform error checking.
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	673 None of it should be very important to understanding how to build or install
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	674 any of the actual packages. It just abstracts away repetitive, uninteresting
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	675 bits.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	676
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	677 <h2>Downloading source code</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	678
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	679 <p>The FWL source distribution does not include any third party source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	680 tarballs. Instead, these are downloaded by running download.sh, which calls
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	681 the shell function <b>download</b>, which calls wget as necessary. The
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	682 download.sh script contains a series of calls to the download function.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	683
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	684 <p>Only calls to the shell function "download" contain version information for
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	685 packages. The scripts that actually build the packages do not, they are as
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	686 version agnostic as possible.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	687
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	688 <p>The following environment variables control the behavior of download.sh,
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	689 and may be set before calling it:</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	690
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	691 <ul>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	692 <li><p><b>EXTRACT_ALL</b> - prepopulate the source cache.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	693
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	694 <p>If set, each call to the download function will extract and patch the
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	695 corresponding source tarball (into the sources/packages cache directory)
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	696 immediately after the download completes, instead of waiting for setupfor to
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	697 do it. (See "Extracting source code", below.)</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	698 </li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	699
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	700 <li><p><b>FORK</b> - Calls to download are usually wrapped in maybe_fork,
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	701 so if this is set they run in parallel.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	702 </li>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	703
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	704 <li><p><b>PREFERRED_MIRROR</b></p> - This contains the URL of a mirror site
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	705 to be checked _before_ downloading from the actual $URL specified in
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	706 download.sh.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	707
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	708 <p>This allows download.sh to fetch some or all of its packages from a local
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	709 mirror of the files, instead of going out to the net. Any files not found
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	710 in this mirror will be fetched from the standard URL, and the fallback mirrors
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	711 as necessary.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	712
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	713 <p>(Note: inside qemu the special address 10.0.2.2 passes through connections
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	714 to 127.0.0.1 on the host, so if you run a web server on your host's loopback
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	715 address you can pass source code into the emulator without going out
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	716 to an external network.)</p></li>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	717 </ul>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	718
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	719 <p>The following environment variables control each call to the
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	720 download function, and are set before each call:<p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	721
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	722 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	723 <li><p><b>URL</b> - The URL from which to download this source package
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	724 into the sources/packages directory.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	725
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	726 <p>In addition to specifying a web location, this URL determines the name of the
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	727 source package to fetch. If this source tarball cannot be fetched from this
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	728 location, the download function tries to download the file from a series of
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	729 fallback mirrors (stored in the variable MIRROR_LIST, set in include.sh).
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	730 The primary mirror is http://impactlinux.com/firmware/mirror
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	731 which should have every source tarball used by the build.</p></li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	732
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	733 <p>The package name is the filename at the end of URL minus any version
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	734 information and file type extensions, so "bash-2.04b.tar.bz2" becomes "bash".
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	735 The shell function "basename" uses a rather complicated regex to
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	736 extract the package name from a URL. This versionless package name is used
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	737 by things like setupfor, allowing the build scripts to mostly ignore the
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	738 versions of the packages they build.</p></li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	739
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	740 <li><p><b>SHA1</b> - The sha1sum of the source tarball to fetch.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	741
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	742 <p>Used to confirm that the downloaded file is correct. If not, it tries
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	743 the next mirror in the list, or calls dienow if out of mirrors.</p>
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	744
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	745 <p>If this value is blank, the sha1sum calculated from the file will be
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	746 displayed but not verified. This means any file will be accepted as correct
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	747 as long as it exists with the right name, but the build won't be able to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	748 detect corrupted or truncated files.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	749
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	750 <p>When updating to a new version of a package, a common trick is to update
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	751 the URL and blank the SHA1, run ./download.sh to fetch the new file, cut and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	752 paste the SHA1 value displayed after the download to set the SHA1 variable,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	753 and then re-run ./download.sh to confirm they match.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	754
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	755 <li><p><b>UNSTABLE</b> - URL to an alternate version of the file, for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	756 testing purposes.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	757
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	758 <p>This version is only downloaded when USE_UNSTABLE contains the name of this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	759 package in its package list. It doesn't fall back to check the mirror list,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	760 and is not affected by PREFERRED_MIRROR.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	761
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	762 <p>Unstable packages are saved as a tarball called "alt-$PACKAGE-0" plus the
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	763 file type extension. (Thus the name to save is based on the filename in the
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	764 normal $URL rather than on what the $UNSTABLE address points to, so even if
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	765 your UNSTABLE address ends with "snapshot.tgz" or "tip.tar.bz2", it will
655 62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	766 still wind up somewhere the rest of the build can find it.)</p></li>
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	767
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	768 <li><p><b>RENAME</b> - regex to rename a downloaded file.</p>
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	769
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	770 <p>This is a "sed -r" extended regular expression with which to rename a file.
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	771 The "setupfor" function expects filenames in "$PACKAGE-$VERSION.$TYPE" format.
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	772 If a source package at $URL isn't named that way (such as squashfs not having
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	773 a dash between the package name and version), you can adjust it with this.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	774 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	775
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	776 <p>At the end of download.sh is a call to the shell function <b>cleanup_oldfiles</b>,
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	777 which deletes unused files. The include.sh snapshots the current time in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	778 the variable $START_TIME, and download calls "touch" to update the timestamp on
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	779 each file it verifies the sha1sum of. Then cleanup_oldfiles deletes every
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	780 file from sources/packages with a date older than $START_TIME. (It does
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	781 not recurse into subdirectories.)</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	782
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	783 <p>Note that download updates the timestamp on stable packages when
1040 b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	784 downloading corresponding unstable packages (and vice versa), so
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	785 cleanup_oldfiles won't delete them. In this special case they're not
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	786 considered "unused files", but it won't verify their integrity or fetch
b630081630a7 Serious upgrade of the "What is?" section. Eric S. Raymond <esr@thyrsus.com> parents: 881 diff changeset	787 them if they're not already there.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	788
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	789 <h2>Extracting source code</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	790
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	791 <p>The function "setupfor" extracts <b>sources/packages/$PACKAGENAME-*</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	792 tarballs. (If $PACKAGENAME is found in the comma separated $USE_UNSTABLE list,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	793 the build adds an "alt-" prefix to the package name.) This populates a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	794 corresponding directory under build/sources, and applies all the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	795 <b>sources/patches/$PACKAGENAME-*.patch</b> files in alphabetical order.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	796 (So if a package has multiple patches that need to be applied in a specific
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	797 order, name them something like "bash-001-dothingy.patch",
655 62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	798 "bash-002-next.patch" to control this.)</p>
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	799
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	800 <p>The trailing "-" before filename wildcards prevents collisions between
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	801 things like "uClibc" and "uClibc++". Packages are allowed to contain dashes
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	802 (such as gcc-core), but cannot have a digit immediately after the dash.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	803
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	804 <p>FWL implements source cacheing. The first call to setupfor extracts the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	805 package into build/sources, and then creates a directory of hard links in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	806 current target's build/temp-$TARGET directory with cp -lfR. Later setupfor
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	807 calls just create the directory of hard links from the existing source tree.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	808 (This is a hybrid approach between building "out of tree" and building
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	809 in-tree.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	810
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	811 <p>The <b>./download.sh --extract</b> option prepopulates the source cache,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	812 extracting and patching each source tarball. This is useful for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	813 scripts such as sources/build-all-targets.sh which perform multiple builds in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	814 parallel.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	815
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	816 <p>The reason for keeping extracted source tarballs around is that extracting
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	817 and patching tarballs is a fairly expensive operation, which uses a significant
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	818 amount of disk space and doesn't parallelize well. (It tends to be disk
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	819 limited as much as CPU limited, so trying for more parallelism wouldn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	820 necessarily help.) In addition, the same packages are repeatedly extracted:
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	821 the cross-compiler and mini-native stages use many of the same packages, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	822 some packages (such as the Linux kernel) are extracted and removed repeatedly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	823 to grab things like kernel headers separately from actually building a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	824 bootable kernel. (Also, different architectures build the exact same
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	825 packages, with the same set of patches. Even patches to fix a bug on a single
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	826 architecture are applied for all architectures; if this causes a problem, it's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	827 not a mergeable patch capable of going upstream.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	828
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	829 <h2>Building host tools</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	830
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	831 <p>The host-tools.sh script sets up the host environment. Usually the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	832 host environment is already in a usable state, but this script explicitly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	833 enumerates exactly what we need to build, and provides our own (known)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	834 versions of everything except the host compiler toolchain in the directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	835 <b>build/host</b>. Once we've finished, the $PATH can be set to just that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	836 directory.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	837
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	838 <p>The build calls seven commands from the host compiler toolchain: ar, as,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	839 nm, cc, gcc, make, and ld. All those have to be in the $PATH, so host-tools.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	840 creates symlinks to those from the original $PATH.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	841
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	842 <p>Next host-tools.sh builds toybox for the "patch" command, because busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	843 patch can't simple handle offsets and is thus extremely brittle in the face of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	844 new package versions. (This is different from "fuzz factor", which removes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	845 context lines to find a place to insert a patch, and tends to break a lot.)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	846 If USE_TOYBOX is enabled, a defconfig toybox is used and all commands are
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	847 installed.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	848
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	849 <p>Next host-tools builds a "defconfig" busybox and installs it into
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	850 build/host. This provides all the other commands the build needs.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	851
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	852 <h3>What's the minimum the build actually needs?</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	853
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	854 <p>When building a new system, environmental dependencies are a big issue.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	855 Figuring out what package needs what, and what order to build things in,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	856 is the hardest part of putting together a system.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	857
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	858 <p>Running the build without build/host calls lots of extra commands, including
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	859 perl, pod2man, flex, bison, info, m4, and so on. This is because the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	860 ./configure stages of the various packages detect optional functionality,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	861 and use it. One big reason to limit the build environment is to consistently
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	862 produce the same output files, no matter what's installed on the host.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	863
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	864 <p>The minimal list of commands needed to build a working system image is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	865 1) a working toolchain (ar, as, nm, cc, gcc, make, ld), 2) /bin/bash (and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	866 a symlink /bin/sh pointing to it), 3) the following command line utilities
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	867 in the $PATH:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	868
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	869 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	870 <p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	871 awk basename bzip2 cat chmod chown cmp cp cut date dd diff
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	872 dirname echo egrep env expr find grep gzip hostname id install ln ls mkdir
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	873 mktemp mv od patch pwd readlink rm rmdir sed sha1sum sleep sort tail tar
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	874 touch tr true uname uniq wc which whoami xargs yes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	875 </p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	876 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	877
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	878 <p>These commands are supplied by current versions of busybox.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	879
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	880 <p>Bash has been the standard Linux shell since before the 0.0.1 release in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	881 1991, and is installed by default on all Linux systems. (Ubuntu broke its
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	882 /bin/sh symlink to point to the Defective Annoying SHell, so many scripts call
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	883 #!/bin/bash explicitly now rather than relying on a broken symlink.) We
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	884 can't stop the build from relying on the host version of this tool; editing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	885 $PATH has no effect on the #!/bin/bash lines of shell scripts.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	886
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	887 <p>The minimal set of commands necessary to build a system image was
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	888 determined experimentally, by running a build with $RECORD_COMMANDS and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	889 then removing commands from the list and checking the effect this had on
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	890 the build. (Note that the minimal set varies slightly from target to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	891 target.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	892
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	893 <p><b>$RECORD_COMMANDS</b> tells host-tools.sh to set up a logging wrapper
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	894 that intercepts each command line in the build and writes it to a log file, so
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	895 you can see what the build actually uses. (Note that when host-tools.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	896 sets up build/wrapper, it doesn't set up build/host, so the build still uses
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	897 the host system's original command line utilities instead of building busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	898 versions. If you'd like to record the build using build/host commands,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	899 run host-too.sh without $RECORD_COMMANDS set and then run it again with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	900 $RECORD_COMMANDS to set up the logging wrapper pointing to the busybox
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	901 tools.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	902
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	903 <p>The way $RECORD_COMMANDS works is by building a logging wrapper
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	904 (sources/toys/wrappy.c) and populating a directory (build/wrapper) with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	905 symlinks to that logging wrapper for each command name in $PATH. When later
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	906 build stages run commands, the wraper appends the command line to the log file
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	907 (specified in the environment variable $WRAPPY_LOGPATH, host-tools.sh sets
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	908 this to "$BUILD/cmdlines.$STAGE_NAME.$PACKAGE_NAME"), recording
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	909 each command run. The logging wrapper then searches $WRAPPY_REALPATH to find
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	910 the actual command to hand its command line off to.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	911
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	912 <h2>Building a cross compiler</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	913
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	914 <p>We cross compile so you don't have to. The point of this project is to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	915 make cross compiling go away, but you need to do some to get past it. So
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	916 let's get it over with.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	917
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	918 <p>The <b>cross-compiler.sh</b> script builds a cross compiler. Its output
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	919 goes into <b>build/cross-compiler-$TARGET</b> directory, which is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	920 deleted at the start of the build if it already exists, so re-running this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	921 script always does a clean build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	922
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	923 <p>Creating a cross compiler is a five step process:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	924
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	925 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	926 <li><p><b>binutils</b> - Build assembler and linker for the target platform.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	927
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	928 <p>This package has no interesting dependencies, and thus can be the first
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	929 thing you build for a target.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	930 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	931
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	932 <li><p><b>gcc</b> - Build C/C++ compiler for the target platform.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	933
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	934 <p>This package needs binutils, and must be built after that. It does not
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	935 need a C library, so can be built before that.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	936
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	937 <p>The mini-native build doesn't require C++ support, but the build adds
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	938 gcc-g++ to the basic gcc-core and enables C++ support so the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	939 <a href="#distcc_trick">distcc accelerator trick</a> can speed up C++
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	940 builds.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	941
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	942 <p>We create an "xgcc" symlink pointing to the host compiler to force gcc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	943 not to attempt to rebuild itself with itself. (It needed to be able to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	944 build xgcc with the host compiler, but doesn't trust the host compiler to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	945 build an actual binary to deploy. Note that this xgcc builds _host_ binaries,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	946 not target binaries.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	947 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	948
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	949 <li><p><b>compiler wrapper</b> - Install a wrapper around gcc to enforce sane
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	950 path logic.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	951
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	952 <p>This builds a wrapper for gcc from "sources/toys/gcc-uClibc.c". This
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	953 compiler wrapper rewrites the gcc command line to start with --nostdinc and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	954 --nostdlib, and then explicitly adds the correct header and library search
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	955 paths, and when linking adds the correct object files and libraries.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	956
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	957 <p>It needs to do this because gcc's path logic has been consistently broken
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	958 for about two decades now. (See <a href="#why_cross_compiling_sucks">why
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	959 cross compiling sucks</a> for more details.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	960
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	961 <p>The compiler hands off the new command line to $ARCH-rawgcc, so the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	962 old $ARCH-gcc gets renamed to that and the wrapper gets the old name.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	963
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	964 <p>To allow the compiler wrapper to easily find the headers and libraries,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	965 the build moves them to known locations. The system headers and libraries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	966 go into "include" and "lib" directories at the same level as the "bin"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	967 directory containing the wrapper script, and gcc's own headers and libraries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	968 go into "gcc/include" and "gcc/lib". The wrapper then finds itself (using
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	969 argv[0] and if necessary searching the $PATH it inherits), and backs up one
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	970 level to find the headers and libraries it needs to add to the gcc path.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	971 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	972
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	973 <li><p><b>linux</b> - kernel headers.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	974
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	975 <p>This package doesn't have any prerequisites, but C libraries need it
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	976 to build themselves. (Kernel headers define the system call API for the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	977 Linux kernel.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	978 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	979
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	980 <li><p><b>uClibc</b> - uClibc (micro C library).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	981
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	982 <p>This package is target code that needs to be built with a cross compiler
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	983 (gcc and binutils), and also needs kernel headers. It requires all three of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	984 the other packages, and thus must be built last.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	985
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	986 <p>Note that we only build a standard C library. We don't build/install
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	987 a standard C++ library (uClibc++), because distcc doesn't need headers
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	988 or libraries in the cross compiler. Thus the cross compiler has enough
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	989 C++ support to be used from the native environment via distcc, but not
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	990 enough to cross compile C++ code on its own.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	991
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	992 <p>The compiler wrapper actually uses links to "libc.so", which is a linker
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	993 script pointing to libuClibc.so.0. We patch uClibc so it doesn't put absolute
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	994 paths into its libc.so; without them the linker searches the supplied library
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	995 search paths, and thus the compiler may be installed in an arbitrary
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	996 location.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	997 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	998 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	999
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1000 <p>Afterwards the build strips some of the binaries, tars up the result,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1001 and performs some quick sanity tests (building dynamic and static versions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1002 of hello world. If the target configuration lists a version of QEMU to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1003 test individual binaries under on the host, it runs the static version
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1004 to make sure it outputs "Hello world".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1005 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1006
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1007 <h2>Building a minimal native development environment for the target system</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1008
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1009 <p>The <b>mini-native.sh</b> script uses the cross compiler from the previous
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1010 step to build a kernel and root filesystem for the target. The resulting
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1011 system should boot and run under an emulator, or on real target hardware.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1012
531 4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1013 <p>If you really want to learn how to cross compile a target system, this
4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1014 is the script you want to read, and possibly append your own packages to.
4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1015 That said: please don't, and here's why:</p>
4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1016
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1017 <p>Because cross-compiling is persnickety and difficult, we do as little of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1018 it as possible. This script should perform all the cross compiling anyone ever
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1019 needs to do. It uses the cross-compiler to generate the simplest possible
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1020 native build environment for the target which is capable of rebuilding itself
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1021 under itself.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1022
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1023 <p>Anything else that needs to be built for the target can then be built
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1024 natively, by running this kernel and root filesystem under an emulator and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1025 building new packages there, bootstrapping up to a full system if necessary.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1026 The emulator we use for this is QEMU. Producing a minimal build environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1027 powerful enough to boot and compile a complete Linux system requires seven
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1028 packages: the Linux kernel, binutils, gcc, uClibc, BusyBox, make, and bash.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1029 We build a few more than that, but those are optional extras.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1030
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1031 <p>This root filesystem can also be packaged using the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1032 <a href=http://www.linuxfromscratch.org/lfs>Linux From Scratch</a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1033 /tools directory approach, staying out of the way so the minimal build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1034 environment doesn't get mixed into the final system, by setting the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1035 $NATIVE_TOOLSDIR environment variable. If you don't know why you'd want
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1036 to do that, you probably don't want to.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1037
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1038 <p>In either configuration, the main target directory the build installs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1039 files into is held in the environment variable "$TOOLS". If $NATIVE_TOOLSDIR
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1040 is set this will be "/tools" in the new root filesystem, otherwise it'll
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1041 be "/usr".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1042
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1043 <p>The steps the script goes through are:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1044
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1045 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1046 <li><p><b>directory setup</b> - Create empty directories for the basic
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1047 filesystem layout.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1048
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1049 <p>If $NATIVE_TOOLSDIR is set, build script will create a Linux From Scratch
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1050 style intermediate system by moving the filesystem layout under /tools,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1051 which means skipping the top level directories and installing most files
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1052 into /tools instead of /usr. This also sets the variable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1053 $UCLIBC_DYNAMIC_LINKER to tell the compiler wrapper to create binaries that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1054 depend on shared libraries in /tools rather than the default
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1055 "/lib/ld-uClibc.so.0". (With the /tools layout, the qemu-setup.sh script can
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1056 recreate most of the top level directories at runtime, often as symlinks
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1057 into /tools.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1058 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1059
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1060 <li><p><b>Copy sources/native</b> - The most important thing here is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1061 the qemu-setup.sh script, but there's also example source code in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1062 src directory.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1063
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1064 <li><p><b>Linux kernel</b> - Build a kernel that can boot under QEMU.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1065
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1066 <p>We need kernel headers to build uClibc, so install those while we've got the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1067 kernel tarball extracted. (We could grab these files directly from the cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1068 compiler, but we rebuild from source to keep the layers cleanly separated.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1069
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1070 <p>The kernel build uses <b>sources/targets/$ARCH/miniconfig-linux</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1071 to configure the kernel for the appropriate QEMU target, and the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1072 $KERNEL_PATH variable to figure out which kernel image file to use.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1073
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1074 <li><p><b>uClibc</b> - Build standard C library.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1075
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1076 <p>The binaries in the target system are dynamically linked, so we need
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1077 shared libraries installed. Again, we could grab these files out of the cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1078 compiler, but we rebuild from source to keep the layers cleanly separated.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1079
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1080 <p>We unconditionally install the development files (headers and static
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1081 libraries), and delete them later if $NATIVE_TOOLCHAIN isn't set.</p>
531 4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1082
4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1083 <p>Right after installing the C library, we export the environment variable
4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1084 $WRAPPER_TOPDIR which tells the compiler wrapper to links against the new
4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1085 headers and shared libraries we've installed into the new root filesystem,
4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1086 rather than the ones out of the cross compiler's include and lib
4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1087 directories.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1088 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1089
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1090 <li><p><b>toybox</b> - Build optional command line utilities.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1091
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1092 <p>This isn't strictly required. If $USE_TOYBOX isn't set, this only
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1093 installs symlinks for the "patch" command and "oneit" commands. (The
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1094 oneit command is similar to init=/bin/sh, except it allows terminal control
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1095 to work and shuts the system down cleanly on exit. It's used by
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1096 qemu-setup.sh to provide a more forgiving command line.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1097
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1098 <p>If $USE_TOYBOX is set, this installs toybox versions of many commands
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1099 instead of the busybox versions. These tend to be simpler, more
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1100 straightforward implementations than the busybox versions. (Note: your author
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1101 is biased here.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1102 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1103
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1104 <li><p><b>busybox</b> - Build busybox command line utilities.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1105
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1106 <p>This provides the bulk of the command line utlities for the new
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1107 system.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1108
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1109 <p>Once upon a time, "make defconfig" provided the largest sane configuration
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1110 in busybox, enabling every working command and feature that didn't have
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1111 undesirable side effects (such as debugging options) or require special
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1112 configuration to use (such as SELINUX). Unfortunately, over time this goal
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1113 was lost and make defconfig bit-rotted into a fairly random configuration.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1114
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1115 <p>To recapture the original "largest sane configuration" goal, the build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1116 starts with "<b>make allyesconfig</b>" and applies
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1117 <b>sources/trimconfig-busybox</b> to remove features that would otherwise
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1118 cause problems. The trimconfig file has comments in it if you're wondering
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1119 why specific features are disabled.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1120 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1121
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1122 <li><p><b>Check $NATIVE_TOOLCHAIN</b> - Build a native development toolchain
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1123 only if $NATIVE_TOOLCHAIN is set.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1124
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1125 <p>$NATIVE_TOOLCHAIN is the only configuration option set by default. You
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1126 can disable it in "config" if you want to build skeletal target system and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1127 add your own software to it by hand.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1128
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1129 <p>If it is enabled, the following happens:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1130 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1131 <li><p><b>binutils</b> - Build a native assembler and linker.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1132
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1133 <li><p><b>gcc and libsupc++</b> - Build a native C and C++ compiler.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1134
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1135 <p>This process is still a bit tangled. The fundamental reason for this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1136 is that the gcc build process is pathologically misdesigned. (See
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1137 <a href="#gcc_sucks">what the hell is wrong with gcc</a> for a long
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1138 digression into the details.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1139
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1140 <p>The secondary reason is that libstdc++ is built into gcc, which makes as much
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1141 sense as building glibc into gcc. GCC's C++ support is not cleanly separated
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1142 into layers, so replacing their built-in libstdc++ with the much smaller
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1143 uClibc++ requires performing additional surgery on the gcc build process
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1144 to get it to stop being actively stupid. (For simplicity we punted on this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1145 while building the cross compiler, but now we need to make it work.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1146
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1147 <p>So after beating gcc over the head with almost a dozen different environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1148 variables and a bunch of ./configure options to get it to cross compile like
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1149 a normal program, we then have to chdir into the <b>libsupc++</b> subdirectory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1150 to build a static library which uClibc++ needs in order to interface properly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1151 with the compiler. (It defines things like stack unwinding and the current
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1152 exception model, which the C++ library needs to know but which gcc doesn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1153 cleanly export for external use.) Logically this step belongs with the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1154 uClibc++ build, but we have to export this information from the gcc source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1155 directory because that's where it lives.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1156
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1157 <p>We also clean up after a bug where gcc uses multilib directories (such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1158 /lib64) on some systems even when we explicitly told it we didn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1159 want multilib. (This package isn't very good at taking "no" for an answer.)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1160 And we create a "cc" symlink to gcc, because some packages use that as their
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1161 compiler and SUSv3 says we should have one.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1162 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1163
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1164 <li><p><b>compiler wrapper</b> - Wrap gcc, to control path logic.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1165
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1166 <p>The native build still installs the compiler wrapper
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1167 (from sources/toys/gcc-uClibc.c) to rewrite gcc's command line arguments
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1168 and bypass its built-in path logic. In theory native
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1169 compiling is less tricky and the final location we're installing the compiler
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1170 at is known at compile time, so we could just patch the compiler's source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1171 code to check the right paths. But going there rapidly turns into a nightmare
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1172 of tangled historical scar tissue, and breaks in new and exciting ways
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1173 with each new gcc release. The only way to get gcc to use sane paths is to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1174 take path decisions out of its hands entirely.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1175
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1176 <p>This does the same header/library shuffling and symlink creation as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1177 the cross compiler did, but without a prefix on the symlink names this
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1178 time.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1179 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1180
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1181 <li><p><b>uClibc++</b> - Build micro standard c++ library.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1182
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1183 <p>C++ has its own standard library, and its own standard header files,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1184 without which the overloaded bit shift operators can't even perform I/O.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1185 The package uClibc++ provides much smaller and simpler versions of these
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1186 than the libstdc++-v3 metastasized through gcc-g++.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1187
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1188 <p>This package mostly builds out of the box, assuming the cross compiler has
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1189 minimal c++ support and you have the right pliers to extract libsupc++ from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1190 the gcc build. We start with the defconfig, switch off TLS and LONG_DOUBLE
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1191 support that uClibc doesn't currently provide, and blank the RUNTIME_PREFIX
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1192 so it installs where we tell it to. Then we shuffle the libraries around
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1193 so the compiler wrapper can find them and make symlinks from the generic
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1194 "libstdc++.{so,a}" names to the corresponding libuClibc++ files.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1195 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1196
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1197 <li><p><b>make</b> - Build make</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1198
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1199 <p>A toolchain doesn't do you much good without the "make" command. Fairly
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1200 straightforward to build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1201 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1202
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1203 <li><p><b>bash</b> - The standard Linux command shell.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1204
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1205 <p>Bash has been the standard Linux command shell since 1991, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1206 lots of scripts explicitly say #!/bin/bash. In addition, bash extensions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1207 like curly brace filename expansion are in common use.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1208
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1209 <p>Someday, busybox might provide a decent replacement for bash, but since
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1210 busybox has four different shells (lash, hush, msh, ash) which don't share a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1211 lot of code, development is fragmented and proceeds slowly. A bash replacement
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1212 will need to be callable as "#!/bin/bash" since debian pointed #!/bin/sh at the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1213 Defective Annoying SHell and greatly discouraged use of that symlink.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1214
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1215 <p>We intentionally build an older version of bash (2.04b) which is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1216 sufficient for our purposes, and much smaller and simpler than the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1217 current bash 3.x monsters. We have to hardwire a few things ./configure
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1218 entries because this version doesn't like cross compiling, and we do so
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1219 by supplying a config.cache file with the appropriate entries. It also doesn't
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1220 work if you try to build in parallel, so we don't supply -j.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1221 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1222
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1223 <li><p><b>distcc</b> - command to distribute compiles across a network
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1224 cluster.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1225
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1226 <p>We install this for <a href="#distcc_trick">the distc accelerator
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1227 trick</a>. It's entirely optional.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1228
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1229 <p>We create a $TOOLS/distcc directory full of symlinks to distcc with the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1230 names of gcc, cc, g++, and c++. Inserting that directory at the start of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1231 $PATH makes the build use distcc in place of the normal native compiler.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1232 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1233 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1234
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1235 <p>That's everything in the $NATIVE_TOOLCHAIN. The rest is minor cleanup
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1236 and packaging.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1237
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1238 <li><p><b>Build static and dynamic hello world binaries</b></p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1239
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1240 <p>These are installed into $TOOLS/bin as hello-dynamic and hello-static.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1241 These are debugging tools: If you can't boot the system to a shell prompt,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1242 try running hello-static as init to see if it runs and gives you output.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1243 If that works try hello-dynamic to see if shared libraries are loading.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1244
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1245 <li><p><b>Strip some binaries</b> to save space.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1246
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1247 <li><p><b>Create the mini-native tarball</b> - we're done.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1248 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1249
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1250 <p>In theory, you can add more packages to mini-native.sh, or run another
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1251 similar script to use the cross compiler to produce output into the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1252 mini-native directory. In practice, this is not recommended. Cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1253 compiling is an endless sinkhole of frustration, and the easiest way to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1254 deal with it is not to go there.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1255
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1256 <h2>Packaging up a system image to run under emulation</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1257
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1258 <p>The <b>package-mini-native.sh</b> script packages a system image for
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1259 use by QEMU. Its output goes into <b>build/system-image-$TARGET</b>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1260 directory, which is deleted at the start of the build if it already exists,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1261 so re-running this script always does a clean build.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1262
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1263 <p>The steps here are:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1264
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1265 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1266 <li><p><b>use genext2fs</b> to package the output of mini-native.sh as an 64
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1267 megabyte ext2 image.</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1268
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1269 <li><p><b>create run-emulator.sh</b> by appending an emulator invocation
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1270 command line to </b>sources/toys/run-emulator.sh</b>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1271
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1272 <p>This calls a shell function "emulator_command" from the target architecture
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1273 definition, passing in the name of the ext2 image containing the root filesystem
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1274 and the kernel image to boot. A shell function "qemu_defaults" is defined
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1275 to let emulator_command grab logs of common boilerplate, such as kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1276 command line options. (In theory run_emulator is free to use a different
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1277 emulator, or even output a command to send the files to real hardware
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1278 through a network connection or jtag or some such.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1279
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1280 <p>The path for some or the run-emulator.sh kernel command line
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1281 arguments is also adjusted based on $NATIVE_TOOLSDIR.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1282 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1283
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1284 <li><p>For the powerpc architecture, ppc_rom.bin is copied from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1285 sources/toys. (This architecture needs a custom boot rom for qemu
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1286 to be able to boot a bzImage via -kernel.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1287
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1288 <li><p>Tar up the result</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1289 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1290
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1291 <h2>Running on real hardware</a></h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1292
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1293 <p>To run a system on real hardware (not just under an emulator), you need to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1294 do several things. Dealing with myriad individual devices is beyond the scope
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1295 of this project, but the general theory is:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1296
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1297 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1298 <li><p>Figure out how to flash your device (often a jtag with openocd)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1299
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1300 <li><p>Configure and install a bootloader (uboot, apex, etc.)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1301
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1302 <li><p>Build and install a kernel targeted to your hardware (in the kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1303 source, see arch/$ARCH/configs for default .config files for various
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1304 boards)</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1305
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1306 <li><p>Package and install the root filesystem appropriately for your system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1307 (ext2, initramfs, jffs2).</p></li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1308 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1309
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1310 <a name="distcc_trick"><h2>Speeding up emulated builds (the distcc accelerator trick)</h2></a>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1311
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1312 <p>Cross compiling is fast but unreliable. The ./configure stage is
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1313 designed wrong (it asks questions about the host system it's building
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1314 on, and thinks the answers apply to the target binary it's creating).
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1315
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1316 <p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1317
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1318 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1319
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1320 <a name="why"><h1>Why do things this way</h1></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1321
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1322 <h1>UNDER DEVELOPMENT</h1>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1323
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1324 <p>This entire section is a dumping ground for historical information.
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1325 It's incomplete, lots of it's out of date, and it hasn't been integrated into
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1326 a coherent whole yet. What is here is in no obvious order.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1327
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1328 <a name="cross_compiling"><h2>Why cross compiling sucks</h2></a>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1329
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1330 <p>Cross compiling is fast but unreliable. Most builds go "./configure; make;
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1331 make install", but entire ./configure stage is designed wrong for cross
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1332 compiling: it asks questions about the host system it's building
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1333 on, and thinks the answers apply to the target binary it's creating.</p>
531 4828f4597ede Documentation tweak talking about mini-native, and warning that the last Rob Landley <rob@landley.net> parents: 526 diff changeset	1334
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1335 <p>Build processes often create temporary binaries which run during the
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1336 build (to generate header files, parse configuration information ala
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1337 kconfig, various "makedep" style dependency generators...). These builds
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1338 need two compilers, one for the host and one for the target, and need to
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1339 keep straight when to use each one.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1340
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1341 <p>Cross compilers leak host data, falling back to the host's headers and
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1342 libraries if they can't find the target files they need.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1343
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1344 <p>TODO: finish this.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1345
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1346 <a name="distcc_trick"><h2>Speeding up emulated builds (the distcc accelerator trick)</h2></a>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1347
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1348 <p>TODO: FILL THIS OUT</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1349
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1350 <h2>The basic theory</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1351
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1352 <p>The Linux From Scratch approach is to build a minimal intermediate system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1353 with just enough packages to be able to compile stuff, chroot into that, and
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1354 build the final system from there.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1355
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1356 <p>This approach completely isolates the host from the
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1357 target, which means you should be able to run the FWL build under a wide
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1358 variety of Linux distributions, and since the final system is built with a
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1359 known set of tools you should get a consistent result. It also means you
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1360 could run a prebuilt system image under a different host operating system
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1361 entirely (such as MacOS X, or an arm version of linux on an x86-64 host)
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1362 as long as you have an appropriate emulator.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1363
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1364 <p>A minimal build environment consists of a compiler, command line tools,
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1365 and a C library. In theory you just need three packages:</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1366
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1367 <ul>
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1368 <li>A C compiler.</li>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1369 <li>BusyBox</li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1370 <li>A C library (uClibc)</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1371 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1372
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1373 <p>Unfortunately, that doesn't work yet.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1374
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1375 <h2>Some differences between theory and reality.</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1376
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1377 <p>We actually need seven packages (linux, uClibc, busybox, binutils, gcc,
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1378 make, and bash) to create a working build environment. We also add an optional
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1379 package for speed (distcc), and use two more (genext2fs and QEMU) to package
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1380 and run the result.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1381
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1382 <h3>Environmental dependencies.</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1383
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1384 <p>Environmental dependencies are things that need to be installed before you
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1385 can build or run a given package. Lots of packages depend on things like zlib,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1386 SDL, texinfo, and all sorts of other strange things. (The GnuCash project
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1387 stalled years ago after it released a version with so many environmental
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1388 dependencies it was virtually impossible to build or install. Environmental
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1389 dependencies have a complexity cost, and are thus something to be
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1390 minimized.)</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1391
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1392 <p>A good build system will scan its environment to figure out what it has
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1393 available, and disable functionality that depends on anything that isn't.
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1394 (This is generally done with autoconf, which is disgusting but
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1395 suffers from a lack of alternatives.) That way, the complexity cost is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1396 optional: you can build a minimal version of the package if that's all you
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1397 need.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1398
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1399 <p>A really good build system can be told that the environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1400 it's building in and the environment the result will run in are different,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1401 so just because it finds zlib on the build system doesn't mean that the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1402 target system will have zlib installed on it. (And even if it does, it may not
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1403 be the same version. This is one of the big things that makes cross-compiling
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1404 such a pain. One big reason for statically linking programs is to eliminate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1405 this kind of environmental dependency.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1406
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1407 <p>The Firmware Linux build process is structured the way it is to eliminate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1408 as many environmental dependencies as possible. Some are unavoidable (such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1409 C libraries needing kernel headers or gcc needing binutils), but the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1410 intermediate system is the minimal fully functional Linux development
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1411 environment we currently know how to build, and then we switch into that and
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1412 work our way back up from there by building more packages in the new
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1413 environment.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1414
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1415 <h3>Resolving environmental dependencies.</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1416
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1417 <p><b>To build uClibc you need kernel headers</b> identifying the syscalls and
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1418 such it can make to the OS. We get them from the Linux kernel source tarball,
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1419 using the "make headers_install" infrastructure created by David Woodhouse.
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1420 This runs various scripts against the Linux kernel source code to sanitize
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1421 the kernel's own headers for use by userspace. (This was merged in 2.6.18-rc1,
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1422 and was more or less debugged by 2.6.19.)</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1423
702 a8854df93e15 Fix typo that left a tag unclosed. Rob Landley <rob@landley.net> parents: 655 diff changeset	1424 <p><b>We install bash</b> because the busybox shell situation is a mess.
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1425 Busybox has several different shell implementations which share little or no
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1426 code. (It's better now than it was a few years ago, but thanks to Ubuntu
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1427 breaking the #!/bin/sh symlink with the Defective Annoying SHell, many
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1428 scripts point explicitly at #!/bin/bash and BusyBox can't use that name for
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1429 any of its shells yet.)</p>
526 ad14395e1f0c Update, fix a couple dead links. The last 1/3 of the documentation still needs a major rewrite... Rob Landley <rob@landley.net> parents: 520 diff changeset	1430
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1431 <p><b>Most packages expect gcc</b>. The gnu compiler "toolchain" actually
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1432 consists of three packages <b>(binutils, gcc, and make)</b>. (The split
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1433 between binutils and gcc is for purely historical reasons, and you have
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1434 to match the right versions with each other or things break.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1435
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1436 <p>Adding an SUSv3
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1437 <a href=http://www.opengroup.org/onlinepubs/009695399/utilities/make.html>make</a>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1438 implementation to busybox or toybox isn't a major problem, but until a viable
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1439 GCC replacement emerges there's not much point.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1440
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1441 <p>None of the other compilers under development are a drop-in replacement for
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1442 gcc yet, especially for building the Linux kernel (which makes extensive use of
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1443 gcc extensions). <a href=http://www.intel.com/cd/software/products/asmo-na/eng/277618.htm>Intel's C Compiler</a>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1444 implemented the necessary gcc extensions to build the Linux kernel, but it's
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1445 a closed source package only supporting x86 and x86-64 targets. Since
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1446 the introduction of C99, the Linux kernel has replaced many of these gcc
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1447 extensions with equivalent C99 idioms, so in theory building the Linux kernel
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1448 with other compilers is now easier.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1449
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1450 <p>With the introduction of GPLv3, the Free Software Foundation has pissed off
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1451 enough people that work on an open source replacement for gcc is ongoing on
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1452 several fronts. The most promising is probably
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1453 <a href=http://pcc.ludd.ltu.se/>PCC</a>, which is
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1454 supported by what's left of the BSD community. Apple sponsors another
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1455 significant effort, <a href=http://clang.llvm.org/>LLVM/Clang</a>. Both are
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1456 worth watching.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1457
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1458 <p>Several others (Such as TinyCC and Open Watcom) once showed promise but have
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1459 been essentially moribund since about 2005, which is when compilers that only
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1460 ran on 32 bit hosts and supported C89 stopped being interesting. (A
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1461 significant amount of effort is required to retool an existing compiler to
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1462 cleanly run on an x86-64 host and support the full C99 feature set, let alone
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1463 produce output for the dozens of hardware platforms supported by Linux, or
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1464 produce similarly optimized binaries.)</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1465
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1466 <h2>Additional complications</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1467
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1468 <h3>Cross-compiling and avoiding root access</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1469
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1470 <p>Any time you create target binaries that won't run on the host system, you're
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1471 cross compiling. Even when both the host and target are on the same processor,
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1472 if they're sufficiently different that one can't run the other's binaries, then
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1473 you're cross-compiling. In our case, the host is usually running both a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1474 different C library and an older kernel version than the target, even when
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1475 it's the same processor.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1476
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1477 <p>We want to avoid requiring root access to build Firmware Linux. If the
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1478 build can run as a normal user, it's a lot more portable and a lot less likely
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1479 to muck up the host system if something goes wrong. This means we can't modify
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1480 the host's / directory (making anything that requires absolute paths
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1481 problematic). We also can't mknod, chown, chgrp, mount (for --bind, loopback,
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1482 tmpfs)...</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1483
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1484 <p>In addition, the gnu toolchain (gcc/binutils) is chock-full of hardwired
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1485 assumptions, such as what C library it's linking binaries against, where to look
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1486 for #included headers, where to look for libraries, the absolute path the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1487 compiler is installed at... Silliest of all, it assumes that if the host and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1488 target use the same processor, you're not cross-compiling (even if they have
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1489 a different C library and a different kernel, and even if you ./configure it
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1490 for cross-compiling it switches that back off because it knows better than
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1491 you do). This makes it very brittle, and it also tends to leak its assumptions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1492 into the programs it builds. New versions may someday fix this, but for now we
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1493 have to hit it on the head repeatedly with a metal bar to get anything remotely
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1494 useful out of it, and run it in a separate filesystem (chroot environment) so
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1495 it can't reach out and grab the wrong headers or wrong libraries despite
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1496 everything we've told it.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1497
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1498 <p>The absolute paths problem affects target binaries because all dynamically
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1499 linked apps expect their shared library loader to live at an absolute path
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1500 (in this case /lib/ld-uClibc.so.0). This directory is only writeable by root,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1501 and even if we could install it there polluting the host like that is just
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1502 ugly.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1503
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1504 <p>The Firmware Linux build has to assume it's cross-compiling because the host
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1505 is generally running glibc, and the target is running uClibc, so the libraries
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1506 the target binaries need aren't installed on the host. Even if they're
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1507 statically linked (which also mitigates the absolute paths problem somewhat),
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1508 the target often has a newer kernel than the host, so the set of syscalls
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1509 uClibc makes (thinking it's talking to the new kernel, since that's what the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1510 ABI the kernel headers it was built against describe) may not be entirely
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1511 understood by the old kernel, leading to segfaults. (One of the reasons glibc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1512 is larger than uClibc is it checks the kernel to see if it supports things
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1513 like long filenames or 32-bit device nodes before trying to use them. uClibc
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1514 should always work on a newer kernel than the one it was built to expect, but
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1515 not necessarily an older one.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1516
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1517 <h2>Ways to make it all work</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1518
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1519 <h3>Cross compiling vs native compiling under emulation</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1520
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1521 <p>Cross compiling is a pain. There are a lot of ways to get it to sort of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1522 kinda work for certain versions of certain packages built on certain versions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1523 of certain distributions. But making it reliable or generally applicable is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1524 hard to do.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1525
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1526 <p>I wrote an <a href=/writing/docs/cross-compiling.html>introduction
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1527 to cross-compiling</a> which explains the terminology, plusses and minuses,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1528 and why you might want to do it. Keep in mind that I wrote that for a company
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1529 that specializes in cross-compiling. Personally, I consider cross-compiling
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1530 a necessary evil to be minimized, and that's how Firmware Linux is designed.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1531 We cross-compile just enough stuff to get a working native build environment
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1532 for the new platform, which we then run under emulation.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1533
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1534 <h3>Which emulator?</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1535
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1536 <p>The emulator Firmware Linux 0.8x used was User Mode Linux (here's a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1537 <a href=http://www.landley.net/writing/docs/UML.html>UML mini-howto</a> I wrote
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1538 while getting this to work). Since we already need the linux-kernel source
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1539 tarball anyway, building User Mode Linux from it was convenient and minimized
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1540 the number of packages we needed to build the minimal system.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1541
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1542 <p>The first stage of the build compiled a UML kernel and ran the rest of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1543 build under that, using UML's hostfs to mount the parent's root filesystem as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1544 the root filesystem for the new UML kernel. This solved both the kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1545 version and the root access problems. The UML kernel was the new version, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1546 supported all the new syscalls and ioctls and such that the uClibc was built to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1547 expect, translating them to calls to the host system's C library as necessary.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1548 Processes running under User Mode Linux had root access (at least as far as UML
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1549 was concerned), and although they couldn't write to the hostfs mounted root
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1550 partition, they could create an ext2 image file, loopback mount it, --bind
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1551 mount in directories from the hostfs partition to get the apps they needed,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1552 and chroot into it. Which is what the build did.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1553
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1554 <p>Current Firmware Linux has switched to a different emulator, QEMU, because
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1555 as long as we're we're cross-compiling anyway we might as well have the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1556 ability to cross-compile for non-x86 targets. We still build a new kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1557 to run the uClibc binaries with the new kernel ABI, we just build a bootable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1558 kernel and run it under QEMU.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1559
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1560 <p>The main difference with QEMU is a sharper dividing line between the host
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1561 system and the emulated target. Under UML we could switch to the emulated
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1562 system early and still run host binaries (via the hostfs mount). This meant
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1563 we could be much more relaxed about cross compiling, because we had one
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1564 environment that ran both types of binaries. But this doesn't work if we're
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1565 building an ARM, PPC, or x86-64 system on an x86 host.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1566
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1567 <p>Instead, we need to sequence more carefully. We build a cross-compiler,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1568 use that to cross-compile a minimal intermediate system from the seven packages
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1569 listed earlier, and build a kernel and QEMU. Then we run the kernel under QEMU
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1570 with the new intermediate system, and have it build the rest natively.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1571
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1572 <p>It's possible to use other emulators instead of QEMU, and I have a todo
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1573 item to look at armulator from uClinux. (I looked at another nommu system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1574 simulator at Ottawa Linux Symposium, but after resolving the third unnecessary
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1575 environmental dependency and still not being able to get it to finish compiling
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1576 yet, I gave up. Armulator may be a patch against an obsolete version of gdb,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1577 but I could at least get it to build.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1578
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	1579 <h2>Packaging</h2>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1580
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1581 <h2>Filesystem Layout</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1582
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1583 <p>Firmware Linux's directory hierarchy is a bit idiosyncratic: some redundant
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1584 directories have been merged, with symlinks from the standard positions
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1585 pointing to their new positions. On the bright side, this makes it easy to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1586 make the root partition read-only.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1587
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1588 <h3>Simplifying the $PATH.</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1589
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1590 <p>The set "bin->usr/bin, sbin->usr/sbin, lib->usr/lib" all serve to consolidate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1591 all the executables under /usr. This has a bunch of nice effects: making a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1592 a read-only run-from-CD filesystem easier to do, allowing du /usr to show
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1593 the whole system size, allowing everything outside of there to be mounted
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1594 noexec, and of course having just one place to look for everything. (Normal
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1595 executables are in /usr/bin. Root only executables are in /usr/sbin.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1596 Libraries are in /usr/lib.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1597
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1598 <p>For those of you wondering why /bin and /usr/sbin were split in the first
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1599 place, the answer is it's because Ken Thompson and Dennis Ritchie ran out
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1600 of space on the original 2.5 megabyte RK-05 disk pack their root partition
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1601 lived on in 1971, and leaked the OS into their second RK-05 disk pack where
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1602 the user home directories lived. When they got more disk space, they created
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1603 a new direct (/home) and moved all the user home directories there.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1604
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1605 <p>The real reason we kept it is tradition. The execuse is that the root
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1606 partition contains early boot stuff and /usr may get mounted later, but these
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1607 days we use initial ramdisks (initrd and initramfs) to handle that sort of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1608 thing. The version skew issues of actually trying to mix and match different
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1609 versions of /lib/libc.so.* living on a local hard drive with a /usr/bin/*
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1610 from the network mount are not pretty.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1611
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1612 <p>I.E. The seperation is just a historical relic, and I've consolidated it in
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1613 the name of simplicity.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1614
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1615 <p>On a related note, there's no reason for "/opt". After the original Unix
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1616 leaked into /usr, Unix shipped out into the world in semi-standardized forms
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1617 (Version 7, System III, the Berkeley Software Distribution...) and sites that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1618 installed these wanted places to add their own packages to the system without
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1619 mixing their additions in with the base system. So they created "/usr/local"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1620 and created a third instance of bin/sbin/lib and so on under there. Then
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1621 Linux distributors wanted a place to install optional packages, and they had
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1622 /bin, /usr/bin, and /usr/local/bin to choose from, but the problem with each
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1623 of those is that they were already in use and thus might be cluttered by who
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1624 knows what. So a new directory was created, /opt, for "optional" packages
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1625 like firefox or open office.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1626
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1627 <p>It's only a matter of time before somebody suggests /opt/local, and I'm
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1628 not humoring this. Executables for everybody go in /usr/bin, ones usable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1629 only by root go in /usr/sbin. There's no /usr/local or /opt. /bin and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1630 /sbin are symlinks to the corresponding /usr directories, but there's no
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1631 reason to put them in the $PATH.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1632
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1633 <h3>Consolidating writeable directories.</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1634
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1635 <p>All the editable stuff has been moved under "var", starting with symlinking
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1636 tmp->var/tmp. Although /tmp is much less useful these days than it used to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1637 be, some things (like X) still love to stick things like named pipes in there.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1638 Long ago in the days of little hard drive space and even less ram, people made
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1639 extensive use of temporary files and they threw them in /tmp because ~home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1640 had an ironclad quota. These days, putting anything in /tmp with a predictable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1641 filename is a security issue (symlink attacks, you can be made to overwrite
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1642 any arbitrary file you have access to). Most temporary files for things
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1643 like the printer or email migrated to /var/spool (where there are
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1644 persistent subdirectories with known ownership and permissions) or in the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1645 user's home directory under something like "~/.kde".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1646
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1647 <p>The theoretical difference between /tmp and /var/tmp is that the contents
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1648 of /tmp should be deleted by the system init scripts on every
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1649 reboot, but the contents of /var/tmp may be preserved across reboots. Except
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1650 there's no guarantee that the contents of any temp directory won't be deleted.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1651 So any program that actually depends on the contents of /var/tmp being
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1652 preserved across a reboot is obviously broken, and there's no reason not to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1653 just symlink them together.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1654
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1655 <p>(I case it hasn't become apparent yet, there's 30 years of accumulated cruft
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1656 in the standards, convering a lot of cases that don't apply outside of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1657 supercomputing centers where 500 people share accounts on a mainframe that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1658 has a dedicated support staff. They serve no purpose on a laptop, let alone
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1659 an embedded system.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1660
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1661 <p>The corner case is /etc, which can be writeable (we symlink it to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1662 var/etc) or a read-only part of the / partition. It's really a question of
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1663 whether you want to update configuration information and user accounts in a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1664 running system, or whether that stuff should be fixed before deploying.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1665 We're doing some cleanup, but leaving /etc writeable (as a symlink to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1666 /var/etc). Firmware Linux symlinks /etc/mtab->/proc/mounts, which
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1667 is required by modern stuff like shared subtrees. If you want a read-only
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1668 /etc, use "find /etc -type f \| xargs ls -lt" to see what gets updated on the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1669 live system. Some specific cases are that /etc/adjtime was moved to /var
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1670 by LSB and /etc/resolv.conf should be a symlink somewhere writeable.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1671
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1672 <h3>The resulting mount points</h3>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1673
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1674 <p>The result of all this is that a running system can have / be mounted read
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1675 only (with /usr living under that), /var can be ramfs or tmpfs with a tarball
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1676 extracted to initialize it on boot, /dev can be ramfs/tmpfs managed by udev or
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1677 mdev (with /dev/pts as devpts under that: note that /dev/shm naturally inherits
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1678 /dev's tmpfs and some things like User Mode Linux get upset if /dev/shm is
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1679 mounted noexec), /proc can be procfs, /sys can bs sysfs. Optionally, /home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1680 can be be an actual writeable filesystem on a hard drive or the network.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1681
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1682 <p>Remember to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1683 put root's home directory somewhere writeable (I.E. /root should move to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1684 either /var/root or /home/root, change the passwd entry to do this), and life
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1685 is good.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1686
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1687
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1688 <p>Firmware Linux is an embedded Linux distribution builder, which creates a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1689 bootable single file Linux system based on uClibc and BusyBox/toybox. It's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1690 basically a shell script that builds a complete Linux system from source code
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1691 for an arbitrary target hardware platform.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1692
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1693 <p>The FWL script starts by building a cross-compiler for the appropriate
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1694 target. Then it cross-compiles a small Linux system for the target, which
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1695 is capable of acting as a native development environment when run on the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1696 appropriate hardware (or under an emulator such as QEMU). Finally the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1697 build script creates an ext2 root filesystem image, and packages it with
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1698 a kernel configured to boot under QEMU and shell scripts to invoke qemu
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1699 appropriately.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1700
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1701
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1702 <p>The FWL boot script for qemu (/tools/bin/qemu-setup.sh) populates /dev
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1703 from sysfs, sets up an emulated (masquerading) network (so you can wget
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1704 source packages or talk to <a href="#distcc_trick">distcc</a>), and creates
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1705 a few symlinks needed to test build normal software packages (such as making
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1706 /lib point to /tools/lib). It also mounts /dev/hdb (or /dev/sdb) on /home
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1707 if a second emulated drive is present.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1708
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1709 <p>For most platforms, exiting the command shell will exit the emulator.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1710 (Some, such as powerpc, don't support this yet. For those you have to kill
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1711 qemu from another window, or exit the xterm. I'm working on it.)</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1712
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1713 <p>To use this emulated system as a native build environment, see
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1714 <a href="#native_compiling">native compiling</a>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1715
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1716
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1717 <a name="new_platform"><h1>Adding a new target platform</h1></a>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1718
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1719 <p>The differences between platforms are confined to a single directory,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1720 sources/targets. Each subdirectory under that contains all the configuration
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1721 information for a specific target platform FWL can produce system images
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1722 for. The same scripts build the same packages for each platform, differing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1723 only in which configuration directory they pull data from.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1724
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1725 <p>Each target configuration directory has three interesting files:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1726
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1727 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1728 <li><p><b>details</b> - sets a bunch of environment variables</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1729 <li><p><b>miniconfig-uClibc</b> - configuration for uClibc.</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1730 <li><p><b>miniconfig-linux</b> - configuration for the Linux kernel</li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1731 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1732
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1733 <p>These configuration files are read and processed by the script
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1734 <b>include.sh</b>.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1735
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1736 <h2>Target name.</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1737
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1738 <p>The name of the target directory is saved in the variable "$ARCH", and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1739 used to form a "tuple" for gcc and binutils by appending "-unknown-linux" to
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1740 the directory name. So the first thing to do is find out what platform name
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1741 gcc and binutils want for your target platform, and name your target directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1742 appropriately.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1743
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1744 <p>(Note: if your platform really can't use an "${ARCH}-unknown-linux" style
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1745 tuple, and instead needs a tuple like "bfin-elf", you can set the variable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1746 CROSS_TARGET in the "details" file to override the default value and feed
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1747 some other --target to gcc and binutils. You really shouldn't have to do
655 62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	1748 this unless gcc doesn't yet fully support Linux on your platform, or unless
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	1749 you're doing multiple variants of the same target such as powerpc and ppc440.
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	1750 Try the default first, and fix it if necessary.)</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1751
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1752 <p>The name of the target directory is also used in the name of the various
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1753 directories generated during the build (temp-$ARCH, cross-compiler-$ARCH,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1754 and mini-native-$ARCH, all in the build/ directory), and as the prefix of the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1755 cross compiler binaries ($ARCH-gcc and friends).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1756
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1757 <h2>$ARCH/details</h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1758
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1759 <p>The following environment variables may be set in the "<b>details</b>"
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1760 file:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1761
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1762 <ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1763 <li><p><b>CROSS_TARGET</b> - By default this is set to "${ARCH}-unknown-linux".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1764
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1765 <p>This is used by binutils and gcc. If your target really can't use that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1766 tuple name (perhaps needing a tuple like "bfin-elf" instead), you can set the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1767 variable CROSS_TARGET in the "details" file to override the default value and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1768 feed some other --target to gcc and binutils.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1769
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1770 <p>You usually shouldn't have to set this yourself unless gcc doesn't yet fully
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1771 support Linux on your platform. Try the default first, and fix it if
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1772 necessary.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1773 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1774
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1775 <li><p><b>KARCH</b> - architecture value for the Linux kernel (ARCH=$KARCH).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1776
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1777 <p>The Linux kernel uses different names for architectures than gcc or binutils
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1778 do. To see all your options, list the "arch" directory of the linux kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1779 source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1780 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1781
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1782 <li><p><b>KERNEL_PATH</b> - Path in the linux kernel source tree where the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1783 bootable kernel image is generated.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1784
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1785 <p>This is the file saved out of the kernel build, to be fed to qemu's -kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1786 option. Usually "arch/${KARCH}/boot/zImage", but
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1787 sometimes bzImage or image in that directory, sometimes vmlinux in the top
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1788 level directory...</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1789 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1790
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1791 <li><p><b>GCC_FLAGS</b> - Any extra flags needed by gcc.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1792
655 62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	1793 <p>Usually blank, but sometimes used to specify a floating point coprocessor,
62a727f52a18 Update docs to mention download RENAME option. Rob Landley <rob@landley.net> parents: 635 diff changeset	1794 ABI, or --with-cpu.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1795 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1796
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1797 <li><p><b>BINUTILS_FLAGS</b> - Any extra flags needed by binutils.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1798
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1799 <p>Usually blank.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1800 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1801
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1802 <li><p><b>QEMU_TEST</b> - Optional emulator for sanity test.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1803
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1804 <p>At the end of the cross compiler build, a quick sanity
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1805 test builds static and dynamic "Hello world!" executables with the new cross
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1806 compiler. If QEMU_TEST isn't blank and a file qemu-$QEMU_TEST
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1807 exists in the $PATH, the cross compiler build script will then run qemu's
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1808 application emulation against the static version of "hello world" as an
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1809 additional sanity test, to make sure it runs on the target processor and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1810 outputs "Hello world!".</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1811
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1812 <p>Leave it blank to skip this test.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1813 </li>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1814
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1815 <li><p><b>emulator_command</b> - Shell function run to generate the actual
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1816 emulator invocation at the end of the run-$ARCH.sh shell script in the system
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1817 image tarball.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1818
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1819 <p>This is actually a shell function, not an environment variable. It's
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1820 called from package-mini-native.sh to output an emulator command line to
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1821 stdout (generally using "echo").<p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1822
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1823 <p>The function receives two arguments: $1 is the name of the ext2 image
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1824 containing the root filesystem, and $2 is the name of the kernel image.
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1825 The function can also call another shell function, <b>qemu_defaults</b>,
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1826 which is defined in package-mini-native.sh and which provides most of
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1827 the qemu command line. (If you use a different emulator, you don't have to
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1828 call this function, but if you use qemu it makes things a lot easier and
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1829 more consistent.) The qemu_command function outputs $ROOT and $CONSOLE
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1830 variables for its root= and console= kernel command line arguments, so
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1831 set those before calling it.</p>
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1832 </li>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1833
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1834 </ul>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1835
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1836 <a name="miniconfig"><h2>Miniconfig files</h2></a>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1837
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1838 <p>The expanded .config files used to build both Linux and uClibc are copied
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1839 into the /usr/src directory of mini-native filesystems during the build,
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1840 and kept for future reference.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1841
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1842 <p>The Linux kernel and uClibc each need a configuration file to build.
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1843 Firmware Linux uses the "miniconfig" file format, which contains only the configuration
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1844 symbols a user would have to switch on in menuconfig if they started from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1845 allnoconfig.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1846
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1847 <p>To generate a miniconfig, first configure your kernel with menuconfig,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1848 then copy the resulting .config file to a temporary filename (such as
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1849 "tempfile"). Then run the miniconfig.sh script in the sources/toys directory
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1850 with the temporary file name as your argument and with the environment variable
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1851 ARCH set to the $KARCH value in your new config file (and exported if
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1852 necessary). This should produce a new file, "mini.config", which is your
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1853 .config file converted to miniconfig format.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1854
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1855 <p>For example, to produce a miniconfig for a given platform:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1856 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1857 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1858 make ARCH=$KARCH menuconfig
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1859 mv .config tempfile
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1860 ARCH=$KARCH miniconfig.sh tempfile
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1861 ls -l mini.config
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1862 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1863 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1864
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1865 <p>To expand a mini.config back into a full .config file (to build a kernel
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1866 by hand, or for further editing with menuconfig), you can go:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1867
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1868 <blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1869 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1870 make ARCH=$KARCH allnoconfig KCONFIG_ALLCONFIG=mini.config
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1871 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1872 </blockquote>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1873
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1874 <p>Remember to supply an actual value for $KARCH.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1875
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1876 <h2>$ARCH/miniconfig-linux</h2>
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1877 <p>This is the miniconfig file to build a Linux kernel for the appropriate
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1878 target. This is usually aimed at booting under QEMU, but if you'd like
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1879 to come up with your own configuration for actual target hardware, feel
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1880 free.</p>
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1881
635 c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1882 <p>The starting point for kernel configs is generally one of the defconfig
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1883 files from the Linux kernel source code, usually at
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1884 "arch/$ARCH/configs/*_defconfig". Copy that to .config at the top of the
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1885 kernel source, run menuconfig to edit it, then shrink it into a miniconfig.</p>
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1886
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1887 <p>Kernels to run system images under qemu generally require the following
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1888 hardware: serial port (for /dev/console), hard drive (for hda and hdb images),
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1889 network card (for distcc), and a persistent realtime clock (make gets unhappy
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1890 if source files are newer than the current time). The ability to address
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1891 at least 512 megs of memory is also nice, although some targets (such as mips)
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1892 are limited to less than that by the hardware. The "qemu-system-$ARCH -M ?"
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1893 and "qemu-system-$ARCH -cpu ?" options may be informative here, also
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1894 the <a href=http://www.nongnu.org/qemu/qemu-doc.html#SEC60>QEMU System
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1895 emulator for non PC targets</a> documentation.</p>
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1896
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1897 <h2>$ARCH/miniconfig-uClibc</h2>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1898
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1899 <p>Just like the Linux kernel, uClibc needs a .config file to build, and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1900 so the Firmware Linux configuration file supplies a miniconfig. Note that
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1901 uClibc doesn't require an ARCH= value, because all its architecture information
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1902 is stored in the config file. Otherwise the procedure for creating and using
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1903 it is the same as for the Linux kernel, just with a different filename and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1904 contents.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1905
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1906 <p>Most of each miniconfig-uClibc is identical from platform to platform.
635 c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1907 Usually only the "Target Architecture" changes (and occasionally an entry
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1908 or two out of Target Architecture Features and Options). At some point
c796d3ebf179 Fluff out docs a bit (miniconfig: what hardware does qemu need?) Rob Landley <rob@landley.net> parents: 584 diff changeset	1909 in the future the rest of the uClibc configuration might be factored out into
520 16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1910 a common file, but so far removing the duplication hasn't been worth the
16adbffb58b0 More documentation tweaks, adding a new target section. Rob Landley <rob@landley.net> parents: 518 diff changeset	1911 extra complexity.</p>
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1912
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1913 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1914
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1915 <!--
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1916
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1917 <p>This root filesystem acts as a minimal native build environment for the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1918 target platform. This means it contains a compiler and associated build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1919 tools capable of building a complete new Linux system under itself. If you're
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1920 interested in building a more complex development environment within this one,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1921 see the <a href=http://www.linuxfromscratch.org/lfs/view/6.4/>Linux From
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1922 Scratch</a> project for ideas on how to bootstrap your way up (adding
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1923 zlip and perl and such before building more complicated packages).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1924
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1925 <p>Note that FWL can build a LFS "temporary system", but that the packaged
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1926 mini-native tarballs and system images are not configured that way.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1927
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1928 <p>The vast majority of the space taken up by this filesystem is the
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1929 development toolchain and associated support files (mostly header files and
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1930 libraries).</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1931
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1932 <p>If you're doing anything fancy, you'll probably want to rebuild it from
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1933 source.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1934
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1935 GCC is bad at cross compiling.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1936
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1937 <p>A cross compiler reads input files and writes output files. So does a
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1938 docbook to PDF converter; this is nothing special. A program can take both
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1939 explicit and implicit input. Explicit input is listed on the command line,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1940 or perhaps piped into stdin. Implicit input in the case of the docbook->pdf
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1941 converter would include fonts and stylesheets, which might live at some
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1942 common path on the host, or be handed out by a server.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1943
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1944 <p>Compilers take implicit input from five places:</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1945
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1946 compiler #includes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1947 system library #includes
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1948 compiler libs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1949 system libs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1950
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1951
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1952 <p>In theory, someday busybox may provide a decent /bin/bash replacement,
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1953 but unfortunately busybox shell development is terminally fragmented (between
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1954 lash, hush, msh, and ash, which do not share significant amounts of code),
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1955 so don't hold your breath.</p>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1956
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1957 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1958 variables for invoking download
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1959 mirrors
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1960 --extract, setupfor, cacheing
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1961 why hard links instead of symlinks
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1962 Packages that modify distributed source files without breaking links: bad.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1963
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1964 CPUS, make -j (As much parallelism as possible; SMP increasing.)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1965
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1966 Minimal native build environment, seven packages.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1967 Why bash?
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1968 Busybox ash buggy. Busybox has four shells, not one scalable shell.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1969 Older version of bash, smaller and simpler.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1970
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1971 sources/native
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1972 src
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1973 qemu-setup.sh/chroot-setup.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1974 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1975
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1976 <hr>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1977
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1978 <h2><a name="native_compiling">Native compiling under emulation</a></h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1979
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1980 <pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1981 Why do this?
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1982 the distcc trick
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1983 run-with-distcc.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1984 hdb for working space.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1985 run-with-home.sh
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1986 Building on nfs sucks rocks.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1987 Building out of tree with cp -rs
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1988
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1989 /tools (Linux From Scratch chapter 5).
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1990 qemu-setup.sh and the /lib symlink.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1991
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1992 Deficiencies in the current mini-native filesystem:
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1993 host-tools.sh, bzip2, coreutils, diffutils.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1994
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1995 Building glibc is your problem.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1996 It requires perl. Statically linking "hello world" is 400k. It's evil.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1997 Still, building it natively sucks less than trying to cross compile it.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1998 Pretty much follow the non-cross Linux From Scratch procedures.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	1999
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2000 Building a distro:
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2001 Linux From Scratch.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2002 Gentoo embedded.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2003 Debian/ubuntu.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2004
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2005 anatomy_of_include
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2006 </pre>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2007
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2008 <h2><a name="customize">Customizing mini-native</a></h2>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2009
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2010 Adding packages (to the script, without the script)
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2011
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2012 build/temp-$ARCH
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2013 build/cross-compiler-$ARCH
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2014 build/mini-native-$ARCH
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2015 build/qemu-image-*.tar
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2016
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2017
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2018 <a name="gcc_sucks><h2>What the hell is wrong with GCC?</h2></a>
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2019
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2020 <p>First of all, gcc wants to build itself with itself. When you build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2021 gcc it wants to compile a temporary version of itself, and then build
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2022 itself again with that temporary compiler, and then build itself a _third_
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2023 time with the second compiler.
f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2024
584 26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2025 <h1>Packaging</h1>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2026
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2027 <p>The single file packaging combines a linux kernel, initramfs, squashfs
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2028 partition, and cryptographic signature.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2029
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2030 <p>In Linux 2.6, the kernel and initramfs are already combined into a single
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2031 file. At the start of this file is either the obsolete floppy boot sector
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2032 (just a stub in 2.6), or an ELF header which has 12 used bytes followed by 8
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2033 unused bytes. Either way, we can generally use the 4 bytes starting at offset
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2034 12 to store the original length of the kernel image, then append a squashfs
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2035 root partition to the file, followed by a whole-file cryptographic
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2036 signature.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2037
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2038 <p>Loading an ELF kernel (such as User Mode Linux or a non-x86 ELF kernel)
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2039 is controlled by the ELF segments, so the appended data is ignored.
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2040 (Note: don't strip the file or the appended data will be lost.) Loading an x86
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2041 bzImage kernel requires a modified boot loader that can be told the original
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2042 size of the kernel, rather than querying the current file length (which would
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2043 be too long). Hence the patch to Lilo allowing a "length=xxx" argument in the
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2044 config file.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2045
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2046 <p>Upon boot, the kernel runs the initramfs code which finds the firmware
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2047 file. In the case of User Mode Linux, the symlink /proc/self/exe points
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2048 to the path of the file. A bootable kernel needs a command line argument
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2049 of the form firmware=device:/path/to/file (it can lookup the device in
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2050 /sys/block and create a temporary device node to mount it with; this is
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2051 in expectation of dynamic major/minor happening sooner or later).
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2052 Once the file is found, /dev/loop0 is bound to it with an offset (losetup -o,
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2053 with a value extracted from the 4 bytes stored at offset 12 in the file), and
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2054 the resulting squashfs is used as the new root partition.</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2055
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2056 <p>The cryptographic signature can be verified on boot, but more importantly
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2057 it can be verified when upgrading the firmware. New firmware images can
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2058 be installed beside old firmware, and LILO can be updated with boot options
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2059 for both firmware, with a default pointing to the _old_ firmware. The
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2060 lilo -R option sets the command line for the next boot only, and that can
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2061 be used to boot into the new firmware. The new firmware can run whatever
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2062 self-diagnostic is desired before permanently changing the default. If the
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2063 new firmware doesn't boot (or fails its diagnostic), power cycle the machine
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2064 and the old firmware comes up. (Note that grub does not have an equivalent
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2065 for LILO's -R option; which would mean that if the new firmware doesn't run,
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2066 you have a brick.)</p>
26a5ac0c01ce Random update to docs I did a while ago, need to do more. Rob Landley <rob@landley.net> parents: 531 diff changeset	2067
518 f109c324cca4 Massive documentation rewrite. Not quite finished, but better. Rob Landley <rob@landley.net> parents: diff changeset	2068 -->
881 0da87d1ef528 Update the web page. Put the left-side nav bar back, move about page to "about.html" and make news.html the default index. Add architectures page. Note that IRC discussion is now on #edev. Rob Landley <rob@landley.net> parents: 830 diff changeset	2069
0da87d1ef528 Update the web page. Put the left-side nav bar back, move about page to "about.html" and make news.html the default index. Add architectures page. Note that IRC discussion is now on #edev. Rob Landley <rob@landley.net> parents: 830 diff changeset	2070 <!--#include file="footer.html" -->

Mercurial > hg > aboriginal

annotate www/documentation.html @ 1229:313c702a0984