changeset 5:2907d5193cf6

A real web page, and description of the build process.
author Rob Landley <>
date Mon, 27 Nov 2006 19:36:31 -0500
parents 9f52b6d1c4ab
children e039588b3189
files www/build-process.html www/index.html
diffstat 2 files changed, 287 insertions(+), 2 deletions(-) [+]
line wrap: on
line diff
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/www/build-process.html	Mon Nov 27 19:36:31 2006 -0500
@@ -0,0 +1,68 @@
+<title>The Firmware Linux build process</title>
+<p>FWL builds a cross-compiler and then uses it to build a minimal system with
+a native compiler, BusyBox and uClibc.  Then it runs this minimal system
+under an emulator (QEMU) and natively builds the final system.  It then
+packages the resulting system (kernel, initramfs, and root filesystem) into
+a single file that can boot and run (using a modified version of LILO on
+<p>Firmware Linux builds in stages:</p>
+<h2>Stage 1: Build a cross-compiler.</h2>
+<p>The first stage builds a cross-compiler,
+which runs on the host system and produces binaries that run on the target
+system.  (See my <a href=/writing/docs/cross-compiling.html>Introduction to
+cross compiling</a> if you're unfamiliar with this.)</p>
+<p>We have to cross-compile even if the host and target system are both
+x86, because the host probably use different C libraries.  If the host has
+glibc and the target uses uClibc, then the (dynamically linked) target binaries
+we produce won't run on the host.  This is what distinguishes cross-compiling
+from native compiling: different processors are just one reason the binaries
+might not run.  Of course, as long as we've got the cross-compiling support
+anyway, we might as well support building for x86_64, arm, mips, or ppc
+<p>Building a cross-compiler toolchain requires four packages.  The bulk of
+it is binutils, gcc, and uClibc, but building those requires header files from
+the Linux kernel which describe the target system.</p>
+<h2>Stage 2: Use the cross-compiler to build a native build environment
+for the target.</h2>
+<p>Because cross-compiling is persnickety and difficult, we do as little of
+it as possible.  We use the cross-compiler to generate a native build
+environment for the target, and then run the rest of the build under an
+<p>The minimal build environment you can boot into and build a complete Linux
+system under is the Linux kernel, binutils, gcc, uClibc, BusyBox, make, and
+bash.  The emulator we use to run this is QEMU, so we build that too.</p>
+<h2>Stage 3: Run the target's native build environment under an emulator to
+build the final system.</h2>
+<p>Running a native build under QEMU is about 1/3 the speed of cross-compiling,
+but it's a lot easier and more reliable.</p>
+<p>A trick to accelerate the build is to use distcc to call out to the
+cross-compiler, feeding the results back into the emulator through the virtual
+network.  This is still a TODO item.</p>
+<p>Stage 3 is a fairly straightforward
+<a href=>Linux From Scratch</a> approach,
+except that we use BusyBox and uClibc in place of a couple dozen other
+<h2>Stage 4: Package the system into a firmware file.</h2>
+<p>The reason for the name Firmware Linux is that the entire operating system
+(kernel, initramfs, and read-only squashfs root filesystem) are glued together
+into a single file.  A modified version of LILO is included which can boot and
+run this file on x86.</p>
--- a/www/index.html	Mon Nov 27 19:34:36 2006 -0500
+++ b/www/index.html	Mon Nov 27 19:36:31 2006 -0500
@@ -1,6 +1,223 @@
+<title>Firmware Linux</title>
-<p>Nothing to see yet, try <a href="">the old site</a> and email me if you're actually interested.</p>
+<h1>Firmware Linux</h1>
+<b><h2>What is it?</h2></b>
+<p>Firmware Linux is an embedded Linux distribution builder.  It's basically
+a shell script that builds a complete Linux system from source code.</p>
+<p>FWL builds a cross-compiler and then uses it to build a minimal system
+containing a native compiler, BusyBox and uClibc.  Then it runs this minimal
+system under an emulator (QEMU) and natively builds the final system.  Finally
+it packages the resulting system (kernel, initramfs, and root filesystem) into
+one big bootable file.</p>
+<p>Here is a description of <a href=design.html>the design of Firmware
+Linux</a>.  That's the new (QEMU-based, capable of cross-compiling for non-x86)
+design I'm working on now.  The old (UML-based, x86 only) design is described
+<p>The current stuff is available from <a href=/hg/firmware>the mercurial
+repository.  That's the new (QEMU-based, capable of cross-compiling for
+different hardware platforms) design I'm working on now, and where new
+development happens.</p>
+<p>The old (UML-based, x86 only) design is still available from <a href=old>the
+old website</a>, which is hideously out of date but contains a working
+(ancient) version.</p>
+<p>Here's a quick <a href=build-process.html>overview of the Firmware Linux
+build process</a>.</p>
+<p>Here is a description of <a href=design.html>the design of Firmware
+<p>I've been working on this project on and off since 1999, it's what 
+got me into BusyBox and uClibc and compilers and so on.  Now it's where I put
+together everything else I'm doing (like toybox and tinygcc) to see what
+actually works and give it a good stress-test.  (Eating your own dogfood,
+and all that.)</p>
+<p>The project stalled while I was BusyBox maintainer (2005-2006) due
+to lack of time, and since then most of my spare programming time has gone
+into launching toybox.  But sinice one of the main goals of toybox is to
+replace BusyBox in Firmware Linux, as toybox matures it'll naturally lead
+to spending more time working on FWL.</p>
+<p>This server does not currently run on Firmware Linux.  Making it do so
+is a TODO item.  After that, I'd like to get it to the point where I can
+use it on my laptop. :)</p>
+<p>Firmware Linux is a bootable single file linux system, based on busybox and
+uClibc.  After downloading the
+<a href=downloads/firmware-build-0.8.9.tar.bz2>source code</a>, extract
+it, read the README, and run "./".  This will (eventually)
+create a self-contained firmware-uml executable you can use to try out an
+emulated version of Firmware Linux.</p>
+<p>Prebuilt versions are available: The <a href=downloads/base-uml>basic
+build</a> is 2.5 megabytes (which includes the linux kernel, all command line
+utilities, and the minimal set of shared libraries).  The
+<a href=downloads/devel-uml>full development environment</a> is 15 megabytes
+(the base system plus enough development tools for Firmware Linux to rebuild
+itself from source code).</p>
+<p>Read <a href=/notes.html>my development log</a> to see what I've been up
+to on this project.</p>
+<p>Here's the <a href=lilo-length.patch>length patch</a> I'm using on lilo.</p>
+<p>I wrote a <a href=/writing/docs/UML.html>Quick and Dirty User Mode Linux HOWTO</a> if you've never played with UML before.</p>
+<h2>What is it?</h2>
+<h2>Firmware Linux is a bootable single file linux system.</h2>
+<p>Firmware Linux is one file containing a kernel, initramfs, read-only root
+filesystem, and cryptographic signature.  You can boot Linux from this file
+as if it was a normal kernel image (a slightly modified LILO is required,
+patches for GRUB and other bootloaders are a to-do item).  You can upgrade
+your entire OS (and any applications in the root filesystem) atomically, by
+downloading a new file and pointing your bootloader at it.</p>
+<h2>Firmware Linux is a Linux distro using busybox and uClibc as the basis for
+a self-hosting development environment.</h2>
+<p>When the Firmware Linux project started, busybox applets like sed and sort
+weren't powerful enough to handle the "./configure; make; make install" of
+packages like binutils or gcc.  Busybox was usable in an embedded router or
+rescue floppy, but trying to get real work done with it revealed numerous
+bugs and limitations.</p>
+<p>Busybox has now been fixed, and in Firmware Linux Busybox functions as an
+effective replacement for bzip2, coreutils, e2fsprogs, file, findutils, gawk,
+grep, inetutils, less, modutils, net-tools, patch, procps, sed, shadow,
+sysklogd, sysvinit, tar, util-linux, and vim.  (Eventually, it should be
+capable of replacing bash and diffutils as well, but it's not there yet.)</p>
+<p>The base system consists of uClibc-0.9.28, busybox-1.1-pre1, and
+linux-  (Currently, bash-2.05 is also included due to limitations in
+the busybox built-in shell.  This is a temporary measure until Busybox
+is further improved.)</p>
-<p>Some <a href=design.html>design notes</a>.</p>
+<p>The build toolchain uses the base system plus binutils, gcc-core, bison,
+linux-libc-headers, and make.</p>
+<p>The install software uses lilo, which needs bin86 and as86 to build it.</p>
+<p>The full development system (which creates a development environment
+sufficient for Firmware Linux to rebuild itself from source) adds
+m4, flex, bison, diffutils, zlib, bin86, and nasm.</p>
+<p>Busybox is effectively replacing all the following packages:
+bzip2, coreutils, e2fsprogs, file, findutils, gawk, grep, inetutils, less,
+modutils, net-tools, patch, procps, sed, shadow, sysklogd, sysvinit, tar,
+util-linux, and vim.  (Eventually, it should be capable of replacing bash
+and diffutils as well, but it's not there yet.)</p>
+<p>The single file packaging combines a linux kernel (either a bootable kernel
+or User Mode Linux executable), initramfs, squashfs partition, and
+cryptographic signature.</p>
+<p>In 2.6, the kernel and initramfs are already combined into a single file.
+At the start of this file is either the obsolete floppy boot sector (just
+a stub in 2.6), or an ELF header which has 12 used bytes followed by 8 unused
+bytes.  Either way, we can use the 4 bytes starting at offset 12 to store the
+original length of the kernel image, then append a squashfs root partition
+to the file, followed by a whole-file cryptographic signature.</p>
+<p>A User Mode Linux executable should still run just fine (loading is
+controlled by the ELF segments, the appended data is ignored).  Note: don't
+strip the file or the appended data will be lost.</p>
+<p>Loading the bootable kernel image requires a modified boot loader that
+can be told the original size of the kernel, rather than querying the current
+file length which would be too long.  Hence the patch to Lilo allowing
+a "length=xxx" argument in the config file.</p>
+<p>Upon boot, the kernel runs the initramfs code which finds the firmware
+file.  In the case of User Mode Linux, the symlink /proc/self/exe points
+to the path of the file.  A bootable kernel needs a command line argument
+of the form firmware=device:/path/to/file (it can lookup the device in
+/sys/block and create a temporary device node to mount it with; this is
+in expectation of dynamic major/minor happening sooner or later).
+Once the file is found, /dev/loop0 is bound to it with an offset (losetup -o,
+with a value extracted from the 4 bytes stored at offset 12 in the file), and
+the resulting squashfs is used as the new root partition.</p>
+<p>The cryptographic signature can be verified on boot, but more importantly
+it can be verified when upgrading the firmware.  New firmware images can
+be installed beside old firmware, and LILO can be updated with boot options
+for both firmware, with a default pointing to the _old_ firmware.  The
+lilo -R option sets the command line for the next boot only, and that can
+be used to boot into the new firmware.  The new firmware can run whatever
+self-diagnostic is desired before permanently changing the default.  If the
+new firmware doesn't boot (or fails its diagnostic), power cycle the machine
+and the old firmware comes up.  (Note that grub does not have an equivalent
+for LILO's -R option; which would mean that if the new firmware doesn't run,
+you have a brick.)</p>
+<p>Currently, the version of gcc it builds and uses only has a C compiler, not
+c++.  This restricts the packages I can build with it, and you'd be amazed
+what kind of things need c++.  (Python is the one I'm really missing at the
+moment.)  Possibly uclibc++ could help here, but I'm also looking at tcc
+instead of gcc and binutils.  (tcc doesn't quite build an unmodified Linux
+kernel yet, and who knows what other packages would need to be tweaked,
+but it's definitely worth a look once version 1.0 comes out.  Perhaps
+some kind of front-end could make it do c++?)</p>
+<p>User Mode Linux is used during the build for a number of reasons.  The
+kernel headers used to build the C library may be newer than the kernel the
+system doing the build is using, and this may result in programs linked against
+this C library trying to use new features the existing kernel doesn't have.
+This tends to result in programs segfaulting.  Building a UML kernel to
+run these programs under during the build solves this problem, by translating
+the calls into ones the host system understands.</p>
+<p>UML also avoids the need to run the build as root.  The build needs to
+mount partitions, associate files with looback devices, create device nodes,
+create absolute paths requiring new entries in the root directory, chroot,
+and so on.  Doing all of this within the emulated UML environment avoids the
+need for root permissions on the host.</p>
+<p>That said, if you're running the same kernel version the Firmware Linux
+build is using, and you have root access, you can skip the UML wrapper to
+speed up the build and make things more easily debuggable.  I need to make a
+wrapper script for this, but basically in sources/scripts, stage 0.0, 1.1,
+and 2.2 are still needed, then 2.3 to package the final result (although 2.3
+depends on an executable built in 0.1).</p>
+<h2>How to build it</h2>
+<p>Run "./".  This runs all the stages (numbered files in
+sources/scripts) in sequence.  It'll start by downloading all the source code
+needed to build everything (which it'll keep around in the sources/packages
+directory for future builds).  If you just want to download the source,
+run "sources/scripts/0.0-*".  The 1-* stages create a cross-compile environment
+independent of the parent system.  The 2-* stages build the final system
+by using that cross-compile environment.</p>
+<p>My name is Rob Landley and my email address is
+My <a href=notes.html>development log</a> is probably the best way to keep
+track of what I'm working on, although I'll start a mailing list if enough
+people pester me.</p>