view www/news.html @ 928:0291e9591d18

Update downloads/binaries/README
author Rob Landley <rob@landley.net>
date Tue, 08 Dec 2009 20:22:53 -0600
parents e546c8767a0e
children 9d5a42a83de2
line wrap: on
line source

<!--#include file="header.html" -->

<b><h1>News</h1></b>

<h2><a name="12-08-2009" />December 8, 2009</h2>
<p><a href=downloads/firmware-0.9.9.tar.bz2>Version 0.9.9</a> is out,
based on <a href=http://impactlinux.com/hg/firmware/shortlog/876>hg commit 921</a>.</p>

<p>Just a checkpoint on the way to 1.0, which is still planned for
around new years, but there's been some schedule slippage already.</p>

<ul>
<li>
<p><b>New documentation</b></p>
<p>The presentation slides have been converted to
<a href=presentation.html>HTML</a>, although the much prettier (and much
bigger) <a href=downloads/presentation.pdf>PDF</a> is still available.</p>

<p>There is also a <a href=FAQ.html>FAQ</a>, which is still somewhat sparse.
(Ask away.)</p>
</li>

<li><p><b>Prebuilt binaries</b></p></li>

<p>Added shm support to uClibc.  Bugfix to the c++
compiler (it should work again).  The system images now have a "guest"
user/group so packages can more easily be built as a non-root user.
The shell prompt now includes the $HOST name.</p>
</li>

<li><p><b>Build scripts</b></p>
<p>A new trivial-but-convenient "./clean.sh" script lets you delete just the
target builds out of "build".  (If you delete that whole directory it has to
re-extract all the package tarballs and rebuilt the host tools, which is
often unnecessary work.)</p>

<p>Command recording has been completely redone.  No more RECORD_COMMANDS
variable, instead run sources/more/record-commands.sh to wrap the current
$PATH in build/wrappy (either after or instead of running host-tools.sh),
then run your build stages, and finally either run
sources/more/report-recorded-commands.sh to get a summary or
look at the raw command line lists in build/logs.</p>

<p>The Great Refactoring continues in sources/sections: broke up
binutils-gcc.sh into individual binutils, gcc, and ccwrap
stages, split off linux-headers and uClibc++ from uClibc.sh,
and spun off toybox.sh from busybox.sh.</p>

<p>Also broke kernel_cmdline() out of qemu_defaults() to make it easier to add
non-qemu sources/targets configurations without reinventing the wheel,
and fixed up dependencies for hw-target builds (which should work a lot
more reliably now).</p>

<p>Bumped up the number of $CPUS used by the build to 1.5x the actual number
(as long as your host has at least 512 megs of ram per actual CPU).  That
should help keep larger servers busy.</p>

<p>Some self-hosting fixes from Natanael Copa (making sure host-tools.sh
builds on a uClibc host): added --disable-nls to the e2fsprogs build
(what exactly is ./configure running all these tests for again?), and a
squashfs patch (substituting a function uClibc hasn't got).</p>

<p>Simplified system-image.sh to use killtree() instead of mysetsid.
Simplified the cross-compiler/cross-static logic to set the $PATH to
use the right one, rather than moving them around during the build.
Updated smoketest-all.sh to show "NONE" for system images that didn't build.
Added more support for SKIP_STRIP and CFLAGS=-g to the rest of the
packages.  Cleaned out the old USE_COLOR stuff which was superceded by
set_titlebar.</p>
</li>
</ul>

<hr>
<h2><a name=11-07-2009 />November 7, 2009</h2>
<p><a href=downloads/firmware-0.9.8.tar.bz2>Version 0.9.8</a> is out,
based on hg commit 876.  (If you want to see all the changes in this release,
look at <a href=http://impactlinux.com/hg/firmware/shortlog/876>commits 810 through 876</a>.)</p>

<p.The current plan is to cut one more release at the end of the month, and
then have the 1.0 release around new year's.  This could be considered a
1.0-pre1, if you like.</p>

<p>This release upgrades linux to 2.6.31.4 and busybox to 1.15.2.
(The uClibc version is still 0.9.30.1, they haven't had a new release
yet.)</p>

<p>The old #firmware irc channel on freenode.net has been merged into #edev,
so go there now to ask questions about this project.  (A _lot_ of
knowledgeable, helpful people hang there who can answer your embedded
development questions.)</p>

<h3><b>Bug tracker and roadmap</b></h3>

<blockquote>
<p>The <a href=http://redmine.impactlinux.com/projects/impact>new bug
tracker</a> has a <a href=http://redmine.impactlinux.com/projects/impact/roadmap>roadmap</a>
showing what we plan to implement for the next release or two.</p>

<p>It's also a place to file bugs, athough sending them to the <a href=http://lists.impactlinux.com/listinfo.cgi/firmware-impactlinux.com>mailing list</a>
is still the fastest way to get a fix.</p>
</blockquote>

<h3><b>New screenshots page</b></h3>

<blockquote>
<p><a href=screenshots>This page</a> lists every target, with
a "screenshot" of the target's boot messages under qemu, and associated
links to all the various prebuilt binaries for that target.</p>
</blockquote>

<h3><b>Fun with static linking</b></h3>

<blockquote>
<p>The cross-compiler-$ARCH tarballs are all compiled for i686 and
statically linked against uClibc for maximum portability.</p>

<p>The busybox binary in the root filesystem images is now built statically by
default, providing a 20% speed improvement in ./configure times under qemu
(due to the reduced page translation overhead).  You can switch this back
to dynamic with "BUILD_STATIC=none".</p>

<p>Static dropbear and strace binaries for each target are also available,
built by the new sources/more/native-static-build.sh script.</p>
</blockquote>

<h3><b>System Image interface changes</b></h3>

<blockquote>
<p>The run-from-build.sh script used to set up a development environment
out of the build directory, but didn't provide an easy way to do so based
on downloaded tarballs.  (You had to work out and provide extensive
command line arguments to the run-emulator.sh script.)  This has been
fixed.</p>

<p>The run-emulator.sh script in each system interface tarball is now much
simpler.  It now looks for distccd and $ARCH-cc in its $PATH, and sets up
distcc acceleration automatically if it finds them.  (It announces whether
or not it has managed to do so, and init.sh announces whether or not it
thinks it has distcc acceleration enabled just before launching a command
prompt.)  So setting up the distcc accelerator should be less of a pain.</p>

<p>Also, run-emulator.sh no longer takes any command line arguments.  Instead
the environment variable $HDB indicates a /dev/hdb disk image to mount on
/home, $HDBMEGS specifies the size of a (sparse ext2) HDB image to create if
it doesn't already exist, and QEMU_MEMORY indicates how many megabytes of
physical memory the emulator should allocate for the virtual system.</p>

<p>Each system-image also provides a simple wrapper dev-environment.sh
which calls run-emulator.sh with HDBMEGS=2048, HDB=hdb.img, and
QEMU_MEMORY=256.  (These are the values run-from-build.sh used to provide,
and give a reasonable build environment with 256 megs of physical memory
and 2 gigabytes of writeable /home space.)  The run-from-build.sh script
itself now merely adds the specified build/cross-compiler-$ARCH directory
to the $PATH, changes directory to the appropriate build/system-image-$ARCH,
and calls ./dev-environment.sh.  It's 3 lines.</p>

<p>The system images can now also automatically mount a third disk image
($HDC, specifying an /dev/hdc image to mount onto the /mnt directory), and if
that filesystem contains an executable "init" program in its root directory
the sbin/init.sh script will run that instead of launching a command prompt.</p>

<p>The new native-static-build.sh uses this HDC capability to automatically
run some native package builds without having to drive the build through
/dev/console via a wrapper script (the way smoketest.sh does).  See
sources/more/setup-native-static-build.sh for the script which generates
the hdc.sqf image for this.</p>

<p>Note that some targets (such as sh4 and powerpc) don't yet support /dev/hdc
due to QEMU board emulation limitations.</p>
</blockquote>

<h3><b>Build interface changes</b></h3>

<blockquote>
<p>The new "sources/more" directory contains scripts the end user can
call, but which aren't important enough to clutter up the top level
directory with.  The buildall.sh and smoketest-all.sh scripts moved there,
as did the cronjob.sh used to build the nightly snapshots.</p>

<p>The new sources/more/native-static-build.sh natively compiles dropbear and
strace under the emulator, and copies them out to the host through the virtual
network (into the build/cron-temp directory) using busybox ftpd on the host.
This provides an example of how to use the new HDC /mnt/init functionality
to perform automatic builds under the emulator, and copy the results out
through the network.  It uses the new sources/timeout.sh wrapper to detect
hung builds (defined as builds that haven't produced a line of output within
a given number of seconds).</p>

<p>The new BINARY_PACKAGE_TARBALLS variable (in the file "config") tells
the build to create a separate tarball of the binaries produced building
each package.  (These more granular tarballs aren't shipped yet, but may
be in a future release.)</p>

<p>The new SKIP_STRIP variable tells the build not to strip binaries.
(Add in CFLAGS=-g for enormous binaries full of debug info.)</p>
</blockquote>

<h3><b>Target upgrades</b></h3>

<blockquote>
<p>Powerpc and sh4 should now work fine with stock qemu 0.11.0.</p>

<p>The arm platform now has an armv4tl target.  This is a little-endian armv4
eabi, for armv4 chips offering the "thumb" extension which eabi requires.
(The plain armv4l target is still oabi.  The oabi/eabi thing is different
binary ABIs, a bit like Linux vs BSD binaries or aout vs elf.  The eabi
standard is the new one, but some of the older arm hardware can't support
it.  This takes it down as far as it can go.)</p>
</blockquote>

<h3><b>Infrastructure</b></h3>

<blockquote>
<p>Behind the scenes, some extensive work is going on to simplify the
build scripts.  (The documentation's fallen a bit behind reality here,
but should be back up to date next release.)</p>

<p>The uClibc miniconfigs are now automatically generated, with the
bulk of the config living in sources/baseconfig-uClibc and the few
target specific lines being appended from the UCLIBC_CONFIG variable
defined in the target's settings file.</p>

<p>The other big change is factoring out lots of common code, moving it into
the sources/sections directory, so things like the uClibc, busybox, and
toolchain builds now live in one place and are called from the various
other scripts that need them.  (This may eventually allow alternate build
scripts for these packages, potentially swapping out glibc for uClibc, or
llvm/clang for gcc.  In the meantime, it eliminates a lot of redundant code.)</p>

<p>This required lots of small cleanups, such as making the scripts now
consistently use the $STAGE_DIR variable for their output directory.
Some other cleanups were primarily cosmetic, such as moving the extracted
tarball cache directory from build/sources to build/packages.  (Tarballs
live in packages, extracted tarballs live in build/packages.)</p>

<p>The root filesystem now contains simple /etc/passwd and /etc/group files,
because dropbear won't work without them (not even to ssh out).</p>

<p>Several small bugfixes (ccwrap no longer segfaults if $PATH isn't
exported, /etc/resolv.conf doesn't get overwritten unnecessarily by init.sh,
etc).</p>
</blockquote>

<hr>
<h2><a name=08-20-2009 />August 20, 2009</h2>
<p><a href=downloads/firmware-0.9.7.tar.bz2>Version 0.9.7</a> is out, a little
over a month and a half late.  It's a larger update than usual, so the
release notes are a bit long.</p>

<p>This release is based on
<a href=http://impactlinux.com/hg/firmware/log/807>mercurial version 807</a>
of the build scripts,
and includes <b>Linux 2.6.30.4</b> and <b>BusyBox 1.14.3</b>.
No new uClibc release is out since last time.</p>

<p>Lots of little bugs got fixed.  For example, this release should build on
Fedora 11 (and any other platform that hasn't got "which" installed by
default), and now supports distcc and ccache on the host (and will use them
automatically if they are installed).</p>

<h3><b>Changes to build stages: new root-filesystem, cross-static, and
native-compiler tarballs, upgraded buildall.sh and smoketest-all.sh.</b></h3>

<blockquote>
<p>The old "mini-native" stage has been renamed "root-filesystem"
(the same way package-mini-native became system-image last release).  The
script names (and resulting tarballs) now indicate what each stage makes.
So build.sh calls host-tools.sh, cross-compiler.sh, root-filesystem.sh,
and system-image.sh, in that order.</p>

<p>The old build-static-toolchains.sh script is gone, with its functionality
integrated into buildall.sh (which now builds i686 hosted static toolchains
for each target by default).  This is implemented by two new environment
variables triggering optional behavior: STATIC_CROSS_COMPILER_HOST
and BUILD_STATIC_NATIVE_COMPILER.</p>

<p>Setting STATIC_CROSS_COMPILER_HOST to an architecture name creates a
cross compiler statically linked to run on that host (via a technique
known as canadian cross compile).  For example, the invocation:</p>

<blockquote>
<p>BUILD_CROSS_COMPILER_HOST=i686 ./build.sh armv4l</P>
</blockquote>

<p>Would create a tarball called "cross-static-armv4l.tar.bz2" which was
built to run on an i686 host, and was statically linked (against uClibc)
on that host to be distribution independent.</p>

<p>Setting BUILD_STATIC_NATIVE_COMPILER to any non-empty value creates a
native compiler for the target (packaged as native-compiler-$ARCH.tar.bz2),
which is statically linked against uClibc so it can be extracted and run on
just about any system of the appropriate type.  (So if you can get a Linux
shell prompt and have enough storage and memory, you should be able to
compile stuff.)</p>

<p>The upgraded smoketest-all.sh script runs smoketest.sh on each
target built.  (The smoketest.sh script attempts to compile and run "hello
world" natively within a system image under qemu, and indicates whether or
not it worked via "pass" or "fail".)  Note that smoketest-all.sh now
autodetects which targets have been built (have system images in the build
directory), and smoketest.sh times out after 60 seconds so targets that hang
will return failure eventually.</p>

</blockquote>

<h3><b>Squashfs 4.0 now the default system image type</b></h3>

<blockquote>
<p>The root filesystem image type is now squashfs by default.  (To change it set
SYSIMAGE_TYPE to ext2 or initramfs, either in your environment or the file
"configure".)</p>

<p>This involved upgrading the userspace squashfs support package
to the version 4.0 supported by the upstream kernel, and numerous small fixes
to work with a read-only root filesystem (with writeable spaced mounted on
/home and /tmp).</p>
</blockquote>

<h3><b>Target upgrades</b></h3>
<blockquote>
<h3><b>ARM upgrades</b></h3>

<blockquote>
<p>Lots of work on the arm targets, with the addition of a new armv6l target
and conversion of the armv5l target to
<a href=http://www.linuxfordevices.com/c/a/Linux-For-Devices-Articles/Why-ARMs-EABI-matters/>ARM EABI</a> (which used to be documented <a href=http://www.arm.com/products/DevTools/ABI.html>here</a>,
but ARM screwed up its website and now you have to go
<a href=http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.subset.swdev.abi/index.html>here</a>).  The armv4l target still provides an ARM v4
little endian soft float OABI target (which is "the i386 of ARM", meaning it
should run on just about any arm hardware, if a bit slowly).  A new
armv4be target is the same as armv4l except big endian.</p>

<p>A new hw-tct-hammer target supports the <a href=http://www.tincantools.com/product.php?productid=16143>Tin Can Tools hammer/nail board</a>.</p>
</blockquote>

<h3><b>Updated sparc support, almost works now.  (uClibc for sparc still sucks.)</b></h3>

<blockquote>
<p>The sparc target has also been upgraded to the point where it now gives
a shell prompt.  (And immediately dies with a bus error or a hang if you try
to do anything, because uClibc for sparc still doesn't work.  The "upgrade"
consisted of statically linking everything because uClibc's dynamic loader
doesn't work for sparc.)</p>
</blockquote>

<h3><b>PowerPC remains fiddly due to QEMU issues, new powerpc-440fp target</b></h3>

<blockquote>
<p>PowerPC support in QEMU is
<a href=http://lists.gnu.org/archive/html/qemu-devel/2009-07/msg01007.html>officially unstable</a>, so although FWL's powerpc target hasn't significantly
changed from last release, the version of QEMU in source control no longer
runs it (nor the binary powerpc system images released with FWL 0.9.6) and
QEMU's developers do not consider this a regression.  While we search for a new
Linux kernel .config that matches QEMU's current expectations (and await a
release considered "stable"), note that setting HOST_BUILD_EXTRA=1 before
running host-tools.sh will build a version of qemu that does run powerpc and
put it in build/host where run-from-build.sh can find it.  (Note also that
building qemu from source still takes significantly longer than the rest of the 
host-tools.sh stage combined.)</p>

<p>A new powerpc-440fp target produces code for the powerpc 440 with hardware
floating point.  (Theoretically QEMU's "bamboo" board emulation should
run this, but QEMU doesn't emulate a 440 processor yet.  We compromise by
running a ppc440 filesystem under qemu's power mac emulation, to take
advantage of the ppc440 instruction set being almost a subset of full powerpc.
This gets us a shell prompt, but throws illegal instruction errors if you
try to do anything fancy.)</p>

<p>(Note that powerpc-440fp is not a hw-target for powerpc, it's a different
processor instruction set like i586 vs i686.  Alas, one that qemu doesn't
properly emulate yet.)</p>

</blockquote>

</blockquote>

<h3><b>Now using last GPLv2 release of GCC, and future plans</b></h3>

<blockquote>
<p>The upgrade to <b>gcc 4.2.1</b> gives us
<a href=http://gcc.gnu.org/ml/gcc-announce/2007/msg00003.html>the last
GPLv2 licensed version of gcc</a> (thus avoiding
<a href=http://lwn.net/Articles/343608/>various problems</a>).
<a href=http://savannah.gnu.org/forum/forum.php?forum_id=4932>Make 3.81</a>
was already the last GPLv2 release of that project.</p>

<p>Note that <b>bash 2.05b</b> is still intentionally ancient (to avoid the
bloat of 3.x), and may be replaced with busybox ash as the default FWL shell
in a future release.  Similarly, although <b>binutils 2.17</b> was the last
GPLv2 version from the FSF, an upgrade to
<a href=http://ftp.kernel.org/pub/linux/devel/binutils/release.binutils-2.18.50.0.1>the
last GPLv2 release of Red Hat's binutils fork, 2.17.5.0.17</a> is under
consideration for a future release.  But mostly the GNU compiler toolchain
has been upgraded as far as possible for the forseeable future, either because
new versions provide no significant benefits or because they're not available
under a license compatible with the Linux kernel's.</p>

<p>Future upgrades to these packages will either be via individual patches
(maintaining GPLv2 license) or by investigating alternative non-FSF
packages such as OpenBSD's <a href=http://lwn.net/Articles/255558>PCC</a> or
Apple's <a href=http://llvm.org>LLVM</a>.  In preparation for support of
alternate compilers, the build scripts have moved from using "gcc" to the
more generic "cc" for the compiler name wherever possible, but currently no
drop-in replacement compiler is available.</p>

</blockquote>

<h3><b>Implementation details</b></h3>

<blockquote>
<p>Each target configuration renamed "details" to "settings", and added
a "description".  (This makes Gentoo From Scratch's kconfig easier.)</p>

<p>The download.sh stage now populates a "packages" directory instead of
"sources/packages", meaning that everything in the sources directory now
comes from the FWL source tarball.  To do the equivalent of "make clean"
you can "rm -rf build", and to do a "make distclean" you can
"rm -rf packages build".</p>

<p>The build no longer deletes the alt-* tarballs when UNSTABLE isn't
set, you must do that by hand if you want an updated alt- version of a
package.  The packages/MANIFEST file creation can now use the extended
tarball information to detect git version numbers.</p>

<p>The === bars now show the current stage name and architecture each
package is being built for.  (Pipe the build output to grep "^===" to see
just the start of each new package build.)  The same information is sent
to the title bar of your xterm, export NO_TITLE_BAR=1 if you don't want the
build to update the title bar.</p>

<p>The BUILD_STATIC option now applies to all packages built in
root-filesystem.sh, and can be set from a target configuration's settings file
(as sparc is now doing).</p>

<p>The BUILD_VERBOSE option passes V=1 to the linux, busybox, and uClibc
builds that otherwise sanitize their output.</p>

<p>You can now set NATIVE_TOOLCHAIN=only to build just the compiler
and not the other parts (busybox, make, toybox, bash...)  This is used
by the canadian cross steps in build.sh to create static cross and native
compilers.</p>

<p>This release broke up scripts/include.sh so that script just
sets lots of environment variables, and now includes scripts/functions.sh
to define shell functions.  The architecture setup (reading architecture
files and creating temporary directories and such) is now done via the
"read_arch_dir $ARCH" shell function, so merely including sources/include.sh
should no longer have significant side effects.  Similarly, setupfor now
has an EXTRACT_ONLY environment variable instead of depending on a blank
$ARCH (which host-tools.sh has, despite wanting to snapshot source so it can
build stuff).</p>

<p>Various fixes to make parallel builds work better.  (So it neither leaves
background processes running nor kills the parent shell and closes your
xterm when you hit ctrl-c.)  The new "killtree" shell function,
which kills a process and all its children and grandchildren recursively with
pgrep.  We also build our own sources/toys/mysetsid.c program with the host
compiler (because the normal command line "setsid" doesn't also do a tcsetpgrp
on stdin so signal handling isn't forwarded to the new session).  This should
more reliably allow ctrl-c to exit smoketest.sh and system-image.sh and such,
and let FORK=1 ./buildall.sh stop all its background processes when
interrupted.  (Note that smoketest-all.sh also honors FORK=1.)</p>

<p>The logs from buildall.sh now live in build/logs.</p>

<p>The new config option ROOT_NODIRS avoids creating the normal set of empty
directories in the new root filesystem.  (The old /tools support went
away, but a BUILD_STATIC ROOT_NODIRS build can be extracted into /tools
and used from there.  Then either symlink "/lib" to "/tools/lib" or "export
UCLIBC_DYNAMIC_LINKER=/tools/lib/ld-uClibc.so.0" to add more stuff to
/tools natively.)</p>

<p>The compiler wrapper (sources/toys/ccwrap.c) can now use
architecture-specific WRAPPER_TOPDIR values.  (I.E. armv4l-cc could
"export armv4l_WRAPPER_TOPDIR=/path/to/directory".)  This makes
canadian cross compiles a lot easier.</p>
</blockquote>

<hr>
<h2><a name=04-02-2009 />April 2, 2009</h2>
<p><a href=downloads/firmware-0.9.6.tar.bz2>Version 0.9.6</a> includes
Linux 2.6.29 and uClibc 0.9.30.1, now with support for sh4 and upgraded powerpc
support.</p>

<p>The big news is the <a href=http://impactlinux.com/code/gfs/>Gentoo
From Scratch</a> project, which extends a FWL system image into a Gentoo
Stage 1 environment, building natively inside qemu.</p>

<p>The <a href=documentation.html>documentation</a> has been extensively
rewritten (but needs more).  The source now contains a README and each image
has a usr/src/MANIFEST file listing the packages it built from.</p>

<p>The cross and native toolchains now include ldd, readelf, and
ldconfig.  Added support for hardware targets (an unfinished example
of which is hw-wrt610n) that use the same cross compiler and system
image as a qemu platform, but build/package a different kernel to boot
on actual hardware.</p>

<p>Several new build options:</p>
<ul>
<li><p><b>$SYSIMAGE_TYPE</b> defaults to "ext2" but can now
produce and boot "initramfs" images as well. (And squashfs, but that doesn't
work yet due to missing features in squashfs tools version 3.4.)</p></li>
<li><p>Setting <b>$HOST_BUILD_EXTRA</b> compiles qemu 0.10.1 in host-tools.sh.
You currently need to do this to run sh4 or powerpc, 0.10.1 has some bugs
needing patches.  Not on by default because it takes a _long_ time.</p></li>
<li><p><b>$NO_CLEANUP</b> keeps source around after builds for debugging purposes.</p></li>
<li><p><b>$SYSIMAGE_HDA_MEGS</b> lets you control the size of the ext2 system images.</p></li>
<li><p><b>$SNAPSHOT_SYMLINK</b> lets the extracted source live on a different
filesystem than the build happens on.  (Once the updated squashfs userspace
tools ship, build-static-toolchains.sh may be updated to use this.)</p></li>
<li><p><b>$CROSS_SMOKE_TEST</b> use qemu application emulation to test run hello
world program at the end of the cross compiler build stage.</p></li>
<li><p><b>$SKIP_STAGE_TARBALLS</b> don't create tarballs of each stage,
just populate the directories.</p></li>
<li><p><b>$NATIVE_RETROFIT_CXX</b> retroactively add uClibc++ to the cross
compiler tarball during the native build stage.</p></li>
</ul>

<p>The run-emulator.sh scripts of system images have several new options,
do "./run-emulator.sh --help" for a list.  Takes several environment
variables such as $QEMU_EXTRA and $KERNEL_EXTRA.  Root filesystems can
now run in read only mode with "KERNEL_EXTRA=ro".</p>

<p>Lots of build infrastructure upgrades.  The build stages have been
reorganized somewhat (most noticeably package-mini-native.sh is now called
system-image.sh).  The "sources/packages" directory has moved up a
directory (to separate repository files from stuff downloaded at runtime).
Upgrades to the cc wrapper and the USE_UNSTABLE infrastructure.  Better
stripping of the cross compilers and system images, so everything's a bit
smaller.  The mirror list now lives in download.sh.  New buildall.sh is a
wrapper around build.sh.  New build-static-toolchains.sh builds them under
qemu; set FORK=1 to build them in paralell just like buildall.sh.  The
ext2 system images are now built at a small size with genext2fs and
then expanded with resize2fs (because genext2fs slows down exponentially
as size increases; this means e2fsprogs is built in host-tools.sh now).
Distcc has been upgraded, and should work more reliably now.</p>

<p>Too many little bug fixes to list.</p>

<hr>
<h2><a name=12-16-2008 />December 16, 2008</h2>
<p>The mailing list moved to firmware at impactlinux.com.  The link in
the nav bar on the left has changed, but the
<a href=http://www2.them.com:8080/cgi-bin/mailman/listinfo/firmware>old
archives</a> are still there for the moment.</p>

</hr>
<h2><a name=12-03-2008 />December 3, 2008</h2>
<p>After several false starts (I.E. last minute bugs, yes I test this thing
before shipping it),
<a href=downloads/firmware-0.9.5.tar.bz2>Version 0.9.5</a> is out, inching
towards a 1.0 release.  Lots of new work this time.</p>

<p>The biggest news is uClibc 0.9.30 and uClibc++ 0.2.2.  The native toolchain
now supports C++, hooked up to distcc and everything.</p>

<p>The <a href=documentation.html>documentation</a> has been extensively
rewritten.  (There's always more to do, but most of it isn't horribly out
of date anymore.)</p>

<p>The packaging stage now uses genext2fs instead of User Mode Linux, so
the most brittle part of the build has been removed.  FWL can now fully
rebuild itself under itself, all the way through the packaging stage.</p>

<p>My personal website's getting a touch overwhelmed, so a higher bandwidth
site is now available at
<a href=http://impactlinux.com/fwl>impactlinux.com</a>.  This should become
the new primary website going forward.</p>

<p>BusyBox has also been upgraded (to 1.13.0), and the new USE_UNSTABLE
infrastructure allows test versions of any package to be downloaded and built
without forking the build scripts, so future package upgrades can be dealt
with as side projects without blocking other important work.</p>

<p>The packaging stage now uses genext2fs instead of User Mode Linux, so
the most brittle part of the build has been removed.  FWL can now fully
rebuild itself under itself, all the way through the packaging stage.</p>

<p>The old forkbomb script has been replaced with sources/build-all-targets.sh.
with an argument --fork to build all targets in parallel, and "--fork 3"
to limit the parallelism.  This script can run from a cron job, producing
a "buildall" directory with tarballs, compressed logs, and a README file
listing the version of each package.  The environment variable USE_STATIC_HOST
specifies an $ARCH to create statically linked cross compilers
for.  (The build will then try to use them for the later stages,
and won't get far if you specify an arch that can't run on the host.)</p>

<p>The include.sh script has been broken up, with a new sources/functions.sh
containing shell functions (like download, setupfor, and cleanup) and
a new "config" setting the user-editable environment variables that control
the build.  Several of these variables have been renamed and their number has
grown.  You can now specify the colors for each build stage using the variables
at the end of config.</p>

<p>Building mini-native in a Linux From Scratch style /tools directory is
no longer the default.  Instead set NATIVE_TOOLSDIR=1 if you want that
kind of layout instead of a more conventional layout.  (Similarly,
NATIVE_TOOLCHAIN=1 now enables the native toolchain, but that variable
defaults to enabled in config.)</p>

<p>See the <a href=documentation.html>docs</a> for details.</p>

<hr>
<h2><a name=08-06-2008 />August 6, 2008</h2>
<p><a href=downloads/firmware-0.9.0.tar.bz2>Version 0.9.0</a> is out
(<a href=http://landley.net/hg/firmware/shortlog/378>changeset 378</a>)
and can rebuild itself under itself.  (The packaging step still requires User
Mode Linux to create ext2 images, which only works on x86 and x86-64 hosts.
The next release should replace that with something more portable.)
The root filesystem is now based on BusyBox 1.11.1 (plus a few bug fixes).</p>

<p>The prebuilt binary images have been moved into their own
<a href=downloads/binaries>binaries</a> directory.
The prebuilt binary <a href=downloads/binaries/cross-compiler>cross
compilers</a> are statically linked against uClibc for better portability.
Updated the <a href=downloads/README>README</a> to better describe the
contents of the <a href=downloads/binaries/system-image>system-image
tarballs</a>.</p>

<p>The forkbomb.sh script can now take a number of parallel targets to
launch, for example "./forkbomb.sh --fork 3".  (The resulting display is kind
of horrible but the out-$ARCH.txt files are still legible.)  Leaving off the
number still builds all targets in parallel, with the original display.</p>

<p>The include.sh script was reorganized so that interesting environment
variables the user might want to export for themselves are set near the top
of the script.  Currently this includes:</p>

<ul>
<li><p><b>BUILD_SHORT</b> - Makes the mini-native.sh stage skip building
development tools.  The resulting mini-native root filesystem contains uClibc
and busybox, but nothing else.  (Export "BUILD_SHORT=headers" to put back the
toolchain headers for use by another compiler, such as tinycc.)</p>

<li><p><b>BUILD_STATIC</b> - Tell cross-compiler.sh to create static
binaries, for a more portable cross compiler toolchain.  (If you want them
linked against uClibc, build a mini-native for your host, chroot into it, and
then rebuild under that.)</li>

<li><p><b>PREFERRED_MIRROR</b> - Tell download.sh to fetch packages from a
non-default location (falling back to the standard mirror list if it can't get
one from there).</p></li>

<li><p><b>RECORD_COMMANDS</b> - Record a list of all command lines used during
each build stage log files named "cmdlines.$STAGE.$PACKAGE" under the build
directory.  Each file can be turned into a list of the command names used via
"awk '{print $1}' build/cmdlines* | sort -u".  The script
sources/toys/report_recorded_commands.sh gives a report of all commands
used by each package after a call to "RECORD_COMMANDS=1 ./forkbomb.sh".
(This is an easy way to find out exactly what environmental dependencies a
package needs to build, or at least what it uses out of $PATH).</p></li>
</ul>

<p>Updated documentation is in progress...</p>

<hr>
<h2><a name=06-06-2008 />June 6, 2008</h2>
<p><a href=downloads/firmware-0.4.0.tar.bz2>Version 0.4.0</a> is out
(<a href=http://landley.net/hg/firmware/shortlog/345>changeset 345</a>),
with kernel 2.6.25.4 and the "distcc trick" working out of the box.</p>

<p>The distcc trick accelerates a native build by calling out to the
cross compiler.  To use it, download the appropriate cross-compiler-$ARCH and
system-image-$ARCH tarballs, run the "run-with-distcc.sh" from the system image
directory with the path to the cross compiler directory as its first argument.
(You need to have distcc installed on the host system.)</p>

<p>After building from source, the cross-compiler and system-image directories
remain in the build directory.  The "emulator-build.sh" runs these with
distcc acceration set up.  The script "smoketest.sh" is a wrapper
around emulator-build.sh which feeds a script into qemu's stdin to compile
and run a multi-threaded "hello world" under the emulator, using distcc and
the cross compiler.</p>

<p>If smoketest.sh works for a platform, it means a lot of stuff worked to get
that far.  The cross compiler worked to build the system, and the emulated
system booted so QEMU, the uClibc config, and the kernel config agreed.
The emulated system has a working virtual hard drive and serial port, and for
distcc to work it has a working virtual network connection.  Running "make"
would also require a working realtime clock.</p>

<p>The seven commands missing in order to run the FWL build under itself
are bzip2, sort, diff, wget, install, od, and find.  (The busybox versions,
where available, had a bug, and they're not in toybox yet.)  I'm trying to
get those in for the next release in September.</p>

<p>Sorry I missed a release back in April.  Shouldn't happen again.</p>

<hr>
<h2><a name=01-29-2008 />January 29, 2008</h2>
<p><a href=downloads/firmware-0.3.1.tar.bz2>Version 0.3.1</a> is out
(<a href=http://landley.net/hg/firmware/shortlog/275>changeset 275</a>), with
kernel 2.6.24.  The <a href=downloads/images>images</a>
are now tarballs each containing the ext2, zImage, and run script files.
The run scripts now run qemu-setup.sh by default so /proc, /sys, /dev
and the virtual network are initialized, and the default shell is busybox
ash (so cursor keys and history work).</p>

<p>Sparc is still broken (tracked it down to a dynamic linking bug, I think,
haven't fixed it yet) and m68k is failing to build with an internal compiler
error (next release I should upgrade the compiler).  All the other targets
have a working virtual network, and all but powerpc exit qemu when you
exit the PID 1 command shell.</p>

<p>Building on an x86-64 host< should no longer be insanely slow (the
gcc-min-heapsize arguments that allow 32-bit hosts to build in
128 megs of ram do very impolite things to 64-bit hosts), and the resulting
x86-64 cross compiler should now be properly relocatable.</p>

<p>Better build dependency tracking: the host-tools script is now
populating a single directory with all the commands needed by the build, and
restricting $PATH to that directory (so it can't accidentally call any
commands that aren't explicitly listed).  It's building toybox and busybox to
provide most of these commands (the remaining ones are listed towards the end
of include.sh), and creating symlinks to the remaining ones out
of the original $PATH.  This allows it to build on a wider range of host
distributions.</p>

<hr>
<h2><a name=11-02-2007 />November 2, 2007</h2>
<p><a href=downloads/firmware-0.3.0.tar.bz2>Version 0.3.0</a> is out
(changeset 242), with kernel 2.6.23.  PowerPC finally works under
qemu (network and everything), and it's building a m68k target
(although I have no emulator to run that).  That means there are
ten targets, of which eight run under qemu.  (Sparc is still
broken, let me know if anybody actually cares.)</p>

<p>The new "./forkbomb.sh --fork" script builds all the targets in
parallel, or use --nofork if you haven't got the memory for that
(and --watch to watch and existing build if you ctrl-c out of
the display before it's done).</p>

<p>The build is now smarter about extracting only one copy of the
source code (in build/sources) and re-using it for multiple builds,
building out of tree (via cp -rs to create a tree full of symlinks
so the package doesn't need native out-of-tree support).</p>

<hr>
<h2><a name=09-02-2007 />September 2, 2007</h2>
<p>Added <a href=new_platform.html>a porting guide</a>.  I'm working on
m68k, ppc, big endian arm, and maybe blackfin targets for the next release,
and thought other people might be interested in the process.  (The vast
majority of it is fiddly debugging.  These are the other steps.)</p>

<hr>
<h2><a name-06-27-2007 />June 27, 2007</h2>
<p><a href=downloads/firmware-0.2.2.tar.bz2>Version 0.2.2</a> is out
(changeset 185).  Using uClibc 0.9.29 release, several small bugfixes,
new mips big-endian config, actually work when /bin/sh->dash (mostly
by explicitly saying /bin/bash everywhere), automatically detect
the number of CPUs in the machine to feed -j to make, and better
diagnostic output.</p>

<hr>
<h2><a name=04-09-2007 />April 9, 2007</h2>
<p><a href=downloads/firmware-0.2.1.tar.bz2>Version 0.2.1</a> is out
(changeset 156).  Significant upgrades to armv4l: the native compiler works
now, and it's using soft-float.  Added armv5l and i586 targets to show how
to do that kind of variant.  Added preliminary powerpc support (although
qemu can't run it yet).  Reorganized the download directory and rewrote
the <a href=downloads/README>README</a>.  Added squashfs back (the kernel
supports it, but it's not using it yet).  Moved uClibc to miniconfig.
(Yes, the patch for this is huge, and yes I need to push it all upstream into
the Linux kernel.  On the bright side, this one builds on a host that doesn't
have curses installed.)  Stopped trying to build qemu (at least until they
get the gcc 3.x dependencies cleaned out), now just warn if it's not there
(so far the build only uses it to sanity-test the cross compiler, which is
now skipped automatically if it's not there).</p>

<p>I note that some versions of Ubuntu (Edgy and up) are pushing a political
agenda.  They install bash, but point /bin/sh to dash instead.  (So why
install bash if you're not going to use it?)  This breaks all sorts of things,
and if somebody can tell me how to force make to run "/bin/bash" instead of
"/bin/sh", I'll consider trying to deal with the mess.  Until then, I suggest
"rm /bin/sh; ln -s bash /bin/sh" to get a working system.  (I'm no fan of
bash, but dash is just broken.)</p>

<p>(Those of you who use vi and want to be able to use the cursor keys while
in insert mode might want to do something similar with vi->vim.  Ubuntu seems
to be trying to train users to change their habits.  Dog biscuits would be a
more honest approach.)</p>

<p>Update: I've been pointed at
<a href=https://wiki.ubuntu.com/DashAsBinSh>this</a> as evidence Ubuntu isn't
pushing a political agenda.  Ok, maybe it's just really bad technical judgement.
Are they honestly saying that people write shell scripts for the speed?  And
how do you make a size argument and then install bash by default in ADDITION
to the other shell?  Boggle.  In 1991 after Linus Torvalds taught his term
program to understand the minix filesystem so he could upload and download
without rebooting, he taught it to handle the system calls of bash so he could
rm/mv/mkdir without rebooting.  That's where Linux 0.0.1
came from.  Bash was the default Linux shell _before_ 0.0.1.  Be very
careful messing with that kind of a de-facto standard, guys.  Dash is
garbage.  Background a process with &amp; and then hit ctrl-c: it'll kill it.
It doesn't understand blah/{blah,blah} file list syntax.  It can't do
"source file.sh" (just ". file.sh", which is unreadable).  I repeat: I'm not a
fan of bash (yeah, the bloat's getting crazy), but dash is broken.</p>

<p>And that still doesn't explain why "vi" can't use the cursor keys in
insert mode, but "vim" can...</p>

<hr>
<h2><a name=03-06-2007 />March 6, 2007</h2>
<p>Version 0.2.0 is out.  This one builds ext2 images which can boot up to a
shell prompt under qemu.  This works on i686, armv4l, x86_64, and mipsel.
(Sparc's broken.  Anyone who cares about sparc is welcome to debug it.)</p>

<p>There are now prebuilt ext2 image files in the download directory.
The kernels are in the mini-native tarballs.  (Yeah, I should repackage that
next release.)  The system emulator invocation for each platform is
the appropriate sources/configs file, and run-mini-native.sh invokes 'em.</p>

<p>The mailing list is in the usual place if you have any questions.</p>

<hr>
<h2><a name=01-14-2007 />January 14, 2007</h2>
<p>There's a new irc channel for the project, #firmware on freenode.</p>

<p><a href=http://landley.net/hg/firmware?cl=88>Changeset 88</a> builds
i686, x86_64, armv4l, mips, and sparc.  I've made a release tarball of
that (<a href=downloads/firmware-0.1.1.tar.bz2>firmware 0.1.1</a>, "It works
for me"), and updated the prebuilt cross-compiler tarballs in the
<a href=downloads>downloads</a> directory.  I've added a big README and
prebuilt mini-native tarballs for each platform (although packing them up into
something qemu can boot is currently left as an exercise for the reader).</p>

<p>Speaking of which, I'm currently working on adding ext2 packaging
(via <a href=http://landley.net/code/toybox>toybox</a>) to the build scripts,
so qemu system emulation can boot the result and then run the next stage
automatically.</p>

<hr>
<h2><a name=12-28-2006 />December 28, 2006</h2>
<p><a href=http://landley.net/hg/firmware?cl=68>Changeset 68</a> builds
a native build environment with a working toolchain.  And in celebration, I've
revamped the website with an actual navigation bar and content and such.</p>

<p>I should put out a release soon.</p>

<hr>
<h2><a name=12-05-2006 />December 5, 2006</h2>
<p><a href=http://landley.net/hg/firmware?cl=27>Changeset 27</a> builds
a relocatable armv4l cross-compile toolchain!  Download the tarball,
run ./download.sh, then run ./build.sh, then grab the "build/cross-compiler"
directory and use "bin/armv4l-unknown-linux-gnu-gcc" out of that (which
I admit is a bit of a mouthful, for which I blame the FSF).</p>

<hr>
<h2><a name=08-06-2006 />August 6, 2006</h2>
<p>Mecurial repository created.  Nothing to see yet, move along...</p>

<!--#include file="footer.html" -->