view www/news.html @ 694:55a526672f43 0.9.6

Announce 0.9.6 release.
author Rob Landley <rob@landley.net>
date Thu, 02 Apr 2009 21:27:25 -0500
parents e85e59bd76aa
children d326c68aa747
line wrap: on
line source

<!--#include file="header.html" -->
<b><h1>News</h1></b>

<h2>April 2, 2009</h2>
<p><a href=downloads/firmware-0.9.6.tar.bz2>Version 0.9.6</a> includes
Linux 2.6.29 and uClibc 0.9.30.1, now with support for sh4 and upgraded powerpc
support.</p>

<p>The big news is the <a href=http://impactlinux.com/code/gfs/>Gentoo
From Scratch</a> project, which extends a FWL system image into a Gentoo
Stage 1 environment, building natively inside qemu.</p>

<p>The <a href=documentation.html>documentation</a> has been extensively
rewritten (but needs more).  The source now contains a README and each image
has a usr/src/MANIFEST file listing the packages it built from.</p>

<p>The cross and native toolchains now include ldd, readelf, and
ldconfig.  Added support for hardware targets (an unfinished example
of which is hw-wrt610n) that use the same cross compiler and system
image as a qemu platform, but build/package a different kernel to boot
on actual hardware.</p>

<p>Several new build options:</p>
<ul>
<li><p><b>$SYSIMAGE_TYPE</b> defaults to "ext2" but can now
produce and boot "initramfs" images as well. (And squashfs, but that doesn't
work yet due to missing features in squashfs tools version 3.4.)</p></li>
<li><p>Setting <b>$HOST_BUILD_EXTRA</b> compiles qemu 0.10.1 in host-tools.sh.
You currently need to do this to run sh4 or powerpc, 0.10.1 has some bugs
needing patches.  Not on by default because it takes a _long_ time.</p></li>
<li><p><b>$NO_CLEANUP</b> keeps source around after builds for debugging purposes.</p></li>
<li><p><b>$SYSIMAGE_HDA_MEGS</b> lets you control the size of the ext2 system images.</p></li>
<li><p><b>$SNAPSHOT_SYMLINK</b> lets the extracted source live on a different
filesystem than the build happens on.  (Once the updated squashfs userspace
tools ship, build-static-toolchains.sh may be updated to use this.)</p></li>
<li><p><b>$CROSS_SMOKE_TEST</b> use qemu application emulation to test run hello
world program at the end of the cross compiler build stage.</p></li>
<li><p><b>$SKIP_STAGE_TARBALLS</b> don't create tarballs of each stage,
just populate the directories.</p></li>
<li><p><b>$NATIVE_RETROFIT_CXX</b> retroactively add uClibc++ to the cross
compiler tarball during the native build stage.</p></li>
</ul>

<p>The run-emulator.sh scripts of system images have several new options,
do "./run-emulator.sh --help" for a list.  Takes several environment
variables such as $QEMU_EXTRA and $KERNEL_EXTRA.  Root filesystems can
now run in read only mode with "KERNEL_EXTRA=ro".</p>

<p>Lots of build infrastructure upgrades.  The build stages have been
reorganized somewhat (most noticeably package-mini-native.sh is now called
system-image.sh).  The "sources/packages" directory has moved up a
directory (to separate repository files from stuff downloaded at runtime).
Upgrades to the cc wrapper and the USE_UNSTABLE infrastructure.  Better
stripping of the cross compilers and system images, so everything's a bit
smaller.  The mirror list now lives in download.sh.  New buildall.sh is a
wrapper around build.sh.  New build-static-toolchains.sh builds them under
qemu; set FORK=1 to build them in paralell just like buildall.sh.  The
ext2 system images are now built at a small size with genext2fs and
then expanded with resize2fs (because genext2fs slows down exponentially
as size increases; this means e2fsprogs is built in host-tools.sh now).
Distcc has been upgraded, and should work more reliably now.</p>

<p>Too many little bug fixes to list.</p>

<h2>December 16, 2008</h2>
<p>The mailing list moved to firmware at impactlinux.com.  The link in
the nav bar on the left has changed, but the
<a href=http://www2.them.com:8080/cgi-bin/mailman/listinfo/firmware>old
archives</a> are still there for the moment.</p>

<h2>December 3, 2008</h2>
<p>After several false starts (I.E. last minute bugs, yes I test this thing
before shipping it),
<a href=downloads/firmware-0.9.5.tar.bz2>Version 0.9.5</a> is out, inching
towards a 1.0 release.  Lots of new work this time.</p>

<p>The biggest news is uClibc 0.9.30 and uClibc++ 0.2.2.  The native toolchain
now supports C++, hooked up to distcc and everything.</p>

<p>The <a href=documentation.html>documentation</a> has been extensively
rewritten.  (There's always more to do, but most of it isn't horribly out
of date anymore.)</p>

<p>The packaging stage now uses genext2fs instead of User Mode Linux, so
the most brittle part of the build has been removed.  FWL can now fully
rebuild itself under itself, all the way through the packaging stage.</p>

<p>My personal website's getting a touch overwhelmed, so a higher bandwidth
site is now available at
<a href=http://impactlinux.com/fwl>impactlinux.com</a>.  This should become
the new primary website going forward.</p>

<p>BusyBox has also been upgraded (to 1.13.0), and the new USE_UNSTABLE
infrastructure allows test versions of any package to be downloaded and built
without forking the build scripts, so future package upgrades can be dealt
with as side projects without blocking other important work.</p>

<p>The packaging stage now uses genext2fs instead of User Mode Linux, so
the most brittle part of the build has been removed.  FWL can now fully
rebuild itself under itself, all the way through the packaging stage.</p>

<p>The old forkbomb script has been replaced with sources/build-all-targets.sh.
with an argument --fork to build all targets in parallel, and "--fork 3"
to limit the parallelism.  This script can run from a cron job, producing
a "buildall" directory with tarballs, compressed logs, and a README file
listing the version of each package.  The environment variable USE_STATIC_HOST
specifies an $ARCH to create statically linked cross compilers
for.  (The build will then try to use them for the later stages,
and won't get far if you specify an arch that can't run on the host.)</p>

<p>The include.sh script has been broken up, with a new sources/functions.sh
containing shell functions (like download, setupfor, and cleanup) and
a new "config" setting the user-editable environment variables that control
the build.  Several of these variables have been renamed and their number has
grown.  You can now specify the colors for each build stage using the variables
at the end of config.</p>

<p>Building mini-native in a Linux From Scratch style /tools directory is
no longer the default.  Instead set NATIVE_TOOLSDIR=1 if you want that
kind of layout instead of a more conventional layout.  (Similarly,
NATIVE_TOOLCHAIN=1 now enables the native toolchain, but that variable
defaults to enabled in config.)</p>

<p>See the <a href=documentation.html>docs</a> for details.</p>

<h2>August 6, 2008</h2>
<p><a href=downloads/firmware-0.9.0.tar.bz2>Version 0.9.0</a> is out
(<a href=http://landley.net/hg/firmware/shortlog/378>changeset 378</a>)
and can rebuild itself under itself.  (The packaging step still requires User
Mode Linux to create ext2 images, which only works on x86 and x86-64 hosts.
The next release should replace that with something more portable.)
The root filesystem is now based on BusyBox 1.11.1 (plus a few bug fixes).</p>

<p>The prebuilt binary images have been moved into their own
<a href=downloads/binaries>binaries</a> directory.
The prebuilt binary <a href=downloads/binaries/cross-compiler>cross
compilers</a> are statically linked against uClibc for better portability.
Updated the <a href=downloads/README>README</a> to better describe the
contents of the <a href=downloads/binaries/system-image>system-image
tarballs</a>.</p>

<p>The forkbomb.sh script can now take a number of parallel targets to
launch, for example "./forkbomb.sh --fork 3".  (The resulting display is kind
of horrible but the out-$ARCH.txt files are still legible.)  Leaving off the
number still builds all targets in parallel, with the original display.</p>

<p>The include.sh script was reorganized so that interesting environment
variables the user might want to export for themselves are set near the top
of the script.  Currently this includes:</p>

<ul>
<li><p><b>BUILD_SHORT</b> - Makes the mini-native.sh stage skip building
development tools.  The resulting mini-native root filesystem contains uClibc
and busybox, but nothing else.  (Export "BUILD_SHORT=headers" to put back the
toolchain headers for use by another compiler, such as tinycc.)</p>

<li><p><b>BUILD_STATIC</b> - Tell cross-compiler.sh to create static
binaries, for a more portable cross compiler toolchain.  (If you want them
linked against uClibc, build a mini-native for your host, chroot into it, and
then rebuild under that.)</li>

<li><p><b>PREFERRED_MIRROR</b> - Tell download.sh to fetch packages from a
non-default location (falling back to the standard mirror list if it can't get
one from there).</p></li>

<li><p><b>RECORD_COMMANDS</b> - Record a list of all command lines used during
each build stage log files named "cmdlines.$STAGE.$PACKAGE" under the build
directory.  Each file can be turned into a list of the command names used via
"awk '{print $1}' build/cmdlines* | sort -u".  The script
sources/toys/report_recorded_commands.sh gives a report of all commands
used by each package after a call to "RECORD_COMMANDS=1 ./forkbomb.sh".
(This is an easy way to find out exactly what environmental dependencies a
package needs to build, or at least what it uses out of $PATH).</p></li>
</ul>

<p>Updated documentation is in progress...</p>

<h2>June 6, 2008</h2>
<p><a href=downloads/firmware-0.4.0.tar.bz2>Version 0.4.0</a> is out
(<a href=http://landley.net/hg/firmware/shortlog/345>changeset 345</a>),
with kernel 2.6.25.4 and the "distcc trick" working out of the box.</p>

<p>The distcc trick accelerates a native build by calling out to the
cross compiler.  To use it, download the appropriate cross-compiler-$ARCH and
system-image-$ARCH tarballs, run the "run-with-distcc.sh" from the system image
directory with the path to the cross compiler directory as its first argument.
(You need to have distcc installed on the host system.)</p>

<p>After building from source, the cross-compiler and system-image directories
remain in the build directory.  The "emulator-build.sh" runs these with
distcc acceration set up.  The script "smoketest.sh" is a wrapper
around emulator-build.sh which feeds a script into qemu's stdin to compile
and run a multi-threaded "hello world" under the emulator, using distcc and
the cross compiler.</p>

<p>If smoketest.sh works for a platform, it means a lot of stuff worked to get
that far.  The cross compiler worked to build the system, and the emulated
system booted so QEMU, the uClibc config, and the kernel config agreed.
The emulated system has a working virtual hard drive and serial port, and for
distcc to work it has a working virtual network connection.  Running "make"
would also require a working realtime clock.</p>

<p>The seven commands missing in order to run the FWL build under itself
are bzip2, sort, diff, wget, install, od, and find.  (The busybox versions,
where available, had a bug, and they're not in toybox yet.)  I'm trying to
get those in for the next release in September.</p>

<p>Sorry I missed a release back in April.  Shouldn't happen again.</p>

<h2>January 29, 2008</h2>
<p><a href=downloads/firmware-0.3.1.tar.bz2>Version 0.3.1</a> is out
(<a href=http://landley.net/hg/firmware/shortlog/275>changeset 275</a>), with
kernel 2.6.24.  The <a href=downloads/images>images</a>
are now tarballs each containing the ext2, zImage, and run script files.
The run scripts now run qemu-setup.sh by default so /proc, /sys, /dev
and the virtual network are initialized, and the default shell is busybox
ash (so cursor keys and history work).</p>

<p>Sparc is still broken (tracked it down to a dynamic linking bug, I think,
haven't fixed it yet) and m68k is failing to build with an internal compiler
error (next release I should upgrade the compiler).  All the other targets
have a working virtual network, and all but powerpc exit qemu when you
exit the PID 1 command shell.</p>

<p>Building on an x86-64 host< should no longer be insanely slow (the
gcc-min-heapsize arguments that allow 32-bit hosts to build in
128 megs of ram do very impolite things to 64-bit hosts), and the resulting
x86-64 cross compiler should now be properly relocatable.</p>

<p>Better build dependency tracking: the host-tools script is now
populating a single directory with all the commands needed by the build, and
restricting $PATH to that directory (so it can't accidentally call any
commands that aren't explicitly listed).  It's building toybox and busybox to
provide most of these commands (the remaining ones are listed towards the end
of include.sh), and creating symlinks to the remaining ones out
of the original $PATH.  This allows it to build on a wider range of host
distributions.</p>

<h2>November 2, 2007</h2>
<p><a href=downloads/firmware-0.3.0.tar.bz2>Version 0.3.0</a> is out
(changeset 242), with kernel 2.6.23.  PowerPC finally works under
qemu (network and everything), and it's building a m68k target
(although I have no emulator to run that).  That means there are
ten targets, of which eight run under qemu.  (Sparc is still
broken, let me know if anybody actually cares.)</p>

<p>The new "./forkbomb.sh --fork" script builds all the targets in
parallel, or use --nofork if you haven't got the memory for that
(and --watch to watch and existing build if you ctrl-c out of
the display before it's done).</p>

<p>The build is now smarter about extracting only one copy of the
source code (in build/sources) and re-using it for multiple builds,
building out of tree (via cp -rs to create a tree full of symlinks
so the package doesn't need native out-of-tree support).</p>

<h2>September 2, 2007</h2>
<p>Added <a href=new_platform.html>a porting guide</a>.  I'm working on
m68k, ppc, big endian arm, and maybe blackfin targets for the next release,
and thought other people might be interested in the process.  (The vast
majority of it is fiddly debugging.  These are the other steps.)</p>

<h2>June 27, 2007</h2>
<p><a href=downloads/firmware-0.2.2.tar.bz2>Version 0.2.2</a> is out
(changeset 185).  Using uClibc 0.9.29 release, several small bugfixes,
new mips big-endian config, actually work when /bin/sh->dash (mostly
by explicitly saying /bin/bash everywhere), automatically detect
the number of CPUs in the machine to feed -j to make, and better
diagnostic output.</p>

<h2>April 9, 2007</h2>
<p><a href=downloads/firmware-0.2.1.tar.bz2>Version 0.2.1</a> is out
(changeset 156).  Significant upgrades to armv4l: the native compiler works
now, and it's using soft-float.  Added armv5l and i586 targets to show how
to do that kind of variant.  Added preliminary powerpc support (although
qemu can't run it yet).  Reorganized the download directory and rewrote
the <a href=downloads/README>README</a>.  Added squashfs back (the kernel
supports it, but it's not using it yet).  Moved uClibc to miniconfig.
(Yes, the patch for this is huge, and yes I need to push it all upstream into
the Linux kernel.  On the bright side, this one builds on a host that doesn't
have curses installed.)  Stopped trying to build qemu (at least until they
get the gcc 3.x dependencies cleaned out), now just warn if it's not there
(so far the build only uses it to sanity-test the cross compiler, which is
now skipped automatically if it's not there).</p>

<p>I note that some versions of Ubuntu (Edgy and up) are pushing a political
agenda.  They install bash, but point /bin/sh to dash instead.  (So why
install bash if you're not going to use it?)  This breaks all sorts of things,
and if somebody can tell me how to force make to run "/bin/bash" instead of
"/bin/sh", I'll consider trying to deal with the mess.  Until then, I suggest
"rm /bin/sh; ln -s bash /bin/sh" to get a working system.  (I'm no fan of
bash, but dash is just broken.)</p>

<p>(Those of you who use vi and want to be able to use the cursor keys while
in insert mode might want to do something similar with vi->vim.  Ubuntu seems
to be trying to train users to change their habits.  Dog biscuits would be a
more honest approach.)</p>

<p>Update: I've been pointed at
<a href=https://wiki.ubuntu.com/DashAsBinSh>this</a> as evidence Ubuntu isn't
pushing a political agenda.  Ok, maybe it's just really bad technical judgement.
Are they honestly saying that people write shell scripts for the speed?  And
how do you make a size argument and then install bash by default in ADDITION
to the other shell?  Boggle.  In 1991 after Linus Torvalds taught his term
program to understand the minix filesystem so he could upload and download
without rebooting, he taught it to handle the system calls of bash so he could
rm/mv/mkdir without rebooting.  That's where Linux 0.0.1
came from.  Bash was the default Linux shell _before_ 0.0.1.  Be very
careful messing with that kind of a de-facto standard, guys.  Dash is
garbage.  Background a process with &amp; and then hit ctrl-c: it'll kill it.
It doesn't understand blah/{blah,blah} file list syntax.  It can't do
"source file.sh" (just ". file.sh", which is unreadable).  I repeat: I'm not a
fan of bash (yeah, the bloat's getting crazy), but dash is broken.</p>

<p>And that still doesn't explain why "vi" can't use the cursor keys in
insert mode, but "vim" can...</p>

<h2>March 6, 2007</h2>
<p>Version 0.2.0 is out.  This one builds ext2 images which can boot up to a
shell prompt under qemu.  This works on i686, armv4l, x86_64, and mipsel.
(Sparc's broken.  Anyone who cares about sparc is welcome to debug it.)</p>

<p>There are now prebuilt ext2 image files in the download directory.
The kernels are in the mini-native tarballs.  (Yeah, I should repackage that
next release.)  The system emulator invocation for each platform is
the appropriate sources/configs file, and run-mini-native.sh invokes 'em.</p>

<p>The mailing list is in the usual place if you have any questions.</p>

<h2>January 14, 2007</h2>
<p>There's a new irc channel for the project, #firmware on freenode.</p>

<p><a href=http://landley.net/hg/firmware?cl=88>Changeset 88</a> builds
i686, x86_64, armv4l, mips, and sparc.  I've made a release tarball of
that (<a href=downloads/firmware-0.1.1.tar.bz2>firmware 0.1.1</a>, "It works
for me"), and updated the prebuilt cross-compiler tarballs in the
<a href=downloads>downloads</a> directory.  I've added a big README and
prebuilt mini-native tarballs for each platform (although packing them up into
something qemu can boot is currently left as an exercise for the reader).</p>

<p>Speaking of which, I'm currently working on adding ext2 packaging
(via <a href=http://landley.net/code/toybox>toybox</a>) to the build scripts,
so qemu system emulation can boot the result and then run the next stage
automatically.</p>

<h2>December 28, 2006</h2>
<p><a href=http://landley.net/hg/firmware?cl=68>Changeset 68</a> builds
a native build environment with a working toolchain.  And in celebration, I've
revamped the website with an actual navigation bar and content and such.</p>

<p>I should put out a release soon.</p>

<h2>December 5, 2006</h2>
<p><a href=http://landley.net/hg/firmware?cl=27>Changeset 27</a> builds
a relocatable armv4l cross-compile toolchain!  Download the tarball,
run ./download.sh, then run ./build.sh, then grab the "build/cross-compiler"
directory and use "bin/armv4l-unknown-linux-gnu-gcc" out of that (which
I admit is a bit of a mouthful, for which I blame the FSF).</p>

<h2>August 6, 2006</h2>
<p>Mecurial repository created.  Nothing to see yet, move along...</p>
<!--#include file="footer.html" -->