From cbc3e2c9889eab9e99127570cbbfa1d5f350d52d Mon Sep 17 00:00:00 2001 From: Rob Landley Date: Sun, 30 Jul 2023 02:32:28 -0500 Subject: [PATCH] Release notes for 0.8.10 --- www/news.html | 267 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 266 insertions(+), 1 deletion(-) diff --git a/www/news.html b/www/news.html index c109c7b4..8d82c879 100644 --- a/www/news.html +++ b/www/news.html @@ -8,6 +8,271 @@ a development environment. See the links on the left for details.

News

+ + +

July 30, 2023

 +
+

The way it functioned was very interesting. When the Drink button was +pressed it made an instant but highly detailed examination of the subject's +taste buds, a spectroscopic analysis of the subject's metabolism and then +sent tiny experimental signals down the neural pathways to the taste centers +of the subject's brain to see what was likely to go down well. However, no +one knew quite why it did this because it invariably delivered a cupful of +liquid that was almost, but not quite, entirely unlike tea.

+

- The Hitchhiker's Guide to the Galaxy

+
+ +

Toybox 0.8.10 +(git commit) +is out, with prebuilt static binaries and +mkroot images +bootable under QEMU (built using a lightly patched linux-6.4).

+ +

New command shuf, and Elliott added i2ctransfer. +Finished cleaning up dd a promoted it out of pending.

+ +

Lots of changes to mkroot, working +on updated documentation but the release is already overdue...

+ +

Features: +Added a bunch of options to tar (-s --ignore-case --wildcards +--anchored --wildcards-match-slash --one-file-system), ls --sort +(which takes a comma separated list of fallback sorts, and new +sort types to sort by extension, case insensitive, and directory first), +cpio -R to set user/group of all files and cpio -L to +follow symlinks, gzip -t to test file integrity, find -readable, +cut -DFR now has --longopts aliases (as suggested by the coreutils list). +Taught blkid to recognize iso (cdrom) image files, oneit now +autodetects console from sysfs with no -c argument, added "next address" +support to i2cget and enforced bus limit, +added posix 1x2x3 support to dd (with overflow detection, and 0x +at the start still means hexadecimal).

+ +

Jarno Mäkipää taught i2cget to read without passing a command byte. +Elliott added xxd -e (little endian), lspci -x (hex dump of +pci config space), ls --block-size, widened the default +VSIZE field width in Android's default ps display (they're up to 10 gigabytes of +ASLR padding for hello world), and increased the vi/top +output buffer size to reduce flicker redrawing the screen.

+ +

Bugfixes: +Taught switch_root to move mountpoints into the new root. +The head command now uses lseek to restore unused data from seekable +inputs (tested with read i; echo =$i; head -n 1; done < input). +In lspci handle PCI bridges and don't show "revision 0". +Missing /proc entries no longer cause vmstat errors, the output +fits into 80 columns more often, and adds reclaimable to the cached field. +Fix help -u -au, and the "see othercommand" generation. +Adding fuzz support subtly broke patch in a couple places: fixed -R and +matching EOF. Trying to create a directory that already exists is no longer an +error for cpio -i, and -d just warns about directory creation +failure instead of exiting. Adding --wildcards logic to tar +broke the full string +matching (it checked after every / but not at the start of the string +when there weren't any), and tar now waits for the compression program +to exit and passes on any error result instead of killing it (which could +truncate output). Fixed more realpath/readlink corner cases, with tests. +Fix a sed bug when s//&/ (the copy-matched-text operator) was the +first change made to the string. Fix error message for find -commands +that need an argument but were at the end of the command line (it said +"unknown command" instead of "command needs argument"). Fixed killall5 +parsing of command names with spaces in them. Fixed grep -m -A used +together (-m stops new matches but -A still shows lines after match). +Fixed mv to --preserve=all when copying between filesystems. +Moved the EPIPE skip from perror_exit() to verror_msg() so we never +output a "hey, our output pipe unexpected closed!" error message when +we're not the last command in a pipeline.

+ +

Franke Busse reported that bunzip2 was deleting its input file +on error, and date -s +abc was segfaulting. +Jooyung Han fixed nsenter using -C and -t together. +More fixes from Eric Roshan-Eisner's toysh fuzzing, and he also sent a fix +for an error message in the patch command. +Elliott fixed stdin handling in log and logger to read a line at +a time and log each one separately, fixed an off by one bounds check +in readelf (and factored out the bounds checking logic so it's +common between sections, and improved the error messages), fixed +a gpiod crash.

+ +

Library: +Implemented DIRTREE_BREADTH to do breadth first processing of directories +(among other things, allowing entries to be sorted and processed in a +consistent order). The dirtree plumbing now has macros for the mode bits +instead of hardwired numbers, +better adler32 calculation in deflate.c, +and fs_type_name() can now convert most of the linux/magic.h constants +into "mount -t TYPE" names.

+ +

Mkroot: +Moved mkroot into its own directory: you now run mkroot/mkroot.sh instead +of scripts/mkroot.sh, and all the mkroot plumbing should be under there now. +What was scripts/root is now mkroot/packages, and I added a "busybox" +package to download and build that (useful for testing command +implementations side by side).

+ +

Reorganized the contents of each root/$ARCH build directory so only +the files necessary to run are at the top level, and everything else +(kernel configs) is in a docs/ subdirectory. Added linux-microconfig +with the three CSV lines actually used by the build. +New mkroot/tar-for-web.sh script that generates tarballs for +uploading to the web page and adds two README files (one at the top level +and one to each tarball's docs directory). The generated tarballs don't +contain the fs/ dir, instead the README explains how to extract +and recreate the cpio.gz archive.

+ +

Taught KCONF=value,value,value... CSV format to understand NAME=VALUE +entries, and merged most KERNEL_CONFIG lines into KCONF. +(Note that while it can handle NAME="this that" you need to escape it as +NAME=\"this\ that\" to work right.) Added the legacy NLS +dependencies the VFAT driver needs (even though we selected UTF8) to the +base config. The init script now tests if stdin is open via +"2>/dev/null <0 || blah" so only does the exec redirect for /dev/console +when necessary. +Fixed the .config checker to replace the toybox config when CONFIG_SH=y +isn't set, so "make defconfig; mkroot/mkroot.sh" should work now. +Switched initramfs packaging to use more portable cpio -R 0:0 +instead of --no-preserve-owner. Moved the -m 256 before $@ in run-qemu.sh +so setting memory size on the command line overrides the default. +The mkroot/record-commands setup script will now use logpath +out of the $PATH if it exists, rather than having to compile it each time. +Bo Svangård fixed a typo in the cpio command line that's ignored by toybox but +breaks with the host tools.

+ +Pending: + +

Moritz Weber put in more work on git (still very unfinished). +Eli Lipsitz fixed bugs in init and login. Elliott added risc-v support +to ptrace, and fixed :w in vi (it didn't +takes a filename argument and would crash if you started with no filename) +and improved error reporting. Jarno Mäkipää taught vi about backspace +and improved empty file handling. Rob fixed vi -s. Various cleanups to +tcpsvd.

+ +

More toysh work: command line option parsing stops at first nonoption argument, +all piped segments are now implicitly subshelled so variable +lifetimes match bash's and loops don't have to exit before proceedinng to the +next one (ala "while true; do echo hello; done | while read i; do echo $i; done" +hanging on a full output pipe). Several changes to HERE document processing +(variable expansions can cause line continuations in a HERE document, etc), +and better line continuation handling in general, added set -u, +fixed if/while return codes, wired up the "local" builtin, fixed an escape +passthrough bug reported by Mingliang Hu, support undelimited redirects +(ala cat<file with no spaces), fixed ${X::} with no arguments.

+ +Cleanup: + +

Converted chgrp to FLAG() macros and did some minor cleanup on bootchartd, +(which needs tests), basename, bzcat, test, +i2cdetect. Elliott switched flock +to FLAG() macros, switched nohup to use octal permissions insted of macros, +widened the default VSIZE field width in Android's default ps display +(which is now inserting 10 gigabytes of ASLR padding into "hello world"), and increased +the vi/top output buffer size to reduce flicker redrawing the screen. Multiple cleanups to +cpio. Added requested --gratuitous-longopt synonyms to ls. +Consistently capitalize TODO comments in the tree so they're easier to search for. +Silence yet more perennial "warning: variable is never used uninitialized" gcc false +positives.

+ +Portability: +utf-8 support still isn't automatic in libc (you have to set an appropriate locale to +enable it), and locales aren't entirely standardized: try C.UTF-8 (which MacOS hasn't got) +and fall back to en_US.UTF-8 (which Gentoo hasn't got). +Removed itoa() from seq.c because cygwin pollutes the namespace with +DOS debris.

+ +

Dan Bryant renamed xrunread()'s "stdin" argument to "to_stdin" to work around +broken headers that #define stdin as a macro instead of a global variable. +Elliott enabled copy_file_range() on Android.

+ +Documentation: +

Calling command --help and toybox --help command shows +the toybox URL banner, but "help command" doesn't. +Reorder the find help text so match filters that take an argument are mostly +in the left column. Minor help text tweaks in sleep, find, xxd, cpio. +The README now points to the same binary cross compilers as the FAQ. +Update sed help text now that -e and -f get processed in command +line order when used together.

+ +Plumbing: +The FLAG() macros now always return 1 or 0, so we can CONSTANT*FLAG(X) without needing +a !! in there. The help plumbing now has HELP_FLAGS. +Each command's option string in NEWTOY() may now use octal escapes with the +high bit set to use control characters as option flags, with the resulting +FLAG_X## macro using the 2 digit hex value (without the high bit set). For +example if you needed "command -:" then ":" is ascii 58 (hex 3A, +toybox's ascii command is useful for this), +128 is 186, which in octal +becomes "\272" and tested with if (FLAG(X3A)) in the code.

+ +Test suite: + +

Each command.test now runs in an empty subdirectory, and working files like +expected/actual go elsewhere (only "input" is created for you in there, and only +when you provide input in the current test's 4th argument), so the directory +listing doesn't contain anything the test didn't put there. (This simplified +several tests.) +The realpath test broke on mksh (which doesn't export $PWD by default). +Elliott fixed the ifconfig tests on android. General tidying and adding +tests to sed, ls, modinfo. The tar tests broke on github because it +switched to a filesystem that can't do sparse files with 4k granularity, +added a check to skip those tests (and also skip them on MacOS, which +fails for a different reason.) +When a test's name field was blank (meaning the actual test command line +should be displayed as the name) the test plumbing was adding the command +name to the display twice. +The make tests target now defaults to building toybox with ASAN when +not testing an existing binary. (You still have to specify ASAN=1 manually +for individual command tests.)

+ +

Fixed demo_utf8towc which had the range check backwards (0x10ffff is +the biggest _output_, not input). The range check was an attempt to avoid +spending a long time going through all 4 billion entries, but wasn't testing +everything it needed to, so now it advances the counter with a bit shift +based on the number of bytes consumed by the match (avoiding large swaths +of redundant checks but still theoretically testing all meaningfully different +4 byte inputs).

+ +Build: +Fix return code of verbose mode in scripts/build.sh so make V=1 succeeds. +Speed up the MacOS build: their gsed is slow to launch so avoid one repeated call in a +loop, and reimplement the wait $PID codepath because their bash is 17 years old. +The llvm+bionic toolchain warns about printf(" "+FLAG(m)) unless you add an +extra ,0 argument, but gcc+glibc warns if you DO add the extra argument... so +I used the write() system call to avoid the compiler "helping" in a way I +can't make it STOP. Similarly, now that FLAG(u) is always 1 or 0, +size = FLAG(u) ? : sizeof(toybuf) +is either 1 or 4096, which is what I _want_, and the compiler warning about it +is counterproductive. LLVM needs a new -Wno-invalid-source-encoding flag +or else it complains about octal escapes in string constants. (When I want +to put utf-8 in there I can just do so, why would I octal escape it? +Stop "helping".)

+

January 10, 2023

"Why," Arthur said, "is there a sofa in that field?"
@@ -15,7 +280,7 @@ a development environment. See the links on the left for details.

continuum!"
"And this is his sofa, is it?"

- The Hitchhiker's Guide to the Galaxy

-

+

Toybox 0.8.9 (git commit) -- 2.39.2