From 122bbe602f50b7fe747751370035f6fd55e674d0 Mon Sep 17 00:00:00 2001 From: Rob Landley Date: Mon, 8 Apr 2024 20:29:53 -0500 Subject: [PATCH] Version 0.8.11 --- toys.h | 2 +- www/header.html | 2 +- www/news.html | 298 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 300 insertions(+), 2 deletions(-) diff --git a/toys.h b/toys.h index 88b7a837..94d0c4b5 100644 --- a/toys.h +++ b/toys.h @@ -141,5 +141,5 @@ extern char **environ, *toybox_version, toybuf[4096], libbuf[4096]; #ifndef TOYBOX_VENDOR #define TOYBOX_VENDOR "" #endif -#define TOYBOX_VERSION "0.8.10"TOYBOX_VENDOR +#define TOYBOX_VERSION "0.8.11"TOYBOX_VENDOR #endif diff --git a/www/header.html b/www/header.html index e72481f4..ec0d2969 100644 --- a/www/header.html +++ b/www/header.html @@ -55,5 +55,5 @@ -

Current release 0.8.10 (July 30, 2023)

+

Current release 0.8.11 (April 8, 2024)

diff --git a/www/news.html b/www/news.html index f983d6f3..8054c635 100644 --- a/www/news.html +++ b/www/news.html @@ -37,6 +37,304 @@ bootable under QEMU (built using a Build: --> +

April 8, 2024

 +
+

Another thing that got forgotten was the fact that against all probability a +sperm whale had suddenly been called into existence several miles above the +surface of an alien planet. And since this is not a naturally tenable position +for a whale, this poor innocent creature had very little time to come to terms +with its identity as a whale before it then had to come to terms with not being +a whale any more.

+

- The Hitchhiker's Guide to the Galaxy

+
+ +

Toybox 0.8.11 +(git commit) +is out, with prebuilt static binaries and +mkroot images +bootable under QEMU (built using a lightly patched linux-6.8).

+ +

Highlights include a new quickstart +web page, shorter /bin links to prebuilt binaries, +lots of work on mkroot, and a new "canned" toybox build to reduce +dependencies.

+ +

New commands: +Rob added tsort, and promoted fold and getopt. +Oliver Webb added ts and csplit, and +Elliott added memeater.

+ +

And one command got temporarily de-promoted: passwd is "default n" +(disabled in defconfig) this release due to the ongoing lib/passwd.c +infrastructure rewrite that isn't quite done yet. (See the Library section.)

+ +Features: +The new "canned" +toybox build in scripts/prereq/build.sh lets toybox provide its own +build prerequisites by compiling a minimal toybox against saved headers (ala +"cc -I scripts/prereq/generated file1.c file2.c..."). This provides +most of the commands toybox needs to configure and build itself (except make, +bash, and the compiler toolchain). This may help bootstrap toybox on systems +that don't provide a modern Linux command line out of the box: install +toybox-prereq at the start of the $PATH, add +a .config file, and run scripts/make.sh.

+ +

New mv -x option to atomically swap two files, +env -e FILE executes a different command than argv[0] on the command +line, the reset command now puts raw terminals back into "cooked" mode, +negative offsets in sort -k count right to left, +setsid grabs the tty more forcefully (removing the need for getty) +and the -cc option will try to steal it, +mount now shows file= when displaying loopback mounts, +wc -L shows longest line length, +tar -h now detects a lot more hardlinks, +cp now copies xattrs for directories (not just files).

+ +

Elliott taught file to recognize wasm binary modules and +display JPEG width/height, added the LOWER_UP, DORMANT, and ECHO +flags to ifconfig, added PAC and BTI decoding to readelf, +cache size querying to sysconf (supported by glibc and bionic but +not musl-libc), and added a "paste" menu to microcom. Kalesh Singh +taught readelf to decode a new note type (NT_ANDROID_TYPE_PAD_SEGMENT). +Christopher Ferris improved readelf section flags handling, +with test. Oliver Webb suggested count -l. +Aditya submitted netcat -z.

+ +Bugfixes: +Fixed printf \0 and grep -w (which never quite worked right), +ls -k now switches off --block-size, +install -dm 02750 works (sticky bit support for -d), +tar is better at following symlinks and extracting records with broken +metadata, gzip now knows +knows how to rename between "tgz/tar" extensions, +mkpasswd -m sha* salt length can now range from 8-16, not just 16, +ps can now handle an session ID of 0 (which is what PID 1 starts with +before the first call to setsid()), +pwgen can now use ~ in output, +xargs now stops when a child exits with 255, +iconv now exits with error if any characters failed to convert with -c, +things like truncate -s 1e2 were saying "too large" and now correctly +say "not integer" (it's not scientific notation, it's exabyte suffix with +trailing garbage), +timeout now takes its exit code from SIGCHLD instead of wait() +to fix a race condition (SIGCHLD can happen before calling wait and the +kernel won't deliver status twice, so you can't wait AND have the +signal handler). The usual round of gcc warnings got tracked down, confirmed +to be useless, and removed, but despite gcc constantly warning "variable is +not actually used uninitialized" there were some uninitialized variables in +dd it DIDN'T warn about: found and fixed.

+ +

Elliott added an error message to catch xxd -r receiving -p format +input without -p, +fixed readelf -n for x86-64 ibt/shstk notes +(whatever those +are), +and fixed blkid not to check filesystem signatures that would continue +past the end of the loaded buffer. +Michael Shavit pointed out that devmem command line number parsing +wasn't unsigned (although fixing that means it no longer parses the kmg unit +suffixes). +Peter Collingbourne fixed inotifyd skipping the first mask character. +Taisuke Matsushiro fixed a fencepost error preventing stty from +setting serial baud rate to 57600. +Askar Safin fixed a typo in mkroot/packages/busybox.

+ +Library: +The in-progress rewrite of lib/password.c no longer uses shadow.h +(which lets it build against bionic), +and a new lib/hash.c where toybox implements its own crypt() using +the md5/sha1/sha3 etc hash functions we've already got (to work around glibc's +ongoing deprecation of posix features), but neither are quite finished yet. +New lib/elf.c with plumbing shared by file and readelf, with +additional arm and riscv flag decoding, +the peek() family of functions now return long long instead of +int64_t so the base types are consistent on 32/64 bit, +lib/args.c now handles the high bit octal escape sequences in trailing +[-abc] blocks, so you can have punctuation participate in option groups, +switching off an option via grouping now sets it back to its default +value (not to zero), and numeric and date parsing report out of range numeric +or millisecond values.

+ +Mkroot: +There's a README in the mkroot directory now. (Microsoft Github was +picking up the README.root file used by the tar-for-web.sh script +and showing that instead.) +Extended run-qemu.sh so it can be run from other directories +(prepending $(dirname $0)/ to linux-kernel and initramfs.cpio.gz when +loading them into qemu). +The initramfs now has /etc/os-release with the toybox $VERSION. +The big if/else staircase of linux kernel target configurations +moved into its own get_target_config() shell function, with +a big comment block right before it explaining what all the variables +it sets mean, and the generic kconfig symbol list got moved out to +a GENERIC_KCONF variable assignment right after that function. +The microconfig format got compressed slightly more +by use of bash brace expansion, decoded to comma separated values by +the be2csv function ala KCONF="$(be2csv SPI{,_BITBANG} IOMMU_SUPPORT)" . +The init script no longer calls oneit, instead doing the work in the +init script. (Redirect stdin/out/err to the "real" console device on +the last line of /sys/class/tty/console/active, call setsid, trap SIGCHLD, +reboot when the shell exits. This (and the setsid tty stealing upgrades) also +fixed ps not showing any processes in mkroot because stdin/stdout weren't +properly claimed as the controlling tty. +The base kernel config now uses CONFIG_PANIC_TIMEOUT=1 so +we don't have to say panic=1 on the kernel command line anymore. +The baseconfig now automatically adds CONFIG_MODULES and +CONFIG_MODULES_UNLOAD when any $MODULES are listed. +Moved the kernel modules archive into the "docs" directory. +New $NOTOYBOX variable skips the toybox build if set. +The $DTB files no longer have a path, instead using "find" +to install them. +The build now creates the externally loadable cpio.gz file even when +building initramfs into the kernel, but sticks it in the docs/ subdirectory +when it's not used.

+ +

Several new mcm-buildall.sh targets (or1k, riscv32, riscv64, sh4eb) +taking advantage of upstream musl-cross-make updating itself to use +musl-1.2.5. The target list in mcm-buildall.sh moved into a TARGETS=() +array with a big comment before it explaining the format, and the script +is better about building host toolchains +to build other cross compilers with: it tries to autodetect the host +type (moving the relevant one to the front of the build list), and +will also build a missing host toolchain when given targets on the +command line. Added microblaze and or1k kernel configs, neither of which +know how to exit qemu yet. (If Linux has got a reboot mechanism it knows +how to poke that qemu is listening for on either virtual board, I haven't +found it yet.) Also added sh4eb with FDPIC support (see kernel patch, which +allows qemu-system-sh4eb to run the sh2eb nommu root filesystem), and armv4l +(the 486 of arm).

+ +Pending: +Elliott fixed and improved strace, translating more system calls. +Rob enabled command recursion in toysh (so it can call non-builtins as +builtins), fixed calling "exec" on NOFORK commands (which MUST run +in the shell's process context), redid the code to handle trailing +backslashes gluing lines together, and it no longer leaks script +filehandles into child processes. +Oliver Webb noticed that OLDTOY(MAYFORK) aliases didn't become shell +builtins, added tr -t, in vi added line gotos in ex mode, +'g', 'v', 'j' commands, and backwards search, cleaned up stuff in +xzcat, getopt, bc, and removed a bunch of useless autogenerated +bc tests. Jarno Mäkipää added the list of supported commands to vi's +help text. +Mathieu Anquetin fixed ip link set parsing, and NUL byte handling +in hexdump. +General cleanup passes on expr (which now uses 64 bit comparisons), +tr, and dhcpd.

+ +Cleanup: +pass over lsattr, getconf, acpi, and blkid. +Use FLAG() macros in ifconfig and restorecon. Removed TODO block +from nbd-client. +More error_msg("%s", str) converted to error_msg_raw(str). +Trimmed the GLOBALS() sizes (using scripts/probes/GLOBALS): +removed 1k of global data from cksum and switched a 2k global +in grep to a malloc. Simplified scripts/probes/bloatcheck and +mkroot/record-commands.

+ +Portability: +Fixed truncate -s 8g on 32 bit hosts, +updated mcm-buildall.sh to autodetect host type and automatically +select newer package versions, +and moved the linux32 command to its own file because BSD can't +build it. +MacOS had a use-after-free with the getpwuid() return value being recycled +a more aggressively than Linux. +Elliott fixed diff.test to work with mksh, and posix_fallocate() +to work with MacOS. +More commands enabled in make bsd_defconfig and make +macos_defconfig. +Added some (void) typecasts to netstat, oneit, and watchdog to try to +shut up compilers that insist on responding to uncorrectable failures. +(Such as fgets() reading known /proc data from the kernel, writes to +the watchdog timer that will reboot the system if they fail, or PID 1 calling +dup() on stdin to create stdout and stderr after closing the old ones. +If any of that goes wrong, what are we supposed to do about it?)

+ +Documentation: +The website now has a quickstart page, +and the site now defaults to the about page (updated index.html symlink). +The "current release" is now in the common header displayed by most pages. +The mkroot dir has a +README, and its faq entry +has been updated. +New option to compress help text with gzip, and help text size now shows +up in make bloatcheck. +Fix sed --help to show full help, patch -F is now mentioned +in the help text, tweaks to help text in cut, grep, +blkdiscard. +Elliott did a cleanup pass over "usage:" lines and alphabetized more help text +options. +Christian Koestlin fixed a typo in the dmesg docs. +The release +procedure checklist has been updated, and the README generated +by mkroot/tar-for-web.sh got updated. +New faq entry explaining why toybox doesn't have (or need) cttyhack, +and the architecture list got updated to match mcm-buildall.sh. +The usual roadmap updates.

+ +

New scripts/probes/cmd2dpkg script shows what +debian package (if any) each toybox command lives in, but only checks +installed packages (dpkg-query -S "$(which $COMMAND)") and debian breaks +up source packages strangely, so did not used it to redo the roadmap section +listing packages (created using Linux From Scratch's list of commands +installed by each source package).

+ +Plumbing: +The yes command now uses writev() to go way faster, mostly to +prove it can. +The default stdout buffer type switched to block buffering to make +Elliott happy, which is a rich source of bugs (many of which Elliott +has already whack-a-moled) and means among other things +that mixing write() and fwrite() calls now requires manual flushing. +Meanwhile xflush() became xferror() because it usually +wasn't flushing anyway, and now that's more noticeable. +Removed TOYFLAG_LOCALE and just set utf8 locale for all commands. +The dirtree code now sets DIRTREE_SYMFOLLOW in "again" when we followed +a symlink to get to this node, callers checking DIRTREE_COMEAGAIN need +to mask out the bit they're interested in. +Improved scripts/probes/findglobals output to be more legible.

+ +Test suite: +Added make test_help test_install targets. (The standalone +command targets conflict with the help/install make targets, but you +can still build them via scripts/single.sh and there's no reason not +to be able to individually test them.) +New utf8locale shell function attempts to enable a UTF8 locale for +TEST_HOST, which was used to fix wc.test. +Colin Cross fixed pidof.test when stdout is a socket. +Something called "ecryptfs" has a maximum filename length of 143 bytes, +so our tar.test for long filenames support needs to be skipped there +because the filesystem can't handle it. Similarly, our truncate.test +was failing because storing nothing on ecryptfs wasts more space than expected. +The tar tests now fetch user/group names with "stat" because +between Linux, Android, FreeBSD, and MacOS, there aren't really consistent +user and group names for any existing files. Added nbd-client.test +Thiébaud Weksteen fixed the getfattr/setfattr tests on +filesystems using selinux. Oliver Webb added sha3sum tests. +Elliott stabilized another slightly racy ifconfig test (big test +farm, hits the weird corner cases).

+ +Build: +Lots of changes to scripts/make.sh and friends to speed up and parallelize +header generation. The build now figures out which toys/*/*.c files +to compile by searching for unindented USE() macros wrapping NEW(commandname), +which among other things fixes enabling the builtins when selecting just +CONFIG_SH in menuconfig (instead of doing a "make sh" single command build). +It also probes for optional shared libraries in parallel. +New scripts/probes directory to collect scripts that aren't actually +used by the build, with a README in it. +Updated .gitignore so it doesn't complain about spurious name collisions in +subdirectories. Elliott shut up a +truly sad +warning where despite sizeof(ptrdiff_t) always being sizeof(long), gcc +nevertheless warns if you printf("%ld", ptr-ptr) and wants a magic invented +"%td" type on 32 bit systems (but is just fine on 64 bit). (Or we could add a +literally NOP typecast to long.) And silenced a spurious gcc 13.2 warning +in date.c. And Rob shut up more broken gcc warnings. +Updated .gitignore so it doesn't complain about spurious name collisions in +subdirectories.

+

July 30, 2023

The way it functioned was very interesting. When the Drink button was -- 2.39.2