Rob's Blog rss feed old livejournal old twitter

2021 2020 2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2002


May 17, 2022

Dear bash: why does a trailing || pull in the next line, but a trailing | does not? (Yes I know what the standard says, but WHY?)

Still ripping the generated/flags.h plumbing a new one. What I'd LIKE to do is turn something like the "mkfifo" command's optstring "<1"USE_MKFIFO_Z("Z:")"m:" into:

#define FLAG_Z ((USE_MKFIFO_Z(1)+FORCED_FLAG)<<1)
#define FLAG_m (1<<0)

And then generated/flags.h doesn't need to be regenerated when .config changes.

I also kind of want it to be generated via bash script instead of C (tricky), and I also want to collapse together the two FORCED_FLAG macros and just have it always be 1LL. Unfortunately, it turns out gcc's optimizer does "long long" math in 64 bit mode and then truncates the result to 32 bits even when it CAN'T MATTER, ala:

#include 
#include 
int main(int argc, char *argv[])
{
  unsigned long ul = 1LL<<atoi(argv[1]);
  printf("%lu\n", ul);
}

Compiling that with 1LL vs 1 before the shift and then diff -u <(objdump -d one) <(objdump -d two) says that it produces different code, even though the assignment to ul will truncate at 32 bits so the left shift might as well be 32 bits, the conversion-to-int can move one step up in the math because shift left can't have the bottom 32 bits influenced by anything in the top 32 bits. Even on 64 bit platforms it's an extra instruction (because then the truncate becomes explicit). I'm guessing on 32 bit platforms it's calling the library functions to do 64 bit FOIL algebra instead of the simple 32 bit math it has CPU instructions for.

Sigh, that's why I had the two macros in the first place (and switched to using the big one only when it was shifted by enough to matter). Hmmm. And with llvm it's an even bigger difference between the two assembly outputs. Sigh, you'd think with all the work they put into the optimizer, gcc or llvm would pick this up, but apparently not...

Right now, mkflags.c takes two different optstr inputs, one for allyesconfig and one for the current config; it resolved the macros with gcc -E but did it twice and then diffs the strings. If I switch to preserving the USE_X() macro inside the FLAG_X(macro) then it only needs the one (unpreprocessed) input, which more or less means rewriting mkflags.c, and if I'm going to rewrite it anyway I might as well do it as shell instead of C. Except C string syntax is kind of nontrivial, ala "abc\"USE_"USE_POTATO("abc")"def" and yes that is an escaped literal quote and USE_ is legitimate optstr payload although I think I can guarantee it'll never actually occur in that order in a string... I mean it COULD show up in a longopt, but I'm willing to bet it won't and fix it up if it does?

Anyway, trying to process this in bash turns out to be kind of unpleasant. I made a sed invocation to give me the command name and option string for each NEWTOY:

sed -ne ':a;s/^USE_[^(]*(\(.*\))$/\1/;ta;/^NEWTOY(/!d;s/NEWTOY(\([^,]*\),/\1/' \
    -e 's/, *[^"]*) *$//;/"/p' toys/*/*.c | while read NAME ARGS; do
  echo NAME="$NAME"
  echo ARGS="$ARGS"
done

But taking the result and emitting flag macros for it... thats easier to do in C than in bash? Which means I'm writing a new C scripts/mkflags.c which is not ideal, but still... could be worse?

Except... The reason I have the low-ascii skip handling in lib/args.c is so when it hasn't got -Z the parser will error on -Z instead of silently accepting but ignoring it. Even if I chop the optstr contents out with macros instead of replacing them with spacers, the flag positions change so the FLAG macros have to be regenerated, which means rewriting generated/flags.h anyway.

Sigh, I WANT to simplify this infrastructure, but it's all got reasons for what it's doing. Needs more design thought.

In the meantime, I started down this path because tar --transform and --xform were synonyms without a short option to collate them (which matters because they take arguments and put them in two different buckets which don't maintain order relative to each other), and I should implement a solution to that in the current plumbing if this isn't a good near-term excuse to redo said plumbing.


May 16, 2022

Gmail is back on its bullshit. So many "bounce action notifications", refusing delivery of a message gets retried by dreamhost until the number of attempts exceeds the unsubscribe threshold. I can't get an actual human at either side to fix the design assumption conflict (can you just discard the one MESSAGE and not unsubscribe the USER), just like I can't get gmail and thunderbird to talk to each other about their conflicting imap design assumptions (both the "All Mail" vs "Inbox" folder problem that screws up threading in mailing lists, and the problem "you can only permanently delete out of the 'Trash" folder but thunderbird implements imap "move mail" as "copy" then "delete" so trying to move messages to the trash before deleting them DOESN'T HELP.)

The common element here is, of course gmail. I asked Elliott but he isn't aware of any humans working at gmail from his perspective either.

I went through and fixed all the mailing list accounts up by hand in the web interface again. Something like 1/4 of the list users are gmail. Nobody else got kicked into "hold:B" state, just gmail, as usual.

And I still need to move OWN email off gmail by the end of the month due to gmail's forced migration to a pay service of my old account type (which I don't have the admin login info for and thus couldn't migrate anyway even if I DID want to pay for it, which I don't). But I still can't start the migration yet because I want this "getting paid by the middleman" thing sorted out first.

Speaking of which, the middleman emailed me this morning with the reply they got from their outsourced payment processor:

So it looks like we need to ensure transaction is rejected at the bank and the money is refunded first before we can proceed. It should normally take a few business days but I hope to have all of this sorted for you this week.

Please let me know if you have any questions.

Oh do I have questions, most of which I didn't ask. I DID ask if "sorted out this week" involved just reversing the first transaction, or if actually getting money into MY account might happen this week, because of the whole "email needs to change ISPs at the end of the month" and me just assuming it will go as wrong as this because it often does for me. I break everything. It SHOULD take maybe 24 hours for the DNS to update. I EXPECT to be unable to receive email at landley.net through maybe the end of June? No idea.

Questions I DIDN'T ask include whether they could elaborate on the word "ensure": is there a strong possibility of the money NOT being refunded? If the money went into a random account that didn't have my name on it and somebody got dollar signs in their eyes and absconded with it, is there some sort of insurance here? (I have zero visibility into this process. I've heard of large windfalls winding up in people's accounts due to bank error and them getting in trouble for spending it, but they still spent it.)

And of course, is the transaction being rejected because we asked them to reverse it, or because the transaction that "successfully completed" couldn't have successfully completed, both because you can't do a domestic transfer via swift and because there's no destination account with that name/number pair at that bank? (The name "Landley" was made up during World War I because a german immigrant named "Landecker" wanted a less german-sounding name fleeing a war to the side that WOULDN'T draft him; there's not a lot of us. Google says a few people not related to me have taken it as a stage name, usually as their first name.) If "rejected" means rejected rather than cancelled, then how did it "complete successfully" first? What was the three day waiting period for if "destination account exists" was not part of the success criteria?

I had more questions, which I decided to edit out of the email reply as "not helpful", so I saved the longer version as a draft in thunderbird, edited it down, sent it, and watched thunderbird delete from the drafts folder, without backup or undo option. Thank you thunderbird. You wrote a copy of the short version in the sent mail folder, but completely deleted the long version I explicitly saved. Bra fscking vo.

Yes, this is a normal weekday for me, why do you ask? I break everything, all the time. Nothing is foolproof because fools are so ingenious, and whatever soup of neuroatypicality I have combines with knowing just enough to be dangerous to near Newton Pulsifer levels at times. I am trying very hard not to have to migrate my email until the Q2 bank deposit is sorted, meaning I'm on a deadline. Wheee.

And yes, I make an EXCELLENT TESTER because of this. Doing it for other people hasn't been how I prefer to spend my time, in part because Murphy's Law only fails when you try to demonstrate it so RELYING on being able to break things seems like it'd break, but if you wonder why the software I make is like it is, this is what works for me. Achieving basic functionality (not gonna say reliability) by process of elimination. The Linux USB maintainers do not use dodgy cables nearly often enough, or they'd have fixed whatever locking problem causes ports get tied up by a device that's been unplugged. (My laptop has 3 of them, but I've run out before. Luckily software suspend unloads and reloads the USB subsystem as part of its power management shenanigans, which seems to fix the issue.)

People worry about fork bombs, I worry about "tr '\0' a /dev/zero | sort" or "while true; do mkdir a; cd a; done". (Has anyone ever actually TESTED inode exhaustion in most modern filesystems? Running out of SPACE, sure, but ext4 has fixed-size inode tables, if you touch a zillion files but leave them zero length, you run out of inodes without running out of disk space. What do OTHER filesystems do, and when were those error paths really last exercised?)

I have so much practice at debugging because "how did I break it THIS time?" is a question other people have consistently been unable to answer. Still dunno what was wrong with my phone's volume down button. It's happy again now. Could go out again at any time. Same as it ever was...


May 15, 2022

Last time I sat down to do "tar --transform" the problem was "tar --xform" is a synonym, but there's no short option. Short options can have as many --longopts as you want as synonyms, and they all get collated which means when they can collect a single * argument list and all goes into the same variable. So if "tar -Z" was a synonym the optstr "Z(transform)(xform)*" would let the command line "tar --transform s/abc/def/ --xform s/ghi/jkl/ --transform s/mno/pqr/" put all three in the same list in the original order. But without a short option to collate them under, leading "bare" longopts don't collate, so each would collect into its own arg_list and processing them after the fact would have lost which order they arrived in relative to each other.

It's not a BIG flaw, and this problem has come up before: sed doesn't remember what order it saw -e vs -f in when both are on the same command line. A certain amount of detail being lost is inherent to having generic code do the option parsing before calling main(): the neatly collated data can miss some subtleties of the original. But this time it bothered me enough I shelved it, and now I want to fiddle with the option parsing infrastucture to provide a way to at least collate bare longopts.

Except... $HOSTCC scripts/mkflags.c running on the host to generate a header file at all is a bit of a wart in the design. MOST of generated/*.h is created with a shell script calling the existing toybox commands. I've slowly been deprecating scripts/config2help.c so maybe at some point I could yank it (possibly replacing some of what it does with USE_COMMAND() macros in the help text?) and it really SEEMS like what mkflags.c does can be done with bash/sed. Especially if the result was more like:

#define FLAG_q ((USE_THINGY(1LL)|FORCED_FLAGS)<<42)

I.E. when a string snippet is in a macro (INSIDE the string, I don't care about the guard macro around the NEWTOY() line here), I could just put that macro in the flag definition, and this avoids the whole dance where I curently run the arguments through the cc -E preprocessor twice to produce different option strings and then have mkflags.c compare them. (Which is a BIG part of the reason the headers have to be regenerated whenever the config changes!) It can be done in ONE pass through the data, and the result would be the same no matter what .config said.

Except flag macros can theoretically nest, and teaching sed to push and pop them... would not be pretty. Hmmm.

ANYWAY, what I should do is add an "invisible short option" that lets me collate longopts but doesn't result in a new short option, and teach the existing mkflags.c to handle that. What I DID was a fairly extensive cleanup pass on the build plumbing that doesn't address this at all, but makes the output a little more readable and makes explaining it in a video easier. :)


May 14, 2022

If a Pixel 3a's "volume down" button sticks, the phone is completely bricked. Specifically, when you reboot it, the bootloader menu comes up and then can't be dismissed while the "down" button is held, so it will never reboot back into android again. (And you can't power it OFF either, holding down the power button only works when it's not also holding volume down. I have to wait for the battery to die.)

The phone repair place in Hancock center didn't survive the pandemic, neither did the one that used to be in Fiesta across the street. The closest phone repair place I can find is the "1up" phone repair place on Guadalupe at UT (which I've walked past late at night and it specifically says they repair Android devices on the window). Dug around to find a paper book to read on the bus. (No bluetooth headphones. No podcasts or audiobooks. No e-books. No phone games...)

The 1up phone repair said a Pixel 3a is unserviceable because the case is designed not to come apart (and wouldn't go back together again if it did), so even if it doesn't need a replacement part (which they DEFINITELY couldn't get), they can't get it open to service anything inside without destroying the phone. A stuck button is the most trivial POSSIBLE hardware problem (other than maybe the usb port filling up with pocket lint, which you can fix with a needle or a sim card ejector tool), but the phone is a brick now. Because of the contact switch under a button.

The thing is, I bought this phone through Google instead of tmobile because I specifically wanted an unlockable one. (Even though the bootloader says I never actually bothered to unlock it.) And mail ordering another one would take a week. I could presumably pop the sim card out and put it in an old phone, but I stopped using my Nexus 5 because it got wet (and was never quite happy afterwards), and besides it was 32 bit so stopped getting security updates. I expected THIS phone to become my backup phone someday, and possibly be a thing I'd unlock and reimage and test software on...

Really not impressed with Google's hardware policy right now. I've had this thing for less than 3 years but it is now completely unserviceable. (I haven't even scratched the display, the problem is OBVIOUSLY TRIVIAL, but no. It's a brick. Thanks Google.)

Sigh, what was that cheap Motorola phone Louis Rossman liked? (Can't google it right now because I'm still out at UT and have no phone tethering because brick. Ah, UT guest wifi. It's the moto g. Which is $190 new and has an sd card slot, something the pixel did not. Its processor is cortex-a53, 64 bit, 8 cores maxing out at 2.3 ghz. In theory that's faster than my laptop. Only 4 gigs ram instead of 16, of course, but THIS phone accepts a microsd card and I've got a half-terrabyte one of those somewhere...) Still multiple days to mail-order it, but there's a T-mobile store in Hancock center. Back to the bus...

HA! The guy at the t-mobile place was able to unstick the button! Even he's not quite sure how he did it (I took it out of the case and pressed, scraped with the SIM ejector pin, and whacked it against tables and such for like an hour) but it's working again! For the moment, anyway. Limp by until money comes in again, at least.


May 13, 2022

As Sam Vimes said, you do the job that is in front of you. Right, back on the horse.

Finished editing and uploading the second video (first was how to download and use binaries, second is how to build from source, including basic cross compiling if you have cross compilers lying around). I need to start fiddling with playlists...

The dropbear problem is because dropbear's autoconf is stupid. Zlib built for arm64 just fine, but then dropbear did:

checking for aarch64-linux-musleabi-gcc... no
checking for gcc... gcc

Because IT'S NOT -GCC, it's -CC. So dropbear has the same hardwired headache Linux does if you don't hit it with a rock. (Why did the absolute path version bypass that but the search-in-path version didn't? Autoconf! Why is it NATIVELY COMPILING WHEN IT CAN'T FIND THE CROSS COMPILER DESPITE THAT NOT BEING WHAT IT WAS ASKED TO DO? Because gnu/autoconf has "gnu" in it, so it doesn't care what it was ASKED to do, it thinks it knows better than any unwashed masses outside its cathedral (it was the original one) and will impose its will upon them for their own good, "this hurts me more than it hurts you", "look what you made me do", etc. Of course there's a middle aged white man at the helm, you can't be properly patronizing without "pater", the latin word for "father", as in The Fatherland. And of course and OF COURSE. Why did Matt decide to use autoconf? Sigh. I object to the design philosophy behind this tool. Oh well, usual whack-a-mole solution...)

And moving the package builds before the toybox build (because the new scripts/root/dynamic needs to run before toybox builds if toybox is to be dynamically linked on the target) revealed the bug that my dropbear build script isn't returning to the top level menu. (The package builds are sourced, not run as child processes, because dropbear has to set an environment variable that writes extra stuff into the qemu-*.sh launch script. But that ALSO means each build script has to clean up after itself for things like $PWD. The problem in this case is that dropbear builds two packages, zlib and dropbear, and the lifetime of those package builds overlap because the zlib build creates a static target library dropbear links against but which is not installed to the target (no, I didn't modify it to be aware of the new dynamic build option), and setupfor/cleanup does not nest so "setupfor zlib; cd ..; setupfor dropbear; cleanup" puts us back in the directory "cd .." got us to...)

Hmmm. I kind of want a debug function that compares "env" output before and after, and $PWD, but... eh, that's not SIMPLE. Defeats the purpose. Really, the package builds should work as child processes. Maybe the QEMU_MORE stuff should live a file somehow. Hmmm...

Anyway, fix up what's currently there. For the directory I can just pushd . before the source and popd afterwards. Bit of a band-aid but it works.

And while I'm at it have there been new releases of zlib or dropbear? Yes to both. Except zlib 1.2.12 isn't on the sourceforge mirrors. The previous release (from 2017) is there, but the curent release (from 2 months ago) is not. Despite https://zlib.net still linking to them as "mirrors". Time to email the maintainer. (If I link to the copy on zlib.net, the build will break as soon as he releases a new version because he deletes the old ones when he does that. "Because building against it would be insecure", he says. That's not how this works for any other package. Ok, it took him 5 years to get this release out because it's a very stable project that does not do "heartbeat" releases to show the maintainer hasn't quietly died without anybody noticing. But still. Back in aboriginal linux I had my own mirror page I put the tarballs on, but manually maintaining that and other people depending on me to do that going forward isn't SIMPLE.)

Eh, I can update the dropbear version, anyway...


May 12, 2022

Sigh, miscommunication within the houshold resulted in the mortgage check being sent out on time (so it wouldn't be late), and bouncing today because the money wasn't in the account yet. (Fade then got a cash advance against her credit card to put enough money in the account if they resubmit it. If I knew the money would take this long to get here she could have done that last week, or I could have taken another contract after the last one, or sold stock, or tapped a Roth IRA... Two recruiters have called me this week to ask if "things are finally panning out with Google" and I replied that I honestly don't know. Xeno's invoice.)

The money SHOULD have been in the account because the dot-com company that the middleman outsourced the wire transfer to said it completed at 8:15 this morning, but the money wasn't in our account. Fade called the bank people, and they said there was no trace of a transaction. Cue hours of digging...

So of course it turns out I filled out the bank deposit information wrong. I typed in the info it asked for accurately: the account number was right, my name was right, my mistake was when it asked for a swift code I gave it one. The bank guy Fade talked to assured her you can't use a swift code for a domestic transfer, it's ONLY for international transfers. (And yet the transfer went through...? "Yes, but what happens if you DO?" "Oh. Huh. Ummm..." Ah, that old refrain. Story of my life.)

The middleman company's web form defaulted to "swift" instead of "ABA routing number" (and thus there was no place to type a routing number since I didn't notice you could click on and switch the type), and their system did not check that it was trying to do a domestic transfer to a US address via swift code. Instead it did the transfer to "Wells Fargo Bank International" (which apparently isn't the same as the domestic Wells Fargo Bank?). The account number was right... for the domestic bank. There apparently WAS an account with that number to receive the money, since IT WENT THROUGH. Not minding that my name was not on that account?

So now I've asked the middleman company to ask their outsourced wire transfer company to reverse the transaction, although this took hours to do because the person I was emailing last time turns out to be in New Zealand and didn't get into work until around 4pm my time, so I'd sent four emails and created an account on the middleman's slack by the time I got a reply.

They say I should wait for the transaction to reverse (3 business days), then change the deposit info and resubmit (3 more business days). So hopefully sometime next week I might actually see money. And I shouldn't fiddle with anything while it's reversing which rules out a parallel invoice with the remaining money parked at the middleman. (I only invoiced myself for 3 months because I did not trust this process. A few weeks back I explained to Elliott why Fade handles all our online purchases, and he tried to reassure me as to their safety. Did I mention that as far as I can tell, none of my computerized medical records still exist? You literally CAN'T get rid of that stuff, and yet. Welcome to my world...)

That ate my energy for the day. Curled up into a ball for the rest of it and finished reading Harbinger... which ended on a cliffhanger. Book 4 came out in 2014, EIGHT YEARS AGO. Sigh. I really hope it doesn't take her another 8 years to resolve the cliffhanger, she turns 60 next year. (I'd subscribe to her patreon except for the stupid screen blanking bug every time I visit her profile, no of COURSE patreon hasn't fixed that. I should do that on my phone, but "my phone being authorized to spend money, even indirectly" is a line I have not crossed. The orwellian tracking device with cameras, GPS, and whole-room microphone that's on and with me 24/7 does not get to spend money. Yes I am contributing signifcant chunks of software to its operating system, but that's one lump in the katamari. They try very very hard to secure it, and state level actors and criminal syndicates (venn diagram's not QUITE a circle there but between trump/putin/xi it's pretty darn close at times) push back the other way with a literal body count. The hardware was made in china and I'm supposed to TRUST it? My main defense here is being largely uninteresting and blending in with the crowd.)


May 11, 2022

I got a bug report about disabling logging changing the cross compiler behavior in a way that broke the dropbear build. It looks like an absolute path to the cross compiler prefix (thus any tool name you append becomes an absolute path to that prefixed tool) becomes an unpathed prefix with the $PATH adjusted to have the set of tools in it. In THEORY both should behave the same, in practice the dropbear build started using gnu/autoconf back in 2009 and thus only ever works by coincidence.

But when I tried to reproduce this problem in a clean directory, I got:

scripts/root/plumbing: line 17: root_download/zlib-1.2.11.tar.gz: No such file or directory
wget: unsupported protocol: https://downloads.sf.net/libpng/zlib-1.2.11.tar.gz

Oops. I promoted wget and updated the airlock list to replace host wget with my wget, but the "defconfig" build doesn't include openssl (and thus https support). And I really don't want mkroot to DEPEND on the host having openssl installed. Long-term I want to do what busybox did and implement my own https plumbing. (Denys did it from the RFC, it's not an unmanageable lift. I'd REALLY like to make puppy eyes at him and use his code under 0BSD but he's been incommunicado for a bit.)

In the meantime, possibly I should modify the dropbear build to use /usr/bin/wget for https downloads. (Or at least fall back to it on a second attempt...?)


May 10, 2022

Wen Spencer's "harbinger" is out, so I'm re-reading the series up to that point before reading the new book. The new one's book 5, so this may take a while. (It's been out for a month but I was waiting for the audiobook, and I've stopped waiting. I prefer actually reading books, but since eyestrain is my limiting factor for working...)


May 9, 2022

My problem with $(echo||) being unable to return an error is that pipe_subshell() returns an fd, but not the PID, without which we can't wait() to get the exit code. That's fine, the shell won't accumulate zombies because wait_job() will eventually harvest the child data (and discard it as not in the jobs table). But it does mean that the return code from the subshell doesn't get propagated anywhere.

Which is actually what bash does: echo -n $(false); echo $? prints 0. What barfs is a syntax error, which bash notices early because it's the host shell recursively parsing the shell syntax within the $() and thus noticing the unbalanced statements before the end of the parenthetical block. Which means this isn't JUST an obscure issue of using case statements in shell substitutions (which I've never actually seen done), this is easily user visible syntax error checking. Hmmm.

Darn it, I have to make $() parsing catch syntax errors at parse time. Which means parsing has to be recursive, which it's REALLY not set up to do. Grrr.

Probably I should finish the rewrite to let while true; do echo hello; done | head -n 3 work properly first. (The first pipe segment needs an implicit subshell, the while can't run to completion in the host shell context before launching the next pipe segment. Also, head closing its stdin needs to terminate the while loop. Right now if you add explicit parentheses around the first pipeline segment yourself, "echo" is noticing the output is gone but the while true isn't and thus doesn't terminate, so the pipeline hangs eating CPU after outputting its three lines.)

All of this is tangled into terminal and job control, of course. I did large plumbing changes for this and got pulled away before they were finished, and need to reverse engineer my own code so I can complete it. Which is a well I haven't dived back into yet because I need to pick off some of the items on Elliott's list first: at least dd, tr, readlink, and tar --transform since all those are ALSO open cans of worms already...


May 8, 2022

Darn it, it's been long enough since I've edited videos I'm not rembering the key bindings. I need to make myself a cheat sheet. Let's see...

I did blender setup and saved defaults so it starts in video editing mode with a good panel layout, so I don't need to do anything with that, but I should dig up and bookmark the video I used to do the setup in case I need to install it on a new machine.

Plug the USB headphones in first, click the xfce volume icon and select "audio mixer" at the bottom of the drop down, go to "output devices" and click the green checkbox, go to "input devices" and click the green checkbox, THEN run blender (because you won't be able to move what it's outputting to otherwise and for some reason plugging in the headphones doesn't make them the default input/output device). This is more important for recording than editing, but it's still nice to have headphones when editing.

Dismiss the stupid advertising pop-up it always starts with, then at the bottom click "add->movie" (in view/select/marker/add/frame/strip). navigate to the first mp4 file from simplescreenrecorder and select it. Mouse over the display window and hit "home" to zoom it, mouse over the edit strip at the bottom and hit "home" to zoom that. In the upper left (properties) area scroll to the top and update the "end frame" field to the number of frames in the clip. (That has to be manually updated every time material is added, "home" doesn't change it. The default of 250 is insanely short.)

Now down at the bottom press "play". If it loops back to the start you've gone past the "end frame" and need to adjust. If you pause and advance the frame number with the arrows it plays just one frame, and at 5/sec it's reasonable to navigate by that. The vertical green line is the current frame number. Playing forward is more coherent than playing backward. It cuts at the current frame it's the START of the current frame, so if you advance to where a sound plays or a screen change happens, your ending frame is ON that change (not the frame before it). The starting frame for a cut is AFTER the last frame you want to keep. Left clicking also moves the line (selecting a strip is right click, yes it's crazy/backwards). Or hit "b" to batch select, hold down the left mouse button to drag the rectangle over both strips (audio is on top, video on the bottom), and release the mouse button. No there isn't a reliable visual indicator of what is and isn't selected, it sort of changes a bit but not in an easily confirmable way.

Hit shift-k to hard cut at the vertical green line. (Only applies to selected strips, and you have to re-select after every move. Non-shifted k is a "soft cut" which has never worked right for me.) Right click the little deleted bit, hit x to delete, confirm in the pop-up menu (sigh, it's got ctrl-Z to undo with a VERY deep stack but no, it needs a confirmation pop-up every time), select the remaining strips with "b" again, hit "g" to move, and move them left to fill in the gap (cursor left, then enter when done. If it overlaps the existing one it'll go red, but it bumps it back right to not overlap when you release so you don't have to be too precise. If there's a black dropout, you accidentally left a gap so move it again.)

Video editing is basically repeating that over and over to chop out the 1/5 second intervals that shouldn't be there until done. Add the image trailer at the end, then in resolution select "100%" in scale for render resolution (it keeps going back to 50%, maybe I need to update the save thingy?), and then at the top render->render animation. This writes out the file, very very slowly, to a random-ish name in whatever directory it started in (it doesn't prompt you for an output file name, and when you navigate to another directory to read the filename, it doesn't care).

Watch the result in vlc to make sure it's acceptable, then upload it to prudetube.


May 7, 2022

I'm still on some University of Wisconsin mailing list from that japanese class I took in Milwaukee, and they're having a "Trauma in our community" seminar. (The Boomers are not dying fast enough. Alito is 72. Clarence Thomas is 73. That's almost a decade younger than Biden and Pelosi, but still most of a decade past what used to be mandatory retirement age (65) before Boomers refused to step down while still alive. And that's ignoring the 40 years of breathing leaded-gas fumes that gradually made the lot of them measurably dumber even before they started going senile.)

Slowly getting pulled back into shell headspace. I have toysh to-test notes for things like echo $(if true) and echo $(true &&) that should produce syntax errors (and thus an exit code of 1 despite inability to run the command, same general idea as ${unset?complaint} erroring out in the command setup before having something to run). Making that work requires some poking at sh.c. It also raises that problem that if it just produced error output without changing the return code (luckily not the case here), I'd have no idea how to make the test suite TEST that properly, with TEST_HOST also passing. I kind of need a regex match on the produced output. Or "did it produce any output"? Hmmm... I could probably do it with txpect.

The next fun todo item in that same list is echo $(case a in a) echo hello;; esac) which toysh can't parse because $() contents just matches up parentheses, it doesn't do full recursive parsing (I'm waiting for somebody to complain who actually NEEDs it) because it would have to either discard the parse results and re-parse it again later anyway, or else result in a nasty branching tree structure that would be awkward to free() again. Right now I parse shell script into a circular doubly linked list with each node containing an array of argv[]. (Ok, it's an array-of-arrays with the first being the command line and the later ones being HERE documents; reality is ALREADY pretty complicated.) Having each element of that argv[] potentially have MORE THAN ONE entire subsidiary shell script circular-linked-list-containing-array-of-argv[] is just... really dowanna? That's why I'm keeping $(substitutions) as a string until it's time to execute them. You've still gotta get the sequencing right:

$ X='$((1+2))'; echo $($X)
bash: $((1+2)): command not found

But that's why I have/need so many tests...


May 6, 2022

I got the release out, after the usual shenanigans.

Back to poking at dd again, and hating dd conceptually. This keyword=value thing was really not a good interface (posix never even says what conv= is short for), and the DEFAULTS are all wrong: Why is there no seek=end? Without conv=notrunc dd defaults to truncating the file before writing to it, but oflag=append does not imply notrunc? (The toybox one probably should; that's a deviation from what upstream does but it's a NOP otherwise! Or undefined behavior when two oflag=append stomp some of each other's initial data and then interlace the output.)

Right, nose to the grindstone. Pickaxe to the coalface. Read the spec again, and the man page. Oh, speaking of which, I'm still subscribed to the coreutils mailing list (camping the spawn of "cut -DF") and I saw a MUCH nicer syntax for something wander by... here. iseek and oseek instead of remembering which one's seek and which one's skip. I definitely want toybox dd to support that. Although the adjacent commit is damn awkward to implement because toybox passes all this through to "atolx()" which is case insensitive, and "b" is already 512 byte blocks... Wait, no, it was originally bytes? Why did that change... because "most things" wanted b to be 512. Which things? Apparently dd! Probably posix says that. And indeed, dd if=/dev/zero count=1 bs=1b | wc gives me 512 bytes. So they added a new capital letter that means something different than the lower case letter. Bra filesystem checking vo, gnu/dammit guys.

Over on github Elliott wrote a thing more or less in favor of assert() to check for "should never happen" errors. Personally, I lean towards "never test for an error condition you don't know how to handle".

The problem is half the time a "can't fail" error is benign. I've seen spurious -EINTR leak through because the process was suspended and resumed, or -ENOENT when a device got hot-unplugged (which should have been treated as EOF but wasn't; I.E. same general reason for nerfing SIGPIPE). In a lot of cases the correct thing to do is ignore it and continue, because it doesn't actually indicate a problem. Exiting in such a case just makes the program more brittle.

An assert() is fundamentally about being afraid to do the wrong thing, thus exiting. But exiting can BE the wrong thing. (Especially when your caller doesn't notice and carries on with the truncated data, but simple denial of service is enough to be wrong when it would otherwise have worked.) In trying not to do the wrong thing, you do the wrong thing. In biology, this is the histamine problem. We deploy antihistamines at everything because our bodies respond to a WIDE RANGE of stuff with swelling, which is often a bigger problem than the original issue... but not always. Exiting is not always the wrong thing: I've got buckets of xfunctions() in lib/xwrap.c that exit when they detect something going wrong. But I've also removed a LOT of spurious asserts from code over the years where it was complaining about a non-problem (and thus the proper fix _was_ to remove the assert).

"What is the best thing to do here" is a DECISION. I do a lot of staring and pondering at cases, trying to figure out possible inputs and their ramifications. But "if the unexpected happens, do this"... having a stock response to the unexpected is a contradiction in terms. When I don't know what the correct response is, I lean towards not making it worse. If I don't know how to respond, I do not prepare a response. (And "suppose the kernel is compromised and sends you malicious attack data when you're reading /proc/filesystems"... I can't fix that. I'll suppose a future kernel produces more than 4k of input, sure, but a filesystem name of "../../../../etc/shadow" is not an attack vector I'm willing to add code to defend against: in that circumstance they can probably already ptrace me. I'm out.)

What I really want is a proper test suite that triggers every possible error path, but that turns out to be really hard to do. An error handler that is never tested can BE the problem. Case in point just now: I got the order of operations wrong because that error path has never triggered. How do I add a test to trigger it? I don't remember how you can fail to create a new variable, but I remember there was a way to do it? I need to read through setvar_found() to see why it can fail... the two goto bad; cases are because VAR_READONLY, or += on VAR_INT failing calculate(). Except (vv = addvar(s, ff))->flags = VAR_NOFREE; does not set either of those flags? Oh what was this case... something about dangling symlinks with VAR_NAMEREF? Have I not implemented the variant that can fail yet?)

I gotta set aside a block of time to get back into this headspace. If I can't come up with a test case, then the error condition can't occur here and I don't need a handler for it. (And that's not a "should never happen" because a function I wrote is calling another function I wrote. When you change the function, you check the callers before committing it. And then you add tests for the externally visible behavior your new code causes/allows/prevents/changes.)

Sigh. It is entirely possible I'm wrong, but I don't think going EITHER way is always right. Elliott leans one way, I lean the other, but it's an individual judgement call in each case what to do.


May 5, 2022

I did not put a jar of mayonaise in the sink and take a picture of it today. Not feeling the holiday spirit.

Today was the day that money from google was supposed to make it to the middleman organization. (45 day payment terms in invoices, and Fortune 500 companies always wait until the last day to pay so they can eke out tiny little bits of interest on payables. That's late stage capitalism for you.) The website balance is still reading $0.00, but I sat down and figured out how to do the paperwork anyway (with Fade handholding via phone). I invoiced myself for Q2 (13 weeks, at exactly halfway between the hourly rate the last 2 gigs were paying me), approved my own invoice, entered my banking info for deposit, filled out the third party form they use (docusign is neither secure nor particularly legal, and yet...) to hand over my identity theft number so they can do tax reporting (although not witholding)... and now we wait. Their documentation says they process approved invoices twice a week. I'm assuming the invoice existing before money comes in won't cause a problem (it'd just get bumped until next time)...

Almost got the toybox release out. I actually uploaded a version of my release checklist which isn't entirely up to date but gives a general idea. Wrote up the release notes, and edited them into parseable HTML. Built the mkroot images with the right version tag and uploaded them. (I cheated slightly and undid the tag, committed two more things, and redid the tag after the build, but both changes were just to documentation and wouldn't affect the binary). The toybox static binaries come from mkroot these days, out of root/$ARCH/fs/bin/toybox in each target build. Yeah it pulls in two things out of $PENDING but I'm ok with that.

I spent April shoveling through open tags and backlog of half finished stuff, trying to close down for a release, but what I really need to do now is get serious about the dev environment stuff. Google gave me a list of stuff they want for AOSP to hermetically build itself, which is the priority to get done first, that's only part of the picture.

I want to run the toybox test suite under mkroot, which is MOSTLY a question of toysh being good enough. Right now, when I run "./sh scripts/test.sh" it goes "source: syntax error 'scripts/runtest.sh'@221: ;;" and yes I had to upgrade the shell to beat that much detail out of the error message. (It was initially just saying "source: syntax error ;;" which... isn't helpful.) This is the same part of the code (case/esac parsing) that's preventing /usr/bin/ldd from running under toysh. (Gnu ldd involves a large shell script, which starts with #!/bin/bash so once again ubuntu's decision to switch /bin/sh to point to dash remains stupid 15 years later. You can't NOT have bash on the system. You can TOTALLY not have dash.) Once I've got the tests running under mkroot, I have a lot MORE tests I need to add that run as root and/or require a known environment containing things like the ethernet "dummy" driver.

I also want to build toybox under mkroot with a native.sqf compiler loopback mounted. This immediately dies due to "make" not being there, but assuming I harvest static binaries from an old aboriginal linux build and stick them after toybox in the $PATH, I can presumably attempt that with toybox providing what it can, and both verify that the toybox commands are load-bearing in this context, and get a list of what's missing via scripts/record-commands.

I eventually want to do a new automated Linux From Scratch build under mkroot. I'm tempted to take my old LFS 6.3 build I automated years ago and run that (because it was known working at one point), except more than one of those old packages have #if/#else staircases ending in "#error what is musl libc". This was the case even when I was trying to use musl in my old "last gplv2 release of everything" toolchain under aboriginal (and was the main thing delaying me from switching that project over to musl: I'd lose the ability to build LFS natively unde the result). Using an even newer toolchain with the old packages will not have improved this.

This sort of version skew problem happens a lot. On monday SJ Hill emailed me asking about initramfs console issues in a 2.6 kernel, and I couldn't build a 2.6.39 kernel from a git checkout to test my answers with the current debian toolchain because apparently newer gcc versions default to producing pic code now? And then when I applied that fix, the 2.6 build tried to #include linux/compiler-gcc8.h which doesn't exist. It really really should not CARE, but does. He said he's building it under an old debian version running in a VM, which was more effort than I wanted to go to just answering email related to documentation I wrote in 2005. Anyway, we got his problem fixed and he wandered off to do his work. The point is, old packages not working with new build environments isn't always a problem with the new build environment, but with the old package making stupid assumptions. When "gnu" is involved, you can just about guarantee it.


May 4, 2022

A google developer is sending me fixes for toysh. I am simultaneously happy somebody's paying attention to it and embarassed it's not ready yet. It's like a realtor coming to show somebody the house when you haven't tidied up. I'm suppressing the urge to preemptively apologize. They're reviewing an unfinished draft, but I've been working on this for over 2 years now so it's my fault it's not entirely there yet. Speaking of which....

I recorded two large chunks of what should be the next video, on building toybox from source. Now I need to edit it into... at least 2 videos, it's 20 minutes of raw footage. I need to remember to type "reset" and clear my terminal window more often, because my editing decisions are a lot more constrained when I don't. (The backscroll in the terminal window jumping around would be distracting.)


May 3, 2022

The GOP was especially evil today. Too stressed to focus much. And yes, this is the same supreme court that ruled in favor of Nestle in the recent child slavery case. The Boomers are not dying fast enough.

I'm trying to remind myself that youtube becoming prudetube isn't JUST a sign of US culture being co-opted by witch-burning puritan Boomers clutching their pearl rosaries as they sink beneath the waves, it's mostly about youtube being incompetent and self-destructing by slowly driving away its creators. (Note how in this great resignation howto video at 3:13 the word "onlyfans" is bleeped out, in a list of patreon/etsy sites people use to make money online.) And that Amazon becoming an SEO hellscape being called out by celebrities is about them having bait-and-switched from customer-driven search to bid-for-placement while even screwing over sellers with its own copycat products that get the top five search spaces for free.

All platforms have a lifespan, and if Youtube is going the way of Vine, Twitter going the way of Livejournal, and Amazon going the way of Woolworth's that's not unexpected. More like inevitable. Similarly, I now go entire weeks without remembering microsoft EXISTS, which did not used to be the case. They're still around, but after 30 years of unassilable dominance they have no presence in the phone space, can no longer exclude macs and chromebooks from the PC world, and have their own version of "Win-OS/2" (which meant back in 1993 anybody deciding whether to ship an OS/2 binary or a windows binary could just ship a windows binary and cover both systems, quickly ending most native OS/2 development).

There's plenty of places to buy stuff other than amazon. Even if I'm not personally active these days, plenty of people are still promoting nudism, and holding events with reasonable turnout. (I'm told Star Ranch's naked 5k run had record attendance this year. I wanted to go but it was too close to my return flight from Minneapolis.)


May 2, 2022

Trying to update various toybox documentation, and it's HARD. I very carefully phrased was was there, over a large number of attempts, and now I need to change what it's saying in a way that resequences stuff, reorders categories, changes category divisions... Hmmm.

Xeno's release process continues as usual. Half way to halfway... Meanwhile the bug reports continue to come in. *shrug* The tree's in pretty good shape, I'd just like to nail it down and SNAPSHOT it already. But that involves a LOT of documenting, which is a special kind of turning over rocks to see what's underneath. (Honestly, I'm not really that good at this. I'm just persistent!)


May 1, 2022

My wife is getting a degree in classics, doing her dissertation on "Social interactions between (the group of marginalized people composed of) slaves and sex workers in Plautus". She's currently taking a class on sex work in the ancient world.

A recent thread from Dr. Sarah Taber on how men took over the egg trade from women (replacing "egg money" with factory farming and Salmonella) neatly explained why prositution is illegal. The relevant observation is "Men force women into the worst-paying parts of the... system. But once those trades start making [money], the women have to go." Here's a 1983 paper how men pushed women out of computer science as soon as there was money in it. Centuries ago a woman invented kabuki dancing and later only men were allowed to do it.

Prostitution doesn't cause any actual PROBLEMS, it's just that women always make more money at it than men do, thus it can't be allowed. Yet it's been legal all my life in Nevada and Amsterdam which have no problem with it. The current war on camgirls is related to the blowback against "the great resignation": young women achieving financial independence working from home cannot be allowed by a patriarchal dominance hierarchy. Everything else is just an excuse.

Centuries of home weaving and sewing were forced to give way to the Triangle Shirtwaist Fire, an example of human trafficing that had nothing to do with sex work. Most human trafficing today is in agriculture. Blaming sex is the usual pearl clutching diversion.

Fade's tumblr is full of references to, among other things, "our stronger nudity taboo", and as a (largely lapsed) nudist, I am sad about said stronger nudity taboo. Made recently much, much stronger by senile lead-poisoned boomers.

Fade, Fuzzy and myself were recently trying to watch Thermae Romae on Netflix, a remake of an anime about a roman bath designer time traveling to modern japan to steal ideas. It's TV-MA... for no apparent reason. They never show a single dick. Awkward camera angles, shiny water, and anachronistic towels. (In the first episode they put all the bathing romans in towels, and then a few episodes later one of the great discoveries the roman bath architect time-travels to modern Japan to appropriate is... towels.)

Seriously? What is this nonsense? People have bodies. I showered with my younger brother until I was ten. Parents taking naked pictures of their children was entirely normal before the Boomers took over; nudity is not the same thing as sex, and this A) drawings, B) already a pay service, C) an entire story about people having baths.

Daryl Hannah's naked body in the 1984 movie Splash was PG, but the Disney+ version recently deepfaked dog hair over her butt. (Oddly enough Tubi had the original unedited version last month, because an ad-supported service was less scared of itself than a billion dollar conglomerate that regularly rewrites the law.) The video for Wrecking Ball is still on youtube, 1.1 billion views over the past 8 years, which somehow has not led to the collapse of civilization or even the singer (despite the Disney child star curse).

As for full frontal male nudity, Richard Donner's 1978 Superman showed a naked little boy's dick (when Kal-El lands in Kansas and stands up) in a PG movie. In R-rated films nonsexual male full frontal nudity was everywhere from The Terminator to Life of Brian, as recently as 2009 Doctor Manhattan spent most of Watchmen naked.

But Netflix decided to rate something TV-MA:nudity (no other reason given) and then NOT show anything in ANIMATION (drawings, not even pictures of real people) where everybody is repeatedly naked for UNAVOIDABLE PLOT REASONS? Seriously? Either show the nudity, or don't make it TV-MA:nudity. Pick one!

The senile lead poisoned Boomers (no really, they are uniquely measurably stupid) are not dying fast enough. From climate change to fascism (which always includes racism, sexism, and homophobia in the definition of "fatherland" because everyone must perform their assigned societal roles full time no exceptions), it's a race between enough Boomers dying and the collapse of civilization. I'm aware this is a rounding error compared to that, but it's personally annoying.


April 30, 2022

Oh good, Denys is posting to the busybox mailing list again. That means I can stop paying attention. (I noticed in the first place because there was a guy emailing me about a bug disclosure that he thought might be a significant security vulnerability, and I was trying to help him. He's posted it publicly now. Yes, I still get emailed about busybox stuff, and try to at least gracefully hand off...)

I've copied the toybox release note slush pile into the start of news.html, grabbed a new hitchhiker's guide quote for the top, and am trying to beat it into some semblance of order to cut a release. There's still a zillion trailing loose ends but it's been FIVE MONTHS since the last release and I'd really rather not make it six.

Blah, what is my todo list: Release notes. Youtube channel for the videos. Update nav bar with youtube and patreon links, and collate the documentation somehow. While I'm at it, the main toybox page should go to about.html but with release a release note link from there... probably that goes at the top of the page?

The last time I tried to collate that I hit the fact that the summary at the top of news.html and the summary at the top of about.html are redundant to each other, but not as easily mergeable as I liked. Really, I should have a summary blog in header.html (which produces the nav bar via ssi, and yes I have a todo item to make toybox httpd support ssi...)

This is my problem doing this sort of work, it's fractal. Attempting to close tabs winds up opening more tabs. Doing the work makes the todo list longer, and there's sort of a hydra thing going on where trying to find half-finished stuff and finish it results in more half-finished stuff...

(You can tell I'm in debug mode. I'm angry at software design again.)


April 29, 2022

As always, updating documentation leads onto a development tangent. In this case, now that I've got binary toolchains uploaded I want to explain CROSS_COMPILE= as its own part of the FAQ, and the examples I have for that are both statically linked, and I want to teach mkroot to set up dynamic library directories in the target (as a new "dynamic" argument like dropbear), which means writing a new scripts/root/dynamic so you can "scripts/mkroot.sh dynamic" to add the shared libraries the way you can add dropbear. (And then move those builds before toybox, so the toybox build can test -e $ROOT/lib/libc.so and do a dynamic build if it's there.)

But despite all the talk about merging /bin and /usr/bin, what Devuan did NOT merge is /lib and /usr/lib, which is annoying for me trying to add a scripts/root/dynamic that works with the host toolchain too, because libc.so lives in /usr/lib and libc.so.6 lives in /lib. (Well, both add a gratuitous x86_64-linux-gnu-gnu-gnu-stallman-forever-ftaghn-ia-ia-all-hail which has nothing whatsoever to do with Linux and never did. The migration from the libc5 the kernel guys wrote to libc6 (Ulrich DrPepper's fork of glibc is where "glibc 2.0" came from, pretty much the same as egcs where the original project was long dead and linux people frankensteined the corpse into something useful) was because Ulrich's project was all about adding thread support (I.E. spraying everything down with gratuitous locking) and the flood of Java people in 1998 (tripling the size of the community after Netscape released its source code and credited Linux for the decision) outright DEMANDED thread support because James Gosling hadn't included bindings for unix staples like poll/select in the standard Java library so the ONLY way to do nonblocking I/O was to spawn a blocking thread for each background read, so Java devs who'd learned nothing but threading couldn't function without it. (Sun invented threading because forking processes on Solaris was so heavyweight, Linux could fork processes faster and cheaper than Solaris could spawn threads so had never bothered with threads, but if twice as many cobol programmers had shown up to use Linux than the entire previous community, they'd have gotten good cobol support in a year or two, so... Anyway, the FSF was the largely passive recipient of a lot of external development work that tried to use them as a coordination point largely for historical reasons. The gnu project was announced in 1983 and was already vaporware dinged by Linus' 0.0.1 announcement in 1991. Stallman initially responded by telling people not to use it and then pivoted to trying to take credit for it. And no moving off libc5 wasn't about ELF support, that's what the move from libc4 to libc5 was about. Libc6 was about threading.)

So anyway, I need to copy everything out of the directory that has libc.so, the one that has libc.so.[0-9], and the one... No, hang on. I don't care what the host has, I only care what the toolchain has, and rather than doing library whack-a-mole I need to use cc -print-search-dirs and parse that.

Still need to special case the dynamic linker through. (/lib64 should not be a thing at the top level. Seriously. You ALREADY ADDED multilib directories UNDER /lib, and then you have another one on top. That's very gnu, used here to mean "trying so hard to be clever it wraps back around to stupid". The kind of clever that piles higher and deeper without ever clarifying anything. Eternally accumulating, never simplifying. I'm trying to eliminate problems so you don't have to deal with them anymore, this nonsense confronts you with endless "look what I did, you'll never understand it, worship me" chest beating. When I'm successful at my goals, it doesn't look like I did anything. I'm not building monuments, I'm TIDYING. Grrr.)


April 28, 2022

I'm trying to update qemu, and once again it's "upgraded" itself into uselessness. I want to run "make install" without root access to see where it's trying to install stuff. (Is it in /usr/local/bin or /usr/bin or /opt or what?) But when I run "make install" it pops up a gui window prompting me for my password so it can sudo, and if I don't give it the password it exits with failure. There is no "continue as this user" option so I can see it try (and fail) to install and thus know WHERE it's trying to install stuff.

So I try to find and rip out the call to sudo from the build plumbing, but build/build.ninja is calling meson/meson.py which is calling some library I can't easily find, so second attempt: make a NOP sudo wrapper script that does nothing but run its arguments and stick that in the front of the $PATH... and it's not calling sudo at all, this GUI pop-up is some freedesktop.org thing (I.E. it's from the systemd people). The COMMAND LINE BUILD is going through GUI DESKTOP PLUMBING to INSTALL FILES. That's not right.

I remember now why I haven't upgraded qemu in so long. I WILL NOT run a random script as root to modify my system in ways I can't see before it does the thing. That is not happening. I tried running it under fakeroot, but that said it couldn't LD_PRELOAD fakeroot.so in one of the child processes and popped up the same GUI prompt.

Ok, what I needed to do (after reading files under qemu's "meson" subdirectory, which have spaces in them and lots of references to running on windows) was add --dry-run to the install command line in build/build.ninja, and THEN it says it's trying to install stuff to /usr/local. Ok then.

The new qemu-system-ppc64 is 102976760 bytes. That's over a hundred megabytes. It is not statically linked, ldd lists 68 shared libraries on TOP of that nonsense. Grand total 3.7 gigabytes of JUST /usr/local/bin/qemu-* files, and that's not even counting the crap it installed in /usr/local/share and who knows where else. Bloatware of the highest order. Meanwhile, Fabrice's Bellard's new tinyemu is a 250k source tarball.

(Ok, as Rich points out, stripping qemu-system-ppc64 brings it down to only 20 megs. But that's still insanely large, and qemu installed the unstripped binaries by default. The Devuan install ISO is only a 1.2 gigabyte download, QEMU's default install is THREE TIMES BIGGER THAN THE DOWNLOADING THE ENTIRE OPERATING SYSTEM. Yeah sure, compressed vs non-compressed, I don't care. That's NOT RIGHT.)


April 27, 2022

A problem sei/coresemi/j-core hit a lot was collapsed manufacturing chains, I.E. this is no longer manufactured, there's just an existing inventory of irreplaceable parts. Apparently this is now coming up trying to send enough stinger missiles to ukraine. The USA has a stockpile of them but hasn't ordered any new ones in 15 years, and manufacturing more requires redesigning the electronics to replace parts that are no longer available. Of course once you're aware of this category of problem, scratch the surface ANYWHERE in the Boomers' world and you find stuff the Greatest Generation did that nobody inherited responsibility for. Sometimes hobbyists have reverse engineered enough to know how to do it, but not at scale, and the original sources have lost that institutional memory (let alone ready production capacity) in some capitalist merger/layoff/consolidation.

This morning I MEANT to sit down and fix the sha1sum endianness issue, but instead I got sucked into cataloging various data types the file command doesn't recognize, and now I'm trying to design file -i support, to output mime types instead of the ad-hoc text format file usually produces. There was zero provision for that in Elliott's original design, and it's not easy to retrofit.

Ok, technically I want to implement file --mime-type which is JUST the mime type without the stupid ;charset=potatosalad decoration gnu/file keeps wanting to crap over things. But I have an aversion to long options without corresponding short options (it's not unix!) and thus want to an -i that does the concise/simple output, EXCEPT that makes TEST_HOST harder to support in the test suite.

(If Elliott hadn't submitted his own file.c, I'd probably have done one that JUST produced mime output to start with, and added the long text version later when people asked, and then probably not for everything... but that's not where we are today. Hmmm... This needs a long walk and a design think.)

Ahem, but AFTER I get a release out. And a bunch of videos filmed edited and posted. (Put the shiny thing down and back away slowly. Doesn't matter that I've just half-assed it based on file extension in httpd.c, I need to CLOSE tabs...)


April 26, 2022

Grinding along doing release prep: building all the targets and testing that the network actually works in them, by running my new httpd on loopback on the host and having wget fetch the toybox README file and sha1sum it. (Remember: qemu's 10.0.2.2 address passes through to 127.0.0.1 on the host, so ./netcat -p 8080 -s 127.0.0.1 -L ./httpd matches up with wget http://10.0.2.2:8080/README -O - | sha1sum which would be nice to automate somehow. I have not reopened the can of worms that is build control images yet, but I'm working towards it.

This is a kind of test I've done piecemeal over the years, but there's been nothing systematic up until now because I didn't have the pieces, and now that I _AM_ being systematic-ish about it (or at least retesting all the targets)... my armv7l fix for the network card wasn't applied right. (It's a QEMU bug: QEMU's arm device tree is the same for 32 bit and 64 bit platforms and the address it gives for the network adapter is up above 4 gigs and thus inaccesable on 32 bit targets. You can work around this by enabling the page addressing extensions on the kernel so it can map high memory down into the 32 bit address space, but you really shouldn't HAVE to. The QEMU bug that was opened about this got marked invalid, so we're stuck with the workaround. Anyway, the PROBLEM is when I added the symbol I added CONFIG_ARM_LPAE to the csv list for the armv7l target, and the plumbing prepends the CONFIG_ itself so it was setting CONFIG_CONFIG_ARM_LPAE=y which didn't work.)

Meanwhile, on m68k the wget seems to be working fine, but sha1sum is producing the wrong hash! (Not just for this, echo hello | sha1sum produces the wrong hash too. I thought I had the endianness and alignment right for sha1sum? Hmmm, mips is getting it wrong the same way, that's looking like endianness. (Yup, mipsel is getting the right answer.)

Let me guess: In 2014 Daniel Verkamp did a commit which made md5sum and sha1sum run faster, but made the code bigger and more complicated. Shortly thereafter I did a commit which made them work on big endian. Then in 2016 Elliott did a commit to add a config option to pull the assembly optimized versions of md5sum and sha1sum (and sha256 and friends) out of openssl. Then in 2019 I did a commit which reverted Daniel Verkamp's changes putting the non-openssl versions back to the slow-but-tiny versions. So the question is: did I forget to re-apply the big endian fix?

The other thing I'm not really testing this pass is block device support. Everything's still running out of initramfs, no "create an ext2 formatted hda.img and mount it on /home". That's an important step for doing native builds.

Oh hey, powerpc64le found a new way to fail. Can't tell if this is a qemu bug or a kernel bug, so I asked both lists. Probably going to be ignored by both too, due to the list traffic volume. (The powerpc list isn't that high traffic, but it gets giant piles of inappropriate crap cc'd to it. Standard kernel-dev failure mode for the past decade, signal buried in noise.)

Huh, sh4 is doing a similar failure, except it doesn't "catch up" when you hit a key, the extra data has been discarded. (It got the sha1sum of the fetched file right when I typed it in manually though.)


April 25, 2022

Still congested.

I caught up on the git commit log, and have (unedited) release notes covering all the commits. Now I need to figure out what else should and should not go in the release, so I'm back to picking through the "git diff" in my toybox dirs. I have a conversion of nbd to work on nommu (it daemonizes which was using fork and should now use vfork(), but htat's not a trivial conversion because fork-and-continue is not available with vfork, you have to re-exec yourself). But it's not very tested?

And THIS reminds me that I was trying to come up with a nommu test environment for qemu, because having to pull out my turtle board and sneakernet files onto it via sd card is a slow and awkward way to test compared to mkroot and qemu. (Sure I can manually set CONFIG_MUSL_LIES_ABOUT_HAVING_FORK, but I never quite trust that isn't leaking something that won't work on a REAL nommu system, and having an actual nommu target in the mkroot list gives me testing organically.)

And specifically the nommu target I was trying to bring up a week or two back was coldfire, ala nommu m68k which qemu supported long before it supported full m68k.

... and my laptop decided to power down walking to the table, so instead of resuming from suspend I've lost all my open windows again.


April 24, 2022

On my way out to the table after sundown I noticed an abandoned walker on the corner of Red River and 38th, the owner of which was an old man standing in the middle of the intersection. I talked to him (his name's Glen) and he was VERY lost and tired, having been walking since morning. (With a walker, due to a back injury he's got surgery scheduled for next month.) His cell phone battery had died and he could barely see at night, he'd been trying to read the street signs up above the stoplights. He was trying to get to the driver's license office to deal with citation paperwork (related to nighttime "driving while unable to see") so he could move his truck out of the parking lot he'd had to leave it in after the police guy stopped him and told him not to drive until he sorted the citation.

I used my phone to work out the bus route to get back to his truck, which was only like 4 blocks from the driver's license office. (It's that one up on Lamar across from whatsaburger. He knew that area because he used to live in an apartment there years ago, but his truck was parked half a mile east of there and he went in the wrong direction when he headed out this morning, hence winding up 30 blocks away from where he needed to be at the end of the day.) I got on the bus with him (day passes are lovely), and took him to the relevant whataburger (open 24 hours), and from there we walked very very slowly east to his truck. (He only fell off the curb into the street twice. Once because his left wheel went off the edge of the 4 inch curb, and the second time because he tried to sit down on the curb where there was a storm drain in the gutter.)

As we talked, it turned out that the reason he's in this much trouble is his wife (ok, "girlfriend of 9 years") died suddenly 3 months ago, leaving him living on his own about a year away from qualifying for full Social Security. He has a brother, but he's out of town until monday (attending the funeral of a friend Glen couldn't go to because he had to work and deal with the citation on the truck or he can't get to the construction jobs he works at).

After a couple hours we finally made it to his truck, which he said he was fine sleeping in. (And it started fine; he was worried he'd drained the battery the previous night.) He said he can see well enough during the day to make it back to the driver's license office, and this time knows which direction it's in. He couldn't find his car charger in the first 5 minutes so I left him my worse phone battery and one of my zillion USB charger cables so his phone would be useful again in the morning.

I walked home from there, which took me by my house and I stopped in for a beverage and a bathroom break, and then wound up staying instead of heading out to the table again.

So I got my exercise, but missed out on about half the day's programming.


April 23, 2022

Huh. According to "git log --format=fuller" it looks like back on wednesday Bernhard Fischer commited a half-dozen back patches to the busybox git repo? (That's the guy who took over uClibc from Erik Andersen and let it die.) Still no word from Denys...

I bit the bullet and uploaded binary toolchains (cross and native, for all the targets I've gotten working so far except hexagon) to my website. I told Patreon about it on the 15th (my first locked post!), and will probably add a link to the FAQ or something as part of release prep. It's not too hard to figure out where they are, but I haven't been publicizing it yet because... ew. GPLv3. Ouch. Dowanna. (Yeah there's a source tarball-of-tarballs in the same directory, it's still horriffic to provide ANY exposed surface to the people who sued Mepis because even explicit endorsement from Mark Shuttleworth was not enough to let Mepis point to Ubuntu's servers for source packages their Ubuntu derivative hadn't modified: this one man garage operation was forced to mirror everything locally or get sued by the FSF.)

But my old aboriginal linux toolchains have fallen off the map (too old to build current kernels), and the LACK of proper toolchains I can point people at has become limiting. I need to do videos building toybox and mkroot and so on that kind of need "wget toolchain" as a step I can SHOW rather than "here's the procecure for building a zillion toolchains at once, building just one is kinda hinky because it builds an i686 musl toolchain first and then builds the other toolchains with that because they're static linked on the host for portability, and if you just run it without rather complex command line arguments it's going to build dozens of different toolchains and take a double digit number of hours to finish..."

Yeah, not exactly intro material. "wget this tarball, extract it, put it here" is 15 seconds of instruction and maybe 60 seconds of demonstration while explaining why. It's very limiting NOT to have them up somewhere.


April 22, 2022

I just hit a real world example of why "infrastructure in search of a user" is a bad thing. Years ago I implemented fileunderdir(file, dir) in lib/lib.c, on the theory that "take the absolute path of both and compare them" is an expensive but reliable way to avoid all the "../../.." and symlink escapes for things like web servers. That approach potentially kills a whole category of tricksy security nonsense. The current function only has one caller in the tree so far (cp.c), but I put it in lib anyway because I expected more to come. And here I am doing httpd, so I call fileunderdir() and...

Its API is wrong on multiple levels. When you "wget http://domain.com" that turns into a fetch of "/" which (once adjusted) is exactly EQUAL to the constraining path, and this wants the file to be UNDER the constraining path so an exact match doesn't work. Meaning I tried "wget https://127.0.0.1:8080" and got a 404 error, and had to debug why. Plus the function returns a malloc()ed string (to the abspath of the file) and I was treating its return value as "true or false" so was leaking that without even noticing. And changing the semantics of the existing function would subtly break cp in ways I'm not sure I have a test for yet...

So for the moment, I've added my own "static int isunder(dir, file);" to httpd with the semantics I want, and then I might want to move the other one into its only user (cp.c) and wait for a third instance to show up before trying to work out common semantics they'd all be happy with.


April 21, 2022

Hmmm. I think I need to rip scripts/config2help.c out.

I just built a defconfig-ish toybox in a mostly clean source dir, and netcat --help is not showing the "server" options, but when I "netcat -l" it does listen on a local port. The web help generated last release shows the extra options, so this broke recently-ish, but I really don't feel like spending a lot of effort debugging it?

Way back when, config2help would shuffle together help text for commands with config options that could change what sets of command line options it supported, but HAVING that sort of micromanagement was design idea left over from busybox. That project had the goal of being as small as possible at the expense of compatibility, and as I pushed it into being something you could use on a desktop (as Alpine Linux does), being able to yank features out of individual commands to save 35 bytes at a time got less interesting. Toybox never had as many config sub-options, and I've been slowly eliminating the ones I did implement, because I want toybox commands to behave consistently. "Does toybox cp support --preserve" should be a question you can answer just by knowing which version you're running.

The only two commands I'm spotting in menuconfig that still have these kind of sub-options are cat and netcat. In the case of cat it's because of the catv command, which is my fault (I took some bad advice). Nobody uses it, they just use cat -v, and google isn't even finding mention of its existence outside of busybox command lists. I'm not finding any scripts using it, no human writeup mentioning it, etc. Android's toybox config does not enable it, but DOES have -v in cat itself.

As for netcat, the argument was that offering server functionality was "dangerous", although... that's why iptables exists? And dialing out is just as dangerous? Android DOES enable netcat's server mode, and they're as paranoid as you get...

Sigh, I zapped both those two but there's still MKDIR_Z and such. (Commands with features enabled via build environment probes.) I think the correct fix here is to run the config help text through the preprocessor so USE_COMMAND(blah) macros can apply to it? What have we got left: MKNOD_Z, PASSWD_SAD, WGET_LIBTLS, ID_Z, MKDIR_Z, MKFIFO_Z, SORT_FLOAT...

Anyway, I went down this giant tangent because I got an initial httpd implementation to the point where I need to start testing it, but I only implemented inetd-style support so far (not a standalone daemon server engine), so I was gonna run it under netcat -l (which provides the same stdin/stdout network wrapper semantics) and I was doing --help to remember the server options and the options weren't there...


April 20, 2022

Strangely under the weather. Sore throat, sneezing, general lethargy. Not sure if this is a cold or if it's from all the smoke I inhaled last night. (We had marshmallows! We had hot dogs! We had wind shifting around and blowing smoke straight AT me no matter how many times I moved.)

Finally got wget promoted. And in order to properly test wget (in a regression test way instead of fetching stuff from landley.net that I put there via ssh), I need a web server I can run on loopback as part of wget.test. So of course I've added an httpd.c to pending and am filling it out...


April 19, 2022

Today's email reply I did NOT send, regarding my ongoing attempt to get posix to notice freadahead():

On 4/18/22 07:36, Chet Ramey wrote:
> On 4/18/22 12:53 AM, Rob Landley wrote:
>> https://github.com/coreutils/gnulib/blob/master/lib/freadahead.c
>
> So the gnulib folks looked at a bunch of different stdio implementations
> and used non-public (or at least non-standard) portions of the
> implementation to agument the stdio API.
>
> If that's what you want to do,

I thought starting "is this already in here and I missed it" was more polite than "you should add a thing that may already be there and I just haven't spotted it", but I'd assumed one would flow naturally into the other due to the nature of the venue. My mistake.

(And hey, I just saw Geoff's proposed workaround and I think I can use it. Still problematic in a threaded context, but what isn't?)

> propose adding freadahead to the standard.

Yes, I would love to. How?

http://www.opengroup.org/austin/papers/posix_faq.html does not say how, googling "how do I propose an addition to the posix standard" brought up the faq, the wikipedia page (also doesn't say) some rant from Richard Stallman (not even clicking)...

The front matter of the standard itself has "participants" (a credits roster), and before that an "updating IEEE standards" section that points to https://standards.ieee.org/ which is a glossy marketing brochure site but I dug down to https://standards.ieee.org/develop/maintaining-standard/errors/ which does not obviously help?

(I did not expect "ask on the list" to require monkey paw phrasing.)

> Or reimplement the gnulib work and accept that the stdio implementation
> can potentially change out from under you.

Or ask the bionic maintainer to add the API the musl maintainer already added (which is basically what's in dragonfly bsd and z/os already, and is also the api gnulib implements with that staircase), so two of the five C libraries I care about would export an explicit API for this, helping a cleaner way to do this gain more widespread availability...

> Current POSIX provides no help here.

No, Geoff had a good suggestion. I just hadn't checked my spam filter promptly.

>> I was just wondering if there was a _clean_ way to do it.
>
> OK. Do you think you've gotten an answer to that?

"Does posix have a way to do this?"

"No."

"It would be nice if it got standardized."

"Maybe it would, but that's a different question."

When I asked the C standards committee chair about adding this, and reported back his observation that the C spec does not have filehandles, and my interpretation that this makes posix the relevant context to add such a function which would be analogous to the existing fileno(), the reason was I wanted to establish that posix was the relevant context to add such a function.

This is because I was hoping to lay the groundwork to convince posix to add such a function.

>> The C99 guys point out they haven't got file descriptors and thus this would
>> logically belong in posix, for the same reason fileno() does. "But FILE *
>> doesn't have a way to fetch the file descriptor" was answered by adding
>> fileno(). That is ALSO grabbing an integer out of the guts of FILE *.
>
> Sure. And adding that to the standard would require the usual things, for
> which there's a process.

What is this process? Where does one find documentation on this process? Is it in one of the documents linked from https://www.opengroup.org/austin? It's not in https://www.opengroup.org/austin/docs/austin_14r2.pdf, and https://www.opengroup.org/austin/docs/austin_sd6.txt describes the defect reporting mechanism, is a new feature considered a defect? Ah yes, halfway through that document, it says:

> Everything starts out as a defect report, unless raised
> during a plenary session when the ORs are present.

(I wonder if by "plenary session" they mean the thursday conference calls?)

Ok, so by "there is a process" you meant "add it to http://www.austingroupbugs.net". That would be the process.

Which I should have known because who hasn't read down to line 138 of "Committee Maintenance Procedures for the Approved Standard" document of the "Committee Draft Development Procedures" section near the bottom of the righthand column of the project's about page. How did I miss it?

>> This exists. It would be nice if it got standardized.
>
> Maybe it would. But that's a different question.

Out of curiosity, why do _you_ think I brought this issue up on the posix mailing list specifically?

Anyway, I've got a workaround now, and I don't care about making it work with threads, so...

Thanks Geoff,

Rob


April 18, 2022

A tree fell into our back yard recently. It was growing up between our fence and the neighbors' and Tornado Warning Du Jour brought it down while I was up at Fade's. (That whole "storm of the century every few weeks this time of year" thing hasn't really gone away, we just did many thousands of dollars worth of landscaping after the fourth time the house flooded so the water part isn't as big a problem. We did have to repaint that corner of the house scoured bare by Hurricane Harvey.)

It's been hanging OVER our fence ever since, because the part where it broke is just a little above the fence and the tree itself is in our yard, and we didn't want to let it fall the rest of the way down and take a chunk out of said fence, so Fuzzy went to Home Despot and bought a saw, and spent several hours today performing topiary with extreme prejudice. And now we have a large pile of wood bits, so of course she put some bricks in a circle and tried burning some. (After trying to google for the tree type to make sure it wasn't one of the more obviously poisonous ones.)

Green wood does not actually burn all that well (the tree was alive until it came down, and yes this is the SECOND tree to rot and fall into our yard, at least this one wasn't full of bees), but we still had our dead christmas tree awaiting disposal, and THAT burns quite easily and gets the other wood going eventually. And we have zillions of fallen branches that have been accumulating because breaking them up to stuff in the compost bin or lawn bags was a chore.

We need to buy marshmallows. We only have the little mini ones you roast over a candle with a toothpick.


April 17, 2022

I'm looking at shipping container batteries.

No, not Ambri, they suck. Bog standard ivory tower academia needlessly complicating things, with a full decade of failure behind them. Solar projects storing energy in molten salt is an old idea and they were all very expensive failures. Putting the molten salt in a shipping container so you can have it downtown instead of out in the desert is NOT an improvement. Observing "you know, we could make big batteries out of shipping containers" ten years ago was laudable, but not THAT creative given that converting shipping containers into diesel generators was an established business model. Back in 2012 people were already living in them, using them for greenhouses, and one had already been converted into a starbucks. Deciding it was a good idea to FILL THEM WITH LAVA and park them downtown in population centers was very... grey haired white man with academic titles fundraising from silicon valley. (Where accidentally reinventing things that already exist is a regular event. A place where "new to me personally" = revolutionary.)

Anyway, the ongoing problem is that solar panel deployments are storage limited far earlier than they should be, because curtailment triggers loss aversion, one of many well known bugs in the human brain. We won't install solar panels if we're sometimes "wasting" their output, because monkey brain says it's better to let rain fall on pavement than install water barrels that might sometimes overflow. Failure to capture EVERYTHING is wasteful, so don't capture ANYTHING. Brilliant! If you never even start, you can't do an imperfect job!

So we need batteries and lots of them. The chronic first mover in the psychological hype space deploying a half-assed solution in perpetual development has of course installed space heaters, which as usual have plenty of competitors trying to do exactly the same thing. GE has its own shipping container full of lithium cells called the reservoir system, which was also announced years ago and then silence, but it technically still exists so yay I guess? Samsung's stab at this was via partnership with a chinese company so I assume all their IP was stolen and the venture folded when their partner ran off with the money, but I can't say I've actually checked. That's just how most chinese partnerships go...

But all those guys are doing lithium, which got popular providing PORTABLE power. It's light and energy dense, explodes on contact with water (or various other things), and is fairly rare. Annually we only mine about 30 times as much lithium as gold. We mine over 6800 times as much copper as gold each year, and copper is still expensive enough for people to steal pipes and wires out of empty buildings.

But when you're filling shipping containers with batteries, and can stack said shipping containers on top of each other, weight and density are far less of an issue than cost and durability. Something cheap and plentiful, which doesn't burn (let alone explode), can be recharged thousands of times without losing capacity, and isn't made out of toxic waste or blood diamonds.

Which is why I'm interested in Form Energy, the people making shipping container batteries out of iron and salt water. Iron gives off electricity when it rusts, and if you run a curent through the rust it gives up the oxygen and turns back into iron. This is how aircraft carriers made out of steel avoid rusting: they run a low-level electric current through the hull. It's called "impressed curent cathodic protection" and according to wikipedia[citation needed] they've been doing it via batteries since 1824 and with allgator clamps from a generator since 1928.

Form Energy makes "iron air" batteries using big piles of iron pellets (which are about as explosive as metallic iron usually is), and salt water (which will actually put OUT a fire if poured on it, and if it wasn't safe to pour straight into the sewers we couldn't use road salt to de-ice roads in the winter). We make buildings out of steel girders and live in them, so I'm not too worried about toxicity, nor about sourcing the materials. Any modern really big cheap human thing probably has a lot of iron in it. Even concrete is full of rebar.

Form Energy more or less fills a shipping container with PVC pipe, iron pellets, water, and road salt. To speed up the reaction they add a little vinegar to acidify the liquid, and wire the whole thing up to accept and output electricity. The reaction still isn't very fast so they make it big, and you get a battery that takes 10 hours to fully charge and 10 hours to fully discharge which is what you WANT for municipal battery applications. If you short it across the terminals whatever you're shorting it with gets hot, but the battery itself isn't likely to care. (It's iron soaked with salt water. What's it gonna do? "Rust faster!")

Pretty much all batteries have side reactions (when you recharge them the energy dosn't ALL go into reversing the discharge reaction, it makes Weird Other Chemicals too), but the "salt, water, iron, and air" reaction is simple enough (especially since they're using ROAD salt, not table salt, so you haven't got any chlorine involved) that they can slap an upscale pool/aquarium filter on there and recycle it all back into the original chemicals without too much effort. (They call this bit the "kidneys" and it's where all their patents are.) The main other thing they care about is temperature control (because water has to stay liquid: even in death valley it's not going to get anywhere near boiling and the water is FULL OF SALT so it can get pretty cold without freezing, but minnesota winter can still manage to freeze the pipes and the reaction would slow down in the cold anyway, so you're going to want to heat it a bit in the cold, or at least insulate the container...)

The limit on how much capacity one of these can store is apparently the weight of the iron: it's really big and really heavy. The maximum legal weight of a shipping container is around 20 thousand kilograms (petty much the same for 20 foot or 40 foot ones), so while they CAN make individual batteries that last longer than 10 hours the containers they're in wouldn't be portable or stackable. But the duration of ONE battery isn't really limiting: if you have twenty batteries that last an hour each, you have twenty hours of batteries. The reason not to do that with the lithium ones was the space, the expense, the limited manufacturing capacity due to the shortage of materials to make them from, and of course the tendency for even non-Tesla lithium to catch fire in any sort of quantity.

I'm glossing over some details, but less than with most battery technologies. "Let's use 200 year old chemistry plus pool supplies to make batteries out of a material so abundant we build oil tankers and skyscrapers out of it, which wouldn't burn if you threw it on a bonfire and left it there overnight". Seems like a win to me.

Anyway, here's a video with a tour of their factory, and a talk given by one of the women who run the company talking about how fast they're hiring and growing. (They've got a bunch of women in actual positions of authority.)


April 16, 2022

My attempt to organize an explanation of the toybox infrastructure has resulted in me outlining a complete C tutorial (and checking it against other tutorials to see what I've missed). Hands up everybody surprised by this.

So I now have a whole script for a quick and dirty C tutorial, which I'm pretty sure I could cover in a one hour talk at ELC or similar. Can I do it in half an hour? Hmmm...

Of course I haven't really explained ELF file format and Linux process attributes and so on... ("man 2 execve" and "man 2 fork" have lists of process atributes, "man 7 environ" has another chunk...) Or walked anybody through the kernel's ELF loader. I already have gratuitous mount documentation lying around...

Ahem.

Meanwhile, the representative of "512 solar" came by to see my roof (I was napping on the couch, he never called or knocked during this "visit" and only texted me afterwards), and claimed it is impossible for us to have solar panels because our property has trees on it. I mentioned I was looking into taking two of them down over the best southern facing exposure, and he said it didn't matter, his mind was made up, solar panels would explode or something if put on our roof. As far as I can tell, this person did not go into the back yard to see the southern facing part of our roof and the yard that only has trees around the edges. I know this because the way the leaves are piled up in front of the gate at the moment totally jams it (and said leaves have not been disturbed anyway), by far the easiest way to the back yard is through the house. He just didn't want to bother with our house at all.

Yeah I know, I should really pay off the home equity loan before fiddling with that. (And when Fade finishes her dissertation who knows where she'll get a job, we may sell this place and move.) But I'm still disappointed at a rooftop solar panel installer that doesn't want to finish the quote process. (Yes demand surge after the ukraine invasion has caused the price of solar panels and batteries to go UP for the first time in decades, which even Trump's tariffs couldn't manage. But refusing to do your job without even saying "I've changed my mind and don't want to do this one" is just unprofessional.)

Speaking of which, the preiliminary quote they gave us was going to use panels from "Mission Solar" in San Antonio, which has plenty of nice puff pieces about how they manufacture their own panels... except they don't. They USED to back in 2014 (although it didn't say where they got the silicon wafers), but in 2016 they abandoned that business model and started buying cheap asian solar cells like everybody else. Then in 2018 they "doubled production" (back to pre-layoff levels perhaps?) but as that article clarifies they hadn't gone back to making their own cells, they were basically assembling Ikea flat packs from china like everybody else. And then in 2021 they were hoping Biden would expand their business, but nothing about going back to manufacturing their own cells. Just assembling them into panels.

If the whole "china supporting russia and threatening taiwan while having endless covid lockdowns because their vaccine can't handle variants" thing goes sour, does Mission Solar have a sustainable business model? I'd love to hear that they do, but their website does not answer this question. (In fact they list solar cells as a "raw material", which would be a bit like Intel listing silicon chips as a raw material and saying they just package Samsung's output into little ceramic containers with pins. Which sure is a viable business model but it's not the same thing as IC design and photolithography. Outsourcing your core business is not a sign of health.)

(Oddly, this is exactly the problem Russia is having with sanctions: they THOUGHT they made their own stuff but it's full of inputs they can't reproduce, there was an excellent thread about that last month. Possibly the most relevant tweet in the thread is this one, about how their drive to BECOME independent turned into them lying to themselves and hiding the origins of stuff.)


April 15, 2022

Nope, adding gzip support to wget would be trivial if the http standads bureaucracy hadn't hallucinated "chunked" encoding. Because it can return anything "chunked" and can gzip the chunked, and making gzip and chunked work together turns out to be stupidly difficult. (Piping it to zcat works fine when it isn't chunked, does not work when it is. That's just broken.)

I grepped busybox wget.c (which is 22 years old at this point) for the strings "gzip" or "deflate", which it does not contain. And their wget hasn't got -d but -o log.txt shows an uncompressed transfer. Busybox wget has not gone unmodified for 22 years, but if it's been in USE for 22 years without growing support for this feature yet, nobody cares much and I'm pretty happy skipping it.

(This might also explain why pulling pages from cnn.com or lwn.net could never give me "deflate" encoding, only gzip. The spec says it exists, but nobody's bothered to implement it.)

Design question: if I escape an URL I'm given so space becomes %20 and such, what do I do if I'm given a URL that already has escapes in it? Because % is an illegal character that needs escaping...

Hmmm, looks like the server unescapes whatever it's given.


April 14, 2022

The lib/deflate.c plumbing I wrote strongly assumes it's reading from an fd and writing to an fd. This is to avoid trying to return and restart in the middle of stuff, the "get next X bits" and "write next X bits" funtions just handle buffers emptying and filing up behind the scenes because the data has somewhere to go.

What I could do is make empty() and refill() callback functions that get called instead of the read() and write(), and then if either is non-NULL call that instead of reading the fd. (Because the fd legitimately could be zero, reading from stdin is a thing.) But I'm not sure that's LESS awkward that just forking a background process to pipe the data through? Especially if it's a MAYFORK so it avoids the exec. (Which on Linux with MMU is 90% of the overhead, the fork itself is close to free. On nommu fork requires an exec and there still isn't a "re-exec-self" syscall, despite me repeatedly asking on linux-kernel, so it's dependent on /proc/self/exe being mounted.)

One problem the fork-and-fd approach is there's no standard zlib or deflate command line utility to pipe stuff through the way there is for gzip format. The obvious thing to do is add flags to zcat, but non-toybox zcat wouldn't have them. (Once upon a time I was trying to turn "decompress" into the general purpose swiss army knife decompressor with command line options selecting the type for compression and autodetecting decompression, posix be damned since their "deflate" is for the .Z algorithm that was abandoned in the 1980s for patent reasons and less efficient than deflate anyway... but that kind of fell by the wayside a while ago.)

Another problem is wget.c doesn't provide a clean filehandle to read from, because https support. Instead wget_read() is a wrapper around three different functions (read(), tls_read(), or SSL_read()), two of which take a structure instead of a filehandle as their "from" argument. The design for adding decompression support to wget with https is trying very hard to boil down to for (;;) { weird_read_func(in, buf, len); weird_write_func(out, buf, len); } in a loop, which is... not ideal? Gratuitous data shoveling through a buffer I'm both the producer into and consumer from, without even reblocking the data. Hmmm...


April 13, 2022

Called the tax lady and had her file an extension. Still haven't seen a dime from Google, and the piecemeal contracts I've worked over the past 6 months have not included "money for taxes". Well, they did tax witholding, but not "pay the backlog from Jeff not witholding"? I took zero deductions to try to catch up, but I haven't gotten a tax REFEND since before I first worked for Jeff, and I gotta pay the preparer too. I REALLY don't want to have to sell stock or something to pay taxes, because I have no idea what my cost basis on years of Divident Reinvestment Plan purchases is? (I don't want to file tax paperwork on something I sell to pay taxes. The Sorceror's Apprentice music from Fantasia would start playing on a financial level. I can if I have to, but... sigh. Waiting for this would be a lot easier if I hadn't ridden down yet another start-up and lost tens of thousands of dollars to the house repeatedly flooding. I am REALLY appreciating patreon right now.)

Moritz is working on his git implementation, and git is basically a multiplexer with a bunch of subcommands. I want to reuse the toybox mutiplexer plumbing so "git clone" can work like "toybox ls", which seems straightforward. In main.c, split toy_find() into two functions, with the new toy_list_find() taking the list to search and the length as arguments. (The old name would still prefix-match "toybox" at the start and pass toy_list_find() an adjusted list to skip that out of order first entry.)

The git_main() call then looks something like:

struct toy_list *which, toy_list[] = {
  {"clone", git_clone_main, "<1", 0},
  {"init", git_init_main, 0, 0}
};

which = toy_list_find(name, toy_list, ARRAY_LEN(toy_list));
toy_init(which, toys.optargs);

Which assumes the git top level command's option string has the "stop at first non-option agument" thing, because "git -C dir pull" is a thing...

But the problem is 1) if this parses the new option string how do I get corresponding FLAG() macros? 2) help text: "git --help pull" and "git pull --help" are both a thing. (There's also "git help pull"...)

To get the flag macros and help, I need the subcommands to have NEWTOY() macros and config blocks like normal. I need them to exist in the main toy_list[], but not be externally visible. And THAT means they're basically shell builtins. I probably need a new TOYFLAG_SUBCOMMAND flag which includes TOYFLAG_NOFORK (so they don't show up in the "toybox" output), but also a way to tell the shell not to make them available as shell builtins. (can't be called directly from the shell and don't show up in "help").

(There's a lot of "I already did this but it's not generic enough...")

Ok, probably if I chord together NOFORK and MAYFORK, this can give me the behavior I want. If all the subcommands have a common prefix, I can make a function that collects them together into a temporary list and searches just that list, but returns their position in the original list.

No, wrong: I can put the search function back how it was because this doesn't need binary search. THAT was an optimization to minimize the latency of running toybox commands in the common entry path. This is just "search the list for the prefixed entries" via brute force. I care a lot how fast "true" runs in the shell, but don't care at all if "git pull" takes an extra half a millisecond. A shell script can run a hundred commands per second, fractions of milliseconds add up there.


April 12, 2022

Working on toybox release notes, but as I go through the commit list I keep finding half-finished things that need more work.

Also working on toybox video scripts. What I really need to film is an introduction to the infrastructure, which is a big topic that's hard to break down into bite size chunks. I've tried multiple times already, and keep writing more or less the same script over and over, and not being happy enough with it to record. (This SHOULD be simple, both to explain and to use...)

I kind of want to approach it with a video walking through "hello world" (toys/example/hello.c) and explaining the comment block at the start (top line describing the file, copyright line, links to standards*, NEWTOY() line(s)* have USE() guards* and three arguments each (command name*, option string*, flags*), kconfig block*), then include toys.h*, GLOBALS()*, and finally command_main()*.

But each asterisk above is a can of worms requiring its own video. For example, the NEWTOY lines are SORT of collected via grep -w 'NEWTOY(' toys/*/*.c > generated/newtoys.h except they're sorted in alphabetical order by "first argument to newtoy" and you can't use sort -t -k for that because the relevant field starts with an open parentheses and ends with a comma, and -t only lets you choose one delimiter, so instead we use sed magic before and after the sort to copy the relevant field to the start of the string and then strip it back off again. Another can of worms is how the "command name" argument from newtoy is used as both a string (toys.which->name) and a symbol name (the command_main function pointer) to populate toy_list[] in main.c via macro magic...

I kinda have to explain how generated/*.h gets created by grepping data out of each command's source file. (Except it's usually sed, not grep, because I modify the data slightly as it's extracted.) There's five generated/*.h files, two Config.* files...

What is instlist doing there, that should go in unstripped. (Goes off to poke at scripts/*.sh, it's used in change.sh and install.sh neither of which are using the $UNSTRIPPED variable, because install.sh. is including ./configure not scripts/portability.sh, although another problem is UNSTRIPPED is defined in scripts/make.sh not ./configure, and while I'm at it ASAN is used in both configure and portability.sh and half what it's defining overlaps. And scripts/change.sh is not including portability.sh OR ./configure, because it's calling scripts/make.sh and can porability.sh be included twice without glitching stuff? Hmmm, yes if I unset ASAN after consuming it...

Ahem. This is another reason doing this takes so long. Documenting stuff always leads to fixing things rather than trying to explain them.

ANYWAY, at some point I need to do a birds and the bees talk on how command_main() gets called, walking through main.c from the top level main() to the command_main() receiving control. Including argument parsing and toy_find() and toy_init() and so on. Plus the standalone non-multiplexer codepath too.

And then the NEXT thing to explain is lib/*.c. How many commands are currently exported from there?

$ sed '/^$/d;/^[ {}/#]/d;:r;/, *$/{N;s/\n//;s/ [ ]*/ /g;br};/) *$/p;d' lib/*.c | grep -v ^static | wc -l
283

Hmmm, let's organize that a bit better...

for i in $(ls lib/*.c | sort); do echo -e "\n--- $i\n"; sed '/^$/d;/^[ {}/#]/d;:r;/, *$/{N;s/\n//;s/ [ ]*/ /g;br};/^static/d;/) *$/p;d' $i; done

Right. EACH of those files is a video unto itself. You'd think lib/args.c would be easy with only one entry point, but that one's a doozie, however that's spelled. And these command lists are moving targets enough that I'm reluctant to record said videos without a mechanism in place to keep them up to date? (Re-recording them and editing the playlists? Some sort of notifiation to rewatch, except if 90% of it's still the same... Hmmm.)


April 11, 2022

While looking for something else, I stumbled across Hacker News noticing toybox exists again 6 months ago. And half the comments are Gavin Howard going "But my bc! It's the most important thing toybox could possibly have! And toybox is useless because it hasn't been paid attention to!"

Makes me want to remove bc and start over from scratch when I get around to that. Probably easier than trying to trim SIX THOUSAND lines of code down to something manageable. (I just checked, and the one in busybox is 7500 lines. Which he considers an improvement.)

Back when it was first presented to me, the development team for this bc exploded into a cloud of drama. And out of that drama stepped a guy who claimed someone ELSE was the source of drama, yet stirs up a cloud of drama in the comments here. (Drama magnets are always surrounded by other people who are At Fault For All The Drama, and they constantly go find new groups of people where Drama Inexplicably Occurs as soon as they show up. Dunno how much of this is drama magnets finding each other, and how much... isn't.)

Honestly, the most useful part of the whole process was probably the test suite. Except the only USE of bc I've noticed is the one script the kernel uses, because Peter Anvin blocked my perl removal patch with a rewrite in bc. Before which neither Linux From Scratch nor Gentoo had bc in their base install, because it's obsolete and useless otherwise. Sure it's in posix. So are the 1977 edition of Fortran, the compress utility for the .Z extension which was abandoned due to patents that expired in 2004, an ancient source control system that's five generations out of date (sccs was replaced by rcs, which was replaced by cvs, which was replaced by svn, which was replaced by git, and Linux switched to git SIXTEEN YEARS AGO and it's gone through multiple user interfaces and file formats since then), and of course a bunch of commands for managing batch job queues...

Posix is supposedly coming out with Issue 8 soon. I doubt they'll bring "tar" back (stop trying to make "pax" happen), or cpio (still the basis for RPM and initramfs)...


April 10, 2022

Huh. Busybox hasn't commited anything to the repo for over a month, nor has Denys posted to the list. I hope he's all right. (His country is still being invaded, although he was living elsewhere in Europe last I checked.)

Yay, Rich finally got the 1.2.3 musl release out. Woo! I should rebuild the toolchains and then bite the bullet and upload the binaries to my website.


April 9, 2022

Following up on:

As for adding coldfire support, let's see... google for "qemu coldfire" first hit is https://qemu.readthedocs.io/en/latest/system/target-m68k.html which says the default board in qemu-system-m68k is coldfire and has run uclinux. There's a defconfig for it (arch/m68k/configs/m5208evb_defconfig) so:

$ make ARCH=m68k m5208evb_defconfig
...
$ CROSS_COMPILE=m68k-linux-musl- make ARCH-m68k
...
$ qemu-system-m68k -nographic -kernel vmlinux

Hey, console output on serial using my existing m68k toolchain. Good sign. Ok, let's see, can I get a userspace...

I did a quick stab at trying to get fdpic support working. (I haven't got an fdpic toolchain for the target, but the loader should be able to handle ELF PIE binaries, if somewhat less efficiently.)

The next steps to trying to get a mkroot target for it would be:

$ cp .config walrus
$ ARCH=m68k ~/aboriginal/more/miniconfig.sh
Calculating mini.config...
[1] 135/135 lines 3260 bytes 100%
$ grep -v '=y$' mini.config
CONFIG_LOG_BUF_SHIFT=14
CONFIG_RAMBASE=0x40000000
CONFIG_RAMSIZE=0x2000000
CONFIG_VECTORBASE=0x40000000
CONFIG_KERNELBASE=0x40020000
CONFIG_SERIAL_MCF_BAUDRATE=115200
CONFIG_BOOTPARAM_STRING="root=/dev/mtdblock0"

I.E. make ARCH=blah relevant_defconfig, digest it down to a miniconfig, and pull out the config symbols that AREN'T boolean. (No modules here so I can skip dealing with them; they can't be needed to boot an initramfs but may be needed for things like ethernet support or mounting block devices...)

Those symbols get glued together into the KERNEL_CONFIG= setting in the mkroot target. I probably don't need to change the LOG_BUF size from whatever the default is... where is that, init/Kconfig and it's 17, which is 128k. Meh, close enough. I probably don't need to change the baudrate (it's virtual, qemu won't care) and the bootparam string needs revisiting. (I'm not faking up an mtdblock device for qemu, I'm booting initramfs. But if I _do_ want a block device for native builds...)

As for the config symbols that are enabled, let's filter out the ones all targets set:

$ sed -n 's/CONFIG_\([^=]*\)=y$/\1/p' mini.config | egrep -v "^($(grep -o 'BINFMT_ELF,[^ ]*' ~/toybox/toybox/scripts/mkroot.sh | tr , '|'))\$"

And now I'm working through that config symbol list, to see which ones are relevant. Currently I'm trying to figure out why taking out why switching off CONFIG_EXPERT makes all serial console output go away. (I have a sneaking suspicion I've tracked this issue down before. It feels familiar...)


April 8, 2022

Jeff's finishing up the first J-core ASIC (first round is just a test chip in a shuttle, maybe 50 chips resulting and they're proof of concept rather than general purpose), and needed to fill an extra 100 bytes of ROM space with chip demo, so I tried to come up with the smallest CRC32 implementation I could so the ROM can checksum itself and then flash the LEDs in sequence to indicate the resulting bits. (We can't fit the proper CPU test program in the ROM space we've got, and this chip doesn't have serial output to display the results anyway.)

unsigned crc32(unsigned char *data, unsigned len)
{
  unsigned int i, j, k, c;

  // Calculate little endian crc32 with pre and postinversion
  c = ~0;
  for (i = 0; i<len; i++) {
    k = (c^data[i])&0xff;
    for (j = 8; j; j--) k = (k&1) ? (k>>1)^0xEDB88320 : k>>1;
    c = k^(c>>8);
  }

  return ~c;
}

This produces the same as the "crc32" command line program, but calculates each "table" entry as needed because there isn't a spare 1024 bytes of sram. Inefficient use of CPU but it's only got to grind through a couple kilobytes of ROM, and everything it's doing should fit in registers.


April 7, 2022

Scientists researching indoor air pollution from gas stoves found it was so bad they switched their own homes to electric induction. (Quote: "All the researchers were pretty horrified.")

Sigh. Now that I've forced the USB phone tether to stop grabbing a gratuitous ipv6 address (by switching off ipv6 support in the kernel)... Devuan's udhcp daemon keeps exiting? My address expires and I have to re-run the daemon. I suspect it's throwing some sort of stupid error I can't see (because daemon) and exiting? Buggy piece of crap, I need to replace it with toybox's dhcp client so I can fix any problems. (Such as -4 being ignored and the darn thing grabbing an ipv6 address I did not ask for and ACTIVELY DO NOT WANT because it SABOTAGES SSH'S MAN IN THE MIDDLE PREVENTION.)

Huh. Well, there appear to be something like 30 instances of dhclient running, and when I killed them all and ran a new "dhclient -d -4 usb0" (because dhclient said -d4 was an unknown argument: facepalm), it complained "RTNETLINK answers: file exists" which is the error it gives if the interface is already up with an address... and then says renewal in 1624 seconds. (Um, ok? So... did it or did it not work? I thought that error made it exit, and that I had to run it again. Also, when I unplug the USB so the interface goes away, this does NOT make dhclient exit? It just leaves an old copy talking to a no longer existing device?)

Not opening the dhclient can of worms right now, I'm trying to finish cleaning up "wget" so it can go into this release. Closing tabs, not opening new ones...


April 6, 2022

Ok, the Google guys say that the middleman organization invoiced them and was approved on March 23, and given 45 day payment terms they should have the money by Star Wars Day early next month, at which point I can submit an invoice to the middleman and finally get paid for working on toybox! Woo!

Which means I shouldn't have to get another contract to keep the lights on, I can start tackling the toybox todo backlog now and invoice for this month at the end of the month. (Yay!)

Probably starting with getting a release out, since that's hugely overdue. (I've put some time into it this past week but I'm like a third of the way there...)


April 5, 2022

Japan points out how much of the "carbon capture" technology silicon valley techbros are working on belongs in the "you've reinvented trains, badly" bucket because plants are already amazingly good at sucking carbon out of the air. (They're solar powered and everything.) All we have to do is process the resulting biomass to convert that carbon into whatever format we want it to be in, and "mixing it into dirt" turns out to grab rather a lot for potentially quite a long time.

Sigh, I need to resubscribe to Linux Weekly News. I'm terrible about spending money online. (I know too much about how the sausage is made to trust typing payment information into any keyboard. And my phone does NOT have authorization to spend money. I generally ask Fade to do it for me, which I KNOW is silly and yet... I should buy gift cards or something.)

Anyway, I am reminded of this because I'm waiting to read the updated story about the Debian /usr merge, which is a follow-up on their story from six years ago, which is apparently my fault. Yet ANOTHER case where I yodeled in an avalanche zone and then couldn't steer the result. (The high water mark of which is probably still the patch penguin thread, which just goes to show that "Don't ask questions, post errors" sometimes really works. Here's an old news article about it, here it's linked from a syllabus, here's somebody's disseration in which I am at least 3 people. (Mostly just "Landley", sometimes Rob or Robert, on pages 36 and 37 it's "Langley", and page 173 quotes a Roger Landley.) It's that old "Don't ask questions, post errors" thing.)

Anyway, back in the Yellowbox days symlinking /bin and such under /usr let me mount a read only zisofs image on /usr that contained more or less the entire operating system. (Zisofs was a compressed CDROM image format that predated squashfs. The read only root filesystem was partly a security measure: I wanted something you couldn't mount -o remount,rw if you'd partially cracked the system and "isofs" fit the bill.) My yellowbox init was doing unnatural things that Linux didn't quite support yet, since then the kernel has grown "mount --move" and lazy unmounts. (And yes, my habit of posting things to lkml that get completely ignored goes back quite a ways, doesn't it?)

Anyway, I had the luxury to merge /bin with /usr/bin in 2002 because I didn't have a package management system to worry about. "Where files live" is kind of important to those, and the various packages all have to agree or it gets weird. Debian has tens of thousands of packages and decades of historical cruft accumulation. Plus there are hardwired "#!/bin/bash" in zillions of scripts, so the symlinks aren't going away.


April 4, 2022

URGH, I HATE HATE HATE the way ipv6 lobotomizes ssh's man-in-the-middle protection, so that every "git push" to github does:

Warning: Permanently added the RSA host key for IP address '2607:7700:0:2d:0:1:c01e:ff71' to the list of known hosts

I've been running "dhclient -4 usb0" with phone tethering to disable that, but apparently devuan "upgraded" it so that it requests both address types at the same time now even when you EXPLICITLY TELL IT NOT TO.

So now to stop it, I have to do:

sysctl -w net.ipv6.conf.all.disable_ipv6=1
sysctl -w net.ipv6.conf.default.disable_ipv6=1

Only then does ssh uses a consistent address which DOES NOT GET AUTO ADDED WITHOUT ASKING ME EVERY TIME. (A reminder that the 0.0.0.0/8 and 240.0.0.0/4 block and most of the multicast address space can totally be released for general use, as Dave Taht as trying to do. That's somewhere around 300 million more ipv4 addresses freed up without redistributing any of the existing ones. (And 127.0.0.1 does NOT need netmask 255.0.0.0, there's another 16 million...) But no, the IPv6 loons see doing that as an existential threat and mobilize to block it. They can't win unless everone else loses, because a proper fix to the problem renders them unnecessary. IPv6 is such a clusterfsck that the only way for it to succeed is for IPv4 to fail, so it must be sabotaged.


April 3, 2022

Attempting to put together scripts to record videos from. The problem of needing to explain everything before explaining everything else remains, but I suppose that's more a "what order do I put video lists in" rather than what order I RECORD material in.

Tempted to try to optimize "factor", but the largest positive signed 64 bit prime seems to be:

$ time toybox factor $(((1<<63)-25))
9223372036854775783: 9223372036854775783

Which takes 14 seconds to (fail to) factor on my 10 year old laptop. That's probably good enough.

(This laptop is newer than that to ME, because I bought four of them from surplus clearance so I have spares if it breaks, but this model was introduced a decade ago now and all I did was stick more memory and a big SSD into it. The CPU is a quad core-i5 at 2.7ghz. Faster than a speeding raspberry pi at 1.5ghz, but not by all THAT much. Factor may not be snappy in a loop, but it's USABLE on any 64 bit value and isn't using a bignum library so that's all it can do anyway.)


April 2, 2022

I get emails:

> Good morning!
> Attn. Mr Rob Landley :
> Recently my Huawei Nova 2 Plus Android phone started to show me some signs
> of being rooted in a very different way with some elements like a toybox
> /toolbox as I'm gonna show you : Please take a look to all of the images
> above attached.
> All I hope Mr Landley is to receive some free advise on the issue I'm
> dealing with I mean how to get rid of this nasty problem and is ruining my
> phone I mean I can not run apps mainly those of banking services and stuff
> like that!
> Thanks in advance for any can provide to help me out with this issue and
> resuming my activities normally.
> Best regards
> Sergio XXXXX
>
> Sent from my Huawei Mobile

I make one specific component of the system, which is incorporated by Google into the Android Open Source Project with over a thousand other packages, and then the version you're using was extensively modified by Huawei (a Chinese company).

This is a bit like your car getting infested with ants, and emailing the designer of your car's fuel tank on the theory the car can't run without a fuel tank so I must know what to do about ants.

Have you tried a factory reset?

Rob


April 1, 2022

A few months back Joe Biden tried to change the IRS so any account with $600 in it must report every single transaction (no matter how small) to federal authorities. (This neatly disarmed the "double the IRS's budget so they can go after Billionaires" push from the left, by offering to double the IRS's budget so they can go after the poor. This doubling is of course undoing decades of cuts by the GOP reducing the IRS to a shadow of its former self, it's shrunk 20% just since 2010 because the rich hate taxes and the GOP wants to shrink the federal government to the point they can "drown it in the bathtub". This would allow billionaires to become kings and keep slaves again, the basic goal of plutocracy.)

Now Boomer Biden's taking his next step in the War on Cash: he wants all online sellers to provide the government with Photo ID to Protect the Children... no wait (checks notes) because intellectual property law.

A recent Paul Krugman Column included the paragraph:

I still sometimes encounter people who say that we live in a digital age, so we should be using digital money. But we already do! Like many people, I pay for most things by clicking a mouse, tapping my debit card or pressing a button on my phone. I used to keep singles in my wallet to buy fruit and vegetables from New York’s ubiquitous sidewalk stands, but these days even they often accept Venmo.

And later in the same column:

And we shouldn’t discount the importance of illegal activity. There’s about $1.6 trillion worth of $100 bills in circulation — 80 percent of all U.S. currency — even though large-denomination bills are very hard for ordinary consumers to spend. What do you think people are doing with all those Benjamins?

So 69 year old Krugman notes that the ability to make anonymous financial transactions on a daily basis is going away, and tuts that using cash for any reason is inherently suspicious (only criminals use cash), and never questions whether that's a good thing.

Meanwhile, the GOP is empowering vigilantes to sue anyone who "assists" someone to have an abortion, and make traveling across state lines to have one a felony. (The GOP has always relied on vigilanties, they just used to wear white hoods and lynch people instead of suing them.) Even if the credit card company or regulatory agency collecting this data ISN'T captured directly by nazis, the next data leak will give a complete list of donors to Planned Parenthood or the DNC so they know who to target with vigilante armament bill du jour. And Biden's refusal to make the Supreme Court "go to eleven" means said laws proliferate and remain in force, while Merrick Garland lets all the statutes of limitations expire on rich white men who might otherwise be prosecuted ever for anything.

Krugman's a Boomer, which means baked into his worldview is the assumption that anything they inherited has always been like this therefore it requires no maintenance and no amount of abuse Boomers heap upon it can possibly damage it in any way. The Boomer world is eternal and unchanging and required no effort (from a Boomer) to be like this in the first place. Therefore nothing important is ever really destroyed or goes extinct, no matter how incompetent those lead-poisoned narcisists are at dunning-krugering their through life. Impostor syndrome only happens to people with the slightest trace of self-awareness.


March 31, 2022

Followed Fade to her graduate office, which is a nice quiet place to record videos from when nobody else is here. Except I need to prepare scripts. And trying to explain stuff keeps leading me off on tangents to fix things I've just reexamined and gone "that's a rough edge, could be better"... Plus I'm leaving myself a lot of notes like:

Can prlimit query stack size (ala ulimit) for comparison in main() instead of hardwired stack limit for command recursion? Is that worth doing, and does it work on nommu where stack size is a per-process attribute in the FDPIC ELF headers? Does prlimit show the correct process-local stack size on nommu, and if not should I poke Rich to fix the j-core kernel...


March 30, 2022

Finished the board bringup contract yesterday: the board is essentially up now. All three issues were hardware, but were easily fixable (in hardware) once diagnosed. So I have once again done a software project which resulted in zero lines of code, and yet fixed the problem. (Yay?)

The most interesting was that the ATWILC3000 and ATWINC3400 wifi chips are pin-compatible, so when one was out of stock (ongoing chip shortage) they substituted the other. Except being electrically compatible does not make them REMOTELY the same chip: the 3000 is a standard-ish WIFI adapter with a linux driver in the vanilla tree, and the 3400 is a router-on-a-chip that boots its own multitasking OS from built-in SPI flash and doesn't just have its own TCP/IP stack but runs its own web server, does its own crypto, it runs SNTP to set its onboard clock so it can check https certificate expiration... The 3400 is designed to act as a router for deeply embedded arduino-style systems too small to do their own networking, and its manual does not contain the word "Linux" because it's not FOR that.

The fix was to find 3000s in stock at digikey and swap the chips back. In THEORY we might have botched up some variant of a PPP connection to let Linux use the 3400 as a gateway to talk to the net through an extra hop, but that would be a HARDER hardware change because they hadn't wired up the serial port pins (which the 3000 only uses for testing).

There was a whole lot of head scratching along the way to figure out why trying to load 3000 firmware into the 3400 providing the WEIRDEST driver failures. (Oh, the driver will panic the kernel if the firmware isn't in the format it expects, because it only checks that the SOURCE ranges it's copying data from fit, not that the TARGET ranges it's copying data to fit. Needless to say the 3400's SPI flash image and the 3000's SRAM image aren't quite compatible.) Whole lot of sticking printfs into the wifi driver to answer The Wrong Questions. Searching for keywords in the giant programming manual and not finding them (because they weren't there) was also frustrating. Reading the sales brochure is what finally clarified things: oh, wait, this is the wrong category of product. (And it's pin compatible? Really? Yes, yes it is... Why?)

When you have a really FRUSTRATING debugging session, often you've missed something more fundamental/obvious than you're examining. This was an "I am in the wrong house" level mistaken assumption. Took quite some backing up until I could see it.


March 29, 2022

My laptop kept powering itself back up fifteen seconds after suspending, so I went through the dance of closing every window in all eight desktops until I could actually shut the thing down. This involved scribbling a note about the state of many open terminal windows into a text file to track which todo item that open terminal represented. (I semi-regularly go through and close tabs that CAN be closed, but a lot are something I was in the middle of and context-switched away...)

On Linux I can fix most things without a reboot, but "the BIOS has gotten itself confused" is not one of them.


March 27, 2022

"Streaming" means things vanish. You either have your own copy, or you don't. (This is a subset of "intellectual property law destroys intellectual property". Years ago Cory Doctrow said we're living in a dark age, because so much will fail to be preserved from today due to IP law that future historians will have a much clearer picture of the 1800s than today. Or was it lawrence lessig who said that? I googled, but the talk has fallen off the net...)


March 26, 2022

Visiting Fade and doing board bringup contract. Not very active on other stuff at the moment.

Twitter has gotten deeply confused. The 12 hour timeout expired, and now I can read things again, and tweet, but I can't favorite things? (Cached CSS entry maybe? Meh, not worth trying to debug.)


March 25, 2022

Fade set up a new twitter account for me so I can read threads without Twitter's stupid paywall page blanking the screen as soon as you scroll down three tweets, and while I didn't intend to post anything with it I got tempted into replying to stuff. Yesterday, I replied "We're all just waiting for the Boomers to die" and today:

We have determined that this account violated the Twitter Rules. Specifically, for:

Violating our rules against hateful conduct.

You may not promote violence against, threaten, or harass other people on the basis of race, ethnicity, national origin, sexual orientation, gender, gender identity, religious affiliation, age, disability, or serious disease.

As a result, we’ve temporarily limited some of your account features. While in this state, you can still browse Twitter, but you’re limited to only sending Direct Messages to your followers –– no Tweets, Retweets, Fleets, follows, or likes. Learn more. Your account will be restored to full functionality in: 12 hours and 0 minutes.

You can start your countdown and continue to Twitter once you:

Verify your phone number

Delete the content that violates our Rules

1 Tweet

If you think we’ve made a mistake, contact our support team.

So twitter considers "waiting" to be hate speech. (This Onion article is from twenty years ago before the Boomers got NEARLY this bad.) Meanwhile, the "your account can still browse twitter" is clearly untrue because now it's the phone number entry paywall at every page (paying with information is still paying, and my spouse already GAVE THEM her phone number to create the account so they HAVE THIS INFORMATION, it can only be some kind of "we know who you are" threat?), and they want me to performatively delete a tweet they've already blocked.

This is not a healthy website.


March 24, 2022

When busybox ash encounters an elf binary it doesn't recognize (such as an arm64 binary on an armv7l system) it tries to run it as a shell script.

root@stm32mp1:~# /boot/toybox-arm64 
/boot/toybox-arm64: line 1: ELF��@@�C
                                     @8@
                                        : not found
/boot/toybox-arm64: line 2: syntax error: unexpected "(" (expecting ")")

Meanwhile toybox notes any utf8 parsing failures on the first line (I.E. anything >127 that doesn't decode as valid utf8) and goes "this is a binary". Not the best heuristic, but it catches most things.


March 23, 2022

In minneapolis.

Here's a tangent I edited out of an email I was sending:

I'm trying to make a minimal self-hosting system for a bunch of reasons, and C remains the obvious language to do that in. But "where the source code comes from" is not strictly required to be part of that core. My Aboriginal Linux system would wget tarballs, and the automated building plumbing at http://landley.net/aboriginal/control-images/ created squashfs images of pre-extracted source code so the VM could just refer to a mount point with source code for each package in a separate directory.

The source code download mechanism doesn't affect "here is a classroom of students studying everything to understand it all". It doesn't affect the "Ken Thompson's Trusting Trust attack" analysis. It doesn't affect "here's an archival system you can rebuild known state from" (because if you can't reproduce it from scratch under laboratory conditions what you're doing isn't science). If you don't everything locally cached before starting you've thrown the "reproducibility" part out the window.

(Sigh, I need to do DESIGN videos. Lots of explanations about _why_...)


March 22, 2022

Flying to Minneapolis.

The Posix committee has approved adding strlcpy() to the next release (SUSv5, Issue 8, and POSIX-whatever year they get around to publishing it). Which raises an interesting issue of how to deal with that in portability.c: I independently invented "strlcpy()" back under Turbo C on DOS, and was happy to find it in SunOS' toolchain and BSD and so on, but it was never in glibc because Urlich DrPepper hated it for some reason. Fine, easy enough to implement my own. But if it gets added to libc and the standard #include headers, that's going to clash. (We're 100% going to disagree about "const" if nothing else.)

Do I put the definitions inside #if __STDC_VERSION__ < V4_RELEASE_DATE perhaps? Dunno. I've been using c99 and basically ignoring c11 (and bits of c99 to be honest, LP64 is better for known integer type sizes and declaring variables at the start of a block lets you FIND them), but I haven't been saying -std=c99 because I haven't needed to: both gcc and llvm have given me reasonable behavior as the default. I never had to tell /bin/sh -std=susv4 to specify which version of posix to use either, because THAT'S STUPID. I just run a sufficiently capable version of the shell that can handle my scripts, and it does the thing. If you find yourself having to say "python3" instead of python, it means the python transition faceplanted spectacularly.

Hmmm, looks like somebody on BSD complained (they have their own version in libc) and I already removed it from toybox. (I forgot.) Oh well, bullet dodged I guess?


March 21, 2022

The USA had nine mass shootings over the weekend.

Completely lost in the noise of course, but no party is "pro life" or "protecting children" when active shooter drills have replaced singing "duck and cover" and THEY ARE OK WITH THIS.


March 20, 2022

Back to work on the stm board. Serial console works now!

Something about the yocto install on this board puts it in a mode where the mouse wheel scrolls through command history instead of backscroll. The fix is "tput rmcup" which isn't installed on the board, but running that on my laptop and piping it to hd says "echo -e '\e[?1049l\e[23;0;0t'" should fix it (and did). The man page implies that the first one is involved in "mouse tracking" support but doesn't really properly explain (it says waht ?1000l does but not ?1049l), and if it mentions the "t" one at all I can't find it. Still, it worked, and that's the important part.

So the first thing I'm trying to fix is the gigabit ethernet jack. In theory it's the same chipset and driver as the raspberry pi form factor evaluation board, but something about it is unhappy. I set up a static PTP connection between it and my laptop, and the board never sees packets sent from my laptop, and my laptop increases the rx error count for each packet (although occasionally some get through!) Hmmm...


March 19, 2022

Boomer Prudishness has gone completely insane.


March 18, 2022

Trying to close tabs, in this case tabbed terminal windows within which I have a bunch of half-finished commits, and also the various "clean1, clean2, clean3..." directories in my toybox workspace where I was doing something off on its own branch. (Mostly toysh changes.)

One of the abandoned forks was a very long rathole trying to figure out toysh crash where free() was aborting on one of the targets, which turned out to be 2 things: 1) I was popping an entry off of a list but had a cached pointer to it that needed to be updated, 2) I was hitting a bug in musl-libc's realloc() that corrupted the heap. Apparently this bug only existed briefly a year ago, and of course the toolchain I built (with git commit du jour since musl hasn't had a release in over a year) was within the range that had it.


March 17, 2022

Evangelical christianity continues to suck

I really hope the fall of Putin takes down the GOP he funded. (And yes, Manchin is a republican.) Texas suppressed 19% of the mail-in ballots this last vote.

Boomer Media sucks. Defund the police. The LAPD has been terrible for a hundred years, and they even made a movie about it. But as notable as that instance is, it's not exactly unique.


March 16, 2022

So, definitely not vimeo then. (They're charging patreon creators thousands of dollars per month for hundreds of views per month, as part of pivoting away from hosting video for individuals into hosting video for corporations. Patreon has not yet caught up with the change, and continues to think it has a partnership with them.)

Started on that very brief contract (just through the end of the month) to keep the lights on while the Google guys work out how to get money from point A to point B through a legal briar patch, and we just worked out that the board I'm doing bringup work on was damaged in transit, which is why I couldn't get serial output. So I packed it up and overnighted it back to them, and am taking tomorrow and friday off.


March 15, 2022

Tesla sucks. (All billionaires suck. USA's own oligarchs.)


March 14, 2022

Started the board bringup contract. Oh goddess another Yocto build. But I guess that's why they need help: if they weren't using Yocto they'd be done by now.

The ELC-in-Austin CFP closes in an hour and a half, and I still can't come up with the enthusiasm to submit proposals. I mean, I could do "building the simplest possible linux system" again, presumably much less jetlagged this time and with mkroot actually functioning. I could do "the state of toybox". I could propose a talk on mkroot. I could do a talk on android convergence. Heck, that stupid colocated "community leadership conference" suggests "Open Source Governance and Models", meaning I could dig up "the prototype and the fan club" talk again and try to get a version properly recorded.

But I'm just not feeling it. I don't feel connected to this community. This is The Linux Foundation: stamping out "community" is what they do. Chasing away hobbyists so the suits can charge rent. I wouldn't be going there to see anybody...

*shrug* Looks like it's moot anyway, the website closed the CFP earlier than the posted time. Probably using client-side javascript to use my timezone instead of the stated pacific time, but I'm not resetting my laptop clock to get around that.


March 13, 2022

The dollar value of Asset Forfeiture has surpassed burglary in the USA. The police are now MEASURABLY stealing more money than the "criminals". They need to be defunded.


March 12, 2022

I picked up a brief contract through Imperial Staffing (2-3 weeks of board bringup) to keep the lights on while Google works out how to give me the money they have sitting in an account already approved for toybox development. (There are many lawyers involved, because I am not a corporation, a state Google's bureaucracy cannot comprehend.)

Going through my various toybox/clean[1-9]* directories where I forked off some task and then got distracted from it before it was finished. One of them involved sticking several dozen dprintf(2, "thingy") debug prints into toysh, and running the mkroot init script. Unfortunately, what I DIDN'T record was the actual TEST I was running. The dates say these files were last updated July 5-10, 2021, and my blog says I was trying to track down an error manifesting in musl's free() on 64 bit platforms. Aha! I ran the i686 mkroot image but not the x86-64 mkroot image, and that one is segfaulting. Right.


March 11, 2022

The Boomers' prudish pearl clutching censorship continues. It will escalate continuously until they die.

The GOP's new strategy is modeled on the "fugitive slave laws", where they pass a state law and make crossing state lines to avoid it a felony. This is OBVIOUSLY unconstitutional, but they've packed the supreme court with confederate loons and Biden will never make it "go to eleven" because he's a Boomer and Boomers solve nothing.

Disney squeezing blood from a stone: pay to watch ads. (This is why it's "late stage" capitalism: they can never have enough. They can never stop. They must ALWAYS escalate to the indefensible and trigger a revolt.)

We know what policies work to make a good sustainable society. We always have. The problem is oligarchs lying, cornering the market, inserting unnecessary middlemen to bleed the system dry, burning down the homes people own so they must pay rent forevermore. This is why brexit is privatizing britain's NHS. This is why doctors in the USA no longer make house calls, and the surgery for a broken arm ($16k) costs more than a year's minimum wage salary ($15k, and that's before taxes).


March 10, 2022

Trying to finish up the "cd in deleted directory" stuff and I am annoyed at bash again:

~$ mkdir sub
~$ cd sub
~/sub$ rmdir ../sub
~/sub$ cd -P ..
~$

What is the -P for then, exactly? What, does it fall BACK to -L to try to... Ah, I see. This is a Linux kernel thing. The ".." symlink is retained active in the dead directory, pointing to the directory it WAS in. And when I rmdir the parent directory as well it's still pinned (ls .. shows it as empty but doesn't error, and stat .. gives it an inode), and then cd .. fails but cd -P .. squirts back up to the next non-deleted directory.

So I don't think I have to handle this specially, because the kernel is already being weird about it.


March 9, 2022

The EU has just ordered that the RT inteview with the late David Graeber be scrubbed from the internet, because ten years later a dictator went nuts and therefore everything they've ever posted is damnatio memorae. Not just the original posts, but reposts must be removed. And not region-locked out of the EU, they get to export their censorship globally. Sigh. I can see blocking new stuff, but removing the old stuff has a lot of splash damage. (And once again, a reminder that "streaming is not ownership".)

Of course the worst kind of censorship is self-censorship. These days a 1970s Australian public service announcement that drew cartoon humans that look like humans is considered AMAZINGLY WEIRD because acknowleding how humans look is JUST NOT DONE anymore.

America is one of the LEAST sexually healthy or body positive countries in the world today. What messages do people grow up with here? Your body is shameful and no one must ever see it, but other people's bodies are fascinating and you deseperately want to see them. If what YOU have is bad, and what THEY have is good... Sure it's not the only issue (historically half a percent of humanity seems to always have been trans, which in the USA's population of 330 million would be 1.6 million people), but perhaps a bit of a thumb on the scales? If most actual physical differences are scrupulously hidden from view at all times, at what point does sexual dimorphism become an abstract concept and thus a matter of opinion? Of course no attempt to understand early development influences is any excuse for nazis to persecute people. My point is just that we live in a DEEPLY repressed, unhealthy society. (And as always, the capitalist corporations latch on to any social flaw and make it worse.)

Sadly, the world is complicated. For example, Putin's "denazification" is standard right-wing projection, his forces marching under the Zwastika are mass-murdering people to depose a jewish president. Meanwhile Israel is one of the few countries that's refused to condemn the invasion, and their government is settling Ukranian refugees on occupied Palestinian land while they're the main worldwide source of spyware repressive governments use to monitor dissidents.

And the usual issues remain at full volume: Defund the police, the GOP is still literally a death cult...


March 8, 2022

So I made a Vimeo account and uploaded a video (a lightly edited version of the first one-take tutorial video on downloading and using a toybox static binary), tried to make a video post on patreon, and... it says I can't do this with a "Vimeo basic" account, I need one of the pay accounts.

I logged into archive.org with my google account (well, one of them) and uploaded the video there, and then tried to post THAT to patreon as a video post and instead of embedding the video the summary box is a notice about archive.org having an outage for scheduled maintenance, and then when you click through it goes to the download page where there are inexplicably five files (I only uploaded one). Although archive.org has a nice video player (that goes to 3x when youtube's maxes out at 2x), you have to click "parent directory" to get to it for some reason.

I'm not sure if this is actual incompetence on patreon's part, or strategic incompetence promoting specific partners in a plausibly deniable fashion?

Meanwhile, the Google guys suggested I make an account on something called "Open Source Collective", so I can go through an approved vendor and thus bypass the lawyers boggling endlessly at the concept of an individual human being existing as standalone entitity rather than part of a corporation. (Doesn't EVERYONE have a corporate shell? Are you truly real without a corporation wrapped around you?)

This collective of course takes a 10% tithe, and I asked if just sponsoring my Patreon was doable (I have the old-style account there that takes less than that), and apparently the tax people doing a bunch of head-scratching about that. (So giving 10% of the money to some organization I've never heard of is doable, but going through the one that many open source developers actually use is not currently an option. This reminds me of IBM's lawyers spending the whole of 1998 frowning at GPLv2.)


March 7, 2022

Yet another study shows that lead poisoning cost Boomers several IQ points.

Capitalism intentionally creates scarcity, and some of that scarcity is permanent.

Here's a good thread on how deregulation under capitalism screwed up rock music in 1996. (With bonus misogyny.) And here's a good writeup on how Boomer Capitalism isolated everyone and destroyed our social skills.

And here's a 20 year old article explaining how Russia was already a plutocracy, I.E. capitalism is to blame for Putin. "Oligarch" is just a billionaire with political power.

Cultural misogyny is often internalized, although other times it's explicit (and incompetent, and implausibly deniable).

Defund the police.


March 6, 2022

Fuzzy was up all night barfing again, right after I was sick/tired/anxious again. This is definitely acting like that long-covid sine wave thing, which sucks. (And would imply we got Omicron late last year. Wheee.)

So I'm out at the table trying to edit another video I recorded, and I've hit a snag.

For years I've left the house to work, which might be an unmedicated ADHD thing? I'm more productive OUT somewhere, at a coffee shop or fast food booth or the picnic table on the UT Geology Building's porch. The hotel rooms in Japan were marvelously productive, but the apartment I had there stopped being so after the first week. Same 4 walls does not productivity make, for some reason. Not exactly writer's block, just really hard to get into the zone and WAY easy to get distracted.

But recording video at the table has the problem that it isn't QUIET. Right now there's cicadas, and even when there isn't there's some sort of climate control hum from across the street. (Most likely a multi-story building installation of heat pumps.) I'm assured that the hum is tolerable in the recordings (the true/false videos were recorded here), but my problem right now is I recorded several minutes of footage on the "tty" command (usage and implementation) back at home where it actually IS quiet-ish, and took it here to edit, but now I want to supplement the usage part because during editing I realized I never really explained what a TTY _is_. That's easy enough, I've worked out a brief script pointing at man 4 tty and basically saying it's a pipe with extra side channel info such as screen width and height, cursor position, and also process membership for signal delivery so the shell can batch signal ever process associated with that TTY so ctrl-C kills everything in a pipeline and ctrl-Z suspends the whole pipeline (and then bg or fg can resume it all). (On the implementation side I should probably mention you can go look at the man pages for tcgetpgrp()( and tcgetsid() if you want to know how that's implemented but it's out of scope for now.)

The problem is, if I splice together footage taken at the table with footage NOT taken at the table, the background air conditioner hum cutting in and out probably gets a lot more noticeable. (The cicadas certainly would.) So I need to edit as much as I can here and then go somewhere quieter to record supplementary footage.

Re-recording turns out to be harder than I thought, because on later takes of the same thing I wind up tongue tied and... tired? Dunno. It's strangely exhausting to explain things into a microphone, I don't quite know why.

This is WAY harder than just standing in front of people and talking to them. When I can actually SEE my audience I don't care so much if I repeat myself a little or explain stuff out of order and then backfill. Live talks are never GOING to be perfect, so they don't have to be. I can TELL if the audience is confused or bored or following it. Even giving talks over zoom, there's still people listening live. This is like trying to do stand up comedy without anybody laughing. Maybe it's because I'm new at it and have been speaking to groups of people FOREVER. (I taught community college courses back during the dot-com boom in front of rooms full of people at a time. That part's old hat. One on one I can do, one to many I can do, one to NONE I lose traction somehow...)


March 5, 2022

A right wing loon added an amendment to ban postal banking to the current post office repair bill, which would include the $21 billion of money orders the post office sold last year. It's another part of the "war on cash", the US Dollar has been privatized and may now only be spent through the monopsony of Visa, Mastercard, and Chexsystems who will ban you for life if you do anything they consider "immoral" such as visit a nudist resort. (Of course spray-painting moral judgements over everything is how they prevented AIDS research in the 1980s.)

Hertz is actively hurting its customers for no obvious reason by reporting its properly rented cars stolen and having the renter arrested. They interviewed one man who wound up homeless, another who spent 6 months in jail. It apparently does this over 3000 times per year and knowingly refuses to provide accurate information to the police because if it did the police wouldn't accept its theft reports. (Leto's Law has been covering this story for a while.) The moral of the story: corporate personhood allows people to do this sort of thing and escape punishment because the PEOPLE who did it aren't liable, the ablative corporate shell is.

The GOP remains funded by russia even today. Of course capitalism in general has that problem, but right now right wing loons are actively blocking US military operations near Ukraine.

Meanwhile, Defund, Guillotine, as usual.


March 4, 2022

Blah. Sick again. Or at least VERY VERY TIRED all day, for no reason. And borderline anxiety.

Posted an email to the list about the ongoing attempt to do videos.

I need to figure out where to host the videos I'm making, but youtube's copyright claims are just so amazingly stupid that I really don't want to get it on me, but there isn't an obvious second choice yet? They're like AOL, Myspace, or Livejournal: totally unassailably dominant. An american institution like Sears. (Although in this case it's mostly that Patreon has good youtube integration, and hasn't bothered to implement that for most other places, including raw URLs.)

So there's a bug report about toybox md5sum treating directories as empty files, and this is a more general problem that should be fixed in lib/ somewhere. If I put an fstat() into notstdio()... it doesn't have a good way to report back that it failed. Hmmm. Maybe just in loopfiles()? Grrr. Trying to retrofit a design constraint.


March 3, 2022

Happy International Sex Worker's Rights day. (That thread is from a bookstore, highlighting a lot of books about the racism and sexism behind the attacks on the industry.) Speaking of, here's an old video showing that 80% of american men admitted to watching porn and pornhub had literally BILLIONS of visits per day, and yet the Boomer prudes think that prohibition's going to somehow improve matters here.

The USA defeated Stalin's Soviet Union and Mao's Red China by converting them from communism to plutocracy, which turned out to be an even bigger problem in the long run. Late Stage Capitalism is now the toxic problem threatening to destroy the world, with global warming and nazis and countries invading each other (China's plutocrats eyeing Taiwan).

The plutocrats' playbook is to take over public services and then destroy them, both to convince people that government is unworkable (with tories in charge) and to make people desperate enough to beg for crumbs from oligarchs. (Bill Gates does not hire Warren Buffett to wash his car, they need piles of starving peasants to boss around or being a zillionaire means nothing. If they can embezzle the funds along the way, that's a bonus. As is writing laws to prevent themselves from being prosecuted, such as needing to prove intent. Sure you can show what I _did_ but without reading my mind you can never know what I MEANT by it...)

The most recent scummy replublican thing is to outlaw postal banking, including the $21 billion annual business selling money orders the post office currently does. (It's part of the War on Cash, all financial transactions must go through one of three private companies that can track and veto each purchase. This is why we need to guillotine the billionaires, as soon as the last boomer dies.)

Here in the USA we have the return of slumlords. Still trying to buy up every residence in the country and corner the market on housing.

Russia bombing a nuclear reactor is alarming but not as bad as it could be, because nobody outside of Russia is stupid enough to still run RBMK reactors. (Meanwhile, Russia still has ten of them on its territory, each of which could still go full Chernobyl. Luckily, Russia's post-soviet turn to capitalism gave it massive embezzlement and fraud which has hollowed out its army.)


March 2, 2022

Sarah Taber has great coverage of the strategic importance of ukranian agriculture.

HEB's 88 cent small avacadoes now have a purple sticker, saying they're from the Dominican Republic. The big ones still say from Mexico, presumably they take longer to ripen in the warehouse? (There's a nonzero chance Avacado Laundering is going on, but I expect they really were trucked through a third country if so.)

This laptop has bluetooth built in (my previous one didn't but the Dell Latitude E6230 does, I was reminded when rewriting lsusb recently). It's on an internal USB bus and has EXACTLY the same problem that the external USB bluetooth dongles did: the GPL Linux bluetooth stack is incompetent. (Android's got a working one, they threw out the GPL version and wrote an Apache licensed version from scratch. But nobody's ported that back to Debian.) The xfce GUI dropdown menu for bluetooth sees my headphones just fine (iJoy Logo), but when I click "select" it can't identify them (the options are "serial port", "audio sync", or "handsfree"), and when I select any of those it thinks for a few seconds and then goes "Device added successfully, but failed to connect". This has been the case with the Linux Bluetooth stack for at least 7 years now, completely unchanged. Linux on the desktop is not just dead but embalmed.

I need to do a video on the toybox build infrastructure, but first I want to split header generation out of scripts/make.sh just so it's easier to explain.

There's already a NOBUILD=1 that tells it to return early (just regenerate the headers, don't compile code), but that still leaves this script being hundreds of lines long and doing all SORTS of stuff before actually compiling *.c into *.o. And the basic technique for "make -j" in shell script seems like it would be of use to other projects. (You really don't NEED ninja and so on, 90% of the problem here is that "gcc *.c -o blah" won't use multiple processors because compilers are still stuck in the 20th century which has been over for two full decades now.)

The problem with splitting up scripts/make.sh is that it cheats: the file generated/build.sh is both a standalone script you can run to rebuild the toybox binary from this source and generated/*.h with a simple "gcc *.c -o blah" invocation (sadly currently broken again because I don't regression test it enough), but it's ALSO used by the plumbing to see if we need to re-run the library probes. (The compiler can eliminate used libraries via --as-needed, but if you tell it to link against a library it can't FIND that's a hard error, so we need to probe for what libraries are available and only include those.) And the "figuring out what to build" logic that theoretically only goes in the build-not-headers part gets used to create generated/build.sh. Hmmm, I suppose I can have it generate the start of the script as a separate file and then "cat file; make_more >> file". Bit of an intrusive change...


March 1, 2022

There was apparently exactly one airplane that could move the largest pumps and cranes and such around the world, and Russia destroyed it attacking the airport in Ukraine. I should apparently add the book "Kleptopia" to the to-read pile because it annoys the right people. New research shows that the glass ceiling is held in place by insecure, poorly performing men.

The GOP is reviving its family separation policy, this time taking native american children away from their families so they can be raised by white people (or sent to catholic boarding schools to be quietly murdered).


February 28, 2022

Still anxious, but largely trying to dismiss it as lingering stomach upset.

Lars at Google is back from his week off and says that legal came back with more questions and it'll be at least another week of processing. Fade moved some savings around to pay the mortgage, but I should probably sell my Coca-cola stock to have enough cash cushion to start job hunting again if it takes longer than that. (Ok, so not ENTIRELY lingering stomach upset. And my incredibly generous patreon donor edited his pledge down by 50%, which is still incredibly generous, especially considering I owe like 8 videos by this point. But even the original incredibly generous rate was a small fraction of my mortgage payment, so does not strongly influence the "can I afford to keep doing this" decisions.)


February 27, 2022

Darn it, decided to experiment since I was feeling "tired but not anxious" and HEB was out of the milk tea I like, so I got a bottle of the japanese milk tea (not the brand I get when I'm in japan, but they don't have that here, but this is still a fairly standard sweet black tea with milk), which I'd stopped drinking because the green tea version of it seemed to be triggering anxiety (I.E. upsetting my digestion for a day or more at a time, in a way that messes with my emotions). Two hours later, I'm full of anxiety. Funny that.

Walking around hancock center in a circle watching the blender video tutorial series on my phone. Everybody keeps taking "I have one video I want to cut chunks out of and stitch together" as too trivial a case to ACTUALLY SHOW AN EXAMPLE OF. (I'm not trying to overlay picture in picture. I'm not trying to do transitions. I just want to edit my one long take down to something that sounds more intentional.)


February 26, 2022

Speaking of the USA being uniquely prudish... (Because we were colonized by the Quakers and Puritans, people who fled pre-Victorian England for being too licentious. The mayflower landed at Plymouth Rock in 1620, between this portrait of english court fashion in 1616, and this in 1625. The thanksgiving images of buckled hats don't just look stupid by MODERN standards, they were silly cult stuff at the time, without even taking into account how much colder England's climate was back before the industrial revolution and coal burning steam locomotives and such kicked off global warming.)

So I was sick and then Ukraine got invaded which is quite distracting. I haven't emailed Denys to see if he's ok because I don't want to bother him right now, but I did finally give the busybox list a heads up that he might be a bit distracted. He's posted a single terse message to the list this month.

Found a situation where bash is leaking internal state, I think? Running this in my home directory (and WITHOUT the usual prompt trimming I do for these shell examples):

landley@driftwood:~$ mkdir one
landley@driftwood:~$ cd one
landley@driftwood:~/one$ rmdir ../one
landley@driftwood:~/one$ cd .
cd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
landley@driftwood:~/one/.$ declare -p | grep one
declare -x OLDPWD="/home/landley/one"
declare -x PWD="/home/landley/one/."
landley@driftwood:~/one/.$ unset PWD OLDPWD
landley@driftwood:.$ cd .
cd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
landley@driftwood:~/one/./.$

How did it get "one" back at the end there? The directory no longer exists and I zapped the environment variables that contained that string.

I don't think I want to emulate this one in toysh?


February 25, 2022

I've done almost no work since I got sick, partly because I'm still recovering and partially because Ukraine got invaded, which is hugely distracting. I have no special expertise here and there's nothing I can do about it, but strangely despite draining my energy it's also reducing my anxiety, by taking things that seemed unresolvable and making them... much less abstract.

You know how in Persona 5 the Phantom Thieves have to send a calling card to manifest the treasure so it becomes real enough they can steal it? That's how I feel about Ukraine. All the nebulous Russian influence behind Trump and Brexit where the Mueller report can come out and be ignored because untouchable oligarchs endlessly throw oil money around to run troll farms and bribe politicians and talking heads. The guy who ran Trump's presidential campain got a presidential pardon and the russian oligarch who got Boris Johnson elected was elevated to the House of Lords.

Russia's invasion of Ukraine brings all that stuff to a head so it can actually be addressed. The invasion is NOT plausibly deniable, and the responses to it aren't endlessly delayable until the statute of limitations expires. Tanks on the ground means now, and as with Saddam Hussein invading Kuwait there's no middle ground to hide in.

Faux News can't be propped up by Russian money when Europe has permanently cut BOTH nordstream pipelines and blocked Russia's largest banks. (Creating "the unbanked" is fine when it's punching up, doing it to a country in response to firing missiles at apartment buildings is way different than doing it to individual civilians for being seen naked.) Putin turns 70 in October: the math on a dictator for life changes when he's 3 years from the average russian male lifespan (BEFORE covid) and the sanctions only come off the day he dies and not one second earlier...

This defines the rest of Putin's life. Russians are out in the streets protesting again despite ALL the crackdowns. The morale of Russian troops is terrible: one group of deserters said they were told they'd be welcomed as liberators, a group of POWs say they were told the invasion was just more "exercises" until they were well past the border. Putin turns 70 in October: the math on a dictator for life changes when he's 3 years from the average russian male lifespan (BEFORE covid) and the sanctions only come off the day he dies and not one second earlier...

There's a kind of "big boat stuck" aspect as well, a concrete problem where everybody who doesn't agree what the issues ARE is outing themselves as a transparent shill (and even those are backpedaling). Alas this one is killing people and turning children into refugees and has a small but nonzero chance of escalating into World War III, while also bringing a bunch of other looming nebulous sword of damocles issues down into reach of resolution.

So yeah, this is distracting me and draining all my energy. (The weather being too cold and rainy to go to the table isn't helping there.)

I've still got a backlog of other links, as usual: Guillotine the billionaires, the GOP are all hyopcrites, defund the police, the USA's capitalist medical system is useless... And here's a strong argument for school lunch being free for all students: if you're legally required to be there and can't leave, how do they get to charge for feeding you?


February 24, 2022

Still sick. Got nothing done today.


February 23, 2022

Sick all day, lying on the couch. Nauseous, exhausted, and my sense of smell is misidentifying the catbox as the burnt dust smell you get from the vents when you first start the heater up in the fall, and my checkerboard tea tastes like band-aids. Fuzzy was barfing all night when she had this a few days ago, which is the _third_ time she's had this (about a week apart). Starting to suspect we got Omicron and this is the long tail sine wave thing where you feel better then it comes back then goes away again then comes back for several weeks. (As with the Flu, getting the new strain annually is only to be expected. It's Covid 19 because 2019, it's now 2022, welcome to YEAR THREE of the pandemic. Got the 'rona again. Same old same old.)

Russia invaded Ukraine and late stage capitalism still can't manage to wean itself off Russian money (and not just the right wing loons in Russia's pocket). The modern economy is literally built around money laundering for oligarchs, with corrupt regulators and complicit media. Of course we aren't sending them people or hardware, Late State Capitalism can only conceive of responding in financial terms. Meanwhile Russia's billionaires are commiting their own acts of war in preparation of taking territory directly from NATO countries, while we wait for the other shoe to drop.

The GOP is evil because they don't consider anyone else to be people just puppets, to be manipulated.

Plutocratic prudishness is a new thing. And of COURSE it doesn't stop with sex workers, they're already censoring legal abortion information and attacking gay children. The Evangelical Boomers will continue to get more extreme until their dying day.

Luckily, there's light at the end of the tunnel on that one. Turns out the great resignation is also the great retirement.


February 22, 2022

A good interview with Spike Trotman about leaving kickstarter. She was an early pioneer of the platform, and one of the first to go when they decided to "switch to blockchain" whatever that means. (Probably an attempt to escape that whole "the unbanked" thing where visa/mastercard/swift veto any financial transactions based on "I know it when I see it" moral judgements based entirely on evangelical christian religious prejudices: if your book acknowledges teenagers can be gay it can't be purchased for US dollars, and if you object too hard YOU PERSONALLY may be banned from ever having a bank account again. But at the same time, "if blockchain is the answer you're asking the wrong question", and Kickstarter is being vague about its reasoning because they don't want to point fingers at the payment processing monopoly and wind up with a retaliatory chexsystems ban on their executives. This is another reason we desperately need postal banking, where every american citizen has the inalienable right to a bank account. Once again easily fixed once the Boomers get out of the way...)

So Spike left and is now crowdfunding on her own website, using a wordpress plugin. I'm all for increased decentralization (I.E. undoing of web3), good to see the pendulum swing back the other way. (It's done this many times over the course of the computer industry, that's why X11 workstations tried to be a big thing back in the 1990s.)


February 21, 2022

Apparrently Apple's "security" is entirely performative. Youtube has a new feature to make sure the next video of police misconduct posted on a brand new youtube account can't go viral on their platform. The united states remains uniquely prudish and easily scammable via sex panics.

Fascists are murdering protestors on US soil again. It's only ever the GOP doing this, with the police equally blaming the victims. (The protests were against the police, the police say the protesters deserved to be shot, pointing out the large number of witnesses contradicting their story makes the situation "very complicated". This is AFTER portland police were documented attacking protestors 6000 times and got censured by the Department of Justice.)

Another day, another leak of secret billionaire bank account info. Who owns $80 billion of swiss bank accounts and what did they spend it on.

Of course the scandal isn't that "they still bank here": adding individuals to "the unbanked" is bad whoever they are. The scandal is the tax evasion and that billionaires anywhere remain unguillotined. "Punching up" is good, "punching down" is bad, even though both actions are punching. Basic income (plus a National Health Service and rent control) fixes most of this: let people afford to do what they want to do. And stop criminalizing sex work: we don't criminalize chocolate harvesting despite modern corporations enslaving children to do that. The triangle shirtwaist fire was about textile workers locked in and unable to leave when a fire broke out. Sex is NOT THE PROBLEM here.

The above article about the most recent billionaire banking leak also details the path by which Visa/Mastercard/Chexsystems achieved a proprietary stranglehold on the US dollar so it can unbank "immoral" customers like marijuana dispensaries and camgirls and escalate into the "war on cash" to eliminate all untracked transactions. I'm hoping that The Great Retirement (followed by The Great Die-off) will take the wind out of their sails, but really the problem of cornering the market and creating oligarchs is inherent in capitalism, going back to the Guilded Age with railroad robber barons in the 1800s, who were only displaced by the spanish flu pandemic, World War I, the 1929 stock market crash leading to a decade long Great Depression, World War II, and finally Sputnik and the "atomic bomb delivered by ICBM that no amount of radar and fighter jets can defend against so we will REORDER SOCIETY because of it". The period from New Deal through Cold War beat back the oligarchs for most of a century, but that century is up. AOC's Green New Deal was not NEARLY aggressive enough, capitalism needs to END, along with corporate personhood and civil forfeiture and the existence of billionaires. (Both as a category and as individuals. Guillotines are designed to divid. Proper tool for the proper job. Until then, Billionaires continue to literally kill people.)


February 20, 2022

Among the many MANY failures of IPV6 is the way it defeats half the purpose of ssh: every time I connect to a server via an ipv6 address it's a DIFFERENT address, and it goes "Warning: permanently added host key for address BLAH to known hosts" which provides ZERO protection from man in the middle attacks. (I know the "Earn It" act is poised to outlaw encryption in the USA entirely in the name of locking up all the Boomers who took naked baby pictures of their kids and are thus guilty of child porn by insane modern standards, while also denying that one House episode was actually describing a real thing because THESE days nobody does that under the age of 30. But it's still annoying for the infrastructure to get this stupidly wrong in the meantime.)

Grinding away merging lspci and lsusb, so they can share the .ids file loading/parsing/searching infrastructure, and while we're at it the parser for the uevent keyword=value stuff is pretty much the same too. (The lspci DISPLAY logic is nuts, but that's because it's got a half-dozen different output formats. -m, -n, -nm, -nn, -nnm, plus -e and -k modifiers for all that. And -nn puts the data in a DIFFERENT ORDER than all the others, the vendor:device data gets collated. Everything else is string [numeric] with fields optionally dropping out, but that one emits string string [numeric:numeric] unlike anything else. Hysterical raisins.)

This tweet reminds me I miss the non-STEM parts of college. I got an english minor, and had a bunch of courses in sociology and psychology and cultural anthropology and comparative religion and so on, and those were the actually useful ones. (Open source is an example of a potlatch culture!) The computer science I taught myself starting in 5th grade, and only wound up taking classes in it to pull my GPA up after the disastrous foreign language requirements. (The only F I ever got in high school was the second year of German: I remembered nothing from the first year. The only D I got in college was French: my french teacher taught french entirely in french. I tried to explain that if I knew french I wouldn't need to be there, but it didn't work because he wanted me to explain it to him in french. I WORKED for that D. Don't remember a darn thing from it, I eventually traversed the foreign language requirement with two back-to-back summer courses in spanish where the compressed schedule plus grading on a curve meant 50% was a C+.)

Oligarchs try to divert all funding to STEM and away from humanities because STEM isn't a political threat to oligarchy, and humanities are. Silicon valley techbros are politically fungible.


February 19, 2022

Good news: the copyright office refused to register AI-generated work, and the number of people "certain God exists" has fallen below 50%. Here's a good argument that Faceboot is following Myspace's end-of-life playbook. People mobilizing against the ongoing Boomer Stupidity and outing fascist funders. Meanwhile, california's sued Elon Musk for racism.

We still need to defund the police. Each misdemeanor arrest costs NYC $1750, so buying and giving away diapers and food would be signifcantly cheaper than policing petty thefts of essentials. And here's police teaming up with fossil fuel interests to stage an attack on themselves as an excuse to evade an eviction order by the native american tribes who legally own the land. They don't have to fool anyone, they just need a paper-thin excuse to "yeah but" silence protests at a casual glance: "Isn't that horrible?" answered with "Oh no, she was asking for it..."

Law enforcement is always protecting the powerful against the powerless. and the silver-spoon crowd are unused to being on the wrong side of that, so every time they meet a bigger fish or swim out of their little pond they cry like babies over the injustice of this thing that has never happened to anyone else before.

The war on cash continues. All purchases must be trackable back to an individual in realtime, and the recipient of the money potentially culled from the financial system. Switching everything digital is a bad idea. Once upon a time you could do the NYT crossword without having your data harvested, but of course Wordle is full of trackers under NYT ownership.

If the Earn-It Act punches a big hole in Section 230, Google's ad business could wind up with an awful lot of legal liability, and they'll still somehow manage to avoid going after the real problems (which existing laws apply to just fine). Meanwhile creeping prudishness continues to have knock-on effects, and the GOP is openly attacking contraception already. They consider taking away abortion a given at this point because of the supreme court packing, they're on to the next misogny. And as always, Boomer Biden fixes nothing.

Scammy right wing loons never stop fighting to chip off the most vulnerable 10% of everyone else.


February 18, 2022

I was recently pondering that the switch to electric cars means you can't drive around the country anonymously anymore. Even if you "own" one, and somehow disabled the built-in cellular tracker that does the "software updates", you can fill up a car at a gas station with cash bu all the chargers are credit card only, and I expect the vehicle talks to the charger reporting its VIN.

I've long considered atheism a religion the way zero is a number, and that agnosticism is the "lack of belief", because you can't prove a negative. Whether it's the existence of thor, zeus, ganesha, ameratsu, pele, anansi, osiris, morrigan, sun wukong, baba yaga, or santa claus. Religious people have lots of distraction arguments, like "was santa claus based on a real historial person", because obviously if that person did exist that proves they're still around today as an omniscient arbiter of morality seeing you when you're sleeping or awake and judging you to be bad or good while conveyed by a flying sleigh around the world to millions of houses in a single night single night with a dimensionally transcendental toy sack filled by elves living at the north pole. (Frosty's hat is clearly "Emet" brand, and has a label on the brim.) But since we all know the songs and can name the reindeer, and when we were young our parents told us it was true, all that counts as evidence for it being true. And obviously you aren't qualified to argue Klingons aren't real if you don't speak Klingon and haven't first read every novel and seen every episode and movie Klingons appear in (including the cartoons!) to confirm that proof of their existence isn't in there somewhere.

But some people strongly disagree with "agnosticism" being the default, and consider atheism to be the "lack of belief" one. It's a semantic argument, really. Personally I got disgusted with the atheist "movement" a decade ago prostelytizing (ala "I know better than you do what you really are, and what you must call yourself"), and then either turning into outright fascists or (eventually) stepping way back and reevaluating their lives. If you have missionaries, try to convert people, and have meetings about your belief structure, you just might be a religion. Firm belief in nothing is still a firm belief. Absence of information is not the basis for belief. Black swans turned out to exist. The saddest thing about Elon Musk (and it's a long list) is he did NOT put a Utah teapot into his stupid red sportscar he launched into space. In honor of Bertrand Russell, of course.


February 17, 2022

There are still avacadoes at HEB today, despite the import ban. (I'm not sure how much of this is "don't threaten US officials" and how much of it is the US government telling mexico to get its act together with all the reshoring we're doing there.)

New covid strain dropped.. 30-50% more contagious than Omicron, as injurious as Delta (it's got that s-gene back), mutated enough to reinfect, detected in 47 states, just as the CDC is declaring the pandemic over (just like Denmark did). So looks like March is gonna be a fun month, although as long as you're fully vaccinated you should be ok, modulo no hospital beds for heart attacks, car crashes, appendicitis...

Meanwhile in global warming news...

Hmmm, got a bug report that sha1sum and friends treat directories as empty files, when they should treat them as -EISDIR and skip them with error_msg().

This is actually a library code problem: an awful lot of stuff should refuse to accept a directory in place of a file. There doesn't seem to be an faccess() variant that works on a file, just fstat(), so the check is doable but a bit more heavyweight than I like. (Eh, the overhead of copying a bunch of dentry data into a struct is probably swamped by the system call entry/exit code.) The two obvious places to put this are in loopfiles_rw() or in notstdio(), and although the second is the more thorough fix, adding a stat() to basicaly every open() call makes me wince, and auditing all the calls for correctness (69 xopen and 23 xcreate under toys/ according to a quick grep, plus more in lib/ and possibly main.c) seems uncomfortable. And those are just the DIRECT calls, not through something else.

Speaking of which, there are 48 loopfiles calls under toys/*/*.c (each of which goes through xopen), and just auditing those if that's where I put this fix is... annoying. But that's probably the first place to fix it.

I checked in bash and "echo hello < dirname" is not an error. (No, I'm not poking the maintainer. "The implementation is the specification" is hard enough to cope with, and then he keeps FIXING things I point out to him when I just want to understand why it's doing that.)


February 16, 2022

Naiomi Wu just did a good thread about how Youtube's CEO "has decided to deliberately target channels this year that *aren't* breaking the rules but that she just doesn't like. One of the things Susan doesn't like is women in STEM channels since there are none in the top 100 tech channels." Here's a previous thread she did (and a study linked in the comments).

Meanwhile, since youtube's music playists are now "ad for ulcerative coitus, one song, two 30 second ads for bitcoin, one longer song interrupted midway by another ad", I decided to dig up adb and use my USB cable to copy mp3s onto my phone, but "Android Play" has been discontinued in favor of youtube music. Android has LOST THE ABILITY to act as an ipod replacement, they've ceded that market entirely to iPhone (although the usual reason iPhone users give for 2/3 of the US phone market having gone iPhone is the blue/green texting thing. Android taking multiple years to maybe eventually also block Facebook's tracking cookies might also factor in somewhere, although Apple has plenty of its own problems there).

I am old enough to own easily rippable CDs, and digital downloads that aren't permanent rentals locked inside a walled garden ready to go away when they do. I have my own music, I wanna put it on my phone and listen to it, and stock Android is no longer capable of that. So I downloaded VLC for Android, which wants global read/write access to all files on my phone. No, I'm putting files in a directory and pointing it at them, IT DOES NOT GET WRITE ACCESS. But it can't do read access without write access, just net play. (Which sounds kinky, but I'm busy.) So I installed "simple music player" which does the normal "allow app to access photos and media". (That's what it needs to do its job, and if it uploads my cat pictures to Russia, oh well.)


February 15, 2022

Boomers solve nothing. Boomers can never resist letting Russia spend money at them, which is why they're so easily hired to push Russian agitprop. Sedition is just another commercial endorsement deal to them, like hawking shoes or burgers. They keep letting the same clowns cause trouble over and over, without learning anything ever. (Speaking of which, Canadian truck loons are still in the news.) Everything the Boomers did was ultimately self-defeating, and the lead poisoning combined with senility has made them really stupid.

Oligarchy uses racism as a shield and a distraction. (Keri Leigh Merritt wrote a whole book about this in 2017.) Meanwhile actual racists are stupid and spend their time manufactuing victimhood and being profoundly hypocritical.

Quartz' translation of the spotify apologia is weapons grade snark. And here's a a long 4 part thread of actual analysis of the podcasts triggering the backlash. But the rot at spotify goes far beyond Joe Rogan.

Amazon needs to be broken up: turns out Audible's shafting creators too. Abolish ICE. Remove cars. Slumlords are a problem. An excellent analysis of why all blockchains suck. Uber is a scam. Facebook is also a scam. Youtube continues to suck. The great resignation is happening for many reasons. Religious parables are often kind of horrifying if you understand what they made a story about. Defund the police, then defund the police some more. Funds meant for other things keep getting diverted to police.

Don't forget how bad the guy behind the Comstock Act was. historical censorship has always been insanely stupid, but the right wing loves endless creeping censorship, intimidation, voter suppression, and conspiracy.

The earn it act continues to be horrific. First they came for trans people, then they came for sex workers, then they came for gay teens (and teachers), then they came for abortion rights... How any of the "freeze peach" advocates support the end of anonymity and banning encryption in the name of being shriveled up old prudes is beyond me. (Sex workers look out for each other, the reason there's so much trafficing is they're prevented fromm doing so, and the ones who are preventing them from helping each other are the ones victimizing them: the same rich white men who are openly persecuting them are paying for the trafficing. It's the old "homophobe turns out to be gay" issue. No catholic priest gets to weigh in on this issue.)

No it's NOT the stimulus payments: inflation is happening because the cost of shipping things from china went from $3k/container to $30k/container with a month long wait to unload in the port of los angeles, and it goes back down when we finish onshoring/reshoring the manufacturing within NAFTA. Playing with the demand side (via money) only has an impact when supply can't keep up, it's SUPPLY constraints that cause inflation, not demand. The classic example is rare comic books where there's only a few dozen bidders for that issue in the WORLD and if you found a box of 1000 of Batman's first appearance in an attic somewhere the price of all of them combined would plummet to less than what ONE costs now. "Act now supplies running out" is a classic sales line trying to induce demand via the illusion of scarcity. Cornering the market raises the price, it's how Late Stage Capitalism works: when everyone is adequately supplied, the thing being produced is cheap. Only when there isn't enough to go around do prices get bid up. It doesn't matter what the DEMAND is, air is still free and clean water coming out of the tap is cheap enough to shower in the stuff without a timer. Inflation is happening because of disrupted SUPPLY chains, not a change in demand.

The reason I keep coming back to twitter isn't what twitter thinks it is. Unfortunately last year twitter was purchased by a right wing loon, just like the Wall Street Journal and Newsweek were. "Conservatives" are all about censorship. They want everything they say to be mandatory viewing (clockwork orange style when at all possible), but to be the only ones allowed to speak and immune from criticism.

Elon Musk, "incompetent narcissist". He should have it on his business cards. Guillotine the billionaires.


February 14, 2022

The Atlantic just ran an article on how the USA's doctor shortage is intentionally caused by the AMA imposing medical school graduation quotas, restricting visas for foreign doctors, and preventing nurses from taking on doctors' tasks. It's basically the same story from this excellent 2009 writeup, and of course I've done my own deep dives into the topic before: we can't fix healthcare in this country until the cartel intentionally screwing it up for profit gets broken up.

The reason for fiddling with struct string_list yesterday is I wanted to add struct dev_list next to it, because lsusb and lspci are reading basically the same file format for their device ID databases. There's a third layer to pci.ids, but toybox's lspci isn't using it (because it doesn't implement -v, which turns out to be kinda fraught).

But I've sort of changed my mind and decided that merging lspci and lsusb into the same file probably makes more sense? Because I don't see a third caller showing up any time soon, and neither of them is particularly big (each a little over 100 lines).

Meanwhile, I REALLY need to break up ps.c and move the shared code to lib/ps.c, and that's... a can of worms. But triaging commands to do a new round of documention on them (videos) is resulting in various cleanups. As documentation always does. It's easier to fix it than explain it. (Which can be read as BOTH avoidance productivity and filing off corner cases that are too much backstory to digress into during the writeup.)


February 13, 2022

Reading the C99 specification is no fun. I forget what I was trying to look up but along the way I convinced myself it did NOT require unmentioned structure members to be zero initialized when you did &(struct boing){.potato=37;} (I mean, it WORKS, and you can see the explicit initialization via objdump -d, but I wasn't sure it was RELIABLE because {1,2,3} and {.a=1,.b=2,.c=3} seemed to be treated different by the c99-draft? The confusing part is that section 6.7.8.9 says "Unnamed members of structure objects have indeterminate value even after initialization", but apparently section 6.7.8.21 is the part that says to zero initialize them, and "elements or members" clarifies it applies to both styles of initialization. (According to grep '[)]{ *[.]' main.c lib/*.c toys/*/*.c the only command using member initialization in compound literals is host.c, which I rewrote NOT to do that and then didn't check in because... no, it's fine as is. I think.)

I then tried to look up what "unnamed members" _ARE_ but it seems to be 1) a bitfield thing (might have seen that but I can never trust bitfield endianness and the compilers traditionally generate HORRIBLE code for bitfield access, so I always shift and mask myself) and 2) a way to add padding to a struct? (Never used it, never encountered it. Lots of stuff adds padding to structs but they always call it stuff like char __unused1[2];)

Ah right, I was trying to look up whether the variable length array element at the end of a struct should have [0] or just [] and it's section 6.7.2.1.15 that says it's char blah[]; with nothing in it. (It of course adds some nonsense about access being "undefined" and then immediately subverts it via the examples in 16-18, and I refuse to be baited twice by the same standard in the same session. This one I KNOW, I'm just making sure that removing the 0 from struct string_list in lib/lib.h is the correct thing to do. They're equivalent with or without the 0, but static checkers throw a wobbly and people keep trying to use them...)

Heh, I apparently need a current writeup of my whole "Linux uncontaminated by gnu" plan to make a system you CANNOT call "gnu-slash-linux" because it isn't even built with gcc. I've mentioned this multiple times over the years, I think it was in my 2013 "rise and fall of copyleft" talk at ohio linuxfest for example, and probably on the old aboriginal linux mailing list long before that. Heck, it was probably in my old livejournal back in the busybox days, and a quick google finds people referencing me talking about it back in 2007. And, of course, my original sin that interested me in undoing the damage from the toxic waste spill I'd initially triggered in the first place...


February 12, 2022

Apparently the reason for the boil water notice (attachment B, page 1, "How exactly did this happen") is that Austin adds lime (calcium carbonate) to the water to soften it, and somebody left the lime running overnight until it clogged up the filters. Whether this was related to somebody being unable to get to work because of the ice storm is not specified.

Yay, I figured out how to switch the keyboard backlight in this laptop on and off. (Despite poking the geology building's management at UT a couple months back, they have yet to fix the timer that turns the lights on at midnight, so if I get there at 10pm I'm in the dark for a couple hours, and the keyboard backlight is very convenient.) It's function right-arrow. While fiddling with that, I toggled something with a moon symbol, and something that's three squares fit together. (I was thinking it MIGHT be a keyboard but it's probably supposed to look like the mousepad and the left and right click buttons under it?) I hope selecting whatever it is an even number of times turns it back off so I don't notice some weird new symptom I forget how I triggered days from now. Ah, it turns out there is a webpage explaining that stuff, and most of it doesn't work on Linux anyway. Oh good.

I'm going through lots of little toybox commands that might make good short intro videos I can both get out easily and use to learn video editing. (This is resulting in lots of little cleanups to commands I haven't looked at in a while.) There's a lot of "I need to explain everything before I explain everything else" circular dependency stuff in this, but there's pretty clearly gonna have to be playlists: how to use commands, how commands are implemented, common infrastructure, building bootable Linux systems...

I want to do videos with GROUPS of small commands, because true, false, clear, reset, sync... they're all basically one function call. Another genre is "open a file and call an ioctl on it", maybe with or without a loopfiles(), which covers freeramdisk, fsfreeze, partprobe... (Huh, I have a "detach.c" in my tree that isn't checked in, I should look into that...) It seems a bit silly to have a "how to use unlink" video, and then an "unlink.c implementation walkthrough" when each is under a minute: the implementation is "if (unlink(*toys.optargs)) error_exit("message");" plus the usual command boilerplate and help text.

But at the same time if each command has its usage video and it's implementation video, it's easier to do a table of them. A few commands (ps, sed, sh...) will probably need multiple implementation videos though. I REALLY need to do the breakup work on ps.c to make the 5 commands in it have their own files and move the rest to lib/ps.c before doing an explainer video on that. (The shell and sed are just big, I expect awk and vi to be too. I see tar.c has crept over 1000 lines too, that one's not CONCEPTUALLY hard to explain but somewhat voluminous.)

Everything else (that's completed and not in pending) probably fits reasonably well into a single implementation walkthrough video that isn't exhausting to watch. But even the "how to use" video for sed or sh is gonna be a bit of a marathon viewing if it's all one go...


February 11, 2022

To avoid burying the lead: people are organizing against the latest monstrosity, and here's Stephen Fry on Graham Norton decades ago explaining the fundamental problem.

So the anti-sex evangelical loons have a new bill to make showing ankle on the internet illegal: the Earn It Act. (Of course the main people stopping sex trafficing are sex workers, but victim-blaming is fundamental to christianity. Eve listened to the snake that God made and let into the garden, so God held a grudge for thousands of years and had to arrange a human sacrifice to give himself permission to (conditionally) forgive their descendants. That is literally the core story of christanity, so if the church admits Eden is a "metaphor" then original sin needing to be "forgiven" by this benevolent god is...?)

Wikipedia blacked out to stop SOPA and PIPA, and white guys spoke out against it, but not FOSTA and SESTA because that was "icky speech". Germany's nazi party started out by treating trans people as non-human, and expanded it to concentration camps for millions of people. Of COURSE the misogynist evangelicals going after camgirls won't stop there, they want to rigidly censor the internet and bring back the Comstock Act.

But the real driver is to force everyone participating in the great resignation back into the crappy jobs they left making plutocrats money. And yes, they're still using other tools like civil forfeiture (except that example was Amazon convincing a friendly judge to steal the money straight from bank accounts), and of course both of the news stories linked from that gofundme are behind paywalls.

The evangelical clowns burning a zillion books are never addressing actual problems. The actual fixes are all undoing damage the GOP intentionally caused in the first place (or at least pushing back against it). There are zero bright lines with prudery, just one big long gradient smoothly shading through every possible grey. The censors will never have enough, and will keep grasping until the Boomers die and we end capitalism.

Speaking of capitalism, Faceboot's cover-up of its role in Cambridge Analytica just blew up in its face. The billionaire whose seed money was inheriting blood emeralds from apartheid south africa might be a giant racist (who could have seen that coming) and he continues to be arrogantly incompetent.


February 10, 2022

It warmed up a bit, and I went to the table and actually stayed for a while and got work done. (Also, HEB restocked on my lovely checkerboard tea.) Enjoyable evening, on the whole.

Except I tried to walk down to the river first (and then walk back TO the table) to get a longer walk since it's an exercise-poor week, and the construction downtown is just NUTS. And extremely pedestrian unfriendly. You can't walk south on the east side of the statium, it's all fenced off, so I kept going east along the fence until it finally gave out IN THE I-35 FRONTAGE ROAD. No sidewalk, having to walk IN the road for multiple blocks down to... somewhere past MLK I think? With a 4 foot high crash divider a high chain link fence right on the other side of it. And then when I finally could get back out of the road and went west a block before continuing south... that road gave out after half a block and was all fenced off by construction and cranes. So I went back north, another block west, started south again... same thing. Wound up walking back to the table from there (still through a lot of construction and backtracking, but less bad when I got away from I-35).

I have no idea what they're doing. I vaguely expect it's related to the I-35 expansion that's tearing down the upper deck and killing the bat colony at 42nd and turning the whole thing into a belowground flooding deathtrap? Yes, looks like it is. What a horrible project. It's ALREADY making downtown unwalkable just in the CONSTRUCTION phase (no pedestrian through access for literally multiple blocks). We've known about induced demand for decades now, Expanding the highway is totally self-defeating. The proper thing to do is reduce car dependence by eliminating single family zoning and parking minimums and so on. You don't have to drive when you can walk to the dentist, but you can't walk to the dentist if zoning doesn't allow a dentist to set up shop within a mile of your house, and literally half the space in your city is legally required to be parking lot.


February 9, 2022

It's fascinating, I can never remember the name of the hackermonthly PDF (let alone the URL for the original slightly wrong explanation I posted to the busybox mailing list off the top of my head a dozen years ago) so I googled for "landley usr bin" to find it and...

No, Ken and Dennis did not have floppies, the PDP-7 used a ramdisk loaded from paper tape. (And /usr wasn't "renamed", it was the user directories with a three letter name like bin, etc, lib, and dev because the pdp-7 had 1024 18-bit words with half running the OS and half being used as a ramdisk so yes the extra letter counted and they kept the name when they moved to the pdp-11.)

No, dynamic linking did not solve any problems with static linking, it made things WORSE so that the proposed "mount this, then that" approach wouldn't work reliably. (And yes, the a.out executable format had shared libraries back before ELF.)

Obviously I can't really throw stones when my own first stab at the explanation got the disk sizes wrong. (I remembered the TOTAL size but not how it was partitioned, nor how MUCH faster the tiny system disk was than the external disk packs. In modern terms they filled up their SSD so leaked the system onto the external USB disk holding user files, and then got a second USB disk later to move the user files to so the system could have all the first one.) That's why I went back and corrected it when a magazine asked to republish it, and gave links to primary sources at the end.


February 8, 2022

I have filled out the "Embark" online paperwork. I already know one thing I got wrong, but that's normal for me filling out paperwork: back in middle school I expected about an 85% on tests where I knew the material cold, just because it was paperwork so I would of course fill some of it out wrong, miss the back of one of the sheets (of the instructions if not the test), and then hand it in to the wrong place. (Programming is all about iteratively fixing the thinkos when the machine points out where you got it wrong, sometimes backing up and frogging entire sections when something's misaligned.)

To relax, I'm trying to get toysh to match bash's behavior for moved and deleted current working directories. I already added a test:

mkdir -p one/two/three
testing 'cd in renamed dir' \
  'cd one/two/three && mv ../../../{one,four} && cd .. && echo ${PWD: -9:9}' \
  '/four/two\n' '' ''
rm -rf one

Which the host passes but toysh doesn't, and the logic is sadly a bit tangled. But then bash is a bit confused too. When launched in a deleted current working directory, bash goes:

$ env -i bash --norc --noprofile --noediting

shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

bash-5.0$

Except I ran it under strace and piped the output to less so I could search it (even with --noediting bash and less are trying to read from the same tty and getting alternating letters until you hit control-C which kills the bash instance but not the less instance for some reason; probably gotta make my less implementation do that too eventually) there was only ONE call to getcwd() (with a fixed size 4k buffer instead of an automatic allocation), so I have no idea what it's talking about "parent directories" plural?

I added support for "cd -" while I was there (and already emailed Chet back on January 20 to add it to "help cd" in bash because seriously, it's in posix: 'cd -' is 'cd "$OLDPWD"' the same way 'cd ~' is 'cd "$HOME"').

It was already exporting $PWD and $OLDPWD only on the first cd, because that's what bash does (and yes I should add a test):

~$ cd sub
~/sub$ cd ..
~$ unset OLDPWD
~$ cd sub
$ declare -p OLDPWD
declare -- OLDPWD="/home/landley"
~/sub$ declare -p PWD
declare -x PWD="/home/landley/sub"
~/sub$ unset PWD
~/sub$ cd ..
~$ declare -p PWD
declare -- PWD="/home/landley"
~$ env -i bash --norc --noprofile -c -- 'cd sub && env'
PWD=/home/landley/sub
SHLVL=0
OLDPWD=/home/landley
_=/usr/bin/env

It SETS them both each time you cd, but only _exports_ them the first time, and just does a local assignment the rest of the time. And I do mean "local":

$ x() { local PWD; cd .; }
$ unset PWD
$ x
$ echo $PWD

So I think my code was already getting that right (I need to add more tests) but before it was wasting an entire int for this in the TT globals block and I switched it to using the high bit of TT.options, and collated the export statements under the same test.

When I say I'm not just writing a shell but specifically a BASH replacement, I'm SERIOUS about that. My spec is not posix, or even the bash man page. It's WHAT BASH DOES. Tied to the comfy chair, poked with the soft cushions, and added to the regression test suite. I'm reluctant to communicate with Chet about stuff because he keeps FIXING THINGS I POINT OUT and thus changing the behavior. Which probably breaks somebody's script somewhere...


February 7, 2022

Still not sleeping well, because anxiety. I did not used to have anxiety. It comes and goes, but having it lurking is INCREDIBLY ANNOYING.

Corporations are not your friend, and regulatory capture is a thing.

Intellectual property law destroys intellectual property. Ten years ago the netflix DVD library had 100,000 titles in it, but the online streaming version they transitioned to keeps losing content and these days has 15,000 and shrinking. All the services are like that: Youtube just showed me a clip from something called "Sakura Trick" which Google says came out in 2014 and was on Crunchyroll, but of course it's not on Crunchyroll anymore. It's about two high schoolers dating; if they were heterosexual it would be "cute" but if they're gay cue the pearl clutching about corrupting the youth. Me, I'm thinking it might be better "I want to learn japanese" background noise to put on without subtitles than Hulu's run of the original Sailor Moon, because watching Usagi murder people and be lauded for it "because protagonist" gets old. Each week she kills somebody who was passing as human just fine at the start of the episode, and I have yet to see any of her opponents actually permanently harm anyone? As I said, watching without subtitles so I couldn't tell you much about the plot. It's hard to find things that aren't so bad they're unwatchable but aren't good enough I turn on the subtitles to follow them after a few minutes.

Of course the Boomer extinction burst of evangelical prudishness is cover for late stage capitalism misoginistically shoving self-employed young women back into retail jobs, and the war on cash (no untracked purchases for-profit corporations can't individually veto and PUNISH you for trying!) has literally escalated to police seizing armored cars. (And no, crypto solves nothing.)

I'm confused by Mark Cuban. He's a billionaire, and he's maybe possibly doing a good thing? Most likely because he wants to run for president (which should DEFINITELY NOT HAPPEN), so it's really for his own personal gain, but... I wouldn't want to STOP him from subsidizing affordable medications, if it is true? It's not Mackenzie Scott's get-out-of-guillotine free pass (checkboxes: did not voluntarily become a billionaire, is convincingly and consistently attempting to STOP being a billionaire, has embraced Uncle Ben's Spider-Mantra in doing so). Mark Cuban isn't absolving himself of the slumlord guilt of having taken a billion dollars from poor people and hoarding it (starting by giving it BACK), he's doing the "little people" a favor by pushing back against a specific societal problem that plutocrats have been consistently exacerbating since 1970 (here's a lovely writeup on that). But that IS one heck of a favor... Maybe it means he goes to the back of the line leading up to the guillotine? Hmmm... (Obviously it's not my call, but I'm trying to figure out what to lobby for. In a just society, hoarding a billion dollars while anybody is homeless would be a capital offense.)

This is an unusual exercise because most billionaires constantly broadcast how bad they are, and even PR-minded twerps like Elon Musk are much clearer: sure Muskrat's cheerleading the advance of cool tech, but none of it's happening because of him. He bought Tesla, he bought Solar City, he bought Maxwell. All the advances in electric cars and solar panels and batteries were already happening and he literally purchased credit for other people's ongoing work. ("Oh but they might not have been sufficiently funded otherwise" that's only because for a hundred years the USA massively funded basic research but we STOPPED when Reagan lowered the anti-billionaire tax shield until plutocrats could hijack the government and iteratively "drown it in a bathtub". The Boomers have been eating the seed corn since they took over.) Musk's assets have been mortgaged to the hilt for decades, all of Tesla's money (like Uber's) comes from friendly financiers willing to hand him tiny slices of the $21 trillion the USA printed after the 2008 mortgage crisis and gave to the BANKS rather than using it to bail out any homeowners or students with loans or the piles of predatory credit card debt... That man is a scam artist. Adam Something and Common Sense Skeptic have multiple video take downs of his endless dumb ideas. Ford's F150 Lightning is still restricted by the dealership model, even though they're inching away from that model they're hamstrung by stupid protectionist laws from years ago, and nothing Musk did to punch holes in that for Tesla fixed the dealership problem for anyone else, because he's not ABOUT fixing anything for anyone else. He's about getting a spotlight shone on him as a savior because he inherited african blood emeralds during Apartheid which gave him seed money to hang out with finance bros during the 1990s dot-com boom.

But even without the threshold being "net positive", it's rare to see a billionaire where guillotining them immediately would have ANY discernable downside. Encountering a second brings us from "Spiders Georg" territory into... black swan event maybe?


February 6, 2022

Walked out to the table for the first time all week. It's been below freezing each night, and the UT Carrilon plays "I've been working on the railroad" (no really!) twice through very slowly starting at 9pm, but in between there's a few hours were it's quiet but not TOO cold yet.

Didn't really get anything done, but oh wow I needed the exercise. Couple hours of walking. Listened to podcasts in the wifi headphones. Good times.


February 5, 2022

Austin has issued a "boil water" notice. Not due to the freeze, due to "human error" which usually means infrastructure mismanagement and underfunding/understaffing.

Various links from my talk went up on the golug website. Basically stuff I'd pasted into the chat channel. There's no archive of the video that I can find, and the page itself doesn't seem to be set up to have a persistent link to any of the entries. Presumably archive.org will snapshot it at some point? Still, I'm glad there was interest, and I need to make my own videos on this.

Some good political news: multiple federal courts have struck down republican gerrymanders in upwards of 5 states. (Doesn't stop the voter suppression, propaganda networks, replacing the people doing the counting with partisan hacks... But still, step in the right direction.)


February 4, 2022

So it looks like logging into Google Embark (their contractor payment thingy) with my main email address has a hiccup, namely that I break everything.

I've had the google account my email is spam filtered through for many years, through multiple upgrades of Google's systems, and it's... kinda borked. For example, to this day I have to log OUT of google to use maps.google.com because otherwise it's not an "authorized service".

Elliott recently pointed me at a bug in issuetracker.google.com (that's what those /b/ links translate to for the rest of us) and I should probably log in to comment on it, but I used my "work account" (at coresemi.io) to view it. The gmail account my personal email goes through is some sort of "google domain" organizational thingy set up in 2008-ish by a friend who has since undergone a religious conversion and moved to Europe. Once upon a time gmail wouldn't accept mail for other domains unless without special setup, and if I ever had the domain-side login to that it would be a half-dozen laptops ago.

Both the organization account and my previous individual Google account had the login "landley" (yes, at the same time: initially different namespaces), and when Google collapsed them together it had a name collision and got itself VERY confused (meaning I couldn't use Google Plus for the first two years of its existence), and Google periodically sends me emails like this one from Thursday:

We are writing to let you know that your G Suite legacy free edition will no longer be available starting July 1, 2022. To maintain your services and accounts, you need to upgrade to Google Workspace.

Which I have historically ignored and they auto-migrated whatever they had to and email still worked so I left it alone, but I do not consider an account with a pending Windows Scheduled Shutdown timeout to be load bearing, and if it STOPS passing email I move the DNS record to dreamhost's mail servers and update the pop3 and smtp server addresses in my client.

I've been thinking of preemptively pointing my domain's MX record at dreamhost's servers anyway to cut the gordian knot, but to be honest have been reluctant to touch it. And now I need a Google account to use Google vendor stuff and should probably interact with android's bug system directly and...

The thing is, Google's "Embark" draws a bright line between "gmail" and "non-gmail" addresses for some reason, and moving the MX record in dreamhost's nameservers (where the domain is registered) doesn't clean out the account association on Google's side: it still thinks "landley.net" goes with the 15 year old fused pair of gmail accounts that have undergone multiple system migrations. The tutorial video warns that if I try to tell Embark my email is an external address when it smells gmail, it will throw an "email mismatch error".

I could use my phone's gmail account, but my phone has NEVER had authorization to spend money, except for one fixed-size gift card I fed it back in Milwaukee. (The problem with programmer paranoia is it tends to be justified, not that intel is any better. It's not that the phone is inherently less secure, it's that the always-on orwellian tracking device with the whole-room microphone, camera pointed at the user's face, and second-by-second GPS tracking is a much higher profile target, and reusing google accounts between phone and payroll smells WAY too much like reusing passwords.)

Besides, until recently I just created a new gmail account for each phone. I kept it for the most recent phone because I remembered the login/password and because I wanted my youtube music playlist, but youtube has lost that capability and no longer lets you listen to two consecutive songs without 60 seconds of advertising in between. Youtube has taken away the POINT of having a persistent account for phones, and I'll probably just create another one once the Pixel 3a stops being supported (or allowing battery replacements).

I asked Elliott if there was somebody at Google I could talk to who could maybe apply cleansing flame to the situation, and he's never even MET anybody who works with gmail, and doesn't believe there are any at the Googleplex. (My own theory is it's entirely automated.)

Anyway, if you're wondering why I compulsively document and make everything be a self-contained reproduction sequence script with minimal external dependencies and regression tests... (And also why I use a vanilla unmodified distro on my laptop with a fresh reinstall each version upgrade and a written checklist of setup changes, and why I leave my phone running the stock Android version with nothing sideloaded.) Bit-rot is sadly real: the context shifts out from under you, the space something lived in stops being maintained, migration copies symlinks but not what they point to (at a dozen different levels of abstraction), they back up the data but not the reader/parser...

I got innoculated aginast this early, by the time I'd moved from a C64 to Amiga to DOS to OS/2 to Linux I was ALREADY expecting "this too shall pass" and becoming a digital pack-rat. But it's hard to be paranoid ENOUGH...


February 3, 2022

Austin is paralyzed by an ice storm. HEB is open but understaffed, which is ok because there's almost nobody here. (And then they closed at 5pm.) This is nothing like last year's blizzard, though, this is just our version of a snow day, we had a worse ice storm than this the first year I moved here. (That was over an inch of ice on everything, this is like, a third of an inch tops. Makes the trees pretty, but they'll live.)

Politics is unchanged: the GOP is a terrorist organization, missouri has legalized lynching, and Oklahoma just outlawed teaching evolution. (Because if christianity is wrong about where we came from, why would it know where we're going to? Must preserve useful societal control where the man molesting the choirboys is the man telling the choirboy what is and isn't moral, and has the child confess all thoughts to him in a small booth where holding back any private thoughts the child DOESN'T confess is its own sin, and then if the child breaks the seal of the confessional to show where on the doll the man in the frock touched him, he will be tortured in hell forever. And when the boy grows up, his stockholm syndrome will fight sex workers and defend priests.)


February 2, 2022

Yay, Lars pointed me at the Google individual contractor signup process (but warned that there are usage limits in a given year, presumably fallout from the Microsoft permatemps lawsuit). So I don't have to go visit my credit union to open a new bank account and so on, yay. They have an unlisted youtube video walking people through the web wizard. (I'm not sure if Embark is the name of the new system or just the wizard part. Still, it's quite straightforward, and I should totally make the account and do the thing. Any time now...)

I poked Jeff about whether he's running payroll, and he wanted me to commit to working for him full time and blowing off Google as a condition of paying me for this month. He wouldn't say it in so many words, of course, but "being committed"... He IS my friend, and I'm sympathetic to his position (he's understaffed and trying to hit ambitious deadlines to get the next tranche of funding), and I want to see everybody's projects succeed, but I'm still recovering from stress burnout from Stenograph (and the pandemic and the trump administration right after riding down SEI in the first place) and CANNOT overcommit myself right now. (I'm already to the "want to lie on the couch for a week and do nothing" stage.)

Speaking of overcommitting myself, I really need to do the toybox videos I prompsed my Patreon supporters. I have scripts, I need to sit down and learn blender video editing (the way I need to sit down and make that Embark account and track down my shoe size and blood type and whatever else it needs; finish watching the videos first, probably. Yes the video editor is a tiny bump on the side of a large project, but it's no siller than Netscape having a built-in email client was, and I used that for years. This one does not require installing half of KDE to boot it.)

Tonight I gave a mkroot talk at Orlando Linux User Group (because their call for talks went across the Silicon Valley LUG list I'm still subscribed to from when Jeff and I gave a talk there years ago, and I decided a practice run at the mkroot talk I need to do for patreon wouldn't hurt). They're using a web page called jit.si that can screen share on Linux assuming you're not using any CPU for anything else. (I have 4 processors in this laptop and it used half of each of them just to do the tiled Google Meet thing even when I wasn't presenting.) I walked people through the three sections of the mkroot source (setup, creating the root filesystem, kernel build and packaging), and referenced the initramfs docs I wrote years ago (and need to update: the new domain for the web archive links is lkml.iu.edu), and the miniconfig docs, plus links to some of the kernel patches I've sent upstream over the years to actually fix issues I'm working around. (Of course they were ignored, I usually give up after resubmitting 3 times and the successful ones need to be consistently resubmitted over and over for at least five years years with full-on dead parrot sketch levels of explaining why.) I even briefly went into the dropbear build to show what adding packages looks like. During the Q&A I diverged into 0BSD with a little backstory of how I wound up there...

Now I need to figure out how to break that material up into a series of 5-10 minute videos that progress logically but aren't TOO hard to understand out of context. Hmmm...

HEB was positively SLAMMED today because there's an ice storm coming, so the line to use the self-service checkout machines ran all the way around the deli and into produce aisles. They apologized for being understaffed (despite offering $17.50/hr to start according to the signs), but it's panic buying. Last year we went down under 10 degrees celsius and STAYED there, all the pipes froze including the gas pipes leading to the turbines generating electricity: it comes straight up out of holes they drilled into the pipes with no processing, which means they didn't bother to remove the steam and such mixed into it, which means if it's cold enough ice plates inside the pipes until they close up.

Meanwhile solar panels are self-de-icing if you install them at a slight angle, because the parts that aren't lit up act as resistors and heat up when current from the lit bits tries to flow through them, so the snow on them melts at the bottom until it slides off, and the newly revealed bits collect more electricity to heat up the remaining parts faster until it's all clear and dry. (This is why casting a shadow over part of a solar panel disproportionately reduces the amount generated, and they use microinverters to wire them up individually instead of in series when they can, so the shaded ones merely don't generate instead of consuming the power produced by the other panels).

But that was a hard freeze lasting days. This is just getting down into the 20s overnight (below freezing but not by that much) with rain putting a thin layer of ice over everything, and then going back up above freezing during the day and then down into the 20s again the following night (repeat for most of a week). That's not even climate change, Austin was doing that when I moved here in 1996. (We have six weeks of winter every year, they're just not consecutive. Last year we got in trouble because two of them WERE consecutive: the destruction of the Polar Vortex means nothing's keeping the the arctic winds up where they belong anymore. So the ice up there doesn't get built up and maintained, and all the cold leaks down here where it's just gonna melt again immediately. Leaving the freezer door open does not mean your frozen peas are going to last longer just because it feels so cold in the kitchen. It's a BAD sign, not a good one. The cold should not be down here, it should be up there: it went walkabout because the oil companies screwed up the weather.


February 1, 2022

Turned 50 today. Forgot until well after I woke up, and then wound up taking the day off and (let's be honest) moping.

Fade ordered me a fresh set of headphones. Yay. (I do break them every 6 months or so; they have planned obsolescence plastic hinges that are always the first thing to go. If those were metal I'd probably still be on my original pair. Capitalism again.)


January 31, 2022

The plan for today was to file new LLC paperwork at the Secretary of State's office when they opened at 8:30 am, but it was pouring out when I woke up. I tried to wait it out (the #10 bus picks up across the street but I still have to walk several blocks downtown and then I'd planned to walk to UT from there to work with my laptop), but by around 2pm it was clear there wasn't going to be a good gap so I went anyway.

Talked to the LLC specialist in the little booth for half an hour, and came away with a stack of paperwork to fill out and return. They say that with covid it could take up to 40 days to process, although I can bribe them an extra $25 to cut that in half, so that's nice.

I have to do name clearance searches for a new name because my old LLC from 2008 (Impact Linux) was apparently never properly shut down and still sort of zombie-exists but not in a useful way. (It went "inactive" in 2011 due to unpaid annual continuing-to-exist fees. The person who handled that at the time was born again into one of those religions where you get a new name, dress differently, and move away from everyone who knew the old you.) At this point properly killing the old LLC off means fully resurrecting it first, which is MORE paperwork and fees than just starting over with name clearance searches for a new name, which is like finding an unused domain or twitter login, only worse and probably requiring trademark searches too (which the initial state search doesn't do). Decisions, decisions...

Got caught in the rain on the way back, dried off at the Wendy's in Jester Center. Which is open 10am to 5pm, and for once I was there during daylight hours. Bought a new frosty keychain and everything, although sadly I didn't have those coupons with me. And then trying to walk home from UT during what SEEMED to be a break in the rain, it started pouring enough I gave up and asked Fade to order me a Lyft while huddling in a parking garage. (I haven't got the app on my phone; nothing on there is authorized to spend money. Yeah, I should have just taken the bus home, but it LOOKED like a gap in the rain. Tricksy weather.)


January 30, 2022

I went out to jack in the box today (the sign on Wendy's said the lobby is only open for take-out orders, which if true would be an improvement) and got a combo. They were out of the #1 and #2 combos but still had the junior burgers. Their self-serve soda machine is down but they can still fill sodas from the drive-in machine. I sat at the wall table near the outlets and read fanfic. Then I went into HEB and bought a can of milk tea from the interdimensional foods aisle and read more fanfic. Then I went home and napped while my electronics recharged, and watched some anime with Fade in the evening, and walked out to the table (2 miles each way, it's good exercise).

(Ever since high school I've tended to absorb the writing style of what I've been reading recently, and that author does a lot of matter of fact descriptive statements. You have been warned.)

At the table, but... really tired. I am not load bearing when it comes to ANY additional stress right now, not even GOOD stress. I am highly pleased to be starting work on Google's toybox hermetic build todo list, but tomorrow I have to fill out paperwork with the state of texas (can I reuse the name Impact Linux from 2008, or do I need to find another one that passes a conflict search), and then do I need a federal EIN or not? I finally got the tax envelope from Triple Crown (now that the box of stuff has returned to Stenograph; packed it as well as I could with those air baggies from amazon and wads of paper, but that thinkpad never had a case, I hope it made it there ok) and need to file taxes for 2021 soon, I hope Juanita can tell me how to do the "passthrough ignore" thing for a single person corporation.

I want to get work done. I need to get a lot of todo items cleared. I wrote a bunch of things I need to remember to do on the back of an envelope this morning, and of course left it at home.

I did a little programming doodle yesterday, which was great stress relief. That's the kind of programming I do for fun. I was going to do a quick ldd and posted about it on the list in hopes somebody had a trick to make dlopen() work from a statically linked context (ideally to portably poke a shared library loader to tell me about foreign libraries in a chroot context; I want the load address and maybe whether all the symbols resolve, but I don't need to USE the result), and Elliott reacted like I'd tried to poison his cat. (We've been in a long thread about it all month.)

My working style has always spun off todo items faster than I can close them, and the really satisfying work sessions are the ones where I can close tabs, finish stuff, document it, and ship it. I'm trying to clear my plate before diving into a new chunk of todo, and... I'm having the programmer vesion of weak noodle-arm can't lift anything. I'm TIRED.

I'm about to sign up to serve two masters, because Jeff's doing the J-Core ASIC that I REALLY WANT TO SEE SHIP, and Google's finally ready to get serious about toybox in a way I REALLY WANT TO SEE FINISH. And first I have to work out the division of my time between them (both want me full time; I would LOVE to do both full time) and do a bunch of paperwork.

I need to do more toybox videos, those should probably be weekly. And I signed up for a lightning talk at golug on wednesday (the topic is an intro to mkroot), and Jeff and I are speaking about making an open ASIC at Japan Technical Jamboree #78. Both online via different streaming clients, of course.

Still haven't decided if I should submit an ELC talk. My "not sure I really want to go" talk strategy has been to submit ONE proposal instead of the usual four or five for the committee to chose from, but then I feel rejected when they don't select it. I'm aware that makes no sense, and yet I know my failure modes. And coping strategies. I have decades of experience representing a project rather than myself, and met my wife when she was working the SJGames table at various conventions I was pitching Linucon at. I could be poised and confident pitching the convention to potential vendors because it was a great convention and other people not seeing that was their loss. I was just coincidentally creating it, the work stands on its own. But "would you like me to talk" can't avoid being at least a little about me, it's not a context in which I can easily intellectualize emperical metrics to moot a subjective judgement, and I can't dismiss it with "this person's opinion doesn't matter" when I'm the one who asked to be judged.

(Oddly, if I submit 4 topics and they choose none of them, it's easy to dismiss the conference because clearly our interests do not overlap. It's less of a personal rejection. I didn't ask them to trust my judgement about what would make a good talk but offered them a menu, and they didn't want anything on that menu because they don't get why any of that is important. Ok then.)


January 29, 2022

Sigh, I need to vent but I'm too tired. Fuzzy and I tried to walk through the Wendy's drive-through today (corporate is still mailing out coupons) but they were too understaffed to serve food. (When we knocked on the little window, the guy behind the register ran away, and the woman who (eventually) answered said come back in half an hour.)

Here's a good story about how the last wave of american prudish censorship got beaten back in the 1980s (between panics about dungeons and dragons being witchcraft and AIDS being a judgement from some god or other). When television ruled the culture they got a strangehold on broadcast television and vetoed every image and word spoken. Now that late stage capitalism has its tentacles everywhere they're making moral judgements about every dollar spent and rendering those they deem insufficiently pure "unbanked", and the gerontocracy's promises about properly funding the IRS to get billionaires and corporations to actually pay taxes turn into monitoring the poor to make sure the have-nots pay every penny of tax on every tip and side hustle. (Of course the increase in the IRS's budget is all about punching down, not up.) Here's a good link about how plutocracy is a problem from my wife's tumblr, and Dr. Sarah Taber pointing out that the war on camgirls is part of plutocracy pushing back against the great resignation: how dare anyone pay their bills WITHOUT flipping burgers for a large corporation.

A clear explanation of how modern christianity turned into a toxic death cult of doomsday preppers, and thus why church attendance is plummeting.

David Graeber talked about "direct action", which takes lot of forms.


January 28, 2022

YES! Google has approved funding to sponsor toybox work! (Specifically to advance the "hermetic build" work I've been poking at for many years.)

At the start of this (a full 6 months ago) Elliott didn't think I'd need to resurrect my old LLC, but now that we're actually doing it... yeah, down to the Secretary of State's office monday morning to fill out paperwork.

And I'm still hip deep in the J-core ASIC work and don't want to leave Jeff in the lurch, so I'm presumably doing each half-time for a bit? No idea how long that's going to take, but I want to see both these things FINISH.

Speaking of which, back to explaining how hardware toolchains work:

The toolchain we're currently trying to make our ASIC mask with is qflow, which is a set of build wrappers around four other tools (greywolf, netgen, qrouter, magic), plus then you need to feed it the fab's #include file with the standard cells for the process you're actually trying to build a mask for.

I mentioned last time how insanely proprietary most fabs are: their SDK (except it's a PDK here, Physical Development Kit) is TOP SECRET. Various NDA material has leaked over the years (especially for the older fabs that have been passing this info around for decades), but if you present a self-made mask to such a fab without a good explanation of how you got it, they will be EXTREMELY unhappy with you. The fab WANTS to get your netlist and make their own mask, because that's a service they can charge you for, and sprinkle in lots of per-chip royalties for their libraries, and it means your design is completely unportable so if you ever want to fab it through a competitor you basically start over from the netlist and pay all over again.

BUT: back at the start of the pandemic Google paid one fab (Skylake) to publicly release their PDK data without an NDA, for a complicated but powerful 130 nanometer fab process with its physical facility up in minnesota somewhere. This includes (quite a large) standard cell library, and corresponding process data such as how thick the P in a P/N junction has to be and how close two wires can be together before they interfere. (There's hundreds of weird little corner cases worked out by scientists in lab coats when they were designing the fab process, which are usually part of the NDA data without which your placer can't place and your router can't route. Sky130 is extra weird because the layers are different sizes: five volts up top, 3 volts in the middle one volt down at the bottom, so it can do a lot of flexible signal processing stuff and is frankly a bit odd to make a CPU with.)

Before this, the other main option was the OSU standard cells, which a university made years ago using extremely conservative estimates for a very old process, and would PROBABLY work... if spaced widely and clocked very slowly. And original targeting 350 nanometers, I think? It's a terrible option, but at least does not involve an NDA, and was used to develop a lot of these open tools back in the day. And I've talked about Graham Petley's work before, but it's more a generator than a ready to use PDK package, and the guy who did it apparently retired to paint watercolors. Jeff was involved in that project back in the day and could pick it up again, but there's a lot of heavy lifting to actualy use it in a real modern process and we'd like to establish a working baseline first. But the advantage there is you could (in theory) assemble an PDK for any process from maybe 25 nanometers on up. Below that it gets weird and quantum, and they cheat by turning the cells on their side to pack them more tightly together...

The standard modern file format for this stuff is LEF (Library Exchange Format) and DEF (Design Exchange Format). The PDK is generally LEF and DEF files, and the final mask file is a big DEF file. The netlist could be supplied as a DEF file instead of as verilog but usually isn't for historical reasons, and qflow isn't JUST a pile of glue scripts but also has a half-dozen little C programs to do file format conversions.

So, back to qflow: Greywolf is the placer, qrouter is the router, netgen does multiple things (most notably collating the list of labeled pins into a list of networks), and magic is the design rule checker (and it's a GUI visualizer, and format converter, and it actually calls back to netgen to perform some of those design rule checks).

The input to qflow is a netlist describing the circuit (which can be a bunch of formats but verilog is popular). This is still literally just a list of components with pins, and a corresponding list of "nets" which are just groups of pins to wire together. (I was calling them "pin pairs" until Jeff corrected me that there can be 3 or more pins in a net, all of which get wired together so current flows between them all. So networks, not just wires.)

The output is a "routed netlist", a bit like a PDF or photoshop file (except in DEF format) with a bunch of layers each containing a bunch of rectangles. Each component from the input netlist has been assigned to a physical location (layer, x, y, and orientation: greywolf places cells in a grid but can rotate or flip each one to make the routing work better) and broken into the series of rectangles of each type of material that implement that component. Each net from the input netlist has also been filled out with the set of rectangles (on metal layers, plus the vertical bridges connecting layers) needed to route the connections (I.E. the output of place and route).

The rectangles on each layer are all filled with the same thing, like "metal" or "insulator" or "dopant" (the chemicals that turn silicon positive or negative) or vertical connectors between layers punching little squares in the insulator layers. These layers are defined in the fab's PDK, and the PDK also has rules for each layer describing how thin the rectangles can be and the minimum gap needed between them, and the design rule checker's job is to verify that none of the PDK's rules are violated by a given DEF file generated by the toolchain: it checks the output for errors after you've generated a mask candidate.

The components don't just connect to each other, they connect to power and ground and clock. On the outside of your chip is a "pad ring" with special cells that talk to the outside world; there's generally a separate pad ring generator that produces all that, and then your place and route fills in the space inside the ring and links up the relevant nets to the ring's connections to satisfy all the references to the outside world.

The hardware version of QEMU is called spice. It's a physics simulator package that can actually "run" a circuit in a mask definition file. (Very slowly and unreliably, and it hasn't benefitted from 30 years of Moore's Law the way the other tools have, although there are ongoing attempts to rewrite it. But if the stars align and you chop your chip into tiny enough pieces, it can show you little pieces of your circuit working before you spend $$$ to fab it.)

So that's what we're TRYING to get the ASIC toolchain to do: combine a verilog netlist (produced by GHDL from the ICE40 J1 repo) with the sky130 PDK to run it through the qflow tools to place and route a mask file. We also need some libraries, most notably an SRAM generator (what's wrong with openram is its own whole writeup, but the tl;dr is it SHOULD have a few bits of SRAM available as a macrocell we can stick in the PDK and place and route instances of like any other component, and instead it tries to takes over a chunk of silicon wafer space and produce raw rectangles, routing its own power lines and everything (badly). In software terms, we want some C functions and it gives us a .o file with a giant binary blob in it that isn't even -fpic).

One persistent bug in qflow is that verilog uses an insane escape sequence for names with punctuation in them: it sticks a backslash on the front and a space at the end (which either gets stripped so the symbol no longer matches, or causes a syntax error, in any other format the symbol name is copied into). The proper thing to do when converting from verilog to any other format is to strip this escaping, and re-escape any characters that need it in the new file's format. (So far, we haven't hit a case where anything WOULD need to be escaped anywhere else, it's mostly that verilog hasn't got a lot of concepts VHDL does so things need to be made very explicit for it, which involves creating compound symbol names to represent components of VHDL objects, and GHDL uses an "illegal" character (period) to glue those together to avoid potential in-band signaling conflicts. Period is a valid name component requiring no escaping in all the other file formats EXCEPT verilog...)

What qflow decided to do instead (which is nuts) was keep the leading backslash and turn the trailing space into another backslash. And it didn't do it CONSISTENTLY, lots of names leak through unconverted. Sometimes this broke the reader of the new format, and other times it meant bits of the design didn't match up (mostly broken nets because it couldn't match up pins when the converted and unconverted names didn't match at each end). The other problem is, while LEF and DEF don't seem to escape anything and just treat backslash as a normal character... json doesn't. And yes, one of the conversions is into and out of json, because that's what some tool wanted. So the json parser threw a syntax error trying to read the file, which we hadn't noticed before because it didn't get far enough to HIT that failure mode before we fixed several of the other conversions. (The name mismatches were showing up as design rule check failures, now they're showing up as a build break, and this is PROGRESS. Sigh.)

So the PROPER fix is to go back and strip out ALL the escaping: when the vhdl reader library encounteres an escaped name, it should use the escaping to parse the name, then strip it. Unfortunately, this both means digging through qflow's vhdl code to figure out WHERE to strip the escaping (it needs it to figure out where symbols end during some of the parsing), AND it means ripping out all the wrong conversions later turning the "\name [123]" format into "\name\[123]". (Whether or not the [123] should be part of the resulting symbol is one of those things I need to dig further into, but... probably? Work out "what would this do if the name didn't need to be escaped" and make it do that.)

It would have been easier if the earlier code had been at all consistent or systematic, but there's a lot of whack-a-mole fixes that happened here over the years, which need to be ripped out again.

Qflow was apparently trying to preserve the escapes because one of the big Design Rule Checks at the end makes sure the input and output files contain all the same components (nothing got dropped), so it wanted a conversion that was trivially reversible. But the question "how do you know when to put the escaping BACK so it looks like the original" was THE WRONG QUESTION to ask. You don't, you consistently strip escapes EVERY time you load data from verilog: you now know the length of the symbol because it's in a C string, and you don't CARE what punctuation is in a null terminated C string. Teach the design rule checker to strip the escapes when it loads verilog, so it doesn't HAVE to escape the data it loads from the DEF file.

And if you DO need to write the data out into a new verilog file, you know to escape that symbol name BECAUSE IT HAS PUNCTUATION IN IT. Same way the escape wound up there in the first place when the original tool wrote out the first verilog file. You don't have to preserve a decision an earlier tool automatically made, you just have to competently make the same decision again as needed.

So yeah: qflow is a set of build wrappers around four other tools (greywolf, netgen, qrouter, magic), and then spice (or Xyce) is a physics simulator package that can actually "run" a mask (badly), and it doesn't quite work right with verliog that was generated from VHDL, and it doesn't quite work right with the sky130 PDK. We're working on it.


January 27, 2022

Poking at toysh. Combining command history with continuations is horrible. Both of these multiline input patterns:

$ for i in a b c
> do echo $i
> done
a
b
c
$ for i
> in a b c
> do echo $i
> done
a
b
c

Produce the same for i in a b c; do echo $i; done history entry when you cursor up. The newline between "for i" and "in" does NOT take a semicolon, and the code stitching the multiple lines together into a single long line knows that. This means we have TWO line continuation types: ends current command and does not end current command. No wait:

$ echo $(
> echo potato)
potato
$ echo "
abc"
abc

When you cursor up, the history entry PRESERVES the newline in both those cases. So three states so far. And you can cursor left and right past the line break just fine, which is why I wanted the code I'm writing to be the basis of my text editor implementation.

Sigh, what does busybox ash do... keeps everything entered as separate lines in the history and does no stitching together. Makes sense, but a lot less useful for rerunning things.

Alas, that's all the time I have for this today. Jeff needs me to fix vlog2Spice and vlog2Def the way I fixed vlog2Cel yesterday, and then look at the other project he found that's also trying to use VHDL with qflow to build a chip, and see if we can collaborate at least on toolchain fixes...


January 26, 2022

Working on the qflow stuff (toolchain for making our own ASIC). Trying to debug various design rule verification failures, and work out what magic's display output is trying to say when there are a bunch of layers... Ok, back up: I should do videos about this stuff too.

We've had a fully open source "VHDL to bitstream" toolchain for a while, which compiles human readable VHDL source code into a "netlist" file describing the circuit, and then the toolchain links that netlist into a bitstream which you can load into an FPGA to run your circuit.

The fully open source toolchain uses GHDL to parse the VHDL source code, and yosys to produce the netlist, then feeds the netlist to icestorm to turn it into a bitstream for Lattice's ice40 or ecp5 FPGAs.

We've recently updated this bitstream toolchain: the previous version was using a package called ghdlsynth-beta that's since been merged into one of the other packages. Unfortunately the new one reads a vital component via dlopen() and thus can't be distributed as a statically linked binary, which isn't exactly an improvement. But anyway, open source FPGA bitstream generation toolchain (for lattice): solved-ish problem.

An FPGA is primarily made up of LUTs and switching fabric. Switching fabric is a bunch of wires connecting all the inputs and outputs of every other circuit in the cell, which run together into junction boxes full of programmable switches that control which lines gets plugged together or kept apart.

LUT is short for Lookup Table, which is a tiny amount of SRAM with a set of input pins and one output pin, so each possible set of inputs can produce a known output (and thus act as any combination of and/or/nand/nor/not gates turning a set of binary inputs into binary output) just by using the input pins as address lines to look up that bit in the SRAM.

Historically there were 3 big FPGA vendors: Xilinx, Lattice, and Altera, but Altera was bought by Intel in 2015 and isn't available to individuals or small business. The J-core stuff builds for Xilinx and Lattice FPGAs.

Lattice uses 4-input LUTs (each indexing 16 bits of SRAM), and Xilinx uses 6-input LUTs (64 bits of SRAM). Neither's really better, it's just the granularity with which you can program the circuits, but it does mean that 1000 Lattice LUTs is potentially less circuitry than 1000 Xilinx LUTs so you can't compare them directly. (Although it also depends on how efficiently the toolchain's optimizer uses those extra inputs and how many are left unconnected, so it's not a strict ratio either. It's ballpark estimates then "try it and see".)

Xilinx was a higher-end FPGA vendor than Lattice, but Lattice has been creeping upmarket. The ice40 line maxes out at 7680 (4-bit) LUTs, while Xilinx's "LX9, LX25, LX45" were how many thousands of (6-bit) LUTs each contained. But Lattice's ecp5 line scales up into tens of thousands of LUTs, albeit at a slower clock speed than the high-end (much more expensive) Xilinx chips. J-core runs around 66mhz on cheap (Spartan 6) Xilinx and 100 mhz on more expensive (Kintex) Xilinx chips. It runs between 12mhz and 20mhz on various ICE40 chips, and isn't expected to be much faster on ecp5 (although we can do the full J2 SMP on there). This also makes things like 100baseT ethernet, GPS correlators, and USB2 noticeably harder to implement on Lattice.

One reason Lattice is hobbyist-friendly is price: an ICE40 is $3.5-$5 even with the chip shortage. An LX45 Spartan 6 is $35-$50 depending on speed, and a similarly sized Kintex is easily twice that. (And that's just the bare chip, bought in volume. Then you need boards to put them in, and Xilinx needs more expensive boards.)

But the real advantage of Lattice is the community has reverse engineered how to program it. Xilinx spent a LOT more on lawyers to sue anybody trying to understand its IP, so you can program Lattice with open source tools quite well (we've completely discarded Lattice's proprietary FPGA toolchain), but the Xilinx equivalents are buggy and incomplete and only furtively developed when Xilinx isn't looking.

So you compile VHDL to a netlist and link it into a bitstream that loads into the FPGA to program the LUTs and switching fabric (and initialize any other SRAM blocks). Great. What's a netlist?

A netlist is actually two lists: a list of hardware components (such as "and gate"), and a list of the named I/O pins sticking out of each component. The linking-ish phase of the hardware build is called "Place and route". The "placer" assigns each component of the circuit into one of the FPGA's programmable cells (including all those LUTs). The "router" programs the FPGA's switching fabric to connect up the various inputs and outputs of each LUT (by matching up the pin labels in the netlist and wiring together everything with the same name).

Conceptually, a hardware toolchain is a bit like a software toolchain. A C compiler produces a bunch of instructions bundled into functions and variables bundled into structures, and then the linker glues them together using a bunch of annotations where thing X needs to connect to thing Y. (In static linking this is all done at compile time, in dynamic linking another program does some of it at load/runtime.)

In a hardware toolchain, the compiler produces components (called "cells") instead of ALU instructions, and instead of jump/branch instructions it has I/O pins to wire together (called "nets", short for wiring networks, because when you connect together 3 or more pins that need to share a signal, or power, or ground, or clock, those wires have to be able to branch.) The toolchain backend reads the netlist so it can place the cells and route the nets.

The main limiting factors on how fast you can clock a circuit are 1) how long are the wires the signal has to travel down, 2) how much electricity does the component at the end need to fill it up? Physically smaller circuitry has shorter wires and less capacitance, so clocks faster. But layout also matters: if a wire "trace" is too long, the signal trying to go down it won't have done its job by the time the clock advances, so your circuit doesn't "meet timing". You can annotate signals with minimum required timing (if this 100baseT transciever is passing 4 bits at 25mhz, the circuitry at the far end has 40 microseconds handle it), and place and route iteratively try to get the best timing and tell you if anything failed to meet its minimums. (You can "floorplan" by manually grouping chunks of circuitry together with more annotation to basically give the placer hints, but that's a more advanced optimization technique usually only done when automatic place and route just can't figure it out.)

The big differences between different targets is what components are available: the VHDL or Verilog parser produces an Abstract Syntax Tree and runs the usual optimizations on it, then generates a component list and net list from that. Each Lattice FPGA family has one set of components, each Xilinx family has a very different list, and each hardware fab has its own list of available components for making photolithography masks from. A given process's list of available components is called its "standard cells", and the last "generic" step in a hardware build is the AST, because you need to #include the target process's standard cell file to generate a netlist.

The reason we still have to use the Xilinx Web ISE package to produce bitstreams for Spartan 6 (the bigger and faster FPGA in the turtle board) is icestorm doesn't know how to place and route for xilinx, and packages like Symbiflow that try to target xilinx aren't as mature because Xilinx is proprietary and litigous and has ways of encrypting its bitstreams. We have an ok list of standard cells for xilinx, so we can use GHDL to create a netlist (and thus use a more modern version of VHDL than Web ISE supports), but then we have to place and route that netlist using Web ISE to make a bitstream. Unfortunately, Web ISE is a binary only package last updated halfway through the Obama administration, which won't run without a registration key: it's basically the Flash plugin of bitstream toolchains, if Flash had required individual registration keys for each install with a working email address and needing a physical street address (ostensibly for export control reasons). They can probably remotely disable it too, I haven't checked. (Xilinx is REALLY trying to push everyone to its new Vivado toolchain, which only supports the newer more expensive FPGA families. Think Microsoft trying desperately to pull a reluctant userbase off of Windows XP without the new releases being interesting enough to move them on their own. But Vivado is no less proprietary or tightly controlled, and can't make a Turtle bitstream anyway because they dropped support for the older cheaper FPGAs.)

The standard cells of an FPGA aren't just LUTs, there are also macrocell "libraries" including clock inputs, phase locked loops, SRAM blocks, integer multipliers, and in extreme cases entire DRAM controllers. It's a bit like pulling functions out of libc that would either be a lot of work to reimplment or which you can't really implement yourself in a portable manner (because they need OS-specific system calls). Clocks need crystals, phase locked loops need capacitors, etc. The gotcha here is that each library component you pull in has a finite number of instances available on that FPGA, which live in a specific place on the chip you have to wire the rest of your circuitry up to, a constraint which makes meeting timing harder.

Compiling for ASIC (I.E. making a hardware mask) is slightly different: we take the AST produced by ghdl+yosys and run it through a different backend to produce a netlist using that fab's standard cells. In ASIC there's no limit to how many of each component it can create, the limit is how much space it has to draw cells in. The ASIC placer puts your standard cells in a big grid, initially randomly distributed (modulo any floorplanning) and then performing a kind of sort, swapping them with each other to collectively minimize the distance between pins needing to be connected to each other. The ASIC placer also tries to leave enough space for the ASIC router to add the necessary wires in the next pass, using various heuristics (fancy word for an automated way to guess). The result is more or less a giant photoshop file full of many layers of colored rectangles, with metadata so you can trace back to where those rectangles came from if you need to debug stuff.

A hardware toolchain still produces components ("cells") instead of processor instructions, but instead of libraries it has the option to bundle them together into "macrocells" (which is just cells containing other cells recursively, but with the routing locally within them already worked out). Each macrocell can be treated like a tiny little chip that has I/O pins, each with a long complicated name saying which OTHER pin on some OTHER cell it needs to connect to.

The whole ASIC has a "pad ring" around it (thing crt0.o and friends), and sucks in "standard cells" (think libc, except this defines things like "how to make a transistor in this fab process".). The pad ring is what connects to the externally facing pins the chip will have connected up to it when it gets cut out of the wafer and packaged into a ceramic case. This has special I/O pad cells (GIANT capacitor-ish thingies to try to protect the chip from static damage, or generate an enormous output current visible from space from outside the chip), and GPIO blocks that let the chip remap which pins do what (so the same ASIC can be packaged into different form factors without making different masks; sometimes there's a runtime-programmable register that lets you swap stuff in and out, and sometimes it's fuses blown during manufacturing).

Fabs treat their standard cell libraries as crown jewels: you sometimes have to sign a contract with them just to get enough to make a netlist with, and then the fab wants your netlist and will do everything else itself. If you want just OUTLINE versions of the standard cells (reserve this much opaque space you can't route through on this many adjacent layers, with I/O pins connecting at these spots) to do your own place and route, that's generally serious NDA territory. And you will NEVER get the full version that allows you to make a complete mask, in software terms you can only make a .o file that THEY link into an executable.

Standard cell libraries aren't just transistors and nor gates, they also include big macrocells like the FPGA libraries do, but fabs charge per-chip royalties for them. Using a fab's SRAM or their DDR3 controller adds extra costs, which you pay again every time you run another wafer from that mask.


January 25, 2022

Back in 2006 I was invited (by David Mandala and Tim Bird) to speak at the Consumer Electronic Linux Forum, which I've attended semi-regularly since. This year it's being held in Austin, and I'm tempted to submit a talk proposal... except I should really be doing Youtube videos directly instead of having someone else record and post them. Maybe it makes sense to do a talk there to advertise my youtube channel? Which would involve me actually starting a youtube channel; so far I have one video and the scripts for several more I haven't recorded yet because I need to learn video editing.

Ok, technically CELF isn't in Austin this year, ELC is. (That's the new name the Linux Foundation peed over it to make it smell like them.) Except ever since the Linux Foundation grabbed it they've been trying to glue it to other random events in hopes its success will rub off on whatever random nonsense emerged from their bureaucratic focus group du jour. This year the Linux Foundation is calling the collected mess the "Open Source Summit" (since O'Reilly is no longer using the name: the Linux Foundation is nothing if not opportunistic, derivative. ostentatious... Is immitation still flattery if they're not sincere?)

The latest katamari they've assembled inclues all SORTS of random padding to look bigger. Linuxcon has always made me sigh a bit at the name, but that one's at least happened before, and I think CloudOpen also previously existed, but ContainerCon isn't realy a name, OSPOCon (Open Source Projects Office) is meta-bureaucracy, SupplyChainSecurityCon was CLEARLY just made up to cash in on current headlines, and then you've got Critical Software Summit, Community Leadership Conference, Emerging OS Forum, Embedded IoT (as opposed to all the non-embedded IoT going on), Diversity Empowerment Summit (really!), Open AI & Data Forum (not even trying anymore), and Open Source On-Ramp. (Because none of the rest has an educational component?)

So yeah, they're holding CELF in Austin this year and trying to pretend it's got 14 other events bolted onto it (not tracks, EVENTS), at least half of which they just made up. Ten years ago the event's T-shirt already had more sponsor logos than your average Nascar jacket and I'm trying to figure out if I should bother with it.


January 24, 2022

Oil producers are treating their entire infrastructure as stranded assets. No matter how high oil prices go, they're not investing in more drilling because the transition to renewables could shrink demand and make the price go negative again at any time. While the USA can scale up production, it's mostly chosing not to (except for finishing about half of the backlog of drilled-but-uncompleted wells it already found oil under but never installed pumps and hooked up to the output pipes, which got shelved and auctioned off when the price went negative and the drillers went through bankruptcy.) Other countries like Russia that have nothing else going for them are already pumping at capacity and can't increase further.

For a clue why the finance guys refuse to invest in drilling, look at the oil tanker market. The "contango" issues from last year where oil tankers were used as extra storage have all worked through, and there's now an increasing surplus, and I quote: "Crude tanker owners have been bleeding cash for over a year, and the market will have to absorb more new tonnage in 2022 than in 2021." Container shipping rates are at historic highs, but coal, oil, and LNG tanker rates are down 70-90% from last year, with one tanker that costs $26,000/day to run is able to charge $800/day now.

Shipping oil from the middle east to china has even gone negative again. When prices went up they invested and increased capacity in a shrinking market, but there was no depth to the demand, less need each year than the previous year. The money guys don't want to play musical chairs and get left holding the bag. (Rare comic books sell for huge prices BECAUSE they're rare: if you discovered a hundred issues of Superman #1 in an attic and sold them all at once, the price would plummet. It's even possible that ALL of them would collectively fetch less than ONE does today.)

That said, LNG tankers are doing way better than oil, because there's demand to take extra gas from the USA to Europe, to reduce the amount they buy from Russia, and it's in the USA's strategic interest to scale that up as much as possible (replacing Russia entirely if they invade Ukraine and trigger sanctions strong enough to personally starve Vladimir Putin to death).

As for oil, the USA became a net oil exporter in November 2019, and our oil exports are booming as the rest of the world pays through the nose for the stuff. These days the USA only imports oil to refine it, but we're converting our refineries to consume fracking output which is chemically much less complex than conventional oil because it just came out of the rocks that produced it and hasn't had time to soak through other stuff and dissolve a lot of contaminants, and converting the refineries means they can't consume imported "heavy" crude anymore. The demand slump during the covid pandemic has been the perfect time to convert idled refineries.

US refinery capacity also fell almost 5% last year, as gasoline demand fell 13% (not just lockdowns but electric cars, telecommuting, the Great Resignation...) Some surplus refineries are being turned into storage, others are becoming biofuel plants (which is stupid, but happening), and more are just plain closing.

As for the increasingly lucrative container shipping side of the Force, The profitability isn't because the industry is HEALTHY, but seems to be because it's been sabotaged by the Biden administration to drive reshoring and weaning supply chains off china. As one of the above articles mentions:

Deutsche Bank analyst Amit Mehrotra... noted that the retail sales-to-inventory ratio is now three standard deviations below its 10-year historical average...

If we assume a reversion to pre-pandemic sales-to-inventory and use current demand, retailers would need $821 billion in inventory. Taking the fastest inventory has grown — which was November 2021, up $12 billion — it would take retailers 17 months to restock.

This drives up the price of goods from china (those record shipping profits are subtracted from manufacturer's and retailer's profits), and adds delay and uncertainty to supply. And it's clearly an intentional strategy because the container ship backlog at US ports isn't decreasing, it's increasing.

This seems to be an intentional (but pluasibly deniable) strategy on the Biden administration's part to respond to China's "Wolf Warrior" diplomacy: US/Mexican manufacturing is faster and cheaper once we stop subsidizing international shipping. (Which is half of what the US defense budget's been doing since the Berlin Wall fell over 30 years ago.)


January 23, 2022

I continue to object to the privatization of government functions. Mastercard's war on porn (with the enthusiastic collusuion of visa) is bad and they need to be driven out of business (via postal banking or something; the phrase "being debanked" should not ever apply to US citizens). It's providing cover for the IRS requiring third party facial recognition to access your own tax data. Late stage capitalism is contracting into a black hole.

This is the same kind of BS that youtube vs verizon was trying to pull back in 2007: everybody uploading anything must preemptively dox themselves and provide legally binding signatures stating that they own all rights to sounds playing on the radio in the background so the corporations can easily financially harvest the little guys at will. Verizon didn't get it then, but they're using pearl-clutching about sex and children to get it now. (Nevermind the self-employed models and camgirls and cartoonists they're putting out of work, or that the real problems have nothing to do with sex: M&M Mars and Nestle are facing a literal child slavery lawsuit right now, and of course their response is to try to distract from it by desexualizing their advertising. Because the antidote to accusations of slavery is performative prudery, just ask the Catholic Church. Even when neither the slavery nor child abuse were sexual in nature, being loudly anti-sex makes you immune to any other attack on moral grounds.)

Right wing loons have been using "family values" to advance fascism for generations: Reagan did it, Nixon did it, and "The Fatherland" of nazi germany was literally named after their family values campaign. The first big nazi book burning was the library of the institute for sexology. Fascists LOVE chastity belts and the Prince Albert for the same reason they love fasting, hairshirts, and mortifiation of the flesh. The less healthy your emotional life is, the more you need the cult leader to channel all your frustrations through.

But regulatory capture so you can't spend american dollars without passing "family values" judgements imposed by private for-profit corporations? That's DISTURBING. It's a pity the Boomercrats aren't even aware what the actualy issues _are_. Upstanding german citizens wouldn't stand up to stop the 1933 nazi book burning because some of those books were about gay people and trans people and the founder of the institute was jewish. First they came for...


January 22, 2022

I get emails:

On 1/22/22 9:51 AM, XXXX wrote:
> Hi sorry to bother you,
>
> I'm a it enthusiast and recently I came across your name associated with some
> mirai sample.
>
> Can we talk about it?

My old https://landley.net/aboriginal prebuilt binary toolchains got used to build rather a lot of weird code over the years, including some old rootkit stuff, and those old versions of gcc leak the build path of the toolchain/libraries into the output. (The were uClibc based and glibc is incompetent at static linking, so the cross compilers you could fish out of debian with the qemu-debootstrap stuff were way less useful for making portable binaries.)

My toolchain got stale because I stayed on the last GPLv2 release of each package that went GPLv3, and I wound up fighting the linux kernel devs to revert stupid stuff (see the sources/patches directory on https://github.com/landley/aboriginal). I switched to musl-cross-make years ago but never posted binaries of those because I don't trust GPLv3 not to turn into a lawsuit without a large corporate legal department standing between me and the loonier parts of the FSF ecosystem.

(I keep meaning to try to convince github to build the toolchains and host the binaries for me as part of the build commit hook stuff, but I'm told NFT clowns abused that infrastructure for coin mining and it caps the CPU usage or something now? It's on the todo list to poke at it...)

Rob

Of course now that I've actually looked at microsoft github's workflow docs, it looks like they delete the retained output files after 90 days. So not a place to put stable links to release binaries. Oh well.


January 21, 2022

I should really get the next video up, it's been a week and a day. The problem's topic granularity: the next thing to explain is making a choot dir with the actual toybox --long paths, and that leads innto chrooting into it and going "ps shows nothing" and mounting /proc, and "ls -l has UIDs but not names" so making a simple /etc/passwd and /etc/group, and the problem is all that stuff is mkroot's job? Hmmm, I suppose a video on how doing is fiddly and deep, and then explaining how mkroot does it. But before talking about mkroot, I need to talk about building toybox from source...

Ooh, I think a contributing factor to the anxiety/queasiness is the matcha milk tea I found at HEB gives me an upset stomach. (Black tea doesn't, but raw tea seems to?) That's unfortunate -- it's very tasty -- but the timeline lines up. One way to find out...

I finally found an online japanese teacher I like (Cure Dolly). Pity she died in october. I was pretty sure the voice behind the CGI cartoon was a little old lady, but the RIP entry on her patreon kind of confirmed it. Still, the youtube videos are up, there's a blog, and she wrote a book. Lot of good work there I can benefit from.

Her estate manager posting about taking the patreon down (and having to manually hit "skip billing this month" each month until then) does highlight the archival format problem again. Ten year old clever essays on livejournal are still there (at least the ones that survived strikethrough) despite the russian mafia basically owning that site for over a decade now, but patreon can't stop charging people money and still retain the content with monthly manual intervention. Everything there is inherently ephemeral because capitalism.

Paid for a rideshare to the big fedex office on guadalupe to finally get that box mailed back to previous $DAYJOB. (The third machine they pointed at the QR code understood it.) Then since I was already 2/3 of the way to the philbin I walked there and hung out in the office space current $DAYJOB is paying for. The experience is profoundly mediocre. Today I noticed that the wifi is plaintext. They have a web page you have to log into to access net through them, but the first hop to the router is unencrypted so everybody in the Dobie Center, one of the largest dormitories in the middle of the sixty thousand student University of Texas main campus, can trivially snoop anything that goes by on a non-https link. Starbucks was already encrypting its wifi in 2006. Sure it's not MUCH protection but the equivalent of listening in via police scanner seems... not thought through?)

I asked at the front desk: connecting to wifi with a password is something they want you to pay extra for. Capitalism again. I'm back to using my phone tethering with the USB cable. Yes everything I access is presumably already encrypted and this extra outer layer is permeable with enough effort, but I don't want have to WORRY about it and I'm insulted they perform a task incompetently for the lower tiers and want to charge extra for performing the exact same task with minimal competence. (There's an ethernet jack on the desk, but I didn't bring a cat 5 cable.)

So far the advantages of this place over the table outside the geology building (or the seats in the biology building courtyard, or...) are:

  1. it's not so cold late at night
  2. hot water you can make ramen cups with
  3. the bathroom is closer
  4. fewer flying insects on warm days

On every other metric, it loses. Starting with "hours of access".


January 20, 2022

Huh, my blog is unsearchable on Google again. I tried explicitly searching for which year I first blogged about the three distinct failure modes that open source user interface development breaks down into (because somebody complained that VLC's wanted to add a video editor front-end forever but never has: that's the "separate engine from interface and hope an improved interface shows up by magic one day" failure mode)... and my blog didn't come up. Not when I quoted "landley" and not even when I added site:landley.net. I had to grep the files locally on my laptop to find that link.

This is probably related to Google popping up a "show simplified view?" thing when I look at my blog on my phone. I'd blamed android update du jour but it seems to be Google search policy changing again. My blog is hand-coded html in vi using a half-dozen tags all from the 1990s (mostly p, a, h2, hr, pre, and b with the occasional completely ignoreable span I keep meaning to teach the rss feed generator about). There is no stylesheet. How exactly do you get "simpler" than that? (And yet the android browser has consistently displayed <pre> blocks in the smallest possible font for years, rather than the same font size as the surrounding text. That's not an Android 12 bug, that was already broken back when I had a Nexus 5.)

Oh well. Google isn't ready to go the way of Sears quite yet, but it's clearly no spring chicken anymore. Constantly losing capabilities it used to have, even if the stuff it nominally decides to keep. (It's ironic that the Google Graveyard I prefer to link to is not the CURRENT google graveyard. It is in fact currently the 7th hit on Google for "google graveyard".)

Sadly, this is another example of large companies favoring centralized sites instead of distributed ones. Google can find twitter. Another blog entry I did years ago was about the Linux Foundation providing a "face" for linux (single point of contact) the way AOL provided a face for the internet back in the 1990s. Neither was ACTUALLY representing what it claimed to, but as with a man repeating a woman's words in a meeting and getting credit for it, they successfully stood in front of the real thing, took credit for it, and got all the attention/control/funding. (Pretty sure google wouldn't be able to find that blog entry either.)


January 19, 2022

Today's high was 81 degrees, and it's supposed to get down around 20 tonight. (For the rest of the world, that's just over 300 kelvin to 266 kelvin.) Not going to the table tonight.

The recently ended job finally emailed me some RMA authorizations, so I packed up the hardware I had from them into a box and taped the printout to it, and instead of dumping it in an unmanned drop box I carried the box a mile north to a walgreens with an actual human in it representing fedex (well, eventually)... who couldn't cope with the QR code the email attachment printed, nor could they accept actual money to mail the box. (They're just a drop-off point, I'd have to go 2 miles to find a real fedex with humans working in it.) In theory fedex can pick up from my house, in practice they require a credit card to set up an account to do that. Try again tomorrow, maybe the location on guadalupe just north of UT can deal with this? I could hire a taxi-equivalent. (Alas the loony governor who eliminated gun permits and registration destroyed the cab business 3 years ago. Lyft is... less evil than uber I guess. I had the rideaustin.com app installed on my phone at one point, but they didn't survive the pandemic, the "temporarily closed" on the website was 2020 and they made it official 3 months later. And of course Austin's last surviving cab company was bought by an app I've never heard of and won't be using.)

*blink* *blink* The tables at HEB are near random televisions showing news or sports or something (I have headphones, don't have to listen), and a commercial came on with a countdown telling you to do nothing for 15 seconds. It's apparently advertising the "calm" app. People need to not just be told to do nothing, but walked through how? Really? And some for-profit entity has decided to attempt to monetize that?

Today's android 12 bug: the recent update has made the phone come on in my pocket multiple times. Fade called and said she got "road noise" while I was walking home with the box fedex wouldn't take. Walking home from the HEB later I pulled out the phone to look something up and the flashlight was on, with the screen on and deep in the pulldown menus. Still later, I pulled the phone out and the screen was on in the google play menus at the "do you really want to disable play protect" confirmation prompt. One of these navigated slack into the info menu of the general channel when I switched to that app.

I think what the update broke is the proximity sensor that disables the screen when you put the phone against your head during a call. It's not noticing it's in the pocket, and the random walking motion of pocket against screen is apparently enough to swipe up sometimes and get out of the just-came-awake screensaver panel, and then do random things with the resulting icons. Alas, I can't cherry pick JUST the parts I want out of any of these "updates", it's all-or-nothing lateral progress where they broke random new stuff each time...

Charles Stross explaining to a reader how to get an unecrypted version of one of his e-books is a good example of bit-rot: this used to work and it stopped working because of a combination of version skew (gratuitous upgrade to python3 breaking existing code that worked fine) and late stage capitalism closing up yet more stuff to corner the market harder. This too shall pass.

I like to tell people that the corrolary of Moore's Law is that 50% of what you know about the software stack is obsolete every 18 months, and the great strength of Unix has always been that it's mostly the SAME 50% cycling out over and over, a reef of C and posix with waves washing over them. It's not immortal, but outlasting the competition has been quite the recipe for success over the years.


January 18, 2022

On Fade's advice (she's been dealing with anxiety for decades and I'm new to it; I had about 15 years of depression but usually not anxiety) I took an ibuprofen and melatonin last night, and did manage to get some sleep. Still under the weather. The second time Jeff got Covid in japan (South Africa variant at the time, now renamed "megatron" because mentioning geography is racist) he had several days where he was anxious and couldn't sleep, so I'm wondering if this is Omicron manifesting through two rounds of J&J and two previous infections? (Annual flu vaccine doesn't stop you from getting the flu, it just makes it suck less when you do. Fade pointed out you can order 4 of Biden's covid tests now, for delivery "eventually". If I had one I'd probably take it.)

Another long design call with Jeff (this time going through the open source ASIC toolchain build stuff: instead of feeding ghdl+yosys output into icestorm, you use qflow and magic and spice and so on), and I was wondering why I was out of brain after only 3 hours... turns out we'd been talking for almost 4 and 1/2. (I have a duration readout for the current call, but I hung up and called back to use wifi instead of my cellular bandwidth, and I thought the first call was only an hour when it was almost 2. Right, that explained it.)

So many toolchains: I build Linux with the sh2eb-fdpic one from musl-cross-make, which comes in cross and native. Jeff insists we need an ELF one to build a bare metal BSP toolchain around. (Speaking of which, picolibc is my new example of how horrible bsd-style license combinations can get, and I quote: "There are currently 78 distinct licenses: 10 versions of the 2-clause BSD license, 37 versions of the 3-clause BSD license, and 31 other licenses.") Then there's the open source GHDL+yosys+icestorm FPGA toolchain targeting Lattice ICE40, or it can output a netlist that we can feed into xilinx webice (swapping out the front-end lets us use a newer VHDL standard than that old xilinx toolchain natively supports). And today's talk was about an ASIC toolchain that produces a mask for the fab, from a LOT of inputs through a LOT of stages and then testing it is... horrific. (The "spice" current flow physics simulator did not blossom under 30 years of Moore's Law the way the rest of the tools did.) There was also a proprietary lattice toolchain but it was crap and we've fully replaced it with the open source one. Someday I hope to have superh support in llvm.

Speaking of the ice40 bitstream toolchain: I got my build script to build the new packages with the new options in the new order, and sent it to Jeff again. The build isn't QUITE producing all static packages: icestorm seems to be ignoring the STATIC=1 option now? It's still there in the plumbing. I may need to bisect for the last version that worked with STATIC=1...

But I should have a new snapshot to upload before long. My login credentials to j-core.org went slightly stale: my ssh key gets me in but the login shell reset to /bin/sh (and thus the Defective Annoying Shell which in this distro doesn't not implement tab completion), and I had to zap the ssh host key because of whatever reinstall it went through when Wale fiddled with the VM. Sigh...

In theory I can also have it build nightly on my tiny server, and then build the J1 repo and run the simulator and catch regressions. That's a todo item.


January 17, 2022

Did not sleep at all last night. Lay awake in bed stressed. Took advantage of the holiday to stay in bed until almost 1, but it didn't result in any more sleep.

Jeff did a new VHDL toolchain build script based on the script he did a couple years back, completely ignoring my rewrite that made it do things like "git update when the repositories already exist instead of git clone", and "allow the build to happen without net access from a known set of local sources rather than trying to git clone the abc repo in the middle of the yosys build", plus the config checks to autodetect whether you have gcc or clang, and use readline if it's installed but don't if it's not... Oh, and of course preventing one of the package builds from calling "git clone" of a subrepo in the middle of the build (by downloading it myself as one of the packages and symlinking it into the build where it goes, so it's an actual tracked and updated package instead of re-downloaded mid-build each time. That's kind of important. And statically linking all the binaries so the resulting directory can be tarred up and used on an arbitrary target system... I did a lot of cleanup and productization work on this build.)

So I've got his new build script (replacing the "beta" package that was kind of sideloaded into yosys with a final package that's more integrated and called a different way), and I've got my old build script, and I'm trying to hammer them together, but I still don't want to run big builds off of battery because my laptop can suddenly power down even with 50% battery left. (It's only done it twice, but it left an impression.) Yeah, I can taskset 1 the build to force it to use just 1 CPU. (Or taskset 3 seems manageable, which contrary to first impression makes it use cpu 2 | cpu 1.)

Not a hugely productive work day with no sleep and "don't think about a purple elephant" anxiety avoidance strategies. (Plutocracy has cornered the market on spending money, not just collecting a fee every time but passing value judgements about what you are and aren't allowed to spend money on, starting with the nominally indefensible and spreading out from there the way the first nazi bookburning in germany was the library of a gender studies institute supporting trans people 100 years ago, hindsight about which is what inspired the "first they came for" poem. Since the dollar is the global reserve currency, we're exporting this "table legs need stockings covering them because otherwise it's indecent" lunacy to other countries. And we can't even focus on this with late stage capitalism pricing everyone out of their homes, a school-to-for-profit-prison pipeline bypassing the 14th amendment, student loans immune to bankruptcy with loan forgiveness being the new pie-in-the-sky dangled in front of people but never delivered by the octagenarian president, the retirement age endlessly increasing as lifespans decrease... Right, don't think about any of that until I can be ANGRY again rather than scared about living in the christian version of saudi arabia.)


January 16, 2022

Stomach bug today, which makes me wonder how much of yesterday's stress was "my intestines are clenched up" reading as stress. (I'm still pissed about all those issues, but I usually get on with things better.)

Looking at a todo list dated December 2020 and going "eh, only a month old" is a "still writing wrong year on my checks" variant. It's "twenty twenty also" already, the whole year "twenty twenty defeated us" was in between...

Android 12 had a system update last night, and now I actualy CAN summon the nav buttons at the side of the screen when watching a fullscreen netflix video! (In a non-obvious way, it's not drag up it's drag from "guess which side it thinks is the bottom" but at least there's a WAY to do it again.) I don't know if this is because the update changed my settings from "always show the darn buttons" to "smartass hiding" again, or if they actually fixed something. I'm just happy not to be faced with it today and am not lifting up that particular rock to see what's under it right now: BUSY WITH OTHER THINGS...

(That said, since the update youtube has decided to freeze and restart itself twice, once with one of those minimized windows jumping halfway across the screen when I tapped on it and another ignoring the nav bar of a video until I hit the "back" button at which point it started replaying the video from the beginning? Wasn't doing that before that I'd noticed. So yeah, they moved the bugs around but getting the nav buttons back is a net win.)

I broke down and sent Version 4 of the CONFIG_DEVTMPFS_MOUNT for initramfs patch to lkml. Debian's had four years to fix its obvious big so I didn't include the workaround this time, but the reason I'm doing this now is I'm using this patch and may wind up shipping it, and I like being able to point where I sent a patch I'm using to lkml and show THEY decided not to merge it when litigous assholes contact me about GPL issues. (Oh, feel free to open that can of worms, I'm pretty sure I can create a PR firestorm to cut GPL usage in HALF given a good excuse like "the guy who started the busybox lawsuits can't escape getting sued". Let me tell you what the FSF did to Mepis and here's a lovely sound bite from the Samba maintainer lamenting how the move to GPLv3 was a mistake in a recorded technical talk, here's the sound bite version of why I left busybox (and the lwn coverage of it for reference), and here's my soundbite about GPL from the 2013 ELC talk, and the "rise and fall of copyleft" talk I gave at Ohio LinuxFest, now let me tell you about 0BSD and the massive decline in GPL usage that motivated the need for a replacement for copyleft... Anger is depression turned inwards, not entirely sure where anxiety gets filed (I'm fairly new to dealing with it) but I admit looking for a "noble cause" (I.E. punching bag I can justify to myself) is tempting. Bad impulse, I know. But if somebody wants to present themselves gift-wrapped...)

Anyway, posted patch upstream so as NOT to incite such. I wasn't the one who started things with GPLv3 either.

The reason I'm fiddling with this patch is the Turtle board has an unhappy trying to run the script to mount devtmpfs itself, because some variant of nommu and that kernel config (or maybe real serial having different tty characteristics from qemu's?) is triggering some error handling path that isn't dealing with low filehandles right. Toybox lib/ has a "notstdio()" function that does a while (fd<3) { int fd2 = dup(fd); close(fd); open("/dev/null", O_RDWR); fd = fd2;}, and xopen() and friends all use that under the covers so things like "tar" aren't going to interlace -v output with the archive being generated even if called with no stdin/out/error. But this cleansing step should NOT happen anywhere in the toysh file handling path because we DO want the special case of init being called with no filehandles so mount devtmpfs and exec redirect stdin,out,and err to /dev/console. Which mkroot is doing and it works on the OTHER architectures... but not on turtle.

I need to properly debug this, but I also needed to get Jeff a usable root filesystem he can rebuild from source LAST WEEK, and keep getting interrupted by other things. Rich gives us filesystems even he can't easily reproduce, because there was no build: he hand-assembled it and keeps adding to what's there over the course of an entire project. I have mkroot building a clean vmlinux+initramfs entirely from source with the ability to overlay arbitrary stuff in a reproducible manner... and it crashes on turtle instead of giving a shell prompt. Frustrating.

I sent the patch to Rich asking him to add it to his linux-sh turtle branch, and he said his branch isn't for the purpose of overriding the kernel maintainers' judgement. Dude, patches OFTEN take years to get merged. My "three strikes and you'r out" was because _I_ got tired of dealing with the insular kernel clique. They were waiting for Debian to fix their bug. "It got posted and nobody noticed/replied" is 100% normal for linux-kernel, and conveys zero information. Probably what'll happen to this one too since I didn't run get_maintainers.pl and cc: 37 random historical people who haven't been involved in years. I got cc'd by Andrew Morton on an madvise patch this week, and have been chronically cc'd on Moudy Ho's mediatek patches for months, and this is DESPITE years of distance from linux-kernel. Back before git, people were literally setting up cron jobs to resend to Linus. (The proper fix was to switch to git and formalize the hierarchy of subsystem maintainers patches go through on their way to Linus, but I had to "don't ask question: post errors" to jolt people out of the cron job orbital they were stuck in.) This patch hasn't been resubmitted recently because I'M the one who got tired of dealing with it.

So I'm maintaining my own Turtle tree now. I need to upload that to j-core.org...


January 15, 2022

The german police used the EU's covid tracking app to track down witnesses to a case unrelated to covid. This is why I buy things with cash whenever I can: I don't want every purchase I make tracked and sold to anyone who wants to buy a copy, and of COURSE governments buy collected data from advertisers.

I'd like to link to a long informative thread about the new comstock act puritanism taking down more sites, but unfortunately twitter just stopped being a public website. If you scroll down in somebody's feed without being logged in, about 4 tweets down you get a full-screen modal pop-up "you must log in to read more" without any dismiss option, and even if you "right click->inspect element->delete node" to force-dismiss the pop-up the javascript has disabled the scroll bar so you can't go further down. (There's a way to inject javascript "enable scroll bar" into the page using the debug functions, but there's like 3 different ways they could have disabled it and... I can't be bothered?) I can still read the website on my phone's browser, but twitter is dead to me on the desktop.

Here is the "tumblr safe version" of the famous 1896 painting by French artist Jean-Léon Gérôme, "Truth Coming Out of Her Well to Shame Mankind". The original can still be on wikipedia because that site is not supported by advertising. AO3 has been explicitly defending itself from this censorship for years.

During prohibition, the US dollar became a currency you couldn't buy alcohol with, and rule of law basically went away. The gerontocracy has now made the US dollar a currency you can't buy legal things in anymore (still clamping down on cash), which is really obvious when there's something to compare it to. The "any rando can sue a woman who MIGHT have had an abortion in texas" which california's copying to sue gun manufactuers was pioneered by Fosta/Sesta. The gerontocracy has forgotten the concept of catharsis, that not everything worthwhile is healthy or pretty.

On balance, the kids are all right, the adults are wrong, and the geezers are wronger. Every time, in every society, throughout history. As they enter their second childhood, the Boomers are reshaping the world into 1950s drudgery. We even have a prominent political McCarthy lying in congress again.

The problem isn't just the fuzzy line where "I know it when I see it" is banned, it's risk-averse vendors self-censoring, and nobody is as risk-averse as Fortune 500 corporations. A lot of adjacent content is still there from years ago, but probably couldn't be posted today, and the situation looks to get worse before it gets better.

Youtube doing things like destroying reaction channels is just a sign of youtube declining, but the gerontocracy is a larger social problem that's corrupted the financial system. Capitalism itself is a problem now. Capitalism stands between people and how they want to live their lives. Capitalism punishes cooperation even though nature is full of cooperation. Like royalty, society has outgrown late stage capitalism which now serves a small and declining number of royal persons whom society would be much better off without, while the majority of the population are exploited and treated as a problem.

This is not stable. The traditional responses from the rank and file range from general strike through malicious compliance, and of course for gerontocracy simply letting the geezers die. The ongoing health care system collapse is rapidly reducing life expectancy in the USA. Preventative geriatric medical care was already failing to keep up with boomer demand but now it's simply no longer widely available. Not just the general health care shortage increasing the mortaility of heart attacks and cancer and car accidents, but "keeping Boomers alive longer" is not what most people want to do with their lives anymore. There were around 2 million elder care workers in 2017, declining to around 1.5 million at the start of the decade, and since the start of the pandemic elder care staffing has declined by 220k workers. Of course AARP estimates that 20% of the population is providing unpaid amateur elder care, which tends not to be sustainable for individuals to do long-term. Multi-generational households provided care for kids and elderly at the same time, these days who can afford to have kids, let alone a house that big?

Speaking of Boomers refusing to look in a mirror (I.E. objecting to teaching "critical race theory"), the number of murdered children found under canadian boarding schools is approaching five digits. There were 130 such schools and they've fully searched 6 so far. Remember how the germans sent teams to the USA in the 1930s to study american racism? Columbus bumped into a continent with something like 130 million people living on it. There are about 4.5 million left in the USA and 1.7 million in canada, both due to centuries of centuries of genocide. Yes, that was a link to two prominent american presidents personally responsible for native genocide, here's a third. The civil war didn't even slow DOWN racism against everyone else. Custer's Last Stand was a Union Brigadier General going out to murder native americans because the USA didn't feel like honoring its treaties and had leftover military resources from the war, which we turned west to clear land for cattle (and thus cowboys): the entire "wild west" was made possible by racist genocide committed by the NORTH after the war. (Hence Boomers playing "cowboys and indians" as children: it was still within living memory.) The Chinese Exclusion Act was passed in 1882. Hawaii was conquered by mercenaries from the Dole pineapple corporation in 1893. The USA rounded up japanese americans (but not German or Italian) and put them in concentration camps by executive order of president Roosevelt in 1942. Here's George Takei (Sulu from Star Trek) talking about his childhood in an internment camp for his ted talk.


January 14, 2022

My Android 12 bug workaround for getting the "back" button out of a reclacitrant app is to use the pull-down menu, hit the gear, and then the menu app listens to rotation and can thus give you back/desktop/applist buttons. EXCEPT when you're already _in_ the pulldown menus, doing the network & internet -> hotspot & tethering -> usb tethering dance to access the control that WAS available with a long press from the "hotspot" pulldown button until they "upgraded" it away. Unfortunately if you went there from a widescreen video, and have now attached USB tethering, you can't exit OUT of the tethering menu without a "back" button, so I have to physically pick up my phone and tilt it despite my USB cable being worn out enough it tends to disconnect if I do that.

Meanwhile, my old "make initramfs honor CONFIG_DEVTMPFS_MOUNT" patch broke because this patch removed the argument from devtmpfs_mount(), which seems REALLY stupid except... it says it was already broken? What?

Queue debugging rathole (test it under mkroot!): I was worried for a bit that "mkdir sub; mount -t devtmpfs sub sub" would try to mount it on /dev. (Mounting devtmpfs in a subdirectory is perfectly legal and a fairly normal part of setting up a chroot or container, doing the stuff requiring root access before you chroot into it and drop permissions)... but when I list it, it's there. And THEN I was worried it hadn't recorded the destination properly (which would screw up df and umount -a and friends), but... no, seems to still work? Ok, it looks like the argument it removed was the SOURCE not the destination. Ignoring the 'from' address in a synthetic filesystem... is acceptable: if it's not using it then sure. Before (circa 2013 when I implemented initmpfs) it was just being passed along and recorded as-is, and now it's hardwired to "same as filesystem type". Right...

I'm still subscribed to the coreutils list but haven't set up a filter and folder for it yet, meaning stuff posted to it winds up in my inbox (where I delete it). I ALMOST replied to one of them, but decided against it because I'm not really a part of that project and going "in toybox I did..." seems rude. But what I wrote and then deleted was:

I made toybox's dirtree plumbing use openat years ago and the subtlety of purely directory local operations is somebody can do a "mv" of the directory you've descended into so "cd .." will wind up somewhere else, so a pure descent counting strategy is insecure. And keeping a filehandle open at each level means you hit filehandle exhaustion when dealing with arbitrarily deep directories[1] so what I wound up doing (at least for rm) was having a structure with the name and stat() info of each directory I've descended through and then when I "cd .." back up into it I'd re-stat it to make sure it was the same, and if not either fail out or descend back down from the top using the names.

[1] The default ulimit -n is 1024 filehandles per process, but posix says ANY limit is unacceptable because rm -r needs to handle anything, because even in the days of MAXPATH you could (mkdir a/a/a/a/a...; cd a/a/a/a/a... mkdir ~/b/b/b/b/b....; mv ~/b .) and then repeat that moving the top "a" to the bottom of a new MAX_PATH stack to make an arbitrarily deep path.

Which once again reminds me that tests/rm.test needs a test for zapping paths longer than 4k, which is queued up for my "tests within mkroot" because I dowanna play with that sort of thing on my host system.

I also truncated a reply to the toybox list, because I _do_ usually blog about this sort of thing rather than posting it there:

As David Graeber pointed out, our current system thinks that "your life has meaning" is a form of pay, and that you should only ever need a financial incentive to do something you don't want to see happen. He wrote at least two full books about this, The Untopia of Rules and Bullshit Jobs. (Three if you count "Debt: The first 5000 years".) And he gave a bunch of interviews.

I'm very happy to have supporters on patreon. I'd love it if that could pay my mortgage. But I've spent my entire career going for "useful/interesting work" over "lucrative", and the question has always been been "how much useful/interesting work can I afford to do before I have to go earn money instead".

The well-paying jobs I take are not BAD jobs. (I turn stuff I really don't want to do down before even interviewing.) Tuesday a recruiter asked if I want to work on a medical device (18 month work-from-home contract), and I could see myself doing that if I was on the market. But I could see any random CS graduate doing that too, they don't particularly need ME to do it.

The "pays really well" job I turned down earlier this month would have paid what I owe on my mortgage in about 14 months. (Admittedly pre-tax and ignoring all other expenses, but probably about 2 years to pay it off realistically. I feel guilty for NOT taking that job, it's irresponsible to work on FUN technology when I could be effectively securing my retirement... but I burned out badly just trying to stay at the recently-ended contract for 6 months, and still have a backlog of stress and anxiety from that. Touch gun-shy of jumping into the unknown again quite so soon, and I'd owe them my best. I'm not currently at my best.)

If I'd stayed at JCI I'd already have paid my mortgage off by now, but the project I was interested in there ended and the new one they transferred me to was a large team doing yocto with systemd on an SMP arm board with gigs of RAM and hundreds of gigs of storage that they really didn't need me for. The Google recruiter du jour trying to get me excited about "20% time" last week had the problem that (according to The Atlantic) it became "120% time" ten years ago, but the bigger problem is that I'm already DOING that. Context switching between projects stops me from doing "deep dive large block of time intense focus" things, which is pretty much what's LEFT on toybox. I've been picking low hanging fruit for years, what's left is mostly hairballs, and the SIMPLER of those are things like "rewrite lib/password.c and change 7 commands to use it at once before you can check any of it in". That one's not conceptually hard, but still a big flag day change, plus rechecking the "to libc or not to libc" reasoning confirming the justification of the new plumbing's existence. (I walk away from that sort of thing and revisit it periodically doing an "eh, do I still agree, did I miss anything..." I agonized similarly over doing dirtree.c instead of using fts.h out of libc, back in the day...)

To be fair the coresemi stuff swap-thrashes out my toybox/mkroot headspace too, but since j-core is another thing I would like to see succeed in the world (which I've already spent years working on), it's still worth doing to me even if it's "instead of toybox". And I've usually gotten more than 20% time to work on toybox from coresemi, since they use it in their products.

Jeff Dionne's also paid for a lot of work hours from the musl maintainer over the years. Jeff not only did uclinux, but back in the day he was Erik Anderson's boss at lineo and thus started the ball rolling on both busybox and uClibc, although Erik continued those projects on his own initiaitve for a long time, and my involvement started after Jeff's had ended. (Unlike the earlier history with Bruce I did know that uClibc had spun out of uClinux, but at the time didn't really know what that WAS. Now I know it was the initial port of Linux to nommu hardware; it needed a userspace because most gnu packages couldn't cope with nommu, and thus sort of became a distro but that was never really its intent.)

I still learn a lot working with Jeff, which is the real incentive to stay. The money is just barely enought to _allow_ me to stay.


January 13, 2022

I got my first toybox video uploaded to my website yesterday, and informed patreon about it. It's a good single take but otherwise unedited, so I'm poking at blender as a linux-friendly video editing solution. That's looking like a truly enormous time sink to properly come up to speed on...

Fade flies back to her doctoral program on friday, so I've been doing some stuff but NOT trying to disappear down a technical rathole for 10 uninterrupted hours just now. That said, I'm trying to ping the toybox list more about my design pondering, because blogging about it has had an editing and posting delay long enough other people can't really react to things before I've done them. (Although for the moment I'm mostly caught up, in part because my patreon made it to the SECOND goal so I should definitely clean up my performance of the first goal: keep the blog updated.) And as I said, I _do_ try to get the bikeshedding out of the way up front. :)

I switched to mostly blogging about programming rather than posting it to my lists after the third-ish time dreamhost punched giant holes in the web archives (see yesterday's "databases inevitably go away"), and while I do have all the posts in mbox files it's split among three different places: the list mbox, my inbox, and my outbox. That fragmentation happened because gmail's done filtering I don't want and have never managed to disable for a dozen years now, which scatters data and breaks threads. To reconstitute it, I'd have to write a program to parse those three mbox files, extract messages to: or cc: the mailing list (oh and some of the messages were cc'd to OTHER lists too so I need to check ALL the mbox files to be safe)... It's not that hard to whip up a python program to do it, I just haven't spent a weekend on this. (And then if I DID, what would I do with the result? Put up my own archive directory somewhere and keep it updated?)


January 12, 2022

Ok, I need to get off of gmail NOW. The net-nanny software has made it unusable.

I was trying to send co-workers a tar.gz of an old right click -> save as snapshot of an internal wiki page about GPS timings over a homemade hardware bus, now that database has gone the way of all databases. (Seriously, we knew about the archival format skew problem thirty years ago; even when you THINK you have backups, new programs can't read old nontrivial formats. Nothing stored in a database is going to be usably retreivable ten years later unless somebody's dumped the output to a real static self-contained storage format, and even then half the time your PDF is a cover page with no contents or an index into a backup system that no longer exists.)

But gmail is doing a net-nanny thing: "oh no, I don't like the contents of that tarball, it's got css files and javascript in it, you're not ALLOWED to send that tarball". I didn't ask it to, and can't STOP it.

This is not a receive spam filter, this is a SEND spam filter. I am not allowed to SEND that email because gmail won't let me. It doesn't properly understand the contents but THINKS it MIGHT be dangerous, and insists it knows better than I do what's in my interest to be allowed to send. It had to decompress the file and untar the file (not a format widely used on windows) in order to scoff at the contents of the file. And note how there's no "here's how to override it", the linked page is "Here is an explanation of why you shouldn't have tried to do that in the first place you naughty child you.".

This is not a service I consider usable anymore. This is not a relationship I want to have with a service provider. Dreamhost provides mail servers as part of its service package, I just hate moving my email because my preferred recovery mechanism for stuff IS email. Grrr... Dowanna, but... really need to.


January 11, 2022

At this point, everything the Boomers touch turns to crap. We're not going to pry their hands off of anything, they're going to destroy it and we'll build a new one from the rubble when they die.

Sadly, this includes the health care system. Texas is redeploying the morticians who perform autopsies to work in the intensive care units; people who have never seen a living patient in their professional careers but are technically qualified to perform surgery, therefore can monitor a pneumonia patient. (Hospitals are bled dry and the Omicron surge is worse than Delta because it spreads faster and infects more people at once; doesn't matter what the average case is like if now is a really bad time to get a heart attack, or cancer, or appendicitis, or an abcessed kidney infection, or in a car accident, or have a fall, or... And of couse, what are we spending money on instead of healthcare? Not forgiving student loans, of course...)

The Boomers are dying in a mushroom could of racism, hypocrisy, and historical revisionism. Can we all just agree that IP law needs to die with the Boomers? Capitalism isn't looking good either. Capitalism is was always self-limiting but add late stage Boomer failure and it starts taking down countries: apparently the mess in Khazakstan is bitcoin's fault. And a lot of the money laundered through there was used to Lobby the US government capitalist gerontocracy.

Boomer Media continues to circle the drain. Did you notice how Boomer news stories are sponsored now? Not just by showing commercials, but in the content: the coverage is spun to advertise some random company used as a nominal example for issues the company in question is always trying to profit from. Because Boomers can't conceive a capitalism NOT being the solution to any problem.

Covid is being hugely underreported. The great resignation isn't just people resigning or finding better jobs, a lot more are dying: deaths from all causes are up 40% from pre-covid levels, with many people dying months AFTER having covid. Not just retirees but line workers exposed to the public and to office environments crammed in with a bunch of other people. (This is the life insurance industry wringing its hands over massively increased payouts, so "who is covered" skews the numbers. In addition to the elderly being really expensive to insure, why would a Boomer want life insurance? They're not the one it would pay out to.) On the bright side, vaccines don't just protect against immediate death from covid, they mitigate long covid.

Mozilla's desperate plans to fund itself with bitcoin are so bad the people who actually wrote the software have publicly come out against the company. Meanwhile to nobody's surprise, the cloud remains fundamentally insecure.

The cop mayor of NYC needs to be defunded out of office.


January 10, 2022

Got up bright and early and so I could walk to the Philbin this morning to have a 10am call with Jeff to try out the new collaboration setup to work on the J1 asic. Except they don't open until 8:30 am and I set out a bit early so I hung out at HEB for a couple hours first (the tables there are a lovely quiet work environment early in the morning and they have purchaseable caffeine), then walked home to heat up a can of coffee and roll it down an incline at Fade at her alarm time (she visited Japan years before I did and fell in love with the vending machines; I do my best to simulate the experience), and THEN walk to UT just in time to make the call with Jeff.

We wound up on the phone (eh, Google Meet + a shared Google Doc + some sort of whiteboarding app) for 4 hours, which I suppose counts as a success. We got a lot of design work done, anyway.

Digging through old todo lists, I just deleted the "make ls sometimes exit with 2 instead of one" patch from a 2019 thread out of my main toybox work directory. There's a certain amount of accumulated shoveling out I've needed to do...


January 9, 2022

Sarah Taber Tweeted about Harry Reid far more charitably than I feel about him. I think justice delayed is justice denied and it's far easier to stall popular things forever than openly oppose them. I think that Harry Reid and his ilk are "good cop" on behalf of the plutocracy, endlessly saying that the suspect being questioned needs to give a little more cooperation and then MAYBE he can help afterwards, once the cops have a full confession signed in blood and you've pled guilty to something you didn't do and served your full life sentence without parole at a for-profit prison where you work without pay for your new owner.

Taber explains how when she was growing up the mormon church told people they were going to hell if they didn't vote solidly republican (and somehow kept their tax exempt status while doing this), and Harry Reid being a prominent mormon Good Cop instead of Bad Cop was a big deal. So his existence was apparently useful in very slightly pushing back against a persistent refusal to enforce existing laws against the rich and powerful fifty years ago. He then spent his entire career blocking progressives from doing anything useful via spineles equivocation and delay and lip service plus strategic incompetence. ("We tried but couldn't get it done, sorry, maybe next time, just a little more and then maybe I can help get you the pie I'm holding up in the sky someday".)

Note that Reid stayed around blocking change well past what USED to be normal retirement age essentially died in the saddle, only finally retiring because at age 75 he fell and broke multiple bones which is how they found the cancer he died of. Reid was 100% part of Nancy Pelosi's gerontocracy that exists to prevent AOC and friends from exercising any real power. Diane Feinstein is coming up on 90 and her memory is gone but like Ruth Bader Ginsberg she is determined to die in office because she's SPECIAL and nobody else could ever possibly do her job, which is hers by divine right for life. (That last article points out the republicans have switched to rotating committee heads, but the democrats use pure seniority and all their leadership is in their 80s. Chuck Grassley is EIGHTY EIGHT YEARS OLD and president of the senate. Pelosi is 81 and speaker of the house. The president they serve turns 80 this year. THIS IS NOT NORMAL. Nor is it remotely effective at addressing any of the real problems we face.)


January 8, 2022

LWN had a good example of why DLL hell (too many package dependencies) is terrible.

Trying to do toybox video scripts (the "write down what words to say in which order" kind) involves inspecting a lot of code I haven't looked at in a while and doing lots of little cleanups. It's the old problem that writing documentation results in changing the thing you were trying to document to be easier to explain.

I have a large pending change to lib/passwd.c including basically a rewrite of update_password() that changes the number of arguments it takes (by specifying which field "entry" replaces). Of course if I change the arguments to a lib/ function, I have to change every user simultaneously in order to check it in without breaking the build, and in this case that means updating the commands passwd, chsh, groupadd, groupdel, useradd, and userdel.

Except only "passwd" has been promoted out of pending, and even that one was kind of a stretch: it was merged back before I had a pending directory yet. I did some cleanup but didn't have proper test infrastructure for any of the /etc/passwd stuff. I'm just now getting there with mkroot.

The backstory on this plumbing is that way back when busybox didn't trust libc to implement it, possibly because uClibc was crap. And these days glibc has decided all access to /etc/passwd go through Pluggable Authentication Modules via dlopen() and thus statically linked binaries can't do this basic stuff either, and of course android installed each java applet as a seperate UID 15 years ago (before spraying the system down with SELinux rules and more recently adding container support), so bionic never implemented conventional posix user/group support. (Well, they've started recently except if you build toybox with the NDK and run "make tests" you get a bunch of "libc: Found user/group name 'root' in '/etc/passwd' without required prefix 'system_'" error messages... so no, this doesn't work there either.)

So I guess I should finish this and use it consistently. And doing so fishes 5 commands out of pending in one go, which isn't bad. And it's good to have in mkroot.

Hmmm, these commands should consistently support -R to work in a chroot, which means they can't use the library getgrnam() and friends with hardwired paths at _all_. (Which again: if we could, we should, so the reason for doing the x:y:z line parsing ourselves is because libc isn't reliably load bearing.) Right now the command implementations are mixing them, which can't be right...

Does this mean EVERYTHING should be using the new ones and not the libc ones? Hmmm. I got xgetgrnam() wrappers already, I could redirect it to new plumbing but... What's the right thing to do here?

I've had passwd compiling in my tree against the new lib functions for a while. Just got chsh and groupadd compiling again, but need to clean out the libc passwd/group/shadow functions from them still...

I note that lib/pending.h only has the lib/password.c stuff left in it, so I need to move stuff out of there and delete that. One more corner case: pending.h has #define MAX_SALT_LEN 20 and that seems kinda disgusting. I'm tempted to just use 32 in the callers? I don't have a #define for the scan_key() scratch buffer, which is something like strlen() of the longest scan_key_list entry plus a null terminator, and I've just been using 16 because it's power of 2 plenty long enough. That's kind of what I want to do here, but... is having the #define for a magic constant sillier than having the callers use magic numbers? Hmmm. (This also argues for static buffers. It's another one of those "the fighting is so vicious because the stakes are so small" things; if it was important the answer would be more obvious.) I should stick a TODO note about this somewhere, especially since I'm redoing the tty stuff at the moment anyway. This dirties mkpasswd.c (which calls get_salt() and thus uses MAX_SALT_LEN).

Ok, once I've got everything going through again, I need to git commit lib/{password.c,pending.h,lib.h} toys/*/{passwd,chsh,groupadd,groupdel,useradd,userdel,mkpasswd}.c. Right. (With a sufficiently dirty tree, tracking checkin batching and then testing it in a clean directory is one of the big scratchpad things I need to track. I'm aware tools like "git stash" exist to help this, but I've never managed to use them in a way that caused me _less_ work.)


January 7, 2022

Moxie Marlinspike's article about web1 being distributed and web2 being centralized (and web3 being a scam) is the same problem I'm dealing with putting a "video" directory under the toybox website: the bandwidth I can afford won't survive becoming popular, and the hosting service will REMOVE it rather than let it get whatever they call slashdotted these days. Napster inspired bittorrent to solve the scalability issue, but our pathologically broken intellectual property system attacked distributed solutions as "capable of being misused".

And yes the Boomers are going after cash now under the same "logic". Coins could be used to buy drugs or child porn or sex trafficing or something, therefore nobody can have coins anymore, all financial transactions must be reported to a central authority in realtime that authorizes or denies them based on whether it's a moral use of the funds, and while we're at it that central authority isn't even the state it's a monopsony of private for-profit corporations and the good-or-bad judgements are made by the collective advertising decisions of every other corporations' sales departments. Oh just FILLS me with confidence that does. This is motivating crypto-loons to do digital goldbuggery because "we senile chronically lead-poisoned Boomers can't be trusted unsupervized, therefore nobody can!" is not a sustainable position. Emergent populist solutions are often worse than the problem, but suppressing populism via police state has never ended well for any government. (The KGB had every phone tapped in Russia, the Stazi had it in germany, China's doing it now and the wheels are coming off over there. Not new, but authoritarians are always SURE that the problem is the lid of the kettle isn't screwed down tight enough. Here's a system built by people and run by people that people can't escape, this time for sure! Interesting times kinda suck.)

Capitalism's been "the state" doing the fighting back against non-monetizable individualism for decades, and it's capitalism screwing up distributed solutions to problems. Bittorrent was blamed for slowing networks when the problem was actually bufferbloat (a problem crunchyroll's servers still stuffer from enormously). I could totally post a torrent tracker on my site and it would scale, but no browser will view said video when pointed at a torrent because it might be pirated! It's not centrally controlled and audited, it could be used naughtily! You aren't to be trusted! (The raw link to the mp4 file could have exactly the same content, but that doesn't scale and thus isn't a threat to "pay me to keep breathing" business models demanding endless growth.)

The entire "right to repair" movement needs to exist because capitalism's drive for endless growth (we are insanely profitable very year and it's NOT ENOUGH, unless we make MORE MONEY THAN WE DID LAST YEAR we have FAILED) always tries to corner the market. For me to win, you must lose. That's where it comes from, and it is HIGHLY destructive.

The return to school despite omicron is because day care is necessary for parents to return to work. It serves no other purpose. People are dying, the CDC only cares about profit.

As always, the solution is A) the Boomers die, B) fix everything afterwards.


January 6, 2022

Anniversary of the insurrection and Garland has indicted nobody. They're pardoning Nixon again via statue of limitations, with the occasional half-assed finger shake. About what you'd expect from septuagenarians.

Dialed into the posix teleconference this morning, and the issue they're arguing about seems... irrelevant? It's great to specify it but from my point of view I need a test case and to see what bash does, I'm never going to usefully extract it from posix legalese. (And I'm pretty sure I already DID check this stuff when I was implementing here documents, and matched what bash did at the time.)

Yes, cut -DF is being discussed on the coreutils list, no I didn't bring it up on the call or on the posix list, because doing so can only hurt, not help. There's enough bikeshedding as is. Speaking of which, so I did NOT send the following reply email (which is somewhere between "selling past the close" and "stirring up a hornet's nest unnecessarily"). Served better by silence I think, but for the record:

On 1/6/22 8:35 AM, Pádraig Brady wrote:

> Thanks for taking the time to consolidate options/functionality
> across different implementations. This is important for users.

Thanks. The Android maintainer asked me to. :)

> Some notes below...
>
> On 05/01/2022 16:23, Rob Landley wrote:
>> Around 5 years ago toybox added the -D, -F, and -O options to cut:
...
>> This lets you do:
...
>
> Cool. I agree that the functionality is useful,
> especially in places where awk may not be available.
>
> As I see it, the main functionalities added here:
> - reordering of selected fields

Disabling the reordering to maintain literal order was suggested to posix years ago. Technically this option makes cut do _less_ work. :)

> - adjusted suppression of lines without matching fields

-D implying -s wasn't strictly necessary, but I couldn't come up with a use case for -D without -s. The awk behavior this replaces doesn't pass though lines without matches verbatim, grep -o doesn't have an option to pass through lines with no matches, and if "cut -DF" seems long "cut -DsF" is longer...

> - regex delimiter support
>
> I see regex support as less important, but still useful.

I tried to implement it without regex support first, but switching from handling "characters" to handling "strings" is inherent in the problem space, which gets us most of the way to regex already.

As soon as you switch from single character delimiter to "run" of whitespace (which could be mix of tabs and spaces, plus all toybox commands handle utf8+unicode so mine has to cope with half-width spaces and ogham non-blank whitespace and such), you're confronted with the need to set a default output delimiter because your field delimiter isn't guaranteed to be the SAME run of whitespace between entries, and there's no "before first match" or "after last match" to pass through anyway. I even tried options like "pass through first input delimiter as first output delimiter" but it REALLY didn't help: you can repeat output fields so you run out of input delimiters (what, rotate through again?), the result was just magic nonsense with no obvious relation between delimiter and output field (echo one two three | -cut 2-3 would use the delimiter between 1 and 2), and of course "echo hello | cut -F 1,1" has literally NO delimiters but should have obvious behavior.

If you hardwire in "single space" as your output, you wind up converting tabs to spaces, so it has to be an overrideable default. I basically wasn't smart enough to figure out how _not_ to do -O with a default value, and the existing --output-delimiter was a string, at which point both input AND output are "string not char".

Another need for regex (other than not wanting to special case "run of whitespace" when "regex" was right there) is that it's not parsing escapes, so if you DO want to support "abc\ def ghi" as "abc def" then "ghi", cut paying attention to that would be policy. The user deciding what is and isn't a separator via arbitrarily complex regex magic (a skill that exists out there and is Not My Problem if I just pull in something libc already has) _isn't_ policy...

>> P.P.S. -D implying -F doesn't help because -F is the one that takes arguments,
>> analogous to -f.
>
> As for the interface,

I try to get bikeshedding out of the way up front on this sort of thing, so back in 2017 I asked for feedback about the pending cut changes on the toybox list. Unfortunately -DF looks to be in one of the Dreamhost Incompetence holes:

http://lists.landley.net/pipermail/toybox-landley.net/2017-October/009196.html

But here's an adjacent cut issue from the same update batch:

http://lists.landley.net/pipermail/toybox-landley.net/2017-October/009214.html

Elliott suggested I poke other projects for compatibility reasons so what we already deployed isn't an outlier, and I started with busybox because I have a history there. That's why I asked here if coreutils wanted to be a third compatible implementation for something already used by a bunch of android package builds and shipped in a billion Android devices a year for the past 4 years. Coreutils implementing a different interface is another way of saying "no, we don't want to be compatible with this", which is of course your call.

But sure, "As for the interface..."

> it's a bit surprising that -F wasn't used to
> switch the field handling mode, rather than -D.

Lower case -f is single character fields and upper case -F is regex fields. Both -f and -F take the same comma separated numeric argument lists and do approximately the same things with them, just using different delimiters. Using -f and using -F are analogous, and -F by itself preserves the old reorder/passthrough behavior just with regex separators, because that's what -f (still) does.

-D was "Don't reorder". It's logically separate.

If you're saying that -D and -F shouldn't have been separate (using -F should have the side effect of -D), that's what I tried to implement first and I hit multiple snags, only some of which are mentioned above.

Not only is -F useful by itself but "cut -d. -Df 3,2" seems a reasonable thing to do, which with forced regex would be 'cut "-d[.]" -DF 3,2'. Or with no -D and -F being "magic run of whitespace" how would you "cut $'-d\t' -Df 3,2" to use tab separators that DON'T collapse together? With separate -D you can ask the existing -f not to reorder.

(Yes, there is still an existing -f, why _can't_ it do this too?)

> I.e. I see the mode
> change more pertaining to field handling, rather than delimiter handling.

You haven't tried to implement it or made nontrivial use of it.

> I don't have a strong opinion on this, but it may be a bit confusing to users.

I tried implementing one argument instead of 2 and it was _more_ confusing because of the multiple magic implied behavior changes. Giving the user less control isn't the unix way, and while "sane defaults" is an excuse for -F to change the -d and -O default values (both of which you can override so no control is lost), having -F imply -D if -f doesn't is inconsistent, and trying to have -D toggle the state BACK means the mnemonic "Don't" is enabling the behavior in one case but disabling it in the other case... Yes I considered that, it was icky.

I could not disturb historic behavior of existing options, so the new feature wound up a little verbose to be consistent, but less verbose than "--df 1,2,3" would have been while providing more control.

Given a choice I'd have had -s be the default behavior all along, had -D be the default behavior all along, and had -F be part of cut 40 years ago. I didn't get to make that call, I had to modify what's there in a way that made sense in context.

> BTW it's useful to note existing edge cases in delimiter handling
> when considering this new interface:
> https://www.pixelbeat.org/docs/coreutils-gotchas.html#cut

Let's see...

1) "cut doesn't work with fields separated by arbitrary whitespace. It's often better to use awk..."

Isn't that what this new feature directly addresses? The fist "edge case" is that coreutils' cut implementation doesn't have this feature yet. Um... yes?

2) "Similarly, if you want to output a blank line when there are no delimiters..."

Your other edge case is that passing through lines with no delimiters is an inconvenient behavior people would like to suppress. Cut added -s instead of adding a "turn them into blank lines" option.

> So to summarize the new interface, and how it might map to / be described in coreutils,
> I see it as:
>
> -F, --regex-fields=LIST
> Like -f, but interpret -d as a regular expression (defaulting
> to a run of whitespace)
> -O,--output-delimiter=STRING
> use STRING as the output delimiter
> default is one space with -F, or input delimiter for -f

For reference:

$ toybox --help cut
usage: cut [-Ds] [-bcCfF LIST] [-dO DELIM] [FILE...]

Print selected parts of lines from each FILE to standard output.

Each selection LIST is comma separated, either numbers (counting from 1)
or dash separated ranges (inclusive, with X- meaning to end of line and -X
from start). By default selection ranges are sorted and collated, use -D
to prevent that.

-b	Select bytes
-c	Select UTF-8 characters
-C	Select unicode columns
-d	Use DELIM (default is TAB for -f, run of whitespace for -F)
-D	Don't sort/collate selections or match -fF lines without delimiter
-f	Select fields (words) separated by single DELIM character
-F	Select fields separated by DELIM regex
-O	Output delimiter (default one space for -F, input delim for -f)
-s	Skip lines without delimiters

If you add --longopts for the other two I can add them to toybox to be consistent.

> cheers,
> Pádraig

Rob


January 5, 2022

December and January are recruiter season (everybody's annual budget renews and there's a scramble to spend it), and although I haven't been looking (working for Jeff and waiting to see if Elliott's boss can get budget to pay me to focus on toybox this year), but the game 7 recruiter came up with something interesting enough I agreed to a phone call with the engineers, and that turned into an offer at the same "pay off the mortgage fast" rate I made at JCI, and then when I said I'm probably too busy to do it they upped it $10/hour and made the initial commitment only 3-6 months. Really hard to say no, but also not what I want to do with my life?

I forwarded the offer to the Google recruiter with a request that if the 6 months of "maybe" are going to turn into 9-12 months of maybe, could he let me know so I could maybe fit this in first? Except I'm working for Jeff this month, so I'm probably saying no either way. But wow, it's tempting.

Had the call with the recruiter. He kept trying to get me excited about using "20% time" on toybox, and spending the other 80% on some other Google project. He wanted to get me excited about one of the opportunties he had. Sigh.

I tried to explain to the Google recruiter how my old distro build work led me to wanting to untangle the AOSP build, so if the job was working on the AOSP base build it would at least be heading towards the mountain, but he really didn't have the backstory to understand it. (I pointed him at my 2013 and 2019 videos again. Doubt he watched them, not really his job.)

But yes, I need to learn how to disentangle AOSP and repot it on top of mkroot and merge the NDK and AOSP toolchains and... poking around in that area would serve my purposes. Except Google isn't hiring me to serve MY purposes, and all the big corporate types only believe infrastructure is going to exist after it already does exist. (There's no such thing as a good idea, only a good time-proven historical implementation. Anything that doesn't exist yet is a fantasy, and anything that already exists was inevitable and happened without their help, QED.) And apparently every Google recruiter sees me and goes "He breaks everything and has thus spent decades learning to debug every problem thrown at him since age 12? Site Reliability Engineer!"

Poked the coreutils list about "cut -DF". Did not directly reference poking the posix list last month, but if we get the feature into coreutils, busybox, and toybox it's a de-facto standard anyway. (I could also poke the freebsd guys at that point, and... is there still any kind of open source darwin effort?)

Busybox added sort -h the day after I yanked it for being badly defined, so I've pinged their list but I'm not 100% sure what question I'm asking. If Busybox had implemented the sane behavior I'd probably use it as leverage to make puppy eyes at coreutils and see if THEY might change, but nope, busybox is blindly doing what coreutils did because it's better to be compatible than make sense.

$ echo -e 9999999K\\n1M | sort -h
9999999K
1M

I suppose I could say I'm waiting for posix to add, that's a solid excuse NOT to for a couple decades.

Hanging out in the new office space Jeff arranged. I haven't got a 24 hour access card yet, and they can't GIVE me one without a dedicated desk, their "lounge access" plan doesn't come with a card. I have to show ID at the desk during 8:30am to 5pm business hours, but then they let me stay after the staff leaves. I can't get back in after that, but there's electricity, wifi that doesn't drain my tethering quota, nearby bathroom, hot water tap I can make ramen with, no mosquitoes so far and I have yet to encounter a bug flying into my can of caffeine (easy to prevent if you drape the mask over the top when you take it off, but it's still a concern), it does NOT get down well below freezing where I'm sitting (although that largely counters the flying insect issues; the raccoons going through the trash cans are tame and fairly polite but again none inside here so far), and if the UT carillon spends 12 minutes playing "I've been working on the railroad" twice over very slowly at 9pm I don't have to listen to it. The downside is the occasional unmasked white male Bro playing ping pong, but so far only during business hours.

(Yes, I know the filk lyrics the school uses: "The eyes of texas are upon you all the live long day, the eyes of texas are upon you you cannot get away, they follow you when you are sleeping and in the bathroom too, the eyes of texas are upon you there's nothing you can do. Hook 'em." I'm aware those last two statements seem contradictory, but that's sportsball for you.)

I'm trying to set up a debootstrap to do a clean toybox build in for an introductory video, but "apt-get install git-core" is trying to install 40 new packages, including perl, libx11 and xauth (a gui), krb5-locales (the kerberos locales packages)... Define "core" please? I can install the "git" meta-package and chose NOT to.


January 4, 2022

Mozilla has fallen so far even its founders are speaking out publicly against it.

Fuzzy left a cup of water right next to my laptop on the dining room table, and Peejee predictably did her thing, and I had to pull the battery out of my laptop for 2 hours while it dried off. (Alas, prevented me from heading out to the Philbin before it closed.) Lost all my open windows, but it otherwise SEEMS ok so far? (And I have 3 other instances of this hardware I bought and can swap the disk and memory into if necessary, but it's a pain.)

Oh goddess, I yank "sort -h" from toybox for being badly defined, and the next day busybox adds it. Strongly suspect we got pinged by the same guy asking for the feature. And he matched the "12345k is less than 1M" broken behavior that I AM NOT DOING. So do I put back the one that's correct rather than conforming, or do I leave it out? Hmmm...

The Google recruiter resurfaced and said "I chatted up Lars and got an understanding of the situation. I think we have the space to make this work. Let's jump on a call this week, it's easier to explain on the phone." Which... I want him to succeed? I really don't care what budget this comes out of as long as I get to do the work? But I've been bait-and-switched a LOT over the years.

I like vaguely threatening nudist winnie-the-pooh so far.


January 3, 2022

Oh hey, Harry "dishrag" Reid (D. Spineless) died and the very next day the Senate is on a deadline to vote on removing the filibuster so they can actually start getting stuff done. Change really DOES happen over Boomers' dead bodies. While still warm.

First day back working for Jeff on J-core stuff. He's trying to get office space in the "Regis" at the Dobie, but it's not ready yet, so I'm working from home, and from HEB, and from the table outside the UT Geology building...

I've circled back around to trying to track down why the mkroot boot process on turtle isn't completing properly. It seems to be something to do with the nommu error path for redirecting a file that doesn't already exist? Unfortunately this sucker is heisenbugging all over the place (change something seemingly unrelated and the bahavior is entirely different), which makes debugging it in chunks hard. Come back and you can't even necessarily reproduce where you left off because you misremembered something insignificant and it went "boing". Probably not uninitialized memory. Could be a misaligned read somewhere. But CURRENTLY it looks like the redirect logic trying to save an "old" filehandle that wasn't open records that fact wrong so the "undo" logic is damaging the filehandle state, which presumably isn't the initial problem but makes debugging way harder.

I need stdin/stdout/stderr to WORK in order to see what I'm doing, and the problem is my patches to fix this in the kernel got rejected, so there's an early userspace dance which is failing and I can't see WHY it's failing. I thought there was some way to syslog() data into printk() so it winds up in dmesg and globally written to /dev/console if the priority is high enough, but I think it's actually something you write into under /proc or /sys which doesn't help with early logging before I've mounted anything.


January 2, 2022

Ooh, no. That's not... No. Yoink. I'm sorry, "sort -h" is just BADLY DESIGNED if that's the expected behavior.

Working on scripts for explanatory toybox videos. The more I sit down and think about it, the more material there is to cover. Pascal's Apology indeed...

I've been referring to them as "youtube videos" but I'm not entirely convinced youtube is the best place to post said videos. Youtube has had chronic issues that are somehow managing to get worse. Most videos for the past couple years have a musical outro so people can upload said outro to spotify and copyright claim their own video so that when someone ELSE inevitably spuriously claims it you at least SHARE the revenue during the dispute period rather than losing all of it. (Because they get the revenue while they've claimed it, and if you win the dispute you just start getting new ad views deposited into your account, you don't get the old money BACK, they get to keep it. And most videos see the majority of their views in the first few days after upload, so the spurious claim may take away most of the views this video ever gets.) I'm not posting these for youtube ad revenue, but even if you upload public domain footage from nasa it gets claimed. And then there's youtube running more and more ads on less and less content, and hiding non-monetized videos, and publicly stating the intention to annoy viewers into subscribing (except paying for youtube wouldn't get the content pandering to advertisers has driven off youtube back because it's not on youtube anymore)...

Can I just post videos directly to patreon? Hmmm... no, but when patreon explains how to link videos from their service, youtube is not one of the suggestions. They seem to have teamed with vimeo? Worth a try...

Eh, first I need to record it. I can post the mp4 to a temporary directory on landley.net to get feedback. Really what I want to do is have landley.net/toybox/video with an index file, but that doesn't scale to lots of simultaneous viewers. (You'd think it would by now given 9600 baud ISP dialup in 1993 turning into my 2 gigabit/second google fiber home connection in 2022, but capitalism got in the way.)


January 1, 2022

Redoing the tty plumbing, and I'm trying to figure out if tty_esc() and friends actually simplify anything? In theory you can just printf("\e[%u;%uH", y+1, x+1) but tty_jump(x, y) kind of acts as documentation, and reminds you that the argument order is swapped, and does the +1 for zero-based coordinates. Is that a win? I've been converting hexedit.c to not rely on the tty_blah() functions for color changing and stuff and just output them with printf("\e[blah") itself, but the result has conversions like:

-    tty_jump(0, TT.rows);
-    tty_esc("K");
-    printf("\e[1m%s: \e[0m%s", prompt, TT.input);
-    tty_esc("?25h");
+    printf("\e[%dH\e[K\e[1m%s: \e[0m%s\e[?25h", TT.rows+1, prompt, TT.input);

Which... works fine? But is kinda incomprehensible. But then how comprehensible was it before? Is this actually a win?

I'm pretty sure that:

-  tty_esc("?25l");
+  printf("\e[?25l");

Is a win? I tried to have a tty_color() at one point but the real question is "which color is 32, and does that set foreground or background color" and I'm NOT making a stack of enums for this.

Maybe I need to link to the "man 4 console_codes" web page at the top. Except... in every command that uses it? In lib/tty.c?

(Toybox development is always like this. It's a lot of work to come up with a simple result because it's hard to define "simple", and you only ever get a local peak anyway. I should have called it dorodango because it's endless polishing.)


Back to 2021