Rob's Blog rss feed old livejournal twitter

2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2002


July 22, 2020

There aren't ten wildcard characters there are eleven, because | in the parenthetical blocks is a special character (when not "quoted" or \escaped). In fact the @(block) only makes sense when you have | because it says match exactly one of the | separated patterns: ?() is 0 or 1, *() is 0 or more, +() is 1 or more, @() is EXACTLY one, and !() is inverted (match anything that does NOT match any of these patterns), which... I need to come up with greedy vs non-greedy tests for the inverted case don't I? To see which one it's doing?

All of which means I get to write my own glob plumbing, which I _last_ did (fresh out of college) for OS/2 Feature Install in 1996. (I remember that ***** consecutively was a pathological case with basically O(N^x) search behavior, and that I later figured out "oh, I can just drop all consecutive * after the first because it can't change the match", but probably didn't because it was unlikely to ever come up. I was young. And IBM was working me 90 hours a week and paying me for 40 because we were "exempt", which is a fancy way of saying software developers have not yet unionized. (Which is great in the "non-competes don't apply to silicon valley, you can go to work across the street the next day" sense, but sucks in most other senses. Basic income and medicare for all beat unions any day: no funky regulatory capture Jimmy Hoffa style bureaucracy becoming the enemy of its original nominal goal, but you're free to walk away from a bad job because you always have the option to NOT do it. And people clutching pearls about the shrinking labor force from THAT should stop and think about what declining birth rates and college getting priced out of anyone's reach is likely to do to skilled labor. But that's for odd days, this is an even numbered day. :)

I still need to remove the extra asterisks: they're an active wildcard, not a literal. But I don't need to do the funky speculative match and unwind thing for consecutive asterisks after the first. (How WOULD Popeye pronounce asterisk? Randy Milholland would know...)

The speculate and unwind thing is because x="banana" being globbed with p="b*a" eats the b at x[0] satisfying p[0] and advancing to p[1]='*', which means advancing to p[2]='a' which matches the a at x[1] and thus continues to p[3] = 0 looking for end of string, does NOT see end of string at x[2] and backs up to p[1], and starts looking at x[1+1] which coincidentally is where it was next but the important thing is it expanded the * range by 1 (we could have advanced further with a larger pattern partially matched, but the asterisk stack pop should resume from there)... and that "failed to match, backing up" means we have a STACK of trial locations, one for each * we've hit so far (with corresponding size of the range it's tried to match so far, starting from zero and incrementing by one each time), which is how *?* can do a LOT of checking. Each time we fail we back up one asterisk in the stack and advance it until it hits EOF, at which point we back up to the previous asterisk, and if we run out of asterisks we didn't match this string at all.

There's optimizations like if you know the length of the string and number of literals left in the patter you can tell if there's enough space left you COULD match however many literals you have, but Linux VFS limits filenames to 255 bytes long, so the common case fits in L1 cache. Yeah case statements can work on arbitrary environment variables, but just grinding the search out should be enough for almost all cases? Wait for somebody to complain...)

Hmmm, can it do a redirect in a NO_SPLIT context? Yes it can. And when it DOES try to split...

$ cat < todo*txt
bash: todo*txt: ambiguous redirect

Custom error message. Of course. Hmmm, what should I do about that...

THIS case isn't a "glue results together with space", this is "multiple results is an error". Which probably goes through the expand_one_arg() function, which catches split attempts and returns an error. Even when it passed _IN_ the NO_SPLIT flag. And I _want_ this to be an error, not an attempt to open "todo.txt todo2.txt"...

Ok, every use of NO_SPLIT is a call into expand_one_arg(), which already has to handle error because ${x?y} and such, so multiple wildcard expansion is just another such error. Having it report the error properly is an interesting question though...

Except X=todo*txt _does_ want the "glue together with space" behavior, there are cases that want one behavior and cases that want the other and my current flags aren't granular enough. Don't perform IFS splitting vs glue together legitimate multiple results returned from wildcard expansion: expand_one_arg() can't currently distinguish the two, possibly it needs to be more than one function, or some calls need to inline what it does and handle the "multiple results" case differently? Hmmm... What else CAN return multiple results? Bracket expansion, IFS splitting, wildcards. Probably something horrible with arrays but I'm still going "la la la" with my fingers in my ears about that, but "$@" definitely can and I special cased gluing that back together for NO_SPLIT (dropping it down to $*).

You know, I'm pretty sure ${x@Q} does NOT then check the result for wildcard expansion. I hope it doesn't...

$ touch "'abc'"
$ X='*'
$ echo ${X@Q}
'abc'

Of course it does. The output just LOOKS quoted, it only really IS quoted when you feed it back in again. Note to self: remember to always double quote ${x@Q} expansions...


July 21, 2020

This thread is a good self-contained explanation of why police budgets need need to be zeroed out. Don't just abolish ICE, abolish police departments. Nancy Pelosi and Chuck Schumer remain utterly useless. which should come as a surprise to nobody. (Voting for Good Cop is still voting for a cop. They can actively cause problems or claim to be powerless to stop them.)

There are no "test cases", fascists continually escalate, constantly pushing to see what they can get away with. This is WHY you punch nazis. They've deployed ICE in cities against the populace we need to find who is doing it and prosecute each and every individual. As with neuremberg, "I was just following orders" is not a defense. And most of what they're doing isn't even following orders. The current prison system serves a purely racist purpose. Bastards continuously lie. Meanwhile antifa continues to do the thing.

There are other quite interesting things I'd love to write more about, but haven't got the energy.


July 20, 2020

I've circled back to wildcard logic, and luckily I left myself a trail of breadcrumbs from last time I was poking at this, because I went "this wildcard detection/recording logic has no business being in the main loop of expand_arg_nobrace(), just have wildcard_add() scan for wildcards since a $VARIABLE can expand to active wildcard characters so you have to do it later anyway"... except the REASON I did it early is my logic removes "quoting" and \backslash escapes before calling arg_add() with the results of IFS splitting, and that indicates which wildcards are live and which are kept as literals. So I need liveness metadata, which is what that was collecting.

Modulo the processing at the start of the loop isn't enough, it needs to process both the input string AND the results of various $(expansions) to collect the full list of live wildcards. And each call to wildcard_add() wrapping the old calls to arg_add() has to flush the wildcard parsing stack which could involve incomplete [brackets] or +(regexes), ala:

$ touch a; A=abc; IFS=b; echo [$A]
[a c]
$ touch a; A=abc; IFS=; echo [$A]
a

July 19, 2020

Portland is under siege and the administration has announced plans to take it national. The inciting incident is a federal building got graffitid, and their fee-fees got hurt. The predictable result of the jackboots is that the size of the protest immediately doubled. The republicans are not a legitimate government ( link link link link link link link... Does he really need to money launder AS president?) It's violent plutocracy which has been based on lies for decades (Heck, they've even managed to violate the geneva convention.)

The boomers need to stop This whole mess is the death throes their worldview. The Boomers' Good Cop candidate is insisting that Warren will shape his domestic policy... Ok, so why did he run against her then? Is he honestly trying to hepeat Warren's entire campaign?

Meanwhile, the useless police arrest emergency workers in the midst of saving lives, another reason to zero out their budgets.

Meanwhile, China has gone full concentration camp, has rolled the long-threatened tanks into Hong Kong, and I strongly suspect they're readying to invade Taiwan. (Because right now they're not even in our top ten political crises, and know it. When will they ever have a better opportunity?)


July 18, 2020

Now that I've got the basic ${x#y} parsing infrastructure into toysh, I'm going through the tests and hitting things like "it doesn't understand the 'case' statement" which is because "abc) echo hello;;" needs wildcard logic for abc.

I have about the first half of wildcard plumbing implemented, then it gets... fiddly. Hmmm...


July 17, 2020

So, Boomerdamarung. The Year of Hindsight. The Trumpocalypse. This is about how I expected the Boomers to go: violent aggressive senility, trying hard to take the planet with them. We have to dispose of the boomers and the billionaires they worship before we can even start addressing things like climate change. Cops continue to explain why all their budgets need to be zeroed out. And the "prison" exception to the 14th amendment banning slavery continues to stretch. Meanwhile, the billionaires taking all the money has led to the entirely rational decision that nobody else can afford to have kids. Turns out your money isn't useful if there's nobody to buy anything FROM. We need to guillotine the billionaires. Change the law. Our current government is completely fictitious. Not just held in place via voter suppression and used as an excuse for massive embezzlement, but what it does do is fraud. David Graeber said there's only ever one revolutionary agenda: cancel the debt, redistribute the land.


July 16, 2020

Jeff and I gave a j-core talk to the British Computer Society today. It was followed by two other interesting FPGA talks: one on the new fully open bitstream compiler based on ghdl and yosys, and one on reverse engineering FPGA boards, because used boards with big FPGAs are much cheaper than buying new ones. In theory all three talks should be up on youtube sometime next week.

We were sent questions afterwards, some of which we responded to and some of which...

There are things Jeff does not want me to say on behalf of the project. Apparently mentioning that Japan had a lost decade gets Paul Krugman a meeting with Japan's prime minister, but if we say that SuperH lost its position as the bestselling chip in the world not because of the technology but due to the 1997 asian economic crisis (because after Hitachi handed it over to Renesas they never even did another die size shrink, but kept selling the same ~200 mhz chip with zero further manufacturing investment until chips made on a 10 year old fab process eventually stopped selling)... no, we can't say that, we'd be blacklisted. Even in a talk to british people, people would dial in and get offended and make it their life's work to stonewall us out of every possible future business opportunity ever, or something?

This is why j-core.org hasn't particularly been updated since 2017, and why I'm mostly silent on the mailing list: there's nothing I'm allowed to SAY without washing it through a committee and having it handed over to marketing people who make powerpoint. I'm used to that from big Fortune 500 companies, but from a startup? I will happily take a "that's not right, this is" correction, but "you're not wrong but we can't say it"? Working through the thicket of NDAs and the "must do a coordinated announcement with our customers" was one thing, but a marketing department and nebulous cultural politics that come and go depending upon context, I am not prepared for.

But there are also things I have trouble saying to my satisfaction, even without external judgement. I write and throw away a LOT of material trying t come up with non-confrontational ways to say things my audience doesn't want to hear, especially when baited by people who want to make their opposition look unappealing, as a professional tactic. (I think that ship has sailed within the confines of my blog, but I'm using it to vent stress since twitter's no longer available.)

Anyway, here's the email I composed but didn't send back to the BCS talk organizer. (Jeff replied independently, I'm assuming that covers it for the talk guys.)

On 7/16/20 2:12 PM, Sevan Janiyan wrote:
> Hi guys,

> Below are the questions/comments for your session, could you make your
> slides available or share them with me to host them if that's ok?
>
>
> Will you be taking advantage of the FOSSi/Google announcement to
> manufcture asics for open source designs over the next year or so? (
> https://fossi-foundation.org/2020/06/30/skywater-pdk ) - Andy Bennett

We've been talking to them for a couple weeks now. We hang out on the #j-core and #vhdl channels on their slack.

> would the xilinix parts be able to use symbiflow? - Drew Fustini

That's a Jeff question.

> for Linux on nommu, is there a way to support shared libraries in
> userspace? - Drew Fustini
> I believe ARM has FDPIC ABI for that? - Drew Fustini

ARM is a latecomer to FDPIC, superh support for fdpic started in 2008 and the first architecture that did it was frv I think?

There are two nommu executable variants: binflt is nommu a.out and fdpic is nommu elf. It's not that binflt doesn't have shared library support, it's that shared libraries in a.out were always horrific (it's not relocatable so you have to reserve a unique address range for each library your system will ever use, at build time). Linux switching from a.out to ELF circa 1995 was primarily driven by better shared library support.

So doing shared libraries in a.out is possible but most people don't bother, and doing shared libraries in fdpic is roughly equivalent to doing it in non-fdpic elf. (In theory it should just work.)

P.S. In THEORY you can make fdpic work on systems with mmu just as easily as nommu systems, but in practice the linux fdpic loader plumbing has gratutitous #ifdef MMU checks in it to break it on mmu systems, which is sad. As I said, some security guys were looking into using it on mmu a few years back, but apparently those patches never made it upstream.

> I enjoyed the presentation at ELC in San Diego a few years ago, is the
> turtle board still planned? - Drew Fustini

Yes, we did an updated version of it late last year (among other things using LX45 instead of LX25) which we're trying to get up on crowd supply now. We just didn't want to talk much about something that's not available yet.

> Do you expect people will choose J-Core over RISC-V because it's more
> tightly integrated? - Andy Bennet.

I don't, no. Jeff might?

Personally I have as much interest in what Risc-V is doing as in what Windows is doing. I find them profoundly technically uninteresting, and have no desire to work on that platform.

I'm seldom interested in any technology whose adherents feel threatened by the existence of competitors. I'm not trying to convert people to a religion, I'm doing engineering work that solves real problems in a way I hope other people find useful. Why did the developers of Linux expect people to choose it over BSD? Why did the gnome developers expect people to choose it over KDE? More people had never used EITHER. When you have 1% and they have 2% you focus on the 97% neither of you have rather than trying to block "the competition", unless you're pathologically insecure.

When the GNU project started, BSD already existed. When busybox started, gnu already existed. I maintain toybox, which I started after leaving busybox. Android ships toybox in its base OS image, which means there were something like 1.3 billion fresh deployments of toybox last year. And yet busybox continues to exist, and the dozens of gnu packages I made busybox functionally replace in a self-hosting build system back before handing it off also continue to exist.

Risc-V spends a lot of time marketing itself as "inevitable" the same way itanium, hillary clinton, and linux on the desktop did. That's not a statement of merit, that says you have no choice and will be forced to use it. If your best argument is inevitability, you've got a REALLY weak hand. (Look how many copies Windows shipped, how many users AOL had, clearly that's superior technology.)

I tend to avoid any technology whose adherents demand everyone else justify NOT using their thing. The way Python 3 was handled did not encourage me to leave Python 2. Yes when C++ guys boggle at my continued use of C, I _can_ go into a long technical explanation of why, but is that what they're really asking?

Then again I've had the "how dare you write new open source code competing with other open source code" argument aimed slightly more directly at me than most (https://lwn.net/Articles/478308/) and might be biased.

> Which requirements do you expect to split the market between J-Core and
> RISC-V? - Andy Bennett

I don't? Again, Jeff might. But me, I find the question baffling.

The linux arch/ directory has alpha, arc, arm, armv64, c6xx, csky, h8300, hexagon, ia64, m68k, microblaze, mips, nds32, nios2, openrisc, parisc, powerpc, riscv, s390, sh, sparc, unicore32, x86, and xtensa. (That's after removing blackfin, cris, frv, m32r, metag, mn10300, score, and tile as unmaintained.)

The first commit to the j-core internal repository was in 2011. The first open source release of j-core VHDL was in 2015. The linux arch/sh directory was added to 2.3.16 in 1999, and git log arch/sh/configs/j2_defconfig says it was added in 2016. All of which was before arch/riscv was added in 2017. Yes they spend way more on marketing than we do, but again so does windows.

I wonder why you're ignoring the continuing existence of arm when discussing "the market". Because obviously nobody's going to resume work on https://en.wikipedia.org/wiki/Amber_(processor_core) now that more patents have expired? Or that if a serious open competitor did emerge as a market threat to arm's dominance (which hasn't happened yet), Arm wouldn't do an open version? Heck, I wouldn't put it past Google to buy them and do that given https://asia.nikkei.com/Business/SoftBank2/SoftBank-pushes-Arm-to-focus-on-chips-ahead-of-possible-IPO (and Apple, which is switching its macs to arm, wouldn't exactly object; they love being open-adjacent. Clang/LLVM was an apple project (hiring the grad student and giving him a dev team), they recruited the freebsd developers to create darwin... Beyond that powerpc is open hardware these days, openrisc always was...

The Risc-V guys are motivated to put forth a marketing story about how inevitable they are and everybody else should just give up now, but Intel couldn't pull off making Itanium inevitable. "It's only us, you can't NOT talk about us, we are always so important that any conversation in this space can only happen relative to us"... meh.

I mentioned some of the things we found attractive about the superh technology in the talk. I'm not intersted in "but red hat enterprise has so much more money employing full-time engineers, and they paid the linux foundation to insert RPM into the LSB as the official package management format of Linux, so why does debian even still exist, let alone anything _else_". I sat through the ELC year when everything was Meego, the year when everything was Tizen... are they still saying everything is Yocto or is there a new one yet?

P.S. Jeff doesn't like me to talk about this, because if you respond to riscv with indifference that directly contradicts its developers' carefully constructed marketing aura of inevitability, some of them perceive it as an attack. It's not. Good luck to 'em. And to nds32 and csky which have been added to the linux arch/ directory since riscv was. I remember when java wanted to become "the software platform" the same way riscv wants to become "the processor". And now it's this javascript assembly "webapps" thing that wants to be "the platform". And lo, we shall succeed so mightily as to become the monoculture forevermore, alelujiah, amen. Where have I heard that before...

Remember when I mentioned the fdpic-on-mmu work doesn't seem to have made it upstream into vanilla linux? Personally I suspect it's because linux-kernel development has aged into a tight exclusive clique:

https://www.zdnet.com/article/graying-linux-developers-look-for-new-blood/
https://thenewstack.io/growing-new-linux-kernel-developers/
https://twitter.com/stillinbeta/status/1278185092792438791

Which of course didn't stop the fuchsia guys from getting a lot of blowback for exploring alternatives to it, of course. How dare they write new code when there was a "winner"...

I then made a second attempt to answer that last question more tactfully::

> Which requirements do you expect to split the market between J-Core and
> RISC-V? - Andy Bennett

Why would the market narrow to that?

The linux arch/ directory has alpha, arc, arm, armv64, c6xx, csky, h8300, hexagon, ia64, m68k, microblaze, mips, nds32, nios2, openrisc, parisc, powerpc, riscv, s390, sh, sparc, unicore32, x86, and xtensa. That's after removing blackfin, cris, frv, m32r, metag, mn10300, score, and tile as unmaintained, for values of "unmaintained" that probably have more to do with:

https://www.zdnet.com/article/graying-linux-developers-look-for-new-blood/
https://thenewstack.io/growing-new-linux-kernel-developers/
https://twitter.com/stillinbeta/status/1278185092792438791

than it does with blackfin's market share. (Or alpha, ia64, and parisc staying in.)

Sure j-core is open (and arch/sh/configs/j2_defconfig was added to Linux a year before the arch/riscv directory), but powerpc is also open and openrisc always was. At the height of "all the world's a vax" it wasn't true, at the height of "all the world's a 386" it wasn't either, and we're just now entering "all the world's an arm" where it won't be.

Speaking of ARM, why are you sure nobody's going to resume work on https://en.wikipedia.org/wiki/Amber_(processor_core) now that more patents have expired? Or that if a serious open competitor did emerge as a market threat to arm's dominance (which from their perspective hasn't remotely happened yet) that Arm itself would never consider pulling a netscape with an open base they could sell proprietary extensions to? Heck, I wouldn't put it past Google to buy them and open it given https://asia.nikkei.com/Business/SoftBank2/SoftBank-pushes-Arm-to-focus-on-chips-ahead-of-possible-IPO (Remember when they bought Motorola's phone business?) Meanwhile Apple, which is switching its macs to arm right now, wouldn't exactly object: they love being open-adjacent ever since they hired FreeBSD devs to create Darwin. Clang/LLVM was an apple project (hiring the grad student and giving him a dev team because they objected to gcc going GPLv3). They _want_ to be the 20% proprietary cap on an 80% open market.

I find "inevitability" arguments tiring. I remember when the Java developers expected all software to be Java. I remember when itanium was The Future. I sat through something like 17 consecutive "year of the linux desktop". Proclaiming "inevitability" as a marketing strategy is as old as Sun-Tzu: convince your opponent that everything _not_ you will be swept away and everyone must jump on board this self-fulfilling prophecy now or be left behind. But personally I've always seen it as a sign of weakness: if that's your big pitch it means you're not leading with actual technical arguments in favor of their thing. Making a populist/intimidation argument that this technology is better funded and may have tied up distribution channels the way Windows was/did does not endear it to me either: I wasn't a Windows developer. I've never had a facebook account. I never had an AOL account. I remember the ELC year when everything was Meego, and the year when everything was Tizen. I believe everything is still currently Yocto unless somebody wrote the Linux Foundation a bigger check.

This isn't meant as an attack on Risc-V, good luck to 'em. And to nds32 and csky which have been added to the linux arch/ directory since riscv was. Maybe the people saying javascript webapps will render "the processor" irrelevant are right, and that will become the one and only type of software everywhere (somehow abstracting away the os kernel, which will be written in webassembly too... hey, Transmeta almost made that sort of thing work once upon a time)... but I doubt it. And I'm not really interested. It's not a new argument to go "And lo, we shall succeed so mightily as to become the new monoculture, perpetual and unassailable, alelujiah, amen, the future is already written", I've heard it before and it's BORING.

I expect ARM to be the dominant "big" processor over the next decade because the sheer momentum of a billion units per year would take a while to wind down, and because there's not a lot of headroom to get under a $5 raspberry pi zero with something cheaper. (You can get a cortex-m0 for 30 cents on alibaba in quantity 10; you can argue of "thumb2" and "aarch64" are the same architecture, but it's the same argument about whether the riscv "compressed" instruction set is the same architecture as the 32 bit riscv instruction set, isn't it? J-core is always 16 bits...)

Beyond the next decade, I really can't predict the future that far ahead? I expect to be surprised a lot.

I can tell you why the rise of x86 and arm happened: The switch to x86 was partly about design wins giving them distribution channels/volume, but the big driver was price/performance ratio (best bang for the buck). The switch to arm was initially driven by power consumption to performance ratio (best bang for the watt). Then arm sacrificed power efficiency to move into x86 performance turf, and x86 realized that power efficiency was a thing and did atom, and the two became less distinguishable and it became an ecosystem thing... I've given multiple talks about this already, http://www.youtube.com/watch?v=SGmtP5Lg_t0 for example.

I'm not sure "lack of per-chip license fees" is a strong argument for "displacing standardized high-volume chip runs" when a 45 nanometer mask costs a million dollars to make, and that is NOT a cutting edge fab. Cutting edge is 5 nanometer:

https://www.bloomberg.com/news/articles/2020-05-21/samsung-takes-another-step-in-116-billion-plan-to-take-on-tsmc

Cost of chip manufacturing has always been primarily a question of unit volume, you amortize the start-up costs over the largest possible production run because the fab is paying off BILLIONS in construction costs. At 5 nanometer, HUNDREDS of billions, with interest. "Everybody can make small runs of their own chip" helps the economics of this how exactly? With OLD fabs that have already paid everything off and are struggling to keep their doors open, sure. But by the time Moore's Law is stone dead enough a 5 nanometer fab is offering cheap academic shuttles, we are SO far into the future for all I know quantum computing and warp drives might have become real by then.

So if Risc-V wants to convince Apple to switch processors _again_, and to migrate Android and ChromeOS, and to take over The Cloud, and to come up with a raspberry pi replacement, and switch the business world off x86 windows machines, and replace arduino, and it's already a given that all this will definitely happen on a predictable schedule... I leave them to it? Clearly they don't need me if it's a done deal.

I think j-core is worth doing. I talked about some technical reasons I like j-core in the talk and could go into more. I could tell you about the relative merits of j2/j1/j0 vs cortex-m or atmel avr (the arduino CPU), new features we want to add, projects we're building around it, the work to extend the J2 SOC implementing patent-free HDMI support in the Turtle bitstream and switching from mmc to sd-1.0 as the patents expired (12.5 megabytes per second is SO much nicer than ~200 kilobytes per second to your main storage device, native compiling in 200k I/O bandwidth was painful even as a proof of concept)... and at the other end teaching j1 to do DMA and hardware threading while still fitting in an ice-40...

I do not find the Risc-V "there can be only one" thesis compelling. When the FSF stated writing the GNU command line tools, BSD already existed. When I maintained busybox, the gnu tools already existed. When I left busybox to create toybox I handed busybox off to the best maintainer I could find and continued to submit fixes and new commands to it for another 5 years (including a major redesign allowing busybox commands to live in a single file instead of spread out across 5 files, als http://lists.busybox.net/pipermail/busybox/2010-March/071855.html). These days toybox is part of the Android base image, meaning 1.3 billion new installations of it shipped last year. And yet busybox still exists, and is successful. The gnu tools still exist, and are successful. BSD still exists, and is the basis of MacOS and iOS.

J-core does not exist in a vacuum: our context includes cortex-m, and avr, and blackfin, and so on. We know what we want to do, and we know what THEY can do, and in that context we can confidently say we're good at our job. Risc-V wants to do absolutely everything, which means it has no idea what it wants to do. It's a solution in search of a problem, and every time anyone else does anything the question "why didn't you use Risc-V" comes up, and I'm very tired of it.

Every time Jeff has talked to the Risc-V developers and pointed out something J-core chose to do differently, the Risc-V developers add an extension to their architecture, making the chip bigger. Then they make the extension optional, so they can SAY it isn't bigger, instead the platform is fragmented and incompatible. But if you ignore that it lets them say their chip is small (because there's a version that is) and that their chip can win any microbenchmark you care to name (because there's a version that can). I am not interested in arguing with that.

You seem to be asking why we're ignoring Risc-V's marketing of itself as "inevitable", in the same way Itanium was (https://www.zdnet.com/article/dell-finally-signs-up-to-itanium-2/), the same way Java Applications and Linux on the Desktop were inevitable, the same way GPLv3 was inevitable, the same way https://www.wired.com/2009/12/st-essay-china/ was "going to have a profound impact on computers everywhere"...

You're asking the question of someone who doesn't have a facebook account, never had an AOL account, and was never a windows programmer. I don't find "inevitable" to be a very interesting argument, one way or the other.

As you can tell, my second attempt... didn't really help.

Anyway, there were a few more questions:

> Has j-core been implemented in any other soc toolsets, eg Migen
> (Symbiflow) or SpinalHDL? - David Price

Jeff question.

> getting people started, have you looked at apio? - David Price

Jeff question.

> I'd love to see debian packages for the tools. Their AVR (and in the
> past, msp430) toolchains have been excellent and easy to install - Andy Bennett

I'd love to see more stuff packaged up. Which tools are you referring to?

I'm building the fdpic toolchain (cross and native) with musl-cross-make, and this week we've been discussing updates to the ghdlsynth build script I posted to the j-core list back in november on the #vhdl channel of mithro's sky130 slack. (If I check the build script into github, it might be possible to have a github trigger actually compile it and host binaries?)

You can run our fdpic binaries using qemu-sh4 but we need to do some work on qemu to add a turtle board emulator and proper j2 -cpu type. (It's sh2 with the sh3 bit shift instructions backported, plus cmpxchg.)

What I want to do is a proper getting started walkthrough as a youtube video series, but that implies people have boards. A couple years ago we tried to get people started using a cheap $50 board from India (the Numato Mimas v2) but other than bootling linux to a shell prompt there wasn't anything you could DO on that board. Turtle has a bunch of GPIO and I/O devices you can play with, and plenty of free space in the FPGA to wire up more hardware. The Mimas... didn't, and turned into quite the dead end for wannabe fpga enthusiasts.

So our next big todo item is getting the turtle boards up on crowd supply, and then maybe we'd make a docker image with all the tools preinstalled to get people started easily?

> Great stuff! Got me inspired, thank you! - Yuri Cauwerts

Glad you liked it. :)

> When will the MMU systems be available for mere mortals? - Valery Ushakov

Jeff question.

I still have the window open. I should make a third attempt at answering the risc-v questions politely and non-confrontationally. (I'm not a Risc-V expert. I wasn't a Windows expert either. People insisting I have to be a Windows expert in order to justify NOT using Windows pretty much guarantee I'm never going to care about their thing, but I don't want to spend all my time talking about THEIR THING instead of about my thing which I do have some knowledge of. Many people I want nothing to do with have a black belt in making everything always be all about them.)


July 14, 2020

Reading Google's approved license list and it's kind of fascinating. There's no actual link to 0BSD (which could use the spdx page which links to my own page on it so two links for one there, although the one paragraph description on Wikipedia[citation needed] is pretty concise), but the more interesting part is right after the license list they have one of those "Public Domain: threat or meanace" sections lawyers tend to do that doesn't QUITE explain what the problem is.

The problem is capitalism wants things to be owned. Even "airspace" has to belong to somebody. You can't just fly over land. Yes birds have been doing it for millions of years but YOU can't, that's somebody's air and their inability to fly doesn't mean they don't own it.

Releasing copyrighted material into the public domain is like releasing land into the public domain. There are public parks, but you can't just buy property and declare it public land: there's insane legal hoops you have to go through to NOT own it anymore, and then they'll say no because who pays the property tax? (It's a field with trees in it. Yes, but those trees get charged a fixed amount every year and must pay because CAPITALISM.)

Before the Baby Boom it was easy to abandon land, and it used to be easy to abandon copyrights into the public domain, but as Boomer Senility has boiled off all nuance, distilling extreme fundamentalist versions of old ideas, Late Stage Capitalism has reached the idea of anything anywhere NOT BEING OWNED as just... inconceivable. Radio frequences are not just owned but auctioned off for billions of dollars. (Isn't that basically a color of light? Yes it is! Don't the sun and stars and thermal noise shine in basically all frequencies? Absolutely! Now pay up.) We'll be assigning property rights in low earth orbit soon (that's MY orbit your satellite's in, pay me rent)...

So these days, if you want public domain equivalent code, you can't actually give up ownership because Boomer Senility won't let you. You have to post a "trespassers welcome" sign saying that everyone, everywhere, is welcome onto YOUR land that you definitely own because you can't NOT own it, and when you die it'll be inherited by some distant relative you've never even met who will try their hardest to squeeze every dime out of it so the permission statement had better be unassailably legally valid and nailed down hard. And half your sign needs to be "no lifeguard on duty, swim at your own risk" disclaimer crap or the lawyers insist you're Doing It Wrong.

When the last Boomer dies the rest of us can admit that capitalism goes on the Dead Philosophy pile with haruspicy, monarchy, the spanish inquisition, and pyramid building. It's a thing we did for a while, then stopped because it made no sense. And intellectual property is a facet of capitalism: even ideas can't NOT be owned. (Most artists want attribution and sponsorship, ownership isn't really either. You only really worry about piracy once you've conquered obscurity, until then it doesn't matter as long as they got your name right.) This is how society always changes, the kids believe something else and the geezers die still screaming defiance that Vinyl was Better than CDs while the rest of us aren't sure when we last used a physical CD drive.


July 13, 2020

All that oil china bought back when prices went to zero is coming into port and they can't store it fast enough. This is unlikely to increase future demand any time soon. 31 US oil companies have gone bankrupt so far this year.

The USA does not have any "self made men" anymore. William H Gates III ("Trey" to his friends) got the IBM PC contract for MS-DOS (despite his company being too small to qualify as a vendor according to IBM's rules) because his mother Mary was on the board of directors of the Red Cross with IBM's CEO and he made an exception for "Mary's boy". Jeff Bezos' parents gave him a quarter million dollars to bail out Amazon in 1995. Tax the plutocracy white and guillotine the billionaires.

Police continue to suck, in part because police unions are guilds, not unions. They're also charging protestors with gratuitous felonies and we pretty much burn down the whole system at that point. We're all just waiting for the Boomers to die. The fixes are easy they just involve guillotining the billionaires and redirecting the amount the government spends each year on maintaining inequality (from fossil fuel subsidies and racist farm subsidies to police protecting plutocrats' assets) to basic income instead. The Boomer media got tired of the protests, but the protests are still happening. The Boomers refuse to stop driving, but every crash is an excuse not to fix their car.


July 12, 2020

Got the next lump of toysh code checked in, after almost exactly a one month gap since the last commit to sh.c. This stuff is hard.

Now I'm debugging it and adding more tests, because "implemented a design that made sense in my head" and "it compiled and didn't introduce obvious nothing-works-anymore regressions" is not remotely the same as "done". In fact the new design implements ${x::} but not ${x//} regex stuff, or the case folding, or...

And one of the tests segfaults, with glibc's useless "here's a stack dump of an executable with no symbols, to scroll off the actually useful information, in a way you can't easily disable". Thanks, glibc: you remain the opposite of helpful. And not in a "you're a cat, you're allowed" way but in a "dishwashing liquid in the dishwasher, suds all over the kitchen" sort of way.

As much as I'm frustrated by musl, at least it's not a self-defeating self-aggrandizing front for fundamentalist prostletyzing that profoundly sucks at their nominal purpose while pulling effort AWAY from all the organizations trying to do it right. The FSF and PETA are the same class of organization: Does Not Do What It Says On The Tin, Instead Actively Harms its Stated Cause. In brief: avoid.

I may be a little stressed right now. I _know_ I'm irritable. Trying not to show it in my blog on even numbered days, though.

Heh, more evidence that the Graphics Interchange Format continues not to be the Giraffe Interchange Format.


July 11, 2020

Seattle Police Department vows that if their budget is cut they'll fire their black officers first. It may be hyperbole to say the cops are such assholes that nature itself is rising against them, but it may also be true.

The protests and Seattle's autonomous zone are having an impact, but it all needs to go. The whole system is rotten, guillotine the billionaires. No, seriously. Make that the law: if you hoard a billion dollars for a full year, it's a capital offense. That's plenty of time to give it away. Don't wimp out at the last minute, have the courage of your convictions. The end of the Boomers is the end of capitalism. Similarly, abolish the police is meant literally. (The Daily Show had a lovely interview on that which is where I learned that "reform" always increases police budgets, which is why deflection trying to switch from "defund" to "reform" never works.) Other movements like prison abolition also mean it literally.

But how could society possibly work without kings, without priests, without slaves, without ritual human sacrifice? (You can't just NOT cut people's hearts out and throw them into a bog, the sun would stop rising!) In reality a society cleansed of Boomer ideas could easily work better than it does now, and we know this because it USED to. The top tax rate was 90% until LBJ lowered it and still 70% in 1980. Before the Boomers corporations WEREN'T people, and before 1970 corporate executives had a responsibility to the employees and customers, NOT just "shareholder value" to the exclusion of all else.

The USA didn't even have an immigration policy until 1889, before that people just showed up. (Sure we persecuted some classes of people, african, native american, chinese, war with mexico... but that was pure racism refusing to acknowledge their humanity under any circumstances, not quotas of people allowed to come in and assimilate as citizens to avoid diluting the purity of WASP blood, that was a new idea the KKK had when they rejoined after the civil war and got back into congress to pass new laws. And yes, this was 50 years AFTER the irish potato famine with millions of people suddenly "showing up" in the USA, and eventually assimilating just fine. As with most suburbs being named after what was paved over to install them ("oak grove", "deerfield", etc) the Statue of Liberty's 1886 dedication marked the end of the poem on its plaque being true. "Yeah, we used to be like that. It made the country huge and powerful. Then parasite plutocrats took over during the Guilded Age, found racism an EXCELLENT tool the 1% could use to divide and conquer the 99%, and we stopped. Here's a gravestone for those ideals, a great big statue for the tomb."

The USA was founded by breaking away from from plutocrats (dumped their tea in the harbor), fought a civil war against plutocrats who insisted upon literally owning people, and stamped down the guilded age plutocrats (whose excessive parasitism caused the Great Depression) with FDR's new Deal and World War II solidarity. Then Reagan brought it all back (tearing out the taxes and regulation we'd put in place to prevent inherited generational wealth from accumulating) and we've got to guillotine the billionaires to get it back under control.

So when people say "Abolish ICE", they mean it. 19 years ago in response to the World Trade Center bombing the George W. Bush administration smashed border patrol and the customs bureau together into a single organization tasked with both letting people in and keeping them out. It was INSANELY STUPID at the time, but so was everything those "duct tape and plastic sheeting" clowns ever did. They were there to embezzle billions via haliburton and blackwater and such, actually running the government was only interesting to them when they could steal stuff or cripple things like the IRS and EPA that cost billionaires money.

ICE literally creates the problem it claims to address. We had an undefended border with canada through Y2K, the only reason the border with mexico was different was racism (and the legacy of our own banana republic meddling with Mexico's neighbors spilling back our way). But even then, the stupid "build a wall" rhetoric is because there still wasn't one in 2016, because we'd never NEEDED one. Before ICE, people walking in from mexico was normal: they pick our crops. Without them, US agriculture doesn't FUNCTION.

The FBI handling domestic investigations and the CIA handling foreign ones were explicitly separated for the same reason: smashing them together into Homeland Security meant domestic surveilance did to US citizens what the CIA was doing to Fidel Castro, and when Snowden told everybody how bad it had gotten 10 years later we all went "gee, what a surprise, TEAR IT ALL DOWN NOW" but the Boomers can't see anything wrong in endless escalation because they're too old to EVER FEEL SAFE ABOUT ANYTHING EVER. Death is coming for them, and they want BARBED WIRE AND GUARD TOWERS WITH SNIPERS to stop old age from rendering them irrelevant. And those kids these days with their rhytmic music, that's bad too. Everybody needs to get off their lawn so they can yell at cloud computing in peace. They are quaking in their boots in fear, but it's because they're senile old fogies about to die, not because the world is different.

We got into this mess because Boomers like Bill Clinton replaced all the other social services we used to have with police. (The same way the federal defense budget ballooned to be bigger than the next 8 countries combined: it's NEVER enough for Boomers to feel safe, nothing ever could be.) John Oliver's excellent coverage of this gives a shout-out to Bill Clinton's massive expansion of the police in the 90's, and works his way up to outright mockery of 77 year old Joe Biden; the Democrats are Good Cop in the plutocrats' two party Good Cop/Bad Cop system. Nancy Pelosi is 80. Chuck Schumer is 69. Mandatory retirment at 65 was commonplace until Ronald Reagan outlawed it in 1986. You can't vote younger than 18 or drink younger than 21, and can't be president younger than 35 (which is why AOC couldn't run this time), but we have an 80 year old speaker of the house and Biden would turn 80 in his first term and that's just fine, and the Supreme Court hinges on Ruth Bader Ginsberg (who turned 80 during Obama's FIRST TERM) not dying. This is 15 years after what used to be mandatory retirement.

Getting back to specifically abolishing the police (and private for-profit prisons), replacing the carceral state with JUST basic income and medicare for all (including mental health and drug treatment) would remove most need for subsistence crime. Educationally addressing racism and misogyny would remove the two largest sources of violence. Also, the FBI says they CATCH $300 billion of white collar crime annually, and the real amounts are generally estimated much higher. When you add in stuff like wage theft that aren't even prosecuted as crimes, the annual $50 billion cost of shoplifting is a rounding error. The big thefts have always been by those in power (and only called thefts when they lose power). Mall cops do not keep anyone safe, they're there to make the property owners FEEL safe.

The Boomers are too old to learn or change, but society constantly does. The society the Boomers lived in is broken and rotting. Tear it all down when they're gone. The Boomers' unquestioned assumptions deserve rejection, the world never had to work that way, didn't used to, and all their crap can just stop when they do.


July 10, 2020

Still wrestling with toysh, almost got the new ${variable/slice/code} working. It's compiling, now I'm trying to fix up enough regressions that checking it in doesn't break basic variable resolution.

There's chunks of job control in this commit because that's what I was working on when I forked off on this tangent, but that's ok. If you type jobs it displays state that never gets populated, wheee. It's still progress, working on it...


July 9, 2020

Police are still lying racists. Military morale is understandably low right now, but so far most of them do NOT seem to be following the white house's orders to commit war crimes.

The supreme court's tax return ruling was carefully crafted to release zero information while SEEMING like it did, to avoid triggering more protests. Meanwhile protests continue, as does the obvious racism and hypocrisy and blame shifting.

Another article about how means testing is just a way of preventing programs from doing their job. If you can't stop a program from being created, adding means testing lets you have it without having it, because nobody ever qualifies.


July 8, 2020

The Alpine devs are taking a look at toybox, which is nice in theory. I got a set of bugfixes to stuff in pending, and explained what pending is for. (There's a README. Possibly the README needs to be more expicit? Not sure how to make it more prominent. The build warns in red about using stuff out of pending...)

One of the proposed fixes is to add an extra header #include that's... already in the next header it #includes, and has been since 2016? The patch "fixes" a bug I not only can't reproduce, but can't figure out how anyone else could experience on the kernel versions they claim to be using?

Alas, it's hard to link to the web archive because whatever strange form of HTML email this dev is using gets completely scrubbed by the archive software, and when I pointed that out they responded by... emailing me privately and not cc-ing the list anymore.


July 6, 2020

What does bash WANT here? Grrr...

$ cat << ABC  DEF
> thing
> ABC DEF
> ABC  DEF
> ABC\ \ DEF
> ^C
$ cat << ABC DEF
> ABC DEF
> "ABC DEF"
> ABC\ DEF
> ^C

Ah, eventually figured it out: if you don't quote the EOF then word splitting happens and it's basically parsing as cat DEF <*lt; << ABC because the redirect operator eats the next argument but that's JUST the ABC. (Whitespace expansion doesn't trigger $IFS word splitting in this case and I tried to test it from the command line, and got confused.

At least I didn't bother Chet with this one. (I've basically reached the end of "ask questions about how this works" and am trying to get code running which I can then debug and correct. Almost there.)


July 5, 2020

In honor of July 4, Oregon police were giving nazi salutes. Meanwhile the GOP is finding still more ways to do voter suppression.

Meanwhile, the democrats are Good Cop. Remember, Good Cop and Bad Cop are just two cops who flipped a coin at the start of the shift to see which role each would play today: the carrot or the stick. You never GET the carrot, it dangles out of your reach endlessly. The stick makes regular contact. This is why the Democrats are incompetent and the Republicans are evil, both are sponsored by plutocrats. Hillary's speeches to Goldman Sachs were not out of character.

Right now it is important to break the stick. NEXT it's important to break the carrot. But most important is to guillotine the billionaires, who are the source of both carrot and stick and can replace them easily if left in power. Billionaires hoarding money is what prevents universal basic income. A parasite class that has collected more than half of all wealth into the hands of less than 1% of the population is why the rest of us DON'T have twice as much right now, or else work half as much for what we have now. They are evil, and they try to explain away simple math with lies. Kings and clergy claimed to be necessary too, turns out they were also lying.


July 4, 2020

Mithro pointed me at the sky130 open fab project which looks interesting on the j-core side of things. I've passed it on...

Oh goddess I broke down and logged into linkedin to respond to a connection request from someone I actually knew, and it's gone duolingo levels of clingy in email. (A while back I tried duolingo's japanese thing because Fuzzy was learning french with it, but A) I didn't really like it, B) CREEPY OWL IS CLINGLY AND ENDLESSLY PESTERING, STOPPIT. After multiple months the pestering tailed off, and I opened the app again and did ONE round of things and it's been emailing me daily updates on my "progress" ever since which I CAN'T SHUT OFF. Yeah I can mail filter 'em to the trash folder but it's the principle of the thing.)

Sigh. Linkedin was abusive BEFORE microsoft bought them. I only ever made an account to see if there was a way to get it to STOP EMAILING ME. (There was not.) Remind me to never log in there again, it wakes the beast...


July 3, 2020

The Boomers have gone to full pearl-clutching prudism attempting to outlaw sex entirely. They've been idolizing altzheimer's for years (since Ronald Reagan dodged Iran Contra because he "couldn't remember" anything about it), because dementia makes white men extra-confident when they can't remember ever having been wrong about anything.

After we defund the police (and replace them with actual mental health and homeless services and so on) we need to impeach a whole bunch of judges too, and bits of the military and there are other fixes necessary on our way to sweep away the remains of the GOP. There is so much we can do if we just stop being racist. As the old saying goes, "this is why we can't have nice things". As in the literal reason we can't have an NHS and significant vacation time like canada and australia and every european country is because racism: if we have nice things then "those people" would get nice things, and we'd rather be homeless than let brown people have nice houses too.

I want basic income. I'm aware that means the entire viewership of Faux News would also get basic income. The idea that 4chan trolls and the incels swatting every gamer girl who dares show her face on twitch having basic income and being able to spend MORE time doing horrible things is distressing to me. But helping everybody means helping people you hate. Deal with it. Hurting yourself to hurt "them" is stupid. Setting your house on fire so it spreads to your neighbors is childish. Means-testing is self defeating, any attempt at filtering so only the "right" people get benefits destroys the benefits program, every time.

The argument isn't really about whether we CAN do these things that every other country has already done, or the new things (like universal basic income) the advance of technology makes possible. Of course this country can literally feed itself a dozen times over, 90% of modern "farm" culture is about getting tax breaks and subsidies on investment property until the condo developers are ready to pave it over (which is why they vote republican: it's rich landowners pulling a scam, the reason they use undocumented immigrants as labor is so they can defraud them too. I'm reading a thread about sweet potato laundering. It's like money laundering, except for sweet potatoes. Really! If any group consistently votes republican, their entire business model is a scam, you just haven't figured out HOW yet).

We deny ourselves things like a USA NHS because racists can't stand the idea of black people benefitting. UBI is easy for real countries to do and it's a necessity for handling the end of capitalism, which would also let us end intellectual property law. Capitalism is past its sell-by date, and it's killing people. It's hard to understand the scope of the problem, but the coming rent default crisis may get some attention.

Meanwhile, the main reason "All Cops Are Bastards" is those who aren't get fired. The "behind the bastards" podcast did a 6 part series on the history of policing in the USA.

I am shocked, shocked to find out that the founder of Cards Against Humanity is having a me too moment.


July 2, 2020

Wow, this tweet sums up my attitude to the linux kernel development community amazingly well. ( My community is terrible.)

I REALLY want a better way to follow Dr. Taber's writing than twitter threads, because she writes amazing stuff which is easy to miss and hard to catch up on if you turn your back for a few days. By the time you're trying to trace tangents back ot the source you start wondering "if I saved a link to the start of the thread, how would I ever find this branch again?"

I occasionally get cc'd on projects switching license to 0BSD, which is cool. I should do a proper "Why 0BSD" talk and/or writeup. I've done lots of pieces of it in various places, but haven't done a single coherent writeup I'm aware of. (Alas, it's kind of a big thing to do it right.)

I suppose it was only a matter of time. Human gene editing on living adults was part of the star trek future. I believe it was also part of the cataclysmic backstory (the collapse of capitalism involved war and a bit of a dark age in that continuity), and had since been somewhere between outlawed and heavily regulated (not just Kahn, but Julian Bashir on DS9 having illegal genetic modifications his parents went to prison for). Still, yay treating heart disease...?


July 1, 2020

Oh hey, twitter went through my history and deleted some _other_ tweets that said "guillotine the billionaires". There's multiple "this tweet is no longer available" holes on the first page. Took 'em long enough, since that's what they cancelled my login for in the first place. Of course they haven't got them all yet. (I noticed because an Amazon recruiter emailed me and wondered if I'd changed my mind about working for them.)

Patreon has been emailing me about sales tax for weeks, and today I sat down to try to do whatever it wants but apparently I have to rewrite every single patreon goal? Because it's assigned tax or no tax to each one via keyword heuristics, but if I go through and edit those to manually set them it has a wizard that insists I add other new information to each goal. My patreon goals have names like "do the thing", it's a glorified tip jar offering people the ability to exprss support for my open source work. The main information from the selections was was whether I should spend time on toolchains, system builder, or toybox.

Anyway, I pressed the "don't charge anybody this month" button again so I don't have to worry about it.


June 30, 2020

I started a cleanup fork of the new j-core build repo.

We never use $TEST_DIRS2, I'm removing it because it's easy enough to add back when there's a user. (Infrastructure in search of a user bit-rots.)

We set $RELEASE using hg instead of git: it should be using "git describe --tags" but we've never tagged anything in the git repo so it can't find a base tag to say what we're X commits ahead of. (Should we tag what we released?)

"make clean" has some things I can figure out how to fix (components/cpu/cache needs an extra ../ in front of several paths), and some I don't understand:

make[1]: Entering directory '/home/landley/jcore/clean2/components/ddr2'
Makefile:14: ../icache/build.mk: No such file or directory
make[1]: *** No rule to make target '../icache/build.mk'.  Stop.

There's no icache directory anywhere in what we released? Hmmm...


June 29, 2020

It's official, the Golden State Killer was a cop. And when not serial killers, the police have always been surprisingly useless. The protests against them continue, as does the police brutality.

The GOP endlessly betrays the military, here's a thread and the latest.

It turns out the reason Japan never had that much problem with coronavirus is masks work.

Remember the thread about how the british were basically nazis a century earlier who just got away with their genocide to the point they could write the history books? (Instead of a thousand year reich on land stolen by infantry plus concentration camps, the sun never set on the British Empire built on land stolen by the navy plus selling drugs and killing anyone who objected, with extra prudishness and capitalism.) Anyway, here's a thread of horrible things the british did to Africa.

So many issues dropped on the floor are still true, but then we knew that, and there's plenty of new ones, and here's some weapons-grade stupid, and the usual white male assholes who think the rules don't apply to them. The GOP continues to demonstrate that the cruelty is the point, not a side effect of their policies but the explicit objective. But then we already already know this stuff.

Captain Awkward had a really good essay.


June 28, 2020

The Dell bios in my latop got confused somewhow, probably by the extension cord I bought so I could sit 3 tables away from the outlet late at night at UT, and social distance even when somebody else is there; not a whole lot of other opportunities to get out of the house and work away from the cats, and the round trip walk is something like 13,000 steps according to the Android "let us track your every move for your own good" app.

The symptom of the Dell BIOS glitch is that Linux would suspend, it would stay suspended for one dim/brighten cycle of the LED, and then it would turn itself back on. Initially it only did this when plugged in (possibly the extension cord put it undervolt slightly, although it was happily charging? But the bios power monitoring stuff set some sort of flag Linux didn't know how to clear, so that it would resume any time it was plugged in.

But I could work around that by unplugging it before suspending it... until today, when it powered itself on in my backpack during the walk (and was VERY hot when I took it out, although still running fine with half the battery left). And rather than getting work done, I spent hours closing windows so I can power it off properly before walking home.

Luckily gmail has decided to allow me to send and receive email through my phone tether again (no idea why, I clicked "send" on an email window I'd finished without thinking and it went through, and it can download emails too). Whatever weirdness in the gmail servers was vetoing it seems to have cleared up? (Whatever it was survived a reboot here so it probably isn't a strange route cached at this end, and it cleared up BEFORE I rebooted the laptop again, since I'm trying to clean up and close down to do that.)


June 27, 2020

11 emails got through my spam filters from the various democratic fundraising panic-mongers today. I must give them money because the wave of recalls of governors could lead to a constitutional convention enshrining the GOP in power, and I must give them money because polls show the GOP's historic unpopularity could allow the democrats to sweep the field. (The fundraising shills see no contradiction.)

On the one hand, I'm sympathetic to their cause. On the other, I have ZERO guilt about deleting all that crap unread and adding more spam filter entries. I gave small amounts of money to elizabeth warren and AOC last year. I never agreed to sign up to any mailing lists when doing so. I'm being emailed from a dozen different sources, and have LEARNED MY LESSON ABOUT EVER GIVING THESE CLOWNS MONEY AGAIN.

The democrats are Good Cop, which is still a servant of the plutocracy. Their nominee is an octegenarian. I'm getting trump campaign ads on youtube about how Biden is too old to be president and going "yeah, good point". There is no positive case to be made for Biden, it's 100% a vote against. An 80 year old man who thinks he should be driving the country is automatically disqualified, but his opponent is a literally demented racist con artist who's surrounded himself with literal nazis and has done a dozen things that would be treason if anyone else did them.

Meanwhile, voter suppression has eliminated my local polling place. The one two blocks away from me has been replaced by one eight blocks away, which isn't as bad as I originally thought (1500 barbara jordan blvd is _not_ 15th street, which is 30 blocks south of 45th street) but still annoying.

You know how we had World War I and World War II? Welcome to Cold War II. (Except Russia has an economy the size of Italy's _while_ being one of the top 3 oil and gas producers in the world. Without that, what's left?)

The army does not lead. The army is not _for_ leading. The entire point of basic training is to turn out obedient soldiers who need to be told what to do.

GOP science denial is the same for coronavirus, climate change, and evolution. The police are worse than you think, but Cops gonna cop. The oil companies aren't going down quietly. Here's an excellent thread.


June 26, 2020

I've known about the arm-based macs for a while. Charles Stross's twitter has been putting together pieces on that for months. But more to the point, I myself predicted this more than ten years ago, and wrote about it repeatedly. And I'm by no means alone in that.

The numbers make it an obvious move: Apple sold 217.7 million arm-based iphones in 2018, and 18.2 million x86-64 macs. That's 10 times as many devices _before_ you add stuff like ipads. "Yeah but margins"... the estimated gross margin on the iPhone X was 64 percent. Way more money has gone into arm for a decade now, which means way more R&D spending. Having a rump x86 platform for historical reasons stopped making sense a while ago, it's just overcoming the friction of migrating everything over.

This transition is mainframe->minicomputer->microcomputer->smartphone. The phrase "All the World's a VAX" was a symptom of the minicomputer's standardization around a common platform that became the category killer due to the positive feedback loop of network effects: users are drawn to the system with the most software, and programmers write software for the system with the most users. When the microcomputer standardized around the IBM PC all the world became a 386 (then the backwards-compatible x86-64), and now both iPhone and Android are arm processors under the hood. Using an x86 PC in 2020 is like using a VAX in 1990: existing users keep at it but nobody new is going there and the developer pool will age out.

Google's been sticking arm in chromebooks, but hasn't really laid down the law about ARM development workstations. It's been cross-compiling from x86 PCs. I've been trying to fix that by making android self-hosting, but it's a hard thing for one guy to do part-time without official backing.

Oh well. Grinding away as best I can.

Meanwhile, I got a message on the toybox list in which its author probably didn't MEAN to say "I assure you, you will fail. Look at what I did instead. My project is so utterly superior to your folly, it's a shame you cannot perceive my greatness, if you only you welcome my project into your heart you will be bowled over by its magnificence and abandon your doomed frivolity, could you but conceive the wondrous splendour tantalizingly beyond the limits of your cognition."

Ok, what he actually said was:

...reading the blog is a bit painful because it's clear you will never finish your shell with the current strategy (and this is an informed opinion, after doing it myself).

I think you're less than 10% done with 3K lines of code. I think you will agree with my assessment if you [look at my stuff]...there are so many features you haven't even begun to think about.

A couple things I would suggest:

1) Learn about grammars and parsing...

2) Use some of the test cases and framework I developed...

It's... difficult to reply to this politely. I've been poking at shell stuff on and off since 2006 and maintained my own compiler fork for years, and he wants me to learn _about_ grammars and parsing? Like... that they exist?

His argument seems to be "tinycc was not designed like gcc, therefore it couldn't possibly work because I only understand one way to do it and must reach out to correct you if you try anything else". Except applied to the shell I'm not using yacc and lex in instead of the compiler I became interested in because it didn't use yacc and lex. (The simple design of tinycc is what attracted me to it in the first place.) And lex and yacc are both on the toybox roadmap, lex because it's in posix and not that hard to do, and yacc because the kernel added it as a build dependency in 2018, so yes I am aware they exist.

*shrug* If I _do_ have to write 9 times more code, I can. But I note that the whole of toybox (outside the pending directory) is currently 32,000 lines of C, so he's saying the shell _must_ be as big as the rest of toybox combined (implementing 206 commands). He can't conceive of it _not_ being that big. He insists it's not possible, and comes to my list to assert that impossibility directly at me, because watching me try is "painful".

I'm happy to take constructive criticism and am the first person to rail at how LONG all this is taking me, but... sigh.


June 25, 2020

Once again, The Onion Merely Documents Reality. There is no exaggeration for hyperbole because in the twilight of the Boomers, there's no room to exaggerate. But the protests are successfully making the Boomers uncomfortable, which is important and valuable work since the boomers stole all the wealth and of course blamed their kids for the lack. (Not stole the _money_, since that's a social construct they can just print at will. But the Boomers bought up all the land and housing, tied up all the avenues of social advancement with BS gatekeeping, and bought laws and politicians to enforce their worldview with them on top, while things like climate change, racism, and increasing inequality are Somebody Else's Problem they will die before addressing.)

The US oil shale industry turns out to have been a giant scam currently forcing lenders to write off $300 billion in loans, which never made a profit and only stayed in operation as long as they could endlessly borrow more money.

Suburbia was a giant federally subsidized racist project to enable white flight, and can't survive without the subsidies. And yet the idea of basic income remains inconceivable to Boomers.

The GOP is a terrorist organization, and they're finding new ways to cheat in elections because voter suppression is the only reason they still hold any office anywhere. Meanwhile the Tories are using Covid as a transparent excuse to abolish trial by jury.

Airbnb is terrible, so nobody's too sad they seem to be hurting financially.

Why is the Comic Book Legal Defense Fund is making women sign non-disclosure agreements preventing them from talking about being sexually harassed? Does Neil Gaiman know about this? He's still working with them...

Today I learned that the original 13th through 18th amendments, passed by congress and signed by president Buchanan, would have enshrined slavery in the constitution, with the 18th amendment literally saying support for slavery could never be repealed. The only reason they didn't get approved by enough states is the south couldn't wait and seceeded after Lincoln's election, starting the civil war. (But of course we don't teach this in school. Stem Stem Uber Alles, you can't be a bad person if you know enough science, as all the Nazi Scientists demonstrated. And therefore the Boomers stopped teaching the humanities, just like they ended manned flights to the moon as soon as it was clear they personally wouldn't be going, because if THEY didn't need it anymore nobody did.)


June 24, 2020

Ok, I THINK the variable expansion prefix cases are ${!} ${!@} ${!@Q} ${!x} ${!x@} ${!x@Q} ${!x#} ${!x[} ${#x} (not counting ${!x[*]} and ${#x[@]} which is a todo until array support). Yes, bash considers echo ${!@Q} an error (bad substitution) but if it's gonna accept ${#@Q} I think I should support both. Another inconsistency is that:

$ xx() { echo "${*@Q}";}; xx a b c d
'a' 'b' 'c' 'd'
$ xx() { echo "${@@Q}";}; xx a b c d
'a' 'b' 'c' 'd'

Should NOT produce the same output, that first one should be 'a b c d' because only "$@" splits words within quotes, I believe "$*" uses the first character of IFS to glue them together (and the default IFS is " \t\n"), which means the oggham nonblank space character sadly is not recognized by default as a word separator. Sadness. I'd ask posix what properly supporting utf8 should mean here, but Jorg Schilling isn't dead yet. Given that sort changed what alphabetical order means between ubuntu releases (because gnu/gnu/gnu/stallman/gnu), I expect adding proper utf8 space support in a future toysh release is within range of historical precedent. (And no, that had nothing to do with Ubuntu's dubious flirtation with the Defective Annoying SHell.)

Sigh. Chet Ramey keeps saying "mksh did that" or "that's what ksh93 does" and... if my question was "bash: who hurt you so you wound up like this", I would have my answer. But it's not what I asked?

C and unix managed to take over the world by being extremely simple for what they did. They staked out local peaks in design phase space and became category killers for them, and there was some local breakage like DOS "near" and "far" pointers or systems where char wasn't a byte, but mostly people carved out a common subset that made sense and the standards bodies documented that. Then the "unix wars" happened (starting with AT&T's drive to close up and commercialize the source), resulting in stupid fragmentation that almost killed unix until Linux came along and established a new de-facto standard. But C and Unix remain VERY GOOD at what they do, and it would be nice if people UNDER the age of 50 could still learn and use these technologies rather than reinventing the wheel badly.

I'm interested in implementing "what bash does" because that was the Linux de-facto standard. Linus Torvalds extended his terminal program to run bash specifically, that's why/how the Linux system call layer was created in the first place. Bash was THE shell of Linux until Ubuntu screwed up 15 years later, and ubuntu's mistake was so _spectacularly_ inept that the world wound up with systemd poorly addressing the problem ubuntu utterly failed to solve. Shuttleworth just never admitted his mistake and undid it.

But any time a single implementation carries the torch for a long while (bash, gcc, vim, netscape, linux, openssh, coreutils), you get a mix of "behavior you want" and "coincidental implementation detail corner cases" that are gratuitous sharp edges which should be filed off. Figuring out which is which is hard and subjective.

Decades of accumulated scar tissue are no excuse for winding up with crap like C++ instead of C. Needing a new incompatible version of the language every few years that everyone has to make a flag day transition porting their code to is not a good thing. Nor is being tied to a single irreproducible codebase that its own developers try and fail to replace.

C99 is still "good enough" 20 years later, and part of the reason is that there have BEEN so many independent implementations that had to work out what the common subset was and how to agree on necessary compatibility. Posix-2008 is still more or less good enough 12 years later and the main reason to use even _that_ instead of an earlier version is the 2008 update acnowledged Linux's openat() and friends to work around PATH_MAX limits and races (and that dprintf() is a thing).

Most of the useful tweaks to posix over the past 20 years have been scalability issues like large file support (terabyte disks) and 64 bit processors with gigs of ram. Other than that, they've removed stuff like tar and cpio that everybody still uses (ignoring their change), while FAILING to get rid of obsolete crap like sccs and ebcdic support in dd. And don't ask me what they were smoking replacing 'cc' with 'c99' because I couldn't tell you: since then there's been c11 and c18 which DO NOT MATTER because you can still feed K&R code into the compiler and it works. Call the compiler "cc", it is the C Compiler. (Not gcc, not clang: cc.) The main reason to move from c89 to c99 was the new uint32_t types with actually specified bit sizes, which LP64 did better anyway (uint32_t is just "unsigned" on every LP64 platform, which means everything except windows which as far as I'm concerned is "everything").

Ahem. Where was I? Implementing ${!@Q}, right.


June 23, 2020

A few days ago I got recruiter email du jour titled "Come Join our Billion Dollar Start Up", with the first paragraph:

Interested in joining one of the fastest-growing, venture-backed startups reimagining the $45B industry of enterprise physical security? At Verkada we are modernizing the way spaces stay secure and safe with our AI-based, easy-to-use management software and industry-grade cameras that enable advanced facial recognition, highly accurate motion detection, and other robust search features.

Which also mentioned that they were impressed by my contributions to the busybox project, which I handed off maintainership of 13 years ago and my last commit to was 9 years ago. (At a guess, it's because Bradley wanted a copyright notice when you typed "busybox" with the 3 maintainers to help lawsuit standing, and although he removed it again when I became a vocal opponent of the lawsuits continuing indefinitely, for several years busybox spat out my name when you ran it, which was not my idea.)

I deleted that email without replying, and today I got a follow-up email which I replied to honestly:

> I wanted to follow up with you about our opportunity here at Verkada. I imagine
> lots of companies reach out, however, I'm very impressed with your background
> and would love the opportunity to connect with you to tell you more about
> Verkada and what we're building.

You said in your first email you're doing AI based facial recognition tech to protect corporate assets, either from the company's own customers and employees or from the tens of millions of people currently protesting in the streets in lieu of sharpening guillotines.

https://www.washingtonpost.com/technology/2020/06/11/microsoft-facial-recognition/

> Let me know a good time to connect this week or early next week?

Half the "Boomers" were born before 1955 and the average US lifespan is 79, putting the LD50 on the Baby Boom somewhere around 2034, which seems a reasonable upper bound on when they lose political power. I do not expect the current form of capitalism to outlive them (the eight richest people in the world owning more than everyone else combined doesn't make sense to anyone younger than Boomers), the question is how bloody its end will be. The DNC's desire to replace the oldest president in history with someone who'd turn 80 in office suggests "very".

Another business model tightening the lid on inequality to let pressure build up is not something I want to stand near, let alone with. I would have thought palantir and emerdata had it covered, but apparently they're not good enough at blaming brown people yet?

https://twitter.com/Chicken3gg/status/1274314622447820801
https://twitter.com/osazuwa/status/1274444300894572546

No thanks,

Rob

I'm wondering if I was too harsh. It's not the recruiter's fault, but the idea of a billion dollar startup focusing on defense of corporate assets via facial recognition doing recruiting in the current political environment seems kinda creepy. Or is it just me?

I mean, the protests are continuing to escalate, the Resident is having dementia episodes so bad that all reporters have to be evacuated from the white house, how much endgame is left here? The world the Boomers knew was always completely artificial. It didn't exist before they did, and won't persist after their death. We have better ways to do basically everything now, we just can't deploy the new tricks until the old dogs roll over and play dead.


June 22, 2020

I meant to do shell programming tonight, but instead I have bug reports to field. Some of which I've been re-pinged about because I didn't get to them earlier...


June 21, 2020

So the GOP's going all in on voter suppression. Not even pretending anymore. Meanwhile the cops are murdering people and confiscating nearby cameras. Occasionally learning to pay lip service while people are looking, to buy time to stock up on military equipment while being utterly useless at their nominal jobs. They literally can't NOT be monsters, even to save their jobs.

Racism and misogyny have always had a divide and conquer aspect, but at their core they're bullies claiming other people as property, and convincing everyone who objects that they're a tiny isolated minority lost in a sea of "them".

I am reminded of a classic Dr. Who quote, "The very powerful and the very stupid have one thing in common. They don't alter their views to fit the facts. They alter the facts to fit the view. Which can be uncomfortable if you happen to be one of the facts that needs altering."

The pinkertons were a private security force guilded age capitalists like Andrew Carnegie hired to murder striking workers a century ago. (He later endowed Carnegie Hall so people would remember him as a philanthropist rather than a capitalist, the same way Alfred Nobel created the Nobel Prize so his obituary wouldn't just say "weapons manufacturer and war profiteer".)

The modern robber barons similarly point to the infrastructure they build, claiming that without rich people exploiting poor people we wouldn't have computers or railroads today. But what they're ACTUALLY trying to do is corner the market to profit from trends happening anyway, by getting there first and locking out the competition. Microsoft did not invent any new software in its entire history, when it didn't outright buy new products it copied other people's stuff and drove the actual inventors out of business. (In the process replacing the word "invention" with "innovation", which means "it's new to _us_, the novelty is that we're doing it now".) Facebook did not invent social media, it just captured the largest userbase via network effects and lock-in, decades of consistently unethical data collection and of course russian money laundering (when talk of zynga's criminal activity got too out of hand they created a game called "mafia wars" (well, cloned one) about the same way Boris Johnson pretended to make bus models out of cardboard). Of course back then, nobody took that sort of accusation seriously...

Modern cops fill the role of Pinkertons, intimidating protesting schoolteachers long before they had direct anti-cop protesters to maim. (They became the problem.)

The protests continue. The thread of police misbehavior is well over 500 entries and still going, and that's just the egregious stuff, not the endless microaggressions. They cannot be reformed.

The Boomers have always wanted more police standing around waiting for something to do. (And as usual with bullshit jobs, they make busy-work for themselves to justify their own existence. It's the civil version of an autoimmune disease.) Until the Boomers die, we're stuck in the past with them among with the rest of the rotting debris of expired Boomerdom, and nothing is going to get properly fixed.

Listening to what little kpop I have in my collection, in honor of the great work they've been doing. (I used to object to the lyric "dance dance dance 'till we run this town" on the theory... it doesn't work like that? I stan corrected: Quotus Ed Demonstratum.)


June 20, 2020

Hard to get traction on technical projects, the world's still on fire. I installed Merge Dragons on my phone, then deleted it again after it ate basically a day of productivity. (It's waaaaaay too skinner boxy for me.)

Still, adding lots of tests for toysh to pass. Here are three different behaviors:

$ abc=${a?bc} echo hello; echo blah
bash: a: bc
$ readonly abc=123; abc=def echo one && echo two
bash: abc: readonly variable
one
two
$ echo potato < /does/not/exist || echo hello
bash: /does/not/exist: No such file or directory
hello

I'm getting the second two right at the moment, but the ${var?error message if unset} syntax is documented to exit the shell (if not interactive) in the bash man page, and that means I need different TYPES of error handling for the same action. (It can error-but-continue or error-and-abort. Great. A todo item.)

Then there's "echo > walroid ${a?blah}" not creating walroid, which toysh and bash are likely to _continue_ to do differently. In this case, the ${a?blah} variable resolution (which is an error) happens before the create-file redirect in bash. But in my shell, it handles the argument expansions in order, and when an argument is a redirect it Does The Thing right then and there, because the NOMMU support needs to exec as shortly as possible after vfork(), and you don't want to do error handling in vfork() child context with the parent blocked if you have _any_ choice int he matter. If you "echo > ${a?blah} walroid" it won't create the file because the argument expansion is a prerequisite for the redirect (and thus evaluated, and aborted, first), but if you "> name" it opens it right after resolving it, before looking at the next argument.

In this case the difference is "supporting nommu" vs "not supporting nommu", and I think the right thing to do is document the deviation and move on. (A deviation from bash, rather than from posix. I don't remember what if anything posix says here, and am unlikely to care while Jorg Schilling remains alive.) It's sequencing in an error recovery path: I suppose if a real user complained I could reevaluate and MAYBE adjust it. (Or maybe not, it would depend...)

A sequencing I do not get right (as in "matching bash") at the moment is:

$ ./sh -c 'abc=def > /does/not/exist; echo abc=$abc'
sh: /does/not/exist: No such file or directory
abc=

Bash prints abc=def because the assignment happens even if the redirect fails. Which is weird because it won't RUN a command but it'll run an ASSIGNMENT and... what? Hmmm. (I can adjust that to match, but ew?)

$ export abc=def > /does/not/exist
bash: /does/not/exist: No such file or directory
$ echo abc=$abc
abc=

That's bash. It's being inconsistent, doing local variable assignments but not assignments via builtins under the same circumstances.

Sigh, there's a certain amount of version skew WITHIN BASH, some of which I'm now responsible for by reporting this sort of thing to the maintainer. :P Do I care about this? Should I wait for somebody to complain and say it broke their script? Hmmm...


June 19, 2020

Oh dear. I think we know what the next financial crash looks like, and the epicenter is Softbank (the giant Japanese company that owns the ARM processor), which just used a company called Greensill to pull financial shenanigans on the level that destroyed Lehman Brothers, Arthur Anderson, and AIG circa 2008. This may be why Softbank is selling $20 billion dollars worth of its stake in T-mobile.

Remember how the 2008 financial crisis was basically the invention of the mortgage bond coming to fruition and the bills coming due? This greensill guy did something similar (used blockchain to securitize receivables). The problem isn't that Softbank invested $1.5 billion in Greensill, that's just a WeWork level mistake. No, the problem is Softbank dogfooded Greensill's product and applied it to all their own stuff, and it's infected all their own supply chains and finances. And this one is growing, maturing, and defaulting at internet speeds.

I'm listening to a podcast about this. Here's a few quotes from the first half hour:

"Companies turn their supply chain into a personal bank to invent new credit... A large company will extend their payment schedule to some ludicrous amount of time (200+ days) then if you want to be paid in 30 days you must use our reverse factoring partner... which turns the company's accounts payable from normal operating expenses into debt which is then owed to a financial institution which can be securitized and sold..."

"Reverse factor debt is recorded as operating costs... once the money leaves the factor to your books, it's not booked as debt... brighthouse had leverage ratios of 90-to-1 when it collapsed. The reverse factor is not legally a bank, it's a tech company... Greensill is the largest non-bank bond issuer in Europe."

Circa 26:30 in the podcast: "Uber isn't a tech company, it's an employment rights circumvention platform. AirBNB i's not a tech company it's a property and rent tenant's rights circumvention platform. Greensil's not a tech company, it's a financial regulation circumvention platform None of these tech companies are actually tech companies."

Ok, blockchain-based invoice processing created a new completely unregulated financial asset class, which is in the process of defaulting and collapsing, it's already somewhere north of a trillion dollars, and softbank at the center of it. It sounds really really bad, and I dunno how far the rot's spread.

Luckily, the current economy is completely fictitous, as I've mentioned rather a lot of times before. It's a bunch of numbers in a computer network that haven't represented anything real in living memory. Unfortunately, "there's no such thing as IBM, just a bunch of people pretending" can be both true and useless. When people riot over sports teams, people are still rioting. When stupid decision makers decide to impose unnecessary austerity, real people get hurt. The boomers will cling to capitalism until they die, they're too old to do anything else, and what THIS nonsense means is the amount of modern capitalism that's literally nothing but a giant scam drowns out the part that ISN'T by such a large ratio you can't even easily measure it.

Instead of yet another bailout, give every US citizen basic income. Politically, you can set the social security elligibility age to zero with a one page bill. You can "fund" it (prevent inflation caused by it) by bringing back the 92% tax rate on any earnings over $10 million/year, taxing stock buybacks the same as dividends, and cutting the military budget to no more than what the next 3 countries spend combined (rather than the next 8).

The Boomers will never stand for it, of course. Biden's would respond "over my dead body" but he turns 80 two years after the election, we can start work on lots of stuff that needs to happen over his dead body without impacting their schedules in any way...


June 18, 2020

At the UT table late at night, desperate need to blow my nose (cedar pollen!) and I dug up a Dotour oshibori out of a pocket in my backpack. This is sort of second derivative rude. (In japan it's hugely rude to blow your nose in public, you go to a restroom for that, but I'm not _in_ japan, this oshibori's been in my backpck so long it dried out through the plastic wrap and is just a napkin now. And in any case the point of coming here at 1am on a weeknight is nobody else being here...)

I'm to the point with anime subtitles where I can't really follow most of what they're saying, but I sometimes go "that's not actually what they just said". I listened to the japanese audio track of Groundhog Day on netflix the other night: I dunno if I've got it because I downloaded it in japan or of it's generally available now, but I'll take it. I suck at foreign languages and Japan's racists have used the pandemic as an excuse to forbid all foreigners visiting the country for the foreseeable future (presumably until the rescheduled olympics), but I'm still TRYING to learn this language...

Elliott is being really nice walking me through AOSP build and testing, but he just pointed me at cuttlefish, which that page puts next to another emulator and the way to figure out which to use is "ask the bionic maintainer". (This is why I compulsively write my own documentation. If you do not already know how to do this, where do you START. I also just spent 5 minutes trying to read the page before it, which is the conceptual equivalent of a forkbomb. I THINK what it's trying to say is:

A Generic System Image (GSI) is vanilla android built from unmodified Android Open Source Project code (with no out-of-tree additions) in a simple default configuration useful for testing. The only configuration selected when building a GSI is to specify what hardware (or emulator) the image should run on.

The GSI provides a common baseline, and is used to regression test that current android still works on the hardware by running the Vendor Test Suite (VTS) and Compatibility Test Suite (CTS).

Anyway, cuttlefish. Trying to build for that...


June 17, 2020

The resident's Progressive Supranulcear Palsy has advanced to his frontal lobe. There is no treatment, it is progressive and fatal. His con artist skills have been dedicated to covering for his increasing incapacity. But given how racist even places like silicon valley are, his supporters refuse to abandon him as long as he's still hurting the people they hate. (And given that he's hurting _everybody_...)

In this video a cop forces a weapon into prone protester's hand to retroactively justify the beating he's giving him. Cops kill people because they're insecure ("respek mah authoritai"), and with good reason. We're going about it wrong have no follow-up (all EMTs, no treatment or surgery) and should unbundle the services. Meanwhile, in response to the police abolition movement, cops offer to pepper spray students in their own classrooms less often (Why did LA's school police have a tank and grenade launchers? No seriously, why?) And of course the Golden State Killer was police officer. The police are the problem, it's not new, and they serve no other purpose.

Remember when the ACLU did the Good People On Both Sides thing defending Nazis from being punched? The protests are bigger than ever and accomplishing good things but the roots of the problem go deep. There's plenty of good videos on how the supposed good guys have been deeply insufficient. This isn't "equivalence", this is "Joe Biden is not the answer, he's a tourniquet at best". The real enemy is capitalists who fund racism as a defense mechanism for inequality. It's always been "let's you and him fight" sicking the poor against each other. (Again, the book "masterless men" is about that. I should read more books, there are some excellent ones out there but I have a copy of Masterless Men I haven't finished, a copy of The Box I'm like 3 pages into, I only made it halfway through Bullshit Jobs, but the audiobook is finally out.)

When the Boomers die, we burn down everything they ever touched. If you can't vote younger than 18, drink younger than 21, and be president younger than 35 then you can't hold elected office OLDER than 64. Mandatory retirement from public office: you get a gold watch and a pension on your 65th birthday and go into an advisory role where you are not the final arbiter of any decision.

Good thread on the intellectual errors at the heart of libertarianism. (From someone who thinks there's a point to it.)


June 16, 2020

I tried again to do an android build, left it running overnight, and today I ran "emulator" which said:

emulator: WARNING: encryption is off
Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"
queryCoreProfileSupport: swap interval not found
emulator: ERROR: VkCommonOperations.cpp:496: Failed to create Vulkan instance.
E0616 13:12:24.550453924 31310 socket_utils_common_posix.cc:201] check for SO_REUSEPORT: {"created":"@1592331144.550415611","description":"SO_REUSEPORT unavailable on compiling system","file":"/mnt/tmpfs/src/android/emu-master-dev/external/grpc/src/core/lib/iomgr/socket_utils_common_posix.cc","file_line":169}
emulator: ERROR: AdbHostServer.cpp:102: Unable to connect to adb daemon on port: 5037

It popped up an empty window, which then sat there eating 100% of one CPU for an hour before I killed it. So that's nice.

Is it because I didn't build an image for it to run? Or didn't specify which image to run? Should it NOTICE it's not connecting to anything rather than spinning eating 100% cpu? And SO_REUSEPORT is in /usr/include/asm-generic/socket.h where it's #defined to 15 so what does it mean by "host system" here? Unknown element "blank"? Is failing to create a Vulkan instance fatal? If so why didn't it exit? Why is the emulator trying to connect to an adb daemon? If running adb itself will launch an adb daemon why won't the emulator launch it if it's not already running? Why didn't the instructions I was following mention the need for adb to...

So yeah, my experience with AOSP is about the same as usual. There's like a semester-long course in it I haven't had, and until then just trying to pick it up as you go along is a weird thing for a hobbyist to do, I guess?

I also broke threading in a reply I composed but haven't sent yet, but ever since Android updated itself a few months ago I haven't been able to access gmail through my phone tether, only through the home connection. (same issue from april, still broken I just live with it.) Neither the pop3 nor smtp servers will accept thunderbird logins when it's using my t-mobile connection, but it works fine from the home router. (Fetching gives the misleading "web login required" error message. Sending just claims the smtp password is wrong for any password.)

I thought it was ipv4 vs ipv6, but forcing dhcp to get an ipv4-only address on the phone tether doesn't help, while the Google Fiber connection "just works". I'm assuming that part of gmail has accumulated a full ogres-are-like-onions speech worth of "layers" in its codebase. I guess if I had Google Fee Fo Fum then my phone tether would also "just work". (I miss "don't be evil". It was good to have goals.)

But as with 2/3 of the ads on Youtube being "pay youtube to remove ads", if the reason to do it is because they broke _not_ doing it, it's Danegeld and goes in the "death first" bucket. I am ALREADY paying for netflix, hulu, and prime. I compromised enough to add crunchycrunch. I am NOT paying for disney+, hbo max, cbs all startrek, bbc iplayer, funimation... The services I _am_ paying for claim to be categorical, figure out how to work together and get paid rather than sending me a seperate bill for each and every show I watch. I object to being nickel and dimed with micropayments, I do not want to have to TRACK this nonsense.

Google could have bought twitter instead of creating Google+ and it could have bought netflix to get me as a customer instead of darth vader altering the bargain pray I don't alter it any further. And literally announcing plans to intentionally annoy your users into giving you money? That not only makes me implacably opposed to ever rewarding that behavior on principle, but to wait for the named decision maker to no longer work for the company before even considering re-evaluating that policy. You won't get a dime from me while the clown who ADMITS to doing that could ever call it a win. That particular Boomer is 60 years old, LD50 on Boomers is 79 years, so 50% chance that resolves by 2039. It's easy to be patient when you're waiting for someone specific to die.

My todo list runneth over and there's lots of stuff I'd love to spend focus on but don't get around to for years. Anything I can put in the "this too shall pass" bucket where just waiting it out DOES theoretically advance the agenda (if in a "the sun's only got so much hydrogen" purely theoretical way I may not personally profit from)... I can do that. I've mostly waited out Windows. I waited out AOL, and can wait out Facebook the same way. I will happily weaponize _not_ doing things. Any time someone thinks I'll come around given enough time and they can wait me out it's just adorable. I'm smiling at the inevitability of the ozymandias poem as it applies to them personally, not just on this issue but on any two atoms whoever it is ever exerted the most indirect influence over still having any discernable relationship to each other. Physics puts an expiration date on the last traces of their influence vanishing into the noise, and that can be my happy place if they want to put themselves in that category. (I've mentioned before having a vindictive streak with a high activation threshold. It mostly manifests these days as "ok, entropy is on my side on this one" and I don't usually explain the full meaning when I say I'm not gonna care about the Posix committee while Jorg Schilling is still involved, because it disturbs people. But the beauty is I don't _have_ to do anything. Just smile and wait.)

This doesn't mean I won't change my mind in response to new information, but if you haven't come up with a better argument or fresh data my opinion is not gonna _age_ into submission. I question my existing beliefs all the time, and I'm still using github despite their acquisition by microsoft (after years of boycotting expedia for less) because monkey-boy handed off to a new CEO who sees microsoft as properly third wave rather than second wave, and thus won't necessarily "embrace and extend" everything each new venture touches into a defense of Windows because third wave companies don't have a "core business" providing a corporate identity as "the company that does X". (Sigh, I wrote about this at length 20 years ago and revisited it 9 years ago. I should do another pass but... todo list runneth over.) But at the same time, Microsoft has earned an awful lot of distrust over the years, and expecting them to just stop monopoly leverage when it was all they did for so long that when you talk about their "bundling" you have to differentiate at least three different types (and when you say "the microsoft antitrust trial" you have to specify Judge Sporkin in 1994 or Judge Jackson in 1998)... I am reluctant to trust them with anything open-ended because there's a nonzero chance this newly sober company will fall off the wagon. I have advanced them a fixed amount of trust, I am currently being asked to advance more and am deeply uncomfortable, but there are valid arguments for it? The longer the new CEO does NOT spring a trap, the more comfortable I am, but today? Hmmm... (And I'd never _not_ have an exit strategy with those guys.)

So backing up about 3 tangents, I had to cut and paste-as-quotation the message I replied to from the web archive so I could compose a reply and send it when I get home, which doesn't marshall the message-id into a reply-to because implicit metadata, and that tends to break threading.

And yes, I'm still using pop3. The reason I don't use gmail's imap server is the issues with _that_ from 2012 never got fixed either, but then "living with obvious problems with gmail" dates back before that. Google's been too big to see who's underfoot for a while now, but I've also been pretty consistently on the wrong side of 80/20 decisions in the computer industry since I was ten years old, and am used to it.


June 15, 2020

One video today: the creator of Veggie Tales (and voice of Bob the Tomato) explains systemic racism in the United States in just under 18 minutes.


June 14, 2020

This chunk of shell is one of those big "bit off more than I can chew" sections, which boils down to me trying to write code and instead coming up with lots of tests and adding them to tests/sh.test to keep track of the things I need to implement. The resulting code can be relatively small and simple, but it does a lot of little fiddly things each of which has to be right, and many of which aren't obvious from the man page. (For example, ${!indirect} is listed several pages up from ${!prefix@} which is different from ${!array[@]} but all three are "saw ${! and did a thing".)

So I have a zillion open tabs with various command lines I've run to explore corner cases of bash variable resolution, and I'm slowly going through and closing them by adding sh.test entries for each one. It's slow and tedious, but that's what counts as "progress" at the moment.

Another thing I have to look forward to is marshalling random global state into subshells, such as the current set of function aguments, or TT.bangpid. Did you know that the $! value gets marshalled to subshells? (With fork() it's free but with vfork() you have to send it through the pipe.) "echo $! &" prints nothing for a fresh shell because $! is last background pid and we just GOT backgrounded but I don't think we include ourselves in that, so you have do something like 'true & echo $! &'. (And to BLOG that I have to type &amp; a lot, along with &lt; and &gt; but I'm used to it by now. You can't spell "evil" without "vi".)


June 13, 2020

Just one thread today. I found it extremely educational: Follow the money.

(Yes, I'm cheating slightly with the even/odd day thing. I should update my hacky old rss feed generator to do "programming.rss" and "politics.rss" and so on, since I've been doing the span tags for _years_, albeit with nothing regression testing that I close them or anything. Eh, it's on the todo list...)


June 12, 2020

Race conditions in job control: process creation fires up a pid that can in THEORY get suspended before it's properly registered with the job list, what should I do about that? Hmmm...

In run_command popen_setup() returns a PID which gets assigned to pp->pid in a local that is not yet on anybody's list, and if I set up a signal handler to catch child signals... If I'm calling wait() it's not gonna do anything until I call wait, but I think I need a signal handler? Hmmm, the command line plumbing reports terminated jobs at prompt time and the jobs command also queries, so possibly those are the points that handle it and update the list so it doesn't have to be asynchronous?

Meanwhile, I'm trying to work out what order of operations leading assignments and redirects happen in:

X=${x?bc} > walrus
>walrus echo ${a?bc}

And neither of those create "walrus". Unfortunately, they also abort the current script, so my tests can't report on the existence of walrus. Sigh, I need to 'toysh -c "firstpart"; secondpart' don't I?

And I'm confused again. This is the debian host bash:

$ touch abc\)d
$ echo ab+(c?d)
abc)d
$ IFS=c ABC="c?d"
$ echo ab+($ABC)
ab+( ?d)
$ ABC='*'
$ echo $ABC
abc)d blah blah.txt

Does wildcard evaluation occur before or after variable resolution? It seems like it's before sometimes and after sometimes? Ah! It's IFS splitting, it's happening after variable resolution but before wildcard evaluation, so the split is making +() into two words and not registering as a wildcard grouper.


June 11, 2020

Today's reason to guillotine the billionaires.

The USA could produce a lot more food than it does if rich idiots didn't own so much farmland running intentionally terrible farms as a tax dodge. Dutch wheat farmers get 3 times as many bushels per acre as american farmers, and half of all corn is now used to make biofuel instead of food. A lot of our food isn't even grown on farms (to the consternation of farmers who hate the competition). But we can afford it because we stole so much farmland from native americans we have may more than we can manage. Sarah Taber did nice thread on that.

Voter suppression in Georgia was even more obvious this time. And of course it's racist.

The Good Cop Democrats continue to be useless. In response to the police abolition movement, Joe "Good Cop" Biden proposes giving the police $300 million more dollars to win Boomer votes. How exactly is this a defensible position in 2020? Here's a thread on ways the Seattle PD's been lying to people. Meanwhile, police in Tulsa come out and say they should be shooting more black people, and the guy the GOP put in charge of elections in Indiana was convicted of felony voter fraud.

Politicians claim a lack of funds to help people, but endless funds to pay cops to beat people. Why does the LAPD have 4 times as much funding as the Los Angeles fire department? Here's an LAPD Lamborghini parked in a disabled spot. Meanwhile, California Highway Patrol just murdered an unarmed 22 year old at a traffic stop, firing over 40 rounds into his car killing him and wounding his pregnant girlfriend. Austin's police department is doing PR puff pieces to distract from the people they've shot. Police in minneapolis admit they're the source of vandalism. Here's a thread explaining how Seattle PD attempted to facilitate a mass murder of protesters. Of course after days of beating protestors, they took the shooter into custody gently because he's white and on their side. Yes, right after he rammed people with his car and then shot somebody.

Remember how the one use of police was as a sort of notary public recording that a crime had happened before filing an insurance claim? It turns out paypal ignores that. (Not only is Paypal terrible, but also racist and misogynist hypocrites. There's a reason I don't use them.)

Here's a thread where an upper middle class white male explains how all the interactions he's ever had with police were at _best_ useless. It's a good thread.

Most of mine were traffic stops, the one time our house was robbed the police didn't even fingerprint anything. (They said it wasn't worth it.) When my car window was smashed on a visit to baltimore (they stole a VHS boxed set of the Hitchhiker's Guide to the Galaxy from the passenger seat and the change from the ashtray) the police didn't bother to come. When my bicycle was stolen in Florida (at 23 I chased the guy who did it for 2 blocks shouting "death") I got a case report number and nothing ever happened. Both times my bike was stolen off the front porch here in Austin they took a report and I got a case number about which nothing ever happened. (The lock cut through with bolt cutters? No, they didn't fingerprint that. I've never seen them fingerprint anything in person.) Literally the only point the police ever had for me was a checkbox on the way to filing an insurance claim.

But they HOUND you for traffic tickets endlessly, they want their money and the price goes up as you fail to pay promptly. Multiple times I've been either unable to find my car insurnace card or didn't have the new one with me and of course that turned into a thing each time because they want not just money but performative obeisance. (They have a computer system that can VERIFY if the card I show them is real, but can't LOOK ME UP so I don't need to give them the card. Because failure to provide proof on demand is a revenue source for them.)

People are out in force because being nice to police doesn't work. The head of virginia's KKK drives his car into protestors and of course HE isn't beaten. No, he's on their side. ICE is attempting to create gas chambers with create gas chambers with plausible deniability. If you're black being an stronaut who's been on star trek doesn't save you from police brutality.

The guy who filmed Eric Garner's murder has finally been released from prison. The police continue to arrest people who film them today, for the crime of filming them.

Police arrested a protester for throwing a teargas cannister back at them (the charge is "assault with a deadly weapon", the fact they shot it at protesters FIRST doesn't register because they're magic and allowed to do anything). They also object to protesters reflecting their lights back at them with aluminum foil.

Possibly the truest quote so far is "The police would rather kill white people than stop killing black people." Meanwhile, republican gun nuts vow that if the police don't murder enough black people they'll do it themselves.

There are a whole bunch of closed cases of police murder that all need to be reopened.

Part of the reason to abolish the police is you'll never get the military weapons back from them otherwise.

The police are still profoundly outnumbered.

Police behavior is ludicrously petty and cliche.

The NYPD is a sea of white guys. No group has that few black people without internal racist exclusion.


June 10, 2020

On the energy front, BP just laid off 15% of its workforce, and refineries have the problem that each barrel refined produces a bunch of different outputs some of which aren't selling. (That's how refining works: you take a mix of stuff and seperate it into individual components.) The price has come back up, but the fossil fuel industry still has epic structural problems.

It's still annoying when the test suite spits out stuff like:

bzcat: I/O or other error, bailing out.  Possible reason follows.
bzcat: Broken pipe
	Input file = (stdin), output file = (stdout)
PASS: tar extract dir/file from tbz2 (autodetect)

And I have to work out "ok, I did 'make test_tar' which means it's using the host (devuan) bzcat which is complaining the output pipe closed before it had written all the data to stdout which (in this case) is NOT AN ERROR because tar had read two consecutive blank headers (all zeroes) which is enough to know that the tarball has ended, even though the gnu/gnu/gnu/stallman/stallman/uber/alles/dammit tar appends several kilobytes of superfluous null headers for no obvious reason, which is why my tar tests trim the output when comparing binary equivalence"...

I.E. "not a real error, just noise, and mine shouldn't do that". (And it's a race condition because the pipe between processes has 32 pages of buffer in kernel adding up to 128k that bzcat can write into before output blocks, and closing with data waiting in the buffer is not an error, so MOST of the time zcat writes all its output into the discarded buffer which is why I haven't seen this error before, but THIS time it wasn't that far ahead and Julian Seward's bzip code felt the need to INFORM me.)

I REALLY should have named this project dorodango.

That busybox thing for work is making me sad. (They want a shell with interactive command history editing, and I can't implement that in time on my shell for this release. So, slap it on top as a "scripts/root/busybox" mkroot plugin.) But for some reason miniconfig isn't working in busybox anymore (KCONFIG_ALLCONFIG is being opened and read, but the symbols in it aren't being set? It complains if they don't exist, but when they do the state change is ignored?) So I did an allnoconfig and switched on a list of symbols WITHOUT dependecy resolution (just using sed), but... there's a lot of them?

Rich built a defconfig busybox, but when I try that under mkroot the airlock step breaks the build because defconfig busybox wants all sorts of build time dependencies the smaller config I was building didn't need. So that's not an easy out, I want the result to be an actual portable build and not "I got it to work once on a magic system I set up by hand without documenting what I'd done". (Rich already BUILT one of those, I'm trying to make it reproducible and portable with strict criteria for both.)


June 9, 2020

The plutocratic racist violence is not specifically a republican thing. Yes the GOP is nazi to such an extreme that 95 year old Auschwitz survivors are protesting them (and, sadly, dying). They're packing the courts to give their fascist takeover the veneer of law, and if they're not defeated there won't be anything left to salvage. But replacing the GOP with democrats is not where this ends. The centrist assholes like Biden are themselves a problem. You don't have a revolution to "return to normal": the Boomers' normal is gone never to return (and good riddance).

But we originally got this racist plutocracy from the british. Did you know the War of 1812 started because the british outlawed the slave trade in the west indies, which stopped US ships delivering slaves to British Guiana. This resulted in the assassination of british prime minister Spencer Percival by liverpool slave traders, the only british prime minister ever to be assassinated. (I watched an excellent movie about this in 2006 called Amazing Grace, which is of course not streaming anywhere. But the thread those tweets are taken from is pretty good.)

And as outright traitorous as the GOP is, I'm sad that the democrats are merely "less bad". When Bill Clinton proudly "doubled the number of cops on the street" it was a BAD THING that directly set up the current debacle. (He finally admitted it was a mistake while Hillary was running for president yet again as the issue was hurting her campaign.) Obama authorized extrajudicial drone killings, kept guantanamo open, allowed warantless wiretaps to continue, and called HIMSELF more conservative than Richard Nixon. The democrats blacklist vendors who work with primary challengers and Nancy Pelosi sees AOC as more of a threat to her power than the republicans.

The democrats and republicans are playing "good cop, bad cop" on behalf of the plutocracy. It's carrot and stick, the Good Cop's job is to dangle a carrot in front of you, eternally just out of reach. That's why Good Cop is always intentionally incompetent, they can't QUITE help you yet but maybe if you do this thing for them, and then do the next thing for them, then they need ONE MORE thing before they can help, ooh so close they can ALMOST help now but just this one more thing they need and maybe THEN they can help... But the reality is Good Cop is still a cop, and each new case the cops flip a coin to see who plays Good Cop this time. Democrats are still servants of a 1% that wants the other 99% kept in their place, that's why they're primarying AOC. Nancy and Hillary and Biden are there to stop Bernie and Warren and The Squad from ever actually doing anything. The only way to fix the plutocracy is to Guillotine the Billionaires.

That's why I'm not sure the democrats can be "pushed to the left". Sure, in the short term the bad cops are BAD. Rand Paul is currently taking a stand in favor of lynching. But the democrats support the racist plutocratic status quo. Most crime is a social construct that's not illegal (or not punished) when rich white people do it, but "loitering" gets you jail time (or murdered) when you're black, native, hispanic... Most theft in the USA is wage theft which is structurally never punished. If you do enough of it, you become president. More cars are stolen by banks than by individuals, and individuals can't generally steal your house while banks do it in bulk.

There are plenty of real crimes, but local police tend to be of dubious utility at best addressing them. After each mass shooting it's "too soon" to do anything about gun control, but not too soon to pass laws to prevent gun manufacturers from being sued (just like the police have immunity from lawsuits). Being able to dial 911 so anonymous unaccountable cops maybe show up 20 minutes later isn't as useful as preventing that guy from getting a gun in the first place. The police aren't there to stop crime, they're there to protect the rich from the poor. The GOP needs to burn this year, that couldn't be more obvious, but when the Boomers die (50% of them should be dead by 2034) the democrats need to go down with them. (And then the corporations...)

(The Boomers are aware we're going to chisel their names off all the monuments the instant they're gone, and seem kinda pissed about it.)


June 8, 2020

Hands up everybody who foresaw microsoft trying "embrace, extend, extinguish" on Linux. I mean that one was just TOO obvious.

Toysh: I'm trying to come up with a test for $! and it's annoying. It's the PID of the last backgrounded process, and although I can 'true & echo $!' easily enough... the number it produces changes. (And isn't QUITE controllable because anything can spawn a process at any time consuming a PID, including kernel threads.)

I can do 'true & jobs -p; echo $!' to get the same number twice via two different mechanisms, but I don't have test infrastructure to say "I don't know what to expect but it should be the same thing twice". Maybe I need to add a new test type.

Hmmm. I guess "bash -c 'true & [ $(jobs -p) = $! ] && echo yes'" works?


June 7, 2020

The idea of police abolition is going mainstream, because ongoing police behavior is surprisingly extreme. Lots of people are making strong cases for it. Police "reform" always increases their budgets, they get money to do new things. Taking the money away is the obvious response, and zeroing out police budgets is the only way to solve the entire problem. (As the protests continue the non-monsters resign, but 6 resignations per day from a force of 36,000 would take 20 years to go through the lot.) The police themselves are so ashamed of their behavior they're literally arresting people for filming it. (I mean they're REALLY fragile. That last link was item #377 on an an ongoing thread collecting instances of police misconduct (and items 376 and 378 were also police assaulting people for the crime of recording police misconduct), meanwhile item #371 is Austin's police department sending itself thank you letters through the US mail for PR purposes. It's quite the thread.) Of course Boomers clutch their pearls at the idea of not having police, but when the NYPD went on strike crime dropped dramatically. And the police we've got now are white supremacists.

If you decriminalize marijuana, sex work, and panhandling, half of what police do goes away. Divert a chunk of the police budget to mental health services so schizophrenic people can get treated rather than arrested. When cars switch to self-driving taxi services the need for traffic cops goes away. Eliminiate private for-profit prisons and the school-to-prison pipeline. Other professions don't get to fight violence with violence (which works about as well as fighting fire with fire: real firefighters mostly use water and foam and such). Police in the UK don't carry guns because they don't NEED them, but police in the USA are gassing crowds (and individuals), driving around in tanks, causing permanent hearing damage with sonic cannons and intentionally blinding people and it's not enough.

Then there's domestic violence: police have always been utterly useless in rape and domestic violence/abuse cases (even intentionally destroying evidence), so eliminating them entirely would at worst cause no change there. And domestic violence is the #1 reason police are called, despite the fact that 40% of police are KNOWN to commit domestic violence, which is one of those things where you know more of them have done it and just not gotten filmed.

The cops feel disrespected (since the protests are specifically against police behavior), so the cops are the ones rioting (it's peaceful when they don't, all the violence is orchestrated by the police). But did you know the police have no legal duty to protect anyone? If you call 911 they don't legally have to show up. (I'm not sure they're legally allowed to stop ambulances from getting to people they don't like, but they're doing it anyway. The police are a protection racket.) Of course it's blatantly unconstitutional that we can't safely film the police, especially when you realize they've been systematically lying to us for decades which isn't exactly news. But then none of this is new it's just a few people are finally starting to notice. Cops are following suspected protestors to their homes, not because they couldn't arrest them on site or pull them over and arrest them on the way home, it's purely theatrical for the intimidation value. (And to intimidate neighbors in their communities into compliance and silence.)

Statistically, any group that's entirely white is racist, but that's sort of the point. Racism is plutocrats playing divide and conquer. (There was a whole book about that specific topic. The tools turned against foreigners and minories are inevitably turned against everyone else.) Of course the only reason the police are giving even the meagre ground they have is due to being drastically outnumbered. The divide and conquer only works if the 99% don't figure out what the 1% is doing.

There's a failure mode where what you're doing isn't working so you do MORE of it, in an endless escalating spiral. The Boomers seem particularly prone to this failure mode. The usa has less than 5% of the world's population and over 1/5 of the world's prisoners. We are spending money to create problems. The "land of the free" is Doing It Wrong.

Of course the Boomers have also done this with wealth concentration: the "Greatest Generation" left the top tax rate at 92% and the Boomers lowered it to 28% (and allowed billionaires and corporations to dodge even that, these days doing their best to destroy the IRS entirely).

Adjusted for inflation, the old 90% tax rate used to kick in after the first $10 million someone earned in a single year: you got to KEEP ten million dollars PER YEAR, which is an insane amount of money. Keep in mind just ONE million dollars per year is 40 years of living expenses. (The "fight for 15" campign is about raising the minimum wage, 52 weeks in a year and 40 hours in a week. Assuming 2 weeks/year of unpaid sick time, $15*50*40=$30000, of which you get to keep around $25k after income tax and social security tax. One million dollars divided by 25k is 40 years of minimum wage spending without even earning interest.) Ten million dollars would be FOUR HUNDRED years of spending at that rate, and under the old tax regime the Boomers' greed destroyed you could keep that EVERY YEAR. These days, "millionaire" is barely mentioned anymore, to the point John Hodgman uses his million dollars of book royalties as a joke. The amount of money actual rich people have these days is truly offensive.

That's why rich bastards always get bailed out. They buy changes to the law: "privilege" literally means private law. They're the new aristocracy. Hence all the guillotines. But who they buy them FROM is coming into question. The decline of the GOP continues apace. Health experts are kind of pulling the same thing the Godwin guy did, telling people that in the current circumstances spreading the virus is the lesser evil, so go ahead and protest (with your mask on). The white house has literally surrounded itself with secret police (including ICE) as the Resident's dementia spirals inwards.

You know the murdering minnesota cop that started the protests? He committed voter fraud in Florida in 2016 and 2018. The GOP is guilty of EVERYTHING they accuse their opponents of, to like cartoonish levels. But some people are finding their limits, and even real courage.

Dear media organizations: equip your people properly. (Better equipment won't stop the police from targeting media, but it can still help.) And protestors in the USA can learn a lot from Hong Kong. There have been lots of good instructional threads, and one-offs like how to remove zip ties, or using half a milk jug and a water bottle to snuff teargas cannisters.

Conventional news outlets have The Stupid, reporting this as this, engaging in truly epic understatement... and then there's the New York Times soliciting a piece from Tom Cotton. (Not "accepting", they approched _him_.) That's just extra-stupid. I have no idea why anybody still buys (or writes for) the New York Times.

There have been a bunch of great protest signs.

Much of the reported "vandalism" and "looting" has been destruction of racist monuments (when it isn't white people starting fires while black people beg them to stop).

The headline "Court OKs barring high IQs for cops" is not a typo.

Slack works with the police, and so does Zoom. Don't organize through either.

Police unions aren't actually unions, they're guilds like the AMA.

Remember when the administration was stealing Covid-19 protective equipment? Now the cops are doing it.

New York has suspended habeas corpus, and can now detain anyone indefinitely without reason. Not just protesters: anyone.

Police slam 70 year old man's head into the concrete, leaves him unconscious on the sidewalk in a pool of his own blood. Chris Hayes did a good piece about it.

Here an officer gropes a woman, she breaks away, and gets batoned for resisting being groped.

You know how we send legal observers to monitor foreign elections? The NYPD is arresting them.

Kettling, 10 years ago in the UK.


June 6, 2020

Every time somebody builds a DIY item in Animal Crossing, I keep waiting for the horns to kick in, because I am old. (It apparently started life as a snippet of the Hawaii Five-O soundtrack, which I had not previously known but once you start down these ratholes...)

Speaking of old, Google is finally shutting down the last dregs of Google+. Why they didn't just buy twitter, I couldn't tell you.

Google+ could have been a good thing, its failure was a self-own by Google. Their UI tried WAY too hard to be facebook and offered no obvious way to get persistent URLs to anything people posted (so linking to content on it was either impossible or broke before long), and rather than improve it Google forced it down people's throats via monopoly leverage (microsoft-style bundling) which had lots of problems but the biggest was creating horrible youtube comment spam problems out of whole cloth (believe it or not before Google+ broke them, youtube comments were NOT a cesspool). Google replied in full Elon Musk Silicon Valley White Man fashion (never admit a mistake, charge boldly forward endlessly doubling down) by trying to dox their entire userbase, and the whole thing just snowballed into epic suck.

Seriously, one of the X-Men comics back in 1985 had Loki create a Plot Device that gave non-mutants superpowers, but took away their creativity to fuel it, and when people noticed and Noped out he tried via Oblatory Fight Sequence to force it on them. Then the Alien Superbeings he was trying to impress with his generosity so they'd give him a cookie showed up and anviled a moral ala "if you'd gracefully taken the no we'd have called it good but you went full supervillain trying to get your way so Bzzzzt, fail".

Whoever was running Google 8 years ago did not read enough X-men comics growing up.


June 5, 2020

Just one link today, a video titled Court denies immunity after officer obtains 23 year false conviction.

It's about a police officer and sexual predator who spent his entire career threatening women into sleeping with him, and when one woman moved and changed her phone number instead, he framed her 17 year old son for murder. Now 25 years later the retired officer is claiming in court he can't be sued because it happened as part of his work as a police officer and police have "qualified immunity" from prosecution.

The video is from the channel "Lawful Masses" which describes and analyzes legal decisions. It's a dispassionate reading and review of court filings that has nothing to do with the current protests: the video is 2 months old, the case is from an unrelated jurisdiction (Kansas City), and this video isn't particularly noteworthy (it got 127k views on a channel with 134k subscrbers). It's just one more unremarkable story of the reality of policing.

But it's also a thorough, compelling description of what police do and who police are. Accident victims in the united states beg bystanders NOT to call an ambulance because it will ruin their lives. At least as many people will never call 911 no matter what because introducing police into a situation, even where their lives are already threatened, can only ever make things worse. And just as anyone can wind up uninsured, anyone can wind up on the wrong side of some random officer's grudge whom the rest of the department will rally around and support unconditionally, forever, every time.

(In this case, since he's retired rather than active duty, and the evidence of his guilt is overwhelming, his old department has merely issued a no comment and gone into full ass-covering mode.)


June 4, 2020

And I forgot to plug my laptop in again when I got home, and the battery died when I wandered away from it. Wheee.

I had 4 email windows I was composing, one was a reply to this which I kept open as a todo item, and one was a reply to a private email from Patrick Oppenlander with instructions on how to test blkdiscard. No idea what the other two were.

If you ever accidentally close a chrome tab you didn't mean to, CTRL-SHIFT-T is your friend. (As with "right click -> inspect element -> delete node", there are some things modern web browsers are a lot less usable without.) After last crash a nice person walked me through changing my thunderbird config so it would save email I was composing, and of course thunderbird didn't.


June 3, 2020

Hmmm, my impression Austin's been mostly peaceful seems premature. It looks like I probably just haven't been targeted due to being white. (And also staying home in quarrantine other than walking to a secluded table in an unused corner of the university after midnight. I feel guilty for not protesting, but I'm still trying to quarantine. Among other things, Fade hasn't had it yet.)

I walked to UT after midnight anyway. I haven't got much other opportunity for exercise, and let's face it: the racist cops are going to see a white guy.

Don't get me started with this "all lives matter" crap. A "wet floor" sign says there is a problem HERE SPECIFICALLY, the only reason to point to the rest of the floor is to draw attention AWAY from the actual problem. If you put up an actual "all floor equally dangerous" sign, there's something wrong with you.

I've stopped collecting links. If you don't know what's going on by now you don't want to know. Fade sent me an excellent summary of what really happened in minneapolis, genericize that to the rest of the country. The police are openly looting in uniform and the military is standing around waiting for the GOP to give the order to murder civilians. Then we see how many of them do. The system has a lower limit on age to hold office so we have to wait for good candidates, but the electoral college and 3/5 compromise let septuagenarians and octagenarians run everything until they die. Ok Boomer.

(No, I didn't collect links. That's what scrolled by while I was writing that paragraph. As the man said, "My the world's on fire, how 'bout yours?")


June 2, 2020

Rich built busybox for the customer release last month, but when I tried to reproduce his setup (as a mkroot build script) the resulting shell (busybox hush) intermittently segfaults. As in you run the same command twice and one of the times it segfaults, the other it doesn't. Smells like an uninitialized variable maybe?

Rather than try to debug an unfamiliar codebase (I haven't looked at the hush source in years) I upgraded from 1.24 to 1.31.1 (current release) in hopes they've fixed it (it wasn't segfaulting for Rich) and... the build breaks because "off_t size is misdetected"? Which the nearby comments say is a workaround busybox added for bionic, but is triggering on a musl build. (Sounds like any libc that hardwires on Large File Support without requiring you to -D_IT_IS_NO_LONGER_1996 on the compiler command line will hit this.)

It's after an #ifdef staircase that's trying to micromanage a manual uoff_t declaration, which is just sad. Toybox goes out of its way to render unto libc that which is libc's. Something is either toybox's problem, or NOT toybox's problem. "Do or do not, there is no spoon." Eh, there's more than one reason I don't do busybox anymore...


June 1, 2020

Oh great, that'll show him. Thanks Nancy and Chuck. Problem solved.

Is anybody surprised that this is how the trump administration turned out? They've been signaling this for years.

The resident has a profound zero sum mindset, and his dementia means he retreats into ways of thinking set in stone over decades because everything else has washed away. He did his money laundering in the real estate industry, where you can't gain a piece of land unless someone else loses it. In real estate, "I can't gain unless you lose" because there is a finite amount of land that's just changing hands. He extended this into "if you lost, I gained" (because even love canal and chernobyl drove up the price of the remaining land you can still live on). Add in dementia and this simplifies to whipping boy politics: hurting people advances his agenda. Given his racist base lives on schedenfreude, that's exactly what they want to see. They don't care about getting hurt themselves as long as THOSE PEOPLE suffer.

Racism has crippled the GOP's attempts to deal with Covid-19 because they cannot allow any aid to reach brown people, so instead they had to deny the existence of the problem with some pretty obvious lies. And now they're gleefully hoping it spreads among "those people", whoever that moving target refers to this week.

We're basically having our Tianamen square moment. If the fascists win, they will hunt down and murder everyone who stood up. They're way beyond any historical version of "normal". Back in 2016 Will Smith said "Racism isn't getting worse, it's getting filmed.". But now it's ALSO getting worse, hopefully coming to a head as it dies. Right now we're in the middle of a quite complicated venn diagram.

Let's start by restating that the inciting incident was an explicit murder (that's a thread from a medical examiner), caught on camera from multiple angles, and the guy who did it has killed before. No amount of blaming the victim changes that. The military rules of engagement call this sort of thing an execution, and arrests those suspected of it pending investigation. (In the military they're not put on leave, they're arrested for murder.)

I have no idea what's helpful to do at the moment. I think the reason Austin is peaceful (so far) is the APD instituted a strong body cam policy circa 2015, with any and all footage available via texas open records request, which had time to sink in and become cultural to the point "you had your body cam off" is ITSELF strong evidence of guilt. And thus the cops have NOT torn the city apart like so many others.

Here's a state by state bail fund but most of the ones in Texas are there to oppose ICE (which is still horrific, murdering people and kidnapping children even more with the spotlight off them). There's specific advice for dealing with tear gas. (Did you know using tear gas is a war crime, it's only "legal" to deploy against citizens in peacetime?)

At least there have been a few useful half measures (from people finally catching up with the group). Military weapons should never have been given to municipalities in the first place. It was a really stupid policy, a side effect of the federal budget having enough wasteful defense spending to create a literal disposal problem. John Oliver did an excellent piece on this and also on police immunity from lawsuits and one on police stealing people's money and stuff in bulk. (I say "half measures" because there are 40 million unemployed in the USA and about 700k police officers, going to 41 million unemployed is a lot less intrusive than what we're living through right now. If none of us can afford to call an ambulance, why would doing without police be more dangerous?)

Years ago David Graeber said the police exist to protect the rich from the poor, and have no other purpose. They no longer even pretend to serve the public. Mayors should order their police forces to stand down. (Here's the guy those particular cops murdered yesterday.) The police don't care about the law, they break the law with impunity and even obviously violate the constitution. I have no idea why municipalities still fund them. Zero out their budgets.

The police are the problem. All these protests are peaceful until the police show up. The police attack peaceful protests. They attack people in their homes, or just standing still. Less than three weeks ago police broke into an EMT's apartment in the middle of the night and murdered her in her bedroom. The protests are because police have been murdering people on a regular basis with impunity for years. They've made themselves immune from lawsuits, they can steal anything they want at any time with no recourse, and they're collecting military weapons in bulk. And they're happy to protect other racists murdering people. It's kind of a problem.

A related problem is right wing loons pretending to be protestors. White men with racist tatoos start the fires when people with cell phone cameras catch them and report them then sometimes there's enough public outcry the police are forced to do their nominal jobs, but they presumably quietly let them go as soon as they're out of sight of the cameras. After all, they're working together, and this sort of thing isn't out of character for undercover cops.

But once the police do (officially) show up, police actions are pure sadism. They tape over their badge numbers and shut down their body cameras and jam cell phone signals before they attack. Police violence is just another kind of gun violence. Adding the national guard didn't help. Adding in the military can't possibly help.

Here's a thread collecting examples of police attacking journalists. And here's a video montage. They HATE being filmed. But the journalists have way more courage than the police do, and everybody has a camera phone now. Remember the photographer who lost her eye? She's not remotely alone, and cops regularly kill people with this stuff. They're targeting the media, here's another long thread of police targeting journalists. Here's an officer shooting directly at a cameraman while being filmed by that camera.

It's pure racism, the cops don't chase white people caught in the act of looting. At best white looters get a talking to. But they arrest each black person annually, hold them without bail for a day, then release them without charge (and demand an impound fee to get their car back). During the protests, white people are putting black people in danger, and refusing to listen. White people are the problem, majority black cities are peaceful.

In Los Angeles, they're planting movie prop police cars and setting them on fire to discredit protestors. In baltimore, cops stole drugs from pharmacies and sold them to drug dealers. (In addition to be reponsible for most merchandise theft in the USA, the police also commit 1 in 16 homicides.)

And or course the GOP are blaming the left, and doubling down on racism (and antisemitism), with "look what you made me do" speeches while quoting famous racists to justify shooting US citizens. The fascists remain a tiny minority mangified by bots and sock puppets (and of course Russia's been doing this for decades, but the billionaires have learned, and created a hermetically sealed propaganda network).

Biden is useless. The best thing you can say about that milquetoast equivocator is he's not actively making it worse. He looks stellar in comparison to screamy orange racist grandpa but so would a potted plant, which ALSO does damage more slowly. This entire article boils down to "the idiot really really really wanted Amy Klobuchar (who doesn't care who he rapedhow stupid an idea that was, the octagenarian can't cope with change".

The history books of this period may be interesting (or like tianamen square, may be erased from history altogether). We need to zero out police budgets everywhere.

By the way, the reason people were protesting outside Target is they donated $300k to the police to install surveilance cameras throughout minneapolis, and the first store attacked was a pilot for more intrusive surveilance.

Everything is a distraction from everything else. distraction from everything else. Guillotine the billionaires already. That's where the real looting is, each billionaire has stolen more money than this entire national uprising combined.

And we REALLY need to start teaching the humanities again, because our history of racism goes back at least 500 years. (All the other ex-british colonies with right wing governments are still being racist too, it's just come to a head here.)

The onion is continuing to merely document reality, and is not alone.

The biblical "log in thine own eye" applies to all the idiots looking for conspiracies when the "conservatives" are right there acting openly.


May 31, 2020

So the guy who triggered my first quarrantine stress meltdown back in april has sent more than 2 dozen seperate pull requests via github this week, most of which are stacks of multiple commits. He's well intentioned, and he can occasionally find real issues, but the net result is I've spent more energy saying no to his patches than getting work done on toybox for days now.

In theory this is a series of "portability" fixes for toybox, half of which seem like pointless churn and most of the rest (in my judgement) make the code worse. This signal to noise ratio in this series is incredibly low, but it's also clear he doesn't understand _why_, and there are some interesting lessons here. These pull requests completely misunderstand what I want, and maybe this is my fault. Perhaps I need to update the design page somehow to explain better?

My first question with this sort of thing is "portability _to_what_"? Nitpicky compliace with an abstract standard is "infrastructure in search of a user", especially if you can't regression test it. If the code's been like that for years and nobody noticed, what did you hit that broke so I can add a test case? In the case of "build fixes", what is the BENEFIT of doing that? Where specifically can you build/run now that you couldn't before? Keep in mind, this sort of cleanup work can always be done later when a user/need presents itself. This lazy binding, just-in-time approach to code changes avoids bit-rot.

(Bit-rot is a bit like the Pentagon hanging on to spare uniforms in a style it no longer uses. It bought in bulk to drive down costs and stored them well, why should it throw them out? The uniforms didn't change, the world around them did, and now they're no longer useful. If you ask "what about the uniform changed to break it" you're asking the wrong question, the answer is "its context".)

Some of these pull requests add unnecessary layers of indirection (that link goes to a comment I wrote explaining why that's bad), others break the semantics and change the callers to use the broken semantics while leaving a trap lying around for future callers (that's a comment explaining why THAT's bad). Others enlarge the code without making it any faster, smaller, simpler, or more readable: this one does so at every callsite, and this hides the expansion in a macro. Sometimes I'd already explained in the code why I was doing what the patch undoes.

This pull request changes the kconfig code I inherited from linux-kernel which I already announced plans to completely replace. The patch adds gratuitous changes vs upstream which make the code's continued use harder to understand (larger delta from what we inherited) and less defensible. "Long ago we took a snapshot of this infrastructure and haven't gotten around to replacing it yet" is a coherent story. "We keep modifying this infrastructure, ours looks nothing like theirs anymore, but it's just an old snapshot we don't actually maintain" is not. Back when I did that kconfig stuff for busybox I posted my changes to linux-kernel, and they didn't want them. If anybody sues over GPLv2 code in the toybox tree (which they shouldn't since the resulting binaries don't SHIP, it just makes turns one kind of text file into another kind of text file during the build), my first defense is "look, changes posted here, archived in the public linux-kernel web history, your fault for not wanting them". I still do that (and am surprised when anybody notices). These new changes would NOT be posted to linux-kernel (they're not relevant because their version's had a dozen years of change ours hasn't), which makes a very different statement. I am not maintaining a FORK of gplv2 code, it's grandfathered in and removing it entirely is on the todo list (and right there in the README). If the build started throwing warnings I might patch them out because the toybox build should not generate warnings, and if you come up with an actual target it causes a build break on I'd grumble but might merge changes for a specific testable reason. But it doesn't get aesthetic or speculative cleanups because it's under the wrong license and is very much deprecated, intended to be replaced.

Some of the patch series replace functions with equivalent functions which are actually less portable. (Just because Jorg Schilling suckered Posix into removing tar in favor of pax doesn't mean any linux system anywhere has pax or doesn't have tar 20 years later, and similarly deprecating/removing cpio didn't stop RPM and initramfs from basing their file formats on "cpio -h newc", and there's even talk of extending cpio for sparse files, xattrs, and 64 bit timestamps).

Aside: microseconds suck, please stop using them. Milliseconds measure human perception/reaction times, nanoseconds measure computer reaction times, microseconds measure NEITHER and must be converted into something else to ever be useful. Just as C++ is in the no-mans-land between C (explicitly describing what the hardware needs to do) and scripting languages (abstracting away lots of implementation details), microseconds are in the no-mans-land between nanoseconds and milliseconds. Figure out whether you want human time or computer time and use the appropriate unit, which is never microseconds. I've ranted about this before.

Don't use PRIu32 and friends when you can avoid it. Toybox uses LP64, which means char is 8 bits, short is 16 bits, int is 32 bits, long long is 64 bits, and long is the same size as pointers (32 bits on 32 bit systems and 64 bits on 64 bit systems). The basic integer types HAVE DEFINED SIZES, and this standard is used by Linux, BSD, MacOS X, iOS, and Android. The only thing that DOESN'T use LP64 is 64-bit windows (on 32 bit windows long and pointer are the same size), and modern windows can literally run Linux binaries. I'm quite happy to break compatibility with the win64 API, we have NEVER supported it and won't.

I'm on the fence about ending structs with array[0] vs array[], because the first has been accepted by compilers going back to the 1980's (really, Turbo C under DOS let you do that), but the standards committee decided the second (which used to be a syntax error) is what they were going to standardize. The first uses conventional K&R syntax from the 1970's to request zero bytes of memory be allocated (sizeof(*array)*0 is 0, the compiler is not confused about what to do), and then calculate how much extra space you want when you malloc() the struct and access it yourself. Then the "add undefined behavior to break existing code" brigade went "but bounds checking of arrays!" So they added a NEW syntax they had to change compilers to NOT treat as an error in order to mean the EXACT SAME THING as what people had already been doing for years, all so you STILL couldn't do bounds checking on these cases because the compiler still doesn't know how long array[] is either. If you instead taught your bounds checker "a zero length array doesn't get bounds checked" everything would have been fixed without having to change existing code.

Easy rule of thumb: assume C coders know what they're doing, and that C++ coders do not. C++ coders have already chosen to use C++ and C coders to use C, and that's a GREAT BIG SIGNAL right up front as to who is making good decisions here.

Standards bodies should DOCUMENT, not LEGISLATE, because every time a standards body tells people to change existing code IT IS A BAD THING. Implement compiler extensions, deprecate them if they turn out to be a bad idea, if they become widely used and adopted by multiple compilers (the way llvm picked up using "x ? : y" as a synonym for "x ? x : y" that doesn't evalute the first argument twice, which gcc's allowed for a couple decades now) THEN the standards body can go "ok, four different shells now support set -o pipefail, write it into the spec already".

But anyway, array[] exists now and has since c99 and I'm fine with using it, and the reason I'm on the fence is it IS one fewer character. If I was writing it in the first place and somebody looking over my shoulder pointed it out, sure I'd change it right then and there. But a patch to change array[0] to array[] is churn, and is it worth the churn? It's a diff that makes noise but doesn't really accomplish anything. As part of a larger change, sure. If you have a cleanup pass moving stuff out of pending, fine. But on its own? If the patch doesn't justify its own existence, having multiple separate commits to do the same thing is even less justified.

Other pull requests ignore allocation context, such as the way setenv() inserts the provided memory directly into the environment array, while putenv() creates a malloc() copy to insert into the array (and thus leaks it if the variable's value is replaced later). I wrote an entire file of infrastructure to handle environment variable allocation lifetimes, and have blogged about the issue repeatedly for years.

Another pull request replaces setlinebuf(stdout) with the equivalent setvbuf(stdout, 0, _IOLBF, 0) and which of those two makes it immediately clear what the function does, and which has a macro starting with an underscore for no immediately obvious reason? And of course the pull request is a stack of 4 seperate commits doing it in 4 places. (If you add .patch to the end of github pull request URLs you get a git format-patch style mbox file you can apply with "git am", and I sometimes edit those patches by hand in vi, to do things like squash 4 commits together into a single commit. In this case I would probably take a patch going the OTHER WAY and consider it a code cleanup because the 4 argument version is larger per call, with more cognitive overhead to understand even WITHOUT having to look up what a "vbuf" is or what _IOLBF does.)

Here's one that just straight-up ignores the stated goal of the toybox project as explained in the about and design pages that toybox is a Linux project building and running on Linux and implementing Linux commands with reference to the Linux man pages and the Linux standard base. Instead he says:

On linux, utmp isn't obsolete, and utmpx is the same as it, but on pretty much any other system utmp is obsolete and utmpx should be used instead.

See that part at the start, "On linux, utmp isn't obsolete" and then add a period. Full stop. "Should be used" is editorializing, this guy saying "what I want is more important than what you want, because I define 'should' and you, the project maintainer, do not". How nice. (This is AFTER the blow-up in april back when I cared WAY more about what he thinks and let my repeated objections being ignored and walked over affect my enthusiasm for the project. At least it was a learning experience for one of us.)

I can almost see the point of not wanting to use void pointer arithmetic and mostly I don't, but I avoid it by changing the type of the variable, NOT by inserting gratuitous typecasts (which are themselves a source of bugs, and which make the code larger and harder to read and reason about). In the called out cases (again ignoring pending which needs systematic analysis and restructuring, not fixing an apostraphe in the Eye of Argon), the types are void for a reason, and you'd insert a new variable just to copy the argument into, which I've done in plenty of places, but in context I'm just so exhausted by the rest of the pull requests I don't want to explain how to change it so that I might accept something that isn't worth doing. I'm not really bothered by it being in the code. Yes a few leaked in, if a compiler shows up that doesn't support it I assume it'll throw an error. If it wasn't for the other 25 pull requests maybe I'd care more, but in context it seems LESS worth doing.

But to be fair, let's end with an almost sort of worthwhile pull request. As with the zero size arrays this patch might be worth doing as one big cleanup pass for trivial issues. As with array[0] vs *array, you can say array[] is slightly fewer characters and toybox has a policy of consistently using the shorter way of expressing something when there are two equivalent ways. (It's under Coding Style on the design page. It's not worth multiple commits to do it, one cleanup pass changing all the instances is more or less a "fix typos in comments" level of "eh, doesn't hurt anything".)

But he's not motivated by that, he's submitted pull requests going the opposite way because he wants to eliminate all estensions for no obvious reason. This is a Linux program, it's not gonna build on windows. I actively don't want it to. Cope.

He's still at it, by the way. he found two tiny legitimate-ish changes that I would actually make... in code in pending that needs massive cleanup anyway. This is like reporting capitalization mistakes in the first draft of a novel: whole paragraphs are going away, these changes are not worth making at this time in this context. This isn't worth one commit, let alone two. (Note that earlier when he found a usleep->msleep change what I wound up doing was a big cleanup pass on the file, which isn't even ALL the cleanup I need to do there but I sat down and did enough work to bother checking in.)

Anyway, that's why I didn't get anything done on the shell again today. I was being a good maintainer, responding to user submissions, looking at and explaining why these changes should NOT go into the codebase. And then I was out of energy and went off to do other things. I've written it up here in hopes that at least SOME good will come of it.

P.S. I tend to think of code in context. For example, I'm implementing job control in the shell and I'm using "int" for the "plus" and "minus" indexes (I.E. the two most recently active jobs in the job array, which get a + and a - after them in "jobs" and can be referred to as "%+" and "%-".) While I _COULD_ use unsigned for this, in order for that int to wrap you'd need 2 billion simultaneously active jobs, and the Linux Process ID table doesn't go up that big (and ain't gonna). In linux the type of pid_t is also "int" (signed!) and the default value for /proc/sys/kernel/pid_max is 32768.

But it's not even a question of future proofing: ignoring kernel internal scalability, and the wattage to run millions of processors in parallel, just the MEMORY is infeasible. In order to have 2 billion simultaneous jobs you'd need at least that many processes (not threads), and if each has an 8k stack and dirties NO other pages (static XIP binary with no PLT or GOT fixups, on a nommu systems so a process doesn't have page tables), that's still 13 terabytes of RAM just for the stacks.

"Oh but future machines..." No. Moore's Law is basically over (because all S-curves bend down eventually and this one ran for 50 years). The 18 month doubling time of memory no longer applies, the rate of increase in computer capacity is decreasing as the curve bends down. A 2010 macbook came with 2 gigs of RAM, a 2015 macbook came with 8 gigs (only 2 doublings in 5 years, when 4 1/2 years is 3 eighteen month periods) and the default memory for a new macbook 5 years later is 16 gigs (only ONE doubling in 5 years, it's slowing fast). Sixteen terabytes (which still isn't enough to DO anything with 2 billion processes, if each was "hello world" it's still bigger than our example) is 10 more doublings from there, so is that 50 years or "never"? (It's going to keep slowing, that's what S-curves do, and the laptop->phone switch adds another 2 doublings.)

But ignoring ALL THAT, the fix would just be "if (jobcount==INT_MAX) error("cannot background");". None of these patches tried to make that particular change, but it's a design decision I just had to make for new code today, and explaining WHY is a bit of a lecture, innit? Either you understand this sort of thing, or you need a huge infodump just to have a frame of reference to make a decision in. It's a conundrum. I don't want to discourage people, but it's a giant timesink and the most dangerous part is when you THINK you know what you're doing and project confidence. Me, I'm here spending more time working out regression tests than writing code!


May 30, 2020

Fade's back in Austin (having always been scheduled to return at the end of the semester) and she's been posting a lot of links about what's been happening in her city despite not usually being all that political on her twitter.

I mentioned on the 16th that Harvard is the rape cover-up place.

Wear safety goggles while protesting because cops aim rubber bullets for people's eyes to cause permanent injury (a technique they picked up from chile and warning about that last link: close up photos of the results).

Here's a long detailed thread showing how the looting and fires are caused by white racists posing as protesters, while actual protestors beg them to stop. Dozens of examples, video from multiple cities. Some undercover cops, some Klan. They're even trying to organize fake protests to get real protestors to show up as cover while they play-act "the purge".

It's not just klan and undercover cops committing atrocities. On-duty in-uniform officers are doing it to. ran his car into a crowd of protesters, and here's a mounted officer running a woman down from behind with his horse. They're doing this openly, and explicitly targeting journalists.

Guillotine the billionaires. I know we're all waiting the worst of them to die of old age, but I'm not sure there's time.

The modern economy is still completely fictitious. Money is a way of keeping score. The government prints it at will, and running out of money is like running out of laws. It's an IOU the government issues and will accept back in payment of property taxes (which are unavoidable unless you're homeless; your landlord pays even if you don't). The united states' highest period of prosperity is the same as when its debt was highest: under the Eisenhower administration just after World War II. Most Government debt is borrowing money from itself, which is a way of printing money without acknowleding you're doing it (because it's perpetually "temporary").

The only downside of printing too much money is causing inflation, which is good for debtors (our debts decrease), and the easy way to reduce inflation is to either tax the rich to get the extra money out of circulation, or increase the supply of stuff to buy with the money. But inflation hasn't been a problem for the US economy for 40 years. Instead the united states economy has the opposite problem, it's stuck in a perpetual demand limited liquidity crisis, even over ten trillion dollars of "quantitiative neasing" in the years after 2008 couldn't get the inflation rate UP to 2%, it's stuck below that.

Having the _opposite_ of inflation means we can easily afford basic income, and politically all you have to do to implement it is pass a one page bill setting the social security elligibility age to 0 and granting everyone at least the "worked 30 years" minimum benefit of just under $900/month. And if the republicans could figure out a way to do that ONLY for white people without letting brown people benefit, they'd have already done it. (The appeal of the Resident has always been the idea that he's hurting "those people" at least as much as he's hurting his supporters. Every article about ICE stealing a baby buys him an indulgence from his base for whatever bad thing happens to them. They live on racist schedenfreude. They also have a zero sum mindset and the Resident even more so: they don't win unless somebody else loses, and whenever they hurt somebody else they MUST have benefitted even if they can't immediately see how.)

The capitalism of the united states has always been racist. Slavery was for-profit. At the time of the first civil war, the dollar value of the slaves was greater than the dollar value of all the real estate (land and buildings combined) of the South. The southerners were fighting over MONEY. And the billionaires today are responsible for (and steering) this mess.


May 29, 2020

Um, doesn't "all OSI approved licenses" include 0BSD, a public domain equivalent license? I guess what they're saying is if you _relicense_ to a proprietary license, then they can sue you.

Darn it, I was building mkroot and this happened:

Compile toybox....scripts/make.sh: line 34: echo: write error: Resource temporarily unavailable
...scripts/make.sh: line 34: echo: write error: Resource temporarily unavailable
............scripts/make.sh: line 34: echo: write error: Resource temporarily unavailable

Line 34 is:

[ ! -z "$V" ] && echo "$@" || echo -n "$DOTPROG"

It's saying stdout to the PIPE is temporarily unavailable. And isn't this the bash built-in echo doing this? Is... wait.

There's nothing in dmesg about this. To the mailing list!.

(Half the reason I say "I break everything" is when something like this happens I STOP and DEAL WITH IT rather than letting it go. For the same reason you don't let a roach escape under the fridge and go "I'm sure it's fine", you call the Letal Chemicals Man to show up in his ice cream truck playing helter skelter and perpetrate chemicals everywhere. Never let a bug go is similar to never let a crisis go to waste. Other people are better at deadlines and not getting distracted, but I'm trying to do science here).


May 28, 2020

Fade is scheduled to fly back to texas tomorrow, and isn't sure she can make it because minneapolis is... unwell.

The riots in minnesota are because people are (peacefully) protesting racism in the police force, so the racist cops are pushing things into riots to discredit the protestors.

The first store vandalized was caught on phone camera (people asking the white guy not to smash the store window and him threatening them with violence, then he ran off towards the police station). He has now been identified as a St. Paul police officer.

Of course none of this is making national news, I'm hearing about it from Fade who's now definitely flying back to Texas tomorrow because a friend gave her a ride to a hotel near the airport, where she's spending the night.

None of this is new, by the way. An interesting contributing factor to LBJ's decision to sign the civil rights act (and thus hand the "solid south" to the GOP, setting off the Southern Strategy) is that his research team figuring out why the detroit riots happened said it was racist cops.

Oh hey, after midnight the idiot state police arrested a CNN camera crew during a live report, and refused to say why. Ok, NOW it's getting some coverage. This is complete third world stuff.

And of course, the onion is on top of it.

Really, their whole main page right now.

Ooh, here's a practical thread on how to get the klan out of police forces. (See also.)

Did you know the police now steal more stuff (through civil asset forfeiture) than the official criminals do? The police have become the problem. (Of course most of the rest of the theft is the rich stealing from poor.)


May 27, 2020

As with Toys-backwards-R-us and Sears, Herz didn't go bankrupt because of the economy, it went bankrupt due to rich bastards stealing its money and loading it up with debt. Under the current administration, billionaires are literally buying off legal enforcement. The only reason Bain Capital (destroyer of Toys R Us) is getting dinged is because Mitt Romney is out of favor with the administration. Scratch the surface of any conservative action and you find a get-rich-quick scheme. That's why "guillotine the billionaires" is the central political idea that makes sense.

Of coure the billionaires know it, and have been expecting it. (They're surprised the rest of us have put up with it this long, they know exactly what kind of monsters they are. Rachel Maddow did an excellent piece on this back in 2010, the video of which has vanished from the the msnbc site from the but at least there's still a transcript).

That's why the billions in defense spending become domestic hardware to keep the populace in line. (Democrats keep falling for this. Clinton doubled police on the streets, it was a bad thing. They keep handing the GOP more stazi to keep the brown people down, and as always the order goes "done to foreigners, done to brown people, done to poor people, done to everybody".)

Meanwhile, the voting system is still extensively rigged.


May 26, 2020

Pipes need to live outside block type evaluation:

$ echo hello | potato() { echo abc; } | echo ha
ha

But enabled/disabled segment evaluation (blk->run) is relative to block type. Blocks 0 and 1 skip if disabled, type 2 changes blk->run, and type 3 loops back when blk->run && "done". Not sure how the layering should work here...

Meanwhile, there's a syntax error checking miss: sh -c 'if true; then echo hello | fi' should complain "unexpected fi". (Ok, fixed it but I had THREE ongoing forks of the same file while I was doing that, and it makes me really nervous.

I wrote a long email message I can't send yet because Google mail has decided that any attempt to access its pop3/smtp servers via t-mobile phone tether is inexpicably invalid somehow (with the misleading "how dare you not use the web UI lock-in for everything" message), but it'll still work via Google fiber because that's their service. The Google proprietary lock-in is getting Google annoying. I'll have to send it from home later. (Yes, I'm still social distancing, the midnight walk to abandoned parts of UT remains very useful.)

The type 0 (simple command) and type1 (start of block) logic both handle pipes and redirection, and I'd _like_ to merge that handling but there's a lot of shufflilng. The combined logic would look a bit like:

end = pl->type ? block_end(pl->next) : pl;
ctl = end->arg->v[end->arg->c];
// Skip disabled block
if (blk && !blk->run) {
  if ((pl = end)) pl = pl->next;
  continue;
}

if (pipe_segments(ctl, pipes, &urd)) break;

...

// Parse and run next command, saving resulting process
dlist_add_nomalloc((void *)&pplist, (void *)run_command(arg));

// If we didn't pipe output to next command's input, wait for exit
if (*pipes == -1) {
  if (ctl && strcmp(ctl, "&")) {

Speaking of which I still have a todo item to change how dlist works, but the churn's a bit unpleasant...


May 25, 2020

Did you know that the forward lean the Resident does _is_ one of the diagnostic factors for progressive supranuclear palsy, going back a hundred and thirty years? (Of course back then it was called Atypical Parkinson Syndrome, as in "a variant of Parkinson's disease that's also Althzehimer's".)

Conservatism isn't conservative, it's just fascist, racist, misogynistic, and plutocratic. But then conservatism boils down to the fundamental idea that some people are destined to own slaves, and others destined to be slaves. (And that's why they think hoarding food during a famine is right and proper, because their right to property trumps anyone else's right to keep living, or else what's the law for? Taxing billionaries to fund food stamp programs is against the natural order of things to conservatives. Peasants live or die at the king's pleasure.) This is of course why the guillotine was invented, because you'll never change their minds any other way.

This is why "conservatives" are all-in on voter suppression. Not only is it their only remaining way to stay in power, but their fundamental world view says "those people should not vote, we're special and they're not, we must rule or the world is wrong". They fundamentally do not believe "all men are created equal", and thus cannot believe in democracy. They KNOW they're a tiny minority, and think that's what makes them special. There have always been far more plebians than patricians, more peasants than nobility, more slaves than plantation owners.

Of course they edit the historical record wherever possible. Because what they say and do looks stupid in retrospect. For example, when Bill Clinton balanced the federal budget and maintained a surplus, his successor George W. Bush's first budget slashed taxes on the wealthy with the rationale (and I quote): "You see, the growing surplus exists because taxes are too high and government is charging more than it needs. The people of America have been overcharged and on their behalf, I'm here asking for a refund." This quickly led to the biggest deficit in history (at the time). He tried to turn paying down debt into a moral evil, because billionaires were being taxed.

It's also why the GOP has just eliminated accurate food labeling in the USA. Poisoning "those people" is fine, the rich can afford the good stuff without metal shards and downer cows in their caviar.

And it's why the GOP started agitating to break quarrantine the day after the new york times and washington post announced that black people were disproportionately hit by the disase. (Because poor people tend to love in food deserts where being malnourished looks like "I need to eat 3x my daily calorie needs to get enough nutrients", haven't got spare time/energy to exercise, can't afford to stay home from work, accumulate health conditions they can't afford to treat before they turn chronic...)

(Oh, did I mention the hospital is pestering Fuzzy for money because she thought she had either a kidney stone or a kidney infection and the second can be life threatening, and the local "charge you $200 up front" clinic's test was inconclusive so she broke down and went to the hospital and they did an MRI and found nothing, sent her home without treatment, and the hospital charged her $11,000 for it? She filled out the "I have no money" forms, they've asked her for more information 3 times already, and meanwhile send ever-increasing bills they're about to send to collections because the low-income paperwork does not pause the billing process. She spends hours on the phone each year confirming she still doesn't qualify for obamacare because Texas turned down the medicaid expansion money.)

Conservatives like stupid leaders. The 4 of the GOP's last 5 presidents were the current dementia patient, George W. "is our children learning" Bush, Ronald "I don't remember iran contra and will die soon of altzheimer's" Reagan, and Richard "not a crook" Nixon. Only Dubyah's father wasn't stupid enough to get a second term. He completely avoided the domestic economy to the point Dave Barry faux-quoted him "The united, whadayacallum, states. Barbara and I have a summer home there." but he showed competence on foreign policy (under Nixon he was head of the CIA), and that made him unpopular with his base.

Oh hey, religion's found a new way to kill people. I mean seriously, they're a disproportionately large source of Covid-19 deaths. (P.S. A common communion wine vessel the entire congregation drinks from was probably _never_ a good idea.)

It's not that the Boomers have all the money, it's that they have all the LAND. The rest of us are still waiting for them to die. Eat the rich and end the boomers.

Did you know that countries with functioning governments have the press hold press conferences and invite politicans to address them? What a concept.


May 24, 2020

Rich is using out of tree kernel headers in musl-cross-make. I'd really REALLY like him to reread the Linux From Scratch's project's attempt to put together a FAQ about Mazur's old out of tree kernel headers package from 2004, and then NOT make that mistake again please. (I am quoted multiple times in that page.)

Tomorrow is two weeks from the toybox 0.8.3 release and Wikipedia[citation needed] hasn't noticed yet. I suppose I should do some sort of mailing list posting on project progress since that section still says "in 2015 it had made it halfway to its goals" and I'm going "um, yeah, that was 5 years ago?" (Ok, wrote a thing about it on the list.)

Ooh, found out that zsh has a shell test suite. I should see if I can cherry pick any useful tests from there.

Ok, where did I leave off implementing job control: with a note that the backgrounding decision needs to be made before the enable/disable decision for a block. Which is wrong, because a disabled block shouldn't spawn a process, there's nothing for it to do. If the process on the left side of the pipe is enabled and running, then the process on the right side has to be, all the enable/disable stuff is either && || gates (which are "wait for left side to finish" not the parallel execution of piles or &) or else it's running a test, and even "if false; then blah; fi" runs false. I suppose "for i in $EMPTY do echo $i; done" expands to an empty variable list and runs nothing, but it still runs a statement to do it. The for is enabled and the list expanded, which can have ${SIDE_EFFECTS:=42}. Even "cat | if false; then blah; fi &" runs the "false", I.E. executes the if test.

So you launch the _entire_ pipeline after the enable/disable for that block level. Hmmm, the thing is, pipelines are a bunch of parallel statements that get executed simultaneously, and the "background" this decision is at the end of the pipeline, which is fine as long as none of the statements _in_ the pipeline block (ala "read i"), which is probably why bash makes every pipeline statement a sub-process even for builtins.


May 23, 2020

QED.

I thought we knew this, but we're learning it all over again.

Twitter remains a hellsite.

The GOP are nazis. This is not hyperbole.

Guillotine the billionaires.


May 22, 2020

Sigh, I have a large diff in toybox/toybox/toys/pending/sh.c implementing wildcard support, and another large diff in toybox/clean/toys/pending/sh.c trying to fix backgrounding& and (subshells) which I thought would be a "quick fix" and turns out to be "because job control isn't implemented and the stubs I put in don't work". (And now I may need a second "clean" directory to deal with the reports about the echo \0 fix being insufficient? Right now "make tests" fails with a build break in sh.)

Ok, circling back around to this: pipelines are trees, not lists. The commands nest, ala:

if true; then echo hello; zcat blah.gz & fi | ssh me@elsewhere thingy

in which case the output of both the echo and the zcat go to ssh, in sequence. And in fact I can add a simple-ish test for that:

{ zcat file.gz & } | wc -c

or even

if true; do { sleep .25; zcat file.gz; } & done | wc -c

And just make sure it gets all the bytes, I.E. the "wc" waits for the backgrounded process to finish. (I tried first with md5sum and sha1sum but they insist on crapping " -" on the end when reading from stdin, with not one but TWO spaces, and I need awk or similar to peel it off and I just dowanna. The toybox -b outputs JUST the hash, which you'd think would be a thing in the existing tools, but no. In the gnu/dammit version "sha1sum -b" is "binary", as in "don't translate dos newline/linefeed pairs", which should not be a thing on the Linux build in 2020 regardless of backwards compatibility, it's like having an ebcdic translation switch in the dos build in 1995).

Anyway, the question is what does the shell WAIT for. What signals "don't advance until command completion". Because an & attached to an enclosing block makes the entire block a subshell, so each block has to check for backgrounding at the same time it checks for redirects. Meanwhile, each block level has to have its own pipeline list, because blocks can be chained into pipelines the same way individual commands can.

So block doesn't have an urd, it has a pplist... ok, the names are problematic.

A "struct blockstack" contains the information about the current set of active blocks we've nested down into: if statements and for loops and curly brackets and so on.

A "struct sh_pipeline" should probably be called sh_segment, because it's a pipeline segment, I.E. a command between semicolons or && or | or newlines. (Maybe it's just sh_command?)

A "struct sh_process" is information about a running process with a pid and an expanded argument list, unredirect() file descriptor cleanup list, and list of memory allocations to *delete (mostly from expanding the argument list).

So run_function() starts with a linked list of sh_pipeline, produces a stack of blockstack as it goes along, and calls run_command() every time it has an individual command to run, which returns an sh_process that gets appended to pplist so we can wait for it. (Builtins commands return an sh_process whose -> pid is 0 because it never spawned a child process, just ran it internally as a function call on the assumption they return basically immediately, which is not true for "read".)

Ok, a block with & after it is logically a (subshell), it's just backgrounded and added to the jobs list instead of waiting for it. A (block) in parenthes is a subshell because its got its own PID and local variable definitions and such don't affect the parent context, and any command with | after it's _also_ a subshell, or at least bash is treating it that way.

Except it's not just commands, blocks with | or & after them become subshells too. I need to unify two levels of plumbing...


May 21, 2020

The only way to not take Covid-19 seriously after January 23 is to be so racist you don't think you can get the same virus as foreigners. (Ha ha, the country that manufactures all our stuff has placed millions of people under quarrantine, look at all those ignorant savages who build our iphones for us, what can THEY know with their 3d printers and passenger drones and pervasive facial recognition cameras?) I was in japan at the time where we all knew it WOULD go global, the only question was the timeline.

Interesting thread on the dynamics of fascism.

Guillotine the billionaires.

Why giving money to the New York Times is a bad idea, part 37,602. I mean seriously, they're terrible. The "grey lady" is very grey, in both the "views differ on shape of planet" and "way too old" senses.

The oil companies blame everyone else for global warming.

Of course half the twitter accounts lobbying to break quarrantine are bots. Is anyone really surprised by this?

The governor who stole the Georgia election from Stacey Abrams is now stealing a state supreme court seat.

The Boomers are not dying fast enough, nor are the even older geezers the Boomers vote for. The Boomers are trying hard to take the USA down with them.

87% of the manufacturing job losses in the decade after y2k were due to automation. (Which still raises the question why capitalism decided to outsource manufacturing to china and service to india.) What remains of the US economy is completely fictitious.

Good thread describing how the GOP imploded into the modern "know nothing party". (Yes it was really called that, yes it was right before the last civil war.)


May 20, 2020

Back at the table at UT but... I'm not sure I'm staying long? It's SWELTERING and the mosquitoes are out. (Not all THAT hot, but really, really humid. I miss my air conditioned corner at Wendy's.)

I've been doing a thread on the toybox list because I'd hoped to get Elliott to comment, but he didn't. The main new observation is:

What I probably should be targeting is "5 years from now there's gonna be a lot of old 64 bit phone hardware with USB-C in the backs of drawers", and people will want to use that as hobbyist development systems the way Linux took over all the old 386 PCs in the 1990's but ignored the 286 systems.

Ala my "I wanna make all those old android devices useful" crusade probably isn't going to reach back before everything switched to 64 bit CPUS with USB-C. This realization was triggered by me dredging up a nexus 7 tablet from the back of a bookcase (Fuzzy wanted to send her mother a tablet, we sent her one of the spare amazon Fire tablets I'd buy cheap on prime day and once again confirm I couldn't reimage with AOSP, then shelve). Since I had the tablet and nobody'd touched it in at least a year I unlocked the bootloader (wiping it) then realized the reason you couldn't upgrade it beyond Android M is that's the last version of Android that shipped 32 bit binaries. Yes LineageOS has backported stuff, but... Linux 0.0.1 in 1991 ran on the 32-bit 386 and ignored the 16-bit 286 hardware. Minix focused on supporting the 16-bit hardware and LOST badly. We've been here before, and focusing on the old stuff was the LOSING side. Huh.

Today's new semi-related news is that microsoft continues to upgrade their Linux support in Windows 10, now running steam games and such with full 3D acceleration. Why are they doing that? Because it makes perfect strategic sense from a mainframe->minicomputer->microcomputer->smartphone perspective: big iron is consolidating and the "server space" has collapsed together before. Linux owns the cloud, so to put Windows in the cloud they need to turn Windows into a Linux. Microsoft wants a slice of the cloud pie, and is thus trying to prove Windows can host a Linux VM as well as anything else can.

Context: Starting around 1970 minicomputers kicked mainframes up into the "server space", then starting around 1987 the PC kicked the minicomputer up into the server space. At which point everybody suddenly found out there's only one server space, and all the systems in it are at least semi-fungible (out of sight, out of mind, make it work, here's a check), which meant IBM and DEC fought to the death. Eventually IBM stole all the VAX customers and drove DEC out of business in 1996, but IBM's technology wasn't unchanged by this. Before minicomputers IBM's Big Iron was punch card JCL ebcdic batch jobs written in cobol, fortran, S/360 assembly, PL/1... Afterwards IBM's big iron had C compilers, ASCII support, and interactive multitasking because that's what the server space looked like now, meaning that was the price of entry.

Around 2007 the PC got kicked up into the server space, this time the process has a marketing budget and was called "the cloud", about the same way individual bugs have names like heartbleed and meltdown these day. ("The cloud" is just existing Linux clustering technology combined with containers and a management GUI for easier administration.) This time around Amazon defeated IBM, but instead of cutting off its head with a big lightning show while "She's the Princess of the Universe" played on electric guitars, IBM survived by running away and buying Red Hat, pretty much the same way AOL bought Time Warner in the dot-com days. (The company that did the buying is the one that got eaten by its acquisition: IBM is Red Hat now, doo dah doo dah.) Today "big iron" means Linux container images transparently live migrating across interchangeable anonymous hardware instances, and that transition is OVER. It has happened, past tense, and the loser has fled the field. The server space is "cloud" as far as the eye can see.

Intel and Microsoft are PC companies, and the PC is big iron now. Just as IBM turned itself into a Linux company to keep cashing big iron checks, Microsoft also has to turn itself into a Linux company if it wants to play. And it can: being the original source of a technology means little in the business world. 3com inventing ethernet is a historical footnote, IBM was irrelevant to the future of PC hardware by the end of 1987, and AT&T's unix commercialization efforts lasted something like 7 years before losing the BSDi suit (which Linux rendered largely moot anyway). Microsoft has a history of "embrace and extend", and they'd happily pull that again if they could (become the most widely used version via monopoly leverage bundling, introduce proprietary extensions which become widely used because they're widely deployed, render competitors incompatible with a moving target), but Amazon is unlikely to LET them. The server space is _fungible_.

But the INTERESTING thing is that's all focused on the cloud. It means PC tech is ready to retire to the big iron rest home. The fact we're still forcing people to buy a Microvax in order to build dos apps in C (but we'll let them write all the Basic they want under DOS) is a FAILURE on the part of phones to be self-hosting. The phone needs to become a first class development platform so that having a PC does not make you a "better" developer, because there's nothing you can ONLY do with a PC that you can't do (possibly more slowly) on a phone.

Working on it...


May 19, 2020

At UT again. Late at night, heavily caffeinated. Spending effort to NOT put comments like "Heisenborg may have assimilated here" into the code I'm writing.

It belatedly occurred to me that the reason UT was so busy late at night on friday and saturday nights is... weekends are a thing! (Oh right.)

At least part of the problem with gmail's pop/mail servers randomly refusing to work is that ipv6 addresses don't work and ipv4 addresses do. (The error messages are TERRIBLE, but the behavior is consistent.) And it seems to cache however it talked to the server last, and then it's hard to dissuade it from doing it again.

The sh -c '(echo hello)' segfault turned out to be a use after free error. I need to switch on some variant of poisoning. (Valgrind is supposed to do all this stuff, and I should sit down and try to figure out how to get it to do its thing at some point.)

Except... the REASON for the use after free is expectation mismatch between bits of the code about whether I've implemented job control yet or not, and there's design work to do here.

Ok, at the end of a shell block you can have redirects or gating logic ala sh -c "{ true; } > /dev/null && echo hello" and the redirects get undone when you exit the block, so I was saving them in the block structure. I was parsing "} > /dev/null &&" as a normal (if nonsensical) command with pp = expand_redirect(), and then I grabed the unredirect list out of the resulting pp (an sh_process structure) and stuck it in the block structure, then freed the temporary pp. But the (subshell) logic would fork() (or do the elaborate nommu equivalent), save the new pid to pp->pid, and add it to the list list of running processes. Except that pp has already been freed after having its guts harvested because of the earlier case.

The bigger problem is that type 1 statements start a new flow control block, and type 0 statements execute commands which can spawn child processes and add them to the running process list. This means:

if (type 0) {
  do process list stuff
} else if (type 1) {
  I neither have, nor wait for, a list of child processes
}

But (subshell) | if true; then sed s/a/b/; fi | { tr x y; } is ALSO a list of simultaneously running child processes. Am I already handling that case? If so, where does that process list structure live?

Sigh, git annotate says I wrote this code in August of last year, and I've already forgotten how it works. THIS is why I was going "if I down tools and step away from the shell for any length of time, I'll probably never get back to it". I'm already spending more time reading what's there and trying to reverse engineer it than writing more code. This is why commands shouldn't be longer than 1000 lines, I'm not smart enough to keep it all loaded into my head at once...


May 18, 2020

Cool, solar and wind are on course to out-generate coal in the USA this year. Here's a lovely chart showing when it becomes cheaper to install new solar+battery farms than to continue running existing fossil fuel plants (answer: already is in some places). That's not "what new ones should I build", that's "rip out the existing one, it costs too much to feed". And given that the same is happening in transportation, soon fossil fuels become just a petrochemical feedstock, which oddly enough can probably be displaced by corn, since corn is a platform (and US agriculture is a real estate hustle).

I first noticed how INTERESTING solar had become because of a tedx talk on youtube (which I haven't been able to find again because the tedx channel posts 100 mostly useless talks/day) about somebody who visited a very rural third world village where everybody was replacing kerosene lighting with $15 solar camping kits running a string of LED christmas lights off a battery. These people had BEEN spending $10/month on kerosene forever, and now a one time payment of $15 meant they never had to pay for kerosene again, and selling kerosene to the third world for lighting was a $1 billion/year annual business for the fossil fuel industry.

This led me to a talk by Avery Lovins on how kerosene had replaced whale oil for lighting, how whaling declined not because of a shortage of whales but because a new technology rendered it commercially irrelevant before it could drive whales extinct, and how the same thing is happening again now to crude oil. Oil use isn't declining because oil is running out or getting more expensive, it's becoming commercially irrelevant due to new better technology.

Eventually I found Tony Seba's book tour talk (which was so popular he gave it to dozens of audiences, and still occasionally does updated versions). I later found a mutual fund manager's presentation to his investors of his own independent analysis of Seba's talk, and a recording of Seba's class from his professor days explaining the size, scope, and composition of the energy industry (circa 2013).

But I was late to the party on all this. Paul Krugman wrote a column about the rise of solar back in 2011, and the frustrating part is he did NOT mention the new enabling techology that had just unlocked much of its potential. And I had! I tweeted about it at the time! But I read the May 1, 2011 article Microinverters are launching a solar renewable revolution, went "huh", and moved on.

Microinverters are tiny, cheap, efficient inverters you can glue to the back of each solar panel. This decouples the panels so they didn't have to be wired up in series anymore, and that was a BIG DEAL. THAT is the point modern solar really took off, because physics.

Each solar cell generates about 1 volt, which is a small enough current the resistance in long wires can easily dissipate it. It's not a USEFUL voltage, and before microinveters the standard way to deal with that was wiring multiple solar panels up in series, which adds together their voltage to raise it to to usable levels.

The problem is, solar cells only conduct when they're generating. The cells that aren't currently illuminated RESIST current instead of passing it. This means that when solar panels are wired up in series and some of them are partially shaded, the dark bits don't just fail to generate anything, they actually EAT power from the other cells and get hot, subtracting from the energy produced.

(Batteries do something similar, most battery chemical reactions produce a volt or two and get stacked together, for example the 9 volt battery in smoke detectors is 6 stacked carbon-zink cells, each producing 1.5 volts. It's just a physics thing we have to deal with. That's why it's so important that the individual cells in larger batteries match exactly, because some of them going dead before the others, or filling up first when charged, makes the chemistry inside those cells eat itself when the rest of the battery tries to discharge that cell below zero or charge it past 100%. Silicon solar cells can be get very hot without being particularly damaged by it, battery cells tend to leak acid or explode.)

But an inverter turns DC current into AC current, and AC current can be sent through a transformer to exchange voltage for amperage and vice versa. (Transformers are famously one of the most efficient machines ever invented by humans, it's just a ring magnet with different amounts of wire coiled around two different sides of it, the big industrial ones are 99.7% efficient and even cheap crappy ones are generally at least 95% efficient, which means they lose less than 5% of the energy going through them as heat. The tool for converting AC to DC is called a rectifier and is just a clever pile of diodes, and none of _that_ is new. Building tiny inverters was the hard bit.)

Electrical power is measured in watts, which is volts times amps. Higher voltage with lower amperage goes further through wires, hence "high voltage lines" on the utility poles taking power across town. It's kind of like water pressure vs water volume. High voltage is very high pressure, but doesn't necessarily mean a lot of water is moving. (Think "hydraulic press".) At the other end a flooding river can be very low pressure but with huge amounts of water moving, which is why floods can be stopped by sandbags. The amount of water moving is what encounters resistance (and leaks out as heat), so higher voltage at lower amperage transmits the same amount of power with less loss in the wires.

Hooking a microinverter up to each solar panel means you can take each individual panel's 1 volt output and turn it a more standard 12 or 24 volts before sending it any real distance through wires, which lets you connect the panels together in parallel instead of in series, meaning if a tree branch waves overhead or a bird lands on one, that panel merely stops generating power instead of glowing red hot as all the OTHER panels try to force their current through the sudden resistance.

The exponential cost declines, increasing efficiency, and increasing lifespan of solar technology all helped, but Solar panels USED to work like the old christmas tree lights where one bulb burned out and the whole string went dark. In 2011 we got microinverters, and now you could use them on a roof where pipes and tree branches occasionally shade some panels at certain times of day without it being a big deal.

Meanwhile, Michael "Ok Boomer" Moore just release a hit piece against renewable energy for some reason, decrying the state the technology was in 15 years ago. No really, he's literally using footage and interviews from 2005 to say we all need to stick with coal or something, as pointed out by the many people people who've had to waste time debunking this old fogey's clueless rant.

Imagine if someone had released a documentary in 2005 about how the internet could never take off because dialup was too slow and AOL was a walled garden and Netscape was a monopoly and microsoft declared MSN an internet killer... such a position doesn't even manage to be WRONG. It's just... irrelevant? Describing a 10-15 year old view of a rapidly advancing technology is somewhere between "actively stupid" and "outright lies", but I'm not sure he's even AWARE out out of touch he is. He just got old.

P.S. What happened with the internet is in 1993 the NSF AUP changed to allow for-profit ISPs. (That's the National Science Foundation Acceptable Use Policy for the internet backbone it was funding after it took over from DARPA). That change led to netscape and geocities and napster over the next few years. Netscape released its source code in 1999, mass adoption of broadband started in 2000, youtube launched in 2005, and so on. Yes netflix used to mail DVDs to people and Blockbuster was a thing. Times change. People who don't get out of touch. It's the fundamental problem with "conservatism", but nominal liberals can get there too. "Kids these days" are RIGHT, every time. Because the people saying they _aren't_ die and are replaced by said kids. Might as well listen to what's coming.


May 17, 2020

Now is not the time for avoidance productivity, and yet. I'm halfway through working out the wirldcard parsing logic, but work REALLY NEEDS a drop with two bugs fixed, but my tree is dirty with all the wildcard stuff and I dowanna stop and back it out to fix those bugs (I'll lose my place!), nor do I want to fix stuff in another tree and have to deal with merges, I can't say how long wildcards will take to finish. (It's one of those keep digging until the hole goes clear through things.)

So what am I doing? Finishing a FAQ entry, editing my accumulated unposted blog entries, and catching up on an anime series on crunchycrunch that's treating tropes as poses its floor routine needs to hit rapidly in sequence, which the judges will be grading it on.

Now it's after midnight, but I walked to UT again with caffeine, and am trying to get stuff cleared by deadline.

I need to at least get the wildcard stuff to a good STOPPING point so I can check it in even if it's currently doing NOP, and all the interlocking wildcard, [square bracket], and @(nested pattern) parsing nonsense requires two stacks, and THAT's cheating by having each stack store two pieces of information. The bracket stack stores the ii/oo offset (in the input and output string, respectively) where the wildcard takes effect. (The input part shows you what the wildcard IS, the output shows you where to insert the data you find. The wildcard itself doesn't live in the new string, the result of the wildcard is inserted in there so I don't have to figure out how much to _remove_ to replace it.

Except... that doesn't work. The !(ranges) mean I can't just say "the wildcard itself isn't copied to the output", because there's literals tested in the !() patterns which I need to preserve? (If I need to chop stuff out of the output and replace it, that'll be annoying.) With ranges there isn't _A_ wildcard, there's wildcards plural....

Well... Hm. Wildcards are operations. And there is a series of operations, and *() groups are basically a flow control statement saying to loop, sort of. So they don't need to go into the output as long as I can keep track of what they DO... Hm, when traversing to match a string, non-wildcard text needs to be matched exactly, and the active bits say "when you get here, do a thing on the input, which might get copied to output".

Ah, it's not "offset into source string" I need to record, I need offset into target string and WHAT THE WILDCARD IS. And the control character can be two characters long at MOST, and there's 5 unique @( starters three of which (+, !, @) are already unique as wildcards. So if I call *( and ?( something else (to distinguish them from * and ? by themselves), then I just have a low byte with an offset<<8 in the output going onto the stack.

In fact the full manifest of wildcard ops is just 10 entries long: [, ], *, ?, ), +, @, !, and whatever I call *( and ?(...

And instead of recording where I added the starting "[", I should record where I need to INSERT the starting [ if and only if I hit a corresponding ']' (because yanking it back out again later is a flaming pain otherwise, there's at least 3 exit paths for "we didn't find it". A then I don't need to record the ] because I can just search ahead to...

Nope, problem: [ranges] and +(sub-patterns) have arbitrary contents, even if I've recorded all the wildcards I need the literal characters to match and to know when both end. And that has to be metadata because "]" and ")" already got de-quoted in the resolved target data, so I need to know when to match them or when not to match them as literals. And that MUST be in the target string because it's been resolved ala [$RANGE] so a variable expansion could have provided the contents I'm matching. Pointer into the source string was never going to work because * and ? can be in variables, and act as wildcards once resolved (if unquoted).

Ok, so if each wildcard metadata annotation records both its type and a corresponding TARGET position it applies to, ranges also record the END of the range as a second metadata entry...

Nope, problem: [*] needs to keep the * as a literal, and [+()] needs to keep all three characters as literal. When I discard the metadata entries, I need to KEEP the in-band signalling part of it because that's the [square bracket search contents]. Which means I'm back to recording JUST a destination offset in the resolved string where the "active" wildcard character(s) get copied, and I need to remove them from the output when matched. (Well, not copy them into the new output string it's generating, wildcards can expand to multiple entries and thus multiple strings.)

Ha! Parenthetical sub-patterns are _not_ guaranteed to close!

$ IFS=x; ABC=cxd; for i in +($ABC); do echo =$i=; done
=+(c=
=d)=

And when I try that line via bash -c the parsing has a syntax error, but it works fine from the command line. Time for another email to Chet.

Grr, I'm replacing array_add() with arg_add that takes a struct sh_arg * instead of separate char * and len (because almost all the callers are doing array_add(&arg->v, arg->c++, data) that anyway, I need to convert the "a=b command" environment parsing to have an arg wrapper), and there's two users that aren't incrementing arg->c. The first case is the line terminators ala && and | where arg->v[arg->c] is the teminator rather than the NULL, but I can just arg->c-- when I detect that.

The SECOND case is when it hits ")" which also ends a line early, but isn't a terminator? But it seems to be storing it as the line terminator in that case anyway, except... the terminator is functionally null, the ) should be at the start of the next argv[] list?

So I go "wait, how is the current code handling this", and do an ./sh -c '(echo hello)' in a clean checkout, and... segfault! (Wait, I don't have a test for simple parenthetical?)

Tangent from tangent! Drink!

(P.S. Why does wikipedia's "Silver Spoon" entry not link to the TV series "Silver Spoons"? They say the first needs additional citations that the saying was ever even a thing, and here's a TV show with a theme song explaining it. I'd add a note to the talk page but wikipedia has banned edits from the entire ipv6 address range, and even though I've got both v4 and v6 addresses it's decided to route there via v6 for no obvious reason, and it's all automatic wicd/dhcp nonsense happening behind the scenes I don't feel like fighting with. Oh well.)


May 16, 2020

Would someone please let Rachel Maddow know it's ok to say "dementia"? Everyone else is noticing, and yet his "lack of object permanence" policy is literally the one being carried out by his cronies, because they're getting what they want out of the laziest of faire, everything deregulated while they loot and pillage and lynch brown people.

I want this on a t-shirt (The phrase emotional support guillotine is 2020 in a nutshell.) Billionaires have set the system up to delegate blame, send it back where it belongs. Remember, guillotining the billionaires is far more realistic than taxing them. Don't go for half-measures, lobby for capital offences receiving capital punishment.

"A degree from Harvard" now means "you went to the rape-cover-up place", pass it on.

Is anyone really protesting for the right to work, or are they demanding the the right to be served? White people with guns are screaming "you can't get good help these days". Meanwhile the GOP is just coming out and admitting they're cheating at the polls on live television. ICE is still separating families and now they're using COVID-19 to do it. Globally, racists are pulling out all the stops. The GOP are more or less distributing silver cake spray for everyone's teeth and having us practice chanting "witness me, shiny and chrome". (A hero risks their life to tackle a problem created by the rich and powerful ignoring externalities. They want everyone except themselves to be heroes, always and forever.) Social distancing bought us time, and the republicans squandered that time. All modern "conservatives" are complete hypocritres who believe in nothing. They have filled the US healthcare system with sexism and racism that kills people.

We got in this mess because we stopped teaching the humanities. You have to teach people to be people rather than assholes, but the libertarians went "STEM STEM uber alles, if you know enough science that makes you a good person, there were no such thing as nazi scientists or mad scientists or the tobacco institute or the ethyl institute knowingly killing people for money, and as Ayn Rand shows it doesn't even have to be REAL science you can pull it out of your ass and sound convincing and that counts".

Atlas Shrugged gives a 6 year old boy's tantrum speech a high school vocabulary and sticks it in the mouth of an adult meat puppet ("John Galt") who delivers an author rant for multiple pages. "I'll take my ball and go home and the world will collapse without me and that'll show them because I'm such a magic irreplacable special genius the likes of which history has never seen that the absence of me IS the end of the world, I was born in a log cabin I built with my own two hands and grew my own food from birth and speak a language I invented myself and taught to all of you, so there". The result appeals to idiot silver spoon failsons like Paul Ryan with the emotional maturity of 6 year olds who love their existing worldviews validated. THAT is why it still sells so many copies, because we have so many stunted abuser manchildren in positions of power. The british navy famously had "Sodomy, Rum, and the Lash", the united states has 300 years of plutocracy, racism, and misogyny. "All (rich white) men are created equal" but women and brown people and poors were raped, enslaved, and lynched. The "trail of tears" relocations continued for a decade after the civil war (which managed to convert outright slavery into Jim Crow), and General Custer of "custer's last stand" fought for the _north_ in that war. Women didn't get the vote until 50 years later, after World War I. (The USA sent enough men overseas it needed women's labor for the war effort, and they demanded concessions. Then we did it AGAIN a few years later, and a lot of the 1960's "women's lib" stuff was fallout from the WACs and WAVES of WWII telling their daughters about Rosie the Riveter. Never let a crisis go to waste, change happens under stress.)

The british were basically the nazis a century earlier, and if you think that's an exaggeration look up the opium wars, the boxer rebellion, and ask how exactly did the British East India company wind up OWNING INDIA (a country that HAD a perfectly good population long before the british got there). The difference between India and North America or Australia is the europeans committed genocide against those native populations to the point less than 5% of them are left (which is still millions of people, being horribly treated to this day), but India had encountered Alexander the Great and Marco Polo and such already long before the british got there, and wasn't taken COMPLETELY by surprise by european brutality and slaughtered the way the american and australian native populations were.

If the Nazis had won World War II and got to write all the history books for a century or two, you'd get something that looked a lot like the history of the united states, and we KINDA need to acknowledge that if we're to avoid doing it again.

Revisitionist "we didn't kill millions of people, there was hardly anybody HERE before we arrived, you're imagining it" is the exact same B.S. the Nazis tried to pull. The british just did it first and got away with it, and the early americans WERE british and continued the slaughter to the point "playing cowboys and indians" was still a popular passtime while the Boomers were growing up. There was an awful lot of "how bad are the Nazis really" in the 1930's, and the USA turned away refugees and had a strong "america first" movement. But in 1941 the USA had liberals in charge (FDR cleaning up Hoover's conservative "bail out the rich and let the rest starve" policies that caused the great depression) and with our own civil war against slavery still within living memory, FDR went "You know what? Let's NOT allow racist empire-building and genocide this time around", and while the USA was at it we made Britain give all the other "british empire" countries BACK (at least where enough of the original owners survived) and set up a "can we not do this again any time soon" club called the United Nations.

And no, none of this is "ancient history", it's all still happening every day, and I am very tired of it. Teach the humanities again. The idiot boomers can normalize anything as long as nobody rocks the boat for their last few years, the rest of us have to learn this stuff and take over if we're to have a planet after they finally die.


May 15, 2020

Apple, the #3 company on the Fortune 500, has worked out a deal to pay less than 1% of last year's profits (and less than 1/5 of 1% of its annual gross revenue) to settle a class action lawsuit that it sabotaged its own older devices to force people to upgrade. The settlement apple wants would cost it $25 for each affected user, but they wouldn't get that much each because the lawyers get paid first. (This is why we had sherman antitrust breakups, and why we had guillotines before that. The republicans can plug up various pressure relief outlets, but can't stop the pressure from rising.)

Sigh. Posting a patch to linux-kernel yesterday was my mistake. Here's the reply I did NOT post to linux-kernel. (With the usual awkwardness of in-band URLs turned into links because html vs email.)

> Hi Rob,
>
> You need to send this patch to some maintainer who could merge it.

To be honest I didn't think anyone would notice this. They don't usually.

I tend to give up trying to get things merged after three strikes, going back a while. If people want it, the idea's out there, and it still works for me. (Although in that example's case I've moved on to a microconfig format that mostly fits on one line, ala the KCONF= stuff in toybox's built in make root.)

In _this_ patch's case, the "v4" is because it seems polite to publicly post code I'm shipping in a product, and I had to update it to make it apply to a v5.x kernel because sys_blah() is now ksys_blah() for some reason.

But I gave up on trying to get anything into linux-kernel after v3 back in 2017 (not just this patch series, I gave up on regular participation in the linux-kernel dev community in general), and blogged about it at the time.

> And it uses the wrong multi-line comment format.
$ wc Documentation/process/*
...
11363  70930 475446 total

Would it make a difference? *shrug* Ok, attached.

The thing is, I'm a hobbyist. Linux-kernel is no longer a place for hobbyists.

Last month I drilled down into a syscall implementation (to see which clock_gettime() call gettimeofday() translated into) and hit a FIXME comment which said:

...which we don't want to do late in the release cycle. For now...

And according to "git annotate" in my linux-fullhist tree that comment was added June 21, 2007 (the "release cycle" in question would have been 2.6.22). It's still there in 2020, having survived 13 years in "kernel/time/posix-timers.c" (used by all architectures) because the whole "enough eyeballs find all bugs" thing assumes there are still any hobbyists at all involved in the project, looking at stuff their boss didn't tell them to. As far as I can tell, there are not.

The Linux Foundation literally sells seats on its board of directors for five hundred thousand dollars per year (all "platinum members" get a board seat so anyone rich enough can buy one), and the current board is listed by company: Oracle, Sony, Microsoft, Huawei, Comcast, IBM, Hitachi, Renesas, Panasonic, Microsoft's Github, the Deptuty General Council of Microfocus (the proud manager of 60 attourneys), VMWare, Cisco, Qualcomm, Samsung, AT&T, Facebook, Google, and the token "silver member" from Gitlab, plus the executive director of the Linux Foundation.

Returning to Documentation/process, the "code of conduct" (fine) has a seperate "interpretation document" (danger will robinson) that ends with a resolution to establish an updated process for committee staffing. No, I did not exaggerate that, let me QUOTE the end of Documentation/process/code-of-conduct-interpretation.rst:

We expect to establish a different process for Code of Conduct Committee staffing beyond the bootstrap period. This document will be updated with that information when this occurs.

It's the paragraph after the one promising the committee will produce a quarterly report summarizing the reports to the committee.

Do you really see a place for hobbyist coders to participate in linux-kernel anymore? Other than submitting bug reports? Linux Weekly News's "are there still any hobbyists?" article was 7 years ago now, and the answer at the time is "what's a hobbyist"?

But sure, I removed the offending comment from the patch.

Rob

Alas, even what I DID post, as cleaned up and chopped down as it was... is still too vitriolic. Randy was helping, and I snapped at him. It's true that I already submitted that patch 3 times, and gave up on the idea of getting it in. (And there's SUCH a surplus of coders in Linux-kernel polishing every exposed surface that "here's a working patch to do this obvious thing" saw no follow-up for 3 years despite the nominal objections this time and last being about COMMENT STYLE.)

Sigh. I tried to swallow it and do the "hey, if you wanna escort this patch in" thing, but A) I just haven't got the emotional energy, B) I honestly don't think it would help? They don't WANT it. Yes I know how to get patches into linux-kernel in the same way I _CAN_ sit in line at the DMV for 4 hours and then come back the next day with different documents to do it again. But after three attempts of the "rejected, start over from scratch filling out a new form after having mailed away for more documents" variety? I generally don't want whatever it is THAT MUCH. (I only got my driver's license back to help Nick move, and these days I don't even have a car anymore. Then again, I'm not alone in that but we all know why and have for a while.)


May 14, 2020

I should edit and upload my blog entires again. I expect it to be about as useful as posting to linux-kernel but eh. Makes it easier for me to point people at old things I wrote rather than endlessly re-explaining myself.

Shell wildcard support is fairly straightforward to implement for *, ?, and []. For a definition of "fairly" that has a big subtlety in the square brackets:

$ touch l; echo [hello"]"
[hello]

The reason echo [hello"]"] doesn't print "l" is because the quoted ] isn't active as a wildcard character, meaning you need to annotate live ] as well as [, and then WITHIN the [] there's touch "[hello]"; echo [hel?o"]"; printing [hello] because the brackets weren't live therefore other wildcards within WERE live. A decision which can only be made retroactively, so it needs undo.

But that's the "fairly straightforward" bit. Then there's *() and friends, the bash "extglob" syntax, which nest arbitrarily deep. So I need to maintain a stack of them? Hmmm...

$ touch "potato(abc"
$ echo *(abc
> ^C
$ echo *"("abc
potato(abc

Ok, I _think_ the parentheses parsing will force these to match up when they're unescaped?

$ touch $'potato(abc\n)'
$ echo *(*
> )
*(*
)
$ ls potato*
potato(abc?)

That _should_ match the file? The parsing didn't allow it to. (In bash, not in my implementation, I haven't done this yet.)

Speaking of parsing, (echo hello) needs to become 4 words, and that's what my current parsing is doing. But +(echo hello) is a quoted escape, meaning it's parsed like $(). It is, in fact, parsed UNCOMFORTABLY like $() because echo +(() really shouldn't prompt for another line because the ( inside the () isn't special: +(pattern-list) "Matches one or more occurrences of the given patterns", and yet bash does prompt for another line because the unquoted parentheses didn't match up.

So I guess recycle/extend the $() parsing, do liveness tagging on the range ends as well as the range starts... Hmmm...

$ 
$ unset X; echo +$X(one)
bash: syntax error near unexpected token `('

Ok, what's happening THERE is that +( isn't recognized at parse time so we don't see +( to act like $( and thus the parentheses aren't interpreted as a form of quoting. Got it.

Which says my deck of wildcards needs to tag [?*+@!]( on the ( rather than the first character (because ? ain't ?( and and ?"(" is ? not ?( and the tagging needs to distiguish the behavior because the quotes are gone by the time wildcards are parsed), and it needs to ONLY tag recognized pairs (not just any old open parentheses) so when we see a tagged ( we know we can back up one to see what kind it was. And the parentheses _must_ close because parsing requires it (same as for $( it's an unterminated quote otherwise, triggering a line continuation), but the [] doesn't need to (that does NOT trigger line continuations, so can be retroactively a literal "[") so wildcards in [brackets] may or may not be active depending on whether it closes, so that needs more work. I think I need some kind of "tag and discard"? Brackets do NOT nest...

$ echo [+(]
> ^C
$ echo [(]
bash: syntax error near unexpected token `('

In fact they're completely ignored by word recognition, which makes sense for:

$ touch b
$ echo [$(echo abc)]
b

But +() isn't parsed to find wildcards until way later than that, but parsed for SYNTAX reasons much much earlier. Bash seems to have shoehorned all this into logic it already had and it's kind of a layering violation? But what else is new...

Oh fun:

$ touch +
$ echo [+()]
+
$ echo [+(])
[+(])

Yeah, about that not nesting. Square brackets don't but @() does and each one of those has its own square bracket undo context! Lovely.

(There's probably official terminology for all this crap which I'm butchering horribly, and I don't care. I'm working out priorities, with test cases to demonstrate them.)


May 13, 2020

3am at UT, sitting on the picnic table near the outlet (the semi-enclosed outdoor area of the Norman Hackerman building, no really), getting work done. I get a mile+ walk here and as much again back, so it's good exercise, I'm out and about but it's late enough at night I'm still more or less quarrantining (haven't seen anyone for two hours)... I needed this.

I got toybox 0.8.3 out monday! Didn't blog about it because I haven't been (publically) blogging, but I wrote extensive release notes on the website.

Now I'm juggling A) fixing the sh2eb target actually _running_ on my turtle board (it runs in qemu-sh4eb but initramfs doesn't launch init), B) implementing wildcard support in the shell, C) adding a scripts/root/tests package to makeroot to run the toybox test suite under QEMU, D) writing a toybox FAQ entry about make root.

I'm doing one of those tangent-from-tangent-from-tangent things but hey: fresh laptop with 8 desktops to fill up with todo items. All those old tabs got closed... with alas rather a LOT of shell test cases I'd run in tabs and meant to move into my sh.tests text file or tests/sh.test actual runnable regression test suite... I got maybe 1/4 of them converted in my gradual "chipping away at trying to shut down so I could reboot my laptop and run a more current kernel than October" since the blog entry where I went "yeah, I should do that". I was working on it! Oh well.

Oddly enough, this A) B) C) D) stack is what my working style looks like when I'm _not_ overwhelmed. I follow the tangents as they occur to me, push what I was working on onto the stack (this is E: blog about what I'm doing), finish the thing and pop it back off to resume what I was working on before that. (The terminal tabs let me know! They're even more or less in order when I haven't opened 6 windows with 30 tabs each.) At the end of a solid day of this, I get back to A and haven't finished it so obviously I got nothing done. (The stuff I committed and posted is just side effects. That's like calling making lunch an accomplishment.) But I tidied my workspace and DIDN'T wind up making more work for myself on the ever-growing todo heap, so I usually call it a win.

(Didn't say it was the healthiest mindset, I just said it's what working at my own pace when it isn't all LOOMING at me is like. At least all the bits go back in their boxes. When I DON'T have the time to follow each tangent as they occur, I open a tab for it or scribble a line in notes.txt or leave a TODO comment in the code and go back to what I was doing, and the todo items accumulate endlessly which means I'm falling BEHIND...)

I updated the link to blackduck but they've got the project's license wrong? Found an email address and poked them about 0BSD, let's see if they respond...

(I proposed 0BSD talks at the last couple conferences I submitted proposals to, but nobody was interested. But hey, Jeremy Allison, the Samba maintainer who switched his project to GPLv3, has come around to my way of thinking. He regrets the GPLv3 on samba, thinks it has harmed his project, and would undo it if he could. And says so in that video. I met him in 2008 when we were on a panel together with Eben Moglen to talk about licensing, back when I was still a plaintiff in all those busybox lawsuits. That's back when I thought The GPL could still be saved...)


May 12, 2020

I googled how to stop google searches from showing up in the chrome URL bar as suggestions (I NEVER type a search in there so as not to pollute the URL autocompletes, I go to google.com and load the page because typing into the fake google page that opens in new window similarly pollutes the URL pulldown, but ever since the reboot chrome has "updated" itself so searches on ACTUAL GOOGLE are polluting my history. This appears to be a recent behavior change, and is "integration" I was actively working to avoid until they broke my circumvention strategy.)

My search found a thread where a bunch of people asked to be able to disable this horrible thing chrome does, a google engineer expressed bafflement at them and told them to use more->help to request the feature, and then when people wouldn't stop saying "I'm requesting the feature here" he locked the thread so nobody else could reply.

I couldn't find "more->help" so I googled it and that had a "was this helpful" at the bottom where I listed all the things my three dots pulldown has which are NOT "help", which seems to be a windows thing. I do not expect a human to look at it, but at least I got to report _something_. And in the meantime, I guess I clear my search history and use an incognito window to type google queries from now on? The workarounds keep getting more elaborate...


May 11, 2020

Sigh, and my laptop rebooted. I unplugged it because thunderstorm, and forgot to plug it back in before taking a nap, and the battery drained all the way. (I guess my recent comment about uptime was asking for it.)

I wrote a long toybox release announcement for the mailing list with a PS as big as the rest of it about my stress levels and suspending my patreon campaign, but I hadn't quite finished and sent it and of course thunderbird doesn't record unfinished edits the way kmail did, or vi does, or chrome does when writing in forms, or... any competently written program, really. But then Thunderbird was a mozilla project from the same people who bloated Mozilla until Galleon forked off from it, then took that over and bloated it until firefox forked off from that, and then took that over until Google wrote chrome in self-defense. But hey, after Thunderbird development was declared dead a few years back it was relaunched in 2016 and again in 2020 so... maybe it'll grow a feature kmail literally had 15 years ago someday soon?

We can but hope. Because it's written in C++ which means I'm not patching it.

The main reason I use chrome instead of mozilla is because "pkill -f renderer" will free up the memory and CPU used by all of the open tabs without losing where the tab points to, and I can reopen it by clicking "reload" later. (Or if it gets confused, which modern chrome does due to stupid threading stuff, close it with ctrl-w and then ctrl-shift-T to reopen it and now it's a FRESH tab with the old info.) This is pretty much what the chrome design promised from day 1, in handy comic book format, and given that tabsplosion is my normal working style (they sort of compost after a while)...

I haven't found anything remotely similar in mozilla. Nope, that's one big process and if a tab crashes the whole browser closes. If a tab eats CPU there mozilla expects you to close the tab, discard the information, and never get back to it. (And from both browsers I have wanted a "save tab as tar/zip for later offline viewing" feature for YEARS, not just bookmark but SNAPSHOT. You can right click "save as" and there's "web page, complete" in the pulldown that saves the css and images and such all in a directory, I did that a few times for my old computer history mirror. But there's no integrated way to tell it to migrate a tab to local storage AS a form of bookmarking. You'd think this was obvious, but no. The last browser that could view inside a tarball was Konqueror, which was tied to KDE and went down with that ship for me.)


May 10, 2020

There's stuff I wanted to say about these links but eh, what's the point? The republicans are still trying to destroy social security. Tax the rat farms. Basic income. Etc, etc...


May 9, 2020

Um. I just got cash (a $100 bill) in the mail, from "A Friend" (with a return address in Oregon). Postmarked the day I suspended my patreon.

I... um... thanks?

I used to mail cash almost exactly this same way to webcomic artists, except I'd put my name on the return address and write "make good art" on the paper wrapped around the money. I never expected to be on the receiving end of that. (In part because everybody else is either comfortable with paypal or uses checks, but "birthday card with cash in" always trumped "birthday card with check in" on an emotional level. Better endorphin rush.)

I spent my 20's depressed and still have a bunch of classic ADHD symptoms (except we were just called "hyperactive" and/or "gifted" back then), and now there's something called "rejection sensitivity" (you mean everyone else DIDN'T have that?) which means one negative review outweighs a dozen positive ones. The main difference between now and my 20's is I have more and healthier coping mechanisms, which are just tricks to avoid mental pitfalls. (I basically got good at playing my brain like a pinball machine.)

I used to have a lot of _unhealthy_ coping mechanisms. When I was first dating Fade she got REALLY upset when I'd take my wristwatch off and smash it with a hammer or kick in my own door (in a condo I owned), but in the back of my head I was going "this is $10 to replace" and "this is an hour of carpentry with tools and supplies I've already got" so I was venting at something _safe_. I only stopped because it didn't read that way to _other_people_.

The healthy coping methods are collectively called "adulting" these days, and include using things like code switching to act out a professional role, so that when negative feedback comes in it's not aimed at ME, it's aimed at the character I'm performing. (Less healthy: "I don't care what you think because you're a worthless human being", healtheir: "I only conditionally care what you think in the context of this task, and can STOP caring when go I do something else, your bile doesn't matter I've got gloves on".)

I got a lot of practice at this running and promoting Penguicon and Linucon, where _I_ could never go up to someone and tell them how great I am, but armed with flyers I could gush for hours about how great the project I represent is. Hardison's line in Leverage about his grandmother making him knock on doors and that's how he got his social skills resonated, level grinding for the win.

Unfortunately, I can't put a mask over rejection sensitivity when I can't leave the house for weeks at a time, haven't got a regular schedule, am working on my open source project(s) _as_ an escape from stress, have no seperation between "project communication" and "personal communication" (thanks twitter)... venting into my blog to cope with the stress when I put "update the blog" as the first patreon goal is not good, that's a totally valid criticism. So I stopped patreon AND stopped uploading blog updates.

This pandemic and the general trash fire my culture's going through have left me persistently Out Of Cope, and when I can't keep all the balls in the air I tend to throw one or more of them in the fire so it is No Longer My Problem and not lying around for me to trip over. I no longer smash my own stuff but will quietly throw out accumulated dirty dishes/pots/pans sometimes because I Can't Even and just want it to stop. (Except fuzzy finds them when she takes out the trash and is then offended...)

But being unexpectedly on the receiving end of a thing I used to do... is nice.


May 8, 2020

I've intentionally not blogged stuff since The Event, but I've continued to WORK, and this blog is for me. I don't have to upload it. I already don't remember what the stuff I had to urge to record and didn't over the past few days was, and some of it was technical stuff that's gonna come back to bite me. (I could post random thoughts to the mailing list instead, but that would be rewarding the guy who complained and especially now I've suspended my patreon campaign I'm not doing ANY of this for other people really, just for me. But I need to record my idea of what the result should look like in order to _do_ that.)

Except it's not just for me, I need to make it attractive to the android guys to lead them via trail of breadcrumbs to my desired outcome.

And I care what Jeff thinks because his company's paying my mortgage. And right now he needs dropbear running on the board, and I did 95% of the plumbing for that, adding it to the build and getting it to work on nommu, and now dropbear's working fine from the command line but not starting automatically, and it's a shell problem.

Ok, back up: back when I did standalone mkroot (which also ended because contributors were armchair lawyers simultaneously causing legal issues while voting about what I would do next without asking me), that exteranl variant had module support, with one of the most prominent (and useful) modules being dropbear ssh. But that version of mkroot also required busybox, which means I didn't want the headache of distributing binaries of its output (hi Bradley).

Now that I've merged mkroot into toybox, greatly simplified it, and gotten it to work with JUST toybox (and the Linux kernel if you're booting under qemu, but that's always vanilla linux using unmodified release tarballs, and I'm providing the kernel config in the tarball and I can put a URL to kernel.org in said config as a "written offer of source" so MAYBE I feel ok distributing those, we'll see whether I bother... Maybe I'd have a FAQ entry pointing to kernel.org....)

Anyway, I did some updates to the old project last month to get a current and nommu-friendly version of dropbear building under the old context, but... that's not the way I want to go. I want it building under the CURRENT version, which is built into toybox.

This presents a few problems, the first of which is I optimized away the download/setup/cleanup functions, because the new mkroot is part of toybox so doesn't have to download any external packages to build the filesystem (we're always building in toybox's source directory), and I cheated and went "point me at an externally provided Linux source directory with LINUX= and if you don't I'll just skip that part of the build". But now I need to download zlib and dropbear, so I need that infrastructure BACK, and it doesn't really belong in mkroot.sh OR the new dropbear build script..

But the BIGGER problem right now is the init script needs to be expandable, and the obvious plumbing for running generic "module" init subscripts would be something like:

for i in /etc/rc/*; do . $i; done

Which doesn't work because I haven't implemented wildcard support in the shell yet, and doing wildcards properly is fiddly because "a*b"*c only expands the _second_ asterisk, but X='*' echo $X does expand that asterisk, so the order of operations here conflicts and I have to annotate or speculatively expand or recurse or something?

I'm tempted to channel the Simpsons and "do a half-assed glob()" just to get this ready today so I can cut the toybox release and move on to something else for a while. Maybe one that expands everything whether it was quoted or not.

I'm not entirely sure how this is supposed to work, since:

X='*'; echo $X
echo "*"*".?z"

Implies that quote removal does NOT get done as you go along adding "already processed" data to the output string, as I'm doing. There's a sequencing issue where you either speculatively expand and make multiple passes over the list eliminating stuff as you go (which is both memory and cpu inefficient) or you keep track of "live" vs "non-live" bits for expanding later. (Or produce a wildcard-escaped string and then do an escape removal pass? Hmmm, expanding is just * ? and [ I think, but if \ is still the escape then you need to escape \ too...)

Which is probably why the bash man page implies they keep the quotes to remove in a later pass, but that doesn't help because a variable can expand into a quote so you have the same live-vs-dead text problem, this just moves it to an earlier point, which gets us back to either wildcard escapes or "live vs dead" tracking metadata...

Walked to UT after the sun went down, to sit at the deserted little table near an outlet. It's good to get out of the house.

Wildcard liveness and IFS parsing have almost the same lifetime: if one is active, the other is active. (Modulo the NO_PATH flag.) I've already shoved the IFS parsing into a single codepath everything else drops through, so I can annotate the wildcards then.

But instead of annotating the wildcards inline, I think the right thing to do is array_add() the offsets of the active wildcards and call a function to handle them. There are three calls to array_add_del() that all the "here's a finished argument" stuff channels through, I can make a new function they call instead to do the wildcard expansion, with a deck of wildcard offsets passed in alongside. Hmmm...


May 7, 2020

Huh, this is really cool. (Living in the future!)

Facinating video on how billionaire finances work. Any large expensive thing a billionaire buys is put up as collateral for an "equity line of credit" at 1% interest (less than the inflation rate, so they could make whatever minimum payments there are _forever_ by borrowing more as the price of their underlying "asset" increases each year due to inflation). All the houses and yachts and skyscrapers and such act as cash which they can use to buy MORE houses and yachts, which they then collateralize to borrow more against.

And then they hide it in tax havens. Now that the EU has started cracking down on billionaire tax avoidance (as I recently pointed out, one of the big triggers for brexit), South Dakota (which the GOP swarmed, overwhelmed, and took over the state government of due to the fracking boom drawing in republicans the way picnics attract ants) has become a "mini-switzerland".


May 6, 2020

Suspended my patreon campaign, here's what I posted:

When I lost my twitter account last year I started responding to world events in my blog instead, but recently a patreon supporter asked me to stop, so I did. And since "update the blog" is the only Patreon goal that ever got funded (admittedly after I bumped the amount down so it would), it would be unfair to keep taking money when I'm not even doing that anymore.

Quarrantine has not been great for my mental health, and after almost giving up on toybox development https://landley.net/notes-2020.html#16-04-2020>last month, I've come to the conclusion I can't handle the stress of disappointing my patreon contributors, so I've decided to close this page and stop accepting pledges.

Thank you all very much for your support. I'm sorry it didn't work out, but a year's patreon support doesn't pay a month of my mortgage, which means as much as I'd like it to this can't affect the amount of free time I have to spend on open source in any real way, so you're not really getting anything for your money, are you? I've appreciated the encouragement, but never felt I was earning it.

Rob

P.S. I mostly outgrew ADHD with a lot of caffeine and code switching, but right now I haven't got the energy to juggle coping mechanisms, and it's hard to role play a professional in quarrantine. Adulting is always mostly role play, in my case "I haven't disappointed you, my professional role has disappointed you, which is part of doing that job", but unfortunately that has its failure modes.)

I've hit the "don't bill your patreons next month" button rather than taken the page down because taking the page down would take me out of the grandfathered in 5% patreon fee instead of 8% fee (on top of however much the credit card companies take, and no patreon doesn't let you use the money people send you to sponsor other people's campaigns, every sponsorship MUST pay a tithe to the credit card gods).

But it lets you hit the "skip a month" button indefinitely. I'll see if I feel like making it permanent later.


May 5, 2020

I got an email, and responded:

> I have been following your work and blog for a ling time, learning many things.
> In April 29 I noticed you mentioned USD as commodity. I also found this
> extremely interesting and following few people I found this article
> interesting: https://phenomenalworld.org/analysis/the-class-politics-of-the-dollar-system
>
> I think it explains well why this is a resource curse for most.

Interesting. The article Mark Blyth wrote for that site is also a good read, although I don't have time to read the book he's reviewing.

The best article I know of for explaining the basic "resource curse" is this (which is the same guy who wrote this which I found _very_ helpful.)

Going through the article you pointed me at, the "opaque ownership laws" thing was one of the main drivers of brexit, specifically that the EU was forcing various tax havens to adhere to higher reporting requirements or else be banned from making transactions with the EU banking system. Billionaires _freaked_, but unplugging the UK from the EU didn't stop the EU from doing it. Here's an excellent summary of that, see also this and this and this.

> I thought you might be interested as well, and then I may learn more about it in
> a future blog entry.

It seems like good information, thanks.

Rob

Links du jour:

Don't say nobody could have predicted when the internet never forgets.

Nikola Sturgeon, First Minister of Scotland, says the time has come for universal basic income in scotland.

Meanwhile in the USA...

Late-stage capitalism has nothing for the 99%


May 4, 2020

Oh God. A committee. Biden's VP announcement was that he can't decide and is forming a committee. Can we please get a do-over?

Stop it with the octagenarians. We know Nancy Pelosi's going to cave too, on whatever the issue du jour is. And then turn around and attack her own party for pointing out that the 80 year old is obsolete and out of touch and standing in the way. These geezers need to STOP DRIVING.

Here's a fascinating thread on using diesel to launder money. (I've mentioned before that the way Putin funded his puppet government in ukraine (back before the populace threw the puppet out, so he invaded and stole crimea which was the eastern half of ukraine) was by selling unnecessary middlemen oil at slightly below market value, and letting them sell it on to europe at full price: shaving a dollar or two per barrel off of billions of barrels per year transfers huge amounts of money, and it's technically legal.)

The Texas Railroad Commissioner (which for historical reasons is the guy in charge of regulating the Texas oil industry) has declared himself useless, and that it's too late (and politically impossible) to impose OPEC-style oil quotas on texas production (hence the headline "oil proration dead"). But in the interview he seems sure that nothing's really (ever) going to change, and everything will return to "normal" someday. Horse drawn carriages came back, didn't they?

My roommate Fuzzy is watching conspiracy videos online because "they have some good information" and it makes me so sad. No, they don't. The stuff that's going on IS NOT SECRET.

Look: the Boomers allowed Reagan to lower the top tax rate and balloon the deficit, spraying down the country with financial parasites sucking the money out of everything and exacerbating the racism left over from the civil war. They did this because they were greedy bastards who didn't care who else got hurt, and did not try to hide it. They grew up in the shadow of World War II (which sorted the entire world into Axis and Allies, the "good guys" vs "the enemy"), which transitioned seamlessly into the Cold War (United States vs Russia), and when Russia collapsed in 1990 because the price of oil went down below where it could afford to import food, the Boomers thought they had won the world and history was over now, and had a decade-long touchdown dance under Bill Clinton as the Lone Superpower. Then the world Trade Center attack in 2001 broke the Boomers ( Duct tape and plastic sheeting! Color-coded terror alert levels including red, green, and blue), to the point they allowed Dick Cheney and company to install a surveilance state which Ed Snowden reported to the world in 2013 (the same way Daniel Ellsberg had in 1971, but now the Boomers were "the man" being reported on and didn't like it). This means the Boomers have been pearl-clutching "kids these days offa my lawn" old fogies screaming at clouds for 19 years now, and given a chance the boomers voted for literal nazis.

This isn't a conspiracy. They're not hiding anything. (Ok they're hiding Trump's fronotemporal dementia the same way the GOP hid Reagan's Altzheimer's, but the guy who "I don't remember"-ed his way out of the Iran Contra affair wasn't really hiding it WELL, was he? And neither is the second "oldest president ever".)

We're living in the twilight of the Boomers. Nothing will get better until they're out of power. Their 1980's "greed has good" has collapsed into late stage capitalism where the big modern "innovations" are things like delivery apps that screw over both the restaurants and the drivers. It's not sustainable, and when they die we will chisel their names off all the monuments just like the egyptians did. The two countries currently collapsing from coronavirus are the ones where the Boomers put Ronald Regan and Margaret Thatcher inexplicably in charge. Everywhere else is handling this way better.

The GOP is still lying about everything. The standard trick is to say it's too soon (to do anything, or after the mass shooting to talk about it) until it's too LATE (and old news). "Yes Prime Minister" called it the four stage strategy. The fix, as always, is to guillotine the billionaires. The reason to guillotine the billionaires is there's literally no other exit strategy. Even the excesses Upton Sinclair wrote about the meat packing industry a century ago have returned with a veangance. The Boomers are recreating "typhoid mary" and "the triangle shirtwaist fire" because they've learned NOTHING. (And things like the gulf oil spill only get theatrically addressed when they make headlines, the exact same thing happening right next door can fester for decades.) When the boomers die, their world collapses and we build something new out of the parts.

First to go needs to be the GOP. The pandemic isn't remotely over, hasn't even peaked, but as soon as they found it out kills brown people they clamored to open everything back up and stick it to 'em. They are hypocrites and racists to the core.

Anything the GOP runs ceases to function. Not just through neglect, but active obstruction. The GOP is only only in power because they cheat, it's gotten steadily more extreme as their actual support diminishes, and as the Boomers go under it's turned into directly attacking the constitution.

Remember the writeup about why hospitals in red states are failing? People covering this still don't get it: It's not "paltry medicare reimbursement rates". Your state governor TURNED DOWN THE MONEY.

Here's a good thread about how right wing loons have been craving a power vacuum to fill, but they never get one because that's not how it works.

Evangelicals are dying of Covid-19 in "frightening numbers" because after denying it's a thing, they won't seek treatment when they get sick. meanwhile the Boomers aren't dying fast enough. The Resident is still a dementia patient and cratering fast.

Remember how housing gets suddenly WAY cheaper when airbnb shuts down? Here's hoping they don't come back.

Reopening at 1/4 capacity is like running an engine without oil. Have some more links.

All we have to do to end capitalism is stop bailing it out. This economy is fictitious.

We have plenty of food, capitalism is getting in the _way_ of producing and distributing it.

This administration wants people to die and they lie about it.

AOC's 70% tax rate is a good start but we still need guillotines. (And AOC sets it there to maximize revenue, but the 91% rate after WWII was explicitly to prevent plutocracy. The 1964 revenue act reducing the top tax rate from 91% to 70% was also about maximizing government revenue, and it's what allowed billionaires to re-emerge as a class and hijack the government to reduce the tax rate further. Taxes won't fix this, guillotines can.)


May 3, 2020

I have been asked via patreon to, and I quote, "please ease a up a bit on the political posts".


May 2, 2020

Every time I try to cut a toybox release, writing up the release notes always points out small polishing steps or reminds me of little loose end todo items, and I'm weak and do "just one more quick tweak", and they expand into large things as todo items are wont to do when you pull on a narrative thread. But I really REALLY want to get a release out today because it's another Patreon month and I want to tell them I did the thing.

Especially since this one has the merged make root booting to a shell prompt, which means it's both "work on toybox" and "work on build systems". Keeping my promises! But so much work when you actually try to make it load bearing. All the little things pop out at you...

The toyroot plumbing lets me greatly expand the test suite, to find things like "ls --color (a bare longopt) is acting as a synonym for -x on 32 bit systems". (I'm guessing because FLAG_color is (1LL<<32) and something is getting confused, but need to drill through lib/args.c to find it). The _symptom_ is that ls /bin is giving me multiple entries per line but ls --color /bin isn't, which I hit by hand in the new system but I haven't got a regression test for that because terminal width isn't a constant and the test plumbing isn't faking a tty. I need to do a lot of tests with a pty wrapper to check corner cases in top utf8 handling (such as proper fontmetrics when you have a username with combining characters)...

What I'm saying is my cans of worms _nest_, but I _also_ have a bad habit of accumulating todo items at about the same rate as I clear them at the BEST of times because doing the work reveals more work to do. (And then the main symptom of "I am overworked" swap thrashing is tabsplosion: so many things I had to leave off in the middle of so there's an open tab (desktop switcher has 8 desktops, each with enough windows xfce goes to a pulldown at the edge of the list, and then the Terminals have the scroll left and right arrows when they're showing more than 6 tabs...) and periodically I "close tabs" by moving them to one of my various notes.txt files so they can compost _there_ and don't get lost next time my laptop reboots. Usually this is because I finally have to reboot it for some reason, which takes a half-day to clear properly so I don't lose data. Or because the battery fully died and suspend-to-ram lost context.)

Let's see:

$ cat /proc/uptime
11514340.14 18323433.75

Um, I'm guessing first number is runtime and second is suspend time (in seconds) since last reboot, toybox date -d @$(($(date +%s)-18323433)) says I last rebooted near the start of October. Yeah, sounds about right. I should probably do that so the kernel has a chance to refresh itself for security whatsits.

But first, I need to finish xeno's release process...


May 1, 2020

Today is the first day of the crazy Texas GOP governor's bodies for billionaires campaign encouraging people to die for the economy, so I'm trying not to leave the house at all today. I've been a bad quarrantiner going to the grocery store 2 blocks away or the convenience store down the street almost daily (of course washing my hands and wearing my towel over my face), but given this new "leadership", today I feel guilty about walking to the end of the driveway to put back the trash can.

Drop everything, there is new Good Omens content. It is a good day.

Oh wow, the EU's fossil fuel subsidies may be going away. The two biggest oil producers in europe are the UK (which just gave up its voting membership via brexit) and Norway (which isn't a member), which means thanks to brexit no remaining EU member state is a significant oil producer. Elsewhere, the CEO of Shell admits that oil probably isn't coming back, and so do bloomberg analysists. (Oil hasn't got decades, solar and batteries should continue to get exponentially cheaper for the rest of this decade, and by 2030 it's over. Yes, even for gas stoves: induction cooktops heat faster than gas, and just like gas, the heat stops as soon as you switch it off. No bank wants to finance an oil industry whose graph has loops in it once a reasonable alternative has presented itself, and an INTERESTING point in that last link is the assertion that 3/4 of the US defense budget is protecting our access to oil.

I used to favorite tweets like this as a way to bookmark them so I could find them later. (Didn't work very well, but twitter never had good tools.)


April 30, 2020

So I'm ALMOST ready to cut a toybox release, but "make root" has one more hiccup left: if you're on a QEMU board emulation that hasn't got a battery backed up clock, the init script would call busybox's rdate and ntpd commands, which I've replaced with toybox's sntp command... And musl is returning "Function not implemented" from clock_settime(). Sigh. It's not a kernel config problem, musl 1.2.0 broke 32 bit platforms (I've tried sh4 and i686 so far) when doing its 64 bit time conversion...

Ah, it's CONFIG_COMPAT_32BIT_TIME in the kernel .config. On both i686 and sh4 musl 1.2.0 calls clock_settime32() which isn't there on current kernels unless you switch on the extra config symbol. (The 64 bit syscall is always available, the 32 bit one has a backwards compatibility gate. Why? Is this a musl bug or a kernel bug? Just call the 64 bit syscall when it's available, you can #ifdef the syscall in the kernel headers! In the name of "compatibility" you're failing to run on the kernel I built.)

I keep going down shell ratholes, such as:

$ env -i bash -c env
PWD=/home/landley/jcore/sh
SHLVL=1
_=/usr/bin/env
$ env -i ./toybox sh -c env
SHLVL=1
SHLVL=1

During which I had to tweak the xpopen_both() plumbing to be able to take seperate "path to executable" and "argv[0]" arguments because the shell has to do the path lookup (not the libc execlp() function) in order to know what the path IS and I refuse to allow the race condition of doing it twice and somebody maybe changing it in between for nefarious reasons, nor do I want argv[0] to always be an absolute path when that's not what they typed. (Although it turns out I've already added a callback happening in the new process context, and I can just have THAT do the exec instead, so it all works out but I had to sleep on it to come up with that. Oh, and speaking of which I _could_ try the getenv("_") path and fall back to /proc/self/exe when I want to re-exec self in nommu context. Or more likely vice versa...)

Oh, and if the file you exec is a text file without #!/blah at the top, then both run and exec treat it as a shell script, and basically call sh except bash does it internally.

$ cat > snork <<< 'echo hello $BLAH'
$ chmod +x snork
$ bash -c 'BLAH=123; ./snork'
hello
$ bash -c 'BLAH=123; exec ./snork'
hello
$ bash -c 'BLAH=123 exec ./snork'
hello 123

So that's fun, and I need to implement it.

But because I was taking so long I offered to throw busybox ash into an image I'm making for work to give them a temporary thing to test with, and then replace it with my shell before we ship. And Jeff has a point of pride he's never shipped busybox in anything (he doesn't like its code quality; keep in mind he's the guy who hired Erik Andersen to MAKE busybox in the first place so there's some backstory there I'm not entirely privy to but about 5 years later the corporate context he did it all in became Caldera which became SCO which sued Linux, and apparently the dysfunction was already highly apparent while he was there...)

Anyway, so Jeff sent me the shell he used in uClinux back in the day, which is simple! Tiny! (6 C files and 1 header file! 4968 lines total!) And... wrong.

host$ env -i ./sh -c 'env'
PATH=:/bin:/usr/bin
HOME=/
SHELL=/bin/sh
host$ ./sh
$ echo $(echo hello)
syntax error
$ echo `echo hello`
host$

Here I was going "lack of command history is an issue", and this hasn't got any either. (Cursor up gives you "^[[A".) The main lack seems to be terminal control, and if I'm NOT doing job control (no suspend/fg/bg just ctrl-C) then... yeah, I can do that?


April 29, 2020

Oh hey, Nancy Pelosi noticed that Basic Income is a thing. Thank you for joining the group. (Saying that Nancy Pelosi is a trailing indicator is like calling the Pacific Ocean a damp spot, she's 80 years old. As with impeachment, she strikes several days after the iron has completely cooled.)

I emailed Mark Blyth (he's a professor, his email's on his faculty page) after listening to one of his recent podcasts, which made me realize something at around 15 minutes in:

Your most recent Mark&Carrie podcast made it sound like the USA's global reserve currency status, with corresponding ability to run infinite deficits, is like countries that depend on oil or diamonds for the majority of their income.

The literature calls this the "resource curse".

They don't need the productivity/taxes of their populace to sustain themselves or the quality of life of their elites. They can just import everything they need using the independent revenue stream, which predictably results in repressive totalitarian regimes keeping the populace out of the way of the elites and their revenue source.

Thanks for your time,

Rob

To which he replied:

Hi Rob

Brendan Greeley at the FT is writing a book arguing just that. The US is a commodity exporter. The commodity is the dollar.

best

M

In his next podcast he mentioned the topic briefly (by its older name Dutch Disease), and this explains SO MUCH. (He also read a question I submitted, including the part about solar killing fossil fuels in solid/liquid/gas order.)

As Bylth explained in a pair of talks recently, the world switched off the gold standard last century (because "sound money uber alles" caused the Great Depression, leading to mass unemployment and the rise of fascism). During World War II the USA outproduced the rest of the world, winning the war by producing more guns and bombs and planes and ships and food than all its enemies combined. (Having our fleet wiped out on day 1 at Pearl Harbor proved a minor inconvenience, we just built a bigger one.)

After the war, everybody wanted to buy from us, so everybody wanted US dollars. So the dollar became the global "reserve currency" that other countries filled their vaults with to back their own currencies. Instead of piling up gold, they piled up dollars, because they could buy anything form the country whose bounteous production won the great war.

But that massive US manufacturing capacity hasn't been true for 50 years. Starting in 1980 Ronald Reagan gutted FDR's New Deal regulations to hollow out our economy. The top tax rate went from 92% to 28%, money started to pool in billionaires' pockets, and Milton Friedman convinced corporate executives that their duty was to their shareholders rather than their customers and employees (a thing that was not true before September 13, 1970). American corporations outsourced their core business to third world sweatshops and call centers in india, and manufacturing re-coalesced in China.

Under successive republican administrations the USA gradually lost the ability to make anything or do anything other countries would want to pay for, but the dollar's position as the global reserve currency didn't change. Vaults full of dollars weren't any MORE useless than vaults full of gold had been, so piling them up to "back" other countries' currencies... everybody still did it because everybody else was still doing it.

And this meant that as the world's economy grew (population growth, inflation, technological advances, piling up savings) the rest of the world needed more dollars so they could issue more of their _own_ currency (because if your economy grows from $100B/year to $200B/year you need to issue twice as much money to avoid liquidity shortages), which meant the USA could run endless trade deficits. In doing so we provided _demand_ to the rest of the world, allowing everyone else to have export-driven economies because we were the importer they all sold to. It's a great deal for us, we don't have to do anything and people send us stuff in exchange for our autograph.

(We do still export food, but so do lots of other countries. And except for "factory farms" where we've fully mechanized production of grain and milk/eggs/meat, we import foreign labor to actually tend and harvest all the other crops without which the food rots in the fields. We still have the land, but have completely outsourced the expertise (to John Deere and Monsanto, or to Mexicans). Not that this is anything new, it's another facet of the US's profound racism from the trail of tears to slavery. The people calling themselves "farmers" are really landlords, they own the land and hire others to work it for them.)

This brings us back to the resource curse: we export a commodity, the US dollar, which 99% of the US population cannot help produce. Neither our labor nor our tax revenue is necessary, the USA can run enormous deficits forever because other countries want our treasury bonds. Our financial elites print money endlessly, the banks invent endless derivatives out of nothing, enriching the 1% while the 99% starves. Just like the oil and blood diamonds from Angola, the USA has become a puppet state where the government literally doesn't need the people as long as it has the financial industry.

Oddly enough, this ISN'T the reason we can afford basic income. These days _most_ countries can. I've written written lots of articles on basic income where I ran the numbers, but the tl;dr is that over the past century subsistence work has been automated away. 200 years ago 80% of the population worked on farms and these days (since the invention of the tractor and the combine harvester, the haber-bosch process and Norman Borlaug's dwarf wheat) only about 1% does (mostly the 3 million migrant farm workers picking fruits and vegetables, whom the white "farmers" call ICE to round up and deport right before payday). Domestically we can mass produce cheap wheat, rice, soybeans, corn, milk, beef, eggs, and chicken using chemicals and fossil water and cruelty, but the result is food is cheap and plentiful.

There legitimately was a time when everybody had to work or we couldn't eat, but that hasn't been true in living memory. Back during the great depression Milton Friedman proposed we'd work fewer hours each year, and he was right. The advance of technology automated subsistence work away, to the point lack of work is a recurring crisis. Between railroads and interestate highways, shipping containers, electricity, computers, the internet, GPS satellites, antibiotics... the economy we have now is nothing like the economy of 200 years ago. The pony express took 10 days to convey a letter 2000 miles, the telephone turned that into a few seconds work for an operator to connect the call, and then we automated away the operators most of a century ago. Repeat that in 1000 niches and you get the modern economy. Instead of employing domestic servants we've had dishwashers, washing machines, and microwaves for 50 years. Today we're getting solar panels, self-driving cars, and 3D printers.

The reason we haven't had the increase in leisure time Freidman predicted is our rulers made up bullshit jobs to keep people busy. (And if you work in human resources or IT for a company whose top floor is lawyers or marketing exectives that bring in the revenue from outside that pays for your work, then you have a second order BS job where you do real work in _service_ to bullshit. That's how 90% of the economy can be literal make-work.) The rich resisted any change to the social order where the rich employ the poor, and thus control them. Every rich person has a retinue or they're not important. "How important am I? I have 50 people report to me." (Doing what? Whatever I tell them to. Being at my beck and call. Public school assigns you homework, your day job assigns you busy work along the same lines. Not enough to do? Time for an ISO-9001 compliance audit, six sigma, a reorg, IT upgrade and training...)

And then if you ADD the global reserve currency on top of that, it means unequal distibution of the fruits of labor isn't even the main reason billionaires are soaking up trillions of dollars. Banks literally just invent money, and get bailed out by the federal reserve's printing press if they ever get called on it. There is SO much spare money, it's hard to even measure.

The US left the gold standard in 1971, and by Reagan's election in 1980 the world had coalesced around the US dollar as gold's replacement, giving us the resource curse. Today billionaires get rich because we print dollars. Whatever the 1% does with technology or inheritance or hedge funds is an excuse to _collect_ enough dollars to join the billionaire club, and once in members are endlessly bailed out with freshly printed money so they can't ever fail their way out of it. Once you're in the good ole boys' club you're a different KIND of person, who is not allowed to fail. It's the modern aristocracy.

The Boomers don't see this because the world they grew up in after World War II was the US as superpower, our vast wartime production having defeated the entire world (and invented the ultimate weapon, ICBM delivered nukes literally capable of destroying a city on the other side of the world with the press of a button). Boomers took this for granted, as the natural order of things, so did nothing to preserve it. They fought One Big Enemy that was the source of all evil, like their parents did, and then once that fight was won history was over and they lived heappily ever after. Instead of building or preserving anything, they clearcut and strip-mined their country and society. Now the idea that basic income + medicare for all + rent control is something the USA could _easily_ do is anathema to Boomers because that's just not how it's DONE! (Clutch pearls. Swoon and faint.)

And so we're waiting for them to die, while our financial elites sling around trillion dollar bailouts that get sucked away by billionaries because the current system is rigged for money to flow upward, never downward. But the current system is completely fictitious, the money is printed every time a Billionaire is feeling poor, and 90% of the "work" we're all forced to do is BS accounting games, with most of the rest (haircuts and grocery stocking and burger flipping and garbage collection) consisting of caring for other humans which is paid and valued like dirt (as any housewife will tell you).

It's not real. The Boomers only think it is because they haven't had a new thought in 50 years. Let them die, then guillotine the billionaires and start over.

This system crashed and had to be bailed out in 1991 (savings and loan crisis), 2001 (dot-com bust), 2008 (mortgage crisis), and it's doing it again. If they duct-tape late stage capitalism back together one more time, the wheels will just fall off again in a few years. The mass deaths we're experiencing right now are just a more OBVIOUS form of begging on gofundme for insulin or dying from preventable everything because a hospital visit where they don't FIND anything (she thought it might be a kidney infection, it wasn't, they sent her home) costs over $6k. (Real figure from a friend last month, it's gone to collections because she can't pay it.)


April 28, 2020

Entire lamb legs are available at HEB for $2.99/lb, cheaper than brisket. We have a chest freezer, so we've bought 8 of them so far. Had one for dinner (well, cooked one and had like 1/5 of it for dinner, and the weather report in the kitchen is a post-thanksgiving level of "endless sandwiches"). It's lovely for us, terrible for the farmers. HEB has kept its supply chain intact (with the occasional massive sale as they have more of certain perishable items than they know what to do with), but I'm told large parts of the rest of the country are starting to have... issues.

Twitter remains bad at being twitter. (I have NO idea why you have to scroll _up_ to see tweets this week. They changed their HTML plumbing so the anchor it scrolls to in each new page it loads is _AFTER_ the tweet the link is to, so it starts by showing the first reply and you don't even know there's stuff above that unless you try to scroll up.)

Apple's technology is really going downhill since Steve Jobs died and Johnny Ive left. Here's a good thread on how means testing is terrible, which starts with a link to another good thread on how means testing is terrible. This is the best summary of LBJ I've ever seen, in a single tweet.

The dementia patient in chief is now calling every senior moment "sarcasm". And he's flounced from his diet rallies. Even snopes is calling him out now. Meanwhile the republicans are actively trying to kill people for money. Despite the suppressed covid-19 figures (if they weren't tested, that's not what they died of), so many people are dying newspapers are running out of obituary space, and it's getting worse. Conservative governments everywhere are lying about their PPE stockpiles, but lying is what they do. The Arizona GOP chair instructed the anti-lockdown protestors (paid for by education secretary Betsy DeVoss and her brother Blackwater CEO Eric Prince) to dress like health care workers. Conservatives think the rules don't apply to them. And yes they caused this problem in the first place but they're also also screwing it up now.

The "reopen" push is still about denying people unempoyment benefits. And about preventing insurers from paying out, and letting landlords demand rent.

The true "death panels" are capitalism driving healthcare decisions. And the GOP's drive to "drown the federal government in a bathtub" seems pretty close to succeeding. The "family values" party is also destroying marriages en masse.

Back when Bill Clinton proudly "doubled the number of cops on the street" it was a bad thing.

Our current economy is entirely fictitious. The massive food waste going on is due to distribution monopolies, and efficient still means "brittle" and "all the downsides exported to poor/brown people".

Milwaukee is a nice city. I enjoyed living there. (I had zero social network there, but as a _city_ it was quite nice.)

This thread on what a terrible person Elon Musk is includes an article written by his first wife.

Michael Moore (who is 66 years old) put out a hit piece attacking renewable energy. For example, if you switch to electric cars you _can_ power them with solar and wind, but if you replace your infrastructure in stages and the electricity's initially still coming from coal (because the switch isn't FINISHED yet), you are an imperfect impure horrible person and should never have bothered to do anything ever. (Especially since Moore uses 10 year old figures on where your electricity used to come from in that particular example.)

Misogyny is alive and well.

Billionaire Philanthropy isn't Robin Hood. They steal from the poor and give a small fraction of it back to buy praise.

Still waiting for the Boomers to die.

Floating storage now costs $250,000 per day to store oil, ten times what it did last month.


April 27, 2020

Ha!

$ make CROSS=sh4 LINUX=~/linux/linux root
...
$ (cd root/sh4; ./qemu-*.sh)
sh: exec rdate: No such file or directory
sh: exec ntpd: No such file or directory
sh: exec /etc/rc/*: No such file or directory
Type exit when done.
# ls /sys
block  bus  class  dev  devices  firmware  fs  kernel  module
# exit
reboot: Restarting system
landley@driftwood:~/toybox/toybox$

It works.

(Admittedly the WAY it works is by ignoring the ${i/,*/} and ${CONSOLE:-console} decorators, but that's just missing features.)

Sigh. In libgcc, ssize_t is long but in musl it's int, which is silly: on 32 bit platforms long is 32 bits, and even the one surviving exception to LP64 (winders) breaks small (their long is 32 bit on 64 bit platforms), so defining it as int is silly. The problem is it causes printf("%ld", strlen(blah)) to throw a gratuitous warning.

Aaaaand 32 bit bionic is doing it to in some versions. Great. Pointless gratuitous typecast it is.


April 26, 2020

Wow. In 1986, George Carlin billiantly summed up the Trump administration in 10 seconds (4:59 to 5:09 in that video). It's a pity I dunno how to clip that out as its own clip.

Yes the resident is still cratering from dementia but people still don't see it even though he clearly doesn't remember last month. Election or no, his hard drive's failing and gonna brick itself on its own real soon now.

I suspect Saudi Arabia browbeat its neighbor Jordan into unplugging all its solar panels from its electrical grid, to force it to consume more oil (These are solar panels that exist and are producing electricity now, why ELSE would you unplug them all?)

Covid-19 has now killed as many people in the UK as The Blitz in 1940-41. And that's despite Covid-19 deaths being vastly underreported everywhere conservatives are in charge. But a LOT of people are dying from it. (It reminds me of the star trek episode with the disintegration booths. The casualties of a war, while we all quietly sit at home mostly personally unaffected by the widespread but silent horror.)

Here's a fun new hobby everyone should try. Guess why Labour lost the last election in the UK? (The main opponent of change is moderate old fogies who sold out decades ago and today refuse to acknowledge improvement was never impossible because it means they wasted their life. Pelosi and Biden are the problem, not the solution.) Meanwhile, on the Tory side... which is probably why... Honestly, just guillotine the billionaires.

Why the economy can't "reopen" partially.

An amazing amount of oil is parked off the coast of california right now. Saudi Arabia bought the largest US oil refinery, and intend to run their oil through it whether we need it here or not. Meanwhile we're turning even oil pipelines into storage even though texas normally produces twice as much oil per day as the pipes can store.

Banks rush to rein in financing for oil firms, the article estimates at least 30% less capital available to oil firms already.

The New York Times has issues.

The trump administration is now stealing masks from the Veterans Administration hospitals, embezzling them to sell on the open market. They also appear to have stolen every stimulus check in the whole of Puerto Rico. The republicans not only designed the "stimulus checks" so debt collectors can grab the money before you ever see it, but they're providing pointers on how to steal your check. And even if we got it all, it's a tiny fraction of what functioning governments are providing their citizens. Nobody anywhere else is screwing up as badly as the GOP.

Guillotine the billionaires. Countries that end capitalism will come out of this much stronger. Just do basic income, medicare for all, and rent reform (tax landlords white). We still need public infrastructure, but the current clowns are crit-failing it all today. The post office is like the fire department, police, public schools, orphanages, and hospitals: if it makes a profit something is wrong. Talking about hospital or post office profits is exactly as creepy as talking about fire department or police profits. (And yes, police departments are making huge profits and it's always been a bad thing when that happens.


April 25, 2020

The fiddly bit about redoing so much toysh plumbing to change the order of operations so I can implement all the ${blah#blah} decorated variable resolution plumbing is getting the lifetime rules right. I want to free stuff promptly, and don't want to leak memory, and it's fiddly.

Sigh, I'm going to have to throw a memory tracker at this to find leaks.


April 24, 2020

Does anybody remember the Vox piece from 2 years ago about how Trump clearly has dementia but news organizations consider it unprofessional to talk about it?

The emperor has no brain. It's a problem, but not a secret. It never was a secret, and it's getting worse fast enough people can no longer ignore it. But they still prefer to call him crazy or evil rather than admit a 73 year old man whose father died of altzheimers is sundowning into senility.

The daily coronavirus rallies show the Resident's worsening dementia, not just in specific statements but in patterns of dodging questions he literally can't answer. Here's video where he admits he can't remember what happened last month. People usually attribute this to lying, but as Ronald Reagan's Iran-Contra "don't remember" before dying of Altzheimer's shows, it can also be a degenerative brain disease. Remember that his (remaining) staff regularly lie on his behalf to try to placate him and justify things he said, which means things like "sharpiegate" don't necessarily mean HE drew on a map, it means when one of his staff creates an obvious lie to cheer him up, they fool the senile old man so completely he thinks other people will also find them convincing. He recycles his speeches because he can't absorb and retain new information. When the question a reporter just asked him vanishes into the fog, he pulls out the same old stories over and over like any other geriatric invalid in a nursing home.

He's losing the ability to walk, the day before yesterday he lost his balance and used Melania as a cane twice in 20 seconds (his right arm doesn't swing and his right leg is buckling). And this is _with_ his handlers drugging him to the gills before each public appearance, which is not sufficient, even with a teleprompter (which he can no longer reliably read). It was obvious back in 2017 that this is not a healthy 70+ year old man.

The GOP will never reign him in because they literally want to destroy the federal government, and see his incompetence as a useful tool towards that end. After all, billioniaires can't be kings hunting peasants for sport and sex trafficing a harem chained up in their basements if they have to follow OSHA regulations. (If that was hyperbole how did so many rich and powerful people hang out with Eppstein?) They want us to collapse back into feudalism, which is not hyperbole and not a secret.

Speaking of the GOP, the reason they're reopening Georgia is to avoid paying unemployment, because its right wing loon governor Kansas-ed the state budget and now it can't afford to provide basic services. The "reopen" protests are astroturf. The "reopen" protests are astroturf, and they're also tiny.

I miss having a reliable media ecosystem. Even NPR seems to have faceplanted recently. The situation has been deteriorating for a while but over the past 5 years facebook manipulated the market metrics and bankrupted many outlets. The ones that are left are often so self-destructive they want to make sure nobody sees their content.

This will end when billionaires are guillotined. It will not end before billionaires are guillotined. There's no such thing as a "good billionaire". If they weren't slimy grasping bastards happy to watch people die rather than give up money, they'd never have accumulated so much. (And not just performative philanthropy that exerts control over people and institutions, or a pledge to give (some of) it up when you die. Scraping together that big a pile cannot be done by one person, it means the fruits of your collective labor were not shared remotely equally. You profited and they did not. The CEO should not get a thousand times what the employees get. They didn't used to, back in 1965 the average CEO only made 21 times what their workers did.)

(And yes I linked to a thread where Walt Disney's niece rips the company a new one for laying off 100,000 workers without cutting their excessive executive pay. Current estimates are that Disney is losing $30 million/day right now, which is a $billion/month, which they can afford for about a year.)


April 23, 2020

Expect more of this. The opposite of solar "curtailment" is having demanding loads run dynamically: Aluminum smelting, data centers, manufacturing lines, chemical synthesis (such as making jet fuel from electricity), and so on. Charging up batteries is sort of the null case of this; pure storage is just the start, not the end, of figuring out what to do with an energy surplus. The price of solar panels dropping exponentially has already greatly expanded capacity, but why would it stop at 100% of the OLD power level? We've already gone from 2% to 20% of the power we generate, the next step is going from 20% to 200%, then from 200% to 2000% of what we _used_ to generate. That's how exponential curves work, we didn't just REPLACE mainframes with an equivalent amount of PCs and call it a day. Curtailment and negative electricity prices are relics of the old obsolete system: instead figure out how to use the energy when it's available. Hook up heavy dynamic loads that run while the sun shines and the wind blows, and idle when they don't.

Shell stuff (toysh). So this is annoying:

$ chicken() { echo ${*/b c/ghi}; }; chicken a b c d
a b c d

Although $* expands to include "b c", it hasn't done so YET when the search and replace runs. This requires the sequencing to be that each argument has the regex applied BEFORE they're stitched together. Which with $IFS being variable makes a certain amount of sense, but fights kind of profoundly with the plumbing I've implemented for handling variable expansion. I can stick the ${??} shenanigans in right before IFS substitution, but otherwise I have to make it a function that stuff calls out to, which means marshalling state to it.

Grrr. Alright, the logic wants $@ and $* to get handled in the same route, but the problem is the outputs are very different from an allocation standpoint. I really wanted to do two passes over the data (measure, allocate, then write) but I guess it just has to realloc() a lot. Which is potentially very inefficient but CAN work ok? Sigh.

The other problem is:

$ IFS=x; X=x; eval abc=a${X}b
bash: b: command not found

I want eval to use expand_arg("\"$@\"") to get the glued-together-string (so I can inline the function gluing it together), but it has to use $IFS within each argument but NOT glue them together with the first character of $IFS. Time to add another expand flag.

$ chicken() { for i in "${@:3:5}"; do echo =$i=; done; } ; chicken ab cd ef gh ij kl mn op qr
=ef=
=gh=
=ij=
=kl=
=mn=
$ chicken() { for i in "${*:3:5}"; do echo =$i=; done; } ; chicken ab cd ef gh ij kl mn op qr
=ef gh ij kl mn=

So ${12:1:3} needs to lookup ${12} first to chop the right part out of the result, but ${@:2:3} is operating on the _array_ which then gets stitched together into a result afterwards. (Working out the sequence here is the hard part. Once I've got the order of operations the CODE is mostly just a bunch of typing.)

Ah-_ha_. Bash arrays are genericizing the $@ and ${!blah} results, THAT'S why they exist. So a bash array needs to be a struct sh_var and fall into the same codepath as the others. (For the moment I can implement associative arrays as normal arrays with key=value contents I search with a for loop, possibly reusing the variable infrastructure I've got for handling key=value arrays. Worry about that later.)

$ IFS=x; X=abxcd; echo ${X/bxc/g}
agd

Yup, regex _then_ split. Time to shuffle code around. (Remind me to add all the above to the test suite.)


April 22, 2020

Oil went negative briefly, because storage ran out, but this was just the opening band. The real crash is still coming.

What happened is the monthly futures contracts expired tuesday, so instead of trading paper contract rights people had to take physical delivery of the oil, and traders who hadn't made arrangements to rent expensive storage found nobody wanted to take it off their hands, and if they still owned it come tuesday they had an obligation to _take_ it, or they were in breach of contract (and possibly liable for an oil spill). On the last day of the contract, trading turned into musical chairs and the contract became a hot potato people paid to get RID of before storing the oil became legally their problem.

The result was a localized trading glitch, only applying to WTI oil from landlocked wells in North America. (Remember how it was just one well trading negative last month? Well this time it was "just North America".) The reason the price went negative is oil transportation got overwhelmed and storage filled up. The pipes and rail lines from North America's oil wells go to refineries, but the refineries are all operating at reduced capacity because nobody's buying the resulting petrol/diesel/kerosene/jet fuel, and there's never been much storage for outputs like gasoline. As long as they have crude oil stored near the refinery, they can always just refine more if they need it. You can't put gasoline into crude oil pipes any more than you can put it into a milk tanker, sewage truck, or cement mixer: each liquid needs its own dedicated storage and transportation or everything gets contaminated. Since crude oil is what all the rest is made from, they mostly just store that.

With the refineries overwhelmed crude oil is piling up, storage near the refineries is full, and there aren't a lot of crude oil pipes or tanker train cars going AWAY from the refineries because why would there be? (America used more oil than it produced for decades, most pipes lead _in_.) And thus the price of WTI crude turned negative as the music stopped and there weren't enough chairs. The futures contract expiring forced it to happen for a lot of traders at the same time, so the price they traded it at went suddenly negative.

But A) that contract's expired now and the next one doesn't come due for a month, B) WTI is a minority type of oil mostly only relevant to North America. The rest of the world (2/3 of global supply) is "brent" (named after a British Petroleum oil platform off the coast of Scotland, but generally used to set the price for the "basket" of middle eastern states as well), and THAT oil stayed around $20/barrel. The thing about brent oil is those producers all pipe it to sea to fill up oil tankers which take it to refineries, which means those tankers can head anywhere in the world that still has storage space.

But by this time NEXT month, when the next futures contracts expire and people have to take physical delivery of next month's oil, how much storage will even Brent be able to find? An armada of oil is heading to California for delivery in may. The tankers _themselves_ are already being leased as "floating storage". And brent oil having the kind of seizure WTI just had is a MUCH bigger deal for the oil industry.

The interesting thing is, oil already had a glut problem to the point it needed floating storage three years ago. Global demand for oil wasn't keeping up with supply even before coronavirus. The industry even has a term for this, "contango", which means the price you can sell oil at in the future (according to the futures contracts) is higher than you can buy it for now, and if the gap is big enough it pays to store it until the price goes up. This means the market believes demand for oil will increase in future... except it's believed that for years now. (And they're recently doubled down, inventing "super contango" which I can only assume yells a lot and has pointy yellow hair.)

Oil traders are hoping that the economic damage from coronavirus goes away as quickly as it came (the so called V-shaped recovery as opposed to the persistent U-shaped recovery everybody expects). Because if digging out takes a while, demand for energy won't return to previous levels for years, and by that time how much of it will be for _oil_ if solar and batteries continue their exponential price declines? (So what if oil prices fell 50%, batteries are 87% cheaper than they were a decade ago, and the technology continues to advance as fast as ever.) Therefore it can't possibly happen, because rich white men don't want it to, QED.

That floating storage article from 3 years ago article says the oil traders expected a surge in oil demand from china to raise prices, but I wrote about that last year: china's the biggest source of solar panels and batteries driving the switch to renewables. Collapse in demand from china killed the coal industry most of a decade ago, and china capping its strategic petroleum reserve caused the oil glut in that article.

Oil traders are still hoping for a surge in demand from China today, only now they're hoping it builds ghost oil storage. As late stage capitalism winds down and billionaire parasites suck all the money out of everything leaving workers with flat wages and piles of debt and a crash every 10 years where the rich are bailed out and everyone else loses their homes and careers and retirement savings, the resulting economies suffer persistent lack of demand requiring perpetual stimulus to keep their heads above water. In the USA we do "Quantitative Easing" which means the Federal Reserve prints money and uses it to buy trillions of dollars of paper assets from wall street, so each dodgy get-rich-quick scheme du jour the billionaires come up with gets bailed out as it collapses, so the billionaires never lose money on any of them.

China's version of Quantitative Easing takes the form of hiring construction crews to build "ghost cities", creating "assets" their middle classes can buy from the billionaires. (Real estate scams are as old as the hills.) The leaders of china believe in physical assets, but empty skyscrapers in the middle of nowhere are their own kind of fiction, and when that got ridiculous they turned outward to the "belt and road initiative" building random infrastructure in adjacent countries in hopes those guys might actually use them instead of letting empty buildings fall down. (And also a way of putting adjacent countries, and ones farther away, in both political and financial debt to them, and buying large chunks of their real estate with finance instead of military invasion.)

So it's possible that if oil prices get low enough china will decide to direct the ghost city construction crews it throws its stimulus money at to build oil storage for a while instead, and fill it with cheap oil at under $10/barrel. It's a bit like me buying a chest freezer if T-bone steak goes on sale at 10 cents/pound. I didn't have _plans_ for that steak, but if it's cheap enough and there's enough of it, sure why not. But if I did that, I probably wouldn't buy steak again for quite some time, and that's the problem the oil industry has filling up all this storage. Renting oil storage from somebody else costs a lot of money (same as renting a U-haul or Public Storage), and that motivates people with oil in storage to sell it, eventually even at a loss if the price stays low for long enough. This means the price of oil can't go back up anywhere near where it used to be until it's worked through the backlog, but it can't work through the backlog while oil producers are still pumping. All the storage in the world doesn't fix the problem, it merely delays it (while making it worse).

And oil producers aren't going to stop. The cost of production is complicated, but taxes can be waived at any time (especially for state owned oil firms or as part of a bailout), and some portion of "capital spending" was already paid (financing debt from sinking the well, refinanced or eliminated via bankruptcy), which leaves the "production" and "transport" columns. This means US shale physically costs somewhere between $9.50 and $17/barrel to deliver, with Russia $6-$11 and Saudi Arabia $5.50-$9. This means even at $10/barrel, the gross margin is positive and whole lotta pumping going on from people desperate for cash.

Russia still needs $42/barrel to balance its budget and Saudi Arabia twice that. They'll keep pumping from the wells they've already drilled and running it through the pipes and refineries that still exist as long as there's any profit in doing so, but no bank with two brain cells to rub together is ever going to finance another oil construction project again. It's stranded assets as far as the eye can see, and that lack of new capital spending is likely to be the oil industry's limiting factor this decade. More leaks. More worker injuries. Skill loss due to layoffs and attrition.

And with solar and batteries and telecommuting and electric cars eating away at their customer base, bailout money is likely to vanish into executive bonuses and stock buybacks rather than be invested into a dying cash cow (as with Sears: milk to exhaustion then butcher). Banks lost their shirts on oil in the 1980's because of exactly this kind of price crash (prices stayed low for 17 years, until China started filling its Strategic Petroleum Reserve), and if it takes them another generation to forget their losses renewables should have swept the field by then.

The next question is, of course, what happens to Russia. The 1980's oil crash caused the collapse of the Soviet Union about a decade later: without importing food Russia can't feed itself, and they need to export oil to get hard currency to import food with. (Even in 1997, The Onion often merely documented the truth.) As oil prices went back up, Russia started meddling on the global stage again. They've switched from oil to gas for funding their troll farms, but fossil fuel money directly funds 36% of their government and indirectly it's and over 60% of their export income (down from almost 70% a few years ago because oil prices have come down). Russia's gone thorough brutal austerity that's killed their (admittedly meager) economy to try to build up some cash reserves, and right as they were starting to spend again to suppress political unrest the oil price collapsed six months later. And that's before coronavirus' direct impact.


April 21, 2020

So, shell corner cases.

Not only is ${10} a thing ($1 through $9 show the arguments to the current function, but if there's a double digit number you need the brackets), but ${_} and ${$} and so on are synonyms for $_ and $$. And ${12a} is an error, but ${12#} isn't.

Which means A) I need a lot more tests, B) my variable parsing logic is in the wrong order and I've got to redo a chunk of code. Hmmm.


April 20, 2020

Of course Joe Biden is letting this crisis go to waste, even though the quote "never let a good crisis go to waste" was popularized by Obama's first White House Chief of Staff his first year in office. In case it isn't obvious: Biden is not Obama, is not as good as Obama, and learned nothing from Obama. (And Obama was to the right of Richard Nixon, and yes Obama himself said that.)

Biden is 77, which means he'd turn 80 in his first term. Biden is well past wanting anything new (old dog, new tricks). Even Bernie Sanders was trying to mold the world to fit a vision he came up with 40 years ago (which is not so much a vision of change as a supervillain plot that might coincidentally be useful to other people).

But don't worry, there will be another crisis as long as the Boomers are still driving. They caused the savings and loan crisis of 1991, the dot-com bust of 2001, the mortgage crisis of 2008, and the current Trumpocalypse (which is only a giant crisis because it was utterly mishandled and still is; places that scaled up testing early never even had lockdowns let alone mass deaths.)

We've still got 14 years of Boomer time left. Half of all Boomers were born before 1955, the average lifespan in the USA is 79, so half of them won't have died until 2034. And no Coronavirus isn't changing that (there are around 77 million Boomers in the USA and Coronavirus isn't expected to kill even 1/10th that many.) The Boomers will continue to vote Nazi from their nursing homes until their dying day. And they vote for people older than themselves (I.E. "adults") which is how we got 5 septuagenarians vying against each other for the presidency back in January.

Oh hey, I'm back to angry. Angry is useful.


April 19, 2020

Email's back. I didn't do anything, it just started magically working again. Something at Google's end. Maybe it'll go out again, maybe it won't.

I have a new $25 Patreon donor, who wrote a very nice letter I should thank him for, saying the work I do is appreciated and he wants the bash replacement shell. I should thank him, but I have to log in to Patreon to do that and I haven't got the energy.

The guy who started the thread that kicked me into depression sent an apology. I should reply to that. (It's not his fault. The trumpocalypse quarantine has not been good for my mental health.)


April 18, 2020

Slept most of yesterday, and most of today. Going back to bed.

Planet's still a trash fire. (I didn't know the guy behind Blackwater, Eric Prince, is Betsy DeVos's brother. And that they're the people who funded Mike Pence's political career. When Dubyah talked about the "axis of evil", log in thine own eye dude.)


April 17, 2020

And now my email doesn't work. Great. (I've been half expecting gmail to stop working ever since they warned me it was joining the Google Graveyard last year. And yes it's ironic that the google graveyard is itself no longer maintained.)

Thunderbird's been giving me a "web login required" popup trying to fetch email from gmail's pop severs all day. But logging in to the web page doesn't fix it. I can see the mail _there_ but can't do a pop3 transaction in thunderbird to get it into the client I use with the filters to put 3000 daily linux-kernel and qemu-devel and such messages into relevant folders instead of one big inbox. The server error message popup also provides an utterly useless URL which doesn't relate to web logins at all. Exiting and restarting thunderbird let me check mail once, and then the popup came back. Outgoing mail was also being refused but switching from smtp.googlemail.com to smtp.gmail.com seemed to fix it? (But then I only tried to send one message, maybe it also only fixed it once?) I googled for the error message but all the hits are 3 years old and I haven't changed my config.

I should migrate the domain to dreamhost's mail server, I just... don't have the energy? I'm supposed to be using a work email address at $DAYJOB anyway, and if I really need to contact house repair people or something there's always the burner gmail account Android requires me to create every time I get a new phone. (Old D&D characters from middle school are a good way to find unused account names.) That email address was my 4th since college anyway, they don't last forever. I guess I don't really _need_ to reclaim it?

What I needed that email address for was open source development, and... do I still want to do that? I haven't unsubscribed from the linux-kernel mailing list but don't read it anymore. The old busybox/uClibc nexus died a decade ago. Eric Raymond went crazy a decade ago, and Richard Stallman a decade before that. (Heck, RMS got #metooed out of office recently, just in time for the 67 year old man to collect social security, which means the FSF probably isn't even in need of opposing anymore.)

Google _officially_ doesn't care about toybox or mkroot or 0BSD or anything I've been working on, and never did. They've been willing to accept work done for free, and that is the extent of their interest. (One developer does care in a personal capacity, but according to this interview (starting around 19 minutes) he's A) just as happy to use the BSD utilities, B) affably baffled by my crusade to change the world, C) a manager these days and not a programmer.) If I stop, Google won't care, so why should I? They think the future is java, they rewrote their C library in C++. All my big long-term plans were _my_ plans, not theirs. Android isn't a simple reproducible, understandable, and security-auditable development environment because Google doesn't WANT it to be. If I stop, why would they even notice? (And if I can't tell them because _they_ screwed up the email account I'd use to tell them, I refuse to feel guilty about that.)

Mail going to the old account about initmpfs or cpio not supporting xattrs isn't really my problem anymore. The whole 0BSD thing was so deep in the weeds, who else even knows it exists let alone cares? Fighting back against systemd undermines Red Hat's embrace-and-extend business model, and they're IBM now. I haven't had time to work on QCC in 12 years, at some point "intending to get around to it" stops meaning anything. Current versions of Android don't let you run locally generated binaries and that whole "posix container" idea I had was just a wishlist item that Google hasn't remotely commited to actually do. It's a red queen's race.

But the bigger question I'm asking myself is... Why did I bother? What exactly have I been trying to accomplish? So kids these days learn Python 3 and everything under that is a black box nobody understands anymore, which can only be accessed by giant teams at Fortune 500 corporations guarding proprietary secrets. Oh well. These days Montsanto patents genetically modified corn, lets the pollen blow into neighboring farmers' fields, and then sues those farmers out of business. Until the Boomers die, we suffer through late stage capitalism everywhere. You only exist because a corporation owned by a billionaire allows it. Managerial feudalism.

Huh, I hadn't noticed when my old Aboriginal Linux project turned into a malware platform. More evidence I haven't been helping, I suppose. Time to down tools and find something else to do.

I should take the patreon down if I'm not doing the work.


April 16, 2020

I got dogpiled on github by people who repeatedly ignored my "that's not a design direction I want to go in" until suddenly toybox wasn't fun to work on anymore. Like kernel development, mkroot, Aboriginal Linux, my tinycc fork, busybox... Even twitter I really stopped posting to voluntarily (it's only a 12 hour suspension that won't start until I give them a phone number, I just haven't wanted to) and still have the login to my old livejournal if I cared to use it. I stopped because they're no fun anymore.

The problem is I have momentum on the shell, and if I walk away from it for any length of time coming back up to speed on half-finished code where I don't remember what I did vs what I _meant_ to do (and all the corner cases of _why_ the code works that way when the unfinished code does not yet necessarily work that way so even reverse engineering what's there doesn't provide definitive answers) would be a huge lift. Which means I probably wouldn't come back at all. I'm trying to figure out if I care. I started toybox in September 2006, 14 years ago. Maybe that's long enough? I've been doing volunteer work for the 15th largest company in the Fortune 500, for free, for years. People have told me I'm an idiot for making a mega-corporation more profitable as a hobby, and maybe they're right. I wanted to change the world, but it's a lot bigger than me, and defending things like C (not C++) and Unix and the ability to do general-purpose computing on hardware people actually have... seems kind of quaint, doesn't it? Postponing the inevitable. The people doing systemd have won. You Are Not Expected To Understand This.

I suppose I should at least finish the release I'm working on before downing tools, the same way despite everything I got busybox 1.2.2 out way back when. Which reminds me that I started toybox because I was angry. I'm no longer angry, I'm tired. I should just ignore the world and get on with my work, but if it was _my_ work I wouldn't have people telling me how to do it on github would I? I'm already ignoring so MUCH of the world just to stay sane (as I type Fuzzy's in the backyard yelling into her phone about how mad she is she has to choose between two septuagenarian rapists in November), the tiny sliver of it I was optionally trying to exert some control over becoming a source of stress is not something I have energy for right now. It's not that it's full of people who want me to stop, that's often a sign you're doing the right thing. It's that the supposed beneficiaries of my work insist (at length) I'm not serving their needs, and I guess they'd know wouldn't they? Somebody yells at me for tidying up, I stop tidying up.

I don't really need to cut a toybox release for $DAYJOB, I had an alternate approach weeks ago that I can go back to. This entire tangent was just me soaking up trumpocalypse quarrantine days to try to get bucket list todo done, and that's kind of self-indulgent anyway. I should stop wasting time and get on with my actual responsibilities. Being proud a 232 line shell script builds a bootable Linux system from source for a dozen different hardware architectures is self-indulgent: buildroot exists and far more people use yocto anyway. Even if Antoine de Sait-Exupery was right, so was Voltaire. And besides, it's a bash script: not portable.


April 15, 2020

Oh hey, the USA _does_ have (half-assed) postal banking. Who knew?


April 14, 2020

Sigh, it looks like I'm getting a release out around April 15th but I already used the rice pudding and income tax quote.

Expanding ${} with nothing in it is an error, and ${X:?blah} is an error if X is unset, and ${X;} is an error, and when you have errors like that it doesn't run the command. Which means variable expansion has to be able to abort run_command(), but half-resolved "$X${}" variables that have already allocated memory shouldn't leak, which means it's time for ANOTHER AUDIT OF THE VARIABLE RESOLUTION LOGIC. All of it. Again.

Wheee. (The TWO THINGS that the init script from toyroot still needs are the ${} stanzas in the "ln -sf /proc/self/fd${i/,*/} dev/${i/*,/}" and "exec oneit -c /dev/"${CONSOLE:-console}" $HANDOFF" lines, but ${//} and ${:-} implies error recovery logic for ${}, right after I finished the $'' rathole tangent I probably shouldn't have gone down right then but did...)

Xeno's release candidate.


April 13, 2020

Good News Everyone: If the GOP's fears are right, virginia will never vote republican again. And possibly neither will wisconsin.

Continuing the good news (let's just do good news today), The Pope just came out in favor of Basic Income. (On a related note, this remains a lovely interview.)

Speaking of UBI, while The Onion Cannot Parody, it Merely Documents Reality remains true, people keep getting money wrong. Money isn't an _illusion_, it's a _promise_.

Money is the same kind of promise as laws, employment contracts, titles to property, arrest warrants... Society is built on making and honoring such promises. Specifically money is the promise that the government WILL come and demand taxes from you in the future (and property taxes mean that wherever you live in the country, you or the person renting to you cannot AVOID paying those taxes every year), and when the government comes for its taxes it will accept this currency (and ONLY this) in payment of those taxes. If you can't get enough of it to pay the taxes between now and then, Bad Things will happen to you. And that's why you need dollars, and why everyone ELSE you meet needs them too.

Because the government ONLY accepts the dollars it issues in payment of taxes, those dollars are a uniquely valuable form of currency. If you pile up wealth in land or gold or eggs, you will have to convert some of it into dollars to pay taxes, and doing so at tax time might be extortionate, so people just use dollars all year round. Every once in a while a government is stupid enough to accept payment of taxes in gold or bitcoin or something, at which point its currency immediately collapses. Rich loons who want to avoid taxes and drown-government-in-the-bathtub still think the US dollar would mean something if they succeeded, because they're self-destructive idiots who don't know how anything works, just that their problems have historically gone away when they shout loud enough.

Back to good news: we just had the first March without a school shooting in the united states in 18 years. Here's 60 minutes embarassing an administration official with a montage. Saving the post office means repealing a single law. Here's someone explaining why Crozier (the aircraft carrier guy who called for help when his ship got coronavirus) was unquestionably right to get help when the crew maintaining 8 nuclear reactors started to fall ill.

Oh hey, is that what happened to Austin's Spam Festival?


April 12, 2020

Got an email, replied, might as well share the reply:

> I really wanted to hear
> you talk about device-tree. It is a subject that I am very fuzzy on and I am
> sure whatever you had talked about would have helped my understanding.

Device tree is just a big list of all the hardware in the system, in a known format that can be parsed by generic (architecture-independent) code, so drivers can call functions to ask "do you have one of the devices I manage" and get back data telling it what devices it recognizes are available, how many of them are present, and where to find them. It's a tree instead of a list so it can nest (this bus has these devices hanging under it), and so it can have things like "memory controllers" and "block device controllers" and "character device controllers" and so on grouped together.

Way back when your kernel would be hardwired to expect certain hardware at known locations. (Ala you have a 4 serial ports, here's the IRQ and I/O address of each one). The IBM PC started life as a derivative of the MITS Altair, as described here. There were ways to probe for _some_ stuff (such as the E820 memory map telling you how much physical memory the system had, at what address ranges), but other stuff the kernel just had to know about (hardwired in C, for Linux you'd select it in menuconfig).

Later generations of bus controllers (starting in the early 1990's) were smarter, and allowed you to probe the devices plugged into them (the lspci and lsusb commands do that for pci and usb respectively), but there's always a chicken and egg problem of "I can ask a PCI controller what devices are plugged into the bus once I know where the PCI controller is, but how do I find the PCI controller in the first place?" So there were still some static devices your kernel just had to know about.

Of course when computers boot, they always start by running some ROM (the CPU powers on with the registers resetting known values, which means it starts running code at a known location, and the hardware maps ROM at that location so a control program can take over and bring the system up, this old writeup about the 2.4 kernel explains the ancient 32 bit PC way of doing things before EFI bios was invented, for example.) Since about 2000 they've used flash memory instead of ROM, but the principle's the same: before your OS can run some code has to bring up the hardware enough to hand off to the operating system.

(Tangent: bootloaders actually come in 2 parts: the first part initializes the DRAM controller so the DRAM refresh happens and main memory remembers what's written to it for longer than 1/4 of a second, which turns out to be nontrivial and thus requires software to set it up. Then the second part is your bootloader that reads your OS kernel into memory and jumps to the start of it. Some bootloaders provide just the second part (ala grub) and some do both (such as u-boot and openbios). This is the reason u-boot wouldn't run under QEMU for the longest time is its maintainer refused to split the two parts, so it tried to do dram init under QEMU which doesn't emulate a DRAM controller, which would fail and crash. For years all DRAM init code had to be written in assembly because if you haven't got the DRAM working you pretty much just use the CPU registers, but then the Linuxbios guys did something clever and loaded a single TLB entry into the CPU cache telling it the first 16k of memory (starting at address 0) should only be written back to DRAM when you needed to evict the cache lines to make space for something else, so it would load garbage in from memory, they'd zero it out, set their stack pointer there, and then they had 16k of memory to use as stack so they could start a C function and do their dram init in C instead of assembly. And as long as they were careful not to touch the rest of DRAM before they were done. Everybody does it that way, ad linuxbios got renamed "openbios" a few years later when the BSD guys started using it too.)

Anyway, the bootloader that's flashed onto the board's hardware usually knows what hardware is in the board (at least enough to find the probeable busses like PCI and USB), and a couple generations of Sun Sparc machines used something called "OpenBIOS" with a big hardware table in a tree format, and IBM picked it up for its PowerPC systems, and then the Linux PowerPC port wrote code to parse the PowerPC table out of the firmware (and convert it to a simplified format called "flattened device tree") to see what hardware was available, and then u-boot started PROVIDING a table in that format, and it got genericized and copied to a bunch of targets starting around 2008. Oddly enough, a large chunk of that history is described here.

These days, just about the only architecture that DOESN'T use device tree is x86, because of windows+intel+arm politics. The problem is the Linux infrastructure for creating new device trees, turning a text device tree source (*.dts) file into a binary (*.dtb) file using a device tree compiler (dtc)... well all those dts files are licensed GPL. And that means windows and freebsd cannot (and never will) use them, which means that device tree is a linux-only format. And when windows started to switch over to Arm a few years back, they asked if they could use device tree under a bsd license and were told no, so they sponsored a port of the new x86 bios format (ACPI) to arm which exists ENTIRELY because the Linux guys insisted on GPL for data files and thus the rest of the industry went a different way. Sad, but there you have it.


April 11, 2020

Odd numbered day. I have a backlog of links that collectively call for changing the law to guillotine billionaires.

(Specifically, make retaining control of a billion dollars for longer than 30 days -- so you can give it away -- a capital offense punishable by guillotine. Yeah, we COULD do torches and pitchforks, but changing the law so the state does it for us seems more persistently effective. Sure you can reinstate the 90% top tax rate after the first 10 million/year so there aren't any billionaires and we don't wind up guillotining anybody (after we've cleared out the current lot who would rather die than give up "their" money), but the overton window needs to go well beyond there if you ever hope to acheive the tax thing as a consensus compromise position. Given how many people the current system is killing with covid19 and gofundmes for insulin and civil forfeiture and school-to-for-profit-prison pipelines and "open carry for whites, black 12 year old with a squirt gun shot on sight" and 300 other things, it would take rather a lot of billionaire heads to balance the scales.)


April 10, 2020

Ok, even numbered day. Ignore the ongoing trumpocalypse, be positive, and get work done. (There are still good things in the world.)

I started an ongoing thread on the linux-m68k list about trying to get that to work in toybox's "make root". Not quite sure what the problem is yet, but something's not right.

[Update from the future: it turns out I don't have to use Laurent Viver's out-of-tree qemu fork anymore, "qemu-system-m68k -M q800" support FINALLY GOT MERGED! After only 12 years out of tree! And QEMU and the kernel _can_ agree on an -initrd handoff now, which is nice. The first problem I had is I ran "make defconfig" at the wrong time which switched off CONFIG_SH because it's still in pending so it was legitimately reporting that it couldn't run #!/bin/sh at the start of the init script. But the SECOND problem I had turns out to be what really looks like a compiler bug in gcc 9.2.0 for m68k.]


April 9, 2020

I'm only letting myself rage about the ongoing trumpocalypse on odd days, and trying to get actual work done on the even numbered days. This leads to quite a backlog of links.

It's sad how good the guy who wrote Leverage is at predicting terrible things.

Ok, that's a good counterpoint, and I'm fine with religion as culture the same way I'm fine with santa claus as culture. But only the selfish and evil seem to put a lot of money and power behind using it to control people.

Oh, so that's why he learned the brand name. Meanwhile, it turns out giving the GOP's recommended drug cocktail to Covid-19 patients causes permanent heart damage.

I'm not entirely sure why the Queen's speech included a section clearly designed to act as her eulogy, but there's no way it was an accident.

The navy really refuses to let sick people onto a hospital ship? What is the hospital ship FOR then?

A good thread about why there are milk shortages while farmers are dumping thousands of liters of milk: it's because home use and restaraunt use require different packaging. The home processing lines are saturated while the restaurant ones sit idle, and you can't quickly or cheaply convert one to the other. (And food banks aren't the answer because what's a food bank going to do if a 40k liter tanker of unpasteurized milk rolls up?

This is exactly the problem japan had.

The reason the Dorito is moving like a robot is because it's a dementia symptom. (Not remotely the only one either. He can't remember what he said for fifteen minutes anymore.)

Opec rose from the grave long enough to do a 10 million barrel/day cut, which is less than half the current oversupply. At a guess, they're unable to even find storage (let alone consumers) for oil as fast as they've been pumping it, so they're cutting back to the rate they can store. (Or if my theory from 2 days ago is right, the rate at which china can build new storage with its ghost city construction crews. I wonder if China is going to use this as an opportunity to switch the world oil trade from being done in dollars to being done in Yuan?)

There's no way all these oil storage tanks that haven't been used in forever don't leak. We're scrounging up oil storage capacity that hasn't been used for a _reason_. Everything else is just-in-time delivery with no reserve capacity but oil can store 15 times what it had at the start of the year without an issue? Seems unlikely. There's gonna be issues.

Twitter continues to be bad at being twitter. Its CEO Jack has taken advantage of Coronavirus to announce his move into philanthropy (just like every other billionaire who wants people to be grateful they exist while exerting as much control over as many things as possible in a tax-free manner: he's buying support). The thing is, he didn't give $1 billion to fight coronavirus, he set up a philanthropic fund he controls with an endowment, and then presumably it spends the interest just like all the other billionaire funds do, and one of the things this fund will give money to is fighting coronavirus. This is also why the inventor of dynamite createdthe Nobel Prize after learning that his obituary would be about all the people he'd killed getting rich as an arms manufacturer. A railroad robber baron put his name on Carnegie Hall for the same reason. Billionaires often become philanthropists, and it's always entirely self-serving. (Things can get so bad that a billioniaire potlach is the best available option, but private equity didn't eradicate smallpox. The billionaires' motif is to steal gobs of money and give back 1/10th of it to buy love and redemption. If you dodge taxes and then inadequately take the place of government programs you ended, solely so you can bask in adoration, you still suck.)

It's hard to make substantive change without revolution. Especially when the ruling party maintains its hold entirely by cheating, despite being actively malicious doddering clowns.

Seizing other people's goods in transit and selling them for profit is the literal definition of piracy. They're not even trying to hide it. But you can tell billionaires are driving, because they're giving back a tiny fraction of what they stole and demanding performative gratitude. That's what late stage capitalism is all about. And also why the guillotine was invented.

Joe Biden is a horrible candidate who has almost literaly adopted Hillary's 2016 campaign slogan, "Vote for me, you have no choice". Last year I said he was going to lose, and that the only question was whether it would be in the primary or the general. I really hope I was wrong, because I don't think Ruth Bader-Ginsburg can hold out another 4 years, but if anyone CAN lose this, it's him.

Capitalism has outlived its usefulness and become a cult propped up by distraction du jour, elderly loons, and voter suppression. The old media watchdogs have failed, and even though all the same assholes the Boomers let them get away with it. The USA now has a parasite class who do nothing but make money off what they already own.

Most people are fundamentally nice, they have to be socialized into being liberatarian assholes. (Every 3 year old wants desperately to help out in the kitchen. Which is why basic income would work just fine if you just kept assholes from cornering the market and stealing everyone's lunch money.)

There's a small number of people you have to guillotine to get a functioning society, and right now we're allowing them to be billionaires, despite them committing literal murder. But allowing them to continue (let alone actively supporting them) is a choice. Some people have so much racism they'd rather die than let anyone they don't like thrive.

Meanwhile, the GOP's incompetence is intentional. It's a form of "never let a crisis go to waste", they make things bad so they can blame others for the problems they've caused. These guys are uniquely bad, this infantilizing cult of suck is the cause of these problems. (Which brings us back to the need for guillotines.)

Here's a thread about how the racists have been cheating politically ever since the civil war. (Remember, the GOP and Democrats swapped polarity after 1964: capturing the racist vote LBJ alienated by signing the civil rights act was Barry Goldwater's "southern strategy" that got Richard Nixon elected.)

Oh well, at least the military isn't currently behind them.


April 8, 2020

The main difference between The Hithhiker's Guide to the Galaxy (1980 BBC miniseries version) and Good Omens is that in one the earth was destroyed, and in the other the earth was not destroyed. (And the bug eyed monster? Is green, yes.)

Ok, the current near-term todo list:
make dropbear
  download+build source
  init script
    write/run external init script
    make key
    netcat -L
  boot with sh
  route
toysh
  run init script
  implement "!"
  support {1..2..3}
  pass sh.tests
make test_test (single build broken)

I've tidied up mkroot. It now handles its own cross compiling and logging (no seperate cross.sh script), can once again call external scripts (although doesn't provide download/setupfor/cleanup functionalitiy, I should have an external script for that) and its init script can run child scripts (/etc/rc/* in sort order). I've merged the "disable unwanted default config entries" (switch on CONFIG_EXPERT and then switch OFF the vga /dev/tty* crap and several other things that have no business being on by default, and ALSO switch off the CONFIG_DEBUG nonsense that CONFIG_EXPERT switches on by default when it's enabled).

The result is 228 lines long, and it's wired up so "make root" gives you a working chroot. (Once I get toysh going through the whole init script, anyway.) Or to buildall the supported targets, with a kernel that boots under qemu, using musl-cross-make cross compilers:

git clone https://github.com/torvalds/linux
git clone https://github.com/richfelker/musl-cross-make
git clone https://github.com/landley/toybox
cd musl-cross-make
../toybox/scripts/mcm-buildall.sh
cd ../toybox
ln -s ../musl-cross-make/ccc ccc

make CROSS=all ALL=1 LINUX=../linux 

Then cd (root/sh4; ./qemu-*.sh) and such. (Ok, right now "KARGS=rdinit=/bin/sh ./qemu-*.sh" because of the part where toysh doesn't understand ${abc:-def} and such yet. Working on it...)

It occurs to me that rather than trying to do youtube videos, I should probably try to do _patreon_ videos. I still need to find some decent video editing software, but it seems like a lower bar somehow?

The "shift" command is terrible. If you just type "shift" from an interactive command line, it's basically ignored (there are no arguments), but when there ARE arguments shifting past the end is an error, and shifting by a negative number does NOT recover previous entries (why not?) Ah, I see: shift in interactive mode sets the exit code to 1, but does not produce an error message. Yet another conditionally silent error... oh, it's not conditional, duh, bash NEVER shows an error message. I was test ./sh vs sh again, which points to the Defective Annoying SHell. Oddly, dash aborts the current line when it hits this error, and bash doesn't:

$ sh -c 'shift; echo hello'
sh: 1: shift: can't shift that many
$ bash -c 'shift; echo hello'
hello

Posix is completely useless these days, isn't it? Throw Jorg in with the Boomers and wait 'em out until some glorious day entropy makes the world a better place...


April 7, 2020

As with Friday's Streisand Effect, now "Dr. Drew" is using DMCA takedowns to hide a montage of his earlier statements (telling people coronavirus wasn't dangerous) which contradict what he's saying now (ancknowledging that it very much IS dangerous and not "just the flu" and he'd like not to be sued or lose his license). So a copyright lawyer with a youtube channel did a legal analysis of the taken-down video, which involves showing the video. (It's a beautiful thing. That link skips the first 6 minutes of throat clearing and starts near the actual showing-the-video part.)

The dementia patient in chief assigned Space Farce to moon mining. (Did we have a shortage of domestic mines?) He's still committing piracy and losing the military (there's audio if you have a strong stomach). Meanwhile, last week's insanely high jobless claims doubled this week. And the pandemic playbook Obama left the GOP, which they didn't read because Obama cooties, is available online. (Once they threw it away, it was declassified.)

The mortgage industry is complaining that the 2008 bailout used up all their political capital so they can't currently finagle another bailout. They're not the only vestige of late stage capitalism underperforming.

If Boris Johnson dies of coronavirus, will they announce he's died merely as a precaution? He bragged about shaking hands with coronavirus patients, then assured people his diagnosis was no big deal, then he entered the hospital for routine tests, and now they've moved him to intensive care merely as a precaution. (Not that we can exactly throw stones from our glass house.)

A side effect of the massive lack of testing is undercounting coronavirus deaths, which might explain why New York City has proposed turning one of its parks into a cemetary for coronavirus victims.

The only even SLIGHT upside of the current GOP is that it identifies and burns out the worst people. I'm immensely disappointed in Pelosi and Biden and would ONLY vote for either mid-pandemic, but there we have it. Still, I am comforted by the advanced age and thus self-limiting nature of both of them.

The GOP is only relevant because of voter suppression and they can be as hypocritical as they like about it on "might makes right" grounds. The GOP is basically a death cult at this point. But late stage capitalism is going to continue to crash harder and more frequently until it's replaced. Reagan blew holes in FDR's new deal banking regulation that had prevented a repeat of the Great Depression for 50 years, and since then we've had the savings and loan crisis of 1991, the dot-com bust of 2001, the mortgage crisis of 2008 (and in between those last two the collapse of enron/worldcom/tyco and like 30 others), and ever since then the economy has survived on constant "quantitative easing" which is more or less basic income for billionaire portfolios. If we just guillotine the 600 or so billionaires in the USA and distribute that money to individuals instead, we could all have basic income and medicare for all, and it turns out it doesn't even cause inflation. Of course there are other arguments for guillotines.

The oil industry is starting to admit the changes might be permanent. Let's look at some numbers.

This article on oil running out of storage says the global storage capacity is almost a billion barrels, and that the price decline after the last economic crisis peaked in 2015 storing a backlog of 500 million barrels, which then took 2 years to work through (with opec able to collude production cutbacks and the economy growing at 3.5 percent). So if we fill the billion barrels of storage all the way we're looking at something like 4 years to run through the oil backlog before prices could go up again, and that's assuming the economy and Opec both recover enough to consume at the old rate and do production cuts again. (Then again production cuts might not require opec, since some oil wells can't shut down gracefully and basically have to be re-drilled to resume production. That doesn't apply to any of the US fracking stuff though, so Russia and Saudi arabia _can't_ put those out of business except by keeping oil prices below where they're profitable _forever_. Otherwise whoever's bought the land out of bankruptcy can just start it up again at any time.)

Here's a second article which says there were actually 1.4 billion barrels of free storage at the start of the year (1b on land and 400m in oil tankers at sea) which is over 5 years to run through if we fill it all up. That article also says the storage was nearly empty to start (only 100 million barrels at the start of coronavirus in January, remember Trump was selling the US strategic petroleum reserve to pocket the money) but we've already filled most of it up and will hit 1.3 billion barrels by the end of THIS month. Which would mean storage runs out early next month. I've heard demand is down by 20 million barrels/day and Saudi output increased by 4 million barrels/day, which would be 100 million surplus barrels about every 4 days, which is a billion in 40 days. So yeah, the second article's numbers seem plausible. Oil producers are even filling up the rail cars already.

Meanwhile countries that normally import oil are exporting it due to reduced domestic demand, sending it to asia as a "dumping ground" (both because china, japan, and india still have storage left, and because sending it to the other side of the planet keeps it in the oil tankers longer at business-as-usual "paying to transport oil" instead of "paying to park oil" rental rates). And even countries that import oil may export natural gas which is priced based on oil.

Interestingly, China, home of entire ghost cities because construction is a form of economic stimulus (I.E. instead of inventing financial derivatives, their quantitative easing goes to construction companies and then their middle class buys the empty real estate as its "actually worthless but so lucrative on paper" investments), decided to expand its strategic petroleum reserve as a post-coronavirus stimulus construction project. So when exactly the world runs out seems kinda squishy. Possibly that's what the "storage doesn't run out until june" people are banking on, is China able to build tanks faster than the rest of the world combined can pump oil? But that's not going to raise the price out of single digits, and it's makes the backlog take longer to work through before prices could ever go up again.

And the INTERESTING part is what they plan to do with that oil. They're the people pushing solar and batteries out into the world. If they buy oil at single digit prices for a year or two, and then SELL IT AGAIN to keep the price down if it starts to go back up rather than consuming it domestically... If prices were already going to stay low for 5 years, and they add a few more years with political shenanigans (or just never buy oil more expensive than that from the rest of the world ever again so global demand is permanently down), they can probably bankrupt all the oil producers worldwide. And sell us solar panels and batteries. (No idea what they _plan_ to do, but the options are interesting.)


April 6, 2020

The tests for the toybox "test" command haven't worked since last year, and the reason is "make test" self-identifies as "[" instead of "test", meaning it demands a trailing ] in its argument list.

This broke when I added TOYBOX_MAYFORK late last year, because I added an OLDTOY([) alias for test when I did that. (For some insane historical reason which '[' finds /usr/bin/[ and no I dunno what would break if it's not there, but busybox provides it.)

The problem is, "test" and "[" are both enabled by the same USE_TEST() guard macro, so both are enabled in the "make test" single build because they can't be individually selected, and since it's sorting alphabetically (so toy_find() can binary search) the first one in the list is "[", and the single commands setup just sets this.which to the first entry in the list. So standalone "test" thinks it's "[", and behaves accordingly.

What I'd LIKE to do to fix it is disable all the OLDTOY() macros during single builds (except for sh which is magic because it's where the NOFORK commands run). And this should be fine because OLDTOY() macros should never be enabled when the corresponding NEWTOY() isn't enabled, so in the case of single builds they shouldn't matter.

Except md5sum.c added a bunch of OLDTOY() variants of SHA224, sha256, sha384, and sha512 (all aliases of sha1sum and thus sharing sha1sum_main), each with their own guard macro, without even a "depends on" the base command in their config entries, and that violates this design assumption. (Locally it works, but globally what is and isn't set by various plumbing assumes nobody does that. Hmmm, while I'm looking at md5sum, I should calculate the table when we have floating support enabled, and I never did implement sha256 and friends longhand, did I?)

Anyway, if I teach single builds to disable OLDTOY() aliases I think "make sha256sum" becomes a build break because it doesn't "depends on SHA1SUM". (This file is already kinda inconsistent because sha1sum_main() is just a wrapper for md5sum_main() but sha256sum and friends are oldtoys for the sha1sum_main() wrapper. So they're _sharing_ a _wrapper_.)

Anyway, there's an sha1sum.test but no sha256sum.test so I'm probably just gonna go ahead and break the singlebuild here and throw it on the todo heap to fix it later. It should still build in the multiplexer, "make change" is gonna be unhappy, but:

$ make distclean defconfig
$ make change
...
$ ls change/*.bad
change/halt.bad  change/ping6.bad     change/prlimit.bad
change/nc.bad    change/poweroff.bad

Nothing new there. (It's on the todo list!)

And oddly enough, all of those are OLDTOY() already. (Although it _did_ happily build chown and egrep and fgrep and so on... except they do work, because they "depends on" the NEWTOY, but they probably have the same problem as "test" of anything that checks the name getting the first name...

Darn it, this is fiddly. I need single builds to have just the ONE toy_list entry for the correct command, at the start of the array. Do I need to add a define to the cc command line to make that happen? Hmmm...


April 5, 2020

If this link wasn't real the editor would definitely go "that's a bit a bit on the nose, isn't it?" I mean there's explicit failure, and then there's that.

Meanwhile Faceboot keeps finding new ways to be horrifying. It's kinda lost in the noise at the moment, but I am SO glad I've never had an account on that platform.

I'm not sure the united states will have a healthcare system after this. We're about to burn out all the doctors and nurses. The ones who survive will quit. Here's a good thread about why. (Of course for-profit healthcare has been deeply problematic all along), but it may be coming to a head. The whole system is collapsing financially.

I've never understood the appeal of christianity. The reason they fight so hard against acknowledging evolution is because if they're wrong about where we came from why would they be right about where we're going to? Pascal's wager says you could waste the only life you have bowing before pedophile priests telling you to cut off bits of your childrens' genitals to please God and that women must be silent and subservient and here's a hairshirt, or if you're reincarnated you could wind up back here wasting life after life, and of course if you DO have to pick the right God why _this_ one instead of Osiris/Zeus/Odin/Amaterasu? They offer a "universal" truth tracing back to a single "what some guy said in this particular desert long ago, which can never be independently verified" that tells of an omnipotent omniscient omnibenevolent entity that's hiding forever even in the face of nazi germany, and never even speaks up when two armies fight over different interpretations of the same religion. Why don't any of the elaborate dietary laws include the phrase "boil your drinking water"? They recommend using the organization behind the spanish inquisition and conquistadores as a source of morality when their big master list of don'ts doesn't get to not murdering people until halfway through and puts it equal with saying "Oh God" (name in vain) or working through the weekend (keep sabbath holy)? Their morality is "eternal" and "infallible" but does a 180 ("an eye for an eye" to "turn the other cheek") because God changed his mind. (Yes he destroyed your civilization in a flood, including every child below the age of consent, but he won't do it again have a rainbow. What exactly did Job's wife and daughters do? They didn't get fixed, they got killed to teach Job a lesson and were _replaced_ by a new family so that's ok then? Mysterious ways! Not the droids you're looking for.)

But especially now that christianity in the USA has merged seamlessly with the GOP. The thing to realize about VP Mike Pence is he hasn't made any plans for this part of his life because he honestly thought he would be raptured by now. He does racist misogyny of opportunity, but hasn't been building towards anything because he didn't expect to live this long. (The Rapture! Any day now!) And he's not the only one in this administration to not only expect the literal end of the world but be actively working to bring it about.)

Then again I suppose it's hard to distinguish one grift from another. Accumulate a large enough flock to fleece and a different con man swoops in and grabs your carefully collated profitable rubes, which is how a real estate scam artist and money launderer running a fake university and selling mail-order steaks took over the GOP.

The GOP is currently engaged in ongoing outright piracy. The Trump administration keeps literally stealing masks from the states and selling them for profit. How is this not treason? States are having to sneak around to avoid having their supplies siezed. (Intentional failure to deliver enough supplies, again explicitly and admittedly for profit sounds like treason to. But that's what late stage capitalism has degraded to. Meanwhile, the GOP/capitalists are trying to secure power any way they can. They seem to expect torches and pitchforks. (I've mentioned before this fascinating article about how the 1% can never be satisfied with their wealth because they believe they're going to drive society to collapse at which point they'll have to outbid each other for the titanic's lifeboats. That's the metaphor driving them to accumulate ever-more wealth at whatever the cost in human lives.)

It's also increasingly obvious that most of the United States' culture is based on racism, along with our most critical infrastructure. Confederate loons are actively destroying the union, the main thing slowing them down is their own profound incompetence.

"A republican in authority said it, therefore it is both wrong and stupid" remains true, but let's count the ways on this one: "Russia and Saudi Arabia are colluding" - no, they STOPPED colluding. That's the point. OPEC collapsed, so they're no longer artificially limiting the supply of oil to raise prices. He then said that he wasn't in favor of US supply restrictions, which is the same problem from our end: more oil than the market can consume, filling up all available storage, and driving the price of NEW oil to literally zero because there's nowhere to put it.

The Saudis are engaging in some SERIOUS motivated reasoning (or maybe just derp). They insist the oversupply has two reasons, so their increased supply is not of interest while demand is lower. (Not even a "how could we have known", but instead "whatabout THEM" every time their complicity is pointed out). They also insist that the expensive-to-produce US oil must be forced out of the market so the world can rewind to the days when Saudi sold more oil than anybody else and nobody else could challenge them. (Imagine a 93 year old man who insists he'll be back in fighting shape soon, you'll see: it USED to be true therefore it's the natural order of things.) Meanwhile, oil demand is declining further each month.

Late stage capitalism appears to be failing. The two modern definitions of "efficiency" seem to be 1) make it brittle with no reserve capacity or ability to survive shocks, 2) shift all the problems onto people of lower status. At this point the system's not even serving most white people, just a residual Boomer 1%. Alas "the market can remain irrational longer than you can remain solvent" and Keyenes observation "In the long run we are all dead" remain true. The LD50 of the Boomers pencils out to 2034, and a disease with a 5% mortality rate among septuagenarians doesn't change that much. "Racist grandpa convinced by spam email to vote against his own interest" is not a new story, it's just unpleasant when the senile geezers are collectively driving.


April 4, 2020

I had to build dropbear for a work image, and I have a build script in the old mkroot project, so I started fluffing that up a bit... but it still builds busybox and I don't really want busybox in the image, so instead I started extending "make root" towards what the standalone one could do. And now I've hit the actual design difference, which is that the external one downloads, extracts, and cleans up after package source tarballs, and the one merged into toybox doesn't. Hmmm...

Really, bash? At the start of mkroot.sh I have an 'env -i a="$a" b="$b" c="$c"' construct that I'd prefer to generated with {a,b,c} brace expansion (it's the list of variables to pass through tne environment clearing), so I tried the obvious:

$ X=a=\"\$a\"; echo ${X//a/{x,y,z}}
{x,y,z="${x,y,z"}

Bash does not appear to have taken this well. I set X to 'a="a"' and then tried to regex substitute in the expansion for both a's, with the brace expansion giving me a list of results. The man page says brace expansion happens first, which appears not _entirely_ to be the case here? It looks like ${ is treated as a quote and matches the _first_ } so brace epansion is skipped entirely and "{x,y,z" is what ${// /} substitutes in both places, then there's a trailing unmatched } left over at the end.


April 3, 2020

Streisand effect, activate!

Can we admit the Resident has frontotemporal dementia yet? (Specifically Progressive Supranuclear Palsy?) The ongoing trumpocalypse isn't JUST because he's a sociopath, he's also medically incompetent. Not just an ignorant racist doddering old nepotistic capitalist poster child for Dunning-Kruger, but ALSO incapacitated by a literally terminal brain disease on top of that. That's why he can't pick up a glass of water with one hand, and the way he leans forward is due to the leg brace he wears. (PSP hits a brain area halfway between altzheimer's and parkinson's disease, and expands into both as it kills you.) It's not like his condition is a secret or anything, everybody knows he's increasingly incapacitated. There's no shortage of this stuff.

It takes an awful lot for the president of the united states to lose the loyalty of the US military, but he's getting there. The outright theft (including piracy) and obvious lies aren't necessarily helping, but the senility means it's not because of any kind of chess mastery going on, it's an emu war against an opponent too random and inconsistent to predict, and too dumb and insensitive to even notice any non-fatal damage. There is no goal or end game here, Trump's Razor has been proven time and again by history: the dumbest possible explanation for his actions is correct, and we're all suffering. (Seriously, here's a playlist over 100 videos long titled "Trump's Frontotemporal Dementia with Progressive Supranuclear Palsy (PSP)" interspersing videos about the condition with news footage of the Resident displaying those symptoms.)

Meanwhile, Betteridge's Law applies to a lot of ongoing oil coverage. (There's even a wikipedia[citation needed] article now.) The Dementia Patient in chief said he thinks Russia and Saudi Arabia will stop playing chicken (because he wants them to, just like he wanted coronavirus to magically go away and thus said it would), and oil rallied! Because analysists are facing the end of their livelihoods if the industry they cover becomes 1/10th as lucrative (and thus 1/10 the investment trading volume), and they're desperate for any good news. But no, he didn't know anything, he stated a desire as an expectation because he's a rich white male Boomer and that's how it works, isn't it? Boomer want, therefore Boomer get, QED. (As for calling the Resident "rich", once you're in the club the system will not let you fail. That's why Elon Musk could borrow half a billion dollars to keep Tesla afloat. He WAS rich, therefore he will remain rich with endless bailouts until the good ole boy network that has embraced him as one of its own collapses.)

The global economy has an abusive relationship with the US dollar. The worse things get, the more they cling to it. It's good to be the king. (Mark Blyth pointed out the USA historically has strong courts, which China doesn't. If you want to sue people over a dollar denominated debt or enforce a contract, property owners get preferred treatment here. In china, you will not get a fair hearing, it's bribery at every level and then a party apparatchik takes your stuff. The Euro has strong courts but isn't financially integrated and will have endless financial crises until it works out how to do that, rendering the currency an unsafe store of value. In the USA the blue states fund the red states which is what's kept the union together: the north pays all the deadbeat confederate states' bills, every year, forever.)


April 2, 2020

Today, COVID-19 became the leading cause of death in the USA.

Perfect meme. (Addressing a persistent problem. I blame capitalism, and our increasingly dysfunctional completely fictitious and profoundly abusive society. The Boomers!)

A good quote from CNN's article on oil prices turning negative: "The price is trying to go to a level to force companies to keep the oil in the ground. If it has to go negative to incentivize that behavior, then it will." I wonder what Greta Thunberg has to say about it? China's already recovering which means moore's law style march of solar and battery tech becoming ever cheaper has resumed. We're all hoping the oil industry is permanently damaged by this, "the spice must flow" because when it bad things happen to the physical and financial plumbing. (Yes Dune is an alt-history set in space about Europe moving into the middle east to take the oil a century ago. Arrakis is Iraq, spice is oil, even characters like the Pashidah Emperor and Muad'dib didn't have the serial numbers entirely filed off.)

If the oil industry suffers hysteresis, acquires debt, low prices persist selling through the backlog, has a temporary inability to afford to lobby governments turning into permanent regulatory changes, has trouble financing further investment in wells, pipes, tanks, or refineries (it's stranded assets all the way down), meanwhile the obvious climate effects of reduced fossil fuel usage provide hard numbers from a big natural experiment... (Meanwhile natural gas prices were already negative for two weeks last year because more gas was coming out than they could pipe away, and that was _before_ coronavirus... This definitely confirms that blocking pipeline construction is an extremely effective tactic in reducing fossil fuel consumption. Here's hoping the the whole industry can get into an accelerating negative rut and spiral downwards.)

And twelve years later, they finally close the bug. (Linux on the desktop, smell the usability.)

The toilet paper shortage still isn't because of hoarding. (Just like the grocery stores being out of a lot of food is because people are staying home and cooking rather than eating out; a just-in-time supply chain can't handle a persistent 10% increase when the system hasn't got spare capacity, the shelves empty and can't restock fast enough.)


April 1, 2020

Sigh. Things continue to go about like you'd expect with these clowns in charge (and yes clowns includes joe biden and pelosi). Meanwhile the EU can't kick out Hungary despite it collapsing into dictatorship, which has been coming for a full decade, we're about 6 years behind on that curve.

The oil industry continues to implode.

Toysh: finally got the variable plumbing redesigned through to the end, marched through the missing semicolons and typoed variable names and unused variable warnings to the point it compiles again, and the result immediately segfaults. So, par for the course. (It was inconsistent lifetime rules for the arguments of export() and unexport(), but I changed a lot of variable measuring from "where's the =" to "what are acceptable variable characters, oh you did {abc@def}<<file that's an error lemme catch that... and it's caused regressions where I'm not using the new plumbing right yet.)


March 31, 2020

Bill Gates asks if we've tried turning the country off and back on again. Even Faceboot and Twitter turn out to have a limit break. (Yes the world is still on fire, and yes this whole mess is still entirely the GOP cult's fault, and yes they're still making it worse, and yes the Dementia patient in chief is still an absolute idiot. It's still stressful, but there are other things that need doing too.)

Toysh: the reason TT.vars needs a malloced -> str entry for the magic variables is that you can unexport a magic. I.E. "export SECONDS; export -n SECONDS" means SECONDS is still magic (because it was never _unset_), but the ->str entry is malloced (because when we exported we printed an instance of it, and when we imported we removed it from the globals array but kept it in the ->str field). The problem is when we moved the constant ->str to globals we made a malloc copy and discarded the reference to the constant version, and we can't get the constant version BACK when we unexport it. And if we don't free() that if we later unset it, the allocation leaks.

The other problem with unexport is the environment variable may have come from envp[] at exec time, and never been malloced. So sometimes we need to malloc when we unexport. (Or a VAR_NOFREE flag? Hmmm...)

[Eight hours later] How deep into the weeds am I? xpop_env() can malloc() a copy of a variable if it was inherited from the environment (so the value it removed from the environment is always returned as a freeable heap allocation for consistent object lifetime) but when incrementing $SHLVL for a child process we've already gone to a temporary environ[] array which has pointers to the original entries, so I want to remove the original entry from the new list but NOT free it, and let swapping back the environ[] array put it back. Except if it mallocs a copy I need to know and free THAT (or else avoid the copy in the first place.)

Don't mind me, this is normal around here. MOST of the way through redoing the variable allocation design, which is a maze of twisty object lifetime rules (all subtle) to do at all efficiently, but I knew that going in. Currently auditing all xsetenv() callers. I am SO gonna have to run this under a memory leak checker. How do you use valgrind again?


March 30, 2020

The price of oil is going negative (just in patches at the moment, but 20 million barrels per day is going into storage which is filling up and when there's nowhere to PUT the oil nobody can BUY the oil, meaning about three more months of this would force ALL the oil into "pay to dispose of it" territory), so of course republicans are passing laws criminalizing protests against oil (among other things).

To give an idea how well-supplied the linux kernel is with developers examining every nook and cranny of the code, the FIXME comment in kernel/time/posix-timers.c ending "which we don't want to do late in the release cycle, so for now..." is from June 21, 2007, and still there in today's v5.6 release. (Which, alas, does not have the sh fix.)

I hit it because I was trying to figure out which time fetching function bash was using for seconds. According to ltrace it's gettimeofday() which is obsolete (microseconds, not nanoseconds), and in THEORY that translates to one of the clock_gettime() variants and I wanted to see which one. But I figured out that bash ISN'T saving fractions of a second for SECONDS, because bash -c 'SECONDS=41; sleep .25; echo $SECONDS; sleep .25; echo $SECONDS; sleep .25; echo $SECONDS' has a variable transition boundary.

Still a pain to come up with a reliable $SECONDS test producing predictable output. I can ALMOST trust 'SECONDS=41; sleep 1; echo $SECONDS' to output 42... except the tiny window where it can output 43. (I suppose I can stick a while and test around it?)

Nope, SECONDS is a trash fire in bash and I can't come up with sane tests for it right now. New variable infrastructure, cut a release, then circle back around to this nightmare.


March 29, 2020

Not a surprise. But then what is? There's nothing left in the GOP to salvage. Literally nothing. (The obvious solution is to guillotine the billioniares, which is the classic trolley problem. People are dying, today. If billionaires don't treat people like humans, why should anyone else treat billionaires any different?

Also, ballpark numbers the 2.5% mortality rate when you have enough ventilators becomes 5% when you don't, which would be somewhere north of 15 million people in the USA. (The 20 million cell phone cancellations aren't the only evidence china is hugely downplaying its coronavirus deaths.) If that doesn't cause wholesale structural change, what WOULD? (I hope the american medical association is replaced after all this. I explained my dislike for them last month. Elizabeth Warren's come out for at least half-assed basic income, but it's hard to be satisfied with crumbs.)

In cheerier news, this guy is good. (He talked about autonomous driving 18 months ago. Oil is still tanking.)

I cc'd Linus Torvalds in an email last night because Rich's superh maintainership does not appear to include actually submitting patches upstream. Poking him to ack a patch means he posts a message to the linux-sh mailing list saying "ack". He does NOT submit a pull request to Linus, apparently others do that. (Admittedly when I was Linux Documentation maintainer I didn't have a tree Linus pulled from either, and submitted it through... Randy Dunlap I think? But that was because due to the procedures at the time I needed to get a gpg key signed in-person at a Linux conference. This was a couple years after the kernel.org breakin when they were furiously locking barn doors.)

Alas, said cc: does not appear to have resulted in a checkin to Linus's tree. Going by the checkins from yesterday I think I missed him by about 15 minutes, and he's not gonna do any more today. Oh well, I can use a toolchain with an older binutils version to build the binaries this release.

Back to toysh. The new TT.vars list has index entries for all accessible variables, including globals. Globals are still stored in environ and the way you free those is via xsetenv(), so the "inherited from exec vs malloced" tracking is still done by the existing lib/env.c code.

But the setvar() plumbing I did months ago takes a flags argument, one of which is whether it should take the memory you feed it or strdup() its own copy. The other flags are all status flags: is it readonly, magic, global, etc. I'm not sure setvar() is the right API to change those, because you should NEVER turn a non-magic into a magic (after shell init), readonly variables veto being updated, and globals should already be in TT.vars except during init and export? Having the flags API brings up "ok, so when you DO change them what happens" and it's a minefield of "don't do that" and "this should never happen".

That said, setvar() like findvar() returns the struct sh_vars pointer for the new entry, and if that's never null you can just setvar(blah)->flags = newflags (or |= flag). The one time I might want it to be null is if you update a readonly, but then I'd need to check to avoid segfaults and I dowanna. (Mostly because the checks would be of the "should never happen" type, ala calls during init creating the initial round of magic and readonly variables.)


March 28, 2020

The GOP is using the pandemic as cover to outright steal native people's land (due to a personal grudge). (Can we impeach him again? Yes I know the confederate christians remain racist. This is another reason the 90% top tax rate is important: the rich are assholes because they don't need to be nice.)

Meanwhile, I am so tired of Bernie's profound lack of effectiveness. He's been a professional politician for 40 years and he still can't make a decent case of his own policies.

If your house catches fire, you call the fire department. You don't worry about the bill. If someone breaks into your house you call the police. You don't worry about the bill. The point of medicare for all is even when you have to call an ambulance you don't have to worry about the bill.

This is why we pay taxes. Every kid gets 12 years of full time school their parents don't pay any extra for (and even if you don't have kids you WERE one). You can drive to work and the grocery store on roads that are already paid for, and you don't put coins into every stoplight and streetlight you pass to keep the electricity on. A functioning country has shared infrastructure, including people to run it. Doctors should not be different than teachers, firefighters, and police. It would be INSANE to buy fire insurance or police insurance that only let you use approved providers in-network which you have to identify via phone tree. Giant corporations profiting off the work of teachers and police and firefighters (while understaffing them to the point they have to work 90 hours a week with individual shifts lasting 24 consecutive hours) would be DEEPLY WRONG.

How is this HARD? If you grew up in an abusive environment, you may not realize he's not supposed to get drunk and do that to you, but once you figure it out a light bulb should go off. This is not normal. It needs to stop. The people defending it are monsters.


March 27, 2020

Finally heard back from ELC that the one and only talk proposal I submitted (Building and Booting Linux in a single 250 line shell script) has been turned down. (Well, it's been "waitlisted" and their form email is very apologetic about the volume of applications and so on.) This is why I submit 4 proposals when I'm serious; I halfway expected to be in Tokyo during this. Now I expect it not to happen. (Texas Linuxfest sent me their event cancellation email earlier today, it's been bumped to 2021. The Linux Foundation holds these events for profit, and isn't willing to let go of the money yet.)

Ok, redoing the variable logic to track state (all the "declare" flags). The functions that access the local variable state directly are: setvar getvarbylen zapvar run_subshell subshell_setup unset_main export_main. So that's a get, set, and delete function, an export and import function, and then the unset and export builtin commands.

But _indirectly_, findvar does (TT.locals is passed as an argument, so it cares about the format of the array it's traversing). That function traverses both the local and global lists using the same code, but the local variable format is changing (to include flags) so now they need different code. It's only called 4 times, 2 for locals and 2 for globals, and is just a wrapper around a for loop, so not a big loss to inline it at the call sites.

Hmmm, if I'm flagging SOME globals in the new TT.vars list (replacing TT.locals: change the format, rename the variable), possibly I should have all of them there so I don't have to search both lists? If sets and unsets always through through getvar/setvar/zapvar after shell init, then the string can just point to the environ entry.

Heh, "SECONDS=12345 bash -c 'echo $SECONDS; sleep 1; echo $SECONDS'" uses the inherited SECONDS value to init, but do the same for $RANDOM and it doesn't. Why isn't bash consistent? LINENO and GROUPS also ignore it, the only one that seems to care is seconds. (But you can assign RANDOM at runtime and it matters, I.E. re-seeds the sequence. Is this a security thing or just an oversight?)


March 26, 2020

Ya think? Go HEB! Thoughts and prayers. Of course. This should end well.

In case you were worried the stimulus might cause inflation.

How Covid-19 infects the lungs, and why some people are more vulnerable. (Which is different from why some countries are more vulnerable.)

I'm not sure which part of this to be most concerned about. (Leaning towards THE TRACKING. People have their phones on a beach.)

A reminder that society is something people voluntarily do, and capitalism is something we collectively choose to produce each day. If one day we all decide not to, then it simply doesn't happen. (That observation was in David Graeber's Bullshit Jobs book, but it isn't really a remotely new one.)


March 25, 2020

So the $2.5 trillion bill only spends $250 billion on checks to individuals, 1/10 the total. 90% of it is NOT checks to individuals. They could have sent everybody checks each month for a YEAR with what they're wasting here, but no. They needed 1400 pages of reporting on greenhouse gas emissions (but not fixing them), $32 billion to passenger airlines (the same amount they spent on stock buybacks over the past 5 years), and they gave Steve Mnuchin a slush fund to "loan" to his pet corporations (at zero percent interest forever, I'm guessing) that's TWICE as went directly in checks to indivduals. And, as everyone keeps pointing out, $25 million for the JFK center for the performing arts, because the democrats couldn't help themselves.

What is this nonsense about "unemployment benefits"? I've never ONCE qualified for unemployment in my ENTIRE CAREER. Sure it would be nice but half the medical gofundmes I've asked Fade to send money to (I know too much about how the sausage is made to ever be comfortable entering a credit card number into a website) involve people losing their jobs with never a mention of unemployment. Do these septuagenarian assholes think people still get pensions too? This does nothing for anyone in the gig economy (your lyft driver has no rides, part time hourly worker gets no hours this week at three different jobs), it's a Boomer-only-benefit.

So Fuzzy gets nothing because she can't file her taxes this year or last year because the social security office has her birthday wrong, so the IRS won't accept her return. She didn't see fixing it as a priority because she made less than the federal poverty line last year, so owes nothing. She started _trying_ to fix near the start of the month when the social security office assured us it would be open again on friday. The social security office is now permanently closed and can't handle a birthday change over the phone, and she can't log in to the website unless she can guess how they got the birthday wrong (including what year).

Meanwhile, the checks are the full amount up to $75k and nothing above $99k, so if they're going by last year's taxes when I was in a Milwaukee efficiency apartment working at Johnson Controls (boring but lucrative, I bailed out SO many friends who were in tight spots, at least three of which involved sending them 4 figure checks because their car was totaled) then Fade and I get nothing, and by this year's taxes when I'm back at SEI working from home and earning what they can afford to pay me (it just about pays the bills each month), Fade and I would get the full amount. (Again, you can just give EVERYBODY the checks and then TAX THE RICH. I happily pay 5x as much taxes in years I make more money than in years I make less money. That's how it works.)

So a very tiny amount of this stimulus package MIGHT go to SOME people in need, but leaves people I know out, and the rest is corporate welfare and Boomer Benefits, leaving everyone I actually know still screwed. Great. Just great. UBI is UNIVERSAL. That's THE POINT. STOP WITH THE MEANS TESTING. BUTTGEIG LOST.

The republicans cannot disappoint me. The entire festering plutocratic party is pure unmitigated evil and needs to go the way of the Whigs and Federalists before it. They are completely unsalvageable, there is nothing whatsoever to save. But the democrats are VERY MUCH disappointing me, and they need a hard retirement cutoff at 65 where you get a gold watch and can't be a legislator anymore. You can work as a staffer, be a lobbyist, fine. But not house, senate, judge, or president. (If you qualify to be president at 35, you get a thirty year window. That should be enough. Our youngest presidents include JFK, Bill Clinton, Teddy Roosevelt, and Barack Obama. Our oldest two were Ronald Reagan and the current clown. FDR was 51 and Lincoln 52 when they took office. George Washington left office 10 days after his 65th birthday.)

Meanwhile, I'm told Canada is sending people $2000 a month until this is over and Scotland might make it permanent, Ireland nationalized its healthcare system, and everyone else is starting to notice some obvious truths. (And AirBNB needs to be stopped.)


March 24, 2020

It turns out OPEC basically fell apart last year. Solar continues to kill fossil fuels in solid-liquid-gas order.

Interesting point that you don't need "hoarding" when a 10% increase (from people eating out less) will crash just-in-time supply chains. (Which is also what happened to toilet paper. If the machines were going 24/7 before this, they can't step up production.)

Ok, the problem with the sh4 build turned out to be triggered by the binutils update back in January, just like the problem with m68k and s390x was triggered by the kernel header change back in December. Rich never noticed (or regression tested) either. Wheee.

There's a two part kernel patch that might make it into 5.6, which would build with the new binutils. But for the moment I can specify the non-broken binutils version in the mcm Makefile just like I'm doing for the kernel headers, and build a non-broken toolchain to make root with.

I've poked Rich (who is sick, hopefully not coronavirus) and here's hoping he can get those two patches upstream before sunday's 5.6 kernel release. I'm holding the toybox release until Monday to see how that plays out. If not, I build the make root binaries with 5.5 and the tweaked toolchain. (I'm not posting the toolchain binaries becaue GPLv3, but thalheim might. I should poke him once I know.)

Meanwhile, if I'm delaying the release until monday anyway maybe I can take one more stab at getting toysh to run the init script? (Adding "export" triggered a redo of the variable infrastructure, which is ugly however I look at it, because array_add() elegant-ish-ly maintains an array of char *. I can have an array of structs that array_add() doesn't understand, I can have two arrays (one of flags, one of variables) that have to stay in sync, I can have in-band signalling in the char string (which would mean I can't use the existing global key=value strings inherited from the environment)... It's not _hard_, I just need to pick an approach from a menu of unpalatable options.


March 23, 2020

It is difficult to applaud the democrats for using this crisis to forgive student debt while the republicans are using it to outlaw abortion. JUST SEND PEOPLE THE UBI CHECKS ALREADY YOU BOOMER ASSHOLES! Stop it with all this unrelated wrangling to promote your existing agendas! (Pass seperate bills! You agreed on checks to individuals, you should have passed a clean bill with JUST checks to individuals the day after the White House agreed with Romney on that. Broken clock, twice a day, let's go. It could have been one page, not one thousand, four hundred and four pages, and it could ALL have gone into checks to individuals rather than only 20% of that $2.5 trillion.)

Seriously, HEB is doing "day before thanksgiving" grocery sales every day and giving its employees hazard pay. Dollar General is hiring 50k people to meet demand. If "the economy" is collapsing it's because most of "the economy" is COMPLETELY FICTITIOUS. If the "businesses" going out of business are NOT keeping people alive, why are they a priority now? Yes, I'm sad Lucifer Season 5 has halted production, but we can wait. If the owners and employees of small businesses like The Omelettry and Toy Joy down the street get the same checks as everyone else, then you HAVE helped those small businesses. Maybe not enough, but it's a START.

People have also been pointing out that cruise ships which register in Bahrain to avoid US taxes should ask them for the bailout. And DON'T bail out airlines that spent $36 billion on stock buybacks over the past 5 years rather than build ANY cash reserves. Only the 1% will be able to use them after October anyway. (The TSA needs to be abolished just like ICE does.) We can afford to lose a couple airlines (which is nothing new, remember American, Continental, TWA...?), and whoever buys the planes and hires the mechanics is unlikely to do a WORSE job than united or boeing (pick your article). We have plenty of spare airplanes of the old fossil fuel kind, and what we NEED is to switch at least the short haul ones to electricity (which might be the one place hydrogen fuel cells don't suck, who knows).

Part of the need for airplanes goes away if we get self-driving cars widely deployed (which would happen so much faster if Uber and Tesla would stop trying to "help" here and just let Waymo get on with it in a regulatory environment NOT poisoned by dangerous morons needing to be reigned in from killing people. At this point it looks like we'll just let other countries do it and then import the results as a passive consumer). Eventually self-driving cars can do trips to other cities ala U-haul. If I have to leave for the airport hours before my flight, a car that picks me up at my door gets me to my destination in Houston faster than an airplane can. At 70 mph on interstates you've gone 200 miles (Austin to Dallas) before the plane takes off, and the breakeven point is probably more like 4 hours (Los Vegas to Los Angeles; this is why "tour buses" are a thing for musicians and such).

That's an awful lot of short haul trips, and the self-driving car is one uninterrupted block of reading/watching/working/nap time where you can recline your seat more than 5 degrees. Heck, once the batteries and safety standards are up to it, people comfortable sleeping belted into a reclined seat could leave Austin at 10pm and arrive in Memphis before 8am. Once the infrastructure scales up there's no reason national driving wouldn't be flat rate just like VOIP doesn't charge for "long distance calls". The actual costs are neglibible if the electricity comes from solar and the vehicles last a million miles each. (The electric motors and car bodies already do and the batteries should soon, right now the tires are the weak point.) Yeah, national door-to-door flat rate is probably 50 years off, and it would be slower, but faster isn't always the sustainable business model. The point is bailing out today's airlines without even demanding equity is not something we HAVE to do, it's merely defending the past. Which of course septuagenarians can't help doing.

Right now I care about checks to individuals, the rest of "the economy" can go hang. (The bits that belong to a billionaire can go guillotine.)


March 22, 2020

Alright, think I figured out how to handle labeling globals. Only the ones with attributes (which is not a common case) need to be moved into the locals array, and I can use the standard "delete out of global array" plumbing to do it because I copy and allocate the new version with the extra data when the attribute gets added. Those would then be "export to globals at runtime" but the test is quick (just flags, not a strcmp).

I could even keep them in the globals and just have a blank "NAME=" entry in the locals storing the flags. I should probably do something similar with magic variables. For globals I could keep a counter of how many such annotated globals there are so I don't have to search the locals to export globals when there's no reason to, which I'm torn on. (On the one hand it's fiddly, on the other this is _the_ hot path. But we already go through variable expansion and stuff. Hmmm...)


March 21, 2020

Still reading about oil. Here's a smart guy who was saying the right things 11 days ago (accurately predicting, as opposed to the dinosaurs insisting the meteor is just a fad). So let's see if he's said anything recently... yes, he has a twitter account, and... yup, going to zero. Here's another guy who seems to have the right idea... and he also has interesting tweets, especially the inside baseball soap opera stuff. (It's a pity I can't reliably play podcasts at 2x speed the way I can youtube videos.

Sigh. I wanna go "yes, toybox has this, look at xabspath" but it would be impolite to do that on the busybox mailing list. (I mostly don't post at all, largely because I check up maybe twice a month and the conversations are usually way stale.)

I want to get a toybox release out that can have "make root" binaries for each target, meaning I can post a cpio.gz initramfs for each target that boots to a shell prompt, which is the main reason I've been trying to get toysh to the point it runs the mkroot init script first instead of just tying off what I've got. But I suspect the variable subsystem redesign puts it out of scope unless I can come up with something clever this weekend.

Still, I've got the compiler build fixed and now I'm testing the targets, and sh4 isn't booting? I run qemu and get an almost immediate exit with no output. Hmmm.

Back up to the previous kernel release: no change. Kernel before that: no change. Ok, the kernel version, build script, and possibly QEMU build have all changed since the last time this was known to work, so I need to reestablish a baseline. Clean clone of aboriginal linux, ./build.sh sh4" and run _that_... and it boots! So QEMU is ok. And that gives me a kernel to try to boot the cpio.gz with and... it's the new kernel. Ok, most LIKELY what happened is the kernel config changed and it hasn't got a serial port enabled. So did the kernel version advance or did the build script produce a different config output? Hmmm, that kernel (with the same config) doesn't build a working version with the new toolchain. The aboriginal toolchain is ANCIENT, ummm... build with the old mcm-buildall.sh out of the external mkroot and musl-cross-make checked out from a year ago and... yes, baseline along new path reestablished. Ok, walking forward... toolchain builds take a long time.


March 20, 2020

Last week I started studying the oil price war between Russia and Saudi Arabia. This week I'm following the collapse of the oil market. Some of the more useful coverage is from Saudi Arabia's neighbor Al Jazeera.

The price of oil fell 2/3 recently. It was $60/barrel In February and yesterday it (briefly) hit $18. Demand is down due to coronavirus' impact on the global economy, and production is UP due to a price war between Saudi Arabia and Russia that started last week (more or less the end of the OPEC price-fixing cartel). Analysts think the combination may drive oil prices literally to zero.

Saudi Arabia needs to sell oil at $84/barrel to balance its state budget. Five years ago Russia needed the same, but it recently made signifcant budget cuts (noticeably reducing Putin's popularity) and now only needs oil at $42/barrel to balance its own budget. (This is assuming global consumption remains fixed and each country maintains its market share. Yes oil prices going down slightly increases consumption, but not enough to matter here. People still spend less money on oil when the price is lower. You don't spend twice as long driving because gas is cheaper, your commute stays the same.)

Last month oil was halfway between those two figures, so Saudi Arabia pushed for production cuts to raise the price and Russia refused (arguing the US fracking industry would just make up the difference, but mostly just not needing a higher price to make ends meet). Saudi Arabia's leader MBS (who like Putin murdered his way to the top even before ordering Kashoggi's execution) declared economic war on Russia, increasing Saudi oil output in a simple "he defied me I must punish him" knee-jerk response. MBS did this just as the Coronavirus became a global pandemic, so everybody started self-quarrantining and stopped commuting (let alone traveling long distances) meaning production increased just as demand collapsed, which is why the price of oil has dropped by 2/3 already.

Just like with Trump everybody's seeing faces in clouds in this strategy, and projecting chess master moves onto random thrashing. Both leaders are oligarchs, which means they're bullies who inevitably become surrounded by yes men (by process of elimination). They're more concerned with appearance than achievement because their power rests entirely on the perception of their power (defy me and I will make you regret it, that is why you must obey), which is a feedback loop that goes sour quickly. (If anyone ever defies you and gets away with it, the emperor has no pants.) Analysts' fan theories about deeper reasons aside, neither one can admit a mistake and back down any more than Trump can: they would look weak, which would make them _be_ weak.

Putin and MBS each have about half a trillion in foreign currency reserves which they're spending down, in theory to outlast the other guy, in practice probably in hopes of driving at least third parties out of the market. But at what point is venezuela _not_ going to sell oil if it can make a profit doing so? (They may stop for a bit, but they'd start up again as soon as there's money in it, because they need the money and have few other immediate options.) Both oligarchs are publicly hoping their oil war will at least bankrupt the north american oil producers, but their governments are bailing them out, and Exxon buying up small companies doesn't mean those fields would lie dormant for long.

So all the producers are trying to outlast each other by pumping MORE. The less oil brings in per barrel, the more barrels you must sell to pay the bills. Even if the price falls so low it costs more to pump than you get, those costs are borne by a bank and the cash from the sales goes into your pocket NOW. Drilling the well is a sunk cost, you PUMP to pay it back.

This excess production is filling up all the oil storage in the world, tanks and ships and caves and pipelines. The US stragegic petroleum reserve is 112 days from filling up. (It can store 685k barrels per day to a capacity of 77m barrels.) China's strategic petroleum reserve _started_ full as far as anyone can tell. (The first Al Jazeera report above said China was turning back tankers "due to coronavirus", but with their economy idled and their SPR full why would they buy oil? No matter how cheap, where would the PUT it?) And when the US SPR fills up, that's when analysits think the price of oil may fall to zero. If there's nowhere to put it, you can't sell it at ANY price.

Even if the economy recovers, there isn't an easy recovery from this for the oil producers, because outside of SPR all that storage is finance bros buying to sell it at a higher price later, and unloading those positions is going to compete with any price _rise_. (Especially if it's been a while and they give up and sell rather than waiting for some target number, since renting storage costs money...)

Analysts who don't understand exponential curves insist this is somehow bad news for solar, but it's vehicle costs, not fuel costs, that have been holding back electric cars. (Tesla made electric charging _free_ for its first decade or so, and it's still cheaper than oil for fueling cars even at the new lower prices. And gets cheaper every year.) This curve from 2016 predicted the $30k electric car accurately, and places the $20k electric car in 2022 and $10k in 2026. If the average price of a new fossil fuel car doesn't come down from $36k, buying one stops making sense long before oil prices could recover, no matter how cheap the fuel is. (I've written about this before, and probably the most damning link from that is the loss of institutional memory in oil refining. They're not investing in R&D anymore, no point.)

Which gets us back to the whole "electric transportation as a service displaces vehicle traffic on US highways", and switching trains too (diesel-electric is already electric, maybe you need 4x as many cars to hold the batteries as hold the diesel but the train's already 50 cars long anyway). And THAT leads right into the oil supply chain becoming unprofitable. When the cost to pump, refine, and deliver gasoline to a gas station is higher than the sale price, gas stations close. The end retailer's profit margins are already razor thin.

Energy is 1/6 of the global economy but is concentrated into a small number of hands. That's why of the 10 largest companies in the world, numbers 2, 3, 4, 6, 7, and 8 are oil companies. (#5, China state grid, is electricity. They're pushing for Solar and batteries as much as anything.) But even if there's still demand for diesel for international flights, container ships crossing the ocean, and making plastic and pharmaceuticals, without Beverly Hillbillies levels of profits it's not a big enough industry to let idiots control governments anymore. And that's the point where climate change lawsuits come home to roost.

My next question is if solar is killing fossil fuels in solid/liquid/gas order is what happens to gas now? So far it's lost in the noise...


March 19, 2020

Of course she had a plan for that.

So the literal nazis are saying "checks for everybody", and the democrats have turned that into something that wouldn't help anyone I personally know. Fscking _Mnuchin_ is to the left of the DNC on this one. Unbelievable. Howard Dean is 71. Maxine Waters is 81. Biden is 77. We're all volunteering for martial law to fight off the Boomer Doomer (the same way taking off your shoes at airports was "voluntary" after the shoe bomber until suddenly it wasn't) and the geezers are STILL trying to drive. I am not happy about this.

Still working out how to make a variable global and active at the same time (ala "export RANDOM"). If you "export RANDOM; unset RANDOM; echo $RANDOM" it prints nothing, which means in unset_main() the unset magic part has to fall through to unset global/local because it CAN exist more than once and gotta catch 'em all or you won't get mewtwo.

A rough edge in toysh is that the local and global variable blocks are updated by different infrastructure with a different allocation stride. (What "stride" means here is that the easy way to know when you need to realloc() an array you're appending to is and it with a power of 2 mask, and when it's all zeroes add the stride size, ala if (!(size&(stride-1))) array = realloc(array, (size+stride)*sizeof(*array)); .)

In toysh, TT.locals is updated by array_add() with an allocation stride of 32. The exported globals live in "environ" which xsetenv() updates with an allocation stride of 256. Which means an xsetenv() update can fall off the end of an array_add() array because it assumes enough memory was allocated to get to the next multiple of 256, and the other one only extended it to the next 32..

I wrote lib/env.c to address the classic environment variable memory leak problem, and it did that, but it was designed to work on a single array with a single "inherited variable, do not free" counter. So I genericized it a little, letting you call it on an arbitrary array with an arbitrary counter of "special" entries at the start. But that counter doesn't track how many entries there are, it tracks how many entries were inherited from the environment and thus shouldn't be freed. While the TT.locals counter is how many read-only entries there are which should error out if you try to assign them. (You can unset them, but not overwrite them without unsetting them.)

The array_add() logic is natural to use in toysh, which implies I should do all the variable manipulation locally in toysh. But the env.c logic is useful for things like tar which set and clear environment variables for each file of an arbitrarily long input, so leaking each time (via the normal libc functions) could exhaust memory on small systems or in pathological cases. There's a USE for both. I'd like to unify them, but the assumptions still don't match. Which says I should dismantle the env plumbing some more and try to cleave out a layer that matches both sets of assumptions (and either uses a coherent stride or updates both sides with the same plumbing).

The reason for the shifting assumptions is new use cases coming in. I've tackled shell local, global, magic, and readonly variables. (Magic is like synthetic filesystems: the value is calculated every time you read it, ala $RANDOM or $SECONDS.) I have NOT yet dealt with integer, nameref, indexed or associative array, upper or lower case mapping, or functions.

Darn it, now I see why bash has a single attribute annotated variable table and performs exports at exec time. The darn attributes are combinatorial. You can export magic variables to be global and they remain magic, what other combinations can... Really, bash? REALLY?

$ readonly SECONDS
$ echo $SECONDS
8
$ echo $SECONDS
9
$ SECONDS=42
$ echo $SECONDS
44
$ echo $SECONDS
46
$ declare -p | grep SECONDS
declare -ir SECONDS="46"

You can set the readonly attribute on a magic variable and it _remembers_ it, but ignores it. (Same for RANDOM, assigning a number to that re-seeds it so you get a repeatable series of numbers afterwards. And yes, "readonly PATH" is enforced. And reported by declare -p.)

So this implies I have TT.locals stick a byte at the start of NAME to store the flags, but I want to keep startup fast so #!/bin/bash scripts have minimal starting latency, and copying arbitrary environment data (possibly megabytes of it) into a different format is a potential latency spike. I suppose I could have TT.locals be an array of struct {long;char *} pairs, where the -i flag means the char * is actually another long. The problem is, if I populate the char * with the "NAME=VALUE" string out of environ, I need one of the flags dedicated to saying "inherited, do not free"...

And I've already planned to use pre-digested functions to avoid re-parsing it over and over. So "function" isn't an attribute of a variable, and in fact:

$ potato() { echo hello; }
$ potato=42
$ echo $potato
42
$ potato
hello

So we _can't_ have a unified single table. Locals, functions, and globals are logically distinct. (And magic has to be its own table too.) And I haven't yet opened the array variable can(s) of worms. Hmmm.

What I _want_ is the first byte of locals to be flags (to avoid a wrapper struct storing the flags and a pointer to the "name=value" string) but also to NOT have to copy globals (to avoid the startup-time hit of copying arbitrary amount of environment space). How do I handle readonly globals then?


March 18, 2020

People are going to be studying the climate impact of the coronavirus lockdowns for years. And then when the wheels come off the climate, we're probably going to have to do them again. Meanwhile the new cold war continues, with the GOP being our fifth column.

The first two error messages when a mkroot image tries to boot its init script with toysh are failure to exec "export" and "!", both of which are shell builtins. Of the two export is easier to do because of the "! isn't a good function name" issue. (Gotta either find a main_name() to glue it to or just hardwire it into the run_function() path as a special case.)

So I wrote an export, and redid some of the variable infrastructure along the way (most loops were going s=='_' || !ispunct() to determine how long a variable name is, but one was doing isalnum() instead of !ispunct, and BOTH are wrong because !isspace. Really it's s>' ' && (s=='_' || !ispunct(s)) which allows utf8 variable names, and no it doesn't end early because of utf8 whitespace or punctuation, but I'm ok with that).

Except... bash has very different environment variable behavior than I've designed.

$ export RANDOM
$ echo $RANDOM
31896
$ echo $RANDOM
2222
$ echo $RANDOM
3083
$ env | grep RANDOM
RANDOM=3083
$ env | grep RANDOM
RANDOM=3083
$ env | grep RANDOM
RANDOM=3083
$ echo $RANDOM
18437
$ env | grep RANDOM
RANDOM=3083

So it DOES have separate local and global lists, but it defers copying variables from the local list to the global list to launch time (instead of doing it _in_ export), and then won't re-copy them if they're already there? (Export is both deferred _and_ persistent?)

I just did the move in export (and I try never to have an equivalent variable on both lists), but I haven't handled active (or readonly) variables at all yet. This was me going "so what does it do with exported active variables" and trying one out to experimentally determine the behavior...

I don't really _want_ to copy this bash behavior? I am AWARE of it now, but I think I'll put it in the "wait until somebody complains" bucket because it implies more infrastructure than can possibly be a good idea.

The larger design question is how a global variable remains active, and I was thinking "it probably doesn't" for the same reason "cp /proc/self/stat ." isn't going to update after the cp. But that's not what bash is doing, and thus I have to find a way to not break existing scripts.

I've currently got a char *env[] with counters for both locals and globals, and the counter is "how many entries at the start are magic". For globals it's "how many were inherited from environ and thus should not be freed", and for locals there's a readonly counter and a magic counter after that. But I've ALSO been thinking of switching the arrays to be sorted so they binary search, except that penalizes insert. Probably the right thing to do is have an envc and localc and start searching from the _end_ each time, so you get natural move-to-front behavior and recently set variables are the ones you search for first, which is also gonna be what's in CPU cache. But again, make it WORK first, THEN benchmark, THEN optimize...


March 17, 2020

I wasn't joking either

I did not get on my plane to Japan this morning, instead the trip's rescheduled for the end of April. I don't THINK I have coronavirus? I think I have a mild case of the flu, but I am coughing and short of breath and had a nonzero chance of being stuck in chicago and not allowed on the flight to vancouver. (Yes, it was Austin -> Narita but there were two transfers along the way, one of which is through Canada's semi-sealed border, and the flight there departs from the airport being held up to exemplify "long lines".) Whatever I've got it's been building for about 3 days, and I'm not concentrating at anywhere near capacity right now. (I have all the symptoms of coronavirus _except_ the fever, and they're not intense enough.)

So I work from home. Right now I'm building a raspberry pi 2 buildroot defconfig to see what's in its' root filesystem, so I have a baseline for what we should put in a new Turtle board root filesystem.

I finally made a good current-ish set of toolchains for all the targets, using current musl-cross-make. I backed gcc off to 8.3.0 and switched to the full Linux headers instead of the weird stripped-down ones that don't support m68k or s390x. (I told Rich about this a while back, but he doesn't seem to have committed a change to mcm. Oh well.)

I then built all the targets (scripts/cross.sh all make root) with toysh switched on in the config despite being in pending, and 19 targets built a toybox binary! (This is my new procedure for creating release binaries, which is why getting all this working was tied into getting the next release out.)

This means 3 targets failed to build, but they're armv7r (which has never quite worked, it's nommu armv7 and I need to change the config to static PIE or something, you can't do regular ELF nommu) and x32 (which has always been a bit weird). And powerpc64le didn't build a kernel, but I don't think I ever gave it a kernel config anyway.


March 16, 2020

The CDC says to cancel gatherings of 50 or more people for the next 8 weeks. Airplanes for international flights hold something like 450 people. The first transfer of the flight Jeff wants me to take tomorrow is through Chicago O'Hare. Right.

Fuzzy was hired to sanitize the fencing masks at her club over the break, and she's been trying to get clorox wipes ever since. I thought I'd go to HEB when it opened this morning (they're now closing from 8pm to 8am each day to restock), and... nope. No clorox wipes. Half their shelves are still empty even after 12 hours closed. I expect their warehouses have run dry of rather a lot of stuff. (Last week I was upset they had trouble keeping cans of milk tea in stock in the intergalactic foods aisle. They've got plenty of that now. Bit of a monkey's paw solution, though.) I went to Target to buy more shoes and pants (Tokyo hasn't got either in my size) and they didn't have wipes either. Nor did the walgreens on the walk back.

Has anybody pointed out that "everybody's panic buying so much that grocery stores and target and stuff can't keep anything in stock and the lines to their registers are fifty people long" is coinciding with "the economy is collapsing because economic demand is down"? The ONLY way both statements could be true is if the economy is COMPLETELY FICTITIOUS. The stuff that people actually need to survive is such a tiny part of "the economy" that it doesn't even REGISTER to policy makers. This is why we can totally do basic income, because all the rest of "the economy" is a game billionaires play to keep score. (Hence David Graeber's book, which his publisher tacked "A Theory" onto because they were scared to say it might be true and needed weasel words to sleep at night.)

Lovely article about economics. And there's another bill trying to outlaw encryption using the "some people do horrible things and that's why we must have cameras installed in everyone's retinas" argument. If the thing someone somewhere did is bad enough, that's why your house must be federally bugged just like East Germany. Not buying it? Ok, here's a WORSE thing someone once did, your big brother installer will be there tuesday. (Guillotine the billionaires already.)

Ok, the toysh $() parsing logic starts roughly line 640 of sh.c, and calls skip_quote() which is not working on (). And it fundamentally CAN'T work on $((a)|b) so I need to replace skip_quote() with parse_word, as mentioned yesterday... ok, did that.

And now I hit the fact that the TEST that was failing is "echo $(<input)" which... makes no sense? As in I can implement it but what is BASH doing here, exactly? The child's stdin is redirected. What does that have to do with OUTPUT: <input on a line by itself does not output the file "input". Bash accepts a space between the ( and the < but "echo $(X=Y <input)" doesn't output anything, nor does "echo $(<input X=y)" nor "echo $(<>input)". This is totally a special-purpose hack bash added. So the question is where to stick it to duplicate their horrible special case's logic... Sigh, seriously: this bash hack is ONLY for $(), not even "cat <(<input)" works.


March 15, 2020

Feeling under the weather today. Occasional cough. Very slight sore throat. Maybe shortness of breath or maybe imagining it? It's probably pollen (it's raining outside), but it's really easy to psych yourself out right now.

So the next toysh test that's failing is "echo $(<filename)" which I put in because it's a horrible thing bash does. But the _way_ it's failing is "syntax error: bad )" which means the $(blah) parsing is failing, and... yeah, I need to re-unify the quote traversal, I just hoped I could defer that a bit.

The problem is that parse_word() has plumbing to figure out that $((abc) | d) is NOT a $((math) but is a $( ( subshell ) ), which can only be done retroactively: it's when the quote count goes negative in (( context without ending with )). And no, you don't parse the contents:

$ echo $((echo hello))
bash: echo hello: syntax error in expression (error token is "hello")

It's the decoupled terminal parentheses that do it.

So I implemented that logic in parse_word(), but haven't re-implemented it in skip_quote(). Instead I want to remove skip_quote() and just call parse_word(), but I need a way to signal to parse_word() that it should return whenever it's NOT in a quote context. (Right now it continues to the end of word, I need it to continue to the end of character or quote.)

Meanwhile, I want to test my new shell against the mkroot init script, but building "make root" needs a cross compiler for each target, which reminds me I left off trying to update the toolchain builds where musl-cross-make had switched to a new gcc version that no longer understands --with-float=vfp for the arm targets, and the insane gcc docs now say that I have to micromanage -mfpu=vfpv3-d16 vs neon vs vfpv4-d16 vs neon-vfpvr vs neon-fp-armv8 and I dunno what I _need_?

The armv5l target is running in qemu's -M versatilepb, so I looked in the qemu source code that it's initializing "arm926" which can have an arbitrary FPU as a coprocessor. Grrr. (In theory the kernel can intercept attempts to do VFP and do its own floating point for it but I was pretty sure QEMU was doing this? There's proper hard float with a coprocessor (fastest), there's the kernel faking hard float (slower), and there's software float where the compiler inserts function calls to use integer math instead (slowest). I want the binaries I build to use hardware float on real hardware, because that's the common and fastest case. My kernel config is setting CONFIG_VFP, which doesn't say what TYPE, I don't THINK the intercept-and-emulate plumbing is enabled in the kernels I built for qemu?

Ok, it looks like armv5t is going to have vfpv2, and of course different insane gcc docs mention the older VFP types the previous insane gcc docs didn't. What's current for which gcc release? Who knows? (This project isn't being replaced by llvm fast enough.)

Anyway, reading gcc's configure plumbing in the source, it looks like what it recognizes for --with-float on gcc/config.gcc line 3845 are "hard", "soft", and "softfp". So let's try "hard" and see if it just does the right thing since it has ONE OPTION for armv5tl...

cc1: error: -mfloat-abi=hard: selected processor lacks an FPU
cc1: note: self-tests are not enabled in this build

NO IT DOES NOT! If you're going to use hard float, it's vfpv2! THAT IS YOUR ONLY OPTION. I SUPPLIED THE ARGUMENT BECAUSE THE PROCESSOR DOES _NOT_ LACK AN FPU! WHAT ELSE COULD YOU POSSIBLY THINK I MEANT HERE?

Ok, try adding the +fp to --with-arch... nope, unrecognized. But -march is a RUNTIME option not a CONFIGURE time option. How do I cram this information into GCC's stupid configure system... Ok, "armv5l:eabihf:--with-arch=armv5t --with-fpu=vfpv2 --with-float=hard".

Dear LLVM developers: get a move on please. GCC is not dying fast enough.


March 14, 2020

I wrote a longish email reply I might as well cut and paste here. I get tired of repeating myself, but "nobody knows what you've done" remains true. (I.E. you can give the same talk ten times and STILL most people won't have heard it.)

> but, yeah, the list of stuff you've worked on covers just about
> everything!

I keep saying "I break everything". Nobody takes me seriously, but:

https://landley.livejournal.com/55895.html
https://lists.linuxcontainers.org/pipermail/lxc-devel/2011-January/001615.html
http://lkml.iu.edu/hypermail/linux/kernel/0503.3/1756.html
http://lkml.iu.edu/hypermail/linux/kernel/0407.2/0637.html
http://lkml.iu.edu/hypermail/linux/kernel/0207.2/1182.html
http://lkml.iu.edu/hypermail/linux/kernel/0104.1/1008.html
http://lkml.iu.edu/hypermail/linux/kernel/0105.0/0045.html

Seriously, I'm doing CPU design now. I'm having to study the guts of USB plumbing because we're trying to implement it in FPGA so we can make a chip out of it. (There's a problem that Kintex is fast enough to do 100baseT and maybe USB2 if we push, but gigE and USB3 both clock faster than the FPGA can cope with, so how do you TEST your design before sending it to a fab shuttle, which is 70k? I mean yeah, you can simulate it (entirely in software at 1% real speed), but you can't plug real devices into the simulation...)

> i didn't know you'd worked for parallels until i saw your
> old email address in the 9p thread you linked to.

Specifically I worked for OpenVZ (which Parallels acquired) who were chipping off pieces of their lightweight virtualization solution and rewriting them to get past Linus into the vanilla kernel, where it became known as "containers". (Kir Kolyshkin was there from the beginning, it was invented by a Russian bank back in 1999 to try to secure Linux to do banking transactions on, where it was known as the "bean counters" patch, and then spun out as a separate company...)

There's all sorts of stuff on my old livejournal about that, starting with https://landley.livejournal.com/42214.html and pretty much every post from there through https://landley.livejournal.com/57036.html was about the tech I was working on. I occasionally posted a todo list du jour, ala https://landley.livejournal.com/56696.html

In THEORY my main job on that contract was to containerize network filesystems (such as this) but I suspect the most _useful_ thing I did for them was run their booth at Scale in 2011 (with Kir Kolyshkin) where I think I think I did rather a lot to popularize the concept of containers (something nobody knew about in 2010, they were a russian company with _zero_ competent US marketing people). If you've heard the phrase "chroot on steroids" that's was me. I put together a quick 15 second pitch that turned into a 60 second pitch when I fished somebody from the crowd, and then handed them over to Kir to do a live demo of "this x11 session with animated screensaver is live-migrating from machine to machine, see that 1/3 of a second pause there? Now it's running from _this_ box... and yes with multipath you can plug it into two switches so you can literally swap out the hardware under it with the program still running..."

All this led to "the cloud" a couple years later. There's a _reason_ I was poking Google about using containers years ago. And why I've wanted to do a toybox "container" command. (And have toybox implement criu, although both are post-1.0 material because I have a reasonable idea of the _scope_.) Still:

https://github.com/Fewbytes/rubber-docker

The _plumbing's_ not hard. Modulo criu, but then again for a while the most "viral" random email message I'd written that everybody cited was these two:

http://lkml.iu.edu/hypermail/linux/kernel/0206.2/0835.html
http://lkml.iu.edu/hypermail/linux/kernel/0206.2/1244.html

(Yes, I was thinking about that stuff in 2002. I am weird. At least since I wrote that they've implemented ways to ACCESS all that stuff, although the definition of all is a moving target.)

Rob


March 13, 2020

Well of course. (Now do Warren. Me, I'm still waiting out the boomers.)

Nope, HEB could not keep up. They restocked overnight, but it's all gone again.

Got my taxes filed, which was the last thing keeping me here. My flight back to Tokyo is now booked, scheduled to leave 6am tuesday morning and take 3 different planes to get to Narita. Here's hoping the remaining wheels don't come off between now and then...

I got eval and exec implemented, including making "eval x=y" work properly.

I keep accidentally testing the host "sh" instead of "bash", and wasting time wondering how an obvious annoying bug (like "sh /usr/bin" silently returns immediately with error code 0 even though it's a DIRECTORY) never got noticed, and then figuring out later "oh, that was the Defective Annoying SHell doing that, which nobody uses if they can avoid it. Bash doesn't get that wrong".)

The problem is my command is "sh" and the host command is "bash", and I keep taking the ./ off the front and letting it search the $PATH for the "host version", which is right for every other comand and wrong here. Sigh, I should just redirect the host /bin/sh symlink to point to bash like most people do, but I _don't_ do that to make sure my scripts say #!/bin/bash instead of accidentally saying #!/bin/sh and breaking for other people because of Ubuntu's old mistake.

I'm grinding through the tests/sh.test entries I've already added and fixing them up as I go. I've got most of the infrastructure they use in already. (I was adding tests for stuff I was working on as I figured out what behavior it _should_ have in various corner cases.)


March 12, 2020

The Google Fiber guys came and installed it this morning. I haven't noticed particularly improved throughput, but the latency is way down. Pages are _snappy_ about loading.

Yesterday the Resident gave a speech which so effectively reassured the nation that everyone started panic buying. HEB's shelves have steadily emptied all day, despite double the usual staff deployed restocking. (It's a 24 hour grocery store, so it has 3 shifts to pull from. It looks like 2 of the 3 shifts are here right now earning time and a half. Their supply chain is going "kaching".)

It's almost midnight and Fuzzy and I just watched a pallet of toilet paper have a police escort from the back of the store to the shelves. They're completely out of canned soup, had literally one box of white rice, and half the slots for meat, cheese, cereal, and bread are empty. Fuzzy says the only time she's seen the bacon that depleted was superbowl. Doesn't mean they're exactly _out_ of anything (except hand sanitizer, there's a nationwide shortage). It's the kind of privation where you have to get box soup instead of canned, have to buy _brown_ rice, and "they only had one package of my favorite ramen left, otherwise I'd have to get the spicy flavor". They're completely out of big jugs of white vinegar, you'd have to get a small bottle of fancy vinegar if you can't wait until morning.

Meanwhile there are maybe 50 people FRANTICALLY RESTOCKING while this is going on. I wouldn't be surprised if it's all back in a few hours (to be depleted again when people wake up, of course). I assume the truck drivers are similarly getting serious overtime. And there's still plenty of fruits and vegetables. (But no 10 pound bags of cheap potatoes, you have to get the 5 pound bags or buy one of the slightly more expensive varieties. The donuts and brownies and cakes and pies in the bakery are largely intact. Half the slots for cheap shredded cheese are empty, but on the other side of the store some of the Vegetable and Cheese trays have 50% off clearance stickers on 'em because they expire tomorrow.)

This isn't panic, this is nervousness. Nobody's been told what to DO, and 100 years of capitalist propaganda says the answer to any problem is to buy and/or sell stuff. Something must be done, this is something, therefore we must do it, at least until an adult shows up.


March 11, 2020

Replying to a mailing list posting, I went on a tangent and edited most of it out again, so I cut and pasted it here instead:

I dunno about minimalism, but I'm trying to get toys/pending/sh.c in under 3500 lines. I expected command line editing/history plumbing to live in lib/editline.c so not count against that total, but that was before vi.c was taken, sharing zero code with any editing plumbing I would do, so we'll see...

That's already 3.5 times my _normal_ high water mark for toybox commands. Yes sed.c is over budget by 80 lines but it's another big and complicated one (with a lot of help text so the actual C code doesn't start until line 172). And yeah, I could cheat and say without comment lines and blank lines it's under 800:

sed -e '/[ \t]*\/\/ /d' -e '/^$/d' toys/*/sed.c | wc
  778    3670   27470

But that's cheating: WITH sufficient comments and blank lines, and a header block with good help text, most commands should be under 1000 lines of C. I expanded the budget for sh, and what I have so far of toys/pending/sh.c is 2400 lines, maybe a hundred of which are debug code, so I've got a thousand and change left in that budget.

According to wc my 10 biggest commands so far are:

    483    2011   14522 toys/posix/patch.c
    502    1803   13764 toys/posix/grep.c
    524    1955   15732 toys/posix/cp.c
    550    2033   17399 toys/net/ifconfig.c
    559    2228   17528 toys/posix/ls.c
    714    3030   23657 toys/posix/find.c
    723    3269   23349 toys/other/bzcat.c
    956    3618   29559 toys/posix/tar.c
   1080    4857   34893 toys/posix/sed.c
   1936    8574   65015 toys/posix/ps.c

And yes ps is weird and waaaay over budget. It's actually 5 different commands glued together sharing infrastructure (ps, top, iotop, pkill, pgrep), I should figure out how to move the common infrastructure to lib/ps.c but it's really hard to draw the line between "generic" and "specific" here. (The second table at the start has default ps display lengths in it, is that an attribute of the /proc fields or is that an attribute of ps and top's default formatting? There's help text in the table. I made ps and top accept the same -o and -O field names, which are almost always the names it displays, and then the other three share the same plumbing and thus the same names. In theory this means you could pkill based on CPU utilization percentage, although that would require it to do the 2 pass top thing and I haven't hooked that up there yet, the utilization is calculated based on "ticks since last reading" so needs 2 readings a bit apart to work...)

Basically I have a "1000 lines upper limit for a C file" rule of thumb that I've made exceptions for toys/*/{sed,ps.sh}.c and for lib/{lib,xwrap}.c.

If minimalism here means "intentionally having less functionality", then yes dash is minimalist. My sh binary is currently 61k on x86, /bin/dash is 117k in devuan, so assuming I'm 2/3 through mine may indeed wind up being as big as dash, but I'm trying to include more functionality.

I haven't looked at dash's source (and don't intend to while writing a differently licensed alternative) but I went over to the "statistics" link on the left on the toybox website and searched for dash in black duck's archive and they say it's 13.5k lines of code. If you added ALL of toybox's lib/*.c (~5700 lines), that's a little over 9000 lines for toysh.

(Now maybe a lot of what black duck is seeing in dash is build plumbing. If you wc kconfig/* and kconfig/lxdialog/* that's 18k lines right there, which is a build time tool I've been meaning to replace which only updates a single .config file of keyword=value lines, which is currently 375 lines line.)


March 11, 2020

Actually Jury Dutied today. It was a civil suit, so what was at issue was whether one party should pay another party four figures. (We awarded a slightly lower four figures.)

I paid for a ride "share" both ways (not the "hour and a half with two transfers" bus, but the actual "fuzzy has an app and I made puppy eyes at her", and it's totally not a share if neither person would have gone from point A to point B without me paying them to do so, so that's a TERRIBLE name for it, but that's what Ride Austin calls it, so...) Because of that, I actually have energy left to go out and do some programming today! Woo! (Still sunburned from yesterday, but that was my fault.)

I redid the NOFORK launch plumbing so things like "eval" and "unset" can reach out and touch the shell state. My first stab at it was just to swap out TT.arg and sh_run("\"$@\""); but you can "eval x=5" and that won't work here. I need to factor out the plumbing to glue the command line together (because 'eval echo -n \" hello \" | wc' should say one 7 char word).

But at the moment, I'm shuffling stuff around to try to avoid function prototypes... and there's a loop: sh_run() calls run_function() which calls expand_redir() which calls expand_arg() which calls expand_arg_nobrace() which calls pipe_subshell() which calls run_subshell() which calls sh_run(). Fundamentally the problem is nesting "$($($($($($($blah))))))" -- it's kind of hard to do that non-recursively. And the current code doesn't. (Which means on nommu you can blow the stack doing that, and... I can put in a check I suppose?)

Sigh. For now, leave ONE function prototype.


March 10, 2020

Thought jury duty was today, so paid for a ride way down to south austin... and tuesday is _not_ the 11th. It's tomorrow. (Sigh, I made a note "fly back to Tokyo Tuesday the 11th" back when I left Japan, and although the flight back was delayed by a combination of jury duty and coronavirus, I didn't confirm that tusday IS the 11th.)

Rather than paying for a return trip, since I had the time back I went "I can walk, I don't get enough exercise..." and forgot that even when the temperature is nice (March! Spring!) in Texas sunburn is a thing. Totally fried when I got home, crashed in bed.

Woke up when my phone went off... with an automated reminder that I have jury duty tomorrow. Thanks. Lovely.

Got zero programming done today.


March 9, 2020

I finally got the IFS plumbing redone and checked in (and merged into the calling function's loop rather than as a seperate function), and now I'm grinding through "make test_sh" to try to get at least the converted part of the test backlog to pass, and I've hit the tests that want exec and eval. Both are shell commands (so NOFORK), but they run shell snippets in the current context (with access to all the local variables and such) so they need access to the shell's GLOBALS block. So I kind of need a nofork variant that doesn't re-parse arguments, which is tricky. (Or more likely I need a union at the start of the globals block with the various NOFORK commands' arguments.)

Hmmm... this means NOFORK doesn't blank the GLOBALS block (outside the argument union). There are only two NOFORK commands outside of sh.c itself (test and true, or at least the [ and : aliases for them since NOFORK also means "don't create show this in the command list symlinks are created from"), and neither has arguments. So that's probably ok?


March 8, 2020

In theory I fly back to Japan in a little over a week. (Plane ticket isn't booked yet but the flights are at like 30% occupancy so there should be something cheap.) I'm nervous not because of coronavirus, but because I have to get an apartment working for a company that's still funded month to month.

But at least Tokyo doesn't have bedside financial counseling in its hospitals. You only find that in the USA where the religious loons who founded the place (shakers, quakers, puritans) found money as their new god a few years later, and are happy to literally kill the country's own citizens en masse for profit. (The Boomers consider this normal. We look down on china having erased Tianamen Square from its history, meanwhile we're commiting slow genocide to increase the holdings of billionaires.)


March 7, 2020

I've mostly worked through my recent confusion about some weird bash word splitting corner cases, which turn out to be what happens why you apply the existing rules in a nested fashion. Now I have to redo my $IFS whitespace traversal logic, because I found yet another bug.

I have a function in toysh called split_add() which handles IFS word splitting and adding arguments to arg.v[] when parsing a command line. It takes way too many aguments (currently split_add(arg, delete, before, new, after, quote, offset)), and really should be inlined in the one function that ever calls it, but that function calls it from multiple places and it's not obvious how/where to work it into the big parsing loop.

(For example: "IFS=3;chicken(){ return 3;}; chicken;echo 3$?3" outputs "3 3", yes $? is an expansion that needs to go through IFS splitting. If it went at the start of the loop we'd have to continue and resume from variable state for things like "$@" that need to loop through the current enclosing argv context.)

The "arg" and "delete" arguments are the output: struct sh_arg has arg.c and arg.v[] members and there's an add_arg() function that automatically reallocs arg.v as we append stuff to it. The "delete" linked list is because arg.v[] mixes malloc() and non-malloc() contexts, as in some of these things are fresh allocations (due to variable expansion and such) that need to be freed when we're done with them, and some are the existing unmodified strings we got out of the input that should NOT be freed when this command exits. Since "not modified" is the common case, doing a strdup() on each one just to add it to argv[] is silly, so I add the allocations to a list we can traverse and free when we dispose of the arg.v. (Possibly the delete should move into struct sh_arg, but it hasn't yet.)

What split_add() does is take 3 spans of text: before, new and after, Ala "before${NEW}after". The "new" span is the one that just came from a variable or $(subshell) or some such and needs to be checked for $IFS word breaks. The "before" text has already been expanded as much as it's going to and gets glued the the first word. The "after" is what's left over, and gets glued to the end of the last word. "quote" says whether or not we're currently in a quoted context: we add 1 for every (unescaped/unquoted) double quote we see and 2 for every (unescaped/unquoted) single quote. This means if the bottom bit is set we're in double quote context but should still parse variables (but not split words), and if the bottom bit isn't set we _previously_ had quotes that have since closed (single quote context figures out how much to skip immediately, then loops back around without falling through to this logic), which we need to know because ""$X should add an empty argument even when X isn't set.

But the added "after" data still needs to be parsed (it may have more variables and such in it) so we need to know how MUCH of that last word is already parsed and how much is the copied over "after" data (because if $X resolves to $Y we don't parse it _again_, at least not here, and X=\"; echo "$X"; should not get confused by $X having a quote _in_ it. So we store the length of what's _already_ resolve in offset.

Hmmm, the current "after" string is only needed for allocation length. The parsing never looks at the output (it traverses the input exclusively) and the default behavior of the resolver loop (when it's not a special character) is to assign the next character of input to the next character of output. So we don't even need to copy the data here, it's just convenient to let xmprintf() do so because that allocates the right amount of output size. But all we NEED is enough data to reserve space to copy the remaining literal data into when there are no more escapes. So it doesn't actually need to be a string, just a length...

(I'm having flashbacks to the sed performance optimizations where we changed the "copy before/during/after to a new string each time" into in-place modification. I'm wondering if I should do that here, because it would be faster. But at the _moment_, it's premature optimization. Everybody goes "dash is faster than bash" but I haven't benched anything yet and this is small and simple enough code it should be easy to do this kind of optimization later.)


March 6, 2020

The last time I updated my old Three Waves writeup was in 2011, when I was trying to explain how Google's Alphabet was an attempt to convert itself to stage 3. I spent five days (1 2 3 4 5) giving the basic framework. (Although it trailed off before explaining that Alphabet was about Google fishing individual business units out of its melting pot and transplanting them into a third wave context, now cleanly separated with their roots untangled from each other, because the original mess was potbound and couldn't scale. I even did a talk for Flourish on it, which was recorded but never posted.)

I've referenced it a few times since, and I still want to shout out when it comes up that "yes, certification is important to bureaucracies because delegation of risk", but you need the theoretical framework to understand WHY that's the case.

(And you can't understand modern kernel development culture at ALL until you understand the hobbyist->employee->bureaucrat progression. And that the original disruptive technology insight was about explaining bureaucrat->hobbyist jumps where last year's trees are replaced by sprouting seeds. The old saying "it's the dead wood that holds up the tree" is true but physically _can't_ last last forever. Which is why our political situation is going to keep getting worse until the Boomers die, but once they do there's a lot fewer people left who think the existing system with billionaires _not_ being guillotined makes any sense at all, let alone "is a good idea".)

The unexplained whitespace split from yesterday is because the echo =$i= wasn't quoted. Insert Homer Simpson exclamation here.


March 5, 2020

And there it is. AOC endorsing not-Warren has given us Biden. (Warren understood her opponents and knew how to hurt them. Bernie hasn't got a clue how they work, and Biden wants to make friends with them.) No, I don't see a major difference between the remaining 78 year old white men whose names begin with B. Are either going to restore humanities education instead of diverting its remaining funding to STEM? No? I'm out until November.

When Glacier National Park was founded it had about 150 glaciers. By 2015 it only had 26 still big enough to qualify as glaciers (>0.1 square kilometers), because the park had lost 40% of its ice over the past 50 years. But hey, at least we're finding new islands around antarctica now that it's melting an inch a day.

I don't understand bash's behavior here:

$ IFS=x
$ chicken() { for i in "$@"; do echo =$i=; done;}
$ chicken one abc dxf ghi
=one=
=abc=
=d f=
=ghi=

It gets replaced with a space, but does NOT split? What is the intention here? Did busybox ash implement this? Let's see... busybox ash -c 'IFS=x; chicken() { for i in "$*$*"; do echo =$i=; done;}; chicken one abc dxf ghi' produced... =one abc d f ghione abc d f ghi= so yes, they do that too.

That's unpleasant. Right, $@ and $* work the same way when unquoted, and work SIMILARLY when quoted except $@ flushes word breaks and $* just appends a space to the existing argument. But both of them are doing the stupid space thing for IFS matches, and then doing a whitespace-is-magic split after replacing with spaces if it's not quoted.

Hmmm. The bash man page says quoted "$*" separates with the first character of IFS... And I'm doing utf8 for that, which is a variable length allocation _and_ variable length consumed out of the input string, but hopefully the existing plumbing I wrote can be made to do that.

I don't understand the argument flushing logic here:

$ func() { bash -c 'IFS=x; for i in $@; do echo =$i=; done' blah "$@"; }
$ func one "" two
=one=
==
=two=
$ func one ""
=one=
$ func "" two
==
=two=
$ func "" ""
==

Unquoted $@ is retaining the empty argument only if it's followed by another argument? (Even if THAT argument is empty?)


March 4, 2020

The Bernie Bros have joined Mitch McConnell in urging the "nevertheless she persisted" lady to stop. Bernie continues to perform his function of blocking a more qualified woman while himself losing. Sigh, I held my nose and voted for Hillary when she lost in 2016, I can hold my nose and vote for Biden when he loses in November. I did my part and voted for the candidate I _want_ in the primary, now it's tactical voting 'against' all the way I guess. (But I'm glad I sent her that $100 last week. I got to do something while it still mattered.)

Dear Bash: Seriously? SERIOUSLY?

$ for i
> in one two three
> do echo $i;
> done
one
two
three
$ for i; in one two three; do echo $i; done
bash: syntax error near unexpected token `in'

Be CONSISTENT! (This is like "echo {a..z..-3}" producing "a d g j m p s v y" isn't it? When the implementation is the standard, every weird bug it happens to do is part of the spec.)

Grrr. Anyway, the problem du jour is that toysh is trying to exec "in one two three" each time through the loop, and the problem I hit BEFORE that is the "for" parsing thing (which I'm not doing right either but would LIKE to get right because ; and newline are equivalent). Ok, how does the plumbing I implemented here work again? When you hit "for" or "while" it expects "do", but you can have different things before the "do", so it cheats by adding an extra stage identifier after the NULL terminator. A "while" or "until" starts with "do\0B" which means you can have arbitary command(s) before the do (as with "if"), but a "for" or "select" starts with "do\0A", which means it expects a variable name and either an "in", a "(())", or nothing before the "do", and once it's seen an "in" it transitions to "do\0C" which means "read rest of this line next line must start with do"... except the transition to C currently happens when it sees the "i", at which point you could get an "in" _or_ a "do" next (but not ((;;)) because that's instead of the variable name). And it can be with or without a newline: "sh -c 'for i do echo $i; next' one two three" should print "two\nthree\n" (because the one because $0, which turns out to be the same logic as #!/shell/script/arguments so it makes a certain amount of sense).

And a semicolon is logically equivalent to a newline, but for some reason bash allows multiple newlines but not multiple semicolons. And bash isn't allowing _one_ semicolon here, but IS allowing a newline. Sigh.


March 3, 2020

Fade broke down and got minnesota voting registration so she could vote in a primary. (In theory as a college student she can keep her Texas registration, but after the fiasco with never getting her an absentee ballot and then Frontier cancelling her flight last time, I'm not surprised. Seems Texas once again failed to distribute vote-by-mail ballots to non-republicans.)

I've reached the point where I no longer see a difference between Bernie and Biden. Neither of them has had a new thought in 40 years, both would turn 80 in office, and both are all talk without the ability to pass any legislation. Bernie's role continues to be to prevent a more qualified woman from winning, and Biden's brain's already melting from the strain of the campaign trail. (It's not "gaffes", it's senility.) Both are currently as old as Ronald "I don't remember" Ragan was when he LEFT office.)

I'll vote for a potted plant to keep the GOP out of office, and as far as I can tell a potted plant would be equally effective at legislating as either of them. The real threat to the status quo is warren, and the system has quite effectively mobilized against her to the point people are already abandoning their first choice with 95% of the votes still uncast.

I'm tired of voting against. Guillotine the billionaires already.

I bisected the musl-cross-make breakage to the stupid out-of-tree headers being introduced. Commit before that builds m68k and s390x, that one doesn't. (Remember, "git COMMIT^1" is the commit before that commit. In theory it's "left branch", in practice if it's not a merge it's "previous commit". For merge commits I just git show and cut and paste a commit number, but MOSTLY what I do is "git log --no-merges" and then look at the actual interesting commits, because if anything non-janitorial ever goes into a merge commit somebody screwed up. The toybox repo history doesn't HAVE merge commits, that's one of the nice things about "git am". It applies a patch to top of tree, with attached metadata crediting the author properly in their own words.)

Now I'm doing an all-target build to see what ELSE this might fix (such as the various broken arm targets), but that's kinda slow on this laptop.


March 2, 2020

I owe three closing parentheses from the blog entry on the 27th, so: )))

Tomorrow is "super tuesday", I.E. the official voting day for Texas. I early votinged the weekend before last, and the last day of early voting was the 29th, when the polling place at Fiesta (with like 10 machines) had a line wrapping around the store to the produce section. I'd thought of maybe doing some of the phone bank stuff for Elizabeth Warren since I'm in town, but I've _received_ multiple calls and texts from them already, so presumably they have more volunteers than people to call.

Why is "early voting" and "your official polling place" in different places? Look, it's not really early voting, it's _voting_. The polling place was open for 2 weeks. Last day was the 29th. Now it's LATE voting. It's make-up voting for people who missed the normal voting that was open all day every day (including weekends) for weeks. The marketing on this is just WEIRD.

For some reason the musl-cross-make build for m68k and s390x isn't installing the kernel headers, so anything that tries to #include goes boing. This is separate from the arm targets failing to build because it thinks configure is older than Config.in and tries to run autoconf (which I don't have installed and am not going to start now).

Trying to bisect where Rich introduced this breakage, but it keeps downloading 100 megabyte kernel header tarballs through my slow phone connection? Ah, it's because of a stupid commit where he started using an external kernel headers package again and oh goddess do we really have to repeat this mistake yet again? Seriously, I fought in this war (and had to repeat myself there). I fixed the upstream headers generation for PRECISELY THIS REASON. If you want to remove cruft from the userspace ones, push a patch or maintain a local sed invocation but DON'T PACKAGE UP YOUR OWN HEADERS DU JOUR THAT PREVENT YOU FROM UPGRADING. This is INSANITY. We have BEEN THERE, and LEARNED THIS LESSON, and...

This is why we have nazis again, isn't it? Those who cut history out of the curriculum because STEM STEM UBER ALLES NOTHING BUT STEM are doomed to repeat it.


March 1, 2020

Seeing a lot of speculation that Bloomberg paid Pete "means testing" Buttgeig to drop out, which makes no sense to me? But then his entire run makes no sense to me. I'm also hearing that Pete's voters' second choice is Warren, so I'm hoping she gets a boost. This does now mean that EVERY front-runner is at least 70 years old, so our next president is pretty much guaranteed to be a septuagenarian. I blame the Boomers for the entire slate of the white males beginning with B.

The df and realpath commands were using libc realpath(), which is the same thing as readlink -f I.E. xabspath(arg, 0), and it saves ~300 bytes on a static musl build to _not_ suck in the extra function. It also means I can (eventually) add realpath -e and -m and such, which nobody's asked for yet (presumably because readlink exists).


February 29, 2020

Elliott pointed me at an article that says nice things about 0BSD. The word is spreading.

A big GOP donor just bought twitter. Not missing my account quite so much today.


February 28, 2020

I noticed that several different articles on wikipedia[citation needed] are linking to an old blog entry here (even if they can't get the URL right, kinda needs the #anchor to be useful), and thus I committed the minor sin of fixing things in an old blog entry. (Just "stray end paragraph tag in the middle of a paragraph", "malformed URL tag" and such.) A lot of the URLs it points to have gone down, I should probably fish them out of archive.org or point to the mirrors I made on my history page over the years, but... not meddling with the historical record.

Still, if I was extending that article today I'd point out that the 6-bit vs 8-bit wars were about memory being so expensive that squeezing 6 text characters vs 8 text characters out of every 48 bits of storage (a 25% savings) was a big deal. And when the cost of the extra 2 bits stopped being a big deal, the power of 2 logic (that DIDN'T make you choose between supporting capitalization or punctuation, but not both) won out.

It's always interesting that I'm explicitly not "notable" enough to have a wikipedia page (I've been told this), but I'm mentioned on (and even cited by) over a dozen pages on that site. Oh well. There's a reason I call wikipedia[citation needed] by its full name so often. And I guess I'm used to being scrubbed from stuff. Note how the penguicon page carefully avoids mentioning me, except for linking to my old history writeabout about it in the references. I love how in the media section (dumping liquid nitrogen into the swimming pool), Howard is named but I'm "Attendees". (Not only am I anonymous, there's more than one of me!) There's a reason I haven't been back in a dozen years.

*shrug* The important thing is that the work gets done, not who does it.


February 27, 2020

I was trying to figure out if wchar_t can be unsigned (don't _think_ so?) so I grepped /usr/include and it's unsigned short in curses.h and unsigned long in X11/Xlib.h (which is the same size on 32 bit but NOT the same size on 64 bit), but neither of those are STANDARD header, they're expansion packages. So I checked musl, and it's a mess: it #defines _NEED_wchar_t in more than one place, which means it's pulling it out of a C compiler internal header. (Why?)

So devuan has gcc headers installed at the easy-to-remember /usr/lib/gcc/x86_64-linux-gnu/6/include (thanks EVER so much), and being gcc from the gnu/dammit project run by the Free Stallman Foundation it of course does "stddef.h:typedef __WCHAR_TYPE__ wchar_t;" BECAUSE REASONS, and that of course is "#define __WCHAR_TYPE__ int" at which point I want to punch the gcc developers in the face. (Must be Thursday.)

So that's three definitions of wchar_t, three different types, and the one we're using is the only one that's SIGNED.

I'm think I want to just rip all uses of wchar_t out of toybox (specs be damned) and just have it use "unsigned". I did a deep dive into this crap already, found out that EVERYTHING WAS BROKEN (musl and bionic are now fixed because I know the maintainers, gnu-libc can go hang), and the largest valid utf8 sequence is 0x10ffff (a million and change unicode code points) so short is too small but "unsigned" (defaults to int) is plenty.

Seriously, any type with _t on the end is windows brain damage and there's SOMETHING wrong with it, it may just be subtle enough you won't get bit by it until it's annoying to rip back out. LP64 is a good standard. Use that. (It would be nice if "long long" was defined as EXACTLY 64 bits rather than "at least 64 bits" though. Also, it would be nice if the unix.org pages on this still existed. Once upon a time refspecs.freestandards.org tried t collect and mirror stuff like this, but then they merged with OSDL to form the voltron of bureaucracy that is the Linux Foundation, and while they've still got an equivalent page I think it was last usefully updated 15 years ago? When the Linux Foundation "updated" the Linux Standard Base to 5.0 they broke backwards compatibility and distros refused to go along.)

If Google or somebody ever wanted to give me money so I didn't have to spend the majority of my time and energy on $DAYJOB I might try to put together my own mirror of useful standard material (the linux foundation one doesn't even have c99 or the real posix site (they use a stupid "sign up/pay us for access" one, which seems to have gone down), or the Linux man pages. And maybe I could even integrate it or update it into new specs. (It's kind of what I did for 6 months back in 2006 with the kernel docs page, but I hadn't even finished triaging the problem space when the Linux Foundation pulled the plug. (Yes, I applied to OSDL pre-merger, and wound up working the contract for the Linux Foundation post-merger. I got to see the sausage being made, and was not a fan.)

Sigh. When I put up my patreon page I was thinking maybe I could grow that into "I do open source full time", but... explaining the 8 gazillion things I want to DO is a full time job. Hmmm.


February 26, 2020

Look, we've been over this The reason Frodo couldn't take an Eagle to Mt. Doom is the Eye of Sauron could immediately spot the Ring whenever it had line of sight, and the Nazgul had flying mounts (I.E. "Black Wings"). They were fine when they had mountain ranges between them and it, and Gandalf could take the Eagles into Mordor to sing Hotel California as soon as the ring was destroyed, but _until_ then flying an eagle through Mordor meant Hello Nazgul. (One of the less popular Sanrio lines.) Plus any suspicion they meant to _destroy_ the ring rather than _use_ it would result in a troop of orcs being stationed on Mt. Doom. That's why Aragorn was marching around being a big distraction, and when Sam and Frodo made it in they went and marched on the gates of Mordor to pull aggro.

Seriously, J.R.R. Tolkein was in the trenches in World War I and wrote Lord of the Rings DURING World War II. His son Christopher was in the RAF, and J.R.R. himself both trained as a cryptographer and served as an air raid warden. Sam Gamgee's description of the moon had the same moon phase as the date Tolkein wrote the scene because tracking moon phase and stargazing was part of an air raid warden's job, especially after October 1940 when the Luftwaffe switched to mostly night attacks. Tolkein had sheets of moon phase and such he had to memorize. Yes Tolkein the war veteran, not-quite-spy, and air raid warden understood the importance of air power, and also the importance of radar.


February 25, 2020

Great, Android's browser changed its rendering plumbing again and now this blog only goes halfway across the screen on my phone. Bravo, guys. It's raw HTML text with no stylesheet or anything, and you're rendering it INSANELY.

I know it's not some strange tag I sneaked into a recent blog entry because it's doing it for 2018 and such, which USED to render right. Google's code changed, my html page did not. (And in fact Android 10 wasn't doing this when I upgraded to it, it's a chrome update since then that changed how rendering works. Bra, flipping vo. Golf claps everyone. Golf claps.)

The bash man page says, under command substitution, that "$(cat file) can be replaced by the equivalent but faster $(< file)", which is just... great. You can't "< file" by itself from the command line, so this is an implementation detail bubbling up to the surface, and I may have to special case it. (Dowanna special case it. Grrr.)


February 24, 2020

Huh, the 50 pound baggage weight restriction is actually an OSHA thing restricting how much one baggage handler should be expected to lift. I did not know that. (Hey, I'd tweet these if twitter hadn't gone funny. I'm told the next service I should look at is snapchat, which doesn't do this.)

Gee, ya THINK? (I did a writeup on this earlier this month.)

I need to add a "!" command to toysh as one of the builtins, but that can't use the normal NEWTOY() plumbing because !_main() can't be a function, and while I can feed any name to OLDTOY() as an alias for another command (which is how "true" is also ":"), the problem is there's no existing command to attach "!" to and I don't want to pollute the namespace with "not" or such that isn't an existing command. (There may be a "not" out there and if I don't search $PATH for it I broke somebody's script.)

You can ALMOST use OLDTOY() to call an arbitrary function because this is the implementation in main.c:

#define NEWTOY(name, opts, flags) {#name, name##_main, OPTSTR_##name, flags},
#define OLDTOY(name, oldname, flags) \
  {#name, oldname##_main, OPTSTR_##oldname, flags},

... and nothing enforces oldname##_main being an existing NEWTOY(). The problem is OPTSTR_##oldname has to exist, and that DOES come from the NEWTOY() list (via mkflags.c creating generated/flags.h). In this case it should be a macro defined to NULL. I suppose I could add it to toys.h or something?

Redoing the global plumbing for this seems overkill. I could special case check for "!" in run_command() but I'd rather not? I suppose I could teach "false" to be a "!" alias? Awkward place to put it but not _that_ awkward. (Of course normal false is defined as ignoring its arguments, so it has to check for the "!" name. And I should make it check for CFG_TOYSH so it can all compile out when building false standalone. And if I do that... there's no shared code, so there's no real reason to put it there. Sigh.)

Might as well nail it to one of the existing sh.c builtins like "source", which already has to have a "." alias. At least then it's local to sh.c.


February 23, 2020

I've managed to wrench myself onto a day schedule today... at the cost of being really, really irritable.

I have no idea why the Judoon sing the One Punch Man theme song.

A reminder that ICE is entirely about racism and serves literally no other purpose.

I came up with a third way to do the patch fuzz support which is of course retroactively obvious: use toybuf[] as an array of long to record the line numbers at which fuzz lines were hit. This avoids the constant malloc/free of the linked list approach (since fuzz happens and gets undone for every line that _doesn't_ match, traversing through the patch hunk for as many lines of fuzz as we allow), and avoids the hunk size limit of the bitmap approach (even though 32k is a lot, it's still a limit). This way the limit is the number of allowed fuzz lines (times 2, because context lines are symmetrical, although my fuzz approach just allows X lines to mismatch and doesn't care _where_, any line starting with space in the patch hunk is equivalent as far as my fuzz implementation is concerned. I don't think the other approach lets interstitial context lines in the middle of a hunk fuzz? (Dunno, haven't tried it, but the man page describes silly baked-in assumptions.)

So anyway, this new approach would work up to "patch -F 512" (on 64 bit systems and 1024 on 32 bit) which isn't really a limiting factor.

Hmmm, the change is a bit intrusive and fiddly, and I really want more test patches to run through it. (This was one of the advatages of aboriginal linux, it had buckets of cruft that tested lots of corner cases. Although mkroot is a much cleaner design, it doesn't apply _any_ patches to packages, so I don't wind up naturally dumping a bunch of real-world data on my tools here. Honestly, you only test _fuzz_ by getting it _wrong_. It's hard to get it wrong on purpose in the full variety of ways nature manages by default.)

It works with the patch Elliott sent to the list that sent me down this rathole, but that just has a single fuzz line right at the start. I should test multiple fuzz lines, before and after, including interstitial context. But eh, check it in first, doesn't seem to cause a regression at least.


February 22, 2020

Remember, the reason we don't have an NHS in the USA is the American Medical Association (basically a medieval-style Doctor's Guild) killed Harry Truman's proposal for one in 1946. Since then the AMA has worked dilligently to raise doctors salaries by creating an artificial doctor shortage (leading to the rise of "nurse practitioners" to fill a little of the gap), and today the AMA is still lobbying against medicare for all. We don't allow human organ sales because "your body has a black market value around half a million dollars if properly butchered" is not a world we want to live in (although I'm happy with libertarians living in it; turnabout is fair play). Allowing a profit motive to intrude into other parts of healthcare is also bad.

Blah, way too many todo items piling up. I'm trying to cycle back to finishing the current toysh checkin, haven't finished the patch fuzz stuff yet, haven't checked in the mkroot and toolchain build changes I was doing on the plane, and I got interrupted from all _that_ by a use after free in config2help.c that I can't reproduce but hits somebody (intermittently!) on macos. And someone else is having a problem with rm I can't reproduce. Jeff of course has a half-dozen things I should be working on for $DAYJOB but I'm flaking until monday.

Tried to take a brief nap round 3pm, woke up at 9. Sigh. For some reason, I can get back on a day schedule easily in Tokyo, but here when I get on a night schedule it's really sticky.


February 21, 2020

Still recovering, but I broke down and had caffeine today to try to force my schedule back to Austin from Tokyo. (It didn't work, but I keep trying.)

One of the toybox release blocker bugs I've got is that patch "fuzz" support is broken; it applies the patch hunk when it gets a match, including the original context lines, which were _not_ the context lines when matched with fuzz. (So it reverts the context to the version that was in the patch.)

In trying to fix this, I had to re-read my patch code, and... it's a bit more "clever" than I like. I was fresh off busybox when I did that, and the balance is too far towards small instead of simple. But the other problem is the main patch applying loop does the initialization code twice, and I'm looking to add something to the initialization, and it offends my aesthetic sensibilities to have to do the same thing in two places. (Single Point of Truth and all that.)

My first guess is to use toybuf as a bitfield, which means we can't use fuzz on a hunk longer than 32768 lines long, but I'm ok with that? The other obvious alternative is to keep a linked list of the fuzz mismatch lines so you can drop them in at the appropriate place in the new hunk; if I do a new dlist structure I can trivially add a field.

Which is related to my longstanding want to replace struct double_list, which right now has a single char * data member and means 90% of the uses of the dlist functions require typecasting the pointer to void * because other than patch everything else mostly uses a structure with different contents, that just has its first two members be *next and *prev pointers. (If *next is the first member, it's a valid singly linked list. If *next and *prev are the first two members, it's a valid doubly linked list. If consecutive members are the same type the compiler can't insert padding between them, so typecast to struct double_list and the first two members must be valid.)

The obvious new approach is to have struct dlist { char *next, *prev; }; and then have the first member of other structures be a "struct dlist dl;" so you can "dlist_add(&blah->dl, new);" with no typecast. (A pointer to a structure is equal to a pointer to the first member of the structure, there can't be padding before the first member in C99.) There's code that does "blah->prev->data" and that wouldn't work with a struct *dlist that's only the prev/next pointers, but there would still _be_ an encapsulating structure you can typecast to.

What I'd really like to get away from the problem where some functions take a dlist * and some take a dlist ** (the pop and push functions that may change the list head, and thus need to be able to write to the pointer you give them, thus needing pointer-to-pointer). If you're typecasting, it's easy to feed a single pointer to a pointer-to-pointer and vice versa, and the compiler won't catch it because typecasting. But then you get an instant segfault when you do that so it's easy to spot at runtime, which is how I've gotten away with it so far. (But it's ugly.)

Anyway, this is all a tangent from doing the actual fuzz fix, which is how you can tell I'm tired. That and my new "char * plus line number members" doubly linked list is "struct fuzzymandias", my names always get weird when I'm tired. (It's the same reason as the tabsplosion, I'm not cleaning up after myself to look "adult" as fast as I'm making the mess.)


February 20, 2020

I spent today mostly sleeping. Having no caffeine since Japan finally caught up with me. (Plus all the travel.)

Back in Japan I listened to a nice audio interview with Mark Blyth, which I added to the big playlist. If I was still doing anything like convention organizing, I'd try to do an event flying in him, David Graeber, Clay Shirky, Sarah Taber, and Tony Seba. Just letting them bounce off each other in different combinations for 2 days would pretty much be a full schedule if you ask me.


February 19, 2020

Huh. I've preferred female doctors for years but I didn't know there were actual studies saying you're less likely to die with a female doctor.


February 18, 2020

Onna plane.

Flight back to the states today, flight to Austin tomorrow. I get a day in minneapolis, in theory to see Fade and my sister and the niecephews, in practice probably sleeping a lot.

Yesterday we took a train past the airport to a different hotel (in yokohama, or possibly in 1967 judging by the hotel's decor), so we could have a morning meeting with chip fab people (without getting up at 6am to take the long train ride then), and today we took a train _back_ to the airport for the flight. But first we hung out in "Kawasaki" for a few hours, which was odd because that was also the name of a co-worker a couple years back.

Closing tabs: This is a really good thread. And this is perfectly normal, nothing to see, move along. Once again Boomers lament what a disaster it would be if young people did what Boomers have been consistently telling them to do. (Guys, Japan spent 20 years trying to dig out of this exact problem. Germany can only manage by exporting its problems to other countries.) Also, Justice Ginsberg is too old. (I strongly suspect Bloomberg's policy announcement to let old people die to control healthcare costs is laser-focused from Evil Billionaire Think Tank Du Jour to appeal to millenials and younger. He assumes senile Boomers are gonna vote nazi anyway, teaching an old Boomer new tricks is not a viable electoral strategy, you've gotta shout 'em down. He's still wrong (on this and so many other things), and even if he wasn't I still refuse to support a billionaire's decision to literally buy the presidency.)

I had to add a bug workaround to mkroot because distclean gets confused by cp -sfR and nobody on linux-kernel seems to care. Oh well. (Is that still where you report bugs? There was an lwn article about changes in kernel communication mechanisms but it was paywalled and I never went back to read it after the paywall expired. Speaking of paywalls and working around them, David Graeber's Against Economics was a good article back in December, and given the arguments it makes it being behind a paywall after the first month is officially ironic. Luckily archive.org has it.)

Meanwhile I sent Rich a bug report yesterday (his "cowpatch" script was being confused by ,1 being optional in patches), and today he emailed me that he'd fixed it. So there are still people responding to bug reports. they're just not on linux-kernel.

Finally implementing the "set CONFIG_EXPERT, then switch OFF other config symbols" plumbing in scripts/mkroot.sh. Taking out CONFIG_VT makes the /dev directory a LOT easier to understand (which was the initial motivation for doing this: the symbol is selectable but they WON'T LET ME because they know better than I do what I'm trying to accomplish. Grrr).

Next I'm going "you know, I've wanted to have the option to disable CONFIG_BLOCK for ages. The linux kernel is a giant bloated MESS, my "fairly close to allnoconfig" build creates a 2.7 megabyte _compressed_ bzImage from a 14.6 megabyte vmlinux file (which strips to 13.6), meaning running an x86-64 kernel in 4 megs of ram seems unlikely.


February 17, 2020

Not Buttigieg. Not Biden. Not Bloomberg. And by the time you get to Bernie... being asked to choose between FOUR WHITE DUDES BEGINNING WITH B means something is systemically WRONG.

Bernie would be 79 years old by election day, and would turn 80 his first year in office. He's never built a coalition in his life, and in his 30 years in congress only 7 bills he sponsored ever became law.

Let's look at this 30 years of legislative achievement, shall we? Three of his bills didn't do anything: two named post offices, and a third declared March 4, 1991 to be "Vermont Bicentennial day". That leaves four that MIGHT have done something.

The most useful-seeming of Bernie's bills slightly expanded a national forest in his home state of Vermont back in his first year in office by such a trivial amount the Wikipedia page on that park doesn't even mention the new area. Except his actual bill died but another bill did the same thing, so it's counted as "passed" in the above list.

Another of Bernie's bills granted the consent of congress to existing Vermont legislation negotiating water use quotas with New Hampshire. Not so much a bill as a rubber stamp for an existing bill, and once again Bernie's bill died and instead congress passed Vermont senator Patrick Leahy's bill doing the same thing. But "Bernie proposed a bill" and "congress eventually did that thing" is all it takes to get a win here, and it was a thing that probably should not have NOT happened, so yay?

The other two bills Bernie got passed were about the military, except they weren't really his bills. It's nice that in 2013 Bernie cosponsored veterans legislation with Mitch McConnell (amending existing legislation in a way the bill didn't even bother to summarize) but that wasn't really _his_ bill, it just let Mitch put out a press release about bipartisanship that didn't even mention Sanders by name. And in 2013 Bernie introduced a veterans disability cost of living adjustment, which again was an excuse for multiple republicans to put out press releases about bipartisanship without mentioning Sanders by name.

The timing of Bernie's 7 bills is also interesting: he passed two in his first session of congress (the 102nd), a total of two more over the next 7 sessions (his "me too" to Patrick Lehy's water use bill was in the 104th and he named a post office in the 109th, but got nothing in the 103rd, 105th, 106th, 107th, 108th, 110th, and 111th). And then Bernie got THREE in the 113th, right before he spent most of 2015 and 2016 running for president. This included both of his "this is actually a republican military bill he's letting them call bipartisan", plus one of his bills naming a post office.

It's a bit hard to for me get excited about this legislative record.

Meanwhile, Elizabeth Warren chaired the congressional oversight panel in the 2008 bailout back as a Harvard Law Professor, and was so outraged she got the consumer protection credit bureau created the next year, and only became a senator when republicans wouldn't let her run the new bureau. And SINCE then she's gotten nine bills she sponsored enacted. That means Bernie got 7 in 29 years, and Warren got 9 in 7 years. Yes half of the bills Warren's gotten passed are also fluff, but half of hers _aren't_, and keep in mind half her tenure in the Senate has been under the current kleptocratic administration (Nevertheless, She Persisted).

Bernie Sanders first ran for Senate in 1972. He is a career politician who has never been remotely effective at the job he's held for 30 years, and NPR's piece titled "Bernie Sanders has stuck to the same message for 40 years" came out 5 years ago.

Warren is 8 years younger than Bernie, left a tenured position as a Harvard Law Professor within the past decade to become a Senator because there were specific things she wanted to accomplish, and she is VERY good at doing her job.

Bernie, Buttgeig, and Bloomberg would be ACTIVELY HARMFUL. (Maybe not as much as the current clown, but that's damning with faint praise.) Bernie has been all talk and no action for longer than I've been alive, and his supporters main argument for him is that he "inspires people". I.E. we need a left-leaning charismatic white male septuagenarian blowhard to counteract the OTHER side's charismatic white male septauagenarian blowhard. Bernie's gotten nothing done in the past half century, but in a way that's admirable because... he's an old white male who graduated from a private college in 1964?

Warren gets things done. She has a plan for that, and nevertheless she persisted. I don't want a figurehead, I want Elizabeth Warren to become president of the united states. For once this isn't a vote against, but a vote _for_. It's not a tactical choice, it's who I want to see run the country.


February 16, 2020

Huh, I knew London's economy was primarily based on money laundering but I didn't know that the majority of its import/export trade was ALSO just money laundering in disguise.

In the US the financial industry caused the 2008 financial crisis and they're still at it but the UK doesn't seem to have much else left in its economy? Brexit was apparently a scheme by billionaires to avoid the EU's crackdown on corporate tax avoidance, which puts the UK in a bad spot now they've thrown away their veto power of the EU's blacklisting of their tax havens. (All the people saying we can't afford basic income should watch the video in that first link about how lots of people and giant machines dig up thousands of tons of ore to refine out traces of metal, transport it halfway across the world through an elaborate trading network, just to bury it underground again elsewhere. The gold serves no actual purpose, nobody eats it or builds with it or even uses it for trade -- money lives in computers these days. It just gives the people _permission_ to literally move mountains. IT IS A SOCIAL CONSTRUCT. Our modern religion.)

Today I got an email from someone who stumbled across an old blog post and asked:

> My main question is if it is still not possible to set a different limit
> than 50% of total RAM for rootfs in initramfs after you introduced tmpfs
> as rootfs.

To which I replied:

Dunno. I've stopped trying to engage with the linux kernel mailing list because reasons. I tried again a couple months ago, but it didn't end well.

At runtime you can do something like "mount -o remount,size=60%" but that doesn't help if your initramfs.cpio.gz goes over quota.

Adding a way to address that has been on my todo list forever, but... lkml. I did recently note that the "init has no stdin/stdout when initramfs.cpio.gz didn't have a /dev/console" problem seems to have been fixed in newer kernels? That's nice. Only took 3 years after I reported it.

Speaking of which, I'm trying to force myself to submit talk proposals to ELC, and I just... don't want to? It's in my hometown, I don't have to pay to travel there or for lodging (modulo the whole "will I still be there or have to fly back from Tokyo" thing), but... sigh. I should submit a redo of my 2017 "simplest possible linux system" talk. The description I wrote up for that talk said:

Building the simplest possible Linux system

This tutorial walks you through building and booting the simplest possible Linux system, first under QEMU and then on real hardware. We cover kernel configuration and building, native vs cross compiling, initramfs creation (and other root filesystem options), installing and booting, the init process and system bringup, running an app, adding an example server (sshd), and finally we'll add a native toolchain to compile "hello world" on the target.

And... it didn't, really. As mentioned before, I was SOOOOO jetlagged I didn't manage to edit my notes into a coherent order, and got through maybe half my material in a disappointingly disorganized fashion. (Behold my incoherence.)

I suppose I could just resubmit that description, but A) I'm not sure how to say "give me another shot at this" without badmouthing my own talk, B) it's not what I've been doing recently? What have I been doing recently. Let's see...

Building and booting Linux in a 250 line shell script.

Here is a 250 line shell script that builds a working Linux system from source: https://github.com/landley/toybox/blob/master/scripts/mkroot.sh

The script uses only two source packages: the linux kernel itself and toybox (android's command line utilities, which android uses to compile itself). The script creates a one line wrapper to boot the new system to a shell prompt under QEMU on each of a dozen supported architectures.

For comparison, Linux From Scratch 9.0 is a 368 page PDF that requires downloading 86 packages to build the base system according to the instructions. This talk explains the difference between the two, and the work underway to be able to download and build Linux From Scratch at the mkroot shell prompt.

Eh, it's _true_ but the LFS snark would be much more effective if I was already at the "and it builds LFS under itself" point, which needs an awk implementation and promoting several things out of pending. Plus lex and yacc now thanks to the kernel guys deciding kconfig needed to be turing complete and have the ability to "rm -rf ~" because why not? I think the new shell's 2/3 of the way done, but can't rule out Tom Cargill's ninety-ninety rule.

Anyway, I submitted that talk proposal. When I actually want to go to a conference I generally submit 4 talk proposals and let them pick the one they like, but in this case I could barely manage the enthusiasm (sarcasm?) for one. I'm up for doing the work, yes. But jumping through Linux Foundation hoops to convey old news is a janitorial task, and when I have to fill out forms to convince a committee to give me permission to do so? Wheee.

I should just do youtube videos, but I seem to have a preemptive Cheese Sandwich Problem. In _theory_ the videos I need to produce are something other people could eventually someday care about, but right now they don't, because I haven't done them yet.


February 15, 2020

I have said for YEARS that you don't get waterproof by plugging the holes in a collander. Insert obligatory Antoine de Saint-Exupery quote here.

Meanwhile, the reason for the Clue quote in today's checkin (other than Mrs White being apropos for "My Little IFS: Whitespace is Magic") is that it's a huge pain to figure out WHY it's doing that in that test, and it's basically internal implementation details bubbling up to the surface (and REQUIRING other implementations to work that way).

IFS is the shell Internal Field Separator, which I've never used but has been there forever and is in posix so I have to support it. It's the list of characters (yes I'm supporting UTF8) that separate words for certain kinds of shell parsing, and is completely ignored for others. It kicks in after variable resolution, in backtick output, and a bunch of other places. (But of course NOT for breaking command line options into words when running commands.)

IFS works like this:

$ IFS=x; A=xabcxx; for i in $A; do echo =$i=; done
==
=abc=
==

Yes $A added zero length arguments without quotes. Welcome to IFS.

When IFS isn't set, it's equivalent to " \t\n", and whitespace characters are MAGIC. I don't mean "the shell is resolving the \t and \n here even though it doesn't anywhere else" (although that's true), I mean whitespace IFS characters accept RUNS of characters as word separators, and it's ANY whitespace character in the list. (So if you say IFS="\t\n" it'll accept \t\n\n\t\n\t as a single run of whitespace, but the first actual space character in there will stop it and be treated as a literal. Yes I'm teaching it to accept the non-blank UTF8 whitespace from Nanny Oggham although I'm not adding it to the default 3 just yet).)

And this "runs" thing does _not_ add blank entries, although other parts of the variable resolution logic can:

$ unset IFS; A="   abc   def   "; for i in ""$A""; do echo =$i=; done
==
=abc=
=def=
==

But note that """"$A"" doesn't have _two_ empty arguments at the front, it's the transition from "quoted context" to "unquoted context" that flushes a zero length entry. And in fact:

$ A="   abc   def   "; for i in ""x""$A""; do echo =$i=; done
=x=
=abc=
=def=
==

The "we previously saw quoted material" status is flushed by the separator splitting/commiting the argument, the whitespace run elimination won't proceed into previous material before the variable being expanded, and the previous material (despite being zero length) has a "seen quote" status attached, so is committed as an argument.

Except sometimes, there's a "do not split the argument" status, in which case they're glued together with a space. The most obvious one is when the argument is in double quotes, although there are others:

$ IFS=x; X="onextwoxxthree"; y=$X; echo $Y
one two three

So here's where it gets weird:

$ cc() { echo =$*=; for i in $*; do echo -$i-; done;}; cc "" ""
= =

Here's the test that explains what the heck was happening there:

$ cc() { echo ="$*"=; for i in =$*=; do echo -$i-; done;}; cc "" ""
= =
-=-
-=-
$ cc ""
==
-==-

The =$*= wasn't expanding to "= =", it was expanding to "=" "=" as two arguments. "$*" by itself is expanding to a single space because the quoted empty argument is flushed, a space is added after it, and the next quoted empty argument is flushed.

The annoying/inconsistent part is no output without the flush:

$ cc() { for i in $*; do echo -$i-; done;}; cc "" "" "" "" ""
$ cc() { echo =$1$2=;}; cc "" ""

So it really _looks_ like the next test after my "Flames" comment should produce == instead of = =, but it's introducing a _split_ not a space, then the = gets flushed because it's _not_ empty, where an unquoted empty string would be discarded.

I'm making it work, I'm just not happy about it.


February 14, 2020

I have a japanese residence card, good for 5 years. Picked it up today. I no longer need to carry my passport around with me, the card covers it. (I need to get and register a residence address to be fully in the system, though. I.E. a tokyo apartment.)

ELC's call for papers deadline is today. Their submission form now has an "is this a technical talk yes/no" question. That's a succinct summary of what the Linux Foundation has done to ELC right there. (Alas, I expected this back when the Linux Foundation took over.) Oh well, I should apply anyway...

There's a fairly good battery tech explainer buried in this video, but of course it's wrapped up in the energy/transportation version of "Microsoft is buying up all this technology, let's explain it as if they're inventing it rather than scouring the world of competition and using their existing financial resources to take and take and take". Anyway, that video links to a previous video about another innovative company/technology that just got slurped up by the Big Money. (I kinda doubt the average tenure of their employees being 14 years is likely to continue under the new regime.) I do not consider the ability to throw money at a problem to be laudable, especially when "because you're a billionaire we'll extend you unlimited credit if you overextend yourself" kicks in. The rich can't run OUT of money.

Meanwhile, antarctica is currently room temperature. (You know all that polar ice? Yeah, about that...) And as far as I can tell australia's fires only subsided because there's nothing left to burn.

The coresemi.io domain seems to have expired at midnight eastern (so around 2pm here) and Jeff immediately renewed it, and they charged his card ($75 which is an insane amount for a year of a domain name), but then the domain didn't renew, and when 8am eastern rolled around and people got into the .io office (Indian Ocean, so of course they're on the east coast of the USA) they demanded they demanded an additional $350. Jeff explained that the .io domain is what the bitcoin guys insist upon (you're not a _real_ blockchain without an .io domain), and thus the whole thing has turned into a scam to bilk people. Great. (We can move to another domain, but not with our contact email down. Sort of a catch-22 there.) Personally, I think this is why challenging credit card transactions as fraudulent was invented.


February 13, 2020

The cat. Her name was George.

Sigh. It's not _surprising_ George died. She was a 17 years old and had health problems for many years (special diet to avoid kidney crystals, daily heart pills, twice daily thyroid pills, a special cream to rub on the persistent rash covering half her belly, and they were monitoring her liver function). She was perpetually overweight before the thyroid thing and always profoundly sedentary. (This is the cat who would open a cabinet, crawl in, close the door behind her, and voluntarily sit there in the dark for hours.) She's been eating grass and barfing at every opportunity for a year.

But still. Cat. Fuzzy is kinda shook up about it, George has hung out on her bed more or less 24/7 for years now. See "sedentary", above. Fade did a very nice eulogy. I still haven't got my twitter account back.


February 12, 2020

It's good to know "Ok, Boomer" is still torquing off the Boomers. (Speaking of which, I'm told Bubblegum Crisis is a classic anime about boomers going insane and being fought with heavy artillery, and I should definitely watch it sometime.)

Wrestling with $IFS. It's kind of terrible. (Whitespace is magic.) I'm going through the bash man page trying to find a case where parameter expansion and quote removal don't go together, and I found one: "case word in pattern)" doesn't say quote removal is done for pattern. But I tried case x in "A") and the quotes were removed.

I still hate $IFS, but I now have a much more informed hatred of $IFS. (It's the shell Internal Field Separator. Yes, you can split words based on something other than runs of whitespace. No, "IFS=abc command" doesn't work to change it just for a single command. As in it changes the variable for the child process AFTER the command's arguments get parsed using the old value of $IFS. I'm very tempted to change that for toysh, but that would probably break some existing script somewhere. Also, I have a todo item for adding array support to let IFS be an array variable in which case it's not JUST characters but strings it breaks on. Doing that would fix up the magic whitespace thing where runs of whitespace get removed _and_ leading matched whitespace in the separator takes out empty previous arguments so they don't get added to argc[], but non-whitespace separators only match a single character _and_ it adds zero-length arguments to argc[] (like "" does) when it's the first thing or two of them happen in a row.

See "still hate $IFS", above.


February 11, 2020

I was supposed to fly back today, but my visa isn't ready yet (current estimate is Friday), especially since today is "National Foundation Day" (Japan's version of July 4, except their 1776 is 660BC) so the person processing the visa wasn't in today anyway. (And she doesn't work Mondays.)

We pushed the hotel back a week, and got another week of hotel. On the bright side, this flight we managed to arrange a day layover in Minneapolis so I can see Fade and my sister and the niecephews.

Did you know that buying a seat on the Linux Foundation Board of Directors costs $500k per year? They don't phrase it like that, of course, they say that buys you "platinum membership", and then later say all Platinum members get a board seat. But that's literally what companies do when they buy those. The current board is literally described by company.


February 10, 2020

I proposed introducing two members of the j-core community who've been mailing me off list to each other, and Jeff's immediate response was "No! No no no, don't do that..." without further explanation. This neatly summarizes J-core community management to date.

I have a local patch to rip fork() out of musl for nommu builds (something Rich will never accept), and this breaks the fixincludes build in gcc 8.3 because it hasn't been regression tested in forever. It's easy enough to fix (change one fork() to vfork() and add --enable-twoprocess to GCC_CONFIG_FOR_TARGET right after --enable-fdpic. (Honestly, --enable-fdpic should do it itself, but that would be getting a patch upstream into gcc which would involve signing copyright assignment paperwork which ain't happening.) I submitted the --enable-fdpic to Rich so I'm not patching musl-cross-make in mcm-buildall.sh, and he rejected it because fixincludes builds for him and as he complained on his blog fixincludes is terrible and shouldn't exist. (I agree it's terrible, but if you rm -rf fixincludes in the gcc source before running ./configure (which works for the same reason extracting binutils and a dozen other packages into the gcc directory and building them all as one package works; there's a zillion references to sub-packages in the gcc makefile but they're all guarded with "maybe" plumbing that skips them if they don't exist), then the headers don't get copied at _all_ and the resulting toolchain has no gcc headers and doesn't work. Sigh. Possibly the fix is rm -rf fixincludes but then having a small script cp -a some directory?)

I stripped down a shell function that was testing command line argument parsing to turn it into an sh -c invocation that would test the same argument parsing in a different context, and I left a trailing ;} in the -c string (left over from the function), and I accidentally ran the Defective Annoying Shell instead of bash, and _that's_ why it didn't notice the error:

$ sh -c 'for i in a"$@"b;do echo =$i=;done;}' 123 456 789
=a456=
=789b=

I was staring at that when I noticed, going "there's no way bash is getting that wrong", and indeed bash spots it just fine:

$ bash -c 'for i in a"$@"b;do echo =$i=;done;}' 123 456 789
123: -c: line 0: syntax error near unexpected token `}'
123: -c: line 0: `for i in a"$@"b;do echo =$i=;done;}'

Dash continues to be crap, but we knew this.

(Yes, for some reason the first non-option argument to sh with -c becomes $0. Posix!)


February 9, 2020

Ok, following up, here's an analysis of what toybox does and doesn't replace out of Linux From Scratch, using LFS 9.0 chapter 6.

(Before I start, may I just say that the existence of libasprintf is a damning indictment of everything C++ has ever tried to do or be in its entire history. Most people aren't learning C anymore because the C++ people scream up and down if you do, but the ONLY thing wrong with C is that C++ exists, and C++ is very bad language a lot of people have stockholm syndrome about.)

Here are the packages toybox plans to provide complete-ish replacements for, with pending commands in square brackets, and notes about what we _don't_ implement. (Each "see also" note means this package also installs shared libraries. Toybox probably _can_ provide equivalents for a lot of these libraries, but why?)

  • file: file (see also: libmagic)
  • m4: [m4]
  • bc: [bc] [dc]
  • bison: [yacc] (not: bison, see also: liby)
  • flex: [lex] (not: flex flex++, see also: libfl)
  • make: [make]
  • sed: sed
  • grep: grep egrep fgrep
  • bash: bash sh (not: bashbug)
  • diffutils: cmp [diff] [diff3] [sdiff]
  • gawk: [awk] (not: gawk gawk-5.0.1)
  • findutils: find xargs (not: locate updatedb)
  • less: less (not: lessecho lesskey)
  • gzip: zcat [gzip] [gunzip] [zcmp] [zdiff] [zegrep] [zfgrep] [zgrep] [zless] [zmore] (not: gzexe uncompress zforce znew)
  • make: [make]
  • patch: patch
  • tar: tar
  • procps-ng: free pgrep pidof pkill ps sysctl top uptime vmstat w watch [pmap] [pwdx] [slabtop] (not: tload, see also libprocps)
  • sysklogd: [klogd] [syslogd]
  • sysvinit: [init] halt poweroff reboot killall5 [shutdown] (not telinit runlevel fstab-decode bootlogd)
  • man: man (but not accessdb apropos catman lexgrog mandb manpath whatis, see also libman libmandb)
  • vim: vi xxd (but not ex, rview, rvim, view, vim, vimdiff, vimtutor)
  • sysvinit: [init] halt poweroff reboot killall5 [shutdown] (not telinit runlevel fstab-decode bootlogd)
  • kmod: insmod lsmod rmmod modinfo [modprobe] (not: depmod kmod)
  • attr: [getfattr] setfattr (not: attr, see also: libattr)
  • shadow: [chfn] [chpasswd] [chsh] [groupadd] [groupdel] [groupmod] [newusers] passwd [su] [useradd] [userdel] [usermod] [lastlog] [login] [newgidmap] [newuidmap] (not: chage expiry faillog groupmems grpck logoutd newgrp nologin pwck sg vigr vipw, grpconv grpunconv pwconv pwunconv, chgpasswd gpasswd)
  • psmisc: killall [fuser] [pstree] [peekfd] [prtstat] (not: pslog pstree.x11)
  • inetutils: dnsdomainname [ftp] hostname ifconfig ping ping6 [telnet] [tftp] [traceroute] (not: talk)
  • coreutils: [ base64 basename true cat chgrp chmod chown chroot comm cp cut date dd df dirname du echo env expand factor false fmt fold groups head id install link ln logname ls md5sum mkdir mkfifo mknod mktemp mv nice nl nohup nproc od paste printenv printf pwd readlink realpath rm rmdir seq sha1sum shred sleep sort split stat sync tac tail tee test timeout touch true truncate tty uname uniq unlink wc who whoami yes [expr] [fold] [join] [numfmt] [runcon] [sha224sum] [sha256sum] [sha384sum] [sha512sum] [stty] [b2sum] [tr] [unexpand] (not: b2sum base32 basenc chcon cksum csplit dir dircolors hostid pathchk pinky pr ptx shuf stdbuf sum tsort users vdir, see also libstdbuf)
  • util-linux: blkid blockdev cal chrt dmesg eject fallocate flock hwclock ionice kill logger losetup mcookie mkswap more mount mountpoint nsenter pivot_root prlimit rename renice rev setsid swapoff swapon switch_root taskset umount unshare uuidgen [addpart] [fdisk] [findfs] [findmnt] [fsck] [fsfreeze] [fstrim] [getopt] [hexdump] [linux32] [linux64] [lsblk] [lscpu] [lsns] [setarch] (not: agetty blkdiscard blkzone cfdisk chcpu chmem choom col colcrt colrm column ctrlaltdel delpart fdformat fincore fsck.cramfs fsck.minix ipcmk ipcrm ipcs isosize last lastb ldattach look lsipc lslocks lslogins lsmem mesg mkfs mkfs.bfs mkfs.cramfs mkfs.minix namei partx raw readprofile resizepart rfkill rtcwake script scriptreplay setterm sfdisk sulogin swaplabel ul uname26 utmpdump uuidd uuidparse wall wdctl whereis wipefs i386 x86_64 zramctl)

Commentary: toybox init doesn't do runlevels, man and vim are just the relevant commands without the piles of strange overgrowth, and if you want to call a toybox binary by another name you can create a symlink to a symlink. If somebody really wants to argue for "gzexe" or similar, be my guest, but there's a lot of obsolete crap in shadow, coreutils, util-linux...

I have no idea why LFS is installing inetutils instead of net-tools (which contains arp route ifconfig mii-tool nameif netstat and rarp that toybox does or might implement, and plipconfig slattach that it probably won't.)

Packages toybox plans to provide partial replacements for, as in some of our binaries provide replacements but there are other useful binaries this package provides that toybox does not (generally out of scope for the project):

  • binutils: strings [ar] [nm] [readelf] [size] [objcopy] [strip] (not c++filt, dwp, elfedit, gprof. The following commands belong in qcc: addr2line as ld objdump ranlib)
  • bzip2: bunzip2 bzcat [bzcmp] [bzdiff] [bzegrep] [bzfgrep] [bzgrep] [bzless] [bzmore] (not: bzip2, bzip2recover, see also libbz2)
  • xz: [xzcat] [lzcat] [lzcmp] [lzdiff] [lzegrep] [lzfgrep] [lzgrep] [lzless] [lzmadec, lzmainfo] [lzmore] [unlzma] [unxz] [xzcat] [xzcmp] [xzdec] [xzdiff] [xzegrep] [xzfgrep] [xzgrep] [xzless] [xzmore] (not: compression side, see also: liblzma)
  • ncurses: clear reset (not: everything else, see also: libcurses)
  • e2fsprogs: chattr lsattr [e2fsck] [mkfs.ext2] [mkfs.ext3] [fsck.ext2] [fsck.ext3] [e2label] [resize2fs] [tune2fs] (not badblocks compile_et debugfs dumpe2fse2freefrag e2image e2mmpstatus e2scrub e2scrub_all e2undo e4crypt e4defrag filefrag fsck.ext4 logsave mk_cmds mkfs.ext4 mklost+found)

Toybox should decompress multiple formats, but the only compressor we're providing is deflate (gzip/zlib). The problem with e2fsprogs is I haven't got a good handle on ext4 yet. The "qcc" reference is because someday I'd like to glue QEMU's Tiny Code Generator to Fabrice Bellard's old Tiny C Compiler and make a multicall binary that does cc/ld/as (then use the LLVM C Backend to compile LLVM itself to C for use as a modern replacement for cfront). Until then things like objdump -d (requiring target-specific disassembly for an unbounded number of architectures) are out of scope for toybox. (This means drawing the line somewhere between architecture-specific support in file and strace, and including a full assembler for each architecture.)

The list of LFS chapter 6 packages toybox does NOT replace is: linux-api-headers man-pages glibc zlib readline gmp mpfr mpc gcc pkg-config ncurses acl libcap psmisc iana-etc libtool gdbm gperf expat perl XML::Parser intltool autoconf automake gettext libelf libffi openssl python ninja meson check groff grub libpipeline texinfo.

That said, we implement our own zlib and readline replacements, and presumably _could_ export them as library bindings. Plus we provide our own version of a bunch of section 1 man pages. Possibly libcap and acl are interesting?

The kbd package has over a dozen commands, we only implement chvt. The iproute2 package implements over a dozen commands, there's an "ip" in pending but I'm not a fan (ifconfig and route and such should be extended to work properly). We don't implement eudev, but I wrote busybox mdev way back when (which replaces it) and plan to do a new one for toybox as soon as I work out what subset is needed given devtmpfs.


February 8, 2020

Oh good grief, you know a thing is coming and then when it does people invent endless justifications why the real reason we all knew years ago can't possibly be why it's happening now.

We knew years ago exactly why hospitals in republican-controlled areas were becoming insolvent and closing. It's because GOP legislators sabotaged them, preferring to see their constitutents die en masse (which ironically serves their political goals) than Obamacare succeed.

The backstory is that in the 1980's hospitals could refuse to treat people who couldn't pay, who would then literally die on the sidewalk outside the emergency room, and it turned into a big scandal. In response congress passed a law (which even Reagan was forced to sign because the PR disaster was _that_bad_), requiring emergency rooms to at least _stabilize_ somebody before discharging them, and to pay for it the federal government gave hundreds of millions of dollars of subsidies to hospitals. (GOP loons in the pay of billionaires, who are trying to destroy the government so we collapse back into feudalism and they can be kings, insist this is an "unfunded mandate" because the subsidies are not directly tied to the mandate. I.E. they don't waste money tracking each patient given care and working out how much that should cost, but just periodically cut them a large check.)

25 years later Obamacare redirected those subsidies to medicaid expansion, which the hospitals didn't mind because they still got the money through a different channel... except GOP legislators went to the supreme court to block the medicaid expansions in their states. The problem is, the subsidies didn't come back when they did this. If they refused to receive the same money through the new channel, their state's hospitals just didn't get that money at all anymore. Which turned it _into_ an unfunded mandate in those states. This left hospitals in a bind: they're still required to treat anyone who shows up at the ER (which is now a LOT more people if nobody has insurance, so they only go at the last minute when the alternative is dying), but the federal payments they were getting for this went away in GOP-controlled areas.

Hospitals have compensated by randomly gouging everyone who _does_ pay them, but it's a drowning victim climbing on top of nearby swimmers and dragging them down. (And this is aside from the way the American Medical Association decided in 1970 to raise doctors' salaries by creating an artificial doctor shortage and they've kept up the pressure ever since, constantly warning of a glut of physicians.)

So a video about why hospitals are closing that ignores the reason we've known about for most of a decade now is kinda stupid. (This kind of blind spot is not new but still annoying. Employers in the USA offered health insurance during World War II to work around the wartime salary caps when trying to hire "Rosie the Riveter" away from their competitors while all the able-bodied men were deployed overseas, that's where the USA got its tradition of employer-provided health insurance other countries don't have. How can that video talk about the rise of health insurance at the 3 minute mark without mentioning that?!? Yes years later Reagan allowed HMOs to buy up all their competitors and corner the market, in healthcare as with everything else, but the rest of the world had National Health Services decades before that. In civilized countries the ambulance is like the fire truck, it doesn't refuse to come if you don't have the right employer-provided insurance, nor does it provide you with a giant bill afterwards. The libertarian loons insisting that everyone's lungs should be coin-operated because allowing air to be free would be morally wrong are literally profiting off the pain of others. After all, didn't Jesus charge the lepers 30 pieces of silver each?)

Me, I'm still waiting for the Boomers to die. Our entire trajectory towards becoming a nazi autocracy is 100% the Boomers' fault, and the only thing likely to fix it their collective death. The collapse of the healthcare system can only bring the LD50 of the Boomers closer. (Half of all Boomers in the US were born before 1955, the actuarial tables put the average lifespan at 79, meaning half of them should be dead by the end of 2034. Until then, they'll get increasingly deranged vanishing into racist-grandpa off-my-lawn kids-these days caricature. And use anyone punching _back_ as retroactive justification why they were so horrible in the first place.)


February 7, 2020

It's ironic that automating away all the jobs is even putting stock traders out of business. The billionaires financializing everything create zero real jobs, it's BS jobs or nothing.

It occurred to me recently that bitcoin is basically horse racing. Some people created a betting channel and got people excited about it, and it's a zero-sum game but they skim off a tiny amount of the money going through it. Ten years ago it was online poker, but when that got outlawed it was replaced by day-trading cryptocurrency.

The ballooning of billionaire assets started with Reagan cutting the top tax rate from 70% at the start of his presidency to 28% by the end of it. If you get a graph of the assets of the 1% and the national debt, the two graphs literally mirror each other, it's the same money. But they quickly ran out of places to PUT all that money (without losing value each year to inflation); there's no investment that can absorb the sheer dollar volume the US government is printing. (Yes they're printing it, when the Treasury Department sells bonds to the Federal Reserve, the fed is buying them with newly printed money, that's how the federal government has funded its operations since Reagan stopped taxing billionaires: by printing money.

So several things happened. First civilians (now rebranded "consumers") were all issued credit cards at double digit interest rates, so billionaires could act as loan sharks to the entire US population. Second banks lobbied to lower capital requirements meaning they could loan out more money than they actually had (adding money to your account is just editing a number in a computer; if you then transfer that money to another account within the same back, no money ever leaves the bank. And if it does, the bank can borrow unlimited amounts of money basically for free from the federal reserve which again prints the stuff so can never run out. (In this case creates it electronically, there's a lot more "money" in the world than cash in circulation). Which meant money became a completely fictitious quantity that any bank could create on a whim, and if there was a run on the bank ala Mary Poppins the federal reserve's official money printing capacity would bail it out. (Assuming they hadn't donated twice as much to democrats as republicans and then had a crisis while republicans were in office, which is what happened to Lehman Brothers. The GOP has used its turn as referee to hurt the teams it doesn't like since Richard Nixon sabotaged the vietnam peace talks to prevent Johnson's reelection, and then Reagan did it again with the Iranian hostage crisis to undermine carter's reelection. The Democrats still trying to be impartial are Charlie Brown kicking the football.)

Third, the banks invented new categories of assets called "deriviatives" which are bets on bets. A derivative is a claim against another asset, with a dollar value attached. The classic old types were insurance policies (If future thing happens I get money, but if it doesn't I don't), which expanded to things like "put options" and "call options" (paying money now for the right to buy or sell a stock at a certain price on a future date, meaning an insurance policy against the price going up or going down. If the stock doubles and you have the right to buy it at the old price a year from now, you can sell that right to somebody else today for the price difference times the number of shares it's good for... except the price might change again before then so there's bidding up and down based on guesses about the future). But then in the 1980's the invention of "mortgage bonds" (which were described in the book Liar's Poker which I once wrote a review of) kicked off a revolution in derivatives: they could be made from _anything_. Soon they were creating derivatives of derivatives, which acted as places for billionaires to park money. (It's the same general principle as the art market where what things are worth are literally what rich people say they're worth, and then you can get an insurance policy from a bank to give you that much cash back if they _stop_ being worth that much.)

So the message from the 650 billionaires who own the USA is that we can't possibly afford basic income, but they can just assert a banana duct-taped to a wall is worth 6 figures and suddenly it is. Because they're rich, and the rest of us aren't, so there. They have money _because_ they have money, it's not even compound interest at this point it's a patent of nobility. If something happens to their assets they get a multi-trillion dollar bailout from the printing presses, at the same time as they literally commit bulk fraud against the debtors who weren't bailed out (the robo-signing scandal was that when the banks packaged up the loans into mortgage bonds they threw away the actual legal documents they were legally required to keep if they ever wanted to forclose, and then when they did want to forclose they literally forged new ones and hired random interns to sign them, so that when people sued the interns would take the fall). And then the people who committed that fraud took over the government and got put in charge of regulating their own behavior.

This is why the guillotine was invented, but it still seems like everyone's waiting for the Boomers to die of old age before we get to tackle climate change, student debt, the healthcare system...


February 6, 2020

Taught mcm-buildall.sh to apply the patch I came up with to build a proper nommu toolchain for sh2eb (removing the broken fork() that lies to compile time probes for nommu). The #ifdefs I added are checking for SH_FDPIC because that's the symbol that was set in the toolchain that I wanted to build; making this properly generic would involve testing dozens of #ifdef symbols (per-arch FDPIC and binflt) which is why everybody just checks for the presence of fork().

Also added a #define __MUSL__ to the patch in features.h while I was at it. (I didn't revert the rest of my disagreements because I've already worked around most of it, but that was just low-hanging fruit.)

The downside is nobody's regression tested building a native nommu toolchain, and fixincludes is broken. (It's broken with Rich's failing fork stub too, but it's build break vs runtime failure that gets ignored. Either way it doesn't actually _work_, but the other failure's harder to notice because it compiles a malfunctioning executable.)


February 5, 2020

Ooh, toysh edge case and legitimate behavior difference from bash:

$ echo $$; bash -c "echo \$\$; awk '{print \$4}' < /proc/self/stat"
8191
8238
8238
$ echo $$; toysh -c "echo \$\$; awk '{print \$4}' < /proc/self/stat"
8191
8244
8191

The difference is that /proc/self/stat is opened by the toysh parent process _before_ the fork and inherited by the child, but in bash the fork() happens first and then the child does the opening (and associated error handling if the file doesn't exist). So the awk in bash is seeing the PID of the bash instance that called -c, and the awk in toysh is seeing the PID of the host shell that did the echo $$ before calling toysh.

And I think this is a legitimate difference and I'm ok with it? The reason I did it that way in toysh is to support nommu, which bash doesn't care about. I don't want to do any more than necessary between the vfork() and exec, so I moved all that work into the parent process before vfork(), and wrote code to clean it up again afterwards. It complicates signal handling a bit, but I'm pretty sure I know how to handle all that and it's kind of how I'd need to do it anyway. (The signal handlers have to return and the callers check global status variables they set anyway to get the resource allocation and freeing right, siglongjmp() out of signal handlers is a pile of resource leaks waiting to happen.)

The downside is I was trying to use this to say "what PID is this process" to show that { } and ( ) were behaving differently. I think I still can, but it's a bit awkward getting the comparison value since "readlink /proc/self" gives you the PID of the child process that exec()ed readlink and then exited again? I suppose I can "(exec echo one; echo two)" and compare that with { }. There's also "(exit 1); echo $?" once I implement $? support.


February 4, 2020

Look, not Biden, ok? Quote from that 30 year old video: "The voters are going to have to decide if [Joe Biden] is dishonest or dumb". David Graeber was right about the burned out old centrists insisting that selling out was actually "growing up" when it wasn't, it was just cognitive dissonance. You can't fix anything by appealing to Boomers, they have become the problem and you can only counter them until they die of old age.

I'm working on a musl patch to make it work on nommu (chopping out the broken fork() that always returns -ENODICE but whose presence prevents programs from autodetecting nommu systems so they can vfork() instead), and I wondered what it would take to make bionic more nommu friendly, so I had to do the magic "repo" dance again. (repo pull? No. repo update? No...) Wound up looking up the instructions again (repo sink like a rock, got it) and downloading a fresh copy, and the page says I can do -j but the repo command I just downloaded says unrecognized option.


February 3, 2020

Years ago I watched the first movie of Jar Jar Abrams' Star Trek reboot in theatres, but didn't bother to go see any of the later movies because it was full of sound and fury signifying nothing. It was passable generic space opera, but not particularly memorable and wasn't particularly Star Trek either.

Jar Jar Abrams later did a Star Wars reboot, which I saw the first one of in theatres, and didn't bother to go see the next one because... meh? Rey was interesting, but the rest of the movie wasn't really. The elements of the original they brought back just served to _undermine_ all the new characters. The whole "death star but more" concept was simultaneously repetitive and silly. (No more orbiting the rebel base, instant kill of everything everywhere without moving! That'll show 'em.) Han's return was like "Indiana Jones and the Crystal Skull". Leia had like 3 lines and stayed home rather than adventuring (and apparently had done so for decades while ex-General Han had gone back into the low-rent smuggling business but that character dynamic was never investigated in any way), Luke was a McGuffin not a character (a non-speaking role!), 3PO/R2 were just fanservice. As with the Abrams Star Trek half the decisions ONLY made sense as a nostalgia vehicle aping better movies. (And judging by China, the new films don't seem to work at all without the nostalgia, or at least no better than the vast majority of the DCEU; lots of advertising gets a certain number of butts in seats but not repeat viewings or positive word of mouth.)

When The Last Jedi came out, I was annoyed that Disney went full-on monopoly leverage exploitation against theatres everywhere so thought I'd at least wait out the opening weekend... and then never bothered to see it. I remembered seeing the Phantom Meanace in theatres, being disappointed, and then hearing how much worse the next one was and not seeing it. I thought I might watch it on video, but then Avengers Infinity War pissed me off and I swore off everything disney (which was getting a bit big for comfort anyway, and see also the monopoly leverage thing).

But I heard good things about The Last Jedi. As with the Star Trek reboot, when Abrams wanders off and hands the reins over to someone else the result can be good. And that one was available on Netflix to watch on my phone, so I thought maybe I'd watch it sometime and see the sequel in theatres (but again, no rush if it was contractually obligated to be in theatres for at least a month)... but then for the third one Jar Jar Abrams was back in direct charge, and they cued up the Game of Thrones people to make the sequels to _that_ one right as those guys faceplanted, and I thought I'd wait to see what people thought of it... And it was apparently outright nonsensical DCEU levels of bad, so I wound up not seeing it at all.

But it took me a while to confirm that, because there'd been such a huge misogynist blowback against the second movie (from both gamergators and russian trolls, and in fact if you watch all the above playlist youtube's algorithm goes to dark anti-feminist places). But apparently as with James Gunn Disney's executives fell for the trolling and "Rise of Skywalker" was the _result_. The dumb of this new one is _because_ they repudiated the previous movie that pissed off all the right-wing loons. (And of course the loons are blaming Kathleen Kennedy instead of Jar Jar Abrams. Sigh.)


February 2, 2020

Ha! Changed the page title to 2020 and it's only February.

Remember, George W. Bush's TSA is completely useless, which is not news and never was.

So I'm trying to work out how toysh should handle subshells. A nommu system needs to make the subshell decisions earlier in the parsing, because it has to gather up a todo item and pass it to a subshell through a pipe instead of doing it and then forking after the fact, but that's not a big difference?

The real question is what _is_ a subshell. Obviously (commands; in; parentheses) are explicitly a subshell, but command | pipelines are implicitly in a subshell... except it hasn't got a _shell_, exactly? Bash doesn't fork a child bash that manages those child processes, they're just child processes. Except the child processes can be large chunks of shell function, as in the common "cat | while read i; do stuff; done" pattern where the while body is its own process. (The variables it sets don't persist after the pipeline.)

Ok, here's what happens there:

$ bash -c 'echo hello | while read i; do echo $$; ps; done'
12512
  PID TTY          TIME CMD
12316 pts/120  00:00:00 bash
12512 pts/120  00:00:00 bash
12514 pts/120  00:00:00 bash
12515 pts/120  00:00:00 ps

The 12316 process is the original shell in the terminal tab, 12512 is the shell I launch with -c, and the 12514 is the child shell it runs to do the while loop (reading the output of the echo on stdin, presumably the echo is 12513).

The fiddly bit is that there are _two_ function execution contexts, parsed and unparsed. Escapes like $(blah) and <(blah) are parsed as a form of quotes, and give you an unparsed string. I taught run_subshell(str) to pass that along to a vforked nommu child process through a pipe. But run_function() takes a struct sh_pipeline pointer which contains parsed pipeline segments, which the code did a lot of work to get. Turning them back into a string isn't that hard (it hasn't dequoted them, it's just a question of putting a space between each one, adding the semicolons back when there isn't another terminator... and you've gotta put here documents back. Ok, there's some work). But it's unnecessary work in the common case of "we have fork", where the child inherits the parent's memory with a copy of the processed structures.

Two codepaths means one codepath won't get as much testing, but I don't want to penalize the common case either. (It's the tail seek vs readall problem all over again...)


February 1, 2020

It's my birthday again. I'm old. Being a bit melancholly, as birthdays away from home tend to. I was looking forward to spending 6 months in Japan, but a 5 year visa is a commitment. I don't _know_ anybody here, and I don't know how to _meet_ anybody here?

If I was selling the house in Texas it would make more sense to move here longer term, but I'm still waiting out the cats (which may live another 4 years), and it _is_ an appreciating asset in a city 400 feet above sea level with a reasonable water supply surrounded by viable farmland. (The climate's hot but solar panels and air conditioning work very well together.) I was thinking about this in 1996 but the rest of the world seems to have caught up (especially after hurricanes Katrina and Harvey where it basically _happened_ already, albeit mostly not in a permanent way yet).

Speaking of the environment, methane emissions from animals continue not to be as bad as people with anti-animal agendas insist. (You don't even need the new stuff from that thread; the half-life of methane in earth's atmosphere is 7 years. It oxidizes. Earth used to BE a methane-ammonia atmopshere before oxygen generating bacteria changed it 2 billion years ago.) Raising cows takes literally a thousand times less water per acre than crops for human consumption. Also, cattle love kudzu and it's good for them. Goats, of course, will overgraze the kudzu and kill it given a chance. (It's what goats do.) Rich white guys spray pesticides instead of bringing in goats because they have no idea what they're doing, and every time they rediscover that goats find a wide range of invasive species tasty it's like this big revelation...

But of course, admitting brown people know anything is anathema to racists.


January 31, 2020

Well of course the GOP has disappeared up its own ass, what did we expect as the Boomers went senile? It's not that the politicans have no shame and are thus immune to most of the checks and balances stopping the "tyrrany of the majority" (MLK or Gandhi wouldn't have done very well doing peaceful protest against Hitler or Stalin), it's that their voting base isn't abandoning them for doing so. (Yes, hacked electronic voting, voter suppression, and gerrymandering magnify their vote, but there has to BE a vote to magnify.)

The GOP has succumbed completely to cognitive dissonance, but the real disappointment is that younger voters haven't sufficiently counterbalanced Racist Grandpa. The Boomers cut funding to education as soon as they personally were done with it (pulling the ladder up after them), so everybody stopped teaching the humanities, resulting in a populace with less humanity. If you don't teach voters civics, history, literature, or even "social studies" anymore, and instead focus on all STEM all the time, why would you NOT get nazi scientists?

Remember how after World War II everybody was fighting over whether it was ethical to use medical advances that came from experimenting on people in concentration camps? Or did we want to use Verhner Von Braun to beat the Russians (who captured the OTHER half of the V2 rocket design team) to the moon? Being a nazi didn't stop evil mad science, turns out fascism can be GREAT for that sort of thing. And thus you get agricultural monocultures patented by Monsanto, antibiotic resistant everything, global warming, facial recognition everywhere...

Science and the humanities serve different purposes, and removing one entirely from the curriculum was really bad for society. It would be nice if people started figuring that out soon.


January 30, 2020

A recruiter emailed me a thinly disguised "would you like to work for microsoft", and the answer is still no.

Endless fun shell corner cases!

$ walrus=42
$ readonly walrus
$ walrus=7
bash: walrus: readonly variable
$ echo $?
1
$ walrus=7 echo hello
bash: walrus: readonly variable
hello
$ echo $?
0

I think instead of giving setvar a return type, I just have it complain and set toys.exitval?


January 29, 2020

I really really really should call the senate lizards but I just haven't got the emotional energy right now. (Fade and Fuzzy don't because they never do.) It doesn't help that the times of day their offices are open is either right when I get up or right before I'm going to bed here in Tokyo.

Today's utter weirdness is a gcc bug. I upgraded dropbear from the 2016 release to the 2019 release, and it went boing. I narrowed it down, and Jeff narrowed it down further, and eventually we got to:

$ cat > blah.c << EOF
void fg(int *);
int get_response(int a)
{
  int b;
  if (a) fg(&b);
  return 0;
}
EOF
$ sh2eb-linux-muslfdpic-gcc -c -O -fstack-protector-strong blah.c
blah.c: In function 'get_response':
blah.c:7:1: error: unable to find a register to spill in class 'R0_REGS'
 }
 ^
blah.c:7:1: error: this is the insn:
(insn 28 45 44 4 (set (reg/f:SI 2 r2 [172])
        (mem/u/c:SI (plus:SI (reg/f:SI 1 r1 [173])
                (reg:SI 8 r8 [166])) [0  S4 A32])) "blah.c":7 188 {movsi_i}
     (expr_list:REG_DEAD (reg/f:SI 1 r1 [173])
        (nil)))
blah.c:7: confused by earlier errors, bailing out

Which is exactly the sort of thing you want to see gcc doing. Luckily it seems to be a bad interaction between the stack protector and the optimizer, so removing the stack protector (it's a nommu system, what exactly is it gonna do) is the fix.


January 28, 2020

Yes, exactly. But say this on twitter and they take your account away.

Ok, there's a missing design chunk in toysh and it's a generic quote traversal strategy. I just did utterly half-assed variable expansion to make the two failing "make test_sh" tests pass, but it doesn't do quote removal and it turns out you can't do that as a separate pass at the end because X='"abc"'; echo $abc prints the quotes, but just plain echo "$abc" doesn't, and you need to know what was or wasn't in a variable (already resolved) in order to handle that. There's similar active/inactive logic for wildcard expansion and word splitting (which someday needs to care about $IFS but I have literally never used that feature of any shell that I am aware of).

So I need to do quote traversal as part of variable resolution. I'm reasonably certain no shell context does variable expansion _without_ quote removal, but there's at least one that does quote removal without variable expansion (the here document EOF word has different meanings depending on how it's quoted, and yes you can E"O"F), and that's why they have separate suppression flags.

The first quote removal logic I wrote is in parse_word(), and that's got all the bells and whistles: it can decollate $(( into nested parentheses instead of a math operator, it does everything I could think of to get continuations right and as far as I know it does. It prompts for the next line (or doesn't) as correctly as I could come up with tests for. Unfortunately, it doesn't do anything ELSE. It's not designed to call from other contexts to advance past quote types that the current parsing context should ignore. And that's why I wrote a separate skip_quote() when I was doing brace expansion, because {abc","def} isn't a brace expansion, it's a partially quoted literal string, and there I needed the next unquoted character.

And now I'm frowning at "$((echo $((1+2))thingy) | tr h x)" and going "um...". For ((math)) I handled this so it's broken down into tokens, but $(()) vs $() are two types of quoting, and quoted blocks are a single word. Hmmm. I think I need to feed it back through parse_word() again, minus the leading $...

Strings in a single set of double quotes still get variables expanded inside them. Strings in any _other_ quoting context won't, but each of those contexts get parsed by pretty much exactly the code I'm writing now. single and double quotes get dropped. `subshell` and $(subshell) get evaluated (and the second can nest with any of the other contexts occurring within it, but you basically don't care here because you chop out the appropriate chunk and run_sh() it, except that needs to take a _length_ (unless I want to teach the plumbing that sometimes an unbalanced end parentheses ends a parse context without being an error?) and I need to teach it to report back syntax errors, but not regular errors:

$ echo $(if true)
bash: command substitution: line 2: syntax error near unexpected token `)'
bash: command substitution: line 2: `if true)'
$ echo $?
1
$ echo $(false)

$ echo $?
0

My notes say I still need to do parameter/variable expansion (which includes a couple dozen $$ and $RANDOM magic variables), Ineed to do $(command) (which is also `backquote`), I need to do $((math)) (a can of worms I haven't opened yet; I've written a two-stack math parser but it was a while ago), I need to do word splitting (screw $IFS), and I need to do pathglob (which is a funny name for wildcards, but eh; and yes that also does word splitting, but then again so does "$@" and someday I need to open the array variable can of worms but not yet).

Oh, and $[ ] is an obsolete synonym for $(( )), and ${} has a dozen or so different things you can parse internally. I believe the full set of quoting contexts that need to be handled is ${ $(( $( $[ $' ` " ' but that's only because I left myself a note earlier.


January 27, 2020

Oh wow, I've been approved for a 5 year residence card. (I expected 6 months. I don't quite know how to respond to that.)

Still another week or two of paperwork to do, and they took my passport to do it. Luckily I brought my previous passport with me, which is expired (and has holes punched in it) but is better than nothing until I get the current one back.


January 26, 2020

Twitter continues to be bad at being twitter.

Saw an article on yet another tiny python implementation, a competitor to micropython. I winced a bit at the "python 3.0" part but read on until I got to the "GPLv3" part, at which point I closed the tab. Nope, do not care.


January 25, 2020

Up until way too late last night going down toysh ratholes. Found that 2>&1 wasn't working and fixed it. The output from "ls" is happening WAY later than I thought it would, but it turns out the x86-64 build of toybox is using real fork() for xpopen, not vfork(). (Which I knew, but had forgotten was the case here. I think I had vfork forced on in the config but every time I "mv root root2" and then rebuild without remembering to move it back, I get a blank .config and don't always remember to put everything back...)

Today's kernel debugging timesink: it turns out init_task isn't PID 1, it's PID 0. So comparing its /proc/self/exe entry (in theory init_task.mm->exe_file) with current->mm->exe_file to only print out diagnostic info about toybox processes (same executable as rdinit=/bin/sh) didn't work, because task.mm is NULL. (Not that it would have been much of a filter on a toybox initramfs, but still. Eh, it's got each PID in the debug lines I'm printing out, I just wanted to cut down on the pages and pages of noise.)

The OTHER timesink is that when I finally drilled down to what was going on, yes I totally could have found this with userspace printf(). The reason CLOEXEC wasn't triggering is IT DIDN'T EXEC. (Sigh.) On a system with mmu, xpopen() calls fork, and if we haven't used too much stack it will recurse instead of re-execing itself. Since strace is reporting both fork() and vfork() as clone() for some reason (it's a different syscall number!) it LOOKED like it was doing vfork(). What I should have done is stick a printf() at the start of main() in main.c to see when actualy reentry was happening. (Which I eventually did...)

Also, printf(), dprintf(), and printk() all deciding to randomly delay output so it gets produced out of order? Thanks bunches. I had to stick a getpid() into every single line I printed (LIVE, not stored in a variable) to even ATTEMPT to sort that out, and even within a process it wasn't showing me things in _order_. Grrr. (Also stuck in sleep(1) in more than one place to get parent/child interlacing down to a dull roar. And I'm tempted to move "writing my own strace for toybox" up a bit on the todo list, so at least I'd have one with predictable behavior.)

That said I still want to support <(blah) on a system using fork, so the code needs a bit more cleanup before I can check it in. But at least now I know what was going on.


January 24, 2020

Closing tabs, lots of old links I never blogged about or catalogued anywhere...

I saw a puff piece on sunrun (seriously, "what was your childhood like?" and "how do you lead?"), but eh at least it's not that Musk guy, so I looked up their website and tried to price a solar system from them, and two different tabs refused to give me ANY information without first collecting my address, name, email, blood type, nudes, and the best times to bother me forever endlessly amen, and THEN if a credit check deems me worthy get back to me with a dedicated hard sale team refusing to take no for an answer. (They do not have a catalog, they have a Sales Pipeline Data Collection Engine, and have lost my interest.)


January 23, 2020

I'm largely insulated from impeachment here in Tokyo. Instead everybody's freaking out about the coronavirus. (Which apparently the states are doing too. It's reached the Uighr concentration camps in East Turkistan, where china has a million people forcibly held in racist detention facilities, which is a terrible idea for many, many reasons.)

One of the people I used to follow on twitter noticed I haven't been posting, and sent me a "checking in" DM, which I apparently still get forwarded via email but can't reply to. I can't find an email address for her, so asked Fade to reply with a pointer to one of my blog entries about it.

It looks like twitter changed its stylesheet again. Now when you look at an individual user's feed you have to scroll down past their header intro blob (which used to be a sidebar), then their pinned tweet, then you get four recent tweets, then an advertisement for people you don't follow, then the rest of their tweets are below that. Because chopping up the feed is apparently what twitter thinks people want? (Or they're trying to punish people reading individual feeds without being logged in? I know it won't show the with_replies page without a login anymore, because I tried.)

*shrug* Oh well. I miss the people, but not the service. Twitter was never important, it merely connected me to other users who were coincidentally on the service.

(Huh, and I just remembered I can search for @landley in the twitter search bar to see if anybody poked me and yes they have. Hmmm. I should figure out a way to reply. (And I would appear not to be the only person searching for old tweets. Plus people linking to my talks.)


January 22, 2020

Originally when we scheduled this trip I was going to fly on from here to prepare our linuxconf talk, then fly to australia, give the talk and demonstrate the new round of turtle board prototypes, then come back here to help Jeff finish the open source VHDL turtle device support and upload it to github to prepare for getting the Turtle boards on crowdsupply (including finishing the hdmi bitmap stuff we started in canada last year), and maybe visit singapore or taiwan depending on how other people's schedules worked out, then go back home after a month, and THEN do visa stuff (which would involve a trip from Austin to the Japanese consulate in Houston, probably via greyhound)...

But that's not what happened. The trip to australia fell through, the trips to singapore and taiwan never happened because we're _still_ waiting to hear details of other people's schedules (Jeff might go to Singapore by himself after I head back, but nobody wants to get too near china until the beer virus settles down)... And when we got here Andrew suggested a different way to apply for a visa, and now I don't think I can leave until the visa thing is either approved or denied?

I also can't commit anything to closed source repos from here, because that would be "working on a tourist visa", which is not allowed. (Following Jeff around while he buys a humidifier has never been part of my job description, it's helping a friend move.) But I've done open source programming on a hobbyist basis for many years now, while working dayjobs at a dozen unrelated companies, and I don't go anywhere without my personal laptop.

We put a J-core repo on github last year, and what's there works, but it's a fork that isn't unified with the Turtle board development (let alone J32) so... cleanup time! The ICE40 Turtle repo on github forked off the old hg cpu repo at commit 734, and the turtle boards are running hg 763 (which is pretty much j2 tip; there isn't a branch, but really should be. I'm not sure which commit the second open source tarball on j-core.org was relative to, but it's probably somewhere between those two). Then the J32 tip is hg 813 in the same (sadly mercurial) repo. And that's just the cpu repo, there's a dozen others for things like the DRAM controller and I/O devices and so on, but for ICE40 what we did was fork the CPU repo and stick just that in the tiny FPGA, with a couple raw I/O lines that can drive LEDs and such routed out so you can see it do its thing. Jeff hooked it up to calculator keys and a little calculator display, but that's yet another repo and I've never had a copy of that code or hardware.

So I'm trying to turn the open source branch from github into a patch stack we can forward port to the Turtle branch, and have a unified repo again. Alas, this means rebasing the github repo, which probably means uploading a new repo entirely. Currently it just means reading stuff and trying to understand what we did so we can clean it up and publish it at some point.


January 21, 2020

A TTY was never a typewriter, it was a teletype.

Fun history: the Intel 4004 was commissioned in October of 1969 for Busicomm. That was the first microprocessor (not the first _processor_, the univac went on commercial sale in 1950, but it was the first single chip implementation of a processor instead of a motherboard full of components wired together).

But the 4004 is a historical curiosity, it was a 4-bit processor barely powerful enough to run a calculator. The first 8-bit processor was the Intel 8008, and it was commissioned two months later in a meeting with Victor Poor of Computer Terminals Corporation. As d Ted Hoff said, "they were building what they sometimes referred to as 'glass Teletypes,' computer terminals that used a cathode-ray tube instead of paper to, you know, present the information."

The first 8 bit microprocessor was comissioned for a paperless tty device (not a typewriter) the same year PDP-7 Unix was created. Given that Unix didn't escape Bell labs until the mid 70's (the paper describing it was written in 1973, the ACM talk was delivered in 1974, and Bill Joy took his year off to teach at Berkeley training the students who would create and maintain BSD starting in the fall semester of 1975), the only reason people hooked up old ASR-33 teletypes and so on to Unix machines is they were available cheap on the secondhand market. Bill Joy wrote vi in 1975 to take advantage of "glass tty"'s ability to move the cursor around. (Cursor keys weren't standardized yet so he made a modal editor that used normal letter keys to navigate, and even though cursor and function keys ARE standard now, vi is sadly still a Tardis console without the "telepathic labelling of all the controls" part. (Which explains why the fifth doctor never remapped them, the head injury from falling off the Pharos project tower clearly left his next incarnation telepathically blind, couldn't even recognize the Master face to face. I suspect the celery is a gallifreyan equivalent of the dark glasses and big white cane, letting other members of his culture know he's down a sense. Yeah, "contact" scene in the five doctors but the other incarnations were doing the heavy lifting, shortly after breaking him out of mind control.))

Anyway, as far as I can tellTTY just means "bidirectional pipe plus cursor position and ability to signal attached processes". It's somewhere between "ZIP code no longer stands for Zone Improvement Plan" and "Wi-Fi never actually meant anything in the first place". (Nobody expands ABC, NBC, or CBS anymore either.)


January 20, 2020

So I'm doing ./sh -c 'awk '"'"'{print $4}'"'"' /proc/self/stat ; cat <(/bin/echo hello)' (as you do) and for some reason, fd 5 is losing CLOEXEC. I've looked at strace -f and there's no obvious REASON it's losing CLOEXEC. The syscall succeeds, and if any later syscall anything else is removing it I'm not spotting an obvious candidate. Then after the vfork() the filehandle is still there in the child, so we can never close all the input sides of the pipe, so the output side of the pipe never returns EOF, so the call to cat never exits. Test hangs until you hit ctrl-c.

I fired up mkroot and reproduced this under qemu, which has _so_ many rough edges (the most _annoying_ of which is needing to type "reset" after every qemu run because otherwise the cursor goes up one line in the xterm every time I cursor back in command line history. This is because QEMU is switching the DECAWM attribute off, and I had to teach reset to switch it back on last year.)

Anyway, I've got a cycle of make ARCH=x86 CROSS_COMPILE=~/musl-cross-make/ccc/x86_64-linux-musl-cross/bin/x86_64-linux-musl- -j $(nproc) to rebuild the kernel and qemu-system-x86_64 -nographic -no-reboot -m 256 -kernel arch/x86/boot/bzImage -initrd ../x86_64/x86_64-linux-musl-root.cpio.gz -append "panic=1 HOST=x86_64 console=ttyS0 rdinit=/bin/sh" to boot the virtual system, followed by pasting in the above command line to run the test. Then in the kernel arch/x86/entry/common.c function do_syscall_64() I added printk(KERN_ERR "call %d %lu %lx\n", (int)task_tgid_vnr(current), nr, *(rcu_dereference_check_fdtable(current->files, current->files->fdt)->close_on_exec)); to print out the pid, syscall number, and first 32 close-on-exec bits (including fd 5) every time it makes a syscall. (Again, as you do.)

And it looks like what's happening is a later call to popen() is removing cloexec from the filehandles returned by the previous call to popen(). What the...? I can't be reading that right...


January 19, 2020

Trying to get the next chunk of toysh checked in and... I've narrowed it down to a place where fcntl(pipes[1], F_SETFD, FD_CLOEXEC) is being ignored (the child vfork()ed after that still has the filehandle in pipes[1], which in this case is 5). According to strace the fcntl happens and returns 0, and the right arguments go into it, but the child still has the filehandle anyway.

I tried pipe2() instead of fcntl() (which requires prototyping the libc function myself because I refuse glibc's insane stallman-worship #define for LINUX SYSTEM CALLS) and that didn't fix it.

But if I dup2(5, 12345); and then close(5) and assign pipes[1] = 12345... then everything works. And I have NO IDEA WHY. There's no other filehandle the child could have gotten that end of the pipe from to dup2() it back down via misguided unredirect() plumbing (I dup'd the other end but not this end), the clone() (which is how strace reports vfork) is happening _after_ the fcntl... how do you screw this up? I'm not seeing dubious stuff in strace(), although so much is going on (4 processes working at cross-purposes) that it's hard to spot.

This is "stick printk() calls into the kernel" level of WTF, which means I need to reproduce this under qemu (which means a current kernel and musl instead of devuan's kernel and glibc).

Hmmm...


January 18, 2020

The discussion starting around 18:15 here seems to think that the value of battery walls is to arbitrage electricity from the grid, buying it during cheap times and selling it back to the grid when it's most expensive. That makes no sense at all to me.

First of all, the grid guys can install solar too. Solar has been the cheapest way to add power capacity since 2016, and in 2018 installing new solar became cheaper than running existing coal plants. The downside of solar is it produces a bell curve of power each day, and because artificial lighting shifted everybody's waking hours into the evening back in the "whale oil and gas lamp" days of the 1800's, this gives us the duck curve as the sun goes down while everybody gets home and starts cooking dinner.

Municipal interest in batteries has primarily been about compensating for the duck curve. Extra solar is cheap, and they've been hitting curtailment for years (without storage generation has to match demand, over-generation means you unplug the solar panels and waste the electricity). But if you can stick the extra in batteries and then feed it into the grid in the evenings and overnight, solving the duck curve is just a question of installing enough batteries. Or, since curtailment doesn't actually cost you anything and solar gets cheaper every year, you could just install more solar and if you can't use it all at peak, what have you lost? Solar you don't install is curtailed 100% of the time, being able to use _any_ of its output still puts you ahead.

This means electricity being most expensive in the middle of the day is an artifact of old thinking. The middle of the day is when they're GIVING electricity away, and before the law caught up to allow curtailment they briefly had to pay to dispose of it. So your "energy arbitrage" opportunity, once grid operators catch up to that reality, would mostly be filling your own batteries up during the day and selling them back to the grid at night. And why is that interesting when the utility can just do it themselves by buying their own batteries? The current curve is that batteries get 50% cheaper every 3 years.

With solar panel prices continuing to fall too, batteries are the bottleneck now, not panels. Municipalities can trivially double their solar panel installations at any time, and there's no shortage of land to put it on. Powering the entire US would take 17.5k square miles of solar panels, but that's not actually much space. The US has 3.5k square miles of just golf courses (and some people think golf courses are about enough space). Austin's been sticking solar farms a few miles outside of town for a decade now, but that may not be necessary: there's a LOT of space for rooftop solar. The US has 314 cities above 100k people, and adding up their land area (column 7: square miles in 2016) gives 29,351 square miles, almost 12k _MORE_ than 17.5k, making it entirely plausible that rooftops (and down the sides of skyscrapers) could provide all the electricity we currently use even without trying to make roads and such participate. (To check the math, the list of the 150 largest US cities, minus the alaska entries, the square miles of land area (column 4) adds up to 25.2k square miles, still almost 7k extra just downtown in big cities.

Meanwhile, when I did the math last year I found that a grid tie costs an average of $62.50/month to _NOT_ give you any power, that's just your share of grid maintenance costs. If you have a big enough home battery wall and never feed anything back into the grid (again, extra solar's cheap and curtailment doesn't _cost_ you anything) the grid tie is a gratuitous $750 annual bill, and that's not just a US issue.

If you have your own local battery, being filled from your own solar panels, the main effect is to reduce your dependence on the grid. It might make sense to have ties to your neighbors, maybe wire the whole block together, but a $62.50 monthly insurance payment to pipe in electricity from the other side of town? 99% of the time you wouldn't be using it, just paying for it. It makes more sense to have a $500 backup generator kick in if your batteries go below 10%.

Yes cheap out-of-town solar farms still make sense for electric self-driving app-summonable fleet vehicles, with a self-driving 18 wheeler driving out to the farm to swap a container of dead batteries for a contianer of charged ones -- don't unpack the batteries, plug in the whole _container_ and let them charge in there, it's just wiring and cooling. Then in the city robot arms swap out the batteries and unpack/pack the battery containers. No humans involved anywhere except in security and maintenance of the robots.

But I think the big thing people underestimate (and I didn't hammer on hard enough last time I ruminated about this) is that S-curves don't stop at 100%. Going from 1%-10% is the same as going from 10%-100%, which is the same as going from 100% to 1000%. That link I posted above about people just now realizing curtailment isn't a bad thing (all the solar you DON'T build is curtailed full-time, so being unable to use some of the output at peak is no reason not to install more ever-cheaper solar if you can ever use _any_ of it: the output that comes at a time you CAN use still displaces more coal/oil/gas and is _free_ once installed)...

Of course our energy distribution grid isn't NEARLY ready to pipe a 10x increase in peak daily energy production through cities (you'd need superconductors or solid 10 foot thick copper crossbars, and either emit utterly impractical magnetism). But if you put variable manufacturing out in the boonies and have it do lossy things like "smelt aluminum" or "make jet fuel from air and water using electricity" to consume electricity that would have gone into curtailment, on a start-and-stop "as the energy is available" basis... so what if they're only running 10% of the day? Variable manufacturing's gonna be a thing. If you can't store it, throw it into lossy processes that do SOMETHING with it.

Keep in mind growing crops via photosynthesis is only 11% efficient (half of the 25% modern solar cells give), and then human digestion is about 25% efficient (and we can't even digest cellulose)... but obviously it's still worth doing. If we can take excess would-be-curtailed electricity and make gasoline or kerosene from air, retaining 10% of the original energy? That keeps airplanes and container ships and space flight going even after the oil companies' own reports about global warming from the 1980's turn into a giant smorgasbord for lawyers the way the tobacco companies did. Affordances vanish as the percentage of the populating using them fall below criticial levels. Water troughs, hitching posts, and livery stables vanished from cities as horse ridership declined. Outlawing carbon mining is probably inevitable by the time half the Boomers die (2034 according to the actuarial tables).

There may be an arbitrage market to suck up everyone's excess peak generation, if we can work out the transportation part, but if we continue to pay 1/6 of our economy for energy but get 10x as much, that means the price per watt falls to 1/10 it is now. If it's easy to stick factories 20 miles out of town where space for the sun to hit is cheap, I'm not sure how much exporting from the city is worth? (Plus air conditioners and heat pumps will suck up a lot downtown as the climate continues to go deeply weird.) Once you get used to curtailment, using the extra is nice, but how profitable is it really? (Dunno...)


January 17, 2020

Jeff got most of the GPS correlators up and running on turtle, but it turns out if you just grab the high bit of the nanosecond time counter to have a quick and dirty once-per-second clock signal, the correlators reset on BOTH edges, and thus zero out their millisecond counters _twice_ per second. (It's hard to get the VHDL tools to let you act on both rising and falling clock edges because clock signals have magic routing in the tools, but when you're already clocked and testing != to last time? That it'll do twice, yeah.)


January 16, 2020

I see it's that time of year again. "How can leaving the freezer door open melt the ice in the trays when it's so much colder standing in front of a freezer with the door open!"

We broke the polar vortex, so the cold is no longer staying at the poles, instead it's leaking down here. It should not be down here, the cold should stay up there, that's why the ice is melting. Any extra snow down here won't last the summer. Meanwhile, we're having hurricanes in scotland in the middle of winter and this is such a regular occurrence it's no longer newsworthy.


January 15, 2020

We've started work on GPS on Turtle. Jeff added the correlators to the Turtle build and wired them up to the GPS Hat... which didn't work. We asked for a new schematic for the turtle hats from Martin, apparently the one we have isn't the final version. (The connector was reversed, so his original layout stomped some ground pins. Martin's great at circuit design in boards, but has a bit of I/O dyslexia when it comes to interfacing with the outside world.)

Meanwhile, there's a tcxo (thermally stabilized clock) on the turtle hat, this time with a temperature sensing diode instead of a heater, so it can adjust to temperature changes rather than trying to prevent them (this approach eats _way_ less power and is pretty much what we _have_ to do if we're ever to make it work on ice40 from a coin cell). Unfortunately Martin write down the part number for us so we can't look up how to program it, and it's under the metal shield so we can't look at the chip without destroying one of our 5 GPS hat prototypes. Hmmm...


January 14, 2020

Woke up with my eye bleeding this morning. (Or at least I noticed it was in the mirror while waiting for the elevator down to the lobby at the hotel.) Hoping that I cut it on my fingernail or something without noticing in the shower, rather than something blood pressure related.

I also noticed the linuxconf.au talk slot we couldn't make went past. Told jeff "I assume this means we're never doing that video version of our talk" (which we offered to send them instead of attending, and to which never replied), and he got mad. (While I'm presumably right on substance, he dislikes me being "defeatist". What I meant was, are we going to prioritize doing it in the absence of a pressing deadline that we already got distracted by more pressing concerns from and allowed to go past, or are we going to have it be a "thing we should do" and it still be such 2 years from now because there's _always_ pressing concerns du jour that we're doing instead? Because I know me, and in the absence of an externally imposed deadline, that's usually what happens.)

Meanwhile, I'm kind of proud of today's commit:

 mkroot.sh |  501 +++++++++++---------------------------------------
 1 file changed, 115 insertions(+), 386 deletions(-)

I admit the KCONF= lines being longer than 80 chars is cheating when talking about line count, but it's awkward to break them? I still need to do the "switch off CONFIG_EXPERT and disable these symbols" config category, which would make it 3. (Global symbols, arch symbols, and symbols to disable on second pass.)

Bisecting musl-cross-make to see where Rich broke it. 9b8fcbc4cada works but 38e52db8358c (current) does not... ok, last one that builds is 629189831f61, it's b5b4d47c48b2 that broke it: "update defaults to gcc 8.3.0, binutils 2.32". Updating _both_ at the same time, great. So what broke it was updating gcc from 6.x to 8.x, skipping 7.x entirely. (I dowanna bisect the gcc repository. I really, really don't. It's gnu. That makes it bad.)


January 13, 2020

Jeff poked me into installing "Signal" for work. It requires a phone number to access, and it's so secure it publishes everyone's phone number to everyone else you ever talk to (including group chats). It has pop-ups prompting you to set a config setting that isn't then available in settings. The second text I ever sent had a full-screen popup about changing security numbers (more or less the ssh "adding host key" message, only far more confusing and a MODAL POPUP)...

On the whole, Signal strikes me as having the same user-friendly polish and user interface fit into its problem space as git and "the gimp". But between Signal, Slack, Line, Skype, talky.io, Wechat, freenode, texting, and phone calls, I'm sure we can figure out how to communicate somehow. (I miss twitter.)

Fixed some more shell stuff and I'm trying to boot mkroot with toysh, and it's segfaulting running the init script. (Failing I'm not surprised by, but _segfaulting_ it shouldn't be doing.) First I had to fix the "init has no stdin/stdout/stderr" problem, which is why I sent a patch to fix it back in 2017, but gave up trying to deal with the kernel clique after 3 resubmits, and as far as I know they never did fix it. I just stuck this quick hack at the start of toysh for now:

if (getpid() == 1) {
  sh_run("/bin/mount -t devtmpfs /dev /dev");
  dup(dup(open("/dev/console", O_RDWR)));
}

January 12, 2020

Jeff's excited about a circuit design GUI thing and poked me into getting it built from source, then tried to teach me how to use it. I objected (FIVE open cans or worms already!) and that led to a longish design discussion about ASIC strategy (I.E. why we need to open a sixth can of worms in parallel). We REALLY need to do youtube videos about this stuff...

Wearing the wrist brace Fade got me. It itches, but my mildly sprained left wrist keeps twinging and I think it's trying to turn into RSI with all the typing I do. (It's in the wrong _place_ to be carpal tunnel, but it has high hopes. I guess these days that would be high hopes.)


January 11, 2020

More shopping for the tiny new office. We got a humidifier, a handheld whiteboard, a shelf, and a _second_ router. (Jeff reimagined the first one with openwrt, so of course it's now a brick. He says it's because he usually usses ddwrt, which doesn't support any hardware we can find in stores. I convinced him we're already behind a NAT here, so our second layer of NAT isn't really providing significant security, and we expect to outgrow this office this year anyway.)


January 10, 2020

We have an office! It's a "hello office" which is a Japanese company that rents out office space on the model of Public Storage units, only instead of an 8x10 concrete space behind a garage door, roughly the same amount of space in a corridor of similar doors, each with a little nameplate, in a building with 4 floors of this stuff; the first floor is a Curves gym and the basement is the G-19 subway station. The room has 4 desks, industrial carpet, flourescent lighting, an electrical outlet, and an ethernet jack with gigabit download speed (haven't benched the upload speed yet).

Of course we wasted hours trying to associate with the wifi until we figured out it's a "bring your own router" situation. (The orientation pamphlet didn't say.)

Also met with a lawyer who started filing the Visa Paperwork to get me permission to stay longer than 90 days/year.

Four bottles of milk tea (and a maple milk tea at Excelsior in Akihabra) was probably a bit more caffeine than I should have in a day, but it's SO GOOD.


January 9, 2020

I'm terrible about dating blog entries in Tokyo because I keep my laptop set to tokyo time, and it currently insists it's 5 PM on the 8th when it's 8 am on the 9th here. This was less of an issue when my blog entries were uploaded months behind where I was writing it, and I was mostly writing down "what _did_ I do on tuesday?" (check email, check twitter, slack, line, freenode backscroll, git log...) But now that I at least _intend_ to upload them after I complete each one (ok, currently hung up on finishing the Jan 3 entry because I left myself another "writeup this topic by integrating these 11 ideas into a coherent synthesis" and that's _WORK_...)

Anyway, it's morning in Tokyo. Ronald Regan is still dead. The TV is playing Japanese children's programming, on the theory I'm more likely to pick up snippets of that than the news. (And it's less likely to mention the GOP's _current_ Bastion of Senility, who is lashing out at the staff of his nursing home demanding to know who stole his teeth because he forgot where he put them so obviously they were stolen. Except this involves Iran, which was totally pacified when he took office.)

A duck/robot is singing a (surprisingly upbeat) chiptune song about having a pixelated picture of a broken heart on its chest. And now it's playing drums in a band where a purple/pink squid is on electric guitar and... (I think that's a dog costume?) is on bass. And now the person in the dog costume and... I don't know what the yellow blob with the helmet is supposed to be... are doing some kind of sports commentary. And now it's switched from people in mascot costumes to crayon drawing animation.

I believe I've completed the curly bracket expansion, or at least it's passing all my tests. Somewhere along the way I screwed up an allocation tracking and something's getting freed when it shouldn't, so "echo hello" is now barfing half the time saying it can't exec a string of line noise. (Washed through variable expansion and then freed before use I expect.) But that shouldn't be too hard to track down, and I bypassed the bracket expansion (which is a wrapper function at the start of variable resolution so that's easy to do) and that didn't fix it, so it was something else I did recently and didn't immediately notice (because use-after-free sometimes works, I should switch on a page poisoner) so it doesn't stop me from checking this in to pending.


January 8, 2020

The flight across the international dateline ate a day, so as far as I'm concerned January 8 started at 4pm in the airport Tully's (a coffee shop chain here). I caught an airport shuttle at 2:30 am and flew for SO LONG it's now the afternoon of the following day.

I'm very tired.


January 7, 2020

In airport. The ironic part is I had to stop binging "That time I got reincarnated as a slime" because my flight to Japan was taking off. (Crunchycrunch does not let you download episodes, and does not work in Japan at all due to region locked licenses.)

Despite much sleep dep, I got some debugging done on the flight. There are so many corner cases in curly bracket expansions, and I'm not 100% certain I'm testing them all yet. I've tried to be systematic about it, but keep hitting new weird ones. Wound up checking in a commit that's like 2/3 debug printfs by weight, but it's progress. (It passes TWO of the tests before segfaulting on the third.)


January 6, 2020

Called the doctor up in minnesota about my blood pressure medication not being refilled after a week, and they won't give me any more until I get my kidneys tested. (Because 20 years of caffeine being a stronger diuretic than the stuff they gave me clearly had no effect, but_this_ might.) And they're ok with me being off the blood pressure meds until I get that done, even though I explained I fly to Japan tomorrow for a month, and have already been off the pills since 3 days after I requested the refill a week ago, so they've basically taken me off blood pressure medication for the forseeable future, and they seem to be ok with that. (Oh well, if the medical professional doesn't care I guess that's ok then...)

John Rogers (the guy behind Leverage) linked to an excellent thread about money laundering, but I seriously think the Boomers dying is the only way the strangehold will break: a group of people who _already_ have single digit life expectancy aren't gonna be dissuaded by their decisions dooming us all. (And yes, that's a good photo.)

Scheduled to fly to Japan first thing in the morning, shuttle picking me up at 2:30 am. Trying to get toysh to a good stopping point. Debugging the bracket logic. There's a debugging failure mode I can get into where there's so many dprintf(2, "blah %p\n", varname); that you can't easily follow the _code_ anymore, but I don't want to remove any because as soon as I've answered _this_ question I want the context for the next run...


January 5, 2020

Two days to Japan flight. Racing to get toybox usable. Still wrestling with brace expansion.

Ok, there's 8 kinds of string segments you can output in brace expansion:

// span from start to {
// span from { to }
// span from { to {
// span from } to }
// span from } to {
// span from , to {
// span from } to ,
// span from } to end

Keeping in mind that other expansions haven't happened yet, ala "A= ABC=123; echo $A{BC,''}" outputs "123", we're just outputting literal string segments to the next stage of expansion. Except start->{ and ,->{ and {->{ are basically the same thing, and the same for the trailing three. Hmmm... [3 hours of staring at code before I remember to blog about it]. As always, I go through dozens of culdesacs where I go "wait, but this implies this and that means..." until I come up with something simple looking by process of elimination. It's somewhere between a jigsaw puzzle and untangling a small gordian knot.


January 4, 2020

The bash man page says there are 7 types of expansion, but there are actually 8 if you add quote expansion. And the problem is the order of operations they give, "brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and pathname expansion" does not _include_ quote expansion, and it... sort of goes in multiple places?

$ echo {~,~root}/pwd
/home/landley/pwd /root/pwd
$ echo \{~,~root}/pwd
{~,~root}/pwd
$ echo ""{~,~root}/pwd
~/pwd ~root/pwd

The quotes affect {brace,expansion} but aren't _consumed_ before brace expansion, which implies they have to be parsed more than once, which I'd _really_ like to avoid. (My sh.c is 1796 lines and some of that's fluff that goes away when I'm done. I'm trying to keep the whole mess under 3000 if I can, 3500 if necessary.) Ah, I see it says quotes are removed at the end, but have been PARSED multiple times earlier, which means multiple passes over the string which seems gratuitous? Ok, that's how _they_ did it. Can I do better?

Grrr, this logic kinda wants to be a recursive call and I'm trying hard to do it with loops. But braces nest so it would need a stack.

Sigh, I have to care about $IFS. I hate $IFS. It's SUCH a hack.

I'm treating process substitution as a redirection, meaning "abc<(echo)def" does not become "abc/dev/fd/63def" because WHAT IS THE POINT OF DOING THAT? (Ok, if you've mounted devtmpfs someplace strange then MAYBE you'd want a prefix, but it would still have a hardwired /dev/ under that. And lots of stuff uses /dev/null and friends without caring, not much point special casing that.)

So, parsing {brace,expansion}, the easy thing to do is parse all the way through to the end and then iterate through each segment allocating a new extent a copying the relevant contents into it. I was doing an array of offset, length, and index (for traversing), but said array kind of wants a hardwired length and there isn't inherently a limit on how many of these you can have in an argument. Plus not every { results in one of these extends, you need a { with matching } _and_ a comma in between. With no comma, you leave {contents} with the curly brackets in the result. Or if there's no closing curly brackets, "echo {A,B,C" prints {A,B,C as-is. So if there _is_ a limit, it can get hung up on "echo {}{}{}{}{}{}{}{}{}..." exhausting said limit. (Yeah, unlikely, but not the right thing to do.)

Nope, better approach: make a struct, assemble a linked list of that struct, and pop completed candidates off onto a second list as they're completed, or dispose of them if there was no comma before they ended. (Which can get silly: {{{{A,B},C}D},E}


January 3, 2020

Part of the reason I used to get so far behind on blog entries is I would leave myself notes like this:

New david graeber talk, seems related to Mark whatsisface talk, and the australia guy (rutger bregman?). previous link, stitch together argument

Which means NOTHING to anyone but me. The new talk makes a bunch of good points I should make notes about (which means rewatching it), I put together a playlist of Mark Blyth talks (which I'm sure I tweeted about, but it seems to fall in the gap between the backscroll twitter shows and the last archive I downloaded before they screwed up the format so it was no longer CSV of the actual useful data but instead horrible bloated json you need some sort of tool to parse anything out of). I also tweeted about Rutger Bregman who is an eloquent proponent of basic income who I used to follow on twitter.

And I forget quite _which_ argument I found so compelling at the time, because I haven't rewatched the talk yet. I've combined Blyth's and Graeber's observations into a story before, and I've done my own research to show that 70% of the population was subsistence farmers 200 years ago and now it seriously looks like less than 10% of the population does ANYTHING related to actual human survival. Seriously, that link has numbers: if 90% of the population stopped working entirely the main impact is we might be less entertained (although AO3 fanfic, podcasts, and whatever would replace youtube if capitalism stopped strangling it could probably take over from Disney pretty smoothly).

I've written here before about how Graeber's BS Jobs argument is understated because he focuses on the 1/3 of people who say their jobs are _entirely_ useless, not the people who have 2 hours of real work each 40 hour week, or the dozens of support workers (HR, payroll, cafeteria, janitors, building maintenance, tech support...) every investment banker is propped up by, filling all the floors of the skyscraper below the penthouse by doing useful work in _service_ of some kind of financial scam.

So I left myself work to do. Again. My todo list runneth over: the more work I do, the more work is left to do.


January 2, 2020

I should do a toybox release before flying to japan. This means I should do it on the 6th because I have to leave for the airport something like 3am on the 7th.

I added MAYFORK to pwd, which replies on $PWD being set for -L to work right, which means I'm teaching the toysh "cd" builtin to set pwd, which means I'm implementing -L and -P and so on, which turns out to be a bit of infrastructure with fallout.

Converting "help" to work as a builtin too. Running it with no arguments should list the "usage:" lines of all the builtins, which is more new plumbing. (For one thing, "determine if you're running as a builtin", which is a new high optflag to go along with NODASH.

I should do a FAQ entry writeup that you can't build or test the standalone "help" or "install" through "make", you have to "scripts/single.sh help" and "scripts/test.sh help". (Flat namespace, and those have other meanings.)

Why is "help -au" showing "true" and "test" at the start of the alphabetical list (out of order)? It can't be "that's where all the NOHELP commands go" because sed and false are in alphabetical order. (Sed has to handle its own --help so it can say "this is not gnu sed 9.0" for lying to autoconf when it asks stupid questions.) Yes, I'm aware that show_help() is inefficient when showing a list of help text in order, probably I should do something with a callback. (The help strings are static to lib/help.c and the code parsing them is all localized there, because then I can gzip the help text at some point. For now I'm going "help text display is not something that needs performance optimization"...)

Next problem with cd expansion: the code I wrote assumes tilde expansion is implemented. Hmmm, how crappy can tilde expansion be? What does bash do for 'echo ""~landley'? Answer: it's a literal ~landley, not a path. Ok then, it's a prefix. Ended by : and / as far as I can tell. (Because / is a path segment and : can't be in a /etc/passwd format username field. Presumably \n terminates too, but that's kind of already covered. :)

I implemented the first actual variable expansion type, which means I'm adding the first allocation to the delete list... and double free dump. Which is misleading because what _actually_ happened is I was creating a struct double_list and then traversing it as a (singly linked) arg_list, using "prev" as "data", and freeing the wrong pointer. (Because I have a dlist_add() function so it's easy to create a doubly linked list, but singly linked lists I do by hand and in this case it's 4 steps, borderline "eh that should have a function"...)

I should probably have struct double_list be a structure with _just_ the prev and next pointers, and then have it be the first member of other structures. That would avoid all the typecasting I'm doing whenever I use those functions. (It's easy to use C in an object oriented manner, I just don't bother half the time because unnecessary layering is a variant of infrastructure in search of a user.) But the switchover is an intrusive change and my tree is not clean right now...

Speaking of all this environment variable fiddling, using the getenv/setenv stuff I've already got is actually wrong here, because it searches through the list each time. So it would search through the list doing lots of strncmp to find if it's an external variable, then search _again_ to set it if it is, otherwise search through the local list. Given how hot path variable resolution is, I kinda want that optimized...


January 1, 2020

Twenty years of copyright breakage unblocks today. Yay!

Tickets to Japan are booked, redeye flight on the 7th. Trying to get toysh to a good stopping point...


Back to 2019