So at the end of the Dr. Who christmas special, Capaldi lectures himself for several minutes mansplaining how to be The Doctor, then the regeneration is a big crucifiction cross, then the new female doctor gets to press exactly one button before the Tardis freaks out, opens its doors, turns sideways, and shakes her out from a great height.

Perhaps I'm reading too much into this.

Yay, Elliott fixed that xargs -0 issue. Merry christmas!

Jeff reminded me of Eben Moglen's excellent writeup on the Linux Foundation, where he explained that it's the same kind of Trade Association as the Tobacco Institute and Microsoft's Software Publisher's Association. It's exists entirely to serve its members, and has no mandate to help anyone else. In fact helping the in-group at the expense of everyone else is what this sort of association generally exists to do, and in this case the in group is for-profit companies and the out-group is hobbyist developers. (I've mentioned this before, but again "I wrote about that once 7 years ago" doesn't mean everybody knows about it. I suppose the marketing "rule of seven" works in here somewhere.)

Moglen was the lawyer who co-authored GPLv2 and GPLv3, and is the half of that team that hasn't gone obviously crazy. He's smart and knowledgeable and his article is well worth reading.

That said, Moglen still thinks copyleft is a good thing. Possibly a little too close to the issue to re-evaluate a changed landscape. Forest for the trees sort of thing.

Fade got home on Wednesday, I've been a bit distracted. :)

I got the pidof/killall plumbing redo checked in, and am trying to come up with a proper test suite entry for it. There are several things to test: executable vs script, name vs path, the path logic distinguishing between different executables running with the same name...

Elliott sent a patch trying to make killall behave like "upstream", which is a can of worms. It turns out the bug report he got was that our killall had a behavior difference vs busybox, so he tested against busybox and assumed that's how ubuntu works. I haven't looked at busybox killall recently but I ran a lot of tests against ubuntu to see what it's doing.

The other fun thing is toybox's killall and pidof use the same logic (but pkill/pgrep don't, they use the ps/top logic). Specifically, Ubuntu's pidof treats relative and absolute paths differently (it matches "dir/name" against a command's argv[0] but "/dir/name" against the inode) and killall treats any path as "check the inode", so "pidof ../dir/name; killall ../dir/name" can have pidof find nothing and killall find something.

Ubuntu's killall doesn't distinguish between different paths to the same inode, which means hardlinks get killed together. The pidof behavior of grouping hardlinks together for absolute paths but not relative paths is crazy and inconsistent and I'm not doing it.

I got to introduce somebody to the cp -rs make trick today, something I first optimized _away_ almost ten years ago.

Something that I need to internalize is that successful speakers and educators and such repeat the same darn thing a zillion times, because their audience won't have heard it. I've watched successful speakers give basically the same talk ten times to different audiences. I myself keep thinking "I already talked about that, everybody's sick of it, move on"... but that doesn't seem to be how people work.

The cp -rs trick is where you use cp's symlink or hardlink option to snapshot a tree of source code, then build in the snapshot. This is how Aboriginal Linux's package cache worked.

This has a bunch of advantages: it uses less disk space, snapshots are nearly instantaneous (since you're just copying metadata, not file contents), and if you're doing multiple parallel builds of the same package (for different architectures) you have only one copy of the extracted data in the page cache so you thrash the CPU memory bus a lot less.

It also means that you never really need to implement out-of-tree build infrastructure, you can just cp -s your source tree and build in that. (You can use "cp -l" to make hardlinks instead, but that can't cross filesystem boundaries. You can cp -s from a read-only filesystem (or an NFS mount) to some local build scratch space, build in there, and then rm -rf the symlink tree without disturbing the original. And yes this even works around the NFS weirdness of "rm -rf failed on nfs because some process still had a file open and this created an invisible dot-file that means you can't delete the directory, and that broke the build".)

It's a neat trick, more people should know it. It _can_ break, the old aboriginal FAQ described an old zlib screwup (shipping a generated file and then modifying it in place, the shared copy got modified), and once upon a time bunzip2's "make install" used cp to copy a shipped file under /bin and wound up installing a symlink-to-nothing there. (This is why the "install" command exists. I poked them and they fixed it.) But almost all builds just work (he says having used it to build Linux from Scratch), and you can do a "find" for symlinks on new packages and check for broken ones (somewhere I have a script that did that, some variant of "find -type l | read i; do [ -e "$i" ] && echo "$i"; done", although a find -print0 that you could feed into read would be nice), and it's way less effort debugging that than implementing "make O=".

A report came in via github that xargs -0 is skipping arguments between each line break. Sigh. (Is this a regression or a corner case I never noticed/tested to begin with?)

Throw it on the todo heap...

Got a little cleanup done on stty, although testing changes for regressions is an open question. (I don't have any non-usb serial hardware anymore, and qemu's serial port emulation is just copying bytes to/from a host filehandle; nothing about baud rate, parity or stop bits.)

Cycling back to mkroot, the arm64 defconfig has /dev/vda and eth0 but my miniconfig does not. So, let's run through the isolation protocol again:

What I do is take my existing miniconfig, and append all the new symbols the other .config file enables that aren't already in the miniconfig. Here's a script to do this:

(cat $MINI && egrep -v "^($(sed -n 's/=y//p' $MINI | grep -v '^#' | grep -v '^$' | tr '\n' '|')nope)=." .config ) > config2

That script turns all the NAME=y lines from the miniconfig into a big "^(NAME|NAME|NAME)=' regex, then feeds it to egrep -v to filter out all those lines from the .config, and appends the result to a copy of miniconfig. So config2 starts with a verbatim copy of the miniconfig, followed by all the new symbols that _weren't_ in miniconfig.

(Implementation detail: it's really using NAME|NAME|nope because it's easier to append a string I don't expect to find than strip off the trailing | from the tr '\n' '|' replacing the ending newline.)

Since the new file starts out exactly the same as the miniconfig, as long as I don't touch that part my base system functionality should still work like it used to. The new symbols at the end enable some new feature I want, but only a few of those new symbols are relevant, so I comment out a chunk of the new stuff at the end of the file and rebuild, and if the behavior changed one of those symbols I commented out was important. Rinse repeat until I'm down to the miniconfig and whatever new lines switched on the feature I want.

I usually start by commenting out a couple dozen symbols I recognize that probably _aren't_ important, then do a test build to check that the new stuff I want is still there. The balance is "remove lots of trash so I'm not doing lots of test builds" vs "something I didn't expect turned out to be important and I have no idea which symbol it was because I just switched off buckets of them". There's a tradeoff between doing more builds and backing up a long way if the test fails. Generally I comment a batch out, do a test build, and then delete the ones I commented out before commenting more out, so I can tell where to back up _to_. (They're deleted when I've confirmed they're not needed.)

Eventually I either guess wrong or run out of symbols I recognize. Then it's testing smaller numbers of symbols I'm unsure about, often one at a time, until I find all the symbols that enable the new behavior. (Then I look them up in menuconfig to read the help text and see what they mean, sometimes even digging into the Makefile or C file plumbing that uses them to see what they actually do.)

Still waking up at 1am and being awake until morning, then going to bed around 6pm. This wouldn't be so bad if Peejee didn't climb up into my arms every time I sit down at the computer at home (preventing me from getting any work done), and Austin's once rich array of 24-hour places to program having thinned considerably over the course of "George Dubyah"'s tenure. (I miss Metro.)

I am waiting for my cats to die of old age. A bit like the baby boomers. One of the things I've enjoyed most about trips to tokyo was quiet time alone in a hotel room with no cats, where I could catch up on toybox. (Didn't manage it this most recent trip because there was never a time when we weren't working on crisis du jour; gps debugging, helping Jeff prepare for investor meetings, etc. No time/energy to spare for toybox. Maybe it'd have been different if Jen had gotten on any of the half-dozen flights she promised to be on and then wasn't with a new excuse each time (a pattern apparently stretching back months), but she didn't so I got sucked into shouldering her load.)

It snowed outside this morning. Austin only does this about every 5 years. (It's like a quarter inch and only sticking on half the ground, melting on the other bits. But still quite pretty, and we had to drag Fuzzy's plants inside in rather a hurry.)

One of the recruiters is waving a telecommuting position at me, up in Colorado. Highly tempted. I haven't been to Colorado since I visited Kirsten at her college in something like 2002.

Got email from dreamhost the other day about processes getting killed due to excess resource usage, which read like an attempt to upsell me on some sort of cloud service. I went "huh", and today I got back clarification that it's some site spidering the old mercurial repositories, and launching enough python instances to peg the CPU long enough for dreamhost's monitor thing to start killing them.

Long as the site's not hacked and the static pages load within 15 seconds or so, it's not ideal but eh. Other fish to fry.

Mmmm, fish. I could cook salmon for dinner.

The downside of not following ttwwiitteerr is I bump into mention of "the problems with Patreon" in a couple different places and have no idea what anybody's talking about. Google news is useless, and google for "patreon problems" brings up a dozen different issues going back to 2014 or so on the first page. (Adding "december 2017" doesn't help.) So I wind up digging through the twitter feed of one of the mentioners to find a link to the issue. (Basically patreon is pretending that they charge your credit card individually for every creator you support instead of once a month, and adding a 40 cent fee per pledge, meaning $1 pledges to 30 people are no longer a thing. Since they still DO charge your card just once per month, they pocket these new fees and presumably use them to persecute women.)

And yes, even with 280 the proper description is a link to a screenshot. This 280 thing was stupid and useless and should not have happened and I'm treating it like I did Horrible Retweets which means I miss twitter but do not read ttwwiitteerr.

I haven't got a proper replacement for this yet. Livejournal _used_ to provide my Random Link Farm, and before that slashdot. Tumblr is closest now but the signal to noise ratio is at least as high as twitter ever was with posts that take up to 30 seconds to scroll past (once you've posted 3 pictures of the same staircase, I'm not interested in the other 30) instead of 3 seconds each.

I should set up an RSS reader. Google Reader famously joined the Google Graveyard in 2013, but I still provide a feed here and Fade uses one for her webcomics and such. (I just type webcomics in from memory, which is the modern equivalent of remembering phone numbers instead of having speed dial.)

Airplane back to Austin.

We are so not getting everything done this trip. Not even close.

Emailed the help address at the bottom of my old kernel doc directory to point out they're not linking to the Documentation subdirectory (with the copy of the kernel source's Documentation directory) or linking to the lki mirror. I know they're there because I _put_ them there, but I haven't had write access to that directory for years, because if all you have is git, everything looks like a nail. Or some such.

Sigh. The Linux Foundation wants to "be" Linux the way Facebook wants to "be" the internet. They want to control it, and own it, and convince as many people as possible their walled garden is all there is. I'm not a member of either, and feel dirty every time I have to interact with either one.

But I shouldn't blame the kernel.org maintainer for his employer making me sad.

Hanging out at a tokyo coffee shop called ePronto. It's quite nice. They have a big long desk full of outlets, and in _theory_ they have internet here but in practice I haven't been able to navigate the japanese login screen. (I can recognize the Terms of Service page even if I can't read it, but it goes around in circles. I think it wants me to create an account, maybe?)

I have internet back at the hotel, but the cleaners kick you out at 10:30am (they open every door on a floor and clean all the rooms at once), and you can't get back in until around 3. I could go to the office but I don't have keys to get in, and Jeff isn't in today (he was on the phone with US people until 3am; time zones).

I've fallen way behind on toybox again. And it's slow going poking at mkroot on the netbook because each kernel compile takes half an hour and there's a dozen targets to build. (I didn't bring the big machine this time.)

We walked through the scilab code for GPS fixes and confirmed it's doing more or less what Andrew's version is doing (in a MUCH CLEANER way, although part of that is no coordinate conversions because the units of all the data it uses are consistently light-seconds as double precision floating point numbers).

As with all code review, this turned into a tangent from a tangent and we wound up chasing obscure bugs in the hardware side where we worked out that our fixes were becoming inaccurate when we dropped/added satellites because codephase adjustments made to those dropped satellites aren't being properly discarded but instead applied to the wrong satellite, which means the codephase we think the satellite is at and the codephase the hardware is using don't match. (The hardware doesn't let us read the value back _out_, so the driver has to add up the adjustments it asked for.)

The thing about GPS is once you've read a satellite's packets once (and the most interesting bits repeat every 30 seconds), all you care about after that is timing information.

Subframes 1-3 contain constants you plug into orbit calculations, describing this satellite's orbit very precisely. These are updated at most once a week (at midnight sunday UTC, by ground stations transmitting new info to the satellite's onboard computer if solar wind or the planet's magnetic field or something caused the orbit to drift enough they have to fire the thrusters, but usually they don't). So once you've successfully parsed a given satellite's packets, you don't need to do it again for a week.

You use these equations to calculate where the satellite was at a given time, so the X, Y and Z coordinates for the satellite are calculated from the T (time) coordinate. So every time you get a fix on your position, you have "this week's info" about the satellites and 4 new timestamps for each satellite of exactly when you heard each one's signal. You use those 4 timestamps to calculate 4 corresponding sets of X, Y, Z positions, and then plug them all into matrix math to solve 4 equations with 4 unknowns using a liebnitz determinant.

Getting that accurate timestamp is the hard part. The information you have about each satellite clock has 3 parts: seconds, miliseconds, and code phase.

Seconds come from the parsed packet data, although there's 3 parts to it. The first word of each satellite subframe (the "handoff word") contains a timestamp value, so merely by parsing one packet you can set your clock about as accurately as NTP would. The timestamp is basically a monotonically increasing packet number (which wraps sunday at midnight GPS time, which is UTC but not adjusted for leap seconds since 1980), and then there's a GPS week value only goes 0-1023 so it (which rolls over every couple decades and knowing where "zero" is is left as an exercise for the reader, although we have to care because the next one's April 6, 2019.)

Each packet is 6 seconds apart from the previous packets (measured by ATOMIC CLOCKS), so once you've parsed one (from any satellite) you only need to work out where packet edges are, and don't have to be able to _read_ any of the packet data: the next packet will be exactly 6 seconds after the previous packet. If any packet's too garbled to read you can just count bits: the next packet starts 300 bits after the previous packet, and each bit is 20 codephase rollovers (I.E. milliseconds). Since the satellites send the packets all at the same time (arrival's a bit skewed by the distance, something like 70 milliseconds at the speed of light, but again _transmission_ is measured by atomic clocks), you just need a reasonably set PC clock to figure out which group of 6 seconds you're in.

We have to line the codephase up to read the signal at all (it acts like a one time pad XORed with the signal, it's all just noise until you XOR it back, and since half the bits in the codephase are high and half are low and the thermal noise is randomly distributed, when you get it right this cancels out the noise while amplifying the signal), and that alignment tells you _fraction_ of a millisecond (relative to the other satellites).

The codephase changes 1023 times each millisecond, and when the sequence restarts is exactly (via atomic clock) the start of a new milisecond. So you can count the number of times the codephase has rolled over since the last time a new packet started, and that gives you the current milliseond.

We sample the signal 16 times faster than the codephase changes to find the clock edges more accurately, and 1023*16 gives us 1/16368 of a millisecond, which is 61 nanoseconds. Multiply by the speed of light and that works out to a fix granularity of about 18.3 meters.

So the pieces of timing information you assemble are: timestamp in the packet data (week number and time of week), which codephase reset the first bit of a new packet started transmitting on (indicates exact start of a 6-second period), number of codephase resets counted since then (milliseconds), and the current codephase offset (1/16368th of a millisecond). Of course all those times are at some point in the past: you need to work out your exact distance from the satellite and subtract how long it took the signal to get here at the speed of light.

When you request a fix (which can happen at any time, but to make the math easier we do it at local second boundaries), you snapshot the time you have for all 4 satellites (which is a historical reading because nanoseconds tick by as you process it so you're working out where you were at the snapshot time in the past, not where you are now; the important thing is that the sattelite times match each other and correspond to a timestamp out of your local high resolution clock). For each satellite you have the time you saw the last packet edge, the number of correlator turnovers (milliseconds) since then, and the correlator codephase of that satellite (which passes zero at each bit edge, so the codephase gives you the fraction of a millisecond from the last bit edge that satellite is at when the snapshot happens).

Codephase advances each 1/1023 of a millisecond, but we sample the signal 16 times faster to get 1/16368th of a milisecond. The speed of light is around 300 kilometers per second, which is 300 kilometers per milisecond, which is around 293 meters per codephase, or about 18.3 meters per oversampled codephase. So a 4 satellite fix should have a granularity of somewhere around 20 meters, but if you're not moving you can average a bunch of readings together to get more accuracy (this is called a "survey"). If you work out the rate of change of the codephase (first and second derivatives) you can basically use Bresenham's line drawing algorithm to go "our codephase should be _here_ at the snapshot time" and not just have pixel granularity but the remainder. After all, the satellite's moving at a constant rate through a curved orbit that approximates a straight line in any single digit number of seconds.

So, adding it all up: seconds + milliseconds + 1/16368ths of a second (plus predicted fraction of codephase if you're feeling posh) gives you a very exact time for each satellite. Plug that time into the orbit calculations to get equally accurate X, Y, and Z coordinates to go along with that T (using the many-decimal-place constants out of the satellite packet data), then fling matrix math at the results to solve for where _you_ are if you're seeing those satellite signals at the same instant.

The problems we've been having all boil down to timing mistakes and OH there are so many of them, and THIS is why the hamsternz github code doesn't track real world signals accurately for more than a few seconds at a time. For example, codephase drift (due to the satellite's relative velocity towards or away from you changing as it orbits and affecting its doppler shift) can wrap around zero, which moves the satellite's reporting period from one millisecond to the next, meaning the satellite can report twice in the same period or skip a period depending which direction it's going (ascending in the sky is moving towards you but decelerating, descending is moving away and accelerating; when the satellite's directly overhead its doppler offset is zero), meaning that although you NORMALLY observe exactly 1000 codephase wraps per nominal second, when the codephase wraps you can expect 999 or 1001 depending which way it wrapped. You have to make your codephase tracking code notice the wrap and adjust your expectations (999 milliseconds per nominal second is weird but sometimes right), and we had to add code to do that.

Hardware clock skew's another. We're counting signal samples and saying it's 4 signal samples per codephase (well, per 1/16th of a codephase; our clock isn't 16x the codephase it'x 64x the codephase but we need 4 samples for each sine wave cycle so we can spot the ups and downs), but the clock that drives the receiver isn't the clock that drives the correlator which isn't the clock that drives the satellite. And the same "multiply a slow clock up to a fast one" error magification that screwed up doppler happens here too. So we have to correct for skew between THOSE clocks. (The atomic clock in the satellite is really expensive and works in a vacuum, the ones in the board are under a styrofoam pad yet still skew noticeably when the air conditioner turns on or somebody opens a door.)

And since you can't get a crystal that vibrates at a gigahertz (and couldn't accurately measure the vibrations if you could), what you do is get a crystal that vibrates at a few megahertz and feed the signal through a series of phase locked loops to double the signal rate (the new clock has a rising edge on each of the previous clock's rising and falling edges, so it goes twice as fast). But this multiplies any error in your original clock, so the crystal being off by one tick per million means the drived signal from several consecutive phase locked loops is off by a thousand ticks per billion, and they add up.

And any of these clocks can slide past each other and wrap around and sort of introduce moire patterns into our data, which causes more millisecond slips. And these you can't count directly because you can't predict a gust of air screwing with your temperature a fraction of a degree, you just have to spot when it happens and compensate.

Milliseconds turn out to be the hard thing, that's what all the clock slips tend to screw up. Codephase we can detect when we're off and correct for, in fact the phased locked loops do so dynamically (assuming the darn hardware makes the adjustments we request without losing track of where it IS). But if you're off by a millisecond that's 300 kilometers at the speed of light. Either the matrix math will return "not a number" because there's no point where you could have seen all 4 readings at once, or it'll give you a fix way the heck away from where you really are (both in distance and in time).

Which is why our GPS generally either knows RIGHT where we are or thinks we're in tokyo bay.

Why is the readelf bug back? Because the build overnight used 4.11, not 4.14. Why did it do that? Because modules/kernel didn't get updated. Why not? Because I have two vi instances editing the same file. Lovely.

I still have an urge to _post_ to twitter, but very little interest in reading ttwwiitteerr. I should tack those lines onto here, I suppose.

I bought earbuds for 100 yen in akihabra, out of a bucket of them. Surprisingly, they are NOT the worst earbuds I've ever bought, although they're not in the top third, either. Still, at the rate I go through 'em I should go back and buy a dozen.

I'm taking a day off, trying to catch up on toybox stuff, which diverges pretty quickly into trying to catch up on mkroot stuff. (Aboriginal Linux is what drove what I was doing on buildroot, mkroot is driving what I'm doing on toybox until I can get to the point where AOSP does.)

Along the way, I had to download a new kernel because it's still building 4.10 and 4.14 is current, so I should do that upgrade and test everything, and the wget went boing in a way I'd never seen before: it complained https couldn't decrypt one of the blocks. (Whaaa....?)

It responded by reconnecting and resuming the download (wget --continue), but of course I had to extract the equivalent tarball from git (git archive --prefix=linux-4.14/ > file.tar, and zcat the downloaded tarball) and see if they differ (cmp says they don't). Which means I didn't actually have to download it in the first place, but I wanted to make sure the sha1sum matches the downloaded one (did they gzip or gzip -9)...

Since Jeff talked me out of taking the Big Machine with me (being sure that if I was bringing three computers to tokyo, what with the mac and all, I'd be stopped by customs... The thing is, I telecommute and do open source, so wherever I am I'm working. Kinda hard to answer the 'are you coming to tokyo for work' question when it's kinda quantum. To be honest, I'm coming to tokyo to give Jeff moral support while he finishes the GPS software, almost none of it's my code. Some if it's my _design_ ideas, but I've been arguing with him about that over the phone for the past half-year. (I pulled up a file with a half-finished thing I'd last touched in August.) In person is faster, but definitely not _cheaper_ than just spending six hours a day on voip or skype or something. (I generally pull up talky.io, but haven't managed to get it to work on my current phone and netbook. Worked on the phone before the factory reset...)

Where was I... Oh right: ) compiling kernels for all the musl-cross-make targets on my netbook is SLOW, so I have to leave it running overnight to build qemu-bootable versions of all the mkroot targets.

Exhausted. Jeff found a scilab implementation of the gps position calculation algorithm in Appendix B (page 77) of an old college thesis from 1998 (this is the version that can use more than 4 satellites to increase precision) and we're replacing a GPLv3 function we've had as a placeholder (the hamsternz thing had one file copied from Andrew's code, which we obviously couldn't ship, but this thesis is the same math a decade older so either where it came from in the first place or independently worked out) and Jeff's writing a matrix math library because in scilab there are functions like determinant() you just call, which we need C implementations of. This means I need to understand something called Heap's algorithm for permuatation, so we can use it to calculate Liebnitz's determinant. (Which is more accurate, and only really slower when your matrix size is larger than like 7x7).

Or at least that's where I ran out of brain for the evening.

And Pakistan joins the list of countries where homeowners have the right to sell rooftop solar power back into the national grid. Jeff is convinced that if we can't get our synchrophasor tech deployed soon, grids are going to start collapsing. (Me, I expect a couple of Katrina-style disasters and then it's a crisis which people deal with at the last minute. He'd prefer to avoid people's appliances getting destroyed and thousands without power for months, but given Enron-inspired the rolling blackouts in california and the ongoing mess in Puerto Rico, it'll probably take multiple cities going dark repeatedly for anyone to notice...)

Jeff is also writing a new matrix math library, which means he's sending me links to matrix math primers. Because my brain wasn't already full before that.

In theory both the fixed point library he made and the new matrix math library should be open sourced. We're just too busy trying to get everything to _work_ right now to do the releases yet. (I suppose we could throw 'em up on github, but without a web page and a mailing list it's not a real project. Ok, my mkroot project is using github's readme as its web page, but it has a mailing list!)

After I get toybox+musl+llvm+lld+linux not just rebuilding itself under itself but building Linux From Scratch under the result (as last time did for busybox+uclibc+gcc), my next goal is breaking AOSP into orthogonal stages maybe with kconfig.

That's a base layer I've prototyped at mkroot, and other obvious targets are minijail's android container as used in chromeos, the android init/jvm/selinux complex. Plus a "hermetic build" host airlock with toybox+ninja+toolchain and such (possibly hijacking the NDK for parts)...

Alas nobody else has been interested, and $DAYJOB's eaten every spare second for ~18 months now doing things like GPS software. (It's a startup. They do that.)

*shrug* This _is_ fixable, and Google is not opposed to any of it so far. But nobody ever wants to fund cleanup, and I've grown tired of giving "here's what I _want_ to do" talks at conferences. :)

A Federal Aviation Administration official just emailed me to ask if I want to work as a linux kernel developer on realtime radar data processing for an Air Traffic Control automation system. In other news, the FAA is automating air traffic control. (Good to know? Drones I guess. Hail Skynet.)

Today was the long meeting with investors about potentially funding j-core work going forward. Very exciting, I'm interested in most of the things they want to do with the architecture, things I've been agitating for us to do for _years_... but they haven't signed a check yet.

Alas, in business it's not real until the check clears. They were going to do a small Statement of Work in the run up to this meeting, while trying to get the real funding underway to get us started down their path (very aggressive schedule they want towards their product), but that SoW didn't happen, and there's no deadline by which it's expected to happen. I.E. Maybe Money, Jam Tomorrow. It's _exciting_ Jam Tomorrow, but it's not Jam Today.

Sigh. Jen was supposed to be here and do this. She was supposed to be on a plane from canada to tokyo the same time as me, but didn't get on the plane. she's since scheduled SIX flights that she wasn't on. Something is up there, but life goes on. And thus I got unexpectedly drafted to do her job.

Me, I'm out of my depth here: the lead investor's a kernel developer I vaguely know from way back, so I was happy to say hi, but I'm neither a business guy nor management. I can tell them how great our open source stuff is, but they know that (it's open, the video of our ELC talk is what caught their attention). I enthused at them a bit about synchrophasors, but Jeff's the one who taught _me_ all that. I only got sucked into these meetings because nobody else was available. (Well, Niishi-san and Arakawa-san came over from the other building for an hour or so, but they don't speak english very well and get really flustered when asked questions in it.)

The Python 2->3 conversion continues to suck. And it turns out Python had a 1->2 conversion that sucked almost as much! Backstory time:

So I'm looking through old mailing list postings from the late 90's, trying to figure out if my old blueberry.py script is the reason the kernel developers changed the default kernel output. ("Don't ask questions, post errors.") My quick hack was covered by lwn.net at the time and the commit that added it to the kernel was a few months later, but that commit comment mentioned the "dancing makefiles" kbuild rewrite as its inspiration, and I wondered if that fork copied the idea from blueberry.py between my January 10 post to lkml and the June 5 merge of a different implementation? (Or did they come up with it independently, because it seemed to _me_ really obvious it needed to happen but lots of "really obvious needs to happen' stuff doesn't for years until I break down and do such a terrible half-assed incompetent impelementation that people go to _great_effort_ to nip it in the bud and prevent its widespread adoption.)

This dancing makefiles thing doesn't seem to have been in source control this was back before git, which meant the open source state of the art was CVS (or an explicit CVS derivative like subversion), and most people just didn't bother). It was so long ago its mailing list was on sourceforge. Which has a horrible, horrible archive interface that makes stuff really tedious to find and read, and you basically have to read every message.

While slogging through that, one of the big repeated objections to Eric Raymond's CML2 (the "make menuconfig" rewrite submitted at the same time as the dancing-makefiles build plumbing redo) was CML2 required Python 2. Not that it required Python, but specifically Python _2_.

(This was back before Eric went obviously crazy and started spouting climate change denialism, thinking the book The Bell Curve made good points, that women programmers were trying to "honeypot" Linus Torvalds, and whatever crazy he's gotten into _since_ 2010. I'm told he's worse now, but haven't been paying attention.)

The point is, about 15 years ago when the existing kernel config system was hitting painful limits and a replacement was proposed written in python, there was a painful "Python 1.0 -> Python 2.0" transition going on in the python community, which seems largely lost to history at this point.

Meaning if Python 3 ever does manage to kill off Python 2, we can expect Python 4 to happen and be _just_as_painful_ as soon as the last switch goes down the memory hole and they forget why not to do that.

Meanwhile, the C 2011 standard was a non-event (nothing broke, all the old C99 code still worked, you could ignore it as long as you liked), and C99 offered candy (_finally_ you could explicitly specify 8, 16, 32, or 64 bit integer sizes, why was that so hard?) but you could still compile your ANSI C 89 programs just fine and ignore it as long as you liked, and of course ANSI C grandfathered in K&R syntax. C hasn't even quite gotten to the point where K&R syntax from thye 1970's generates _warnings_ by default (although there's rumblings, and turning on the "warn about functions without prototypes" flag is generally considered a good idea because the 32->64 bit transition around 2005 assumed you had function prototypes rather than expanding the default type promotion of arguments from 32 to 64 bits everywhere; I still think that last part was probably a mistake, but it did save stack space so I can see their point...)

Anyway: Python 3 sucks, turns out python 2 previously sucked but wasn't a _pattern_ of sucking yet, and I'm not waiting for Python 4. Ten years of forcing something the userbase does not _want_ down the userbase's throat is not how open source development should work.

Sigh. I'm helping Jeff prepare for an investor meeting (which should be Jen's job but she's still in Canada for an ever-changing list of reasons, in _theory_ I'm doing the same stuff here I was in Austin just hanging out with Jeff in coffee shops rather than on long phone calls with him), and he keeps trying to tell me about Risc-V (as a "what's the competition doing" thing), and... I cannot BRING myself to care.

I've reached the point where I don't care about Risc-V the same way I don't have a Facebook account or run a Windows system. It may be wildly succesful for a long time (I still have my doubts), but I'm not onboard and can wait for it to go away. (The sun only has so much hydrogen.)

Part of it's that I followed Transmeta, I followed OpenMoko, I waited for OS/2 and Linux on the Desktop to crack 2% market share. I don't see how their strategy of attacking the high end makes them all that much different from everybody ELSE who attacked the high end, from Sparc to the DEC Alpha. There was OpenSparc and Leon Sparc, there was powerpc.org. The Steam guys decided not to do their own game console because the unit volume wasn't there, and cost in this space is almost entirely a question of unit volume. Your design being free but your manufacturing being 1/5 the volume is optimizing for the wrong thing.

And part of it is the incessant hype. I saw a powerpoint where they created several different instruction sets each to win a different benchmark, while repeatedly touting how simple they are. They've taped out a dozen different times and are looking for their first actual user. That's "infrastructure in search of a user", clearly a bad thing in other contexts.

C and Unix were created by people who were using them as tools _while_ creating them, and their first big deployment was a contract with AT&T's patent and trademark office to build a typesetting system which baked the "everything is text" philosophy into the OS. Their development was guided by real use and an immediate testing feedback cycle driving the design. Meanwhile Pascal came out of academia as a "teaching language" that never bothered to standardize how you specify the filename when you open a file because it just didn't come _up_. Sure it was taught to a generation of students as their starting language, but its uptake beyond that? Turbo Pascal and Delphi have sort of wandered off. C is declared dead annually, the same way the Death of the Internet is regularly predicted.

J-core started by trying to improve upon a 20 year old processor design that iterated several times (we're not doing the first version, we're doing the second version incorporating end-user feedback and backporting bits of the third version), and we've got a specific niche (synchrophasors) we're designing products for already incorporating customer feedback from customers in three different countries. I understand the path to success here.

The Risc-V guys meanwhile are going "inevitability, pedigree, and funding", just like Itanium did 20 years ago. "I don't care if you want this, you have no choice." I'm sorry, I went into the booth, held my nose, and voted for that campaign slogan last year. I wasn't exactly excited about it at the time, am now kinda burned out on the concept. And the "get off your butt and vote _against_ if you can't vote for" aspect isn't there: arm64 has a stupid name but isn't torturing puppies.

Meh, I'm too close to this to have anything like an unbiased opinion, but if I wasn't doing J-core I still wouldn't be doing Risc-V. It's just not interesting to me. "I'm going to make an X that will displace Y" is an old, old story I've written about before. Intel couldn't do it with the Itanium, OpenOffice has spent 20 years trying to replace Word... Woody Allen said 80% of success is showing up but the other 20% can be kind of important sometimes. And the inevitability argument has always rubbed me the wrong way even when they pull it off.

(For the record, Mrs. Clinton lost my support in 2014 when she came out against Snowden in a radio interview I was listening to. That's a showstopper for me. I still flew back from tokyo early and pulled the lever on her behalf anyway, but that was a vote against not for, but I'm not surprised so many other people didn't. I flew back because I did _not_ take her election for granted, and didn't want to spent the next four years feeling guilt over not having done my part if the bad thing happened. These clowns implying it is my Duty To Open Source to support their chip? No. No it is not.)

Hah. A recent discussion brought up an old linuxtoday comment which was a story.

[And that's the whole of the entry I left myself for that day. The obvious todo being "tell that story". I was referring to the "back when I worked at Boxxtech I convinced my employer to convince their supplier to create the first SMP Athlon motherboard", but between last entry and this one I've been blocked on editing and uploading these for a bit, and I'm tempted to just move on. The comment tells the basics, but... ok, new paragraph or two:]

Boxx saved Tyan's policial bacon by buying a bunch of rambus motherboards, which Tyan had done the engineering work to design but nobody wanted to _buy_ because of toxic patent troll shenanigans on the part of rambus inserting their technology into the DDR DRAM specification they were on the committee of, and then demanding patent licenses _after_ the standard was approved, in a blatant attempt to hobble their main competition. Everybody else went "ew" and refused to touch rambus' main product. But Boxx ordered just enough Tyan rambus boards to justify doing a small production run so Tyan _didn't_ have to write off the rambus design work as a loss but could instead categorize it as a legitimate R&D expense that resulted in a product sale, which made Tyan's investor numebers look much better. And thus meant Tyan owed Boxx a big favor.

So I convinced the manager in charge of speccing future products that Athlons were kicking Intel's ass while Pentium 3 was stuck at 1ghz (they had to recall the slightly faster one because they'd overclocked their own chip and it was unstable [maybe _this_ was the story I wanted to tell last post?]) and Pentium 4's quest for clock speed _over_ performance (because marketing!) meant they had such an insanely long pipeline that every time they had a cache stall or mispredicted a jump they had to wait 40 clock cycles for the bubbles to work through the pipeline stages and that sucker was terrible... And we all knew Itanium wasn't ready for prime time, that didn't take convincing.

The showstopper was Boxx needed SMP and Athlon "didn't do SMP", but I pointed out (with URLs to contemporary tom's hardare articles) that Athlon was designed to fit in the DEC Alpha's EV6 bus and the only reason you couldn't just stick Athlons in an Alpha motherboard was the boot rom had the wrong type of assembly in it, and swapping out the ROM chips couldn't be that hard. Athlon _was_ designed to do SMP and the only thing preventing it in its first few years was lack of motherboards.

I convinced the guy that this was the way to go (I'd already researched the issue pretty thoroughly for Motley Fool articles I was writing, and could linkbomb him with All The Sources), and he put together a presentation for senior management, and they called in that big favor Tyan owed them due to ordering the Rambus boards, and comissioned a new board that became the Tyan Thunder (which is why boxxtech was mentioned on the press release announcing the board, which was the link from my comment in the old linuxtoday thing that started us off here).

(Yes, I wrote motley fool articles for 3 years on topics like this. They lost my author entry in a database migration so my articles just say "by Motley Fool Staff" now, but some of them mention me by name in the summary the editors typed in or similar, and some even have duplicate bylines with old (link to a 404 page) and new (wrong) info. And a few of the articles that come up on a google search of fool.com for my name aren't ones I wrote. Altogether I wrote almost two hundred articles, I keep meaning to index and miror them properly, I have a couple series pulled out at the end of the writing page, but... todo list. [Seriously. Todo list.])

So in 2001 boxx went from SMP Pentium III to SMP Athlon (the "Boxxtech ServerBOXX R1"). And that's also why the first AMD SMP system ran Linux. (Boxx was an Irix shop that moved to Linux about when it hired me, senior management hated Linux and my boss told me to my face it was "communist" but Silicon Graphics ended Irix development to become a Linux shop so Boxx didn't have much choice.) There's more on that motherboard and its status as the first SMP Athlon in this linuxjournal article

And of course once there was _one_ SMP AMD board, other companies came out with competing products that weren't so high-end and complicated. Tyan got a proper SMP support chipset out of AMD (which was happy to do it, I don't remember if they'd already announced it and there were no board vendors willing to touch it without a proven market for the thing, or if Tyan comissioned it from them. I left Boxxtech long before this new hardware shipped. Being head of Linux development in a company where senior management despises Linux was not fun. The dot-com bust hit not long afterwards and within a year they'd laid off everybody and the senior management that hated Linux was on the factory floor turning screws to keep the company going. They managed, still around today as far as I know.) And once they'd started, Tyan itself went on to do a lot of SMP AMD motherboards. There _was_ demand, and everybody followed the money.

People were waiting for this product to "inevitably exist" for years but nobody wanted to be first, and the reason it finally _did_ happen is I convinced the right people it should happen, leveraging political backroom deals and investment journalism to give the technical arguments weight.

*shrug* A fun little historical footnote, more or less lost to history now. Thing I'm proud of nobody else even remembers, another instance of me setting off an avalanche in the Linux world, although that one went in the direction I'd hoped.

So if you're wondering why I'm trying to turn Android into a self-hosting development environment... it's a thing that should exist/happen and nobody ELSE seems to be doing it, so... grab the snowshoes and dog with the cask and head up into the mountains and start shoveling.

[And writing that up ate over an hour. You wonder why I'm so far behind editing and posting these? Yes, I just wrote all that text now, in April of the following year, even though it isn't in square brackets. This is meta enough as it is, moving on...]

Aha! I _did_ write down that processor rant. Well, bits of it anyway. (This is the one I couldn't find last month.)

There was a Motley Fool part too, let's see... (dig dig...)

[Editorial note: as I come to edit this so I can post it months later [I write this on April 15], I don't remember _which_ story I was referring to. How the Israeli design team took the India design team's head while queen played and fireworks went off [it's a highlander reference] and thus Pentium M unseated Pentium 4? How when Compaq bought the corpse of DEC in 1996 they didn't do chip design so AMD snapped up the Alpha chip designers and went "if you were to make an x86 chip what would it look like" and thus got the Athlon and Opteron and made Intel play catch up for 10 years, something Intel is very very bad at? Since the next few paragraphs are "oh look, I wrote about _this_ too" I'm not entirely sure what topic I was aiming for. Oh well, back to the text I wrote before...]

Heh, these two old point/counterpoint pieces (where somebody would write a "bull" argument on a stock and the other would write a "bear" argument, each side got 2 passes at it) turned out to predict wrong because AMD had a huge pile of debt that hamstrung its moment in the spotlight (all its success went to pay down student loans), because Intel finally broke down and accepted x86-64 into its life (Dell gave it an ultimatum in december 2004 ala "we _will_ ship x86-64 next year and if we can't get them from you we'll get them from AMD"), and because Pentium-M saved its bacon [as described in last paragraph's rant... this entry got kinda meta during editing].

However, the second of those had a list of articles on the topic, starting with the three article series intro to microprocessor fabrication I did way back when, and followed by the series I was thinking of (intel processor evolution from 386 to itanium). [Yes, but which aspect of it?] I've added mirror copies to my local index there. [Did I remember to do that?]

As for the others, the x86-64 launch party and a follow-up on Merced vs Hammer (I.E. Itanium vs x86-64) still stand up as basic technology comparison. And another follow up did describe the problems Intel was suffering from until the Pentium-M resolved the issue. (I wrote stock market _analysis_ columns, not prediction. Intel was in trouble 17 years ago, it just got over it 12 years ago.)

[And it stops there. I clearly remember working my way _to_ a point, but have no idea what it was. Ordinarily I go through and _resolve_ comments I leave myself in square brackets, but I'm just gonna add more, put it up, and move on.]

When did the phrase "Conservative" replace the phrase "Old Fogey"?

A century and change back John Stewart Mill famously said he didn't think conservatives were stupid, just that stupid people are generally conservative. But add a layer of plutocrats and con artists milking the stupid people and you describe modern conservatism pretty thoroughly.

They spent fifty years screaming "better dead than red", and now they're the red states. McCarthyism was all about the looming spectre of communism, pinko meant a little _tinge_ of red was all-the-way bad, there could be no compromise. And now they're rallying behind an obvious Russian puppet who defies congress _not_ to sanction Russia, which still hasn't given Crimea back. (Wasn't the _second_ iraq war still at least nominally punishment for invading Kuwait almost thirty years ago now?)

There are no principles here. Just tribalism.

I've been having an email conversation with Linus Torvalds (this is somewhat unusual for me) about xargs.

Way back when the kernel limited "environment space" to 131072 bytes, I.E. 32 4k pages. The full prototype of a C program's main() actually has 3 arguments:

int main(int argc, char *argv, char *envp)

Usually people just grab the first two, but the _start code (in crti.o, try compiling hello world with the -v option to see the implicit command line options) copies envp into the global variable char **environ; (there's a man page on it, "man 7 environ"). So all 3 get used, even if you write the function prototype to grab the first 2 arguments and ignore the third.

The argv[] and envp[] arguments are basically the same sort of thing, there's an array of pointers with a NULL entry at the end, and a bunch of string data. Each environment variable is a null terminated "keyword=value" string where the first = indicates the end of the name. The arrays and the string data for both arguments and environment variables have to fit in the space limit.

Back when I first implemented xargs, I worked out the limit and very carefully measured the available space... but then a later commit changed the limit in 2.6.22, so each individual entry was 131072 (I.E. 131071 plus null terminator) and you could have 2 billion entries per process.

Well now it's changed again, arbitrarily capped it at 10 megabytes per process. And I don't really care what the limit is, I'm mad it KEEPS CHANGING and there's no way to probe it. (The posix APIs for asking this just return constants hardwired into libc which are different in glibc, musl, and bionic, AND there's just one size API that doesn't tell you size of entry vs cumulative size of all entries.) And if you make it too big, your exec fails. But if you get it too small, there's stuff you can run from the command line as one command that gets split into multiple commands through xargs, which is an unexpected behavior difference that's gonna hit somebody.

Hence the thread. I dunno what the right thing to do here is. Nobody does, so I need a design recommendation from the people who keep breaking changing it.

Joe Ressington emailed me a link to the latest SFC craziness, and I'm so glad not to be involved anymore.

My impression (from a distance, having left before it all went down) was the split between SFLC and conservancy was basically Eben and Bradley not getting along because Eben had moved on from the FSF's shenanigans (hence founding the SFLC instead of staying to run FSF's legal arm) and Bradley remained a true believer (following Eben because sensei, but still wanting to steer the new thing towards militant lunacy; hence the SFLC legal settlements involving appointing a compliance officer who reported quarterly to the FSF despite the FSF having NOTHING TO DO with the enforcement effort).

They managed to stay amicably divorced and not fight for a longish time after that by carefully ignoring each other. That seems to have broken down.

I keep saying "dying business models explode into a cloud of intellectual property litigation" (because it's true, as is the corollary to Moore's Law where 50% of what you know is obsolete every 18 months and the great thing about Unix being it's mostly the same 50% cycling out over and over leaving C99 and posix-2008 still relevant decades later).

But part of what I mean by that is suing people over IP law is a sign of weakness. Like drowning swimmers climbing on top of the ones that can still keep their heads above water, all the splashing and noise signifying exhaustion. It's not a defensive weapon if you initiate an attack, and throwing the first punch means you couldn't cope. Russia didn't invade crimea (or Georgia back in 2008) because they're in good shape: the soviet union collapsed because it couldn't _feed_ itself and that was before a full decade of hysteresis. Picking fights with their neighbors distracts from the fact Russia's pathetic modern GDP puts it somewhere between Italy and Mexico, and that's _with_ all the oil they can pump allowing them buy food from abroad. Their current attempts to weaponize internet trolling are trying to drag everyone else down to their level.

So when I see the FSF lawyering up (sucker-punching Metis, reopening a war with linksys/cisco 5 years after OpenWRT happened, trying to hijack my busybox license enforcement experiment ten years ago into a permanent crusade) I don't see strength. I see weakness. I see an organization that lost relevance back in the dot-com days trying desperately to drag the spotlight back onto itself. It makes me sad.

I should do a write up on the egcs and glibc forks, grub and the gold linker getting crushed and mangled, the ftp site crack after 5 years of stagnation (including debian squashing gentoo), the lwn articles on multiple projects leaving gnu... Meanwhile the Linux world doesn't need any of this crap to reverse engineer hardware (forcedeth etc), run on the most locked-down game console hardware (xbox and the ccc video on the ps3)... I had notes on this before my "rise and fall of copyleft" talk years ago, but I think it was in the 2/3 of the material I didn't have time to cover. :)

Tried to do a video chat with Fade. Both talky.io and google hangouts refuse to produce audio output on my phone. (All four types of volume are up, there's microphone permissions but no speaker permissions. The shop game is producing audio just fine when I run it.) My netbook produces audio just fine but hasn't got a microphone (the lapel mic's back in austin). If I have them on at the same time they do the feedback whistle.

I replied to an email from Linus and attempted to cc: the bionic maintainer, but I used @gmail instead of @google so it bounced. Sigh.

I haven't even left the hotel yet this morning.

November 2, 2017

October 31, 2017

October 29, 2017

Prepping for Japan trip. Soup and/or Shuttle arrives at 4:20 am for my 7:40am flight (because security theatre is ever vigilant against getting their budget cut) so I'm pretty much staying up packing and trying to get stuff done.

October 26, 2017

So Uncle Ben gave Peter the speech about great responsibility, but did _not_ leave him the rice business. Just checking.

October 25, 2017

October 24, 2017

October 21, 2017

October 22, 2017

October 21, 2017

October 20, 2017

October 19, 2017

October 17, 2017

I was raised by cats, but I'm waiting for my current cats to reach the end of their natural lifespans, and not getting more. I've been getting increasingly allergic to them for years now, and it's getting pronounced. Plus Peejee follows me around and climbs up into my lap/arms/shoulder when I'm sitting at the computer (or tries to block the heating vent with her tail over the keyboard), and when she doesn't George bites me. Zabina's been tromping through the poison ivy along the back fence an then coming in to be rub against me. At night Peejee curls up against my face to try to make sure I Breathe Through The Cat. (George just bites my face.)

I wanted children. I got cats. (Well _I_ couldn't have children. That whole "boy" thing. Fade got a dog and another cat on top of previous cats.) At 45 I'm coming to accept that pets are not acting as training wheels for kids, and don't really want more of them.

I type this having left the house with netbook to get away from cats. This means I'm going out to places that require a food/beverage purchase to secure a seat at a table (increasing my calorie intake, especially since "cheap" and "healthy" are so often traded off), and that I'm competing with homeless people for public spaces that let you cheaply sit around for hours in someone else's air conditioning.

Cats.

October 16, 2017

Working on presentation materials with Jeff (via email) for potential j-core investors. Simple stuff, "What's a computer", "What we made (asset inventory, tangible and intangible)", "Market context"... that sort of thing.

Jeff and I have vaguely planned to do podcasts for years, but are both way too busy. One one of my early trips to Japan he bought a tripod, about the way I bought a lapel mic. Neither has made podcasts manifest themselves yet, but we live in hope...

October 15, 2017

In dallas I got a "driving energy" chocolate square (10 grams of chocolate is 50 calories, still well under 200 for the day) that claims to have 150 miligrams of caffeine.

It's... quite effective. And tastes fine, which is an accomplishment given how bitter that much caffeine is. Decent calorie to caffeine ratio. And it was 99 cents. Possibly I should try mail ordering these. (Hmmm, it comes in civilian and professional versions. I had the latter, which has twice as much caffeine...)

There's a warning on the label. Never have more than two of these per day unless you are a thirty ton mega-elephant with bronchial pneumonia. Understood.

Fired up their "store locator" on the website to see if I could buy more individually, and the closest location to Austin is...the store I got it at back in dallas. Ok then.

Got in really late, but the chocolate square kept me up another hour at least. Gave up on fasting and put a beef heart from the chest freezer into the slow cooker for morning, going "maybe I can atkins instead"... and then filled up the extra space with potatoes because potatoes. Yeah, that's usually about how it goes.

October 14, 2017

Driving back to Austin. Picked up my car from my sister last night.

Doing the fasting thing again. My version of which involves eating things that have basically no calories in them (today I had two McDonalds side salads with vinagrette dressing, and a lot of diet Dr. Pepper. Total calories a hundred and something for the day). After the first day I mostly stop being hungry. The problem is being around people who are eating, I'm ok doing this alone, but the longest fast I've done was pretty much the duration of Fade's vacation. She's up in Minneapolis but Fuzzy's home, which means a fridge and pantry full of food. Oh well, something to do on the trip, anyway.

October 13, 2017

October 12, 2017

October 10, 2017

October 8, 2017

Got to Minneapolis in the early evening, drove to my sister's place and had her drive me to Fade's dorm, and then leave with the car. (Her oil light's been stuck on forever, and she used to check it regularly and as soon as she got out of the habit the car _did_ run out of oil and the engine siezed. On the highway, at speed. I so look forward to self-driving electric vehicle subscription services which would have multiple ways of this not being the driver's problem.

October 6, 2017

Phone call from my brother, saying my sister's car died. Oddly enough I'm driving up to visit Fade this weekend, so as long as I get there by monday morning I can give her a ride to work.

A recent tweet reminded me that the US needs to find a new way to fund reporting now that public airwaves no longer come with a public service obligation.

The Communications Act of 1934 required broadcasters to act in the "public interest" in exchange for access to a shared public resource, I.E. the finite broadcast spectrum. This translated to concrete requirements that a certain number of broadcast hours be devoted to nonprofit uses, from news to educational programming.

That's why broadcasters from Edward R. Murrow to Walter Cronkite were special. There was an unassailable wall between "editoral" and "advertising" in each station's nightly news broadcasts: if it wasn't sufficiently objective and "in the public interest" (according to the regulators monitoring the broadcasts), it wouldn't burn off the federal requirement for public service, and they'd have to sacrifice more hours showing things with mediocre average ratings which advertisers didn't particularly want to fund. News NOT being sufficiently factual, objective, and rigidly sourced could cost the station big money.

Then cable TV happened, which wasn't a finite shared resource with a public service requirement as a cost of entry. And one of Richard Nixon's speechwriters started a propaganda network that _pretended_ to put out the same kind of objective news programs the broadcasters did but wasn't even close, and people bought it because 50 years of broadcast regulation had taught them that news in the USA was scrupulously objective and fact-based and would issue a correction if it got anything wrong, and Roger Ailes' "Fox News" leveraged that perception to blatantly lie to people and get away with it. (And as high-fidelity FM radio eliminated demand for crackly AM radio, leaving consistently empty spectrum, AM radio also got exempted from public interest requirements and became infested with conservative talk radio, which also pretended to still be held to a standard no longer enforced against it.)

Then the internet started seriously eroding the advertising revenue of both broadcast news and newspapers (which had their own problems with "yellow journalism" a century earlier but in the face of competition from television news had consistently belonged to a fact-based ecosystem for all of living memory, outside of the supermarket tabloids), and as a result could no longer afford worldwide research bureaus and reporters spending 6 months on assignment tracking down a story. As budgets collapsed the cheap thing to do was present both sides' arguments and imply the truth was the midpoint between them, which was trivial to game with a combination of triangulation and moving the overton window, and the heirs of Roger Ailes went to town. This made uneven progress until the GOP's ongoing atrophy (Rockefeller had warned them that the southern strategy would have that outcome) made the party too weak to keep control of its toys, and con artists like Trump and state actors like Vladimir Putin stepped in and led away the carefully groomed masse of gullible rubes fed 30 years of propaganda.

At this point, I'm just waiting for the mass-senility of the baby boomers to work its way through and hoping there's something left to salvage from the wreckage afterwards.

Sigh. Nobody understands User Mode Linux anymore. (Look, a website still on sourceforge touting a book that came out in 2006! Still the first google hit for "User Mode Linux".)

UML was never ported to most of the architectures Linux runs on, even though in _theory_ it should just care about libc and some memory mapping tricks. Sure it's been overshadowed by qemu, but if you want to stick a printf in the kernel and understand how system call dispatching works, UML is a pretty darn fun toy. (And it's an intermediate between containers and virtualization, given how much effort both of those have absorbed it seems like it would have a niche if it wasn't perceived as abandoned.)

Every time people keep talking about the depth of the talent pool in open source, I keep thinking about ten year old ideas I've had that nobody else has beaten me to. They're generally things that I'm not particularly good at but which I'd try to stick a crowbar in if I had the time. It's probably not _hard_, I just have too many open cans of worms already...

The greying of linux probably works in here somewhere.

Ooh, here's a nice video: a Morgan Stanley Analyst giving a presentation about app-summonable self driving cars. This is what passes for baseline industry knowledge among the financial and industry people, it's why the car companies are so desperately switching over before all their competitors do. (Transportation As A Service needs maybe 10% as many cars as we have now, that's 10% the manufacturing volume, that's 10% as many car companies. Those that want to survive want to get in on the new market and be providing not just the hardware but the _service_. It's a big game of musical chairs with everybody scrambling for a seat as they're taken away.)

That's why the auto industry is crawling over itself to switch to electric as soon as possible. (And look, another riding drone.)

Tony Seba's stanford report says fossil fuel cars stop selling circa 2024 due to collapse of that ecosystem: finding fuel, spare parts, mechanics. The result is a stranded asset. Seba's analysis predicts a cost curve that makes electric vehicles cheaper than fossil fuel vehicles in all categories, so that by 2025 all new cars, vans, trucks, busses, and tractors will be electric. (Except a residual hobbyist market about like horse riding is today. So you'll still be able to watch the Indy 500 just like the Kentucky Derby.) The breakeven point where gasoline and electric are roughly equivalent is 200 mile range and $30k sticker price, and electric vehicle improvement won't stop there while fossil fuel cars are a mature century-old technology without obvious room for rapid improvement.

(Here's another stanford professor who thinks adding lead to solar manufacturing would improve matters. Anybody else spot a downside?)

One reason electric car range can increase is terrible gasoline efficiency: less than 20% of the energy released is harnessed as thrust, the vast majority is waste heat. So even with gasoline's 2 to 1 weight efficiency (due to reacting with atmospheric oxygen, so you only carry half your reaction mass with you), electric vehicles can reach range equivalence at 1/5 the power storage, double it at 2/5. Plus electric cars can devote more weight to batteries since they don't have transmissions, radiators, engine blocks... (Electric cars typically have front _and_ back trunks, it's empty space where a gasoline engine would be. If you really care about range, you can fill it with batteries instead. Some experimental models have already demonstrated a 100,000 kilometer range, but that isn't currently cost-effective and takes too long to recharge.)

Can we really do no better than 20% efficiency? Sure we can! Mixing water with gasoline plus detergent is an old trick, widely known for decades: the heat turns the water to steam converting more of the energy into thrust. Of course gasoline and water don't WANT to mix, so you need a detergent or an ultrasonic vaporizor (called a "bubbler" after a 1960's implementation of it one of my professors told me about in college). In the 90's caterpillar got a patent on basically using powdered sponges to get 80% water and 20% fuel mixed together to run their construction equipment, which worked just like pure fuel but gave you mostly thrust and little waste heat. But the oil industry fought hard against any way to sell _less_ fuel, and preventing your engine from rusting internally or the immiscible liquids seperating back out and stalling out required more R&D than individuals could usually pull off reliably themselves, so the technique was easy to discredit.

Another excellent video is The state of the US Energy Transition (Chris Nelder), The US energy status is: No new coal ever, wind overtook natural gas for new installs in 2015, and solar overtook wind in 2016. Near the end of his talk he explained why hydrogen didn't take over, how tar sands are more expensive than deepwater drilling, and pointed out that high oil prices during the Dubyah administration involved china creating and filling a strategic petroleum reserve, which is now full.

Meanwhile, the Dorito is selling off the US strategic petroleum reserve and presumably pocketing the money via some shell corporation. Drawing down the strategic reserve also opens the possibility of oil shortages and prices spikes for his backers to benefit from.

He's probably familiar with this because the way Putin funded his puppet government in the Ukraine (before they voted them out and he invaded to get back control of the port) was the same way he funneled money to some of Trump's campaign advisors: by selling them a bunch of oil from the state owned oil company at way below market prices, on generous credit terms. They they sold it at market prices, paid for the oil with the proceeds, and pocketed the enormous difference. (When dealing with enormous transactions you don't have to be that blatant abount it, inserting a middleman to shave off even half a percent can still be hundreds of millions of dollars.)

Here's a video on Farming the wind (I.E. wind turbines as new income for farmers.) Meanwhile, Solar has been something you farm for 5 years.

On average, data centers spend 40% of their energy on power conversion and cooling. They need to choose between putting them down south where solar makes power cheap, and putting them up north where cooling is free (most of the year).

People stop deploying solar when the "duck curve" threatens to go negative and you have electricity you can't use. California's crazy regulations once made them pay other states to take it rather than simply unplugging the panels (which is called curtailment, and no it's not bad for the panels). We're so used to electricity having a dollar value attached that when we generate power we can't use we freak out, even when it was free and otherwise would have just heated up some rooftops. Batteries let us store the power.

California now gets about 10% of its power from solar, and 90% of europe's new energy installations were renewables, primarily solar and wind. The worldwide installed base of solar panels has grown by a factor of 6 since 2010.

One more video I have lying around, Ramping up Solar to power the world more or less has a Wadsworth constant of 26 minutes and 30 seconds. (The stuff before it isn't exactly throat clearing, I just didn't find it very interesting.)

Happy first of Halloween!

I should update my Patreon. And I should do a proper pitch for why people should care about mkroot. It inherits most of Aboriginal Linux's goals, especially making Android self-hosting). But Aboriginal had its own web page. The new one has a mailing list, repo, but the only web page is the README in the repo shown by github on the main page. Hmmm...

More on battery technology.

I mentioned sodium's cheap and plentiful, and the next thing down in the periodic table from lithium, so lots of people are researching batteries made out of it. In this ted talk A woman presents her research on the "blue battery", but of course the resulting company is all white dudes because evil. Another group working on this calls its version the Seawater Battery. (There was a marvelous video on how lithium-ion batteries work, but the professor who gave it left to go work at Aqueon and the university let the video go down, because Carnegie Melon isn't in the business of preserving knowledge. I poked their website about it months ago and never got a response.) Other people are trying to make batteries from metalic sodium.

Another approach is to stop worrying about the energy density at all and go big, turning shipping containers into batteries, from ted talk to deployment. (That ted talk is old, the current status is standard startup hell because their molten magnesium/antimony chemistry corroded the shipping container it was in, they needed to develop new seals, and that ate through funding. Technology advances when patents _expire_.)

So, electric and self-driving cars. The existence of app-summonable self-driving car subscription services means that learning to drive a car today is like learning to ride a horse 100 years ago.

There's a timeline of horses going away: New York City newspaper columnists wrote about how the city would be buried in horse manure and have to be abandoned around 1890, which made cars were a huge environment upgrade over the next couple decades. Then cars scaled up transportation in general by 2 orders of magnitude, made suburbs possible, and once people started commuting to work ten miles each way by car, new problems emerged. The solution became the problem because "it's 1% as bad, there's now 100x as much of it, so we're back where we started".

Electric vehicles are going to replace fossil fuel vehicles completely because the gas station network is operating on razor thin margins. 25% of the USA's gas stations have already closed since 1994, and 1/4 of the installed base of gas/diesel cars go away the entire existing gas station supply chain becomes unprofitable.

A self-driving car fleet can charge while nobody's in it. Drive up a ramp, robot version of indy 500 pit crew swaps out batteries, drive off other side. Tesla demoed this back in 2013. Maybe a similar setup could clean (or at least vacuum) the inside of the vehicle. Battery prices have fallen 50% since 2014 and continue to decline (exponential growth), so spare batteries are just a question of time. (Demand is increasing faster than supply and there are raw material production limits, but there are a bunch of battery chemistries possible. Sodium and graphene are made from sea salt and carbon, it doesn't get cheaper or more plentiful than that. All a question of what you're optimizing for, and giving the patents time to expire.)

Volvo announced plans to phase out diesel vehicles, manufacturing only electric engines. (The emissions cheating scandal may have helped.) More than half of new car registrations in europe are for diesel vehicles, because they're more fuel efficient than petrol (gasoline), but in 2021 the EU emissions limit falling from 130 grams of CO2 per kilometer to 95 grams, and that means electric or hybrid cars.

Diesel trains are electric: the diesel drives a generator which powers the electric motor. This has been true since diesel replaced steam locomotives: diesel doesn't have any more motive power than steam, the greater hauling power is due to the greater torque of electric motors.

Caterpillar already sells self-driving trucks and mining equipment, meaning most mining jobs have been automated away. Currently automated sites tend to have skeleton crews to monitor status and respond to theft or vandalism. As time goes on that gets replaced with video feeds and drone patrols, and once you have enough data about what "normal" looks like you can have AI detect deviations from normal and either respond automatically to a known deviation or call a human to deal with it.

Neither millenials nor gen-x really buys into this uber-capitalism, thing, that's all by and for boomers.

Capitalism's domination in our culture is fairly recent. The "postwar economic boom" was an attempt to keep World War II's demand levels going to continue powering the economy. The wartime economy rescued the nation (FDR tried but couldn't get congress to approve _enough_ spending to reach full employment), and then they invented consumerism to keep demand high. It's a recent phenomenon, the Greatest Generation fed it to the baby boomers, and its seems to be dying with the boomers.

A lot's going to die with the boomers. The boomers have the biggest numbers and the biggest megaphone, and they tell people that how the world was for them is how it's always been, but it's just not the case.

The 60's were the 60's because of teenage boomers. Not every boomer went to woodstock or a was a hippie, but the boomer center of mass defined the decade. The 2010's are the 2010's because of retiree boomers turning into "racist grandpa", blaming "kids these days" for everything and screaming at long-haired foriegners to get off their lawn.

Bill clinton was the first boomer president, and he was born the same year as Donald Trump. They're now both 71 years old. The baby boom is on the downswing, and the actuarial tables come into play. The real question is when do enough boomers die for "racist grandpa" to lose political control of the country?

The boomers' parents were the "greatest generation" that fought in World War II, and their actuarial table results are in. Of the 16 million WWII veterans, only 558k are still alive in 2017, which means just under 97% of them have died already. The average age of a soldier serving in world War II was 26, which as a first approximation implies that in 26 years we can expect only 3% of the boomers to be left.

The question is, when do enough of them die to lose political control of the country? Most of the 77 million boomers are still alive (they were still 24% of the population in the 2010 census), and have concentrated wealth and influence into their own hands to punch well above their weight class even in retirement. But if they follow the demographic trends of the "Greatest Generation", 90% of them are going to die in the next 20 years. The last US veteran of the first World War died in 2011, but the question is when do the scales tip and power transfer? The average lifespan in the united states is currently 79 years, loosely implying 50% mortality over the next decade. What will that do to the GOP? What does this look like in real time?

Only one way to find out. Buckle up, it's gonna get bumpy. Wounded animals are at their most dangerous. Dying business models explode into a cloud of IP litigation. Japan's Kamikaze attacks came at the end as they knew they couldn't win and just tried to take their enemies with them. The GOP's southern strategy has tied it to the mast of racist sexist white demographic hegemony propped up by senior citizens, fossil fuels, climate change denial, and capitalist billionaires cornering the market on money and power. The clock's running down on all that simultaneously, and there's no mechanism for a smooth transition of power out of the hands of rich old white men, it's always been from one to another for centuries...

You can't have rich people without poor people. Bill Gates does not hire Warren Buffet to wash his car. It's rich people propping up the system, leveraging their wealth to hire think tanks and marketing companies to pull as many strings as possible to keep the gravy train going.

But capitalism is hollow, a fact widely noticed in the younger generation. Star Trek was post-capitalism. It's not a new concept. If you can't imagine another way of living, you're not describing a technology, you're describing a religion. We're waiting for Margaret Luther to nail a list of theses to Wall Street's door, and stop them selling indulgences to the 1%.

Rsynced down a fresh copy of my website to the big machine. I have like three directories with more or less the website contents, but just enough version skew I'm reluctant to just rsync one over what's there. Now I can triage. (It needs a largeish shoveling out.)

Ok, THAT's an interesting bug. I had a vfat system in need of fsck ("Free cluster summary wrong, 472622 vs. really 470607" according to fsck, although for some reason "fsck /dev/sdb1" would refuse to actually fix anything and say "Leaving filesystem unchanged" after prompting me what to fix and then ignoring what I said. Why? Because Linux is just crammed full of steaming piles of usability. But "fsck.vfat -avw /dev/sdb1" worked without even prompting me about individual tweaks, so that's nice.)

That's not the interesting bug. The interesting bug is that when I tried to copy a file to this filesystem, it would HOTPLUG REMOVE THE BLOCK DEVICE. "cp: cannot create regular file ‘/mnt/vmlinux’: Input/output error" and then suddenly /dev/sdb is now /dev/sdc and I have to re-mount it. (Don't need to unmount it, the mount's gone...)

Ah-ha! In dmesg:

Call Trace:
 dump_stack+0x63/0x87
 warn_slowpath_common+0x86/0xc0
 warn_slowpath_fmt+0x4c/0x50
 ? locked_inode_to_wb_and_lock_list+0x53/0xf0
 __mark_inode_dirty+0x27c/0x370
 mark_fsinfo_dirty+0x2e/0x30
 fat_alloc_clusters+0x37f/0x4e0

Who do I email about that... nothing on the kernel mailing list index mentions "fat". The kernel's MAINTAINERS file has a "VFAT/FAT/MSDOS FILESYSTEM" entry but no mailing list there either, just a personal email for the maintainer, Ogawa Hirofumi. Oh well, send them a message and see if they reply.

(I googled how to submit a bug to ubuntu, but it says I have to create an account to do it, so that's not happening.)

Solar power link dump du jour.

The GOP's war on solar continues. The fact 80% of Russia's net export income comes from oil/gas and the CEO of Exxon is now secretary of state aren't coincidental: solar power threatens the dorito's power base _directly_. The GOP recruited the racists and other single-issue voters in service of their original constituencey, the plutocrats. Four of the five largest companies (according to the annual S&P 500 index) are oil companies (the other one's wal-mart). Energy is 1/6 of the global economy, and until recently that meant exclusively fossil fuels.

That's why literally the first agenda item for the dorito was the Dakota Access Pipeline. The business case for building it erodes by the day, 4 years from now it'll be losing money. They hope that once the infrastructure exists, the fact it's there will provide incentives to keep using it at least long enough to pay off the construction costs. That lets the oil companies pump more oil out of the ground, and turn more of their balance sheet from theoretical money into real money.

Oil companies did lots of searching for oil and discovered oil underground (often in hard to reach places, or stuff like oil shale that's really expensive to extract), and then assigned that underground oil a dollar value on their books (as "inventory"). If it never gets pumped out of the ground, that dollar value is zero, which means they have to mark it down, which means recognizing losses, which means they've failed as capitalists and will get sued. Nobody wants to be left holding the bag, but nobody wants the cash trough the pigs are gorging at to end a second before it has to either... That's why everybody who used to work for the Tobacco Institute switched over to professional climate change denialism thinktanks. Squeezing $$$ from the last gaps of oil is where the blood money is today.

Meanwhile, Australia's power grid switched from coal to solar (and wind) so fast the old supply chain collapsed before the new one finished scaling up. Of course the fossil fuel companies are desperately trying to blame the new stuff (See! Unreliable!) but the reality is nobody wants to pay for the old crap anymore, including the people _maintaining_ the last gaps of the old stuff while it's still in use.

The 9th doctor narrated a BBC documentary called "The Last Miners" about the closing of the last coal mine in britain, but Youtube took it down due to enforcement of the holy secrets or some such.

Moore's Law has stopped applying to microchips and started applying to photovoltaic cells. Please adjust accordingly. This one is really boring and technical but I hadn't realized renewable energy storage was an emerging arbitrage market. Starting in 2020 EU requires all new buildings to generate all power onsite via solar/wind. California tried to copy that, but fossil fuel interests paid the GOP to sabotage it. British power generation achieved its first ever coal-free day. Here's an all-electric flying car, basically a rideable drone. India to elimiate gas stations by 2030, go 100% electric cars. For context, solar prices were $100/watt in 1975 and there were 2 megawatts of it installed worldwide. In 2015 that was 61 cents/watt and 65,000 megawatts worldwide.

Still trying to figure out how the GOP plans to sabotage/delay this...

I can't work at home because cats. Currently poison ivy covered cats, but even without that I'm gradually developing some sort of allergy or something where I get tired the moment I get home (and wake up more tired, usually breathing through whichever cat's curled up against my face this time), so even when I'm awake I can't focus. Perk right up when I leave the house though.

So I do a lot of work out at various places with tables, which usually means there's a food and beverage purchase involved. And I've noticed when I do this how _intensely_ most humans are herd creatures.

Take the HEB deli's dining area, which is less than half full when it's not rush hour so tying up a table for a couple hours isn't a big deal. It's a 4x2 grid of tables, and I try to pick a corner with nobody at the tables next to me. Then half an hour later, all the tables immediately adjacent to me are full, and the far corner is free and has nobody at the adjacent tables. So I move. And half an hour later they're clustered around me again. I mention it because I've moved SIX TIMES this session. (I don't move if there ISN'T a free corner table with nobody sitting at any of the tables next to it.)

I need to get a new pair of active noise cancelling headphones. Earbuds help, but not enough...

I'm starting to see why many self-employed people rent an office.

Spent Friday morning at the hospital emergency room, which couldn't figure out where the blood was coming from either, but it stopped and the cat scan didn't show anything unusual so they let me go home again. (Being very insistent I visit a specialist for follow-up, which I haven't.)

Personally, I suspect the poison ivy found a mucous membrane. (I'm very tired of going to doctors, having them be unable to find anything, and having to diagnose myself. I note I go to doctors _first_, and have them perform the expensive tests to rule out obvious immediate life-threatening whatsis. And then bill me a thousand dollars for not having found anything.)

I _was_ on a night schedule but didn't get out of the hospital until almost noon, and then slept the rest of the day and most of the following night because it was a stressful experience. I have no idea what my schedule is now. Took the big clippers and pruned all the poison ivy I could find, although it doesn't seem to be wilting despite having cut through the base of all those vines. (Wikipedia[citation needed] says urishol is actually a moisture retaining chemical, and humans' reaction to it is more or less coincidence. It would appear to work.)

I should go collect all the vine bits I cut up, but I don't want to touch them even with gloves or an inside-out trash bag. I'm pondering a visit to home despot to get lethal chemicals in a squirt bottle, which might be a nasty thing to do to the beehive in the neighbor's big tree (the half that didn't fall down remains beeful) but this poison ivy business is nasty all around.

Alright, I oversimplified when talking about oil chemistry. The chemical names I gave (butane and octane and such) are for the "simple" versions, where all the carbons are in a single straight line, each bonded once, with nothing else but hydrogen. Reality's weirder, 4 or more carbons can branch, providing a molecule with almost the same atomic weight (hydrogen's light, more or fewer of them is a rounding error) but with a different chemical structure, and by the time you get to gasoline there's a lot of options.

There's an entire category of chemistry ("organic chemistry") about the things carbon does, because it's extra-complicated. Here's a really quick intro.

Most atoms want to fill up their outer orbitals (or empty them out to the previous full layer), but they don't naturally have the right number of electrons, so they share electrons with other atoms creating atomic bonds. (The ones that DO naturally have the right number of electrons are "noble gasses", they don't usually want to bond with anything. They're gasses because each atom is its own molecule, so the molecular weight is lower than if you had multiple atoms glued together.) An atom with unused connections will bond to anything it bumps into (these are called "ions" or "free radicals", electric discharges can make them), so they don't stay unconnected for long.

Hydrogen only connects once and is tiny and plentiful, so the easy way to get a stable atom is to stick a hydrogen on each otherwise unused connection point, and it's such a common default that chemists don't bother drawing the hydrogens when they describe molecules: if nothing's mentioned just assume it's a hydrogen.

Carbon is right in the middle of the periodic table so it needs 4 electrons to fill up its shell (or drop down to the previous one, it's equidistant either way), so it wants to bond 4 times. This is pretty much the most of any atom (only things in that row of the periodic table do that: carbon, silicon, germanium, etc. The smaller the atom is the more chemically active it is because the electrons are closer to the nucleus, so carbon is a special kind of weird. That's why we're carbon-based lifeforms, it's the atom with the most tricks up its sleeve.)

The other thing you need to know is tiny/light molecules tend to be gasses (unless they're polar and stick together sort of magnetically, but only stuff at the left or right edge of the table does that, carbon's right in the middle and thus nonpolar), then bigger/heavier molecule stick together to form liquids (because of london forces, where brownian motion jerks the nucleus to one side of the atom so it momentarily develops a positive end and a negative end, and you get transient weak polar bonds), then even heavier stuff forms solids. (This is why the atoms that naturally _do_ have full electron shells are called "noble gasses". They don't want to bond with anything else, so their molecular weight is their atomic weight, which means they're light molecules and thus gasses. Water _should_ be a gas by atomic weight, but it's so strongly polar the molecules stick together like magnets which makes it a very chemically active liquid. Water's about as chemically weird as carbon, which is why we're _water_ based carbon based lifeforms.)

So: back to the list of molecules from last time. One carbon with a hydrogen bonded to each of its four connection points forms methane, written as "CH4", one carbon four hydrogens. It's a gas at room temperature and so light it floats up to the ceiling in our atmosphere. (Really the heavier air's sinking below it and pushing it up out of the way.)

Ethane is two carbons connected once (with each carbon's other 3 connections going to a hydrogen, so C2H6), and is also a gas at room temperature. But those 2 carbons can also bond to each other twice (think two dungeons and dragons "D4" dice sitting on the same table with an edge touching, so two of the points touch), leaving 2 hydrogens on each carbon. That's ethylene, which makes fruit ripen and it's also what polyethylene is manufactured from. And yes those two carbons can triple bond (those same two D4 dice can have a flat side against each other with the triangles lined up so 3 corners touch, making C2H2), that should be called "ethyne" but is usually called acetylene for historical reasons. (The -ane ending means it's all single bonds, -ene means there's a double bond, -yne means there's a triple bond.)

Similarly charged atoms sort of repel until they stick, like magnets covered in crazy glue, so it takes more energy to push them closer together to make double and triple bonds, and then they shoot apart again hard if the bonds break. That's why acetylene burns hotter than natural gas, you're breaking that triple bond and getting the energy back out. (Turning the bond energy into motion, which becomes brownian motion, which is heat. They're all microscopically vibrating and bouncing all over the place, that's why they're more reactive.)

Of course you don't HAVE to fill all the other binding slots with hydrogen. Reality is full of chlorine and sulfur and oxygen, and all of it binds to carbon. (Oxygen wants to bind twice, and usually one side of it binds to hydrogen anyway when the other side binds to something else, and that's called a "hydroxide group". Hydrogen hydroxide is, of course, water. Carbon with a hydroxide bonded to it is called "alcohol" (from arabic "al" meaning "the" and C O H being the letters for Carbon, Oxygen and Hydrogen), and its ending is "-ol". Methane hydroxide is wood grain alcohol (methane-ol, I.E. methanol), ethane hydroxide is the alcohol in beer and wine and vodka (ethanol), and then propane can bind the hydroxide to the end or two the middle, producing two different molecules. The one where it's bonded in the middle is "isopropyl alcohol" (isoproanol). The iso- prefix means "bonded in the middle rather than at the end". There's a whole vocabulary chemists use, mostly inherited from the 1800's when they were still figuring out how this worked.

And once you hit 6 carbons your chain can wrap around and bind its own ends together, producing a "benzene ring". Benzene itself is carcinogenic because it goes right through cell walls and dissolves DNA like soap (Judge Doom also uses it to kill 'toons), but if you bind a hydroxide to it you get "phenol" which can't get through cell walls (the OH makes it polar and cell walls are a nonpolar mix of fat and protein; smaller alcohols can sneak through anyway but phenol's too big), and that's used in mouthwash. (Most alcohols will kill bacteria, but phenol doesn't get into human cells so it isn't intoxicating, so people are less likely to try to drink it.)

One of the things oil refining does is wash the oil with chemicals to react with all this stuff and strip it off: rip off the sulfur and chlorine and hydroxides, break the double and triple bonds and replace them with hydrogen, cut the benzene rings, and so on. Oil is full of complexity and refining either filters it out or squashes it to the simple stuff, because the point is to get carbon molecules that burn cleanly without side effects, reacting with oxygen to produce carbon dioxide and water. Back before they removed the sulfur in refining, you'd get sulfuric acid as one of the outputs, and then acid rain kills fish and plants and damages buildings.

This gets us back to yesterday, where one carbon with hydrogen plugging all the connections is methane, two carbons connected once to each other and everything else being hydrogen is ethane (well, if the carbons are connected once, if they're double-bonded together so there's only 4 hydrogens needed to plug the rest, it's etylene). Then three carbons is propane, four in a row is butane etc. The short molecules are gasses at room temperature, the longer ones are liquid, and even bigger ones are solid (waxes) that melt like butter (or may stay liquid when mixed with the smaller ones acting as a solvent).

But you've got to simplify their structures before you can handwave away their differences like that.

Going through old tweets. Apparently I started on this whole solar kick back on April 13.

Back then I watched a lovely ted talk about how Guatemala has twice the solar power per square foot as london, and poor countries leapfrog to new technology more easily because they have no old stuff to replace. (Countries that never had wired phones now have cell phones everywhere.) And there were stories about how a small solar panel that plugs into a cheap battery pack, collectively designed to be ~$15 camping equipment in a tent, can power three LED lights and USB charge a phone battery in a rural villager's home. And these are just TAKING OVER some country (not guatemala?) because A) instead of going into town to charge the phone, you can charge it at home, B) your $10/month kerosene bill turns into a one time payment of $15 and then lighting's free, brighter, non-smelly, non-carcinogenic, doesn't cover your house in soot, and doesn't burn your house down.

Except... that was 3 ted talks, and I only saved a link to one. And I can't seem to find the others again. They were quite good. They weren't the only source of this information, but they were succinct and enlightening and how _I_ first learned this stuff, and I wanted to pass it on...

Avery Lovins covered some of the same material in one of his talks, which has really good information buried in the animal noises and sound effects that make him a really irritating speaker to watch. Maybe it's just me, but I find him amazingly smug and regularly picking turns of phrase designed to ensure he's only ever preaching to the choir. (Even if you already agree with what he's going to say, I find it hard to stomach him talking about "the rotted remains of dinosaur goo" or whatever he's sneering about this minute. "I invented the words negabarrel and feebate but I'm pretending I didn't because I'm trying to make them a thing, don't you feel stupid for never having heard them before? I made up PIGS and SEALS acronyms, let's club some seals. And that's not even the reason I make a pig noise in this talk!" I dunno, maybe I'm just tired of hearing from old white guys. It _is_ really good information, but I'm reluctant to forward it to anyone else if I _expect_ the presentation to turn them off before they're halfway through.

Anyway, the collective story I found so fascinating is that the kerosene industry remains a billion dollar/year niche today, but it's imploding as fast as the coal industry because solar panels and battery packs are eating the low end of the market. Kerosene lighting is one of those "payday loan" slumlord niches milking the poor so it flies under the radar, but it's a huge amount of money that was feeding into oil company profit margins. And now that profitable niche is drying up and blowing away.

We've actually seen this before: oil lamps used to burn whale oil, and the reason we didn't drive whales extinct in the 1800's is kerosene refining was discovered as a cheaper alternative. Whales produced a bunch of products: not just oil but meat and whalebone (a strong lightweight construction material, corsets were made out of it for example). But a cheaper replacement for whale oil tipped the scales and made whaling unprofitable, and then people learned to make plastics from oil to replace whalebone, and the US west opening up increased the beef supply (the cowboy era)... But whaling was attacked one product at a time, and that reduced the profitability to the point the whole thing stopped happening _before_ the world quite ran out of whales.

When a walking ship killed a whale, it got blubber (rendered into whale oil), and also meat and whalebone. It couldn't _not_ produce all of them, in a fixed ratio. All they could do is waste the parts they didn't want. Crude oil works the same way: refing it produces butane and kerosene and diesel and gasoline and vasoline and so on, all in parallel. The components of crude oil are mixed together and refining just unmixes them, it doesn't convert them into each other. So if you can't sell some of the outputs, you have to pay to get rid of them. (That's why they keep figuring out how to make dyes and fertilizers and antibiotics out of it, the alternative is paying to neutralize and dispose of massive quantities of highly toxic industrial chemical waste.)

The actual distilling part of oil refining is mostly just separating things by boiling point. The component molecules are categorized by the number of carbons connected together: one carbon is methane, two is ethane, three is propane, four is butane, then pentane, hexane, septane, octane, nonane, decane, and so on. The more carbons the molecule has stuck together the heaver it is, which gives it a higher boiling point. This means the 1-3 carbon molecules are a gas at room temperature, 4 is just barely a liquid (butane is the clear liquid in cigarette lighters that doesn't need to be pressurized to stay liquid, but evaporates at the drop of a hat). Then it's liquids with increasing boiling points up through about 20 carbons, where you start to get solids at room temperature. (Keep in mind if you mix them the act as solvents for each other, so you can have a "solid" soaking up components that would be liquid on their own, or solids dissolved in a liquid.)

The problem is crude oil has _all_ this stuff mixed together, and you can't say "I want to turn this batch of crude oil into pure octane", it doesn't work that way. The refining process primarily just separates the various chemicals by molecular weight. You can tweak the output percentages a bit using different chemical reactions, but that adds to the cost, and modern refineries don't necessarily know how their machinery even works anymore so tweaking the process is slow and expensive and occasionally explodey. Plus each oil well produces slightly different kind of oil (categorized into things like "light sweet crude" and "heavy crude", but also contaiminated with things like iron or sulphur... refineries have their hands full filtering out the weirdness in different batches. And then there's the fun of US refineries designed to deal with middle eastern oil for historical reasons, and sending the canadian shale oil to refineries overseas that specialize in _that_ kind of oil. And retooling any of these is a multi-billion dollar proposition that would have them offline for years.)

The output of oil refining is various products that are generally mixtures, not pure chemicals. Vaseline tries to average about 25 carbons but what they mainly care about is how solid or squishy it is at room temperature. Paraffin wax is anywhere from 20 to 40 carbons, all mixed together. Kerosene is 10 to 16 carbons. Diesel is defined as boiling between 200 and 350 degrees celsius, which means 8 to 21 carbons. Gasoline's "octane rating" originally meant "this liquid acts like this percentage of the liquid is 8-carbon octane, and the rest is some neutral filler that doesn't burn", but as with diesel gasoline is primarily defined by boiling point (30-210 degrees celsius) which is anywhere from 4-12 carbons, and then the cheaper vs more expensive types of gasoline are different refining processes producing different mixtures.

(Yes, that last link says diesel is 170-360C but the earlier wikipedia[citation needed] link said 200-350C. What standards there are for this vary from country to country. And yes, having a little butane in gasoline and a lower end boiling point just above room temperature is why you can smell it so strongly at the gas pump, and why it evaporates so fast when spilled on warm ground.)

This is the context for the loss of the third world kerosene market: the diesel and petroleum markets can't _not_ produce kerosene and vaseline and natural gas as a side effect. They can only skew the ratios of their output just so far, and that raises the price and may affect quality of the product. Refineries used to burn off natural gas because they had it, couldn't profitably turn it into anything else, and couldn't find anybody to buy it. (Not necessarily because people didn't _want_ it, but because capitalism said selling some and destroying the rest was their most profitable move. Selling it all would drive the price down, so they'd make less money overall. Capitalism is a mechanism for regulating scarcity, and in the absence of sufficient scarcity capitalism will create it. The whole point of OPEC existing is to reduce the supply to keep prices up.)

Oil companies today are selling kerosene to the third world because they've got it and can't find anything more profitable to do with it. (They can turn most of the same components into jet fuel, but that's a market with more or less fixed demand, so selling more lowers the price.) So losing a billion dollars a year of kerosene lighting customers comes straight out oil company profit margins. Those customers are now giving money to solar panel and battery producers. They're spending less money on solar than they were on kerosene, but the result is still a combination of "less money going to fossil fuels" and "more money going to solar/battery production and development". And that's a lovely thing.

Back on April 12 I looked at a bunch of Linux video editors, and tweeted about the results but didn't write up a blog entry. So for posterity:

The way to make screen capture work on Linux, to get a video of your desktop with synced audio, is to fire up vlc and under the "media" menu select "open capture device", then select capture mode "desktop", then set the frames per second (default is 1.0, you probably want more like 4), then click the "show more options" checkbox, click the "play another media synchronously" checkbox, and enter "alsa://" in the Extra media field. (This is the magic dance to make it listen to the microphone so it can hear you speak while you type stuff. You may need to plug a microphone into your input jack.)

Then click the pulldown arrow next to "save" and select "convert" and at the bottom entier the Destination file name (with an .mp4 extension). And THEN hit start.

If this seems like an elaborate dance to do every time you want to record something, that's because it's Linux on the desktop. I tried a bunch of other options and they don't WORK.

I still need something to edit the video, so I googled a bunch of options. There's lots! None of which actually work yet. KDEnlive wants to install 124 packages (the whole of kde, including the web browser). Avidemux seems promising (except for lack of "undo") until I saved the video, which started out as a grey screen with changed rectangles filling in. (No initial keyframe! Kind of a bug. I watched a video tutorial on how to use avidemux from 2010, and 7 years later it can't replace keyframes it cuts out when editing an mp4.) Openshot wants to install qtchooser, libqt4-dbus, python-support, and docbook-xml (despite the documentation being a separate package). As far as I can tell if you want to edit video on Linux without bundling more bytes than Microsoft Word + Internet Explorer combined: you buy a mac.

I _can_ edit footage with vlc, or at least save sections as separate files and glue them back together again, using a similarly awkward mess of non-obvious mechanisms manually repeated. It's entirely possible this is what I'll wind up having to do if I want to edit anything.

Then again, when I give talks at conferences the camera starts, the camera stops at the end, and in between sounding coherent is a matter of preparation and practice. Doing my own videos on the netbook without being able to edit them isn't _worse_ than that, and I don't have the "will this video ever go online" problems like with Flourish or the years ELC took months to put stuff up.

I got pinged about lastgplv2.org, which I'd largely forgotten about, and replied explaining why.

As with many things, I should do a proper writeup. But I had an IRC chat today where a question about LP64 turned into the link to "and here's why microsoft didn't do that, and here's the entry from that same blog about why microsoft won the 32 bit transition", which led to "heres why Intel won the IBM PC contract in the first place", and "here's how Cortex-M defeated Itanium inside Intel", and I'm SURE I wrote all those up at one point by my old Motley Fool articles have fallen off the web and most of the old stuff gets weighted down to nothing by google for being too old.

I have "podcasts" as a patreon goal more or less as a placeholder for this. My netbook has a "podiatry" folder with lots of todo lists and partial outlines for such things, but... I have too much else to do.

Up way too late on IRC with Jeff again (turns out texting costs him money; I get it for free).

Poking at mkroot and there's one of those "the fighting's so vicious because the stakes are so small" design issues: should the script create the initramfs cpio.gz image from the root filesystem directory before or after running the kernel build?

The code's currently doing so after, which is slow if all you want to do is rebuild the root filesystem. If it's before, as soon as the kernel starts building you can ctrl-c out and grab the cpio.gz file which has already been updated.

But the REASON it's after is if your kernel build creates modules, it can install those modules into the initramfs and you won't get the updates unless you package up the cpio.gz after the kernel does this. None of MY kernel configs do this (not really a fan of modules), and putting them in initramfs is a bit silly (they're pinned in memory all the time, might as well make 'em static in the kernel), but there are reasons to do it (mostly having to do with deferred initialization and specifying module arguments you don't want to stick in the kernel command line).

I've had a local diff forever that moves the cpio.gz creation up before the kernel build, and I'm tempted to check it in because persistent local diffs are a "code smell", but it would remove flexibility. (Flexibility I'm not personally taking advantage of, but still.)

In theory I could even make "cpio.gz" be a module, and make "squashfs" and "ext4" targets too. Except I don't want to have to specify them on the command line each time to get the "default" behavior. Similarly, I haven't moved the base root filesystem build out to a module, even though having mkroot -d to _not_ do it is kinda silly. I _could_ have a default set of modules that gets run through if you don't specify modules, but... that way lies aboriginal linux's complexity, and right now the build is still simple. How do I hold on to that simplicity but keep it flexible and capable of doing what I want it to do?

And once again it's one of those painful design decisions to make because either way's fine-ish, so neither is clearly _superior_, so there's no obvious right thing to do that stops the internal debate.

And when I'm developmentally swap-thrashing between far too many projects, just trying to keep the plates spinning so they don't come crashing down, something that requires a half-hour walk while I think about it is enough to dump it back on the todo list and cycle to something else, which means it becomes a blocking issue to further development. Not because it's hard, but because it's harder than I'm up for in the 15 minutes I have to poke at this when I'm too tired for "productive" work...

It's not fleas. It's poison ivy. The electric utility guys trimmed the trees and bushes in the back yard and the back fence that used to have lots of bamboo now has lots of (bruised) poison ivy, and the cats have been romping through it and then cuddling up against me, my pillow, the couch, my desk... What fun. Thanks cats. How about NOT being in my lap right now?

I got pinged about qcc a few days ago, and replied that I just haven't had time to work on it.

I've talked about how we the simplest self-hosting development environment is four parts: a kernel, command line, compiler, and C library. All of it written in C (one simple low-level implementation language), and all of it under a public domain equivalent license (or at least something close, ala BSD/Apache).

The reason that's important is so you can understand and audit all of the base system, among other things avoiding Unix creator Ken Thompson's trusting trust problem. Tl;dr: Ken once pranked the BSD guys by adding code to his C compiler that recognized when it was compiling the login program and added a magic root password to the resulting binary, and he also made it recognize when it was compiling the C compiler so it would add _itself_ to the resulting compiler. Then he recompiled the original compiler with his hacked version so the change was there in the binary but not in the source code, so the BSD guys wouldn't see it but it would still be there in any new compilers they built with that compiler. Thus any login programs they built from then on would have his magic root password, even if they audited every line of source code and rebuilt the entire system from source. (He removed it before giving the lecture where he described the hack.)

People have been trying to counter this for a while, but it boils down to "if you're running on a system with a rootkit, nothing you do inside that system is guaranteed to find or remove the rootkit if it's designed to hide from that way of looking for it". You need to look at it from the outside, with a "clean" system. But how do you get a "clean" system?

The way to deal with this is by having multiple implementations of every tool (so they're at least not all hacked the same _way_), being able to inspect all the source _and_ analyze all the binaries, and being able to understand what everything _does_ at least at the base layers so you have at least a chance to spot strangeness when it's there. (You can look at a hex dump of 20k of data and read the disassembly, that sort of reverse engineering is what some security researchers do for a living. But nobody can look at a hex dump of megabytes, the bloat hides the exploit, security through obscurity pointed the other way.)

There are other advantages to having a simple minimal baseline environment: it's the smallest amount of code you need to cross-compile to new hardware before you can start native compiling. And when you're learning how the system works, it's possible to achieve a complete understanding of a real working system you can actually use for something.

My toybox project is working towards such a command line. (I already mostly achieved it under busybox, but that's doomed to a license that's unlikely to outlive the current generation of programmers using it.) Musl-libc is a saneish C library. (I said ish.) These days I'd might point at xv6 instead of Linux, both because it's simpler and because it's got a whole textbook explaining about how it all works, but there's a lot of shoveling to do to make it hold weight. (The Google Fuchsia guys were working on that, but my interest in it never got traction, and Google has a history of abandoning projects so a wait-and-see attitude on anything new seems justified.)

Ideally each of these tools can be completely explained in about one semester undergraduate programming course, so four one-semester courses would get you a basic understanding of a complete usable operating system, under which you could natively compile Linux From Scratch. (and Beyond Linux From Scratch), thus bootstrapping up to arbitrary complexity from that initial "clean" starting point.

The point of qcc was to provide a simple self-contained version of the C compiler, but leverage the work QEMU is doing for multiple architecture support. Fabrice Bellard's Tiny C Compiler provided a working proof of concept, 100k lines of source code that built a bootable Linux system. Yeah, he cheated a little bit but it _worked_.

I maintained a tinycc fork for 3 years, but stopped because reasons. I got permission from Fabrice to BSD license his code (0BSD didn't exist at the time, but might count, I'd have to ask), but there were other people's contributions to the repo (initially quite sporadic), and it fell down my todo list while still in the "trigage" stage.

So still a work in progress. But this is another output of the "make android a self-hosting development environment" project, trying to repot it onto an understandable/reproducible/auditable base. And that's why "let's suck perl and zlib and curses and openssl into the circular dependencies at the root of the tree" is counterproductive. You wanna find a trusting trust exploit in perl? You think you're better at finding it than NSA/KGB/RNC/RIAA/Verizon/China is at hiding it? You think it not being there _today_ means it won't be there tomorrow?

In brief: Avoid.

Some insect bit my hand a couple times on the couch last night and it ITCHES, and then the same hand got bit more when I went to bed, and I'm having one of those allergic reactions that may require cortizone. I note the same cat was curled up in the same position relative to me both times. (No, it's not a cat bite, it's a bug bite.) Yes, the cats that follow me around the house incessantly and won't leave me alone. Last night all three of them were on the couch with me, not sure how many in bed.

Apparently they were supposed to be flea-dropped again at the start of the month, but Fade was doing that and moved her trip up to avoid Hurricane Harvey and didn't leave us a todo list, and it fell through the cracks. And now we spray and wash everything, and I maybe get allergy meds.

I'm tired of cats. I sort of viewed pets as training wheels for having children, but at this point it's pretty obvious that's not happening. I had 2 cats, Fade had 2 cats when we moved in together a dozen years ago, and 4 cats was Too Many Cats and I started to develop cat allergies, but over the years Dragon wandered off and Aubrey died, leaving two cats (George and Peejee) but then Fade wanted a dog, and that dog (Pixel) tried desperately to make friends with the cats who were viruently speciesist, so Fade got a kitten (Zabina) on the theory it wouldn't know better and would hang out with the dog, and then Pixel died suddenly (heart tumor) and then Fade got another dog (Adverb) before Zabina finished growing up because she ships cats and dogs together. And along the way my brother moved and sent me the cat I left with my mother back in 2001 (Foster) who was 17 years old, and we were once again up to 4 cats and my allergies started coming back...

Foster died of old age after about a year (he had a pretty good year though), and Fade took the dog with her up to Minnestoa, but there are still 3 cats here. Peejee and George are 14 years old and Peejee is getting SUPER CLINGY in her old age. I'd be willing to wait Peejee and George out (the age record for cats in my family is 22 but among my siblings and me the record is 19), but Zabina's two and likely to last another fifteen years easy.

They're lovely cats. I am tired of having cats. Too much cat. I can't work at home because cats climb up on my keyboard and on my shoulder and block the laptop's heating vent and WON'T LEAVE ME ALONE.

And now I can't sleep because itchy, and there's so much laundry to do when I get back. Maybe I should get a hotel room.

Witness me, I did NOT reply to Christophe Leroy with:

That was my argument last time, and the answer was "Breaking userspace is bad, mmmkay." Even when userspace is doing something REALLY OBVIOUSLY STUPID and it is _clearly_ their fault, as long as they got there first they've established the status quo and moral arguments about right and wrong are a bit like the native americans asking for their land back.

And I did NOT reply to Greg KH with:

You're right, I forgot step five of twenty-six in the secondary procedure for submitting my form through the approval bureaucracy, Which is also step five in the sixteen steps for the primary procedure.

(Do they have the equivalent of tax professionals to prepare and validate your forms for you yet, or is that still coming?)

The REAL danger of that last bit is SOMEBODY WILL DO IT. And make a lot of money doing it.

(The bureaucracy and bikeshedding in modern Linux Kernel development is the diametrical opposite of fun. I gave a talk about this at Flourish in Chicago but they never put the video up.)

I'm trying yet again to get my CONFIG_DEVTMPFS_MOUNT patch upstream. My third attempt hit a change in how printk() works (I left off the /n and the flushing got deferred and weird), but that was my bug. Just a strange new manifestation of it due to kernel version skew.

My new attempt broke booting for debian derivatives, which took a while to track down but it turns out debian's initramfs boot script is doing "if ! mount -t devtmpfs /dev /dev; then mount -t tmpfs /dev /dev; fi" which is STUPID. If we inherited a working /dev but the devtmpfs mount fails it INTENTIONALYL BREAKS the existing /dev directory. (The mount faiils in this case because it's already there: debian kernels are built with CONFIG_DEVTMPFS_MOUNT but then booted into an initramfs, and previously that was a NOP until my patch fixed it. But if you booted a kernel without CONFIG_DEVTMPFS enabled and provided a static /dev in the initramfs, this would similarly break your boot for no reason. There's no circumstance under which it can work, the tmpfs mount on /dev gives you an empty directory the rest of the boot can't use.)

The underlying problem is Debian's bug, an untested error recovery path triggers and blows up the system. If they didn't do that, it would work fine. The proper fix is for them to STOP doing that.

But until Debian not only fixes itself but the fix propogates through all the derivatives using the broken debian script (Ubuntu LTS has a 5 year support horizon, not likely to be fast), my patch needs to include an ugly workaround (don't return error when you try to mount devtmpfs on top of itself) if it has any chance of getting upstream, and then I need to go "yeah but [large lump of backstory, go talk to Debian]" to everybody who objects, and it's a political issue not a technical one, and see "linux kernel is no fun anymore" rant from earlier in the week.

Hurricane Harvey stripped the paint off of a corner of the house, so we're having it repainted. Technically I suppose that's another $2600 of flood damage, just to the top this time. Then again we haven't had it repainted since we bought it in 2012, so I suppose it's about time anyway.

Trying to get a Toybox release out. Jeff asked for "dd", which makes him like the 9th person to do so. (Awk gets more requests, but that's a LOT more work to implement.) In theory the 3 months since last release mark is the 15th, I.E. Friday. In practice, I'll probably slip it to the following monday.

I recently asked the kernel list for execve(NULL). Never have I seen a more pronounced manifestation of Dunning-Kruger syndrom from otherwise highly technical people. Nommu doesn't have fork()? That must be TRIVIAL to fix, the fact nobody's done it in the past 40 years means this idea off the top of my head must never have been tried before!

So, various people started bikeshedding and coming up with wild compiler redesigns that wouldn't work... Then Alan Cox chimed in with a hilariously misinformed take, advocating the minix approach of using what amounts to the old DOS "overlay" feature, copying all the memory segments out to backup storage every single task switch. On nommu this means that when you fork you need at least twice as much contiguous memory, possibly three times (backup storage for _each_ version plus active storage for this version, you can swap 'em in place if you're tricky about it but as the mappings diverge that gets complicated and potentially buggy). And it means that if you do fork() and exec, you copy all the data for no reason, just to discard it again. (Just try and do a shell script under those circumstances.)

The example case I first heard for this a decade ago is: suppose firefox wants to launch a flash plugin as a child process, which means fork() followed immediately by exec(). So it did a fork() off a 500 megabyte process the immediately discarded all 500 megabytes to launch a 10 megabyte process. Normally you leverage copy-on-write so the actual physical pages aren't copied, it's just reference counts twiddled. If you have to actually copy that you're at _least_ churning the heck out of the memory bus and CPU cache. If you only had 100 megs of memory left then actually copying all the 500 megs in the fork() either fails or swap thrashes for quite a while, just to discard it immediately again.

But reality is worse: because on a nommu system all your pointers point to physical memory, meaning you can't MOVE the data to create a copy. If you copy memory to a different address range, all the pointers point back into the old address range. So you have to evict the old copy of the data (to swap or other memory) so your new copy can run, and then if the program runs long enough for the scheduler ot kick in you have to evict the new copy and copy the old one back to the same address ranges so it can run. Meaning you take this fork overhead every time you schedule _either_ process.

The way vfork() gets around this is by giving the child shared mappings instead of copy on write mappings. Nommu can't implement copy on write (no mmu to intercept illegal access and generate a soft fault to a kernel route that fixes up the illegal access by adjusting the mappings), but it _can_ implement shared memory. Unfortunately if the child and parent both modify the memory they interfere with each other, so vfork() also freezes the parent process until the child calls exec() or exit(), either of which disposes of its copy of those mappings (unsharing them so the parent now has exclusive access to the memory again).

If you exec() almost immediately after the fork there's almost no difference in behavior between fork() and vfork(). But if you don't, it gets dicey fast. If the child writes to the shared memory bad things can happen (which includes writing to the alread-used parts of the shared stack, such as returning form the function that called vfork() (which can stomp the return address the parent uses when _it_ returns from that function).

Sometimes people have the bright idea of copying the stack to give the child its own copy, but A) pointers to stack variables can happen and they'd point to the old copy, B) the stack is potentially a large amount of memory being copied. (It usually isn't on nommu, but it _can_ be, especially since vfork can also get used on with-mmu systems to have a common code path.)

The other problem is if the child blocks, the parent stays blocked. This thread started because ptrace wants the child to SIGSTOP itself so the parent can ptrace_attach() to it before resuming it and performing the exec.

All the tricks to provide more conventional fork() semantics on nommu wind up either reproducing the problems or coming up with worse ones.

So Alan Cox said all we need to do is implement nommu fork() the way minix did. Yes, this is the same minix that had a filesystem maxing out at 64 megs and which couldn't overlap CPU and I/O. Its deficiencies drove Linus to create Linux in the first place, becuase it already didn't scale to a 40 mhz 386 with 2400 bps modem circa 1991. Clearly, we should do what they did.

And I just don't have the energy to give him a large enough backstory dump and try to argue him around. Specifically, I cannot do it POLITELY. (Plus what the THREAD is about is execve(NULL) and vfork is a tangent there, it's useful for something like busybox or toybox to be able to re-exec itself in _general_, without relying on external dependencies like /proc being mounted; vfork() is a _distraction_ from that, it's one example use case he's trying to shoot down because it's there and he thinks the problem he hasn't personally dealt with must have a simple fix a generation of programmers hasn't seen because he personally can't find anything wrong with the first thing he thought of.

I tried to get Jeff Dionne (my boss at $DAYJOB and creator of uclinux) to reply to him, but Jeff doesn't want to get linux-kernel on him. So once again, it's me arguing with everybody else on linux-kernel, the lurkers privately supporting me in email but nobody actually chiming up in PUBLIC to say anything... and I'm tired.

I HATE winding up with "the lurkers support me in email". It means there's a breakdown in the social fabric of the community, and people see me as willing to take on a political role to advocate for their interests while they stay anonymous and avoid participating in said social aspects. This puts ME in an uncomfortable position, doing unpaid basically emotional labor.

The alternative is code staying out of tree forever, and having to maintain patches that upstream intentionally tries to break each upgrade to make the out-of-tree approach expensive, hoping to _force_ people to either confront them or bow to their way of doing it.

Oh well, what I usually do is bring it up again next merge window and hope that between now and then some of the people bikeshedding against it have died of old age. Meanwhile the out-of-tree version works for me.

Walked to ACC instead of UT. (I still have a valid ACC student ID from the japenese class. Which is only fair, since they kept the money.)

It's not quite the same 24 hour availability of the outdoor tables protected from rain with a working electrical outlet at the little niche off of speedway (which has "quiet undistirubed working environment" by virtue of being there at 3am, especially over the summer). But the highland campus is open until 10pm, and it's got all the space in the world (converted shopping mall) and it's all tables and outlets and air conditioned study space, and if I head out around sunset I get a couple hours there before closing time, and then I can head to the McDonalds on I-35 that's open until 1:30 am and get another couple hours there before heading home.

Given that I still can't work at home because cats (so many cats, so clingy), that's not a bad variation. (The Wendy's in hancock center also closes at 10, but I'm usually there earlier in the day for a couple hours of work, and don't particularly want to eat two meals there every day. HEB's deli area used to be open 24/7 so I could go there for a couple hours at 3am, but homeless people started camping out there and now they put away the chairs every day at 7 pm.)

What I really want is a 24 hour Starbucks. I've tried Epoch a few times but it tends to be full, and that's a long walk with no guarantee of a place to sit at the end. Plus the drinks there are about as expensive and as many calories as 4-for-$4 at Wendy's.

I miss Metro. 24 hour coffee shop with a beverage I really liked (Big Train spiced chai, made with steamed milk), next door to a 24 hour video arcade. Pity the Cult of Scientology took over the building and raised the rent to drive them both out.

And GPS is back to being endless feature creep with moving goalposts. Customer doesn't give us money until it demonstrates X. Then Y. Then Z. Still no money. Friday will be the 4th missed paycheck. (The one year anniversary of going down to half pay came and went, that was back in like June.)

We are SO CLOSE to making this work. And I am out of gas.

I tried to switch back to a day schedule after the "Yay GPS working!" part last week, but Jeff has still needed me to help debug GPS stuff several nights since and I've been in that fuzzy "haven't GOT a sleep schedule, just random naps and I'm always tired" state for a week now. It's still really easy for me to get stuck on a night schedule because sunrise makes me really tired, but sleeping until 3pm (with the daily call at 5) means that by the time I'm free to run errands everywhere's closed. (I've been meaning to run an errand at a non-local bank for weeks.)

But closure ain't happening.

Yay! Jeff finally got GPS to do a thing! With live antenna data! There's still several things wrong with it, but there are circumstances under which it does at least some of what it says on the tin.

I'm kind of glad that car dealerships are one of the things self-driving app summonable cars will put out of business in a single digit number of years.

Alas, ever since SCO I've said "dying business models explode into a cloud of IP litigation", which is shorthand for the way drowning swimmers climb on top of would-be rescuers and drag them under. Other people around them aren't struggling, so when they give up trying to keep their head above water under their own power, they grab on to nearby survivors and attempt to suck the success out of them. It's a failure mode you see often, wounded bear syndrome. Nothing to lose.

The current political situation seems to be a combination of the Southern Strategy making the GOP go septic (as was foretold at the 1963 rockefeller vs goldwater convention), the entire fossil fuel industry reacting to solar and wind's exponential growth the way the RIAA and MPAA dealt with streaming media replaced physical recordings, and the baby boomers hitting retirement like a brick wall and going "but we've ALWAYS run the world, the universe can't possibly continue to exist without us".

The feedback between them is uncomfortable. With all three burning down to the ground they'll have nothing left soon, but they're determined to go out in a blaze of embezzlement. The question is how much damage they'll do on the way out, and how much rebuilding work they leave for everyone else when they're gone.

I've been looking at strace which means looking at ptrace, and I'm trying to figure out how it's supposed to work on nommu? The big version at least builds and runs on nommu, but I don't understand how it does launches a stopped child with vfork?

Ordinarily I don't look at other packages' source code (dowanna get gpl on me when writing a public domain implementation), but I couldn't figure this bit out long enough I eventually took a peek (github is a publicly viewable webpage, the answer's theoretically _right_there_) and the reason I couldn't figure out the synchronization mechanism is it doesn't synchronize on nommu, instead it races and probably misses the first few system calls.

The problem is you need to fork(), have the new child kill(SIGSTOP) itself, and then exec(). That way the parent can trace the exec itself, including the syscalls the dynamic loader does and so on. But with vfork() you can't do that because until the exec() happens the parent can't resume, so if the child stops itself first the parent stays stopped.

The solution is do to the "re-exec yourself" trick where you exec /proc/self/exe to restart yourself (thus freeing the parent), and signal to the new exec that it's a re-run either with environment variable data or by changing the argv[] data. But re-execing yourself is brittle because /proc/self/exe isn't guaranteed to be there (in a chroot for example), and argv[0] could be a relative path vs a cwd we already did a chdir() away from. (Assuming argv[0] was set accurately in the first place, execve(filename, argv[], envp[]) supplies filename and argv[0] separately, they don't _have_ to have anything to do with each other.)

Sigh. I wanna execve(0, argv, envp). I should poke people in email and make puppy eyes.

The lack of execve(NULL, argv, argp) to restart the current program is one of my longstanding "why hasn't somebody done this already" things. A process should be able to re-exec itself even if it's in a chroot where the original binary isn't available; on a system with an mmu you can fork() and get a second copy, but on nommu you can only vfork() which blocks the parent process until the child calls exec() or exit().

GPS continues. So much GPS. It's another one of those "three pay periods missed and we need to hit customer milestones to get paid" things.

Very tired.

Also, if you cursor down in less and there's no new input yet, it hangs waiting for more input. The toybox version shouldn't do that, it should scroll if there is input but it should also let you cursor back _up_ before then. How would I write a test suite entry for that instead of manual testing? There's like a whole pty framework needed for that...

The 4.13 kernel adds ktls, I.E. basic https plumbing in kernel space. (The w3c standards loonies renamed https to "Thread Local Storage" because too many people knew what the old name meant, since every URL still exploses it to this day. They also tried to rename URL to URI but the only people who noticed were the same ones who say "kibibyte", and they're both crazy.)

Unfortunately, ktls is only about half the plumbing: it handles an ongoing connection but doesn't do the initial handshake, nor does it handle renegotiation if you switch keys after a while (standard practice for crypto stuff, and I think either end of the connection can initiate it). Instead there's an example userspace package that hands that part off to openssl. (So what's the point of having any of it in the kernel? To make the kernel bigger?)

I didn't tackle my own https support in toybox, in part because the Android guys don't wanna audit another https package (don't blame them). Instead I was looking at an stunnel-style approach of piping data through an external program to encrypt it, but that's still on my todo list because stunnel, openssl, and bearssl's command line utilities to do this all have wildly different command lines. There isn't even a de-facto standard here.

If this really did reduce the burden of implementing my own https plumbing, that would solve a real problem for me. But it looks too half-assed, and I am sad.

Last time I attended Texas LinuxFest I visited the ChickTech booth, and got on their announce list. They just organized their own conference, ACT-W or Advancing The Careers of Technical Women.

There are videos online in one of those random fly-by-night services that will go away again in a year, so I should probably watch them and download the ones I like before they're lost to history (like the Flourish videos).

I wrote a utf8 test program to compare my utf8towc() function to libc's mbrtowc() found that musl, glibc,and bionic were _all_ doing the conversion A) wrong B) differently. (I keep thinking "I can't possibly be the first person to test this", and yet...)

Rich took a bug report about musl (reproduced and agreed it was wrong) and I poked Elliott about Bionic in email (dunno how much they care; the utf8 parsing is right, it's the stupid unicode range restrictions that are funky...) And glibc can go hang, I don't care.

Although utf8 is quite elegant, unicode is INSANE. And the main source of insanity is the legacy utf-16 encoding windows does, which started with the Rich White Guy Assumption that 65535 characters oughta be enough for anybody. (There can't possibly be more letters than that in the world, can there? Answer: currently about 120,000 and still going.)

So utf-16 did this funky straddling thing where values in the 0xd800-0xdfff range mean they come in pairs, which can combined with the base set of values it can code to give a grand total of 1,114,111 values, which is where Wikipedia[citation needed] gets 0x10ffff from.

Oh, and you also exclude the 0xd800 through 0xdfff range as -1 (invalid code point). But NOT the 0xfffe and 0xffff that bionic's also calling invalid code points for some reason. Yes 0x10ffff is a strange number, musl was capping at 0x11ffff instead, and bionic at 0x1fffff. (Meanwhile glibc is not just capping at 0x1fffff but also returning -2 "need more data" for some 4 byte sequences because it's trying to parse to the end of the original utf8 range (up to 7 bytes), and THEN say it's an invalid code point. Jazzhands!)

Sigh. Testing and documentation. Never enough of either. Except both seem to require a certain amount of skill to do right. When I asked for toybox tests as a way to let others contribute usefully, I got lots of tests that weren't really testing anything in toybox. (Looping through mknod 000 through 777, all 512 values, is maybe testing the kernel? But it's not checking that the kernel _does_ anything with those values, just that you can write it into the dentry then read it back...) Similarly, this utf8 stuff isn't testing toybox, it's testing libc to see what toybox should DO when reimplementing part of libc because libc's API maintains unwanted state.

You'd think the simple "do a for loop in an unsigned int trying all 4 billion values, and see if you get the right results" would have been done before, but I guess you do have to know what to compare the results against. I wrote another utf8 parser and was trying to see if it produced the right output. Mine could still be wrong if all four implementations are making the same mistake, but they're making DIFFERENT mistakes...

For some reason GPS coding and toybox fight in my head, or at least require sufficiently different mindsets that I have to _not_ do GPS for a full day before I become unblocked on toybox. (I've carved out an hour and sat down and stared at the toybox code and felt utterly uninspired to do anything with it on multiple occasions. Frustrating. Something about GPS puts me in uber-nitpicky mode where I can't write a simple function because it's not PERFECT and I'm sure I could shave a cycle or two off it with a tighter implementation if I just thought through EXACTLY how to code this, and then the result is so brittle you can't change anything... I'm aware this is a failure mode.)

I keep being awake late enough that Jeff's up and active, and we text, and I speculate about some GPS design issue, and get drawn back into it and then suddenly the sun's coming up and my schedule is trashed again and I won't have any luck doing toybox the following day because wrong headspace...

On the bright side, we're making decent progress on the GPS stuff. Which is the main blocker for actually being able to sell products and get proper funding for the company. (Our synchrophasors measure electrical signals in wires at given points. To correlate the results they need to know not just exactly where they are but exactly _when_ each sensor pack saw the signal. So they need a nanosecond accurate onboard clock, which means doing fancy GPS stuff. Right now we're limping along with third party GPS dongles but that doesn't give us the accuracy we want (for a half-dozen reasons, reporting interface latency is the most obvious but by no means the only one) nor is it cost effective for deployment at high volume.)

GPS processing is a realtime problem: the correlators output one reading per milisecond and if you're tracking 4 satellites you get 4 of those skewed around each milisecond, and you can't drop any (must process this one before the next milisecond or it'll get overwritten) and we have to not just record them but process them in realtime so the delay locked loop and phase locked loop can track the satellite's doppler and code phase, I.E. update the correlator registers before the next reading. (The DLL and PLL are mathematician magic I've spent months trying to understand and just _don't_. I can see _what_ they do, I do not understand _why_, and the explanation always boils down to "and these constants were found experimentally" so I cannot GET an intuitive plumbing-level grasp of what's going on. Jeff understands it, but he's a math guy. Rich is at least comfortable with it, his degree's in math. Jen and Ken understand it too. Oddly enough I was reasonably comfortable with the hardware version where a capacitor is damping out a waveform, but the software version where you add some constant times the first derivative and another constant times the second derivative... Um... does not compute.)

I punted on why code phase is cool last time: the signal is an order of magnitude WEAKER than the ambient thermal noise. (Think analog television static or tape hiss, noise that's just "around" all the time because we're 300 degrees kelvin above absolute zero and everything glows in various radio frequencies even if it's not hot enough to do it in visible light.) How do you hear a signal that's massively out-shouted by ambient static? By inverting half the bits (xor it with a pattern) so the noise statistically cancels itself out. Since the signal you're looking for is _also_ inverted the same way, when you line the codephase up exactly the signal adds but the noise cancels.

Problems this solves:

1) Finding really weak signal in the noise, so a solar-powered satellite can broadcast a signal you can hear from the earth's surface.

2) Precise timing. The correlator resets (the 1023 repeating bits start at a known location) on each bit edge, so matching up the code phase tells you exactly when that happens, which lets your timing data be more precise than the signal you're actually seeing. (The bit edge itself being blurry due to noise doesn't affect your timing, you know when it had to be even if you can't see it. Each time you just need a majority vote of "is this high or is it low" to tell what the bit was, and the codephase tells you where the edge was.)

3) Distinguishing multiple satellites broadcasting on same frequency (if each uses a different xor pattern, they're just noise to each other, and it's noise a tiny fraction as strong as existing background noise anyway so trivially ignorable). So a constellation of ~2 dozen satellites can all use the same frequency, instead of each needing a unique one. And ground signals using the same frequency are just more noise easily discarded. (In theory, that last bit has some practical hiccups I should explain later...)

(Again, it's 1023 not 1024 because prime numbers are easier to make a repeating decimal out of. That's also why the frequency stuff broadcasts on is a multiple of 1023 mhz.)

In theory 1023 codephases/milisecond means your timing accuracy is about 1 microsecond (I.E. 1023*1000 samples/sec, which is still three zeroes short of the 1 billion/second of nanoseconds. The speed of light is _really_fast_.) BUT the timing is really accurate (the bit edges are emitted with nanosecond accuracy) so we oversample the signal, listening at 16 times that speed we expect the codephase to change, and then we try applying the codephase XOR to all those and see which gives us the strongest signal. (If it's switching early or late from the edges we're listening to some of our signal becomes noise and cancels out, but losing 3/16th of the signal just makes it weaker, you can still hear it. A properly correlated signal for a satellite directly overhead (least amount of atmosphere it's going through) can be up to 5x the noise floor.)

Another corollary of #3 is this is a really effective way to HIDE transmissions, because unless you match the correlator frequency exactly you can't see the signal at all, not even to tell it's _there_.

So anyway, we're measuring our GPS processing time in microseconds: our SOC design has a nanoseconds register you can just read from a memory location, it's lovely. The code is running on a processor that runs a little under 100 mhz, with 8k of L1 cache, which doesn't do dual issue so at most 1 instruction per clock cycle. Luckily it's SMP, and in THEORY we're only taking up about 60% of one processor for the realtime bits. (We should be able to optimize that down, but that's fast enough it should work and we can debug it.) In practice, the per-cpu clock tick takes 500 microseconds (half a millisecond) so when that happens we get a latency spike that sometimes drops a sample, and if you diable the timer interrupt on any processor it's RCU stall city and the system goes bye-bye. (Rich thinks he might be able to get RCU grace period processing running on a processor OTHER than the one it's processing for. Sadly a tickless Linux system doesn't mean there aren't ticks, they just aren't regularly scheduled...)

Meanwhile, both Rich and prototes customers are testing have some sort of interference that's causing the analog antenna filter stuff to sqelch the volume of the signal we're interested in. (It scales the input amplitude so the strongest signal we hear is 100%, and if that strongest signal is very loud interference...) Neither Jeff nor myself are seeing this interference? That's another thing we need to track down.

Oh look, sunrise again.

So on the utf8 thing, I need to test my new contextless parser function to make sure it's doing all the corner cases right. Since some random standards body broke UTF8 to max out at 4 bytes of input (even though the original proposal could do 7 and thus encode a much wider range), this means all the valid inputs fit in an unsigned int. So I should be able to loop through them and compare them to the output of mbrtowc(). (Remembering to memset the old one's state structure back to zeroes after any <1 return result, of course.)

First thing I forgot is that mbrtowc() constantly fails unless you call setlocale() first. Because of course it does.

Next thing is this is a little endian machine so the order it's going through the values is nonobvious. But if it was the other way around it'd have leading null bytes for the first several million values, so the mapping's weird either way. Important thing is it tests all of 'em.

I think this even tests all the _invalid_ values. (A first byte with more than 4 leading zeroes counts as a -1 error not -2 insufficient data.)

Oh right, redundant encodings! 0x7f is also 0xc180 but the second is an illegal sequence. (To simplify processing utf8 didn't add the "last code point" of the previous sequence to the start of the current encoding, so they overlap.) Which means I need to -1 for values too _low_ for the number of bytes consumed, but they didn't make the last one a power of 2 and instead it's 0x10ffff which is just sad and needs to be special cased.

The hurricane seems to have wandered off. It's lovely outside. I'm not sure whether to trust it yet. My headache doesn't.

I'm exhausted. Niishi-san is on another vacation. I can't remember the last time I had one. (I've switched to different _types_ of work, but I always had something I was doing or avoiding doing, for years now.)

Sat down to try to do toybox work and just... blah. Burned out. (Then wound up sucked into GPS stuff _again_, and awake all night texting Jeff in Japan. Again. This time I got him to log onto IRC, which at least gives me a full keyboard instead of tiny phone keyboard. So that's something. Lots of VOIP calls too. It's a process...)

I just saw a commercial for some random movie (in which Tom Cruz jumps on a couch ranting about scientology with a female mummy), saying "own it on 4k high def with Google Play". Um, that's a streaming service. That's not "own", that's "lease from a service which is going to cloud rot in our lifetimes". There's more than one graveyard of google services, and other cloud services going away and leaving stranded assets is such a regular occurrence it doesn't even make the news anymore. "You can't play Diablo 3 without connecting to the server? That game will go bye-bye someday" should be part of your purchasing decisions.

Paying a streaming service for access isn't "owning" anything. The main advantage open source ever had over proprietary software has always been that the commercial interests behind the proprietary software will stop investing in it someday, and it will become abandonware. The open source thing isn't going to stop being available. The context in which it runs might be; it might evolve, get forked, even wind up restarted from scratch with a different development team responding to it... But it won't leave users still wanting to use it and having no way to do so.

(There have been some headwinds pushing against this, crap like devfsd and x11 needing hald and Red Hat's current "systemd all the things!" push, but those of us who dowanna can generally wait them out. There's other problems of unwanted sequels (GPLv3 driving everybody away from GPLv2, Python 3 driving everybody away from Python 2...) Reality is complicated. Reiserfs didn't survive Hans Reiser's murder conviction, and node.js is apparently having a bad week. But Erik Andersen took over BusyBox after Debian abandoned it, then I took it over from him when he didn't have time anymore, and I handed it off to Denys Vlasenko when I got too disgusted at the troll who named it to keep working on it. There was no "business case" for any of that (it was independent of capitalism), and I've still never met Erik, and only met Denys at one convention. Similarly I maintained my own tinycc fork for 3 years (before running out of time, but if I really _needed_ it I could dig it up and resume work on it, or somebody else could). It's a "can" vs "will" thing...)

The musl maintainer broke toybox chrt.c, and did so intentionally. He doesn't like what one of the Linux syscalls does so he made the syscall wrapper (and like five others) return -ENOSYS. Here's the commit that did it.

Trying to figure out if I should switch everything to syscall(BLAH) to work around the musl bug, or just say "chrt doesn't work on musl". Leaning towards the second. This is 100% a musl bug.

Hurricane Harvey is driving my sinuses nuts. We're right at the edge of the thing, so the spiral arms of thunderstorm keep going over us, and then the clear gaps between them, so the barometric pressure is BOUNCING.

House hasn't flooded yet. The concrete wall, french drain, and landscaping changes (two trees removed) are collectively holding. I dug a little trench in the french drain's gravel so the water from the downspout at the most problematic corner has a nice clear path to go AWAY from the house. (It's washed all the dirt under the downspout away to expose brick. I didn't know there were bricks under that. I guess the french drain installers put it in.)

(One of the trees we removed was a crepe myrtle, which is trying very hard to grow back. the stump was grated to sawdust a foot below ground level, but what's left is sending runners out all over the yard which I've been regularly plucking for something like a year now. Every few trips to HEB I get another couple salt cannisters and pour salt on where the stump was. That reduces its enthusiasm slightly, but hasn't come close to killing it, and any significant rain washes the salt down the storm drain again. So those legends about "salting the earth" when cathage delenda est seem somewhat exaggerated, or perhaps merely never had to deal with an invasive ornamental plant people in previous decades thought it would be a great idea to plant all over the neighborhood. (See also Austin's "cedar" infestation.)

Oh well, salt's cheap...

Southwest was kind enough to let Fade move her flight back to Minneapolis up a day, to avoid the hurricane. I walked to the grocery store as she was packing up to grab more caffeine (my schedule is flashing 12:00 with all the nights walking to UT and talking to work's Japan office on VOIP, but trying to be awake days for other stuff. A sleep schedule consisting of multiple 3-hour naps isn't necessariy sustainable long term...)

Anyway, there was a positive DELUGE trapping me at HEB when I tried to head back, so Fade got her suitcase and the dog into the car and came and picked me up, and we went to the airport from there. And after I dropped her off... the sky was clear again. And has remained so ever since.

This is a very odd huricane so far. I guess that's what you get when you name them after six foot tall invisible white rabbits.

It's difficult for me to explain just how craptacular lsof is, at least judging by its 2600 line man page.

The basic idea of lsof is clever: it doesn't just check all the /proc/*/fd entries to see what files processes have open, it also checks their "exe", "cwd", and "root" symlinks, and their memory maps to see if anybody's mmaped that file either. It also looks under /proc/net at the files tcp, tcp6, udp, upd6, raw, raw6, unix, and netlink, which tells where open sockets point to (so you can describe the /dev/*/fd entry of a process when it points to a socket).

Unfortunately, the user interface is _insane_. It shows signs of a long history of unconstrained development on a bunch of different fragmented unix variants, maintained by people who glued bits on at random and never said "no" or tried to be consistent about anything.

There's a -b option to avoid using "kernel functions that might block", namely lstat(2), readlink(2), and stat(2). There's ALSO an -e option that skips those for a specific filesystem (although it notes it's path-based, not matching major/minor, so caveat emptor). Except that only skips stat(2) and lstat(2), to skip all three you need to use +e instead. (Lots of arguments have + and - versions, sometimes doing completely unrelated things. For example, -c specifies which command names to show, and +c specifies how many letters of command name to display in the output. Except -c matches the start of commands (so if I say -c b it'll show me all instances of bash and bc and so on). Oh, and you can go -c /regex/ to do regular expression matches.

A lot of commands have _optional_ arguments. How this is parsed I couldn't tell you. If you say -s by itself (or something like "-s -a") it "directs lsof to display file size at all times". But if you say -sTCP:LISTEN it filters for TCP sockets in LISTEN mode. Of course the part after the colon can be a comma separated list of states including things like "FIN_WAIT_2" and "SYN_RCDV". (This is just "some common TCP state names", not an exhaustive list.) For some reason, the TCP states are all upper case and the UDP states have mixed case names like "Idle" and "Unbound". For more information, I'm expected to read the lsof FAQ.

Did I mention the man page is not complete? There's an LSOF FAQ. It says that some other section of the giant man page says where to find it, and I don't care and am not reading it.

There's a -C option to disable use of the "kernel name cache", something I don't think Linux has. There's also -D to disable use of the "device cache file", which again would do what on Linux exactly? This one requires a function letter: b says to build the device cache file, i says to ignore it, r says to read it, u says to read and update it, and ? says to report the device cache file paths (of _course_ there could be more than one). This is a unconscionable level of micromanaging crap to do something dubious that's probably specific to an operating system that stopped being produced in the 1980's.

A positive _masterpiece_ of crap is the -f option, which "by itself clarifies how path name arguments are to be interpreted". It has 7 paragraphs and 2 tables of explanation, can take 5 different letter arguments "in any combination" (including none of them, in which case they recommend -- after it to prevent the next argument from being taken as a parameter), and comes in - and + variants to enable/disable what it's searching for.

The point of -f _seems_ to be that if you specify a mount point as one of the arguments, it automatically notices it's not a regular directory and shows you everything in that mount point. So lsof /home shows nothing (because no program currently has that specific directory open, or as its chroot root, or as its cwd). But lsof /dev shows 461 entries (basicaly open device nodes, mostly /dev/null, /dev/tty, /dev/ptmpx, and /dev/urandom). So if /home _was_ a mount point, it would behave differently (showing everything open uner it, like +D does), but it's not (at the moment) so it doesn't.

That isn't what -f does, the mount point vs not behavior varying is just what lsof does automatically. The point of -f is to _modulate_ that behavior.

Needless to say, the lsof the Android guys sent me isn't treating mount points differently from regular directories. I have yet to find a syntax to make a specific argument recurse when others don't, but I'm not to the end yet and wouldn't be surprised.

I'm trying to figure out what subset of this crap is worth implementing, and what parts I dowanna do. Some is sort of maybe edge-case useful once in a blue moon? The -T option (which takes arguments) displays extra TCP information: queue length, socket options, flags... I'm pretty sure I don't care? Not enough to implement the arguments -T takes specifying WHICH of these things to display.

If it worked like "ps" where you have -p blah,blah,blah where each blah is a magic name tied to a data field, I could add a giant table with all the things and then let you specify what fields you wanted in which order and I could GLUE THIS TO PS AND REUSE THE LOGIC. (It's more code that grovels around in each /proc/$PID directory, seems like a natural fit.) But the command line argument syntax is totally different and the organizing principle is a list of files (where PID is basically an attribute of the file).

Sidelined by a pulled muscle, spent half the day in bed trying not to move. (Thought I'd bruised my tailbone but no, putting more weight on my right leg than my left, even for a moment, is a REALLY bad idea right now.)

In other news, Ibuprofen is a wonder drug.

Still banging on lsof.c. (Under what circumstances does it _not_ have to read everything up front before producing any output?) Well, alternating that and GPS. Jeff made GPS sort of work! Well, bits of it, using software corrlators significantly slower than real time.)

An unexpected benefit of marriage is that if you have a sudden attack of intense intestinal distress having walked an hour away from home late at night, there's someone you call to bring a car full of towels, who is then willing to throw out a pair of shorts while you huddle in the shower being really miserable. (Cell phones are also a wonderful invention. The smart phone part even let me look up and call the UT after-hours maintenance people and apologize profusely for the state of one of their bathrooms.)

All considered, not my most productive evening of programming.

Ok, I went back on caffeine. Too much to do, and that's about as much zombie-ing as I can afford right now.

The big push at $DAYJOB is eating my brain (still no end in sight) and I need the occasional palette cleanser, so I've been poking at lsof cleanup (which has nobody waiting for it, and is thus relaxing). But as with the rest of pending, it hasn't been promoted yet for a reason.

I'm confused about what people _expect_ out of lsof. There's no spec for it, and I'm not sure what it should do. I use lsof -i4 and -i6 which the one Android submitted to toybox doesn't have, but the upstream one was written many years ago (at Purdue university) for non-linux systems, and has a 2600 line man page full of crap like:

To find processes with open files on the NFS file system named /nfs/mount/point whose server is inaccessible, and presuming your mount table supplies the device number for /nfs/mount/point, use "lsof -b /nfs/mount/point".

No. I am not implementing special support for nfs. I have todo items to implement v9fs and smb servers in toybox, and I'll probably do an "nfsmount" wrapper, but even that hasn't been a priority.

Both Android's and ubuntu's lsof are reading all the ps data for all processes up front (which takes 40 seconds on my netbook) before producing any output, which seems crazy.

Bunch of find bug reports through github. I still have pending stuff on ping, dd, and cp --parents.

Sigh. If I'd been able to work on this the past 18 months at the rate I did in 2015, I'd have the 1.0 release out by now.

I was totally burned out and going "I'll just take today off, it's Saturday, won't even carry my netbook with me..."

And Jeff fixed GPS. Position fix within 2 meters, derived from MIT-Licensed code. So I need to look through Jeff's new thing, which is the _fifth_ GPS codebase I've poked at, not counting the code I wrote from scratch or Geoff's hardware correlator acquisition stuff.

This new hamsternz project has two implementations, the one big file version at the top level, and the 300x faster version in a subdirectory that's chopped up into tiny little .c and.h files so you have to choose-your-own-adventure your way through reading it. I question the design choices here: one-big-file-but-slow and many-little-files-but-fast are orthogonal issues, surely? (When I first started poking at tinycc it was one big .c file ten thousand lines long. Ok, that's an extreme case ala Berkshare Hathaway never doing a stock split and hitting a million dollars per share, but still. The two versions of hamsternz have _two_ big differences, and I'd really like one from each pile. Oh well.)

I'm glad we have something that works (albeit needing serious surgery to fit on the hardware, run fast enough to matter, work consistently, and stay up longer than 10 seconds), but I'm _tired_.

Poking at mkroot integration of native toolchains, and the next problem up is packaging, I.E. what goes in which filesystem.

Musl-cross-make's native toolchain extracts to a couple hundred megabytes, so if you add that to the initramfs a qemu image with 256 megs of ram can't extract it. So we need to package it into a block device filesystem we can feed in to /dev/vda or similar. In Aboriginal Linux, I used squashfs.

I haven't currently got mksquashfs in the airlock directory (no host-tools), so I need to build it. And there's an organizational issue: the mksquashfs build needs zlib, which I'm currently building as part of dropbear. The download and build of the package are both in modules/dropbear. I'm trying to build a host version, so having a different build is ok (if not ideal), but having a redundant download makes me a little sad.

In Aboriginal Linux I factored package builds out into sources/sections, which makes it easier to share them but harder to follow what's going on because the order of operations jumps around. One of the big advantages of mkroot is it's a simple script that runs stuff in order. It has includes, but each one is "now run this entire script start to finish", and they're all at the end of the first script. (I'm thinking of factoring out most of the mkroot.sh script and turning into modules/root so the top level ONLY calls modules, but having a top level script you can copy into an empty directory and run and it does what it says on the tin... don't want to give that up either.)

But this "run in order, with all dependencies local" design means if you have to do something twice, you wind up duplicating it. Such as both dropbear and mksquashfs needing zlib, with neither depending on an external zlib build.

*shrug* It's a conflict, you either duplicate stuff inline or you factor it out. Both have downsides. The definition of "simple" has multiple local peaks, and sometimes they exchange gunfire.

Got a feature request for cp --parents, which on further investigation is full of corner cases.

Note: if I haven't responded to a toybox issue after a week, ping me again. If I then don't get to it for another couple days, my policy is to apply your patch as is and then it's on me to clean it up later, without blocking your use case.

Sorry I've been so out of it. Working for a startup on life support it's crisis du jour, you look up and a year's gone by...

There's a s390x user trying to make qemu work under Aboriginal Linux. Cool! I pointed them at mkroot.

Alas, mkroot is the kind of project where you don't necessarily hear back from people because once they've got a kernel config and toolchain that boot a system to a shell prompt on their hardware (or emulator), they can take it from there. It would be nice if you could natively bootstrap things like debian, gentoo, or even buildroot under mkroot just going "I have a shell prompt and a build environment, now build packages with this". But that's the distro hairball problem which I still need to solve. Most notably for AOSP.

That's why I'm trying to get _both_ musl-cross-make and Android's NDK to provide sufficiently capable cross and native toolchains I can both rebuild mkroot and build Linux From Scratch under the result. Orthogonal layers: mkroot should work with multiple toolchain sources.

I keep coming back to Linux From Scratch as the lowest hanging fruit of the distro hairball problem. Building it probably gets us about halfway to solving it for debian (both because it forces toybox to fill out all the corners and because if something _is_ missing LFS has probably built one so you know how to bootstrap up to a more complex build environment for debian). It's maybe 1/5 of the way for AOSP, which can't just swap in a gnu tool (no gpl in userspace, if I want it self-hosting I have to write my own mini-git for repo to clone/pull stuff with).

As for other distros, Alpine Linux might be the next lowest-hanging fruit, not sure. Buildroot is a tardis console that might already be able to do this but I've never figured out how, Gentoo's problems are self-inflicted and not hugely interesting for me to solve, Arch is for people who think Gentoo is too soft on newbies and thus full of "those people" who didn't truly EARN their Linux system, and Red Hat is too corporate to do anything with in a hobbyist context. (And yes, I say that while putting AOSP on the _other_ side of that line.) I've poked at SuSE before but that whole "open build" thing (which I used at Cray in 2013) was way too complicated...

Anyway, that whole struggle is a can of worms I haven't dug my way to yet. LFS first, which has its own giant dependency list to get there...

From today's email:

> I have a RFID Reader with BusyBox on board. I need to execute a stored
> procedure on SQL Server with Java, but from BusyBox I can't execute my
> code. Have you ever tried to do this? On this BusyBox (v 1.14.3) has
> installed JamVM.
> 
> Could you help me?

I handed off busybox development 11 years ago and I still get these. Busybox is seldom actually involved in the problems they describe, and I'm generally not sure how much backstory I have to explain to get them to the point they're even asking the right questions.

Alas, I'm a sucker for help requests, and spend time trying anyway. Here's my stab at it:

Busybox is just a set of command line utilties, mostly implementing the Posix and LSB specs. It's not an operating system. Your operating system is Linux, running on some kind of hardware. (If you run "uname -m" it should tell you what kind of hardware. If that's not there, "cat /proc/cpuinfo". Embedded systems are usually arm or mips, but sometimes they're x86 or powerpc or something else.)

To run userspace programs, you use a C library. The "readelf" or "ldd" commands can usually tell you which one. It might not be installed on the system if your binaries are statically linked, in which case you'd have to copy a binary to another system and run the command on it there. For example, on a normal linux system, run:

readelf /bin/ls | grep interpreter

That'll probably say something like:

[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]

Which means your C library is glibc for x86-64.

To run _any_ binary on an embedded Linux system (such as busybox is usually installed on), you need to build it for the right hardware type, and either statically link it or use the same C library as the target. (If you build the new binary with the same compiler toolchain the system was built with, it should all match up. If not, you need to match it yourself.)

Next question: you said you need to "execute a stored procedure on sql server with Java". Where is this SQL server running? (On the RFID reader?) Which SQL server is it? (Postgresql? Mysql? Oracle? The one actually named "SQL server" is a microsoft product, generally not something running on a Linux system, much less an embedded one.)

Are you trying to get the busybox rfid reader to do a network transaction to a server somewhere, running an sql server and a java runtime? (Which java runtime? Oracle's?)

I strongly suspect you need to talk to a domain expert here. I don't really do databases much.

Sigh. I should add a toybox FAQ entry for that, I suppose. Most of our users are through android but there's some in the embedded space, and it should pick up once I've got a toysh worth using. (Right now it doesn't quite complete the circuit as a useful standalone package. That's what my mkroot project is for, create a simple kernel+libc+cmdline system, I.E. linux+musl+toybox, booting to a shell prompt in initramfs under qemu.)

Asked the kernel guys about the Gratuitous Ping Exclusion (with patch to disable it). So far, crickets chirping in way of reply...

Did a git "clone linux clean" and the new clean doesn't have a branch master. How...? (The linux repo had an older version checked out, but how does that matter?) Had to do a "git checkout origin/master -b master" to get one, and no idea if it'll track.

Seriously, this software was developed by people who saw the Tardis console and went "what a good user interface, let's do that". (Minus the whole telepathic labeling part.)

Also, there's some kind of N^2 algorithmic inefficiency somewhere in "git bisect". On this netbook, "git clone" of my linux-fullhist repo took 2 minutes (22 seconds user, 11 seconds system time, the rest I/O bound; I can tell from the CPU meter it's not pinning a processor). But then if I "git bisect good c319b4d76b9e5" and then "time git bisect master" it pins a processor the entire time and the result is:

real	56m47.585s
user	47m1.816s
sys	0m24.704s

An hour. Wheee. (Most of the difference between user and real isn't I/O bound, it's me doing other cpu-intensive things during that hour. Again, stock ubuntu 14.04 with updates applied.)

So the second bisect is a little over 23 minutes, the one after that 6 minutes, and the one after that 2 minutes. Its job is to figure out which commit is halfway between "good" and "bad".

Of course the kernel developers don't use a fullhist repo due to nitpicking over commit hashes that aren't even in the current tree, so presumably they haven't hit this yet? Or maybe they just all use 8-way SMP 3ghz processors with with 2 megs each of L2 cache and 16 gigs of DDR4 memory. People with less need not apply, I suppose.

The busybox commands mkroot still uses are ping, route, tar, wget, vi, and sh. (Not counting the compression code only enabled because tar needs it; busybox has gz/bz2/xz decompressors and a half-finished gzip compressor.) Those first four don't seem like _that_ big a deal, so yesterday I thought I'd take a quick look at ping, because it's pretty simple, isn't it? Why haven't I already done this one?

Generally, the commands I haven't done yet are the ones that have some sort of can-of-worms design issue once you scratch the surface. In the case of ping, it's seamless ipv6 support. By seamless I mean you should never have to specify -4 or -6 (unless you have multipath and want to force one), it should autodetect. The fiddly bit is you can specify both host and target addresses (which interface to bind to locally), and it's possible that the first one you look at has both ipv4 and ipv6, but the second only has one of those, so you have to defer the decision on which address to use until you've looked everything up.

Banged on it for a few hours until I got "ping -I ::1 127.0.0.1" and "ping -I 127.0.0.1 ::1" giving me reasonable error messages, autodetecting types properly when the match up (or when there's just a target), and letting -4 or -6 veto certain paths. Then I tried to open the port.

Back in 2011 Linux merged ICMP Sockets, which I've meant to use in this all along. To implement ping you used to have to open a raw socket, which requires root access (because you're supplying the full IP header yourself and can fake source addresses or make christmas tree packets and generally bypass all the filtering). The whole point of ICMP packets is to let you implement ping without requiring root access.

But when I _tried_ on my Ubuntu 14.04 host kernel, I got permission denied. And reading more, I found out about the Gratuitous Ping Exclusion, I.E. the fact /proc/sys/net/ipv4/ping_group_range defaults to "1 0" (lowest group that can use this 1, highest group 0), so even ROOT can't use it by default. This is the reason nobody uses this infrastructure and still just deploys suid ping binaries instead.

There's no obvious reason for the Gratuitous Ping Exclusion. If the new ping code in the kernel is exploitable, letting people test it and find OUT in the half dozen years since it went in might be nice. If you do want this kind of restriction, that's why selinux rules are turing-complete (and skynet will be written in them).

I googled and pretty much everybody who enables it turns it on for all users anyway. (Although they disagree about whether that's 0-65535 or 0-0x7fffffff.)

Reminded of the longstanding mount -o remount,ro bug, and this time I set aside a block of time to fix it, which turned out to be two issues (basically it never should have worked in the first place, and the fact it did was its own bug).

But along the way, of course I dug up more design issues, at least things I want to extend mount to do that aren't immediately obvious how to implement. (And I still need to implement nfsmount/smbmount/v9fsmount which all boil down to "prompt for password and feed it in without making it show up as a -o password on the command line visible to other processes in ps".)

Meanwhile, the serial bug in qemu-system-sh4 (where the kernel started depending on a uart buffer that qemu never implemented) still isn't fixed. I have a kernel patch but mkroot builds vanilla packages unpatched and current vanilla kernels don't work. I keep raising the issue and it keeps getting ignored.

The reason people think sh4 is superh is that sh2 was nommu, and then Hitachi had a plan to release a new architecture each year so sh3 was only on the market for a year before sh4 came out. Then the handoff from Hitachi to Renesas (because the 1997 asian economic crisis cut their chip design budget so they spun it off to cut costs) froze sh4 in stone, because Renesas got the design but not the engineers who made it. And the new Renesas engineers wasted years creating an sh5 design nobody wanted (a radical departure from traditional sh that lost most of its advantages), then gave up on superh and started doing arm instead. So that's why qemu-system-superh is called qemu-system-sh4. (Meanwhile, I suspect qemu-system-x86 being being called qemu-system-i386 is because Intel's marketing department paid somebody off. AMD pushed x86 as a generic name to Intel's trademarked i386 with the i standing for Intel. The new 64 bit architecture is called x86-64 because it was designed and thus named by AMD, not Intel.)

Cleaning up dd I got to the "seek" and "skip" options, finally working out the mnemonic for remembering them. (You _must_ seek on stdout to preserve existing data, stdin you can just read the data and discard (skip) the blocks. So seek= only works if stdout is seekable, but skip= works whether or not you can lseek(). I've had to check the man page every single time I've used those suckers in the entire time I've known about dd, which is emabarassingly long.)

This means I need a consume() function to lseek() past input and fall back to reading through it if we can't seek (because zcat | dd). Which brings up "should this be in lib/lib.c" which made me think of tail.c already doing something like this. I've always been embarassed about tail having two complete codepaths for the seekable and nonseekable cases, but without a seek fast path doing tail on a gigabyte log file can take minutes from a slow disk. The performance difference is big enough that _not_ having a second code path is really noticeable, and almost unusable.

However, tail() couldn't use consume(). The reason is we don't know where to seek to, we have to seek to the end of the file and then read _backwards_ to find the appropriate number of lines to display. On the forward pass we have to look at all the data and remember it. Sigh. So less useful as generic infrastructure.

Burned out badly on the GPS stuff, Jeff talked me down. I really hate this kind of signal processing, you keep going down cul-de-sacs where it looks like it was working but then you find it out it was fundamentally never right and you have to rip out and redo the bits you thought were working YET AGAIN.

(Also, the 1995 GPS spec is badly written. The In Nomine core rulebook is clearer and better organized than this.)

And so my fourth attempt to clean up dd.c has started. This time instead of reading the posix spec and trying to implement something sane (and throwing it out once I've been called away from my half-finished work for more than a month and no longer remember where I left off), I'm instead doing small incremental cleanups to the code that's there.

The existing dd is wrong, missing large chunks, and pretty much untestable (how do you check what ibs and obs are doing when pipes collate writes so the only way to see the transaction size is under strace)... I'd throw it out completely but it's one of the commands Android grabbed out of "pending", and I don't want to break them.

I tweaked atolx() a while back to get it closer to what dd.c needs, but there's still 'w' meaning word missing. Add that to lib.c and then I can yank strsuftoll(). (Which is basically atolx_range() except it's non-obvious because "def" is functionally "min". Sigh...)

The other difference is that strsuftoll() returns unsigned long long instead of signed long long. Given the ranges being used in the callers, that only matters for count, seek, and skip, but all of those are filesystem lengths: the maximum file length on ext4 is 16 terabytes and signed long long goes up to 8 exabytes, so standard LLONG_MAX seems appropriate.

There are 6 users, and that means the only remaining difference is whether the smallest allowed value is 0 or 1. Tempted to make a wrapper for that, but small chunks: let's check in what I've got first.

The space guys want another chunk of my time to help debug something, and this time I can work remotely. They fedexed me a board and put a VM image up on an ftp server. Given that work's 2 paychecks behind and will probably be 3 next week, I could use the money. (We haven't dipped into the second half of the home equity loan yet, but it's getting uncomfortable.)

It's been 6 months since I last did anything for them, which means the contract with the consulting company I was going through has expired, so they can send me a direct contract. This also means I can ask for a higher hourly rate, although I only bumped it up $10 (when the consulting company was probably charging double what I was getting) because inertia and I don't plan on doing it long.

Sigh. I really want to finish this GPS stuff, it's a prerequisite for most of the rest of our technology. (For prototypes we're using an external GPS dongle, which both doesn't give us the precision we need and is too expensive to deploy in bulk. We need to process the GPS signal ourselves, from raw antenna data synchornized to our own thermally stabilized clock, and although the hardware's been ready for a bit the VHDL and C code has noticeable gaps in the data pipeline.)

And I really need to spend more time on toybox. I need to get android self-hosting, and I've made no significant progress on that since new year's.

And I need to get mkroot past what Aboriginal Linux was already doing 3 years ago. (Heck, _to_ that point. To do it right I need to write my own make and distcc, and replace the remaining busybox commands with toybox implementations. And that's months of work right there. Even doing it _wrong_ is a solid month I haven't got...)

But yeah, new timesink stacked on top of all that. Keeps the lights on.

The only thing unique about the battery "gigafactory" is the name. Imagine if motorola had created a cell phone "gigafactory" 20 years ago to manufacture its then-popular star-tac flip phone, and advertised about how we'd all be replacing our landlines Real Soon Now and it was a BIG DEAL. Instead it just happened, with another revolution (smartphones) building on top of it immediately after, and sure everything changed but motorola was a bit player in those changes.

Now we've got electric cars, with app-summonable self-driving car subscription services building on top of that, but there's no one company behind it. We've got entrenched buggy-whip interests rigorously defending the Old Way the same way the RIAA and MPAA desperately fought against MP3/MP4 and streaming media, and this time the industry being displaced is 1/6 the entire economy so they didn't just lobby the government, they hacked all those electronic voting machines everybody was warning about since the Cheney administration, and now they've made the CEO of Exxon Secretary of State. This is not _subtle_.

In the long-term, the Fossil Fuel Freakout is unlikely to work better than the CDROM tax did, but what stopped RIAA president Jack Valenti from pearl-clutching about every new media technology since he compared the VCR's ability to record televsion programs to the Boston Strangler killing women in his 1982 testimony before congress (no really) is that he died in 2007 at age 85. D-Day was 72 years ago, the Alleged President turned 71 a couple weeks back. Current US life expectency in the USA is 76.5, expected to go up to 79.5 in 2030 by people who expected there would still be a health care system.

So this problem should solve itself over the next decade or so, but it's really gonna suck getting there. The Baby Boomers who worked _miracles_ as teenagers (woodstock, the moon landing, the internet, stopping the vietnam war via political protests) are now showing us what it's like when the same demographic bulge all hit retirement age and start ossifying into loons. They'll run out soon, but in the meantime drowning dinosaurs climbing on top of other swimmers is the new normal. Render them obsolete and wait the rest out.

Huh. Back when I was wrote about Russia I totally got syria's location wrong. They're at the east edge of the mediterranean. I thought they were basically Morocco. (I.E. Russia's camping the Suez Canal instead of Gibraltar.)

It's easy to forget how big Africa is, and how the same maps overstate the size of russia due to showing a round planet on a flat map being a hard problem. (There was a west wing episode about this.)

If you want to know what ubuntu package a command came from, "dpkg-query -S $(which $COMMAND)" is the magic invocation to do that.

One of the blockers for the dd cleanup is that atolx() is treating the suffix "b" to mean 1 byte, but dd wants 512. So ripping out dd's bespoke suffix parser to use the generic one in lib needs to harmonize that. So I go on a dive through the code trying to figure out where "b" got added (some command needed it), which brings up another trick I call "peelback annotation". Do a "git annotate lib/lib.c", hit forward slash and search for your filename (or hit colon and type in the line number; it's "less" so both work), then highlight and right click->copy the commit hash on the left of the last time the line you're interested in was touched (you probably have to cursor right first to find it), then "q" out of less, and "git show $COMMIT" to see what the change was. If it's not an interesting change, "git annotate lib/lib.c $COMMIT^1" where the ^1 on the end of the commit has says "show me the parent of this commit". (In a merge commit the "1" would say which parent to show, you could do 2 or 3 or so on for an octopus merge. No, I dunno why it isn't zero based.)

Repeating that a bunch of times gives me commit b8ef889cbfae (the current vogue is to show 12 digits of commit hash), and that commit says I added it when implementing "od" back in 2012. Except... the od man page in ubuntu 14.04 documents the "b" suffix as meaning blocks (I.E. 512 bytes). Did it change since I implemented? (Answer: quite possibly.)

Looking around, head, tail, and find... all use b to mean 512. Truncate doesn't specify a "b" but does have +-<>/% prefixes...

I need to use mkroot as a test environment for toybox, and the current low hanging fruit there is netcat, for which I have a longish todo list.

Last time around Aboriginal Linux did native builds of things, but I haven't set up native builds yet. I have to cross compile make and distcc, but when I poked at building distcc I found that project bit-rotted autoconf into a hard build requirement when it moved to github, so I need to abandon that implementation and add it to toybox instead.

I should try adding make, though. It's slow, but it's something. The last gplv2 release is getting a touch creaky, that's _also_ on the "add to toybox" list. (Well, it was on the "add to qcc" list but that project's gone all blue-sky wouldnt-it-be-nice instead of anything I can schedule right now. My time is not my own these days. Sigh.)

I keep trying to update things like my patreon, this blog, and cutting releases of toybox and mkroot, and each time I try the cleanup work to get it ready spins off a bunch of tangents and todo list items.

This has been my working style for well over a decade, but the problem _now_ is every minute taken away from that is another minute I'm not bashing my head against GPS, and my company is starving to death because we need GPS working as a hard blocking requirement to ship product to customers.

Also, since I dunno when I'll get sucked away from stuff and for how long, I'm relucant to check in partial results. Including emails going "oh yeah, I see the problem, looks like a simple fix" without actually doing the fix first. (Half the time it turns out _not_ to be a simple fix, and then I haven't replied to the issue _or_ fixed it when I get yanked away for who knows how long.)

The downside of long interrupts being when I come back I have to reverse engineer my own work to see where I left off. Even if I remember what I meant to do and why, how much of it is actually on the page and how much is just in my head that I _think_ I did but didn't yet? (I'm spinning too many plates to remember the current state of any of them with enough accuracy for more than an hour or two. There's 176 toybox commands in defconfig, and I've got a dozen locally modified but not yet checked in...)

Only way to clear the backlog is to keep chewing. Working on it...

In order to make a functional GPS system, you need to be able to perform something like six stages:

1. read data from live board antenna (w/timestamps of when we saw each reading).
2. find satellites in the noise (acquisition)
3. track satellites as their doppler/phase changes (ongoing: they move fast)
4. Extract bit data for each satellite into 50 bits per second streams
5. packetize and checksum bitstreams into frame 1-5 cycles
6. collate 4 sattelites to solve for x/y/z/t

In theory all these steps are documented in the official GPS specification. (With various important details buried in other documents.) In practice, it's still kind of hard.

We have an implementation of all this in scilab, and it works fine, except that when you run it on an 8-way server with 16 gigs of ram it grinds away for half an hour before giving you the output from 30 seconds worth of input data. It's doing fourier transforms using x86-64 floating point hardware, and we need to get all this working on a 62.5 mhz System-On-Chip (with various coprocessor hardware) in realtime, and it can only eat a certain chunk of that thing's resources because the board's got a lot of other stuff to do.

Steps 1, 2, and 3 all make use of GPS tuning hardare called a "correlator". We sample the input signal really fast (16x the rate of the signal we're looking for, there's an off-the-shelf chip for that costing less than a dollar each), and then run it through a piece of hardware that looks for sine waves at a given frequency. Except GPS uses this funky thing called a "code phase", where it has a PRNG (pseudo random number generator) that repeats after 1023 bits of output, and the sine wave we're looking for is XOR-ed with this PRNG's output bits. (This is actually really clever, for reasons would take a while to explain. Doing it solves multiple problems, but for right now just accept it as a thing we need to do to interpret the signal.)

And the sine wave may be 180 degrees off from where you expect (mirror image) because it's symmetrical and when your start determines what's up and what's down. (Oversampling by 4 means each sample is the same signal rotated 90 degrees.)

The 6 steps above are a bit fuzzy (overlap and bleed into each other) because this is complicated stuff: we need nanosecond-accurate timestamps to attach to raw input signals read from the antenna, and then we have to marshall them along with the data as it's transformed through "correlators" that tune into the frequency of interest (and apply a code phase to it).

We have dedicated hardware for doing #1, and in fact one of our coolest patents is doing it in about 1/8 as much hardware as usual because Jeff and Jen are very clever. I wrote code to do #2 and #3, but it's not sensitive enough (possibly due to a bug in my software correlator implementation) and Jeff insists we must throw it out and take a wholly different approach. (*shrug* Ok.)

The first thing I wrote on this project was code to do #4, then #5, using scilab output.

#6 is math. We were trying to use rtklib for this way back when, but it's a bunch of dispirate functions that don't connect together, with no example code showing how to use them. We've since looked at like 6 other implementations, all of which are crazy/broken in one way or other...

Apparently most of the existing GPS implementation out there are by like the same 2 people, hired as consultants over and over. There's another guy (Andrew Holme) who hacked together his own implementation that actually works, but half of it's in an FPGA and half in a raspberry PI (as in the software passes data back and forth so it's REALLY hard to follow what's doing what) and the code he released is GPLv3 so nobody wants to use it (and we _can't_ ship that into electrical grids, no utility would allow it and we couldn't support it. That license tries to say "authors of this GPLv3 code must be able to upgrade the electrical grid infrastructure, the utilities who own the hardware can't stop them if they pay an electric bill" because Stallman was trying to Free The Cloud or something, and wound up instead excluding his new generation of software from a wide range of uses. Wheee!)

Jeff and Jen understand all the math, there's multiple textbooks out there on how this 1970's technology does its thing. But you need a working reference implementation to test ours against, and if there are a thousand steps having the output of the last one doesn't tell you what the answer to step 22 should be. The failure mode of signal processing is "your output is nonsense static, and you dunno why".

I just got notification of another "payroll delay" at $DAYJOB yesterday. We've been on half pay for a year now, and the paycheck on May 26th didn't happen. We got paid 2 weeks later, but didn't get the missing one filled in. And now they're missing another. They say they'll get new money from a customer on the 1st, but another paycheck's due that friday at which point they'll owe me _three_ paychecks.

Sigh. I really want to see the technology I've been working on since 2014 launched out into the world, we get to open source interesting chunks of it (including time to work on toybox when we're not doing the this-is-fine thing surrounded by fire) and I'm even more excited about it since I've been researching the current state of solar technology. But this is "wheels coming off" territory on the money side. I got a home equity loan to tide the household over through this, but if we spend through that money without the company recovering I can't _afford_ to stay here.

But I _also_ want to turn Android into a self-hosting development environment and the past 18 months have been a huge distraction from that. I sometimes idly wonder if I should switch to focusing on patreon instead of $DAYJOB, but even half pay at dayjob is paying dozens of times what patreon is. It's flattering and encouraging and I'm grateful people care, but I can't remotely live on it. (And I'm supporting two people, three cats, and a house with a yard, not currently in a "ramen and roommates" stage of my life. Although Fade's pretty self-supporting with her scholarship and dorm room now, so if I _did_ sell the house... what to do about the cats though. And Fuzzy doesn't want to move.)

The official Global Positioning System Specification is a 46 page PDF full of incomprehensible math in small print, but what it describes is actually quite cool.

Back in the 1970's the US launched a couple dozen sattelites with atomic clocks so they know EXACTLY what time it is, and the government launches replacement satellites when any of those wears out. (We're on like our 4th generation of the suckers now, but they're backwards compatible.)

Each GPS satellite constantly broadcasts a signal telling you what time the atomic clock onboard the satellite thinks it is, with a 50 bits per second data protocol cycling over and over through 5 different data packet formats ("subframes"). Subframes are 300 bits long, so each one takes exactly 6 seconds to transmit. One of the data fields in each subframe is a timestamp, which says EXACTLY what time it will be (to the nanosecond) at the rising edge of the first bit of the next subframe. (At the tone the time will be... BEEP.)

But that's just the time the satellite thought it was when the signal left it. The signal moves at the speed of light, and Admiral Hopper taught us all that a nonsecond is 11 centimeters at the speed of light, so it takes a whole lot of nanoseconds for the signal to get from the satellite to our receiver on the ground. (Except 11cm/sec is the speed of electricity in wire, the speed of light in vacuum is 30cm/sec, meaning if your time's off by a milisecond your position is off by 300 kilometers. If you want to know your position within 10 meters, you need to know your current time within plus or minus about 15 nanoseconds.)

Other data fields in the 5 frames describe where the satellite thinks it is when the frame was sent, or at least give you data you can plug in to standard equations to work out its orbit. In fact subframes 4 and 5 describe the orbit of the entire constellation (slowly, takes about half an hour for them to cycle through all the satellites to give you the current "almanack", although listening to multiple satellites in parallel gives it to you faster because they're each reciting different parts of it at any given time).

So if you know where the satellite was and what time it thought it was, and you know where _you_ are and what time it is where you are, you can tell exactly how far away the satellite is thanks to the speed of light. (Modulo atmospheric distortion, which there are equations to compensate for.)

Except... we don't know that. Where we are and what time it is here are what we're trying to find OUT. We know where the satellite was and what time it thought it was, and if we have a local clock we can say how long ago we received the packet (exactly when the rising edge of the next packet started, measured backwards from now).

So we reverse the math: we need to know our X, Y, Z, and T (time) coordinates. That's four unknowns, so in order to do the "solve for four equations with four unknowns" thing from high school algebra we need four different inputs (which is why you need to listen to four different satellites to get a lock). If we can get packets from four satellites, each with position and timestamp information (and record exactly when we saw each packet relative to all the other packets), we can compare them to each other and work out which unique position and time could have seen those.

And if we know exactly what time our local clock said it was back then, we can work out how much our local clock is off by, and then keep it accurate as it drifts. All non-atomic clocks drift. We use quartz crystals for timekeeping because of "piezoelectricity", which means if you squeeze them they produce electricity, and if you push electricity into them they flex a little, which means if you run a current across them they _vibrate_, and if you cut them very accurately in special shapes you can count the vibrations and use that to keep time. But "very accurately" here means a digital clock is losing one second per year is ok, but that means it's losing about 43 nanoseconds per second, so if you want to know what nanosecond something happened at that's not good enough. Worse, piezoelectric fluctuations in quartz vary slightly with temperature, meaning your clock rate changes as the clock heats up or cools down so the gain/loss isn't _constant_ unless you put a heater under it to keep it at a constant temperature, and yes those exist. To get _nanosecond_ accuracy you need two nested ovens with special insulation. It's an off the shelf part, I think ours are from Germany. The ones in your phone get around this problem through a special technique called "not caring that much about accuracy". (They work out a correction factor for how many nanoseconds per second your clock is off by on average, an if that varies by 3 or 4 nanoseconds in any given second that's only a few feet so close enough.

Then there's the "a cartesian bear is a polar bear after coordinate translation" issue: the unknowns we're solving for are actually the distance between us and each satellite, and then there's a unique position that far away from each satellite. (Well, sometimes there's two, but one of them's way out in space. Two spheres intersecting have a circle of common points, three spheres give you two intersection circles which have two points in common, and then _four_ should narrow it down... but sometimes after you've set your clock and found your position you drop down to only 3 satellites you can see, and you can still make it work by knowing about where you were an what time it was before so you're solving for fewer unknowns. Your height above sea level is usually the easy one.)

The position each satellite reports is relative to the center of the earth, so we know where that is relative to these points, and we know the planet's radius. It's in our constant table, from the 1984 World Geodesic Survey.) So you work out the satellite positions and your distance from each one, then work out your position in X/Y/Z/T and _then_ you convert that into latitude and longitude using yet _more_ equations. (There's various supplementary material describing how to do a lot of this.)

Except there are TWO LATITUDES, and the one X/Y/Z converts easily to (assuming the earth is spherical, which it isn't) isn't the one mapmakers used hundreds of years ago when wandering out onto the land and siting stars and such through an astrolabe to figure out how far north of the equator you are given a level horizon. The spinning keeps the earth reasonably spherical as far as longitude (east/west) is concerned, but the equatorial bulge from centrifugal force means that once you get north or south away from the equator much towards either pole, the horizon isn't quite level anymore. I.E. a line straight down from the "level" ground would miss the center of the earth by many miles, and a line up from the center of the earth at the same angle would miss _you_ by the same amount, and since that angle is your latitude you've got two coordinate systems to chose from; geocentric or geodetic latitude. And you convert from one to the other with a for loop (iteratively work out about how far you were off by, work out how far _that_ was off by, and keep going until the correction factor is small enough you stop caring). Using a constant value for how bulgy the earth is, which again isn't quite what reality's doing but at this point everybody just sighs and updates the map markings because they weren't THAT accurate before.

All of the above isn't _how_ you make this work. It's _what_ we're trying to make work. The _how_ is DARN FIDDLY on top of all that.

Cut a toybox release, and then a mkroot release. Back on my head, by which I mean endless GPS.

For some reason working on GPS interferes with working on toybox. The mindsets conflict, to the point where if I bang on GPS for half an hour, I'm usless to work on toybox (full blown writer's block) for a couple days afterwards. I can force myself to fix specific bugs, but not make anything approaching a design decision. (I try to load the toybox context back into my head and it won't go, it's all "why did I... what was... I know I had a reason for that, where did I write it down..." Trying to work out the correct approach to take for any new code, NOTHING feels right.) And having multiple days where I DON'T have to do something GPS is pretty rare these days. (Even if I take a weekend off, Saturday and most of sunday I get nothing done, sunday evening I might do a little, and then it's back to GPS again.)

There are big things I very much want to do on toybox, but any time spent on toybox is NOT spent working on GPS, meaning I'm putting my startup out of business. I feel guilty for not making proress on toybox, and even more guilty spending any time on it at all.

Oh, if you're trying to run the i486 target in mkroot you'll need a current qemu. QEMU broke the ability of "qemu-system-i386 -cpu 486" to boot Linux a year ago, I reported it to qemu-devel last month, and the resulting fix went into the tree on the 31st.

Yes, mkroot has already inherited aboriginal's proud tradition of finding and fixing other people's bugs. That's what happens when you run consistency regression tests across multiple architectures, even of simple basic stuff. NOBODY TESTS THIS!

Part of the reason is the embedded Linux world has given up on upstream and uses its own very stale forks of everything, ala the entire Cortex-M Linux world being based off a fork of linux 2.6.33 and not even the _last_ release of uClibc. Yes, I poked Rich about it, but he's busy and nobody's wanted to sponsor the work. Because they have their stale uClibc, and are deploying it.

Nobody fixing/testing GUI stuff is its own can of worms, which reminds me I should really get a new version of my prototype and the fan club talk recorded and up somewhere. The university of Chicago's "Flourish" is not a conference that reliably makes and posts recordings. (I presented there twice, and both times that was going to happen, then didn't.)

But seriously, i486 not working in qemu for a _year_? Sigh...

The help text parser is bring stroppy. I lost hours tracking down a segfault that turned out to be in my debug printfs. (Which were on the wrong side of an if statement so dereferencing a null pointer.)

Blah, I still have a todo item about qemu doing a weird thing. If I do "qemu -nographic blah | grep hello" it works fine from the command line, but doing the same in $(argument) hangs with SIGTTOU. I should track down why, but poking at qemu is unpleasant.

To start with, where does qemu keep its main()? In vl.c it's #defined to qemu_main() but grep -r says there isn't one. (This is why in toybox there's a main.c at the top level that ends with the main() function, and then I wrote up a walkthrough on the website. I can't think how to make it more obvious.)

People keep talking about how well tested open source is, but I keep hitting things where I'm clearly the first person to ever do what I'm trying. Peer review is being defeated by volume.

It reminds me of the way automobiles (the horseless carriage) were initially the _solution_ to a massive environmental problem: the sheer volume of horse manure. It was always a problem (the ubiquitous "road dust" you wore "traveling clothes" to protect against back then was dried horse manure, powdered by the passage of many feet/hooves/wheels until it went airborne). Eventually the sheer volume started rendering places like New York City unlivable in the 1890's. There were newspaper articles about how the city would have to be abandoned due to the stench and other health hazards, teams of full-time municipal street cleaners could barely keep up downtown and had nowhere to haul it off to, and the roads into and out of the city were nearly impassable. Then circa 1901 cars started showing up and the horses were all gone a dozen years later. Suddenly the city was clean and breatheable! And then over the next few decades, the volume of automobile traffic grew to where the solution was now the new problem, previously negligible tailpipe emissions (nothing compared to horses) became a big problem once you had a hundred times as many cars as there had ever been horses...

Everything's only a solution up to a certain scale. Solving a bottleneck just reveals the next bottleneck, every time. "Our limiting factor is now THIS..."

(Meanwhile, I'm hoping that the exponential photovoltaic and battery growth curves do to the oil industry what the automobile did to horses, and more or less coincidentally cuts off the ecological disaster just in time again. The problem is, if we stopped emitting new CO2 _today_ we're already over 400 parts per million, meaning Georgia gets the climate Florida used to have, Minnesota gets Georgia's, and Texas develops the tradition of Siesta because it's too hot to go outside for half each day. And that still puts Florida underwater no matter what we do at this point, and makes Tokyo copy Holland. Wrestling _that_ back into the bottle's not happening in our lifetimes.)

In Dr. Who, one of the seldom-used aspects of the Tardis is its ability to pause the outside world. The 4th doctor used it to change clothes in his regeneration episode, although you could always just park it in deep space a million years in the past anyway. (Hey, time machine. Pop off, spend a year between galaxies, come back 5 minutes after you left.)

I suspect something like that would be the only way to get anything like a handle on my todo list. Of course for a human you'd need some kind of life extension technology too, because there's like 10 years of grinding without significant external input. (A read-only snapshot of the internet would be nice though.)

Meanwhile, I implemented a download thing in mkroot, by which I mean I came up with a hack. Now "mkroot.sh -d" downloads all the packages without building them, by calling sed on the modules to find their download invocations. (I know!) Still doesn't delete the old versions, though.

Building multiple targets with mkroot.sh is slow, and the main slowdown is re-extracting the kernel source tarball. Aboriginal Linux had the package cache to deal with this, which was always one of the big sources of complexity in that design. Heck, it was hard to just to explain it.

That said, it did valuable things (speeding up multiple builds a lot, and making multiple parallel builds take much less disk space, disk cache, and I/O bandwidth), so doing without it is a bit of a pain. Redoing a simpler version of the package cache is tricky.

Right now there are no lifetime rules for source packages. The old aboriginal download() function would delete old tarballs after downloading new ones, this one doesn't. That's a problem going forward, when I add updated versions cruft accumulates and multiple versions might get confused for each other. Hmmm. I _had_ a solution to this, but it was kinda heavyweight and a bit awkward (it wrote a timestamp file at the start, then touched each file it confirmed was current, then did a find -newer and deleted everything that came up. Meaning the download couldn't be conditional, anything you skipped validating got deleted.)

I don't have a separate download.sh this time, and it would be hard to implement because the modules have their own packages in them. So I can't say "download everything but don't build it yet", let alone the EXTRACT_ALL=1 ./download.sh that populated the package cache under aboriginal linux.

So far, I'm not applying any patches to any of the packages I'm building. That's a design decision and one of the big simplifications in the new build. It's also kind of limiting, but for now I'm sticking with it. That makes half the reason for the package cache go away, and if the user wants to extract or git clone the directories themselves, they can. (I could add a helper script to do it for them, but an accumulation of helper scripts is clutter. It's already less obvious than I like that you call ./mkroot.sh to do the build. I suppose I could add a lot of hints and examples to the README, but that turning into a novel doesn't help either...)

Finding "simple" is a lot of work.

Ok, the theory of operation of config2help.c is:

1) read Config.in and populate struct symbol *sym linked list.

2) read .config and set "enabled" on corresponding *sym entries.

3) Loop through *sym entries finding ones with the same "usage:" line and collate those entries, adjusting whitespace, sorting -X options and combining usage lines.

The problem with two commands sharing the same block of text (ala top/iotop) is part 3, with the usage: lines. In ps.c I used "usage: * [blah]" and I never actually implemented that syntax.

It looks like I got the rest of it right, the lifetime rules seem like it's not removing stuff from old entries but copying it into new entries. (It mostly never frees stuff because it's a short-lived program, it's all freed when it exits.) So my first attempt at fixing this was to switch to "usage: iotop|top [blah]" and teach the plumbing to loop checking each name in a pipe separated list and accepting any match. The problem is, the plumbing iterates through in _reverse_ order, so it's hitting the multiple entry first and taking that as the name to look for.

Ubuntu's grep has --include and --exclude so you don't have to do "find . -name blah | xargs grep" if you want a recursive wildcard search. This is simple to implement, but there are two problems:

1) There's no short option for --include. (All long options should have a corresponding short option unless you are _out_of_letters_, upper and lower case both.)

Given that ubuntu already defined an -I, I'm happy using -M for match and -S for skip. Ubuntu's grep (in 14.04) does not define those, and you can use the --include and --exclude long names for compatibility if you want to write scripts that run in both contexts. (Although I haven't currently mentioned the long versions in the help text for -M and -S because I haven't got enough space in two columns. :P )

2) In Ubuntu you still have to say -r, which means "grep blah -M '*.c'" will hang despite seeming, otherwise, really obvious.

Fixing the second one's trickier, because you can go:

find . | xargs grep --exclude '*.c'

and that grep won't -r. So there _might_ be a reason for grep not to imply -r, although I'm not sure it's a real use case? (And you can find -type f if it is.)

My first guess was to have a variant of the "grep -r pattern" no-file period adding logic add the -r when you don't specified what to search with -M, so "grep -M '*.c' pattern" with no files would automatically -r. Except that "grep -M '*.c' pattern dirname' would then return immediately with no results (because no -r). Which is nonintuitive.

Possibly -M should imply -r but -S shouldn't? Hmmm...

Part of the issue is toybox grep doesn't complain when told to look at a directory without -r. (In part because find . without -type F produced a lot of noise...) Maybe it should? I suppose I could just require people to say "grep -rM '*.c' pattern"...

A reminder that when the SFLC said it was fine to relicense OpenSSL, the openbsd developers floated a proposal to use the same process to relicense gcc 4.2.1 from GPLv2 to ISC.

Still waiting to see how that turns out...

(When I relicensed toybox to 0BSD I only had a half-dozen external developers, and I yanked the code of the one I couldn't contact even though he probably would have been fine with it. "Breaks closed" means the default answer is no.)

My ongoing solar research continues. Hard to stop it from getting a bit repetitive, but it's a bit like following microprocessors back in the 80's and internet back in the 90's. It's this THING and it's gonna be HUGE and it's gonna change EVERYTHING and blah blah blah. (Part of the lack of more widespread excitement is human nature: humans continue to be TERRIBLE and comprehending exponential growth. We keep looking at a history of doublings and then projecting forward linearly from wherever we are now, and that's NOT HOW IT WORKS. I grew up with Moore's Law: trust me, this is a thing.)

Some of the updates are things like Tony Seba gave his talk to the Colorado Renewable Energy Society guys. There's some new material, but the majority is the same talk as the last 6 times he's given it. (It's a book tour, S.O.P. Yes, same guy behind the three video series I point everybody at as a good starting point to get up to speed. Plus Avery Lovins' excellent info buried under random animal noises and sound effects.)

Several months back Paul Krugman tweeted a link to an excellent chart showing 80% of Russia's export revenue came from coal and natural gas, but I can't find where I bookmarked it and twitter only scrolls back so far. The closest I'm finding now is figures from 2013 showing that back then 68% of Russia's export revenue came from oil and natural gas. (Ok, 70% to 80% in 4 years isn't a huge stretch, it implies the rest of their export economy's collapsing, but it would be nice to have current figures.) Still, if you're wondering why they hijacked our election to put the climate change deniers in charge, they're as economically petrochemical dependent as Saudi Arabia. (Hence the piplines to nowhere article from last time.)

The exponential growth of solar technology seems accepted as a given at this point (and has its own version of Moore's Law), what's been missing is batteries. Installing more than a certain amount of solar results in "curtailment", I.E. unplugging the panels and wasting the electricity they produce when existing grid demand can't immediately use it. People have been talking about using the surplus electricity to generate storable fuel, but realistic ways to make liquid fuels aren't done yet (note: if you take carbon out of the air to make fuel, burning that fuel just puts that carbon back and the whole cycle is "carbon neutral", it's when you dig it up out of the GROUND that burning it adds carbon to the atmosphere that wasn't there before). And although hydrogen fuel cells are efficient and pollution free (the output is electricity and water, the inputs are hydrogen and oxygen gasses, and you don't need to carry the oxygen with you so the power/weight ratio is similar to other combustible fuels that leverage oxygen's tendency to be available wherever you are on the surface of Earth), _making_ hydrogen is inefficient in a bunch of ways (half the energy from electrolysis is lost as heat and it leaks out of just about anything you try to store and transport it in), so the real excitement is around batteries.

Battery prices have come down 80% in the past 6 years, and we can expect about the same over the next 6. This means at the start of 2023 we'd see 1/5 the price at the start of this year. Pairing batteries with solar panels solves the "duck curve" issue, where solar electrical production drives other power generation needs down near zero during the day (or even below zero, hence curtailment), but then they have to suddenly ramp them up again in the evening as the sun sets and people get home from work, and then keep a lower level running until morning. The ability to store even a few hours of the solar farm's output deals with the ramp-up problem and lets you avoid expensive peaker plants, and 24 hours of storage would let solar power the grid all by itself. (Probably plus wind.)

It's not just price, other big advances have been in lifespan of the technology. Back in the 1970's solar cells used to degrade to uselessness within 5 years. These days the lifespan metric is "how long until they only produce 80% as much power", and the common answer seems to be "around 40 years" and it's getting longer all the time. (One problem is the rooftops they're mounted on need replacing after 40 years.) Battery lifespans are measured by charge/discharge cycle count, and according to Apple a battery that was good for only 300 cycles in 2008 is good for 1000 cycles today (and these days they seem to define needing replacement with the same 80% of original capacity metric). Batteries that don't have to be portable can last much longer.

Laptops funded most battery R&D in the 1990's, cell phones took over in the 2000s, electric cars have been driving development in the 2010's (pun noticed and shrugged over), and next decade it's utility storage and home powerwalls. We're already seeing the first few large utility-scale battery deployments in hawaii, australia, india, china, california... Tesla's advertising this a lot, but samsung's actually _doing_ it, as are germany and china and so on.

We're been getting better battery chemistry (such as Nissan's heat-resistant "lizard battery"), better charging/discharging circuitry, liquid cooling, graphene and glass membranes, and new tricks every year. Lithium still dominates batteries the way silicon dominates solar cells, but people are trying other stuff. Perovskites are a material you _paint_ on surfaces to make a solar cell (at room temperature and everything) with the possibility to tune it to absorb infrared or ultraviolet while letting visible frequencies through, so you could have transparent solar cells turning building windows and tablet screens into power collection surfaces.

Meanwhile, a competitor to lithium I found particularly interesting was the revival of century-old Nickel-Iron battery chemistry. I've known about this for a while but bumped into it again recently when a small colorado company called Iron Edison gave a talk at CRES about their battery technology.

As the name implies they specialize in the Nickel-Iron technology known as "the edison battery" after Thomas Edison bought the patent from the original inventor Waldemar Junger in 1899, and then tried to pretend he'd invented it. (He really was the Bill Gates or Elon Musk of his day, gaining a reputation as an inventor by pouring money and marketing behind other people's ideas, then taking credit for what he'd commercialized.)

This little colorado company is selling Ni-Fe cheaper than Lithium (100 amp hours at 12 volts: $970, or 200 amp hours at 48 volts: $7760. Vs $9920 for their lithium solution. Although they've got a hybrid lithium/iron chemistry too, which sounds new. I'm a bit confused by their website: 48 volts needs 40 cells, so how does amp hour capacity _not_ change going from 12 volts to 48 volts if 100 amp-hours is the smallest they sell? Wouldn't they have a version with fewer cells and thus lower voltage?)

I was really excited about this technology when I first heard about it, because these batteries NEVER WEAR OUT. The "thirty to forty year time horizon" people quote is how long it takes things like tables and bookshelves to wear out. You put a physical object in a building, you tend to replace it after 40 years or so no matter what it is unless it's a historical site with a preservation budget to restore things. There are Nickel-Iron batteries from Edison's day still in use.

But this chemistry has some downsides: the batteries are really heavy (ala lead/acid) so didn't get used for portable applications where power to weight ratio was imporant. And like lead/acid they use a water-based electrolyte you have to top up periodically, but the reason to top it up isn't just evaporation but electrolysis. The current flowing through the battery breaks down the water into hydrogen and oxygen, which is vented into the atmosphere. (The overcharging failure mode is venting a _lot_ of hydrogen when extra electricity goes into the water with no other reaction to consume it.) This hydrogen is why you can't seal them like most modern car batteries.

The hydrogen advocates will immediately go "yay hydrogen", but it's still inefficient (as with all electrolysis the rest of the energy becomes heat, which does evaporate the water), and hydrogen is still a tiny molecule that osmotically leaks through just about everything (current Ni-Fe batteries don't even try to capture it; even if you could doubling the system's complexity for an extra 5% or 10% efficiency's near- pointless).

My real problem with leaking hydrogen isn't lost efficiency: hydrogen is as bad for the ozone layer as freon ever was, and if any does make it up past there it's so light it floats away into space and is permanently lost by the planet. Any energy technology that permanently reduces the amount of water the planet has as a continuous side effect of its operation makes me uncomfortable when people talk about scaling it up to 1/6 of the total economy long-term. (That's not my preferred solution to rising oceans. Water isn't lost to space because it has a molecular weight of 18, hydrogen gas has a molecular weight of 2. Fun fact: you could give the moon a 1ATM atmosphere of Sulfur Hexafluoride if you wanted to, it's 1/6 the gravity but the molecule's 6 times heavier than air.)

The big initial use of Ni-Fe battery chemistry was in fully electric cars; the gasoline car people seem to have switched to lead/acid batteries to avoid Edison's patents the same way all the movie people moved to the other end of the country from Edison's New Jersey offices to start Hollywood in California where he couldn't easily sue them. The reason patents were _invented_ was to encourage disclosure so secrets weren't lost went people died, everything else about them is a massive drag on progress. Places like China that ignore patents out-develop other countries _fast_. So yeah, the first time around Ni-Fe technology was killed off by patents, and by the time they expired it was old news people forgot about.

These days power to weight ratio still rules electric cars, so they're lithium all the way (element #3, atomic weight 7, lighter than aluminum or carbon fiber). They want to suck up as much juice as possible for maximum range, and people have been worried about the charging demand electric cars would add to the grid, let alone whether an average suburban home's rooftop has enough square feet to charge two cars (spoiler: yeah just about, but there's work to do to get there). How do you charge your car overnight if the sun goes down, you'd have to fill up a battery and then drain that battery into your car's battery...

Except that's not how they'll do it. Tesla (Edison Du Jour) demonstrated its 90 second swap-out battery back in 2013. You charge the batteries when they're not in the car, then "refueling" is swapping out the physical battery for one that's already charged. Sufficiently rural areas may convert gas stations to do this, but the business model urban and suburban areas are switching to is app-summonable self driving subscription fleets. (Basically really cheap taxis that come at the press of a button on your phone, and cost less per mile than gas does now, maybe even a flat monthly fee in-area. Not only do you not have to buy or lease a car, but there's no insurance, annual registration, license, oil changes, repairs... In a decade or two owning your own car will be like digging your own well and septic tank instead of city water, only a thing way out in the country.)

Self-driving fleet cars will have battery replacement done by the robot arm version of an Indy 500 pit crew servicing a racecar: between customers the car can drive itself up a ramp and the battery's swapped out when it drives off the other end. This means they can have a stockpile of spare batteries charging all day, to be used later. Set up your own solar farm at the edge of town where land's cheap and pile the batteries on an 18 wheeler overnight to go to a distribution depot in town. (Easier than laying cables. The big technology revolution before microprocessors was containerization, which really did transform the world in a fundamental way, and finding new ways to leverage that remains a common transformative business model today.)

A few years ago the european union passed regulations requiring all new buildings to be zero energy by the end of 2020 (meaning they generate their own power via rooftop solar, most of the gains would come from efficiency increases of the type Avery Lovins specializes in. But the reason you don't hear about it is that recently random lobbying seems to have gotten the weasel word "nearly" inserted into the directive, rendering it basically useless. Still, like everybody they're far ahead of the USA, which has been rapidly turning into a technological backwater since the Cheney administration. Sigh.

I have Fade's cold. Did not get out to see Wonder Woman on opening weekend.

Qemu commit 143021b26ffe is making a couple of instructions sh4a-specific. In theory the technique for adding sh2 and sh3 targets (and j2) wouldn't be that different...

*shrug* Threw it on the todo list...

If you take half of each day out for Japanese class, and then have standard $DAYJOB and open source project bug reports and email fielding, there's not a lot of energy left for programming. :)

Still, trying to get a mkroot release out. I think I've enabled all the targets I'm going to this time, the table looks like:

                boot    clock   disk    net
aarch64         x       x       x       x
armv5l		x	x	x	x
armv7l          x       x       x       x
armv7m
armv7r
i486            x       x       x       x
i686            x       x       x       x
microblaze
mips		x	x	x	x
mips64
mipsel		x	x	x	x
powerpc		x	x	x	x
powerpc64
s390x		x	x	x	x
sh2eb
sh4		*		x	x
x86_64		x	x	x	x

I.E. you can boot to a shell prompt that works, have the clock set to current time (without which "make" gets deeply unhappy), have a working block device and a working network card under qemu on the following platforms: arm64 armv5l armv7l i486 i686 mips mipsel powerpc s390x x86-64.

Microblaze's defconfig doesn't seem to have a working elf loader, I dunno if that's a musl-cross-make toolchain issue, kernel config, or qemu yet. There's also the "nommu/with mmu versions have the same short name" issue I mentioned earlier, it's a todo item.

I punted armv7m and armv7r because they're not fully supported in musl-cross-make, the vanilla kernel, and qemu yet. Musl hasn't added cortex-m fdpic support because the gcc fdpic support patch is still out of tree (so the toolchain is static pie instead), and the kernel defconfigs and qemu board emulations don't overlap for cortex m (qemu can emulate arduino-style boards with 256k of ram that can't run linux; maybe I can plug it into the "-M virt" board next time?). Armv7m is a nommu board that only has the arm thumb2 instruction set, and lots of people seem to use it. Armv7r is a nommu board with the conventional armv7l instruction set and I've so far only encountered two people interested in it. Both targets remain on the todo list but I'm not holding up the release for them.

Mips64 turned out to be broken in musl, there was a type size wrong in the stat structure so things like "ls" don't work. This is a big enough issue (the kernel API to list directories returns nonsense) that I punted the arch until the next musl release. (The upstream patch is here if you want to test it yourself.)

I just haven't gotten around to 64 bit powerpc yet. I'm unaware of anything wrong with it, just haven't put in the time yet. Only IBM seems to be using it, and IBM is dying fast. (I want to get Alpha and with-mmu m68k working someday so that's not a blocker, just... not a priority.)

The sh2eb target hasn't got qemu emulation, that's for j-core boards (turtle and numato). I haven't merged the kernel build yet because it's the only one _not_ targeting qemu and I'm not sure how I want to handle that yet. (Todo for next release.)

The * on sh4 is if you make the kernel change mentioned here and use a current qemu with this patch _then_ it should work. The second problem is that qemu didn't shut down right. The first problem is the kernel guys recently enabled a serial buffer thing that qemu doesn't emulate properly, which means the "serial data has arrived" interrupt only triggers every 16 characters or so (when the buffer fills up; there's no timeout). The THIRD problem is that's the only target that hasn't got a working clock; there are two clock drivers commented out in module/kernel for sh4 and neither of them actually gets data from qemu (despite one being in the r2d defconfig in the kernel).

The correct fix for all of that would be changes to qemu-system-sh4, but I don't really have much contact with those guys. Changing that project to depend on libgnugnugnugnugnuglib was so dumb I largely stopped paying attention to the development side after that. It's another "this got useful enough to large corporations for them to assign bureaucrats to staff it", and there's no trace of hobbyist left. Building the right thing gives way to procedures and certification.

First day of Japanese class at ACC. I'm not officially registered yet, need to email administrative higher-ups and get permission to join a full class. Then I need to get the book and the workbook, which could be a bit of an issue since there's no bookstore on the highland campus. (2/3 of the mall hasn't been digested yet. They say they're making dorms out of some of it.)

Poking at mkroot in the afternoon, the s390x target doesn't have a network card? And the defconfig won't run. How did I... ah right, switch out the machine type symbol for a more primitive processor without this forest of unsupported capability bits that QEMU doesn't implement. (You have runtime probes for what your system supports, but no ability to NOT USE THEM at runtime. So what does the probe accomplish, exactly? Sigh...)

I attempted to add distcc to mkroot, but the current releases of distcc moved to github, which means their source tarball releases are just a snapshot of the github repo, which means they didn't run autoconf to create a .config file, which means autoconf is now a build-time dependency for them, and screw that. It's literally easier for me to implement a simple distcc in toybox than make the current nonsense build. (Especially since I spent a longish time maintaining my own toybox fork and doing ccwrap.c for aboriginal linux, meaning I know more about parsing gcc command lines than is probably healthy.)

Heck, if I start from ccwrap and don't care about compatibility with the old distcc protocol but just do something trivial glued to netcat's server mode, a distcc client/server pair is probably a weekend's work.

But not _this_ weekend...

Thunderbird is open source, but I can't easily build it from scratch. This means it's not USEFULLY open source.

I HATE the way it collapses the "Reply All" and "Reply List" buttons together into something that randomly switches back and forth between them so whenever I'm distracted I wind up doing the wrong one. (I usually want Reply All. I would delete Reply List if I had the ability, but it doesn't give me an orthogonal button, it only gives me the Reply All functionality as part of Gratuitous Bundling of the thing I want glued to crap I don't want.

But as annoying as it is, it's not annoying enough for me to chop it out of ubuntu's package manager, build my own version from source (with giant dependency tree and who knows what invocations), chop my way through the code to figure out how to change this UI thing, argue with the upstream mailing list that my change is worth integrating, instruct the package manager not to replace the version I built for the year or two it takes the upstream change to percolate into the version ubuntu's using...

It's just not worth it. If it gets so bad it's unusable, I either write my own from scratch (ha) or look around for an alternative package (ala my year or so using balsa a while back).

I added cross.sh to mkroot, and if you run it with no arguments, it gives the mcm-buildall architecture list:

aarch64 armv5l armv7l armv7m armv7r i486 i686 microblaze mips mips64 mipsel powerpc powerpc64 s390x sh2eb sh4 x86_64

The arm64, armv7l, s390x, i686, and x86-64 targets are all testable now and should work with the full set of hardware under qemu (boot to shell prompt, disk, network, clock, shutdown). I'm working on filling out and testing the rest.

The armv7m and armv7r targets are nommu, m is thumb2 instruction set only and r is conventional 32 bit arm instructions withotu an mmu. Last I checked, armv7m wasn't quite upstream in qemu and the vanilla kernel (cortex-m linux is available from emcraft in moscow, but they've never tried to push anything upstream; linux had a config for a board qemu doesn't emulate, and qemu had a board emulation with 256 kilobytes of ram). Support for armv7r was worse (I first heard about someone speculating _adding_ Linux support for it in a hallway conversation at ELC in either 2013 or 2015), but maybe it's changed since?

I found a mips64 bug (stat layout is wrong so ls prints nuts info) which Rich has a musl patch for but that needs to go into musl-cross-make. I asked Rich and he said to bump that arch to next release.

Microblaze is funky (doesn't wanna run ELF?), still need to debug why. I also have the problem that there's a with-mmu and a no-mmu version of microblaze and they don't have distinguishing prefixes, not sure how to distinguish that. (Right now the build is sorting by $PREFIX-* in the toolchain name, which was unique per architecture up until now. There's an fdpic suffix, but how that maps to a short name... Need to add plumbing I guess.)

For years Android's had an obvious bug where plugging headphones or earbuds into the headphone jack produces spurious control signals (there are no buttons on these earbuds!), and there's no way to tell it to ignore these spurious signals. I've dealt with this since about 2012 (headphones du jour randomly starting/stopping song playing while I'm listening to podcasts). The current version's iteration is two different sets of earbuds popping up a full-screen "listening..." every 30 seconds while I'm trying to watch youtube videos. (Yes, I've disabled "ok google" everywhere I can in the settings.)

There are complaints online about this going back years. There are of course apps to disable it, the highest ranked of which on the google play store says it was broken by the upgrade to Android Jellybean (the one _before_ kitkat, which came before lollypop, marshmallow, and nougat. Yeah it's alphabetical.) It links to what it claims is a bug report, but which is instead a Google login screen. (If you have to login to _view_ the bug, it's not real.)

Note to self: if I start doing podcasts, bundling is a topic. It's what Microsoft built its business on (then got into antitrust trouble for), and it's what Android is doing with this OK google crap. It's a "feature" I don't _want_ and can't _disable_. I have no control over this without rooting my phone and going deep down the modding rathole.

For posterity, here's that chunk of photovoltaic research I posted to the svlug list (half of which is the same links as last time; I have a bunch more, but this is still the best introduction):

At my dayjob we're making better synchrophasors and hooking them up to the internet.

Electrical grids were designed around the idea of centralized generation from which power flows in one direction to consumers, so you only had to measure it at the generators and maybe the substations. But now we've got solar and wind feeding power back in at the edges, and once you go above around 3% on that the voltages go out of spec and people's electronics get unhappy. So we're retrofitting the grid with sensors so the whole thing can switch over to 100% solar and wind over the next decade. Combine this with batteries and your more optimistic forecasters expect peakers to go away around 2020 and 100% renewable base power by 2030.

The best summary of the state of things is probably a series of three related videos by and about a stanford business professor named Tony Seba. The first video is a class he taught on the energy industry in 2013, then a book talk he gave last year, then that book talk was analyzed by a mutual fund manager in india earlier this year.

Another informative speaker is Avery Lovins, who started on the environmental side of things and became an expert on the technology. I find him really annoying to watch (the first minute and change of this ted talk gives you the idea), but when he's not patronizing his audience, oinking like a pig, or introducing loud clangs into quiet speech, he's got really good info. If you're up for it, this talk is very informative and worth gritting your teeth through.

I could give dozens more links but that trilogy of Tony Seba videos is probably the best starting place. If you want to dig for yourself, about a third of the ones on the Colorado Renewable Energy Society's youtube channel are good (the rest are environmentalism, not technology).

Anyway, the optimistic people are probably right about "renewable energy" adoption, because humans never forecast exponential growth accurately. Moore's Law's been replaced by Swanson's Law, I.E. solar panels were $76.67/watt in 1977, $0.36/watt in 2014, that's a curve that's hard for humans to keep up with and it seems to be _accelerating_, and there's a similar depreciation curve for battery technology too although it started later. (Driven by laptops, then cell phones, and now electric cars and home battery walls are ramping up.) The analogies people keep making are to cars displacing horses (about a decade from 1% to 99%), digital cameras displacing film (ditto), analog to digital phones (cell and voip)... There are one time conversion costs and then the new stuff is _waaaaay_ cheaper.

Speaking of Godwin's Law, did you notice how the vast majority of Russia's export income is from oil and natural gas (without which they basically can't even feed themselves), and the CEO of Exxon was happy to divest himself of his stock assets to become Secretary of State? (Ordinarily a CEO selling all his stock in the company is considered a bad sign, but he found a way.) There's an excellent article explaining that, and why the new administration's #1 priority before ANYTHING else was pushing through the Dakota Access Pipeline.

So yes, the fossil fuel industry's 1/6 of the economy, it's set to dry up and blow away over the next decade, and it's taking the calm dignified approach the RIAA and MPAA did when confronted with Napster. (I.E. they flipped out and did everything short of send assassins for several years.) Add in the GOP's Southern Strategy driving that party outright psychotic and it's "interesting times" indeed. The plutocrats the GOP nominally serves are drowning, and trying to squeeze all the cash they can out of their stranded assets before the loss of value is recognized and their stocks implode. (The coal industry's already lost 99% of its value. It seems to go solid/liquid/gas: oil is next, natural gas after that. Probably something to do with the energy density or ease of mining/storage/transportation?)

The point is, putting solar panels on your roof isn't just good for the environment, and isn't just a good financial move, it's also the most direct financial strike you can make against the current Republican party and the oligarchs supporting it.

Driving back to Austin today, with Fade (and Adverb). Summer vacation! She's got one more meeting and then we can head out in the afternoon.

People on twitter periodically complain about video only tutorials; I'm noticing now as I prepare to do videos. None of the people complaining ever read my blog, so the text I've been writing here for over a decade is lost on them as they notice the popular highly shared videos that many, many, many more people watch, and presumably lament their existence.

*shrug* My motivation is "lemme just show you my screen as I do a thing, and walk you through it explaining as I go". I'm aware it's less keyword searchable and not likely to age as well, but in terms of getting the info out there to a large audience today, television became a major industry for a reason.

I suppose I should break some of my longer writeups out of random mailing list posts and blog entries and into separate papers, but I've done that. Some even got picked up by magazines... which then went defunct. (I got interviewed by Linux Luddites twice (episodes 11 and 88), that ended at the end of 2016). Used to write for three different Motley Fool columns that no longer exist (I was one of several rotating authors for each, but the sections went away; you can still dig a few up if you try but my authorship info seems to have gotten lost in a database migration).

All that's no stranger than anything else on the internet going down, the most ironic of which is the article on institutional memory loss going down at its original location and only being available on archive.org now. This is why I mirrored the computer history articles I was using for research back when I thought I'd have time to write a book on that topic. :)

But it does mean if I want to produce new material, properly composed and edited and organized to be vaguely coherent to other people... well Youtube has the advantage that the first video ever uploaded to the thing is still there. (Stuff goes down for stupid copyright reasons all the time, but it doesn't _expire_.)

Part of it's my recent usage patterns. I've been watching a lot of video on my phone because I have it with me. Half the time I just listen and glance at the screen every few seconds. Trying to read on the phone screen is less convenient (especially while walking), it's ok for twitter but less so for long texts...

And then there's the "how do you find new content" question. Once upon a time there were RSS feeds, but Google killed its reader and the open source community's been worse about those than it's been about email clients. Social indexing goes back to slashdot and livejournal, these days it's twitter but that's a lot more time consuming than livejournal or slashdot used to be, and these days I skip reading my feed for days at a time when I just don't feel up to hearing about the new horrors du jour. (And it goes out of its way to make sure catching up on what you missed is a huge pain.)

Youtube's video sugestions are annoying and sturgeon's law writ large (vast swaths of crap), but I do find interesting new things there when I dunno what I'm specifically looking for.

Collated all the solar power research (or at least URLs and text not files) I did from the big machine, so it's all on the netbook now. I still need to grab a gazillion "watch later" boomarks off my phone, although there's a lot of overlap. I threw a little of it into a post to the ongoing j-core thread on the svlug list, but there's a good hour-long talk here. Alas it's an area I'm not a recognized domain expert in, so I don't get to _give_ that talk anywhere. (And most of it's "go read/watch this, then this, then this..." anyway.)

A couple weeks back Elliott gave an Android NDK status update (about building toybox with it), which linked to info about the nightly builds, which links to a login screen, which is where I stop. Any process that requires a login to download the code is not open source, and I'll wait until they do a release I can download without a login.

It's a pity, I'd love an llvm toolchain that links code against bionic to test toybox against, but bionic doesn't have makefiles so you can only build it as part of the AOSP hairball, at least within the limits of my interest in going down that particular rathole. That leaves me downloading binaries other people built, which only open source the release versions.

Way back at Flourish in 2010 I gave a talk about how open source actually works (project gravity and barriers to interest), but they screwed up the recording. I should redo that talk. (I keep trying to give better versions of talks I already gave at Texas LinuxFest, hoping to get a recording I can point people at. But so far, my txlf talks are... sad and underprepared. Something about a local conference I don't travel for tends to result in everything getting pushed to the last minute. Ok, the time I missed a bus stop and gave myself heatstroke walking to the venue didn't help.)

Speaking of redoing talks, fade ordered a $12 lapel mike off amazon after I watched several youtube videos and came to the conclusion that if I ever have to care about the difference between condenser and non-condenser microphones I'll just use the laptop's builtin-in microphone, and teaching VLC to listen to a USB input is not a can of worms I want to open, but every time I've been "miked" at a conference they just clipped a thing to my shirt collar and I can order one of _those_ and stick it in the microphone jack and call it good. So I'm trying that. (She sent it to Austin, I'm still in Minneapolis, so it's a ways off.)

Visited my sister and the niecephews. They are all getting larger (except my sister, and Ian who is 17 and may be fully tall at this point). Sam's going to be dangerous in a few years. Sean was mostly out with a friend. I watched Carrie fight the ender dragon in creative mode, sit through 15 minutes of fairly pointless end credits to leave, go back in to see if there was another ender dragon (there wasn't), and then work out you can hit escape to skip the end credits when it tried to play them again when she left again.

What did I do today...

Still not sure how to deal with Andrew Morton's email (about my patch for using devtmpfs automount with initramfs). What does he want?

In mkroot I wrote an email about HOST_EXTRA= updated README to mention the mailing list, unquoted $HANDOFF to support multiple arguments, tried to write a checkin comment showing the syntax for using multiple qemu -append instances and found out doing that doesn't work, so I emailed the qemu list. (They said -append doesn't append, it supplies a single set of arguments. The kernel might or might not append them to built-in arguments, but that's the kernel for you. So why is it called append then? No idea...)

I should reply to scsijon's mkroot email about future directions for the project, and post to j-core about turtle orders/pricing...

The guy who keeps trying to turn toybox into a shared library is back, only this time he didn't say that's what he was trying to do until message #5 in the email thread. (If you statically link something that does malloc() and then link it against something else that does malloc() you now have two instances of the heap, and as soon as you allocate from one and free that memory in the other you've got corruption. This is why statically linking shared libraries is dangerous, and thus why libraries depend on other libraries at runtime.)

Ongoing linux-kernel thread about my linux-fullhist tarball, which is actually somebody else's project I mirrored/refreshed when their page went down. (If somebody's looking for a standard for git commit numbers older than Linus's tree I'd much rather they use the existing thing that's been stable for the past decade than invent some new half-assed thing based on bitkeeper.)

Closed the window with the Linux Plumber's Conference call for papers, since the extended deadline of the 11th passed yesterday, and I really don't care about volunteering my time to make money for the Linux Foundation. I should do an updated writeup about the linux foundation, since people showed interest in the last one, and there have been several developments since then.

(A change of scenery is really helpful to my productivity. And Minneapolis doesn't have multiple cats trying to climb onto my keyboard and/or shoulder while I'm typing.)

The phone repair part arrived! (I dropped it twice over the past month, smashing each end of it. Still works fine in the middle but there are sharp edges, and the proximity sensor is jammed so every time I make a call the screen blanks as soon as it connects and you can't use any of the buttons. I switched on "power button ends call" in settings, but I can't dial the passcode for the daily conference call on this phone.

I called around to repair places and they wanted over $200, but the part itself is only $40 and there are youtube videos on how to do it, so...

I followed a youtube video's instructions on how to install the new part. (Which boils down to "take apart old phone like this, move this list of parts to new front panel, reconnect wires, put screws back in, snap on back panel".

The big missing instruction is that lots of stuff is glued together. The guy in the video had a large hair dryer (heat gun?) that I didn't have. The battery especially broke two of the plastic tools over the course of an hour before finally coming out. (I should have ordered a new battery while I was at it, the old one's got greatly reduced capacity at this point. Removing the battery is like the first 1/3 of the disassembly process.

The front panel turns out to be about half the phone's thickness, containing the display, touchpad, front glass, and most of the side case. About half the remaining space is battery. The motherboard (the top 1/3 above the battery) is tiny, connected to the even smaller bottom circuit board (below the battery) by two long (antenna) wires and a pair of ribbon cables (one above, one below the battery). Another ribbon cable plugs the top board into the battery, and two more ribbon cables go from the top board to devices plugged into the front panel (the front camera and I think the proximity sensor?) I had to move the camera, power and volume buttons, two little square rubber earhorns that funnel sound into the top and bottom microphones, and possibly a couple other things I don't remember. All of them were fiddly.

That funky slot I thought was an sdcard is where the sim card goes! In a tray that would be really clever if it wasn't upside down so the sim card falls out when you open it. I somehow got the top motherboad out without noticing, but couldn't put it back in without removing and replacing the tray.

I did not drop any of the ten tiny screws! I don't think I stripped any either, but it was close. The back panel does not want to snap back on. It's got like 8 clips, of which maybe 5 are working. I wonder if I can get a new back panel cheaply?

I'm used to this phone, reasonably comfortable with it, and don't want to lose my apps and bookmarks. (Twitter app user tokens, game progress, etc.) Given that I've called every mandatory google account they make you do setting up a phone "goawaynow" with numbers after it, I have yet to migrate one to a new phone. It may have a password, who knows? I took it to the T-mobile guys to ask if they could do it, and they said Google intended for me to do this and who was I to not go along the path Google laid out for me, for shame. (Yeah yeah, maybe I can fire up adb, copy the filesystem contents, and copy it to the new phone? That sounds easier. But I'd probably have to root it first, and that usually involves a complete reflash. Hmmm...)

I also really like this metal poky thing out of the toolkit. I have no idea what it's called or I'd order another dozen of them, it's basically a flat metal toothpick attached to a flat metal ring and it's SO USEFUL doing this kind of fiddly work.

Ah, Elliott clarified that he _isn't_ going to plumber's this year. They track wiki imported last year's page as a starter, and then never updated it to remove people who weren't returning.

In that email exchange he mentioned the pending list and the first item on it was chrt.c, and I started lookin at cleaning that up. It's not a command I use much myself, but it seems potentially tractable with a bit of research...

Elliott also sent a date %N patch, which I'm pondering. The problem is libc has this thing called "struct tm", which is time broken down into individual fields (year, month, date, hour, minute, second, weakday, timezone, etc). The strftime() function converts struct tm into a string representation, using a printf-like pattern with a bunch of its own escapes to say "put two digit century here" or "abbreviated month name according to local locale". Quite a pain to replicate this yourself, and not really feasible once you take localization into account.

But there's no nanoseconds field in struct tm; it predates computer being fast enough to care about that. You get whole seconds and that's it. These days, lots of things care about fractions of a second, so ubuntu's date added a %N escape, which it parses and replaces before passing the rest on to libc's strftime(). And oh there be dragons there: %%N is a literal "%N" in the output, the other strftime escapes do the printf escape trick of "%34N" padding out the result, and you probably don't want "% %Ns" turning into "% 123456789s". Luckily I factored next_printf() out of seq's insanitize() back when I wrote it, so I've got a function that parses user-supplied printf strings already....

I'm only adding %N to the output side, though. You can set nanoseconds with @UNIXTIME.NANOSECONDS already, and making the parsing work on the input side is noticeably harder (because you're not modifying the format string, you're cherry picking data from the middle of the _input_ string, and figuring out where to take it from is equivalent to correctly parsing all the other escapes...

Meanwhile, I'm participating in a long thread on the Silicon Valley Linux User Group mailing list (which I subscribed to setting up a j-core meetup at CELF and never unsubscribed from). They mentioned j-core and I replied in, and it turned into an ongoing discussion of j-core vs risc-v.

I wonder if I should collate various big writeups I've done and post them as essays under the writing directory? I usually do them as as blog entries, the kind I leave half-finished and then constipate the uploading part of the blog until I get around to finishing them. (I write _newer_ entries, but can't upload them out of order. Yeah, my process often sucks. Or doesn't scale well to the amount of free time/energy I have divided by the number parallel things I'm trying to do. But improving my process is itself a significant demand on my time and energy, so... (And depending on external things like livejournal or sourceforge always turns into technical debt and eventually a _new_ timesink to fix it. No, I don't trust wordpress or github any more than their predecessors, they're fine _now_...)

This is related to my "I shouldn't go to conferences where the talks I'm proudest of tend not to get recorded (sad eyes at Flourish in Chicago which promised to put my 3 waves talk up...), or I run out of time and give an insufficiently prepared talk (symptom: I have 3 hours of material for a 45 minute timeslot and run out of time), and the last few times there was bad jetlag screwing up my ability to present..." I.E. I should just podcast. Except learning to podcast (mostly the editing part) and making time for it... Sigh, it's on the todo heap.

(I'm happy to use youtube for distribution the same way I use github for distribution. I don't _depend_ on them, its a read-only archive of stuff I have backed up locally in at least triplicate.)

Speaking of which, the 2010 Flourish talk I did (which is missing the first 30 seconds and then has buzzing drowning out what I'm saying for the next minute, but the bulk of the talk was at least recorded and I should record a new intro to it maybe? Anyway...) was on something called blip.tv which went away after a year, but I downloaded a snapshot of the video file, except it's a "flash video" format file and what supports that anymore? I should convert it to something else, but last time I tried the audio and video got out of sync. (Keyframes exist for a _reason_, people. Grrr...)

I tried re-giving the talk at Texas Linuxfest, but something about that convention always leads to me giving substandard presentations I'm unhappy with. Maybe because it's local and my subconscious doesn't take them seriously? Dunno. I'm aware there are conflicts here. I need a respectable externally imposed deadline without it resulting in such a time crunch the task I'm traveling to do gets starved for prep time or I wind up too exhausted to perform. Maybe the fundamental problem is that writing and giving talks isn't really part of my job...

May 8, 2017

May 7, 2017

May 6, 2017

May 5, 2017

Dropped my phone again, smashed the screen more. Still usable, it's now cracked along the top and the bottom. Haven't cut my finger on the glass yet,but it's probably only a matter of time.

(I've been taking long nightly walks to UT to use my netbook at a convenient picnic table near an outlet under an overhang so I don't get rained on if the weather turns nasty. Along the way I watch youtube/netflix/hulu videos and play phone games, which means I'm carrying my phone out in an easily droppable position, above a concrete sidewalk. I _try_ to work at home, but cats prevent it. And it's hard to walk any length of time during the day in Texas, the sunlight's brutal. Hence long night walks when telecommuting gives me the schedule freedom to do that.)

May 4, 2017