Thought I might get a couple hours on toybox or aboriginal while the niecephews were at the pool, but Sam doesn't turn 6 until October and in order for her to swim, I have to swim. (Queue Captain Picard shouting "There are four lifeguards".)

On the bright side, in dayjob-land the filesytem I'm backporting through 4 years of kernel development (Pointy Hair Linux is still using a 2.6.32 kernel) now compiles! And mounts! And segfaults when you list the directory contents!

So that's something.

I need to extend the option parsing infrastructure to include FLAG_longopt #defines for each long option. Doing this in shell script with sed is horrible, so I'm pondering writing some C to include lib/args.c and leverage the existing C option parsing infrastructure to output its own macros.

Problem: what to do about flags that are configured out? Right now the shell script is #defining those to 0, which makes if (toys.optflags & FLAG_x) become if (0) and drop out. If I replace the shell script code doing that, the C code doesn't naturally do that either. Since USE_BLAH() macros are done by the preprocessor, it's actually really hard to get C code to deal with both at the same time.

Which means I need to build the C executable twice, with two different preprocessor configurations, and feed output from the first one into the second one...

Sigh. Caught up on email before going to bed, and there's a mount.c submission, from someone using toybox in a product and who needs mount.

I've had an unfinished 100 lines of mount sitting around for months, with no progress on it. Instead, I've been spending what little coding time I can get doing things like cleaning up ifconfig.c (which I'm maybe halfway through). If you got the impression that cleaning up giant external submissions like that is slower than writing new code from scratch, you're not alone.

The new mount submission is over 1400 lines long. The one I wrote for busybox was 600 lines. This is a command I've already written before. It would be faster for me to ignore it and finish mine.

But I haven't had time to do that. The fair thing to do would be to merge this, discard the work I've done, and clean up this thing. Except I haven't had time to finish cleaning up the last giant code dump yet. Or the half-dozen before that...

Instead, I spent today backporting vanilla kernel.org code from an obsolete 3.0 kernel to an obsolete 2.6.32 kernel, because we need 9p working in centos 6.4 and it only supports 9p2000.u not 9p2000.l like SLES does. Nobody will care in 5 years, but we can't wait for Centos 7.0 to ship, and until then Red Hat will continue to ship a kernel that's already seventeen releases out of date.

I'd love to take a year off and work on toybox full time, but I can't afford to. I got married, we moved to a real house with a real car, and she's getting a doctorate. That lifestyle comes with expenses...

I hate blocking other people. I also can't help but be envious that they get to work on toybox at least somewhat in the context of their jobs, and I don't...

Monday: two very attractive women sunbathing outside in bikinis. (Dunno if they've been doing this and I only noticed because I took the day off and the break room with internet access is surrounded by windows.)

Today, it is _snowing_. (Not sticking on the ground, wet grey and overcast all day and I'm not the only one to notice when switched to flakes after sunset.)

Jonathan Coulton's "First of May" song might possibly have presented a very slightly idealized take on the situation, is what I'm trying to say here.

Taking the day off from work. The 3.9 kernel shipped and I need to catch up with open source stuff.

Plugged 3.9 into the aboriginal build, tweaked the patches, and built the quick and dirty version of all the targets, ala:

NO_NATIVE_COMPILER=1 more/for-each-target.sh './build.sh $TARGET'

That's a small enough subset my netbook can churn it out in a finite amount of time, and: arm works, mips works, sparc sort of works ("ls -l /dev" hung once, but ran on the reboot? Hmm...), and the x86 targets work.

That leaves powerpc, sh4, and mips64. (And m68k but qemu still doesn't support that well enough to test and I haven't bothered to set aranym back up yet.) Of those sh4 dies before producing any output, quite possibly a qemu issue.

I also need to get inittmpfs ready to resubmit. There's conceptually three parts: base inittmpfs support, rootrdflags=, and the ability to configure out the root= fallback code.

But mostly, spent the day banging on toybox...

Managed to screw up my local toybox repository again today. (Doing "hg import -f" will suck up changes to your working store, even in very different parts of a file that only has a one line modification. You'd _think_ it would bypass it and just apply changes to the database, but no. And if you do two of them at once, "hg rollback" won't undo the previous one, it's now a permanent part of that repository, because Matt believes that rollback should only do one level otherwise demons fly out of your nose.

(The way to fix it is to clone the copy on the website into a new directory, do the hg import -f into _that_ copy, and then substitute the .hg directory out of that for the one in the old repository. Or at least that's what I did. So far no more nasal demons than usual.)

Meanwhile, I heard back from the first of the tinycc developers about qcc: Daniel Glockner read my old blog entry on the qcc triage todo list and he does _not_ want any of his code used under BSD license terms, so that's 295, 306, and 307 to remove from tinycc. (He added the ARM support, but the point is to replace all the code generation with qemu's TCG anyway, so not a big loss. The ELF plumbing and predefined #ifdefs need to be redone at some point, but it needs to be genericized with a table for all supported architectures.)

But that's after toybox 1.0, and presumably after I switch Aboriginal to musl, toybox, and llvm. And do my own distcc implementation that doesn't assume falling back to compile on the host is something to do on a regular basis.

Behind on everything, and this weekend I visit the niecephews again. In theory the 3.9 kernel drops this weekend, no idea when I'll get to catch up...

Why is distcc making zero speed difference building on my netbook? I kick off two parallel builds in two different windows and they advance at exactly the same rate. What, some kind of strange cache effects? Did I break distcc again in a way that it's totally refusing to tell me about because visibility into what distcc is doing has always sucked mightily? Has the emulated network become a bottleneck that's coincidentally slowing it down to EXACTLY the speed of the non-distcc version?

Sigh. I need to write a distcc replacement. Something that actually _forces_ compiles to be distributed and _never_ falls back to a local build, rather than invisibly deciding not even to try to distribute it for inscrutable reasons.

But not today...

I'm disappointed that our "Democratic" president is to the right of both Richard Nixon and Ronald Regan, but until the baby boomers stop being 24% of the population I can't see it improving.

I respect people searching for the truth, but have a problem with people who claim to have found it. For one thing, the truth changes: we fought World War II against Germany and Japan, which are now our allies. Statements like "Our problem is X", "We must do X", "It's useful to know X", "The highest moral value is X"... they have a shelf life.

Let's take the current universal moral freakout: Traditional Marriage. Before you get all bothered about the fact the only people who actually seem to want to get married anymore are gay, remember that the institution of marriage predates paternity testing, contraceptives, the industrial revolution, electricity, literacy, and most women _not_ dying in childbirth before the age of thirty. "Till Death Do Us Part" averaged less than 20 years before antibiotics, vaccines, blood transfusions, ambulances, defibrilators, refrigerated and canned food, epipens and benadryl for anaphylactic shock, tylenol to bring down fever, inhalers for asthma, surgery for appendicitis, snakebite antivenom, splints and plaster casts so a broken leg doesn't leave you lame for life, prescription eyewear to prevent "legally blind" from being effective reality, and so on. People in developed countries don't die of heatstroke or freeze to death much anymore, are seldom eaten by wolves, and can generally avoid scurvy, beriberi, or going blind due to vitamin-a deficiency. Rats, roaches, mosquitoes, fleas, and worms are things we actually notice rather than just expect about our persons.

Indoor plumbing all by itself is a huge deal: nobody in my neighborhood died of dehydration or cholera this year. I can't currently smell urine or feces. I can't smell myself either, and despite the snow outside did not risk hypothermia in washing any portion of my body.

The point is: this is a recent development. Men used to die of random banditry when they didn't get sucked en masse into a pointless border war or flatted by Genghis Kahn or Xerxes or Napoleon or Stalin. (Not saying we've solved that last one but nuclear weapons have at least kept the scale down: US casualties in Vietnam were about 1/5th of the number of Armenians killed by the Ottoman Empire during World War I.) Women died in childbirth _a_lot_. And then a plague would come through and kill yet another quarter of the population. "Till death do us part" was in the ceremony because getting remarried after your spouse died (and then having more kids) was normal: fairy tales full of stepmothers, stepfathers, and orphans aren't just common because it's a useful plot element, they're common because it was a common condition.

So saying "How DARE you question the rites of marriage handed down to the ancient greeks by Zeus himself! People divorcing after twenty years rather than dying after fifteen is clearly a sign of moral decline, Odin will strike us all with thunderbolts!" Yeah, not buying it. Used to serve a purpose. But we've moved on. Religion gave us dietary laws before we understood nutrition, parasitology, crop rotation, germ theory...

Now that we know what causes malaria, have public schools providing minimal day-care, can actually plan pregnancies and tell who the father is after the fact if we want to, and aren't generally inheriting a plot of land for generations to subsistence farm on (often as a peasant under a local warlord with a big knife), lots of people find marriage less useful. We've found new reasons to keep at it: tax advantages, joint bank accounts, hospital visits, and so on. But the real reason is cultural inertia, and tacking extensions on to a ceremony dating back to the stone age based around buying a woman from her father with sheep? Legitimately questionable if this is the best way to go about meeting society's current needs.

I also note that denialism is not searching for the truth, it's another way of being certain of an existing answer. Specifically it's saying "I know that this is wrong" and then changing your theory about how or why each time your old theory gets debunked. It's just another way of NOT questioning your existing beliefs to see if the world's moved on without you, as the world tends to do.

This is why science makes predictions. Waking up to find your swimming pool empty and guessing that invisible pixies drained it is a very different statement than saying "tonight, invisible pixies will drain the swimming pool" while it's still full. You can then wait and see what happens, and the statment is capable of being proven wrong. You have a way of noticing when the statement is not true, and that's the heart of science. If you only guess about what you already know, you can't tell when you're wrong.

So this is my problem with the baby boomers. If you search for the truth, there's a danger of thinking you've found it, at which point you stop questioning your beliefs, the world changes out from under you, and you slip into denialism about challenges to your existing belief. (This is without even getting into "smartphones have rendered wristwatches irrelevant for anyone under the age of 30, and you're telling me how great vinyl sounds" levels of inertia, but just sticking to the big important stuff.)

If VLC has focus and you type "make", you 1) mute the audio, 2) screw up the aspect ratio, 3) screw up the audio delay, 4) pause the playback by advancing one frame.

Typing "make" in the window again does not fix it.

My phone has bluetooth, but apparently the OS upgrade made the bluetooth file browser go away? Or did I only have that in the old Nexus One and not the Galaxy S?

Either way, it goes "Bluetooth! I can use that to have the laptop display sound, because its speakers are even worse than the phone's!" And that's about it.

Bravo, Google. Bravo.

(Ah, I have to download an app to do what I thought would obviously be built-in functionality. Hmmm... I wonder if this is enough to do podcasts now? Got a recording app, got something to get the files off the device, I need to dig up a screencasty thing...)

Passing comment from someone that Bill Gates dropped out of college so why can't they?

Apart from the fact his father was a lawyer and his mother was on the board of directors of the Red Cross. William H Gates III ("Trey" to his friends) really didn't have to worry about starving to death if he dropped out of the college his parents were paying for.

Just a thought. There's a _reason_ that Steve Jobs needed venture capital funding to launch Apple (costing him control of his company in 1984), and "Trey" didn't to launch Microsoft (he still owned over 40% of all Microsoft stock well into the 90's).

Yes, this sort of detail is important. As we saw when Jobs came back, when the two went head to head Jobs walked all over Gates. Repeatedly. But when Jobs said "Apple II is dead, Macintosh is the future" in 1984 (after the Lisa flopped), his board of directors rebelled and forbid him to divert resources away from the company's cash cow to invest in a sequel to the Lisa, eventually taking all authority away from him when he didn't listen. Gates's pet engineers didn't even BADLY clone the Macintosh until 1990 (and then only because an engineer working on his own initiative surprised them and forced them to chagne direction), and didn't catch up until 1995. Apple squandered a ten year head start and came crawling back to jobs who handed over the Macintosh sequel he'd finished the previous decade, and renamed it the iMac. Meanwhile Jobs had bought Industrial Light and Magic's digital effects arm when George Lucas sold it cheap after the original Tron flopped in theatres, renamed it "Pixar", and turned himself into a Hollywood movie mogul who eventually wound up the largest shareholder in Disney.

Of such details was the computer industry forged...

Whenever I reboot my netbook I lose buckets of state: open command line windows with half-edited files, todo lists, just directories that remind me "oh, this thing you were doing". Balsa won't save its state so I lose windows where I hit "reply" on a message but haven't composed and sent it yet...

Unfortunately, during one of the "apply package updates", Ubuntu swapped out its upstream crypto certificates, deleted the old ones, and needs to reboot to use the new ones. (I think that's what happened.) So I can't do any more package upagrades until I reboot, and that includes installing "debootstrap" and "debuild" so I can once again dink at bootstrapping debian under Aboriginal.

(Well, I could tell it to install packages from untrusted sources, but if the reboot doesn't fix that it's reinstall time.)

Huh. If you google for "toybox linux" the first hit is the toybox about page and the second's the news page... but it says "toybox - Rob Landley". Which is odd, because the <title> tag just says toybox. Where is it getting that? (The top level landley.net page's title is "Hello world!". The only mentions of my name except in release notes are on the license page...

Sigh. Google is trying to help out in the kitchen by dropping an egg on your shoes and getting flour everywhere. Google is helpful in ways you didn't ask it to be! How do I pat google on the head and try to distract it so it stops being helpful...

And _now_ Texas LinuxFest gets back to me, asking if I can do my proposed "why the GPL is dying" as a lightning talk instead of a full slot.

Alas, when they didn't get back to me by their own deadline (the 15th) I booked tickets to visit my family in Austin the previous weekend, where I get that monday off anyway and can spend all the time at home.

No biggie, Eben Moglen just covered the topic, and I already did three minutes on it in the GPL section of my ELC talk. (Yes, you can link to a starting time in a youtube video. :)

April 17, 2013

April 16, 2013

Sleep. Sleep would be good. I should try that sometime.

April 10, 2013

April 6, 2013

April 5, 2013

April 2, 2013

March 31, 2013

March 28, 2013

March 27, 2013

March 25, 2013

And with school letting out early (cancelled), the horde returns...

March 16, 2013

March 15, 2013

March 14, 2013

March 13, 2013

March 9, 2013

March 7, 2013

March 6, 2013

Thanks to Twitter's continuing policy of destroying anything its users like (currently tweetdeck), I have a tumblr now. No idea what to do with it, but there it is.

March 4, 2013

March 3, 2013

Weekend with the niecephews and Adrienne. Exhausting. Think I caught something.

February 28, 2013

February 27, 2013

February 26, 2013

February 25, 2013

February 23, 2013

Travel day to go home. Sitting at the airport sad my netbook battery died overnight and I lost all my open windows. (The acer bios has this "feature" where if the power goes to 5% the suspended system wakes up, presumably so windows can suspend to disk. Ubuntu 12.04 does not suspend to disk in this instance, it just sits there rapidly draining the drags of the battery, even with the lid closed. There is no way to tell the bios NOT to do this.)

February 22, 2013

Finally finished listening to Paul Krugman's End This Depression Now author talk, which was full of brilliant insights (7 minutes 40 seconds: treating the economic crisis as a morality play is wrong because the people suffering are not the people who sinned).

One of the things he said is that if you look at how the population of Canada is clustered along its southern border, "Canada is closer to the US than to itself". What this made me realize is that Canada is what the US would have looked like without slavery.

The book 1493 details (among other things) how the Jamestown colony introduced Malaria to the new world (which persisted until DDT finally eradicated it in the 1940's). Within a century it had spread up and down the east coast, but the mosquitoes that spread it lived much longer in the south than in the cold northern winters. Malaria made the southern half of the country almost uninhabitable, to the point where the practice of "seasoning" arose, importing the european poor to work as servants in america meant giving them nothing to do for the first year because half of them wouldn't survive the malaria they'd inevitably contract.

A 50% mortality rate within the first year made "getting good help" from europe very difficult (and extremely expensive), so plantation owners looked around for people with natural resistance to malaria. They found them where the disease originated; the native people of West Africa have evolved significant resistance to malaria. (In fact the gene that gives you Sickle Cell Anemia if you have two copies gives you near-immunity to malaria if you have one copy. That's why it's persisted in the population: the 1/4 of the population with two sickle-cell genes dies of sickle cell, the 1/4 with two "good" genes dies of malaria, the half with one copy of each gene survive. The massive death is mostly written down as "infant mortality", so they have lots of babies to make up for it. If you were wondering why religious people are so against teaching evolution: suppressing it is easier than trying to explain how a just and loving god would allow it.)

Of course west Africans had no cultural affinity with europeans, so getting them to voluntarily move to north america in large numbers wasn't feasible, so we sent ships over there to kidnap and enslave them. First in carribean island plantations (even harder-hit by malaria), and then on the mainland.

Since 1776 our politics has been dominated by a south full of ignorant bigots. Southern plantation owners were waited on hand and foot, and the rest of the south's population never needed to invent labor saving devices because slaves did the labor (agricultural and domestic). Even those who didn't own slaves grew up conditioned to treat other human beings as not just inherently inferior but as property.

Stop and think about the common practice of fathering children with female slaves, and then owning (even selling) your own children. This was common practice, considered "improving the breeding stock". (Founding father Thomas Jefferson did this, and refused to free even the subset of slaves he'd fathered, not even in his will after his death.) This requires epic hypocrasy and dissociation, so of course they turned to religion to provide it.

White southerners bent their religion around justifying slavery (bending the cain and abel story in the garden of eden to somehow say that whites _weren't_ descended from cain, or at least not as much as blacks), leading to the subtext of Southern Baptist and Evangelical denominations: "we are superior, the chosen people, justified in dominating all others because they aren't really human". Because their superiority was inherent there was no need for them to learn or improve themselves, and the idea of anyone _else_ doing so threatened their superiority which was against god's will. Ignorance became a virtue, all you needed to know was in the bible and if you start questioning whether slaves are people (or anything else) it needs to be whipped out of you. The industrial revolution passed them by, because the chosen people selling King Cotton didn't want any part of the learning required for it, because learning means questioning and we can't have that.

This _poisoned_ the culture of the south, and to this day they "cling to guns and religion" and insist "the south shall rise again" (because that worked out so well last time). An entire area of the country is culturally scarred.

The civil war didn't start this (bloody kansas) and it didn't end it either (jim crow). Since the first Republican president issued the emancipation proclaimation, the bigot vote went to the democrats for the next century (the original "solid south"). This polarity reversed when a Democratic president (Lyndon B. Johnson, Kennedy's vice president who became president when JFK was assassinated) overcame this legacy and in the wake of Martin Luther King Jr.'s assasination signed the Civil Rights Act. This alienated the racist vote, and Richard Nixon got elected with his "southern strategy" of coded appeals to the south's ubiquitous racism. The "Rockefeller vs Goldwater" divide was about explicitly rebasing the party on southern racism, and although Goldwater lost his election he won the battle to reshape his party's identity. This turned the "solid south" into a GOP stronghold that elected Regan, his VP Bush, and Bush's son. But more to the point, it elected all their congressmen and senators. The GOP became the party of rural ignorance, with subtle but pervasive racism as a core plank of the Republican party platform. A few plutocrats at the top steered a vast horde of ignorant racists, whose racism was the central unifying idea allowing them to be gathered and led. (The party would add a few more coalitions of people whose hot button issues allowed them to be led around by the nose: gun nuts, anti-abortionists, libertarians, and so on. People with a red flag that stopped all thought on any other topic. But by far racists were the largest group.)

Eventually the Democrats called them on it by running a black president, and when he got elected the GOP melted down into gibbering denial. They became "the party of NO" filibustering every bill (including ones they sponsored which he then supported), spinning insane conspiracy theories about how he couldn't possibly have actually gotten elected and must somehow not be real (birtherism), dedicating their entire agenda to nothing but preventing his reelection (triggering and then forcing the country to remain in an economic depression to make him look like a failure), willing to let the country default on its debt payments rather than vote for anything he'd sign...

This was far beyond rational hatred, after 40 years the racism of the party's base had seeped into the party's representatives. The ignorant racist bigots the party appealed to elected ignorant racist bigots to represent them, who went full "protecting our precious bodily fluids" nuts when confronted with a black president.

The plutocrats have been "riding a tiger". The goals of the billionaires who fund party activities are to reduce taxes on the wealthy (from 91% in 1963 to 28% under Regan, 15% on "capital gains" income from owning things instead of working, and many large corporations can avoid taxes entirely), reduce government regulation that prevents for-profit corporations from exploiting "caveat emptor" to the fullest, and to increase the amount of influence money buys in politics (see "Citizens United"). But to achieve these goals, they need to rile up their audience and feed them the occasional win in areas the plutocrats don't care about.

The point of all this is canada was too far north for Malaria to survive so had no need of a west african labor force resistant to the disease, and it had separate political representation where southern slaveowners didn't get to shape national politics.

So all this "conservative" mess with ultra-rich bastards cornering the market and treating the poor as inhuman because their culture says they should have slaves, and their religion says that other people deserve to be in bondange... Canda shows us what the USA itself might have looked like without the stain of slavery. Between federal government and population mobility, even the northern half of the USA is tainted with signifcant amounts of the south's legacy. Canada lives on the same continent, comes from the same "northern european dominator party mix"... they're what we might have been if we'd let the conferderacy become another Mexico and washed our hands of the slaveholders.

(Mexico is a different story, mostly settled by the spanish instead of the english. Spain has its own racist past evicting the muslim Moors and then turning the inquisition on themselves when they ran out of rich foriegners to kill and take their stuff.)

If you hear about republicans switching from Rockefeller to Goldwater, it's about the Goldwater's southern strategy explicitly rebasing the party on southern racism.

February 21, 2013

Ah. If we're up against the zero lower bound in short term interest rates, why aren't long term interest rates at zero too? Because we have about 2% annual inflation. So a zero short ter interest rate should translate to about a 2% long term bond yield, and if long term bond yields are below the rate of inflation (which they are), people are willing to lose money to hold these bonds, they're just losing _less_ than if they were holding cash.

Right, that makes sense. (Took me a while to work that out. Listening to a Paul Krugman interview on youtube on my phone remains educational...)

On the CELF front: talk went well. The room seemed full, although it's hard to tell with a light in your face and there was only one question from the audience. My brain is now completely fried, of course, hence blogging about other things.

February 20, 2013

February 19, 2013

California is 2 timezones west of minnesota. My flight on southwest was a bit over 2 hours late. It was the latest one I could get so I didn't have to leave work earlier than necessary. The BART trains stop running at midnight. (Luckily, the airport has shuttlecraft.)

Got to bed at 1am here, which is 3am where I got up, and I got up at 7am. First scheduled event tomorrow is 9am. May be a bit zombie-ish in the morning.

February 14, 2013

February 13, 2013

February 12, 2013

February 11, 2013

February 9, 2013

I generally ignore my Android phone's "Upgrade! Now! Full-screen-popups until you yield to this!" blather because I'd done it exactly once: and the _only_ thing that did was break tethering. (Sprint installed an oh-no-you-don't patch, disguised as a standalone upgrade. First one in the queue. So it worked fine _until_ I upgraded.)

Since then my phone gradually bit-rotted (and probably got who knows what viruses since it had no security patches) until it lost the ability to mount as a USB stick to get photos and video off. It still _charges_, but wouldn't present the "turn on USB storage" dialog. So I gave in and upgraded in hopes it would fix this.

Note: upgrading android has never fixed the specific thing that was so broken I gave in and upgraded. And this is no exception: plugging in as USB still doesn't let me copy data off anymore. Maybe the USB connector got damaged, or the cable went bad: I dunno, it won't tell me.

But it did fix the "app store", so I could upgrade the apps. The netflix client stopped working months ago, so I told it to upgrade everything. Including tweetcaster, which was now unusable because the Java Swing "look and feel" plugin got upgraded out from under it and now there were two layers of un-dismissable menu bars always there vertically along the left side of the screen (!?!?) taking up about 2/3 of the window and there was so little room that it couldn't display one whole tweet (one letter at a time vertically down the right edge, with the occasional "and" or "of" collated) .

So I had to update tweetcaster too, and WOW the new tweetcaster sucks. Have the developers ever tried to use it?

Ok, dismissing the fact that every time I go to a new part of the thing I've been using for years it gives me a full-screen pop-up about what's changed which I dismiss without looking at because it's incredibly rude. It only does that once for every single page and mode in the entire program of which there are far more than there should be.

Half of what I use twitter for is opening links. Android has a built-in web browser, but the tweetcaster guys decided that calling an external app worked too well, so they did a built in sucky modal browser. There's no way to turn this off in "settings" (I looked _hard_). The browser does have an "open in browser" option in the menu which calls the external one, but if you do that tweetcaster will crash when you return and lose your place in the twittervstream. (Every time it boots it jumps to the most recent tweet, not the last one read. In fact it loses all that history, and you have to reload and hope twitter feels like letting you scroll that far back. This is because remembering 1024 or so 256-byte-ish data packets (140 chars plus metadata) would BLOW TWEETCASTER'S TINY LITTLE MIND. 256k of data? THAT'S INCONCEIVABLE!)

You can't pull up the browser menu before tweetcaster's built-in modal browser finishes loading, or it will crash. If you try to load a page with funky javascript (such as any links to the new york times) in the built-in modal browser, it will crash. You can't ever press the "back" button from tweetcaster (instead of the home button) or it will exit and lose all context (I.E. crash). To exit the built-in modal browser, you must press the back button (EXACTLY ONCE).

The "tap on a tweet" menu used to have maybe 3 entries. It's now 9 plus tweet-specific entries, so big you usually have to scroll the screen even though they made the entries narrower and harder to hit. None of these entries are "copy link to clipboard" (or "open in the real browser"), you can only copy the entire tweet to the clipboard, go into the browser, and edit it there. You can also let the tweet load in the built-in browser (which is modal so tweetcaster can do NOTHING until you finish with that page and go back, but only hit back _once_ or it'll lose your tweets), and from the built-in modal browser you can copy just the link. (Except not the expanded link, the t.co URL shortener version that gives you no info about what it points to.)

The Android built-in browser is less useful too. There use to be a "+" button to open a new tab, then hold over the URL part until a "paste" option came up. Now you have to hit the "tabs" button (once you figure out that's what that squiggle means, and it's only there when the URL bar is being displayed so you have to scroll up to make it appear) to see your tabs in funky macintosh-style hover windows and from _there_ you can hit a plus. This gives you a new page that auto-loads google. Hold on the URL bar brings you into edit mode. Delete that URL to get an empty URL bar and then press and hold again and THAT gives you a paste option. Now carefully delete the surplus tweet context that tweetcaster put around the URL you want, and note that the right edge you need to click at so you can remove that last character (because there's no delete to the right of the cursor "del" button, only a delete to the left of the cursor "backspace" button, and no cursor keys, and deleting text never frees up space at the right edge of the URL bar unless the URL is too short to fill up the bar) is exactly TWO PIXELS away from the "delete entire contents of the URL bar with no confirmation or undo" button.

This is just one example. Just about everything that used to be one click in the old android is now two or more, on the same tiny cramped screen. The average number of clicks to do anything has doubled, and they call this an advance. (Oh, and instead of the new page appearing, there's an animated swoop adding a quarter second delay between each page so if you mentally macro redundant clicks together and your thumb goes too fast: too bad, button's not there yet. Of _course_ there's no way to turn this gratuious bling off, not that I've found yet. The stupid "collapse screen to a vertical white line as if it was a 1950's television when you hit the sleep button" is the most annoying so far, it gets old after about the third time, by the 100th its developer is your mortal enemy.)

Yet another regular installment of lateral progress. It's not an upgrade if the new software can't do what the old one could, or can't NOT do what the old one didn't make you do. (Proper open source development can sometimes address this, although not always, as Gnome 3 and KDE 4 demonstrated. But at least people could fork the old ones, or provide alternative projects. But android isn't open source: it's regularly updated abandonware. Periodically releasing stale source code into the wild is not the same as visibility and input into its development. Android allows no access to the actual design or implementation process, you first hear about it long after the fact and take it or leave it with no say in it. If you have comments... who cares about THAT old version, they're already several months into the one that will replace it and there are BIG CHANGES in store, just wait and see!)

What I really want is a fresh security-patched version of the old install. Not something the vendor will ever provide, even thought I keep sending Spring a monthly fee that includes paying for this phone on an installment plan (through October).

Just called them to confirm the "through octboer" thing, and ask how much extra they'd charge me per month to remove the tethering restraint. They say they no longer OFFER that option, instead tried to sell me a "wireless mobile hotspot".

Up yours, sprint. I know what the phone can do. Even if I wasn't an embedded Linux developer, I _did_ it for a month before you disabled it, and the menu entries to do it are still there (it's just the cellular signal strength instantly drops from 4 bars to 0 when you enable that, and jumps right back when you switch it off).

I'm going to go watch netflix videos on my phone. I may leave some long ones running while I do other things, for pleasant background noise...

I just spent the entire evening making my toybox todo heap even longer. It now includes analysis of what's in klibc, sash, sbase, beastiebox, and nash. Tomorrow I need to look at s6. And embutils, elkscmd, 9base, dracut...

I don't intend to use any of their code, but I want to see what list of commands they consider important.

So somebody requested arp, and it's resonably self-contained as network things go, in theory. In practice: the arp man page is horrible.

Writing a new command: not so hard. Figuring out what the old one can be told to do, and how to tell it to do it: hard. Oh well, start with the subset I can work out and wait for people to complain, I suppose.

I'm vaguely amused at the zombie tinycc project, which is trying hard to get a release out after only 3 years of stagnation.

Lemme rephrase that: in early December two different people pestered the maintainer about having a release and he replied (and I quote): "Well, release would be fine for me. Honestly, even better if someone else could do it." (Yes, this is the "maintainer" who opened a "mob branch" anybody could check into, so he didn't even have to review commits.)

So two months later they're still talking about it, and the people who are not him have done some actual work. Not with an eye towards building the kernel or busybox or anything like that. Or building any specific packages at all, that I can tell. Just having a release to have one, it seems. (I haven't been reading too closely, but none of the reports were "I tried to build package X and this broke", because they're just not there yet. Instead they've got threads arguing about how you do a stack trace.)

This burst of effort means that they're scrubbing the tree and finally noticing that they've hardwired gcc into their makefiles, and there's still plenty of code in there they don't understand and are afraid to touch. Most recently, the maintainer suggested doing nothing for a week. (Why? No idea.)

Still, they got the website link to their web archive updated. That only took one year.

This is what they chose instead of my fork, and they're welcome to it. (Actually I'm not bitter, just amused. I've got my hands full with toybox at the moment, and when I do get back around to qcc I've got permission to license Fabrice's original code BSD so I've got triage work to do even on _my_ fork. I honestly couldn't use anything out of that tcc branch anyway, even if it did look interesting. I'm just sad a good codebase got a maintainer who openly disrespects it.)

Finally got a chance to fix the toybox cp command so it doesn't break the Aboriginal Linux build. Which means I need to cut a release, and that's currently the only thing in it, so I'm shaking the tree for low-hanging fruit which would be good to stick into a release. I'm partway through a gazillion things (where "partway" sometimes means I did the research to see what's involved in implementing it, but haven't started yet). The time command's trivial, at least if all you want is posix -p mode. Groups is just id -Gn... and that doesn't work right for any users but the current one. Right.

Sigh. It's hard to come up with automated tests for things that require root access and care about how your system is configured. There's no point in creating tests that only pass on my system and which nobody else can reproduce, and I refuse to make them distro-specific either. Maybe I need to put together an aboriginal linux test chroot.

Amusingly I was pondering putting out an out of sequence Aboriginal release because I had a good stopping point before the kernel was ready. (I actually tested -rc3 for once.) But I've been so busy with other things the kernel's catching up...

If you follow the GOP, you might get the impression that plutocrats actually want poor people to suffer, as a goal in itself. And you'd get that impression because it's true. It's actually a fairly common thread throughout history, the end of slavery and indentured servitude is why "you can't get good help these days". Without starving illiterate peasants, the giant manors of Gosford Park or Downton Abbey can't afford staff. Education ended country manors just as emancipation ended plantations.

The thing is, billionares don't hire other billionaries to do menial tasks; they can't afford to. You need poor people to be rich relative to. Today's minimum-wage burger-flipper has access to prepaid cell phones, antibiotics and insecticides, fruits from the other side of the planet, meat every day if they want it (chicken's cheap and the McDouble is technically beef), the ability to keep food fresh for months or years, air conditioning, access to police and fire services, learning (libraries/Wikipedia/TED talks/Kahn Academy)... We take for granted wealth that kings didn't have a couple centuries back, but a minimum wage burger flipper is not rich.

Being rich is meaningless unless other people are poor, because rich people hire less rich people to work for them. That's what money is all about: you don't pay farms or mines or factories, you pay people to build and run them. Money always ultimately goes to people, not things.

Rich people's ability to hire lots of poor people is what makes them rich. Cheap labor is the end goal of management everywhere, paying less for what you buy and being paid more for what you sell is the heart of business. The ultimate expression of this is slavery, which is as old as recorded history: buy a laborer and own them outright.

Explicit slavery is out of fashion in the USA (but ask child brides sold for dowries how the rest of the world's doing), as is the Jim Crow replacement system of the "company store" where workers were perpetually in debt to their employers and paid with "scrip" instead of cash (non-portable coupons worthless anywhere else) so they couldn't afford to leave. Economic slavery is the end goal of the upper management of most corporations, if you're lucky you get "golden handcuffs". (This is not the same as job security: slaves were sold all the time.)

The people running these systems haven't changed. Plantation owners are now CEOs, and engineer stuff like H1-B visas (leave this job and you're deported, try for citizenship on your own time). They would LOVE to reintroduce slavery or jim crow, they just want to do so out of sight with sweatshops in India or the phillipines where they can get away with it.

Yes, plutocrats want poor people to suffer, even if they don't admit it to themselves. It's inherent in what they do. If everybody is rich, nobody is rich. The ultra-rich aren't trying to expand the pie so _everybody_ gets a bigger slice, they're playing a zero sum game where they can't win unless they beat somebody else.

For the GOP to win, others must lose. Opposition to health care isn't about the cost any more than fighting wars or union-busting. It's about keeping the uppity poor in line so their labor remains cheap.

Hmmm, there's a shell syntax for assigning values to variables if they're not currently set: "${varname:=value}". I'm pondering using it to replace my export_if_blank shell function in aboriginal, but haven't done so because it's not really a cleanup. It looks like evaluating a variable but is actually modifying its contents. It sets locally rather than exporting so I'd need to list the variable name twice (once in the ${} and again in an export statement; exporting is necessary for child processes to inherit the value). And it needs a command context to be evaluated in; you can cheat and use the ":" command (which is basically a synonym for "true"), but you've still got to have that at the start of every line with one of these on it. Which is subtle. I don't like unnecessary subtlety in programs.

So "new tool, very nice, doesn't help".

I'm enjoying working at Cray. Learning all sorts of stuff I didn't expect to. We had a brown bag on ftrace the other day, and even though I sat through the CELF keynote on it it never quite "clicked" the way it did in that brown bag. (I need to write a good intro. I should work out how to screencast videos, actually.) Plus in shell scripting, "exec" with just redirects but no new command name can redirect stdin and such persistently the current script, without resorting to elaborate parentheticals. (It's implicit in the susv4 description of exec, but easy to miss.)

Fun Ubuntu 12.04 bug: if you close the laptop lid while switched to a text console, the suspend process gets a "not authorized" which nicely pops up on the desktop (when you switch back to it), but leaves the thing running with the lid closed.

Oh right, klibc has executables in it. I should add that to the toybox roadmap (yet another use case to replace). Let's see, do a quick "echo $(for i in $(find . -type f); do file $i | grep -q executable && basename $i; done | grep -v '[.]g$' | sort -u)", filter out the *.so entries and the *.shared duplicates of static commands, and...

cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill kinit ln losetup ls minips mkdir mkfifo mknodes mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume run-init sh sha1hash sleep sync true umount uname zcat

Huh, no switch_root? Weird... Oh right, klibc called it run-init and I couldn't put a dash in a busybox command name so as long as I was renaming it anyway I called it switch_root as an analogy to pivot_root. Right. (My computer history research taught me the importance of tracking sources, so I'm usually very good at remembering where things came from... unless it was from me. Anything I could independently reproduce can't be that hard, QED.)

Ok, what does toybox _not_ currently have? (They call sha1sum "sha1hash", and "nuke" is "rm -rf --"... What the heck is mknodes? It... generates a .c source file? What? Ah, it's part of the build infrastructure for dash: mkinit, mksyntax, mknodes, mksignames all "hostprogs-y". And fixdep is another one.)

Ok, looks like:

cpio dd fstype gunzip gzip halt ipconfig kinit minips mount mv nfsmount pivot_root poweroff reboot resume sh umount zcat

Not a bad list. I'm partway through mount/umount already (which should include nfs and a cifs support, and fstype is related), init needs doing (which includes halt, poweroff, and reboot). I did a gzip implementation in java back in the 90's (back before 1.1 added it to the standard library), so that's gzip, gunzip, and zcat I'm confortable doing. I have plans for cpio, dd, mv, pivot_root, and ps. Which leaves ipconfig, resume, and sh, two of which I know about and "resume" is... a strange tool that reads data from a swap partition and passes it to the kernel? I thought the kernel did that built-in? Weird.

Ok, when I get an hour I need to add all that to the roadmap properly. Ooh, and this (ENOENT is not an error when cleaning out initramfs). And if mounts exist in the old directory and there's a mount point for them in the new directory, --move the mount (the klibc code has special case handling of /dev, /proc, and /sys; sigh).

I cut a toybox release. It built the i686 target of aboriginal with cp and readlink enabled (and the corresponding busybox versions switched off), so that's probably a good stopping point. There's losetup in there too and several nice third party contributions and general cleanups.

I've been on the fence about doing an interstitial aboriginal release (the ls bug is embarassing) but the deciding factor is now simply that my poor netbook can't grind out a dozen targets in less than a full day, and I haven't got that much time left. The kernel's up to -rc4, I should have non-work internet again by the time that ships. (Since I've updated the perl patches before the merge window for once, I should post them to the list, but balsa whitespace-damages everything and my python script to send stuff is still getting the header info subtly wrong. More todo items...)

My talk proposal for CELF ELC (on command selection in toybox) got accepted, so I should be in San Francisco in late February. First CELF I've attended since the Linux Foundation katamari'd over it, and I'm not attending the other stuff they've bundled with it, but the kernel.org guys are still nuts about key signing (because physical proximity proves you're harmless, that's why large men in dark alleys are so reassuring), and I should really deal with that. Besides, the hallway track is generally fun.

With a bit more time to think about it, the mknod case of cp -p isn't as horrible as I thought, because I can just create the sucker with the right permissions in the first place. For files the suid bit goes away if you modify the contents of the file, but with mknod I'm not modifying the contents. (Having the suid bit on a device node is kinda strange in the first place, but it's not my place to judge.) So the initial mask is pretty much right, modulo setting umask(0) for -p.

For directories I already make sure the sucker's writeable or I can't create new entries in it. This is another facet of the same atomicity problem: for a file O_RDWR|O_CREAT gives me a writeable filehandle even if the permission bits of the file it creates don't have the write bit. (This doesn't work on an NFS filesystem, which is NFS's problem not mine. Don't use NFS, it sucks in many subtle ways.) Creating a directory doesn't give me a filehandle, and if I do have a filehandle to a directory it's just a reference, permission checks get redone each transaction. (I think.)

Awful lot of fiddliness for 220 lines of code, isn't it? If you were wondering why I held off doing it for so long, I didn't want to leave it half-finished again. I want to get it _right_.

Still gotta do -sHL flags, re-read the posix spec to see what fun corner cases I might have missed, and plug it into aboriginal to find real-world breakage.

So fchmodat(AT_SYMLINK_NOFOLLOW) always fails, which is a glibc bug that uClibc blindly copied. That took a while to track down.

Context: I finally reached the end of a longish rathole making "mknod b blah 0 5 && cp -rp blah blah2" work in toybox cp.

Last time I mentioned I had to fork the -p logic so it could handle with cases where we could and couldn't get a filehandle. The ones where we naturally already have a filehandle to the entity we're trying to set permissions, ownership, and timestamps on are nice and secure: there's no gap between fiddling with the object and fixing up its permissions where somebody could swap in a different object.

But mknod() doesn't give us a filehandle to the node it just created. That's ok, mkdir() doesn't either, in that case we can just open it on the next line and perform a longish dance of verifying that we didn't follow a symlink and that what we have now is a directory. Except that if mknod created a node that refers to a device for which no driver is currently loaded, we can't open it. (Even with 0 in place of O_RDONLY because O_RDONLY already _is_ 0. You can't _not_ request read access.)

Well, that's not quite true: 2.6.39 added O_PATH which gives you a filehandle for use with fchdir() and friends. It's not in Ubuntu 12.04 libc headers yet, but I can look up the value and #define it myself, and then open() accepts it. But if I try to set permissions through the resulting filehandle with fchmodat() it returns "Bad file descriptor". (It doesn't mind opening something that isn't a directory, but then there's _nothing_ you can do with the result.)

So rather than follow the mknod with an open(O_NOFOLLOW) I set the filehandle to AT_FDCWD and down in the -p logic I test for that and use fchownat(), futimensat(), and fchmodat() in that case (which modify a name relative to a directory filehandle), and in the other case use the more secure fchown(), futimens(), and fchmod(). I'm tempted to collapse the two forks together and just always use the lookup-by-name versions, but the original three functions I was using DON'T re-look up the entity by name opening up a race window where somebody can swap in a different one. If I still have the filehandle to the file or directory I was just operating on, from a security perspective I want to use those original ones wherever possible and only fall back to the other three where that isn't an option.

All that I figured out a couple days ago, what ate a lot of debugging time since is the AT_SYMLINK_NOFOLLOW flag, which all three of the new permissions fiddling functions mention in the man pages. That flag tells it not to follow symlinks when modifying this entry, which is the behavior I want: symlinks are already handled further up in the function and -p doesn't apply to them because they don't have their own permission bits. (Hang on, they've got their own ownership and date stamps, so 2/3 of -p does apply. Right, another todo item.)

Anyway, the "race condition" I'm concerned about for cp mknod -p is a malicious user finding some cron job or something where root is doing a cp -a on something that has the suid bit set, setting up an inotify on the destination directory, and right when the resulting file is created deleting it and replacing it with a symlink. If cp then sets the suid bit on that symlink, it can give the user a suid executable with arbitrary contents, and congratulations you've cracked root on the system.

And, as I said at the top of this entry, fchmodat(AT_SYMLINK_NOFOLLOW) always fails. It doesn't even make a syscall, it tests for the flag in libc and throws a temper tantrum if it finds it. (Wrongbot's uClibc code even tests and fails twice: if any other flag is set, fail. If this flag is set, fail. Brilliant.)

They're doing that because there's no syscall, and they can't be bothered to come up with a workaround.

There's also no obvious way to get a filehandle to a symlink, so that _must_ operate by name.

I did an strace on the gnu/dammit version of cp to see what syscalls it uses, and it's doing lchown() with a path from the current directory. That's even _less_ secure, somebody could replace one of the intermediate directories with a symlink.

What I want is for open(O_PATH) to give me a filehandle to a directory entry that lets me modify the metadata but not the contents, and which can open a filehandle to a symlink rather than one the symlink points to. (This way, I don't need two -p codepaths, one that securely operates on filehandles and the other that _insecurely_ operates on names.)

Failing that, I want fchmodat() to implement AT_SYMLINK_NOFOLLOW.

Given that I've got neither of those, I'm going to have to come up with some horrible workaround. Grrr.

I'm continually frustrated by the way people keep calling money an illusion. It's a promise of value. This promise is an "illusion" the same way a signed contract is, or laws are, or shares of stock in a corporation. The difference between a promise and an illusion is that promises are honored and enforced by somebody specific.

Society is built on promises. The bill of rights is a list of promises. Citizenship is a promise. The only reason you can walk around safely is everybody else has promised not to kill you, and if they break that promise there are people promised to defend you a phone call away (police), and people promised to defend THEM (swat teams, national guard, army).

Yes, printing money is making new promises. But so is borrowing money: when you borrow money from a bank, the bank treats your promise to pay the money back as an asset. Every time you use a credit card, you're giving the bank a fresh promise to pay the money back, and the bank records that new promise in its books AS A TYPE OF MONEY. Larger transactions involve a web of interconnected promises: taking out a mortgage loan promises to give them your house if you don't pay (and promises that a third party sheriff will come and evict you from that house on the bank's behalf if necessary). Any of these promises can be broken, and the point of courts and judges and juries is to resolve broken promises without trial by combat or lynching.

I've previously called money just debt with a good makeup artist, but actually it's a slightly different type of promise. Taking out a mortgage loan promises to give them your house if you don't pay (and promises that a third party sheriff will come and evict you from that house on the bank's behalf if necessary). Using a credit card is just the raw promise that you'll pay it back.

The government prints money for the same reason the government passes laws: it's the organization we've delegated the authority to make really BIG promises to, and which is in charge of enforcing those promises. They print money the same way they issue laws. Sometimes it's a bad idea, but sometimes NOT doing it is a bad idea. (There are people who think every law is a bad idea. If they really think so they should boycott dollars, which are backed by laws saying they're legal tender.)

The government can collect taxes as an alternative to printing money, and the reason to do that is to prevent inflation. Except that right now the economy has a shortage of money in circulation, and because nobody is willing to take a pay cut this manifests as high unemployment instead of massive deflation. (Instead of prices going down 10%, everybody who has a job gets zero pay raises for five years while ten percent of the people have no job for all that time. This is called "downward nominal wage rigidity".) A couple years of 5% inflation would actually fix what ails the economy. At the expense of making rich people poorer, and we have the best congress money can buy so they've been fighting to keep it broken because fixing _other_ people's promblems doesn't interest rich people who don't need a job.)

Promises are vitally important to a functioning society. Taxes are backed by a promise to confiscate and imprison if you don't pay. Stop signs and traffic lights and lines painted on roads are backed by a promise that police will arrest you if you don't obey them. Writing a check is a promise that there's money in the account. When you deposit money in the bank, the bank is promising to give it back.

People who don't understand this think commodity money is different. "Gold isn't a promise!" Stick that person on a desert island with all the gold they can eat and come back in a year. Commodity money was always a promise of value. People who think having gold is somehow better than having dollar bills miss the point that neither is useful unless people are wiling to exchange them for things you want at a future date. If people stop being willing to exchange them (as happened to all sorts of commodity money: cocoa beans in south america, cowrie shells in the marshall islands) they lose their value.

Using metals as commodity money is no different, look at the wildly fluctuating price of silver: in 1840 it was $1.29/oz, but extensive silver mining drove down the price to 64 cents by 1900 and 34 cents 1940. Inflation brought that back up to $1.63/oz by 1970, and then commodity speculators drove it up to $16.39/oz in 1980... before crashing back to $4.06/oz in 1990. The value of silver has historically been as volatile as beanie babies and tickle-me-elmo dolls.

The gold bugs try deseprately to pretend that silver was always secondary to gold... except that Rome demanded tribute in talents of silver, the coin of Athens (the drachma, nicknamed the "owl") was 4.3 grams of silver from the mines at Laurion, the bible has Judas paid with thirty pieces of silver, "cross my palm with silver", and so on. In greco-roman times, gold was secondary to silver. (Sure gold was rarer and more valuable, but platinum was rarer and more valuable still, and who actually pays for anything with platinum bars? It doesn't get USED for anything outside of D&D campaigns and the recent attempt to exploit a legal loophole.)

The massive amounts of silver and gold brought back from the Americas changed gold's secondary status, because there was now enough of it to encounter regularly. The huge influx of precious metals also caused massive inflation throughout europe, which was GOOD. The increase in the money supply let the european economy expand without immediately going into the same kind of depression we're in now (a liquidity crisis: not enough money to let all the people buy all the things, so you have unemployed people sitting next to unsold things and no way to connect the two). When people got used to an expanding economy they invented paper money to keep the economy expanding after the extra gold supply dried up; a bigger economy needs more money or it slides into depression.

None of this is new, but it's worked so well for so long we forgot the difference between an aqueduct and a river. That we invented mechanisms to scale things beyond what "just happens naturally", and that those mechanisms need to be _operated_. The same idiots screaming that the melting glaciers are just happening without a cause, that evolution is a lie (and thus it's ok to use 80% of all antibiotics in cattle feed because antibiotic resistance is the will of Zeus instead of a trivial example of evolution in action), screaming to keep the government out of their medicare...

They're in control of the house of representatives and fillibustering the senate, and they're trying to cut spending in a depressed economy. They're literally arguing for an end to unemployment benefits when there are three job seekers for every position nationwide so it mathematically CAN'T matter how motivated the job seekers are, there's still more pegs than holes.

It's really, measurably stupid. There are REASONS they are wrong, if you just bother to understand the system, which they don't because they think they already know everything. But driving a car does not make you a mechanic. Getting rich doesn't teach you how the economy works any more than a farmer whose crops get enough rain learns how to predict (let alone control) the weather. My first year in Austin we had a bumper crop of crickets an inch deep, if there was such a thing as cricket farming somebody would have been rich, and would forever after insist they had earned it.

The "end the fed, printing money is a sin" idiots are the exact same guys who want to drown the government in a bathtub (no new taxes, no new laws), and who insist everyone carry a gun because the promise of protection offered by the police can't possibly be trusted. These people do not believe anyone else's promises, because their own are easily broken. They're dishonest bastards whose word is worthless, so of course they don't trust anybody else, which means they can only function in society with the aid of massive piles of cash to cover their perpetual lapses. They either wind up rich (theft is profitable) or in jail. Society doesn't (can't) work the way they think it does, and when they try to change it to fit their prejudices they destroy whatever portion of it they've managed to steal. (They're great at figuring out what they can get away with. They've got the House due to gerrymandering, is that theft? Losing the popular vote but gaining the electoral vote is their stock in trade, gaming the system is what they do. Because they don't trust promises, their own or anyone else's.)

The whole trillion dollar coin thing was just an end-run around these morons, who've worked a contradiction into the law. The government promised to spend a certain amount of money (the budget congress passed), and the government promised not to tax, borrow, or print enough money to spend the amount in that budget. Paradox. The promise breakers are trying to force the government to break its own promises, to show that all those OTHER promises it's issued (from laws to dollar bills) are as worthless as those from the GOP. The platinum coin loophole let the feds print enough money to keep their promises without hoping the crazies had a moment of clarity.

For a while there, people thought Obama had a spine. In reality, the stiffening effect was a little blue pill Obama takes to sustain elections, and his spine goes all floppy again afterwards.

Back dinking at toybox cp: the mknod should be using st_rdev (device ID of special file) instead of st_dev (device ID of containing filesystem). Oops.

Unfortunately, this doesn't fix the case where we just did a mknod for a major/minor pair with no associated driver, so any attempt to open it fails, so fchown() has no filehandle to operate on to change ownership. The reason is that O_RDONLY is 0, so you can't _not_ request read permission when opening a file. (Hello PDP-11 unix from 1970.) What I need is some kind of O_NOTHING bit that _just_ gives me a filehandle I can manipulate metadata through, but not contents. Some discussion wandered by on linux-kernel about this, but it's not in the open(2) man page of current ubuntu LTS.

For most things, implementing -p just means keeping the original filehandle open and doing some extra stuff on it at the end. This avoids race conditions where the file you copied data to or the directory you created files in gets replaced by another program between your first and second actions on it. (Which could be a security problem. Assume there's a suborned flash plugin running, but it shouldn't be able to crack root by dropping a symlink somewhere root's about to flip the suid bit. And this is why the multi-user nature of minicomputer Unix is still relevant on phones, it means your web browser can't install keylogging device drivers.)

Alas, mknodat() doesn't return a filehandle, so I have to go back and look it up by name to do further operations on it, with a gap that allows a different filesystem object to be substituted. I can feed all three functions SYMLINK_NOFOLLOW, but I'm still uncomfortable.

I'm also uncomfortable with having two codepaths that do the same thing. If I'm going to have fchownat() instead of fchown() shouldn't I force everything through it? (Except mknod is rare, and the filehandle version is more straightforward for the common case of files and directories. And I still need the equivalent of the filehandle anyway to signal whether or _not_ to perform the modifications because you don't for hardlinks and symlinks. Grrr.)

Sigh. The balsa email client is really frustrating.

Right click and copy link doesn't work, so I cut around the link and copy that instead. (Why that works and right click copy link puts nothing in the clipboard, no idea.) Email filtering doesn't work (the rules never triger), so I wrote a python wrapper that runs each time balsa exits and chops up the messages in the inbox mbox, shuffling them to the right folder.

This means the filtering only happens when I exit balsa, which I can't do if I have pending reply windows up, so I tend to read the raw unfiltered inbox a lot. (This means I'm reading a lot more of linux-kernel. It also means I'm days behind on email pretty often.)

So I'm seeing messages containing patches, which I need to collect and deal with later. Cut and paste converts tabs to spaces, but worse the balsa developers didn't bother to implement window scrolling during cut and paste. so I can only cut what's currently showing on the screen, and the patch never fits. So I want to right click and save the message: the balsa guys didn't implement that. So I want to right click and copy the message to different folder (I can create an empty one to collect them, then forward later. Note that fowarding messages _as_attachments_ does whitespace damage). But the balsa developers didn't implement "copy message" either. (Move yes, copy no.) View source: same scroll problem, and it gives me the raw mime with = escapes and wordwrap breakage.

I eventually found out how to get it to do what I want, double click on the message in the message list to pop it up in a window, then "message" (pulldown menu #5), then save current part.

Ubuntu LTS hasn't updated balsa since it shipped, and building it from source still requires installing dozens of packages beyond my patience for dealing with gratuitous dependencies. (I don't want a spell checker. Three spell checker packages and it still wants more: no.)

Sigh. I should restart my search for a viable email client, but trying to find reasonable user interface stuff on linux is an enormous production. (So instead I've written a python script to send email in a way that won't whitespace damage patches, although I've still got to fix the "to" addressing logic.)

I am not a fan of SELinux and friends. Maybe I'm being too hard on them, but I doubt it.

"Release early release often" is good advice, and I want to do that with Toybox. When I checked in cp, I started poking at a release.

My process for cutting a toybox release is to stick it into aboriginal linux and build all the targets. That's how I get the static binaries. It's also a good smoketest that it _does_ build with the new toybox commands replacing yet more of busybox.

Alas, cp didn't. Or rather it _almost_ did, but the gcc source tarball keeps re-extracting itself every time it's set up, which screws up parallel builds (because they try to rm/extract/patch the same thign at the same time, this is why "EXTRACT_ALL=1 ./download.sh" can do it all up front before a parallel build, instead of the lazy binding variety build.sh defaults to.)

The problem boils down to "cp -lf", because -f only applies to the IS_REG() file copy, and -l needs the same "it didn't work, delete the destination and try again" -f behavior to avoid cp returning an error code.

So I need to shuffle the code some more. The _easy_ thing I could do is stick in another goto. (Right now the -p logic to adjust the file's metadata after creating it has a goto so the directory creation and file creation can share the same code.) The -f retry stuff either needs to be duplciated or it needs to be able to jump _back_ to retry. With a counter variable so it doesn't potentially get stuck in a loop. Sigh. (My main objection to adding a big loop around everything is I have to reindent it all and it makes massive diff noise due to the whitespace change. Oh well.)Ok, what are the cases here: directory, hardlink, symlink, mknod (char, block, fifo, socket), file. Hardlink and symlink don't need -p, but the mknod variants do and currently aren't getting that right.

The -f unlink/retry logic is needed for all the cases, but only for failures to update the _target_, but not for failures to read the _source_. So copying a directory without -r, readlinkat() failing, openat() failing mean don't unlink/retry. (And, of course, not having -f.)

Sigh. Went through the work to recombobulate everything, and now it's giving me the 'src' is 'dest' complaint for a bunch of files. (And a different test with "sudo cp -rp /dev/null blah" is giving me a "no such device or address" error attempting to open the file for -p after the mknod.) More digging to do...

So I continue to hate git. If your repository is modified (such as having patches applied), "git show" appends a diff, which craps lots of text into build/MANIFEST that's just trying to show the version number. This broke my manifest generator in sources/functions.sh, so I go to fix it.

In mercurial if I don't remember how to ask for the current commit hash (and nothing else), I go "hg help" (83 lines), look at the list of commands, see "hg identify", do "hg help identify" (29 lines), and find out I can do hg identify -i or -n depending on whether I want the hash or commit number in the local repository. (The number is just the count of commits in the repository when this one was added, and since it's locally unique, a simple increasing number that's easy to understand, and my copy's the master for my projects, I use that to identify versions.)

Trying to ask git how to display the current commit hash, and NOT crap 12 random other thing out so I have to dig what I want out with sed, is a PROJECT. Starting with "git help" just shows a subset of the commands, you need "git help git" (not the same output, and 901 lines) to get the full list. Spending several minutes reading that, I still have no clue where to go next. I tried "man git show" (416 lines), and that wasn't it. I vaguely recalled "git display" which doesn't exist, and dig through my old blog entries to find git describe. So "git help describe" (153 lines) seems to say it can't do it. (The closest is "git describe --exact-match" which spits out an error message if there's no tag for this commit.)

Eventually I tried beating something out of "git log", and on line 789 of the man page it finally told me enough to figure out "git log -1 --format=%H".

This took more than 15 minutes, and the answer was found more or less by trial and error. Compare that to thirty seconds or so mercurial took to figure out the same thing. This is why I hate git.

Heh. Adobe recently released a 10 year old version of photoshop as freeware so they could shut down the old license server, except they say you should only use it if you bought a copy.

This reminds me of the way pirating your own product is a time-honored tradition within the software industry, going all the way back to the creation of proprietary software by the Apple vs Franklin lawsuit in 1983. Pirating your own stuff is a way of both attracting new users and suppressing competition, plus any product that _isn't_ pirated is viewed as a dud because obviously nobody wants it unless forced by their employer. (If nobody _bothered_ to crack it, it must be really bad.)

This is something almost everybody in the industry does, they just won't admit to it (except off the record at tech conferences). I'm not talking about junior engineers sneaking out copies, but the actual owner of the intellectual property uploading their crown jewels to bulletin board systems (or these days, bittorrent) as an intentional part of their business strategy. Years ago an ex-Microsoft engineer told me he watched Steve Ballmer personally upload "cracked" versions of the chinese/korean/indian translation of a new Windows release to various pirate sites because he would rather they run a pirated version of Windows than a non-windows OS. Far easier to convince users of pirated versions to "go legit" next upgrade than to convince Linux (or OS/2, or BeOS, or...) users to pay to switch to a different OS. (Of course Microsoft was founded by a law school dropout whose father is a lawyer, so they sue reflexively. The purpose of Microsoft's "Business Software Alliance" sock puppet was to shake down people who are already using the software. If nobody bothers to pirate it, there's nothing to sue over. Of course this doesn't always work...)

So if you were wondering about the l33t hacking skillz behind all that "zero day warez" stuff... yes there are people who can do that, but their services generally aren't required.

Personally, I prefer the honesty of the open source guys. We're also giving it away for free, the difference is we happily admit it. :)

(And yes, this is the same kind of thing that got Viacom in trouble when they sued youtube... over video Viacom's own employees had uploaded to youtube as part of their jobs. Exact same kind of self-piracy is near-ubiquitous in the software industry. With the exact same kind of secrecy, because they don't want to give up the right to sue.)

Finally got cp checked in. Lotsa fun fiddly corner cases, probably still plenty more I haven't tested yet, but it's to the point where it's worth testing an Aboriginal Linux build against it.

Actually I haven't implemented the mknod() or mkfifo() bits. (Right now it handles those as cat > dest, which isn't going to work particularly well for /dev/zero.) Hmmm, when does that kick in? (For cp -a, probably for cp -p too?) According to "man 2 stat" a file can be regular, extra crispy, rotisserie style... Ahem. Regular, directory, char, block, fifo, symlink or socket. (That's unix domain sockets, "man 7 unix". I have no idea how cp is supposed to handle domain sockets. I wonder if the spec says...?

I submitted three talk proposals to CELF (well, two talks and a BOF). One was "debugging your way out of a paper bag" which is just a bunch of war stories about debugging my way into the kernel and back again (and what happened after) with an eye towards never being intimiated by any problem you can reliably reproduce. (Queue Sark: "gdb can't help you now, my little program".) Another was about my experience rewriting the Linux command line from scratch twice (once in busybox and again in toybox: what a Linux system actually _needs_, the command selection criteria, what the actual standards are and why they're not good enough, etc). The third is a BOF about helping Android replace the PC, which is 5 minutes on my mainframe->minicomputer->micro/personal computer->smartphone rant, the need to be self-hosting, and how to get there. Then the rest of the time open to the floor (and unlike the darn compiler BOF it _NOT_ turning into me rambling for an hour while people ask me questions, although if Rich Felker showed up I'd totally hand the floor to him).

I doubt any of them will be approved because the Linux Foundation annexed the Consumer Electronic Linux Forum and all its functions now belong to the Master Control Program and together they are complete and so on, and those guys do not respond well to hobbyists. (I'm not sure they know what a hobbyist is, they wanted to know what company I represented in the application form.) But it seemed worth a try.

So I was really paranoid about testing the "rm" command, but testing "cp -r" turned out to be the dangerous one. Forgetting to discard "." and ".." while traversing: bad. Forgetting to add the check that source file and target file aren't the same device+inode: bad. (So it opens src/../blah and dest/../blah which are the same file, the latter with O_TRUNCATE, reads nothing from it, writes nothign to it, closes the now zero-length file.)

Having fairly recent backups: good. Still took a couple days to work through what got damaged and what needed to be restored.

Right, moving on from that: I noticed that cp contained a repeated idiom "if (!(x=strrchr(blah, '/'))) x=blah; else x++;" which is x=basename(blah). I started writing my own basename() because the man page said the libc one could modify its argument and couldn't figure out WHY, but while writing it I started adding the code to truncate trailing slashes and went "oh, that's why". Which is a case that the existing repeated code gets wrong, and the test suite doesn't check for ("mkdir one two; ln -s one/ two" and ln complains that two/ exists.)

So going through the tree and fixing that, and inserting some extra tests in the test suite. While doing that, I wound up cleaning up rmdir -p to deal with trailing and repeated slashes and spent the longest time trying to find a reveresed test. (Isn't that always the way: I knew what I meant not what I wrote, so had to go through the whole song and dance of narrowing it down to THIS line is where it deviated from my expectations and why is... because there's a ! missing. Right.)

Meanwhile the -rc2 kernel shipped a couple days ago and that's about the point where I should start paying attention. Alas, my netbook is not much of a build machine and I goodwilled securitybreach (nowhere to set it up, it's LOUD, turns any closet into a sauna, and turning it on only when you need it's no fun because it takes several minutes to boot up with all the Dell server BIOS crap). An equivalent replacement system is about $600 these days (hyper-threaded 4-way with 32 gigs ram and a terrabyte of disk), I might get one in Minnesota.

People are talking about the debt ceiling and how Obama is too spinless to print money (via the platinum coin loophole) to render the nihlist GOP irrelevant. So here's another thing Obama's too spineless to do.

If we hit the debt ceiling, state payments should go to those who voted to honor the government's obligations (I.E. raise the debt ceiling to tax, print, or borrow money they've already passed laws obligating the government to spend). Divide each state's payment by the number of representatives in the blocking body and award one share for each yes vote. So if a state has 7 representatives in the house and 3 voted yes, they get 3/7 of the money they'd otherwise be entitled to. This includes social security checks, medicaid, VA benefits, and all military contracts payable to entities in that state (which are collectively the majority of the budget).

Note this amount will be payable from tax revenues, because the percentage of "yes" votes that goes over the amount payable from tax is enough to raise the debt ceiling. It impartially focuses the default on those who voted to default.

When you cp -R the destination could technically have a symlink where you expect a directory, and thus follow it to copy into areas it shouldn't. This is pretty clearly pilot error, though.

I need to have perror_msg() set TT.exitval = 1 and do a cleanup pass removing all the resulting unnecessary manual exit value setting. Ok, switch gears and deal with that...

Trimming some error messages while I'm here. The point of perror_msg("blah") is that it prints "commandname: blah: syserror output". So instead of realpath going "cannot access '%s'", if it just says "%s" you get "realpath: input/file/name: Discombobulated Inode" or whatever the problem you had was. The _big_ advangtage of this is there's nothing to translate to other languages. The command name's mostly posix, and the error messages come from libc (which should be locale appropriate for us).

Hmmm, there's a repeated idiom (head, cat, wc, cksum, dos2unix...) of:

for(;;) {
  int len = read();
  if (len < 0) error_msg();
  if (len < 1) break;
  do stuff;
}

Not quite sure how to factor that out into a library function. (Ok, I could trivially make a MACRO but that's not the point. That doesn't eliminate duplicate code, it just hides it.)

Sigh. Part of the reason I'm doing a manual pass on all the "find toys -name *.c | xargs grep -A 3 error_msg" output is to find possible cases where error_msg _shouldn't_ set the error return code. I found one: in passwd.c it's doing error_msg("Success") and there is a REASON that command is in the "pending todo items" category in status.html. Need a big cleanup pass on that...

And now cp -p has to care about nanoseconds. (Standards are moving targets when you mothball a project for a few years...)

Memo: when writing a new cp implementation, the first time you test cp -r is _not_ the best time to find out A) you forgot to filter out "." and ".." from the directory traversal, B) you forgot to check the source/target files for dev/inode equivalence before opening O_TRUNC and copying the contents.

Sigh. This is why I have backups...

Digging through the clutter in the guest room and moving bits to storage, I found my previous set of glasses in a box, and... wow. It's _so_much_better_ than my current set, even with the damaged lenses. (Apparently the lens coating was soluble in bug spray.) Distance viewing isn't the greatest, but up close I can FOCUS ON THE SCREEN.

I had no idea how much my ability to concentrate was screwed up by having to reread each line three times at enormous font size to see it, and getting massive eyestrain every ten minutes. (Well, I suspected, but _dude_.)

Right, cp. Dig dig... (I feel like Agatha Heterodyne dealing with Gilgamesh Wulfenbach's falling machine: that can go, that can go...) I need to redesign xreadlink() because I mostly need readlinkat() instead. Really it should do the "feed it NULL and it allocates its own damn buffer" trick.

All the xblah() wrappers exit the program if an error that should never happen actually happens. In the case of memory allocation, malloc() should never return NULL because all it does is allocate virtual address space which gets asynchronously populated later as we actually dirty the memory and generate page faults.

The page fault handler can reclaim physical pages from other users and write them to disk, and use the freed up page to satisfy the allocation. (This is called "page stealing". You give it back when the other process needs it, by stealing it back from _this_ process. Or get another one from somebody else. Or if you're lucky other allocations will be freed by then and there will be actually unused memory. And yes, you can steal pages from elsewhere in the same process. Happens all the time.)

If a page was allocated via mmap() on a file, the page can be written back to that file (or simply discarded if it hasn't changed since it was read from the file, this is why there's a "dirty bit" in the page tables). If it's an "anonymous" mapping (no associated file, technically mmap(NULL)) the page fault handler can write the page out to swap space. Of course what the fault handler _does_ is suspend the faulting program, schedule a DMA transfer, set some _other_ program running, and then when the DMA finishes it sends an interrupt to notify the fault handler the write is done and it can go reuse that page now, so it hooks it up to the faulting process's page tables, either zeroes it out or schedules a _second_ read to pull in the appropriate contents from mmap(file) or swap, and then when _that's_ done it unblocks the program and allows the attempt to access the now-restored page to continue. So you're overlapping processing with I/O wherever possible.

Anyway, the point is you're not really out of memory until you exhaust physical memory AND swap space AND all the mmap() pages. And of course disk cache, which is another big user of physical memory (usually about 1/3 of it; if you allocate all your pages for programs disk access will slow to a crawl, every time you open /var/log/wtmp it has to read through the root directory to find "var", go open the file containing the contents, read through _that_ to find "log", go open the file containing that's contents, read through to find "wtmp", look up where _that_ stores its data... We're talking dozens of disk accesses to track down where 1000 bytes actually lives on disk. You don't notice because it caches all that after the first access.

It turns out that if you pin the system against a wall with allocations it starts thrashing. Long before it actually deadlocks itself (gets into a position where there are NO pages left to satisfy any allocations, and every running process has the next page it needs swapped out and everybody's blocked waiting on everybody else), the system runs out of good decisions to make when it needs to steal a page. So instead of stealing a page that won't be used for a while, it steals a page that the process will need soon. So every time the process is unblocked it's almost immediately re-blocked needing to swap in the next page, and the entire system slows to a crawl spending all of its time swapping out pages that immediately get swapped back in.

A badly thrashing system slows to a crawl, to the point where tasks that ordinarily take a fraction of a second take minutes to complete. If work is coming in externally (such as to a web server), every request starts timing out. A server that gets into this state may never clear itself without external intervention because instead of handling 10,000 requests a second it's handling twelve (and those twelve are erroring out and aborting but it takes 5 minutes of grinding for the system to come to that conclusion).

Note that thrashing means a _transient_ load spike will take a server down and keep it down. If your normal 500 requests per second suddenly spikes to 20,000 requests per second, and then goes back to normal after a minute or so... the server stays down. Because if it's only retiring 12 requests per second, the normal 500/second is still well above what it can handle.

This is why a good web server will respond to load spikes by dropping connections early. It'll back down a bit from the edge and cap its load at maybe 8000 simultaneous transactions it can handle safely and everybody else gets a static "sorry, try again" page (or just a dropped connection with no data). If they hit retry, they'll probably be in the 8000 this time around (or the time after that), and get their page in the same 1/2 second it takes everybody else. This means that transient load spikes clear immediately, and even during the spike people pounding "retry" aren't actually making things worse. And nobody's waiting 3 minutes to see _if_ the page could load.

The linux OOM killer is similar: if the system deadlocks, kill processes until you've freed up enough memory for the survivors to continue. Unfortunately, this is a nasty problem. Ignoring for the moment the decision of WHICH processes to kill (which ain't easy), how do you tell when the system's deadlocked? It goes through thrashing first, and thrashing can literally take DAYS to resolve into an actual deadlock. I rememer the 2.4.7 kernel with a memory manager prone to thrashing when I opened too many browser windows. More than once I left a system thrashing when I went to lunch (no more requests, just finish the tasks you're already processing), and when I got back sometimes it had recovered, sometimes it had deadlocked, and more than once it was STILL THRASHING. The difference from a user's perspective between thrashing and deadlock isn't that big: "I move the mouse and cursor doesn't respond for 30 seconds" is equally useless from both causes.

This _infuriates_ the mathematicians, of course. People who want perfect behavior from the system, and believe that intentionally discarding a user request is just offensive. If you're wondering where the phrase "the perfect is the enemy of the good" comes from, it's these guys.

These are the same people who insist that the return value of malloc() should tell you whether or not there was enough memory. If you just leave an "emergency pool" of enough memory idle and only used in emergency situations... except defining an "emergency situation" is a subset of "figuring out when we're thrashing", and every time you try to work out how big the pool has to be to guarantee you can recover from thrashing you wind up doubling the amount of memory in the system and leaving half of it idle and STILL not being sure it's enough...

Why so much? Because memory is shared, because clean memory can be dirtied, because when a gigabyte-sized pig like firefox or openoffice calls fork() the new copy might dirty all its memory, but most of the time it will just exec() a new program and discard its existing mappings... and you can't tell. Predicting a program's future memory access patterns is PREDICTING THE FUTURE.

The relationship between virtual and physical address space is illusory. The easiest way for a 32 bit system to run out of virtual address space is to attempt to mmap() large files, so when you _do_ run out it's not because you ran out of physical memory. At the other end, running a 1000 copies of "bash" involves each process mapping the bash executable into its address space, and the way dynamic linking works (self-modifying-code!) every page of that mapping _could_ be dirtied. The operating system can't tell. The bash executable is about a megabyte, and bash forks a new short-lived copy of itself every time you use parentheses or pipes. In the real world, how does vetoing process creation halfway through a shell script (because the number of _potential_ writeable pages in the bash executable mapping goes over the physical memory installed in the system) differ from the OOM killer? By the time you ask how many pages of buffer two ends of a pipeline might have in flight between them, how much disk cache a process is allowed to dirty writing files to disk before you block the process and let the buffer drain... the ivory tower burns to the ground before you even get to the _hard_ questions.

So let's get back to "checking for a NULL return from malloc and attempting to actually _recover_ is useless". If this ever actually happens, the system is probably hosed to the point of needing a reboot. Your recovery code will never get any non-artificial testing. All the page fault handler can do is send the program a signal to kill it (the program has moved on by the time the system _actually_ runs out of memory), so the point of the wrapper is to make it so nobody _else_ ever has to check. If it triggers, it's like disk errors coming back from rotating media: the system needs medical attention.

(Note: disk cache means actual problems writing the files to disk often happen after the file is closed and the process has exited. The write() just put data into disk cache, if the program already exited since then there's nobody TO notify of an error. And the _reason_ we do this is waiting around for the results is orders of magnitude slower. The chef does not stand next to the diners to see how big the resulting tip is before starting to prepare the next meal, real life doesn't WORK that way.)

(Yes, nommu systems have no virtual address space, and their allocations can fail due ot fragmentation instead of just resource exhaustion. But again, "kill the program if this happens" is a pretty reasonable reaction.)

December 30, 2012

December 27, 2012

December 26, 2012

December 25, 2012

December 24, 2012

December 23, 2012

How does one test a dog for vampirism? Apparently garlic is bad for dogs anyway, so it's same as the "If you drive a stake through its heart, it dies" test not really being a good way to determine whether something is or isn't vampiric in nature.

Very time consuming dog. Very energy consuming dog. Very seperation anxiety sleep deprivation dog.

December 21, 2012

Why does nobody in washington understand basic economics? This "fiscal cliff" nonsense _can't_ raise interest rates because that's not how interest works.

Interest rates are the return on investment you can get when you loan out money. When the economy sucks, interest rates go down because nobody is making any money, so nobody can afford to make the payments if they borrow more money at high rates. (The people willing to do so don't qualify for the loan. Yes, lenders can always get higher rates by accepting more risk, but beyond a certain point you're just gambling and lose more money to defaults.)

The current economy is stuck in a type of stall we haven't seen since the 1930's: nobody's making any money because nobody's spending any money, and nobody's spending money because nobody's making any money. Living off savings is terrifying if you can't replace them. This problem is easy enough to fix with something like FDR's "new deal" where the entity that can print money (and thus can never run out) buys a bunch of everything to push the economy back up to speed. But we've got a really really BIG economy and it takes a LOT of spending to get it unstuck, and the "stimulus package" of 2009 was maybe a third of what we needed. (Enough to turn "hoovervilles" and the "bonus army" into "occupy wall street".)

Ordinarily when demand goes down below where we need it to be to avoid layoffs, the federal reserve will offer to loan money at even lower interest rates until people are willing to borrow again (sometimes just to refinance their existing debts and lower their monthly minimum payments, thus freeing up new money to spend each month on goods and services). Unfortunately if you get a big enough shock the interest rate you'd need to offer to get monthly spending back up to a rate capable of keeping everybody employed is BELOW 0%, and the federal reserve can't offer that. And because this knob normally works so well at controlling the amount of monthly spending people do, the feds no longer have a backup plan for what to do when the knob hits its end stop. (Asking them to pull out the techniques FDR used in the 1930's is like asking people to pull out candles during a blackout: how quaint, how uncivilized, we don't DO that anymore...)

Unfortunately the old fogies stuck "fighting the last war" are treating this as a supply-side crisis ala the OPEC oil embargo of the 1970's, so they're busy giving water to a drowning man and blocking any attempts to drain the swamp because they cannot CONCEIVE of a problem where customers simply don't buy products producers are selling. The problem MUST be at the producer end, It's not like customers have any CHOICE in the matter, they're just sheep behaving mathematically without volition, right? Only business owners are actually _people_...

Rich people looking for places to park their savings DESPERATELY want rates to go up because right now they're losing to inflation. They don't understand why rates are so low, they're convinced it's a conspiracy on the part of the federal reserve to punish rich people and prevent compound interest from making them richer. They've invented a "bond vigilante" fantasy whereby any day now rates will MAGICALLY go up, and suddenly their vast fortune will be earning, 3%, 4%, 5% above inflation instead of losing money to inflation every year. How will interest rates go up without any increase in people's ability to qualify for new loans or make additional monthly payments? Well, they just HAVE to. Because the alternative would be unthinkable!

Of course that's not how it works, but it turns out that people who made their money by modern white collar piracy ("leveraged buyouts") don't have to understand how the economy works any more than sports players who win via steroids (and retire at 30 with epic health problems) really understand anatomy, biochemistry, neurology... An olympic medal does not qualify you to perform surgery.

Speaking of inflation: it turns out the federal reserve could fake negative interest rates by raising the rate of inflation, because 5% inflation and 1% interest is essentially -4% interest. Go ahead and borrow money: by the time you have to pay it back it won't be worth as much anyway. In fact your existing debts get slowly eroded and less troublesome. But rich people HATE inflation eroding their existing fortune, and will fight to the death to stop this from happening.

P.S. a leveraged buyout is where you borrow a bunch of money to buy a profitable company, often using that company's assets as collateral. (Just like when you get a mortgage on a house, the house you're buying is the collateral.) Once you're in charge not only can you drain the company's bank accounts and pocket the money, but you can transfer your loans into the company's name so the debts you ran up buying the company are no longer your problem. If you haven't maxed out the company's credit rating yet you can have the company borrow MORE money (and pocket it). Next chip off any large assets (buildings, profitable division) and sell them, pocketing that money. Laying off employees can reduce expenses and allow the company to qualify for more loans. Rewrite the employee contracts so any retirement benefits are no longer based on savings and will be paid from future revenues, so you can pocket any existing pension fund. When there's nothing left to loot, sell the dessicated husk of the company and pick a new target.

This is how people like Mitt Romney made their money. Yes, it starts with the ability to borrow millions of dollars, which is easier to do if your daddy used to be the governor of Michigan. If this sounds utterly evil, you obviously don't understand the realities of business where corporations are people and employees are "resources, comma, human".

That said, stealing the Mona Lisa from the Louvre still doesn't make you Leonardo Da Vinci. Obtaining is not making. This is why the correct response to calling rich people "job creators" is to point and laugh.

December 20, 2012

December 19, 2012

December 9, 2012

Got in a long walk today for the first time since I hurt my foot: walked to Dragon's Lair and back. Got to see Randy and several other people (several of whom I was not prepared to meet but said hi anyway).

Stopped at The Donald's along the way, and got some programming in!

Ok, todo items, otherwise known as "procrastinating about the linux 3.7 update". Test current toybox in aboriginal, fix aboriginal's native-build.sh so it actually finds netcat after the busybox->toybox switch, check baseconfig-busybox for more commands toybox provides that I can switch off in busybox (looks like cut, rm, touch, hostname, and switch_root).

Try the lfs-bootstrap build: it breaks in m4 hanging on an rm prompt about deleting a ro file without -f, which is odd because the previous rm didn't and all the rm instances in configure look like they have -f. Try a chroot splice version so I can more easily track down where that's called from, and chroot-splice is saying that the read only bind mount is writeable. And an strace on mount shows it's passing through both the bind and ro flags and the result is still writeable. Is that a regression in Ubuntu 12.04's kernel... no, apparently the read only attribute can only be applied on a remount, not on the initial bind mount, which is CRAZY. (Easily fixed, but still crazy of the kernel to require.)

Ok, the delete is actually from build-one-package.sh in the control image bootstrap infrastructure, which is deleting config.guess and substituting a tiny "gcc -dumpmachine" instead (why config.guess doesn't do that itself...)

This call to rm doesn't have a -f on it, which is fine because it's a symlink (to a read-only bind mount, but the symlink itself is in a writeable directory). The problem seems to be that faccessat() is not honoring AT_SYMLINK_NOFOLLOW. In fact, strace says:

faccessat(AT_FDCWD, "build-aux/config.guess", W_OK) = -1 EROFS (Read-only file system)

And that's not even showing the fourth argument to the syscall. Is there a kernel version limitation? Let's see, cd to linux source, 'find . -name "*.c" | xargs grep sys_faccessat' and it's in fs/open.c and... I don't even have to do a git annotate, it only has 3 arguments. So either the man page is wrong or libc implements wrapper glue that uClibc is getting wrong.

Sigh. I can check the link status in the stat info dirtree is already giving me (symlinks are always chmod 777 in linux), the problem is what if the directory it's in is read only? faccessat() should tell me if the path to here doesn't let me fiddle with it. Then again, if that is the case I can't delete it anyway so prompting is kinda moot...

And ubuntu crashed, for the first time in a while. (It was X11: chrome tabs kept freezing, then _any_ chrome tab would freeze after a few seconds, then the whole of X froze so badly the mouse pointer wouldn't move and ctrl-alt-F1 wouldn't give me the text console. Remember, the system is SO MUCH MORE STABLE if your only way of interacting with it is through a userspace process.)

This means my 8 desktops full of open windows all went bye-bye. Probably a sign I was swap-thrashing, but it's back to my todo lists to try to figure out what I was working on...

Saying "Since the server breakin we've deployed SELINUX" roughly translates to "Since that boat sank we've sprayed WD-40 on everything". Not helping. Really, not helping.

(See also: thinking WD-40 will turn a boat into a submarine, arguing about the merits of WD-40 vs scotchguard for stopping a dripping faucet, allowing people who sell undercoating and extended warantees to design IT security "solutions"...)

Sometimes, posix is so nuts _nobody_ implements it properly.

The posix rm spec, section 2, says how to handle interactive prompts for recursive deletion of directories. Section 2(b) says to prompt before descending into a directory, and 2(d) says to prompt before deleting the now empty directory.

This is not what the gnu/dammit implementation of rm does:

$ mkdir -p temp/sub
$ rm -ri temp
rm: descend into directory `temp'? y
rm: remove directory `temp/sub'? y
rm: remove directory `temp'? y

It only prompts before descending into a non-empty directory. But the spec doesn't say anything about the directory being empty, it says you prompt for 2(b) and you prompt again for 2(d).

Also, section 4 is just awkward. The bits dealing with directories should be 2(e) (because you can't get there unless you made it through 2(a)), and the bits dealing with files should be 3(b).

Oh, and posix requires infinite directory traversal depth (even though filesystems have a finite number of inodes), and explicitly says you can't keep one filehandle open per directory level. This means that A) you have to traverse ".." to get out of the directory you're in, pretty much guaranteeing that two parallel rm -rf instances on the same tree cannot both complete, B) you have to consume an unbounded amount of memory cacheing the directory contents because you can't keep the filehandle open and restarting directory traversal with -i would re-query about files you already said "n" to at the prompt.

Somebody really didn't think this one through, but even _trying_ to make it compliant means I have to start over and write a lot of bespoke code that only applies to rm. Not sure it's worth it. (I'll probably break down and do it, but I'm going to sulk a bit first and call the standards guys names.)

Updated the dirtree infrastructure to feed parent to dirtree_add_node() so it can print the full path to errors. The _other_ thing I need to work out how to do is notify a parent node that one of the child nodes had an error.

I'm going to have to have multiple negative values in parent->data for a COMEAGAIN callback, aren't I?

Oh well, could be worse. The two uses for that field are dirfd for directories and length for symlinks. (Should be zero for normal files.) Symlink length should never be negative and the only negative fd is AT_FDCWD (which is -100, and that's hardwired into the linux ABI at this point).

No, that doesn't work because I can't just reach out and set parent->data at failure time because it's using that filehandle to iterate though a directory and there may be more valid entries after the failing one. So I'd have to defer setting it, which means I need another place to store it which means it should just _stay_ there. I'm going to have to allocate another variable in struct dirtree, aren't I? (I keep being tempted to overload fields in struct stat, but they're just not well defined enough.)

Actually getting the functionality right took a couple hours. Getting error handling/reporting right is coming up on day 3.

I wonder why "rm" cares about the files it's deleting being read-only? The "unlink" command doesn't. Oh well.

Yeah, finally working on The Most Dangerous Command, which I've held off on not because it's hard but because it's most likely to screw up my system if I get it wrong.

The yesno() stuff is wrong, it's checking stdin, stdout, and stderr (in order) to find a tty and using the first one it finds, meaning it's trying to write a prompt to stdin. I can special case my way past this, but in general working out when "yes y | thingy" should feed answers to yesno and when "zcat file | tar" should bypass stdin and bother the tty... I think the caller has to specify the behavior it wants. Gonna have to rewrite that function, but I probably need more than 2 users to work out the right semantics.

The ubuntu man page for rm has a --no-preserve-root option, which is a technical violation of posix but probably a good idea anyway. Except that longopts with no corresponding short opts are kinda tricky. (I _sort_ of have support for them, you can put parentheticals before all the other options and it'll parse them and set a flag. But there's no FLAG_ macro for those, and teaching the shell script to do that sounds painful. (It would wind up being something like FLAG_no_preserve_root anyway.)

I could trivially just ignore "/" (and have the error message be "rm /. if you mean it"), but posix doesn't require it and it conflicts with simplicity of implementation. If you log in as root and "cd /; rm -rf *" or "rm -rf /*" you're equally screwed without hitting the special case. Doing a realpath check for "/" might catch a couple more ".." than you expected, but how is taking out "/home" by accident much better?

Another fun little piece of special case behavior:

mkdir -p sub/other
touch sub/other/file
chmod 400 sub/other
rm -rf sub

This complains about being unable to delete "sub/other/file". It does _not_ complain about being unable to delete sub/other, or sub, so error reporting is suppressed as it works its way back up the three. (But only for parent directories, not siblings and aunts and such.)

My first pass at the code complained about being unable to open ".", "..", and "file", with no paths. Even though . and .. are discarded by the callback what it was complaining about is inability to stat and thus generate a dirtree node, because the directory has r but not x, meaning you can list the contents but not stat the files in it. (How is that useful, you say? No clue.) The dirtree infrastructure notes the stat failure and doesn't construct a node, thus can't call the callback. I can trivially filter out . and .. from error reporting, but giving a path to file means calling dirtree_path() which means having a node to call it on.

I think what I need to do is move the error handling to the callback, which means have it make a node with zeroed stat info. I can probably detect that by looking at st_nlink which should never be zero, but occasionally is. Sigh, inband signalling vs wasting space in every node. And either way I need to go fix the existing users in ls and chgrp and such... Not liking this. What's the alternative? Making generic infrastructure too clever us unpleasant, but having it display the full path to a file it couldn't access is probably the right thing...

Also, really hard to make a test suite that captures the error output in a way the test passes on multiple implementations producing slightly different error messages. Maybe I can just count the number of lines or wash stderr through grep or something...

The losetup command is more complicated now than when I wrote one for busybox, mostly because the kernel keeps growing new features. You can set the _length_ of an association now, not just the starting offset. It's got a "capacity check" thing that updates the loopback device to match a changed file size (which is _not_ the same plumbing as setting the length of the association because the people writing this aren't big into code reuse). You can iterate through all loop devices associated with a given file...

The fun one is /dev/loop-control meaning loop devices come and go now, so the -a, -f, -j, -d, and possibly even the basic association options are more complicated, in a way I'm not sure how to avoid race conditions for, but only after July of last year. (xubuntu 12.04 has this, but 10.04 didn't. And looking at the log of drivers/block/loop.c in the kernel there's a LOT of activity going on in what you'd think would be an ancient stable system. Partition support (tweaked August 2011) with LO_FLAGS_PARTSCAN (no support in the userspace losetup xubuntu's using). And of course the kernel parameter to set the number of pregenerated loop devices (potentially to zero, so you _must_ use loop-control to request new ones).

It's a bit like moving from the old static PTY devices to /dev/pts, except not quite cleanly separated.

One of the musl guys expressed interest in a big endian arm target for aboriginal, so I'm taking a stab at that. It's... tetchy.

Ok, the old arm4eb target built something but qemu couldn't boot it. The main problem here is it's oabi, which is obsolete. Nothing uses OABI anymore, and EABI requires thumb extensions so we need to bump up a notch in processor version to support this.

New target config, armv4teb, based on an unholy union of armv4tl and armv4eb. Diff the two and see what changes armv4tl needs. The gcc/binutils tuple is derived from the filename, so that should be ok. In the uClibc config there's an "ARCH_WANTS_BIG_ENDIAN", set that.

Next problem: the kernel doesn't want the versatile board to be big endian. There's some big endian plumbing support but the key is declaring an ARCH_SUPPORTS_BIG_ENDIAN symbol that currently only the ancient mach-ixp4xx declares, more or less what the armv4eb oabi stuff from before was aimed at. (Apparently you can declare the _existence_ of symbols inside conditional blocks testing on symbols which may be modified at runtime. Wheee.) This big endian plumbing has further derived symbols for armv6 and armv7 processors, but nothing sets it. Leaked infrastructure to support out of tree boards, looks like. The plumbing's there but no board definition uses it.

So, patch the kernel kconfig so the versatile board is even MORE versatile, and then set CONFIG_CPU_BIG_ENDIAN. Now try to build it...

And the sanity test at the end of simple-cross-compiler fails because libc.so is big endian but the compiler is trying to build a little endian hello world. And it's doing that because uClibc feeds a CFLAG to force big endian but the default in the compiler is still little endian. Why is the default wrong? Dig, dig... gcc's libbfd is testing the host compiler to set the target endianness. That's just SAD. Ok, in the target config "export ac_cv_c_bigendian=yes" and try again... and that test is coming out right but it made no difference to the smoke test.

Right, the armv4eb test bit-rotted in current releases but it worked in 1.1.1 so check that out and build a working big endian arm oabi toolchain to compare against... Right, got something that can run hello world under qemu-armeb. Now to look at its build logs.

This is gonna take a while.