"GNU/Linux" is an oxymoron: Linux is GPLv2 only, GNU is GPLv3 or later, the two projects cannot share code. Either it's "mere aggregation", or it's a license violation. Pick one.

I've got an extremely active contributor to toybox (Georgi) who I basically can't keep up with. They've tested toybox in various build environments I can't currently reproduce (macosx, android bionic, musl) and sent various patches.

This discussion resulted in me yanking the "#define _GNU_DAMMIT" from lib/portability.h (because the FSF has no claim on anything I"m doing and if I can _break_ compatability with the Hurd while remaining standards compliant I will do so, the same way I sprinkle c++ keywords into my C99 code as variable names and such. I refuse to #define _ALL_HAIL_RECHARD_STALLMAN in code that not just ISN'T GNU, but is actively ANTI-GNU.)

I only had the macro in there as a temporary hack to get dprintf() (which I can live without) back beore I mothballed the code, and I replaced it with the feature test macros that SUSv4 actually recommends. (Note that SUSv4 doesn't _require_ feature test macros, this incredibly stupid idea only contaminated the standard for "strictly conforming" applications, whatever that means. It's like the "-pedantic" compiler option, I think.) Of course, this breaks stuff.

Let's back up: feature test macros. The glibc headers nominally require you to #define various things in order to get all the function prototypes and constants the headers actually define. Except they don't _actually_ require it: if you NEVER MENTION ANY FEATURE TEST MACROS in your program, then the headers detect this (with a couple of big #ifndef blocks in features.h) and it gives you _BSD_SOURCE, _SVID_SOURCE, _POSIX_SOURCE, and sets _POSIX_C_SOURCE to 200809L, all automatically.

I.E. most people will NEVER HAVE TO CARE ABOUT THIS. But if you _do_ define any feature test macros yourself, it switches all that stuff _off_ unless you do it manually. I.E. touch the knobs and you disable the autopilot and crash horribly. So in general, people don't do this.

There's one special feature test macro, _GNU_SOURCE, which is the Great Big Button That Switches On Everything. In features.h switching that on automatically #defines ten other feature test macros. Everybody who ever bothers with feature test macros just presses the big Go Away button. In fact things like the unshare(2) man page say "#define _GNU_SOURCE" before including sched.h, even though on an Ubuntu 10.04 LTS development system the actual header guard (in bits/sched.h) is __USE_MISC which gets #defined by _either_ _USE_BSD or _USE_SVID (both of which you get for free if you never mention any feature test macros in your source). I.E. the man page tells you to #define something you don't need to define, which would be the wrong symbol to #define _anyway_, just because it's the big Go Away button for feature test macros.

The unshare() function is part of Linux container support, implementing things like network namespaces. The kernel developers invented unshare() in response to a rejecting the OpenVZ patches: Linus and company designed a different interface for the kernel to do container support, and the namespaces bit is via a brand new system call.

The FSF has nothing to do with this, and is not even aware of it. The Hurd does support anything remotely like this.

Needing to say "_GNU_SOURCE" to get unshare() is _really_ _stupid_. The man page there was simply wrong, if you used unshare() without #defining anything, it worked fine in Ubuntu 10.04.

In current ubuntu, they "fixed" that, so you DO need to define _GNU_SOURCE in order to get unshare(). They took the bug in the man page, and changed glibc to match. Right, I can just #include directly and include the unshare() system call prototype directly in my code to work around the glibc regression.

So now we come back to MacOS X (which hasn't got _GNU_SOURCE), and toybox, which is explicitly opposed to the FSF (the project's license is BSD because GPLv3 was that bad)

And then there's musl, which ONLY IMPLEMENTED _GNU_SOURCE (not any of the other feature test macros like _POSIX_SOURCE which are actually _mentioned_ by SUSv4) because nobody ever tries to _use_ granular feature test macros: they just hit the big Go Away Button.

Both musl and uClibc require you to #define _GNU_SOURCE to get strpcpy. That link is to strpcpy in posix 2008: it ain't GNU and requiring _any_ #define to get it is a bug.

tl;dr: Feature test macros are a bad idea.

Well of course Ubuntu is displacing Red Hat. Red Hat abandoned the developer workstation in favor of focusing on the enterprise market, and developers are going to deploy on what they developed on.

Red Hat abandoned its original workstation distro after Red Hat 9 because it figured out how Sun made money. Government and Fortune 500 procurement contracts often cap a bidder's profit at a fraction of the cost of materials, so these bidders spec the most expensive materials they can. They'd rather make a $500 profit off a $5000/seat Solaris license than a $3 profit off a $29.95 retail boxed copy of Red Hat. The engineers would rather use Linux, but the sales force and management insisted on the more expensive component to pad the bottom line. (And of course convinced the clueless _customers_ that obviously Solaris must be superior, since it was so much more expensive. For more on large institutions "protecting" themselves in the foot with both barrels, see this excellent article by Joel Spolsky.)

When Red Hat figured this out (You mean if we raise the price they'll buy _more_ copies?) they came out with the ridiculously expensive "Red Hat Enterprise", everybody switched to the better technology and suddenly their "enterprise" division made ten times as much money as the rest of the company combined. This "tail wagging the dog" situation sucked all Red Hat's attention away from the developer workstation market, which they decided to unload on the open source community in the form of Fedora.

When Red Hat did this, they had a little over 50% of the Linux installed base: desktops, servers, everything. It's what Linus Torvalds himself ran. The default Linux was Red Hat. And then they abandoned the developer workstation.

Unfortunately for Red Hat, Fedora didn't work. They were sort of aiming at something like Debian (without the endless flamewars), and clearly hoped that the community would pitch in and do lots of work for free as unpaid Red Hat employees, without asking for any significant voice in controlling the project. But this is not how open source development works: developers push projects in the direction those developers want the project to go. People are most likely to get out and push when they're trying to steer.

The first year of Fedora development was well-summarized in a parody IRC log: the tension between Red Hat's employee engineers and the would-be volunteers eroded much of the initial momentum. In year five Red Hat vetoed the demand for an independent Fedora Foundation (and in doing so crushed most remaining third party interest). In explaining why no Fedora Foundation would be forthcoming, the main point was "Red Hat has veto power over decisions" and would not be giving it up:

Red Hat *must* maintain a certain amount of control over Fedora decisions, because Red Hat's business model *depends* upon Fedora. Red Hat contributes millions of dollars in staff and resources to the success of Fedora, and Red Hat also accepts all of the legal risk for Fedora. Therefore, Red Hat will sometimes need to make tough decisions about Fedora.

This is a bit like Mozilla back when AOL/Netscape couldn't let go of it, which Jamie Zawinski post-mortemed in his resignation letter. The whole thing's an excellent read, but let me quote a little bit:

The truth is that, by virtue of the fact that the contributors to the Mozilla project included about a hundred full-time Netscape developers, and about thirty part-time outsiders, the project still belonged wholly to Netscape -- because only those who write the code truly control the project.

A similar problem occurred with every open source project Sun ever tried (Java, OpenSolaris, OpenOffice... often exacerbated by license issues and copyright assignment). The company wanted to retain control, but attract volunteer contributors. They wanted obedient galley slaves, free to row in unison alongside their employees but not allowed to nudge the project off course. Open source doesn't work that way.

Red Hat's utter failure to allow outsiders any control over Fedora reduced Fedora to "Red Hat Enterprise Rawhide", I.E. a beta release of the next Red Hat Enterprise, nothing more. In fact, Fedora became _so_ uninteresting that the Centos project was launched to do an open tracking fork of more interesting Red Hat Enterprise.

(How uninteresting is Fedora? Wikipedia considers 70 Ubuntu-based distributions "notable", compared to 19 Fedora based and 11 Red Hat Enterprise based. Those 19 Fedora distros include Fuduntu (combining Ubuntu and Fedora), Yellow Dog Linux (a decade old PowerPC distribution from the days before Red Hat enterprise), sposored projects (K12LTSP is based on Fedora because Red Hat gave them money), corporate-based distributions such as Intel's defunct Moblin project (merged with Maemo to form Meego, and then superceded by either Yocto or Tizen depending on which order you rank the Linux Foundation's quest for relevance), and of course Red Hat Enterprise itself.)

Another failure mode illuminating Fedora's loss of market share was the Solaris x86 debacle. Sun Microsystems' management alienated their development community by repeated killing the x86 version of Solaris, because why would anyone deploy that on a server? But Sun workstations cost tens of thousands of dollars, so developers don't write software on them. Instead they bought cheap x86 workstations, installed Solaris x86, and then ported the results to Sparc hardware when it was done.

The loss of the workstation market shrank the Sun developer pool. The repeated attacks on Solaris x86 strangled it. But Red Hat voluntarily _abandoned_ the Linux developer workstation market.

Red Hat's enterprise distro is not appealing to developers, it's a deployment environment for large conservative instututions. When Ubuntu came along and sucked the developer workstation market away from Fedora (not even bothering to field a server version for its first few releases), Red Hat gradually became a less appealing deployment environment, because it's not what Linux developers write software for.

Red Hat is under fairly standard disruptive attack. Its brand equity among developers has eroded, it's "pointy hair linux" now. It's the distro management uses to run Cobol programs. Small projects deploy something else, and when they grow to be big projects they stay with that something else. Red Hat is still milking brand equity from a dozen years ago.

Downloaded yesterday's Rachel Maddow Show to see if it had stopped being "All Abortion All The Time" (with interludes of Bad Science About Nuclear Power).Alas, no. The first 20 minutes or so of the show: solid abortion coverage, then a guest she could interview about abortion. Sigh.

I'm pro choice, but somewhere between the 3-hour special on Dr. Tiller and the entire episode covered to the history of the Kansas anti-abortion movement, I got kind of tired of listening to Rachel go on about it, and stopped watching. Keith Olbermann's job was to be mad as hell and not take it anymore, and he was a lightning rod that let Rachel be calm and intellectual. Now she's trying to rile up her base, and it's exhausting to watch.

Luckily, vlc has a "playback->faster" option that makes the show chipmunk its way through topics, and her last interview of the show was on an interesting topic, which they danced around but didn't quite directly address:

Rich people get rich by cornering the market: not just selling stuff but preventing _other_ people from selling the same thing, thus driving down the price and taking away their volume. Even the relatively good guys like Warren Buffet talk about how great it is to have a "moat around a business".

The current republican "party of the rich" is all about cornering the market. All the New Jim Crow laws are about cornering the market on voting. Taking away rights from poor/gay/female/young people is about cornering the market on rights, so elderly white men are disproportionately represented. The republicans are trying to corner the market on power.

If you don't have the metaphor of "cornering the market", you can't understand the current Republican strategy. They're think they're playing a zero-sum game, where "I can't succeed unless I fence you out". I.E. "for me to win, you must lose."

Their attacks on contraception or freakout about gay marriage make no sense unless you understand this fundamental tenet of their worldview: anybody else doing well in any way is an existential threat to the mindset of this group of elderly white men. They must be surrounded by suffering to feel good about themselves. Only then are they better than everyone else.

This leads to horribly stupid policies. If a meteor was hurtling towards the planet republicans wouldn't try to _stop_ it, they'd try to build domes so the chosen few could survive. The Tories are going out of their way to dismantle an existing successful health care system because tearing other people down is part of their definition of progress. If the world is a zero sum game, all you have to do is attack your enemies and rewards for you must follow; simple math. The last man on earth must be a trillionaire.

Somebody asked me if replicant is relevant to my interests. It is not. I see it as about on par with Utoto or maybe gNewSense.

Let's start with pragmatism: ReactOS is doing an open clone of windows (essentially porting Wine to the bare metal). Ever heard of it? Know anybody who's used it?

I'm an ex-OS/2 developer: preinstalls matter. Google licensees are shipping something like 100 million android devices per quarter, and providing those guys with a native development environment is a huge opportunity. Replicant _might_ manage 10,000 installs over the project's entire lifetime, all aftermarket.

Cyanogenmod is upgrading Android to be more useful, and even they've got a tiny fraction of stock Android's market share. Replicant is offering _less_ than stock Android, removing working code they disagree with the license terms of. Nobody who just wants to USE their phone has any reason to do this.

Again, BusyBox predates Android. Toybox is not competing with BusyBox on Android: if Google has had over five years to start shipping android and hasn't done so. The shortest path to getting android users more command line functionality than Android's toolbox provides is to write new code compatible with Android's existing license policies, as issued by Google and adhered to by Android device manufacturers. This is not based on "how an ideal world should work". This is starting with existing reality and plotting a course through it.

(Yes, the toolbox/toybox name thing is confusing, but I'd like to point out I named my project before the first Android phone shipped. :)

The Linux Foundation is a really strange bureaucracy. They hold some random invitation-only conference, and then they send out spam emails inviting everyone to "request an invitation". No, really!

One of us is unclear on the concept of "invitation-only". Actually, I think one of us is unclear on the concept of "open source". (And I see having bronze silver and gold sponsors weren't enough, they needed platinum too, because that's the _important_ part.)

Sigh.

I got interviewed by h-online about Toybox.

My old /usr/bin vs /bin rant has been cleaned up slightly and published by Hacker Montly.

They integrated my corrections about the actual drive sizes: / was a fast but tiny 0.5 meg disk, and RK05 disk packs (on /usr and eventually /home) were very slow 2.5 meg external beasties (not 1.5 megs). So the "3 whole megabytes" line was right, but I had the mix wrong in my 2010 post. (Dennis Ritche's website has primary soruce material on this if you know where to look, I pointed 'em at a couple citations.)

I need to get back to working on my computer history book. I need to tackle my aboriginal linux todo list. I need to get toybox finished. I need to start qcc. I need to get hexagon linux booted on my old nexus one. I need to fix chroot in the kernel. I need to do the "hello world" kernel refactoring. I need to start another penguicon/linucon-style convention. I need to start a podcast to get rid of unused ideas. I need more hours in the day...

toybox 0.2.1 is out.

I screwed up the tarball slightly (didn't pregenerate generated/help.h) so it requires python on the host to build. I'll probably do a 0.2.2 next weekend to catch up with the patch backlog and fix that.

Still haven't _quite_ got a toybox release out, too much other stuff eating my time. Thinking this weekend...

Got the roadmap updated a bit though. That should replace most of the random todo list stuff, _and_ be the place listing currently supported commands each release. That's also the place to stick this sort of thing...

I feel bad about the contributors whos patches have gone umerged this week, though. Sorry, I'm catching up as fast as I can! (The roadmap has a list of "probably done" commands, and of course I'm re-auditing them all to make sure they _are_. Well a lot of 'em I haven't looked at in 3-4 years...)

Heh. Forbes asks "Is the cloud catching up with mightly Oratroll?" and never once mentions in-memory databases or the NoSQL movement.

My reaction to the article is "Couldn't happen to a nicer patent troll", but a quick check of my blog didn't find an explanation of what in-memory databases _are_, and wikipedia is useless here. Since that's what's really killing Oracle, here's a quick primer:

An in-memory database is what you get when all your tables fit in RAM at once. IBM's Search and Query Language (SQL) came from its mainframe R-series databases of the 1970's, which were designed around the assumption you only have enough memory to load one record from each table at a time. So it would load one record from disk out of each table, compare them together, and immediately write the results back to disk. All that "stream" and "join" stuff is based on the idea that you can't possibly have more than one record from each table in memory at once, and seeks are bad for performance!

This stopped being relevant about 20 years ago. Moore's Law has doubled memory sizes, and by about 2000 it was starting to become feasible for small databases to keep the entire thing in memory at once and index everything with hash tables, providing literally a 1000 times speedup over disk access.

In-memory databases are not just three orders of magnitude _faster_, they're also correspondingly _simpler_. Suddenly, your "tables" were just python dictionaries, and the entire database program became a thousand lines of python (800 of which implemented the SQL parser; I saw such a program on sourceforge in 2001).

The obvious implementation goes like this: start with a snapshot of your database (a gzipped Python "pickle" file of your hash tables, for example). Every "write" transaction (a query which updates any records anywhere), gets appended to a log file. (You can fsync() the log, it's linear streaming writes so should be pretty fast.) If the power fails, reload the snapshot and replay the log.

When the log gets uncomfortably long: fork the database. The parent process closes the old log file, opens a new one, and continues about its' business. The child starts writing a new snapshot, freeing its memory as it does so.

If the power fails before the child finishes writing the new snapshot, just read the old snapshot and replay both logs in sequence. When the child finishes writing, it can archive/delete the old snapshot and old log because the next shutdown replay doesn't need them.

The interesting bit of the above is that due to the way "fork" works, this doesn't take twice as much physical memory. The parent and child share copy-on-write mappings of all the underlying physical memory, and as long as the child is freeing its copies as fast as the parent is dirtying pages, the system shouldn't run out of memory. Making this take advantage of SMP is bog standard threading/locking.

This is why the department of defense spent millions of dollars buying a 2.4 terabyte ramdisk back in 2004. It was also a big driver for early adoption of 64-bit systems a year later: people wanted more than 4 gigs of memory to fit their databases in; they could afford the chips but needed more than 32 bits of address space to use it all. Now there's plenty.

The entire "nosql" movement boils down to "Hey, if 80% of our remaining database code is implementing SQL, why _bother_ with that? Why not just use the darn hash tables directly, via shared library or something?" SQL itself is an artifact of the R-Series stream-and-join design assumption, it doesn't fit well with randomly seekable hash tables that return results almost instantly.

If you're familiar with disruptive technologies, this is a textbook one. It started with databases too small to interest the existing players; if your database fit entirely in memory in 2000 it simply wasn't worth Oracle's time. Then medium-sized businesses could do it. Now all Oracle's got left are the largest fish like credit card processing companies and stock exchanges (plus the out-of-touch old fogies who Never Got Fired For Buying Oracle), and when those switch over (or in the latter case die off), what's left?

Oracle's problem is that the decades of "stream and join" design optimization is built on top of an obsolete design assumption, that memory is a transitional state your data passes through because memory is tiny and precious; records must be evicted back to disk as soon as possible. This is no longer the case, and the obsolete read/process/write loop at the hard of stream-and-join means Oracle has the best buggy whips in the world, decades of optimization into hiting the horses _just_right_... and nobody cares anymore.

(The stream-and-join metaphor also doesn't translate well to clusters, so very high end data warehousing really isn't their thing either. Like DEC, they're sandwiched between a relentlessly rising foe and a hard ceiling they can't chip through very fast.)

I've developed a theory that dying business models explode into a cloud of IP litigation, like drowning victims climbing on top of anyone who can swim, pushing them under to buy a few more seconds. SCO did it, now Oracle's doing it, the RIAA/MPAA are infamous for it... This is not a sign of health.

Finally got killall.c and kill.c cleaned up and merged.

Fun little corner case: kill lets you do "-s signal" or just "-signal" and lots of signal names begin with s! Thus "kill -stop 12345" gets interpreted as "kill -s top 12345" and it complains "top" is an unknown signal.

To fix that, I added a new option to lib/args.c where you can stick a space after a command letter and it requires its argument to be a separate command line argument, not the remains of the current one.

So now I've stuck a toybox snapshot into aboriginal and am building "TOYBOX=toybox CROSS_COMPILER_ARCH=i686 ./build.sh i686". I had to disable ls because it wants "-di" and "-tc" and who knows what else. And it's doing the xargs segfault thing again. Sigh.

Probably not going to get a release out tonight...

People keep asking me about Qualcomm's Hexagon in email, and I keep writing up long explanations and then never them again. So here's the most recent email I wrote on the topic, for posterity. It's been a year and a half, the basics of what we worked on shipped already, the story can come out now.

Keep in mind my contract to work on this stuff expired in October 2010, so these are my vague recollections from over a year ago:

On 02/24/2012 02:28 AM, [REDACTED] wrote:

> Thanks for info. I have already seen this page, but i have tried all
> branches on the 
> https://www.codeaurora.org/contribute/projects/hexagon-project/ and have
> not found any simulator.

Ah.

They had an in-house simulator that was some horrible proprietary thing they contracted out to a third party to produce, and if I recall right qualcomm's lawyers went out of their way to make sure they _didn't_ get source code because there was a nest of "propreitary is obviously better, duh". I expect that bit them in the ass, because the sucker was useless to the Linux port.

Mostly we just used real hardware. We had these things called "comet" boards that had a snapdragon SOC and ~256M of memory which you could boot and run code on. (No local storage to speak of, I wound up implementing nbd-client in busybox to get some.)

There was an aborted attempt to add hexagon support to qemu (Scott somebody did it, google for "quic qemu" and I think his post to the qemu mailing list is the first hit). Unfortunately, the guy who tried it couldn't wrap his head arount TCG (I think he was mostly a manager), and it never went anywhere. :(

Hexagon is a six stage pipeline running at 600mhz, and the clever thing they did was create six different register profiles and round-robin them down the pipeline, so each pipeline stage is totally independent of the others and they don't need any pipeline interlocks. So it _looks_ like a 6-way SMP chip, and they send NOPs down the pipeline when there's nothing to do (which power the circuitry down completely for that clock cycle). It doesn't do any branch prediction or speculative execution or anything becuase it's designed _not_to_waste_power_, instead each register profile is a separate thread (a bit like hyper-threading, only much simpler). It has ridiculous amounts of parallelism, in addition to having up to six thread profiles in flight at once, instructions are bundled into VLIW "packets", of up to 4 instructions dispatched to 4 execution units. Each execution unit has slightly different capabilities (the last two have the big vector and floating point ops doing the SIMD thing, the first one handles branching, I forget what the second does. They've all got a largeish general-purpose set of instructions they all do too. I had a giant booklet on this explaining the architecture and such.)

As far as Linux is concerned, it's a 6-way SMP chip running at 100mhz, but each cycle it can dispatch up to 4 instructions so you get back closer to 300mhz performance, and then there's the SIMD stuff which makes multimedia things fly. Also, most of the prefetch delays and such happen during the 6 "idle" stages between each batch of 4 instructions getting executed, so it's _really_ good at fairly hard realtime. I'm told that the next generation is going to have a 4-stage pipeline (and thus they're clocking it down to 500 mhz, so it looks like a 4-way SMP running 125 mhz). The lower the clock speed the better the power consumption to performance ratio is, and what they _already_ had was beating the then-current ARM stuff in battery life at a given performance level.

> When i use gdb from windows binaries i see this:
> 
>     /(hexagon-gdb) file DSP2.mbn/
> 
>     /Reading symbols from
>     D:\SHARED\qdsp6\Hexagon_Tools__4.0_windows\gnu\bin/DSP2.mbn...(no
>     debugging symbols found)...done./
> 
>     /(hexagon-gdb) run/
> 
>     /Starting program:
>     D:\SHARED\qdsp6\Hexagon_Tools__4.0_windows\gnu\bin/DSP2.mbn/
> 
>     /hexagonsim_exec_simulation: *Unable to execute hexagon-sim*!/
> 
>     /(hexagon-gdb)/

Sounds like they didn't hook it up all the way.

The snapdragon system-on-chip (which is what you find in the Nexus One and such) actually contains four processors:

1) An ARMv7 "Scorpion" processor qualcomm licensed from ARM and then optimized (at the Raleigh campus, they're protective of their turf, internal politics at qualcomm).

2) A QDSP6 "Scorpion" processor qualcomm developed internally (in Austin). In android this is used as a "multimedia coprocessor", but is actually a powerful 4-issue VLIW general purpose processor with a lot of vector instructions.

3) A QDSP4 (old DSP, does signal processing for the radio side of things). This was an ancestor of QDSP6 the way the 8080 was an ancestor of the Core 2 Duo: it ain't gonna run Linux.

4) An ancient ARMv5 that's the "boot processor". It runs code from flash at power-on, does the DRAM controller init, and then hands off control to either the Scorpion or the Hexagon. having a different chip take control is a question of running different boot code on this ARMv5. (Afterwards in Android it goes off and does signal processing for the radio side of things just like the QDSP4.)

Yes, "snapdragon" and "scorpion" are confusingly similar. (To me, anyway.) Snapdragon = SOC with 4 processors one of which is a Hexagon, Scorpion = yet another ARM implementation.

For the Linux port we powered the other 3 processors down after the Hexagon came up. I really really really wanted a bootloader I could run as an Android app on my Nexus One to boot Linux on the hexagon in that (copy the uboot and kernel+initramfs blobs into memory, kick the ARMv5 boot processor to run that uboot, halt the ARM), but I could never get Richard Kuo or anybody to write one. (And it wouldn't have been useful unless I got at least a usb-to-serial adapter working to get me a serial console, which turned out to be nontrivial. All the Snapdragon peripheral drivers were in the Android tree, but under arch/arm. One of the big things the Linutronix guys were looking at was fishing them out and moving them up to the generic architecture stuff so Hexagon could use 'em. No idea if they ever actually did this.)

Last I checked nobody had written any code to let the Scorpion and Hexagon share main memory: who is accessing what without stomping each other was a thing, so for the Linux port they just powered the Scorpion down completely. (The Android hexagon binary blob can share memory to act as a "multimedia coprocessor", but it does its best to _look_ like a hardware coprocessor, and essentially has dedicated memory buffers handed off the way you hand off texture memory to a 3D chip or sound data to a sound card. All static mappings, I expect. As I said: I didn't really look at that. You could try to objdump -d it if you like. :)

There was occasional talk among the engineers of doing a Snapdragon revision without a Scoprion in it, which would save power and licensing money and die size and so on. But we were careful not to say this where anybody outside the team could hear it because we didn't want the Raleigh guys to start quoting the "From Hell's Heart I Stab At Thee" speech from Star Trek II. (Note: my management mostly shielded me from the politics, so I got this stuff second hand. My exposure to the politics was basically a long list of things I wasn't supposed to talk about outside the department, which meant close your office door because you never know who might be listening...)

We had enough trouble with the lawyers, which were TERRIFIED that this open source stuff was going to undermine their precious patent revenue (the most lucrative of which expire this year anyway, I think). From an engineering perspective "not doing android" simply wasn't an option, and we convinced senior management of this, but the Lawyers insisted that the Qualcomm guys who worked on open source had to be moved to a separate corporate shell ("Qualcomm Innovation Center", I.E. quicinc.com: new email addresses but everything else was the same, they didn't even annotate the names on our office doors with the distinction), and then a SECOND corporate shell was set up (the "code aurora foundation") which was a partnership between Qualcomm and Qualcomm with some random lip service from Intel or somebody, so we had a second layer of indirection to wash everything through protecting Qualcomm's patents from Linux. It was _ridiculous_. Oh, and the lawyers Doomed to Exile in Quic were the junior guys who drew the short straw and couldn't _avoid_ getting the GPL all over them. (They tried not to panic in front of us. Again, this was over a year ago and I have no idea if this recollection is accurate or just me projecting onto them based on incomplete understanding of things that were happily Not My Problem.)

> I also have tried to compile these sources with Cygwin. This was very
> hard, because makefile for gdb has errors with looking of some
> libraries.

I did everything under Linux. We were trying to port Gentoo to it, but hit some unnecessary complexity adding a new architecture to their profiles (undocumented stage 1 black magic, and the need to annotate every single package in the tree with every single architecture it supports, which is _crazy_), and wound up just doing Linux From Scratch plus a bunch of Beyond Linux From Scratch packages.

I'm unaware of anybody ever trying this stuff under Cygwin. I was using Ubuntu, the server was running Debian, and I think a couple developers were using Fedora.

> I have manually downloaded and compiled these libraries, set
> direct path for libraries in make file, successfully make all
> *gnutools*, but have another problem again:
> 
>     Administrator@PC6 ~/Hexagon/bin/gdb
>     $ ls DSP2*
>     DSP2.MBN
...
>     (hexagon-gdb) target hexagon-sim
>     (hexagon-gdb) run
>     Starting program: /home/Administrator/Hexagon/bin/gdb/DSP2.MBN
>     bailing from child.
>     execvp: No such file or directory
>     Switching to remote protocol
>     :15097: Connection timed out.

I know nothing about cygwin. I don't do windows. :)

When the basic Hexagon suport got into Linux 3.1, did they set up a linux-hexagon mailing list? (I told them they really _should_, but I honestly haven't been keeping track since I left. Busy with other things...)

> Also i have seen https://www.codeaurora.org/patches/quic/hlk/ page, but
> there not exist even a gdb files.

Sigh.

Qualcomm outsourced the Linux upstreaming part to Thomas Gleixner's linutronix. (I may have strongly recommended them: pay one of Linus's lieutenants to do the first code review pass in private so you can fix up the code to kernel standards without embarassment or bikeshedding). Thomas is tied up in various NDAs, but now that most of this has gone upstream he might be able to help you more than I can. I'm a year out of date on all this, and left my qualcomm proprietary stuff at work when I left. (I had Aboriginal Linux building a hexagon native development environment, and then building Linux From Scratch and a chunk of Beyond Linux From Scratch natively on a comet board under that. It booted X11 (remotely, the X server was on another machine because comet had a network card but no graphics hardware) and ran a half-dozen X apps: a terminal, xchess, xeyes, and so on. Left it all behind when my contract expired, I no longer have that code. Oh well.)

Hexagon a _very_ interesting chip and I'd _hugely_ love to see it succeed, but as long as Qualcomm's lawyers are steering the company the engineering team has a giant ball and chain dragging it down. They do great stuff and you never hear about it, because "secret is good". Nobody talks about Qualcomm's secret shame, that they actually have a bunch of really smart guys working for them who do cool stuff. (I'm often frustrated at TI being closed and tangled, but it's a lot less trouble finding out their stuff _exists_...)

> I have found /*80-NB419-1_A_hexagon_v2_programmers_ref.pdf*/ document
> with hexagon command representation and looks like my ELF file
> (DSP2.mbn) has Hexagon-based architecture.

I do remember that "qdsp6-objdump -d" works in the toolchain they shipped. (Even the really old ones.)

Note: the qdsp6 name is because Qualcomm's 6th generation digital signal processor grew legs and became a general purpose CPU, _while_ still being good at all the DSP multimedia stuff. The hexagon rebranding is fairly recent (the name's due to the 6-stage pipeline and thus 6-way SMP, even though the newer ones are gonna be 4-way. :)

The big upgrades they did to the toolchain to support Linux were:

1) Add dynamic linking support,

2) Port uClibc and glibc to it,

3) Forward port from gcc 3.4 to something reasonably current.

If all you want is a binary blob to run on the chip, the old toolchain should be fine.

A note on the MMU: the chip hasn't actually got one. Instead it has a set of Translation Lookaside Buffer slots loaded by software. They made a binary blob that acts as an MMU, and their snapdragon port of u-boot (running in the armv5 boot processor, I think) loads this blob and hooks it up to the page fault interrupt so it acts as a software mmu (which their Linux port then depends on).

The lawyers have ADAMANTLY refused to release the code for that because they've got multiple patents in it that will _never_ stand up to scrutiny because software MMUs like this were commonplace back in the 1980's, but as long as you can't examine it you can't collapse the quantum state of the patents and thus they MIGHT be enforceable. (The 3D accelerator guys do something similar: everybody has patents that cover everybody else's chips, but as long as their drivers are closed enough you can't examine them and _show_ that they haven't got some clever way to program the chips that _doesn't_ violate your patents. The open source guys reverse engineering your chips and writing open drivers for them that violate a dozen patents is no problem because that doesn't show that the _closed_ drivers didn't have a workaround for that patent, and the patent-violating open soruce drivers are obviously programming the chip wrong. Plausible deniability! Also an insane waste of time so trolls can leech off the work of others to perpetuate concepts left over from Gutenberg's original hand-cranked printing press, as greedy rich people come up with yet another way to corner a market.)

Also note that the GPL doesn't bite unless you distribute binaries, meaning this code being GPLed translates to an awful lot of stuff that would be trivially easy to reverse engineer if they just shipped a binary never gets let out of the company because it requires an Enormous Legal Review Process to do so, and the political capital to push it through just ain't there. So "we got this to work, but you'll never see it, because it's GPL". (Yes my employment contract can say I can't release this binary outside the company. GPL is about copyright law, not contract law. The _license_ can't require it, but I can personally agree to an individual restriction before they give me the code. You want to argue that with Qualcomm's lawyers, be my guest. I'll be over here.)

The main problem with the Hexagon variant in the existing Snapdragon chips (um, QDSP6v2 I think) is that they don't have enough TLB slots. If you run the full 6-way SMP doing gcc compiles and such, it thrashes the hell out of the cache and slows itself way down. The performance "sweet spot" for that turned out to be around -j3 or -j4 in my testing. I don't think we found this out soon enough to fix QDSP6v3 (although since that's the 4-stage 500mhz variant, it puts less presure on the TLB anyway so is more or less in the sweet spot of what they _do_ have). But QDSP6v4 (in development when I left) adds lots more TLB slots which should greatly improve performance under Linux. Assuming anybody ever uses it under Linux.

The existing multimedia codec binary blob that Android runs uses a lot of static buffers with hugepage mappings, so it doesn't hit the TLB slot imitation. It's all hand-crafted assembly, really. (I assume so, I never looked at it.)

> I have succesfully decompiled
> a little peace of code first with a paper and pencil :), then have
> decompiled elf sections with objdump, but it's very hard to reverse
> entire algo without simulator.

"objdump -d" - disassemble.

> Please, help me, if you can. Maybe, you have any another tools,
> or useful advice about these tools.

I'd love to see Hexagon succeed. I spent half a year working to get Linux on it. The engineers in Austin also greatly want to see it succeed, but they're at the mercy of A) Qualcomm's legal department, B) politics with the Raleigh guys behind the Scorpion yet-another-ARM-chip, (those guys are very proud of their yet-another-ARM-chip. I forget why...), C) quarterly budget renewals. (My contract ended because they couldn't get the funding to renew it without a 2-month gap, so I went off and did other things.)

The reason the lawyers are in charge at Qualcomm is due to an accounting trick: all the patent licensing revenue is credited to the legal department, but all the R&D costs of coming up with that technology in the first place are billed to engineering. So even though engineering makes way more gross revenue than licensing, it looks on paper like licensing brings in 3x the net revenue that engineering does. And thus senior management listens to the lawyers three times as much as it listens to the engineers. Sad really.

Rob

Over the years I've wasted a lot of time talking to people whose minds are already made up, and are merely trying to figure out how to let you have their way. People who don't seem to believe there _is_ a legitimate alternate viewpoint, and are just trying to find a way to explain to us how we're wrong that we'll accept. People who are not there to _be_ convinced of anything.

I'm always trying to figure out "how am I wrong this time". Sometimes it's because I haven't explained myself properly, but often it's because I'm taking the wrong approach, have the wrong goals... I am wrong a lot.

This is why I expect I was wasting my time once more commenting on the recent lwn article where a lawyer reports on his meeting with a conference organizer, and thus the busybox gpl stuff is all resolved now. (I guess if we could get a pizza delivery guy to talk to a random janitor somewhere in rural Oklahoma they could bring about arab-israeli peace? People who ship BusyBox still get sued because of it, Android's still excluding GPL code from their userspace... What's changed, exactly?)

I admit commenting on "the story is over" with "why was this ever news" is sort of counterproductive anyway, but this whole thread was irritating because the original blog post that set this whole mess off was based on more than one false premise:

• Tim isn't behind busybox, I am. Implying Tim (or Sony) is some sort of puppetmaster is both untrue and insulting.

• Busybox is way older than Android, so if Android's 4th release still isn't shipping BusyBox, it's unlikely to start. If some combination of GPLv3, the busybox lawsuits, and the FSF's insanity has rendered GPL in userspace unpalatable to android (which is Google's _official_policy_ here), and waiting around for 5 years didn't make them change their minds, and they're shipping a BILLION devices: there just _might_ be a viable niche which a new project can legitimately address.

• I'm writing new code to obsolete my old code, and people who didn't contribute any code to _either_ project are freaking out about it. So what? Why is this news? It's pure FUD trying to bury the new project and hold out a little longer just in case 5 years isn't enough to confirm the Android guys are serious about not shipping busybox. (Even though there are already 3-4 other projects trying to fill this market vacuum.)

• How would it be bad if Sony (or whomever) _was_ involved? "If you don't like it, write your own code" is exactly what toybox is doing, and it's exactly what's pissing these people off. That's not "standing for freedom", that's insisting on control over the actions of others. These guys are hypocrites.

• Patent trolls and copyright trolls are roughly equivalent. Everybody just assumes the lawsuits were a good thing, nobody's questioning whether or not they might have been a net negative.

• How is "let's switch away from the obvious lawsuit factory" any different whether it's BusyBox or SCO or Oracle you're switching away from? Deciding to _stop_ doing business with somebody is not synonymous with violating their IP, even if "we have patents on other stuff, we'll still sue!" Great. If you want to sue people over the kernel, then do so. If the kernel guys feel that _not_ making "Linux" synonymous with "Lawsuit" it could be because they're _very_smart_.

That last bit I find really annoying. I was the guy who initiated the BusyBox lawsuits. Me, personally. I set the process in motion, I recruited the other people into the action, without me they would not have happened. (Erik was busybox maintainer for years, notice how the lawsuits started _after_ I took over?)

I no longer believe those lawsuits were a good idea, for a number of reasons. (One of which was that the SFLC assured me they had distanced themselves from the FSF... then got back in bed with them to sue Cisco in 2008.) There are excellent conversations to be had around that, but it's apparently not a conversation anybody (other than me) wanted to have.

Instead you get a bunch of hysterical armchair admirals with no copyrights worth enforcing (because they haven't written any interesting code anybody feels the need to use), screaming about how will the poor defenseless kernel survive without busybox to protect it. The kernel literally has a THOUSAND times more contributors than busybox (which hasn't even got one full-time employee working on it: nope, not even Denys). Not only are they far more _capable_ of suing people if they chose to, but Linus still hasn't put his foot down on binary only modules.

The big success in busybox releasing code was the the 2003 negotiations with Linksys that spawned OpenWRT, but that wasn't a lawsuit. GPLv2 gave them leverage, but it was all "walk softly and carry a big stick". Hitting lots of people with the stick broke the stick, and made people distance themselves from your stick. It was a _bad_thing_. (GPLv3 is a board with nails in: I'm not carrying that anywhere, nor going near it.)

Sigh. Ok, I know it's a public wiki, but dude.

So the entire first page of what was the toybox roadmap is now an advertisement for some BSD project, and you have to scroll down to see Toybox is even mentioned. They did no new requirements analysis other than strongly implying they're perfect for the job, despite BSD's ffs and berkeley packet filter being completely irrelevant to Android. They're "proven capable to replace busybox, in general", with 4 fewer commands than Toybox's last release. And now they're the only thing you see when you load that page unless you scroll down a lot, apparently attempting to leverage the work I did while simultaneously obscuring it.

I wonder if they know that gluing together a bunch of existing command implementations into a single binary is not where you _stop_? That the whole _point_ of the exercise is the extensive cleanup and refactoring to simplify and maximize code sharing?)

Sigh. It's an open wiki, obviously they have the right to promote their own project there, and it's not my place to remove stuff other people wrote. But it's also obviously unsuitable to use that page as the toybox roadmap. Let's see...

Needs more cleanup.

Working on release notes for _another_ toybox release, because the amount of progress since the last release is kind of impressive. (Very little of it done by me...)

The "inbox" on gmail accumulates a lot of bounce notifications for spam sent to Japan in my name for some reason, and every couple weeks I remember to go in and clear them out. (Email that's actually to my address gets moved to another folder, this is the bcc: stuff that's not to a recognized list, I.E. almost entirely spam.)Thunderbird's stupidity continues to amaze me. This is an imap folder (not a local folder filters copy stuff to), so when I go in and delete messages it A) freezes for seconds at a time, B) manages to do about two deletes at once even when I hit five (about as many as I can see on the screen at once with their amazingly horrible space-wasting layout and the font size I use).

But those aren't the weird bit. The weird bit is: C) download 2767 message headers, over and over and over. The number isn't even going down, it just sits there, downloading the same message headers, over and over. It takes about 10 seconds each time, and then it does it again. I assume taht since I queued up about 300 message deletions, it's going to do this pointless activity 300 times. At 10 seconds each, this is most of an hour of UTTERLY useless network thrashing.

That's a pretty good summary of thunderbird, really. Some weekend, I need to install a real mail program. (It's moving my email filters over I don't look forward to...)

Oh, it also loses track of folder locking. Even after it finishes with the network (such as me switching the network card _off_, perhaps by having suspended the laptop before its' hour of pointless thrashing was through), it insists something is still using that folder and therefore messages I read don't switch from "unread" to "read" and I can't delete any of them.

The fix for that is to kill thunderibrd and restart it. Really, the "and restart it" is negotiable, I need to install a real mail program. This one has SO many things wrong with it, you'd think nobody'd ever actually _used_ it before.

(Perhaps it's meant to only be used with fetchmail or something, and the imap functionality is vestigial? It certainly isn't _tested_...)

(Alas, the kill thunderbird and restart it also kills any half-finished replies you may have been composing, which I tend to ahve several of up at any given time. I keep forgetting this. Kmail would save them and pop them back open when it restarts. Thunderbird doesn't consider email worth reliable delivery or archiving; it treats it the way Twitter treats tweets. "A mere sideline to what we actually do, whatever that is.")

It's always hard to resist correcting people being wrong on the internet, especially when it's personal. (I point out that pressure put on Cisco in 2003 _without_ a lawsuit was far more effective than the lawsuit in 2008, and the reply is "You're wrong, they did sue, in 2008". No really. Apparently the 2008 lawsuit reached back in time several years to retroactively provide the basis for OpenWRT and such. Good to know.)

But I'm not feeding the trolls over there anymore. I've decided to shut up and show them the code. Have a toybox release.

If you wonder why I think the FSF zealots do more harm than good, the lwn.net threads on toybox are exhibit A. The loonier members there are literally taking the same position as SCO, that a program which replaces another program is automatically "infringing".

Whether it's just FUD or they actually believe it, the public statements are that you _cannot_ legally compete with any existing piece of software using a fresh implementation. "We like the old thing enough that the new thing must somehow be illegal." Under this theory, FireFox on Windows infringes Internet Explorer, and OpenOffice infringes Word, Cyanogenmod must be a "circumvention device" for phone company lockdowns, and so on.

These guys are "on the side" of Linux developers the way BMI and ASCAP are "on the side" of musicians. In the more mature content distribution industries like music and books and video, we have a wealth of evidence (link link link link link link (and link link) link link) that attempting to "protect" content on the internet tends to be hugely counterproductive.

Luckily: SCO lost. Not to the FSF: to _IBM_.

The point of copyleft was to turn copyright against itself. Getting comfortable with copyright enforcement suits to the point where you miss them when they're gone means YOU HAVE TURNED INTO WHAT YOU FIGHT. You're defining yourself by what you hate. That's _sad_.

Proprietary software didn't _exist_ prior to 1983, when the Apple vs Franklin decision extended copyright to apply to the binary ROM images Franklin's Apple II clone had copied verbatim. (Before that, copyright didn't apply to binaries, here's a 1980 audio interview with Bill Gates (transcript) about his efforts to lobby congress to change the law.) Once the law _did_ change, for-profit companies everywhere jumped on the new status quo, from IBM announcing from now on it would distribute Object Code Only (which was not well recieved), through AT&T closing up Unix, to the Xerox printer driver that drove Stallman to start the FSF.

The FSF was a conservative reactionary attempt to defend the status quo against changes in the industry, and wasn't even the only _unix_ based attempt to do so (BSD, Minix, Linux...) The GPL was an attempt to take the bull by the horns and steer. But that was 30 years ago: the disadvantages of proprietary software (abandonware, unfixable bugs, version skew, winner-take-all markets) became apparent to most players in the industry even _before_ the rise of the internet. In a lot of ways, proprietary software was an experiment that ran its course. Javascript isn't open because of the FSF, it's freely viewable because THAT'S WHAT WORKS. Do we really still need to be riding a bull?

When the DMCA passed, everybody thought it was a bad thing, now FSF advocates are trying to not just wield it but advocate aggressive interpretations of it. I'm sure if SOPA passed, a decade later these guys would do the same. That's going in the wrong direction.

The internet has fundamentally eroded the concept of copyright, as I wrote about a dozen years ago. Copyright arose with the printing press, which rendered irrelevant the church scribes' hand-written illuminated manuscripts. This broke their monopoly on literacy, and during the transition many people were put to _death_ for translating the bible into local dialects so people could read it themselves.

Now we've got the internet, which is the printing press all over again. If the printing press was as big a deal as electricity, the internet is room temperature superconductors powered by cold fusion. It changes everything _again_, and copyright _no_longer_makes_sense_ in the new context. The entrenched corporate interests are staging a new inquisition to stop the world from spinning, but all they can do is delay the inevitable.

Most people under the age of 50 take for granted that the RIAA and MPAA are ludicrously misguided dinosaurs struggling to slow their descent into the tarpits. But the people who want stronger GPL and more lawsuits, and think they NEED this in order to propser? They are on THE WRONG SIDE of that exact argument.

In moving from GPL to BSD, I'm asking: do we really _need_ to live in gated communites? Is the outside world really so terrifying we must fence ourselves off from it? GPLv3 is a concrete bunker full of canned goods because the previous gated community wasn't _secure_ enough, and I don't want to live there. Same way I put a creative commons tag at the top of this page (copy what you like, it's polite to attribute it).

Because dude: link link link link link link, link link... Why are we still arguing about this?

Debugging is frustrating. I'm trying to track down the xargs segfault that happens when I build aboriginal with toybox in host-tools (all the defconfig commands overriding the busybox commands), ala:

cd ~/aboriginal/aboriginal
hg clone http://landley.net/hg/toybox
rm -rf build
mkdir -p build/packages
ln -sf ../../toybox build/packages/toybox
TOYBOX=toybox ./host-tools.sh
more/record-commands.sh
CPUS=1 CROSS_COMPILER_HOST=i686 TOYBOX=toybox \
  ./build.sh i586 2>&1 | tee out.txt

When I do this, I get:

[8426193.532368] xargs[4542]: segfault at 0 ip 00007f4e3f21487c sp 00007fff4cb7a238 error 4 in libc-2.12.2.so[7f4e3f107000+15d000]

So in another terminal I ran:

while true
do
  sleep .2
  [ ! -z "$(dmesg | tail | egrep '(segfault|protection)')" ] &&
    killall make configure
done

And then looked at build/logs. This helped me track down the sort bug, but the xargs one _seems_ to be the xargs pipeline at the start of sources/sections/uClibc.build, and I when I extract that and ran it I _think_ I got a segfault once... but not on subsequent runs.

Heisenbugs! Always a pain...

Finally found where archived messages live in the Thunderbird UI. It's not in any of the menus (pull-down or pop-up) that's a red herring.

There are left pointing triangle and right pointing triangle buttons between "All Folders" and "Quick Filter" in the UI. (Under the "Tabs? Why would it have tabs?" level, which are under the "green arrow pointing into a drawer", "inverted carat", "paper with a pencil", "some kind of book maybe", and "dogtag with another carat" icons, which are under the pulldown text menus. (Cluttered UI much?)

If you hit the _left_ triangle you get a different set of folders, one of which is "archive". Which contains, in a flat view, every message ever downloaded by the system, including the spam. Plus _extra_ copies of messages that have been accidentally thrown in there. You can distinguish the extra copies because they actually have bodies when you select them, as opposed to an "oh no, there wouldn't be hot water _today_" message on all the others.

Thunderbird: they put an amazing amount of EFFORT into sucking this badly.

I miss kmail. Too bad it was glued to a desktop that became unusable, and got sucked up into a katamari with calendaring and rss feed reading software I didn't want to use....

I really should thank the guy who blogged about how upset toybox made him. I have contributors now! I'm actually having a hard time keeping up with all the code review I need to do...

If I had a book, I'd encourage him to publicly burn a copy. Closest I can think of is that I'm responsible for maybe half the material in "The Art of Unix Programming", if that's of interest...

I really need to find a better mail program than thunderbird, because its developers are crazy.

The random 4 minute hangs while it decides to go contact the network and refuse to even repaint are annoying enough (even though I told it to NEVER periodically fetch mail, it does so at random intervals anyway. Usually giving it focus reminds it to do so, and thus it

And of course if it's interrupted (such as switching the network _off_ so whatever it's doing HAS TO FAIL so I eventually get control back), it drops empty messages in all my folders with no title and no body.

Contributing to this delay is the fact that A) google's gmail imap server is slow, B) it's got some insane O(N^2) algorithm on folder size, probably doing a quicksort on already sorted content or some other equally "we don't understand how this actually works and are using it in a naieve manner" nonsense. Yes, I have 144,500 unread messages in linux-kernel. I hardly ever GO into that folder because I haven't got the time for this mail program to grind away trying to open it. But YOU SHOULD NOT RANDOMLY RESORT IT WHEN I'M NOT IN IT AND YOU'RE NOT FETCHING MAIL! (I can see the CPU spike and stay at 100% for minutes at a time. I've got a little bar graph that shows it to me. Stop it.)

There's a workaround for this: "file->offline->work offline". Then it only OCCASIONALLY randomly decides to talk to the network. Without that, thunderbird would be completely unusable.

But the thing that's _dangerous_ is the "archive message" option in the right click menu. If I right click on something, it will pop up a menu, usually after about a 3 second delay for thunderbird to be slow and bloated. UNLESS it's too far to the right on the screen, so that the cursor is over the menu when it finally gets around to processing the "release" part of the right click. Then it'll randomly select whatever menu item the cursor was over and execute it.

When it starts randomly popping up a reply window or something when I wanted "make this as unread", that's easy enough to undo. But "archive" means "hide this message from me so I can never find it again". Where do archived messages _go_? I've googled, but nobody seems to know the answer. The only way I've ever found them again was to remember some snippet of text from the message and do a find | xargs | grep on the thunderbird data directory, where I can find the raw text of the message and copy it out by hand.

I rant about this because it just ate another message, and I've given up trying to find it again..

Alas, did not get a toybox release out this weekend. Prepping for a release found too many fiddly little bugs I want to fix first.

Corporations are not people, corporations are machines.

A corporation is a machine the same way an aircraft carrier is a machine: many people must show up to work to operate the machine every day, and the machine can't do anything without them. An airplane and an airline are just different kinds of machines.

Treating a corporation as a person is no different than treating a car or a building as a person. "Don't blame me, it was the car that ran over that pedestrian. Don't blame me, it's the building's doors that were locked when it burned down with all those workers inside."

People drive the machine, and part of the machine is a uniform that people can put on to act on behalf of the machine. Some people who wear the uniform are "just following orders", and will do anything as part of their job.

But the people driving are the ones who rose to the top, often because they want to win at all costs and are willing say or do anything to get what they want. "Of course I'm not HIV positive, baby, come to bed." Some people will say anything to close the deal, to get someone's money, foreclose on their homes, strip-mine national parks, all while eating whale sushi with a side of bald eagle... because they can.

We wind up with crap like "Citizens United" not just because the people steering these machines find them more useful if they're treated as people, shielding the driver from responsibility for who the car hits. It's that the people arguing for this honestly see no difference between people and machines.

You don't have to be a sociopath to become rich, but it does eliminate the temptation to give your money away to others less comfortable than you are, so there's a bit of selection pressure involved at the billionaire level. And being a sociopath means you can hire lawyers, who are just following orders, to argue that machines and people are exactly the same.

Somebody blogged about how my Toybox project is obviously a plot by Sony, even though I've been doing it on and off since 2006, been publicly disgusted with GPLv3 just as long, and repeatedly blogged about how either Android or iPhone is going to replace the PC and I'd much rather it be Android. (For the record Sony hasn't paid me a dime for Toybox, although it would be nice if they did.)

I attempted to explain to him that he was simply _wrong_ in lots of comments on his blog, and on the lwn.net story on it (well, the first one, anyway), but by the time the h-online story came out about it, I decided to just shut up and show them the code. Working on cleanup for a release now.

I am sorry I lost my temper (again) when forcibly reminded that The Failure of Open Source still exists. I get annoyed when people who don't write open source software try to tell those of us who do how to go about it, and when it's all somebody does for a decade and change? Gets old.

Like me, apparently. Birthdays...

SUSv4 continues to be full of subtle assumptions and missing pieces. For example, with xargs the -L option works on "non-empty" lines but doesn't specify whether a line containing whitespace is "empty", or only zero length lines are. (In this case it mentions trailing whitespace indicates continuation, so I guess a line with only whitespace has to be empty due to the continuation rule. I presume trailing whitespace on the last line is not an error.)

Finally got xargs checked in to toybox. It's only got the basic -ns0 options (yes, -0 is a basic options, had to be designed into the tokenizing), but at this point filling out the test suite is the hard part.

What bit me in the test suite? I wanted to use "ls -w" as part of one of the tests, which turns out to be a gnu/dammit extension, and thus doesn't hold weight. Like so much the Free Software Foundation does, ls -w turns out to be utterly useless because they didn't think it through.

When ls's output is a tty it detects the screen width and wraps lines, as required by SUSv4. The -w option is described in the man page as "assume screen width instead of current value", so I should be able to test ls -w against xargs -s and check that the wrapping decisions match, right?

Here's the stupid part: -w only works if you have a tty. If you don't have a tty, the -1 option (list one file per line) is implied. So if you go "ls -w 80 > filename" it does one entry per line just as it would if you hadn't specified -w in the first place. I.E. the most obvious use for -w is exactly where it DOESN'T WORK.

The FSF still sucks at engineering, because it's not what they do. The FSF is somewhere between a religious organization without an invisible friend to venerate, and a lobbying group that never buys appointments with politicians. The common element of those is fundraising, and the Linux Foundation's got them beat there. (Both organizations sponsor a bit of engineering development, and presumably budget it as a marketing expense.)

I've been collapsing /bin and /usr/bin together since forever, and now it's a thing, so I linked to one of my old off-topic busybox posts about it in the LWN discussion comments, from where it got scooped up by Lennart Pottering, and from there became the top "story" on y-combinator, where I do not have an account so can't comment on the the fascinating discussion of my offhanded historical blathering.

I usually have to go to great lengths to track down and mirror computer history stuff, always nice when it comes to me. Although if somebody's going to say I "got many other details wrong" it'd be nice to list them. Good to know that Sun specifically was to blame for /opt, I need to research Sun's "project Lulu" which is going to be hard given Robert Young's Lulu occluding the search term. I remember the bit in "Under the Radar" about it, and the Larry... (Wall? Page? the bitkeeper guy) treatise it links to. I should re-read them.

The backstory is that I started symlinking /bin /sbin and /lib to their /usr counterparts in the yellowbox days back at WebOffice (2001 or thereabouts) because it made read-only vs read-write tracking easier. (All the read-only root filesystem stuff was under /usr, all the writeable stuff was under /var, and everything else at the top level was a symlink or mount point for a non-block backed filesystem.) Keep in mind I'm the guy who wrote up the first initramfs documentation: the first clear /bin vs /usr/bin explanation I got was in the tutorial workbook from Atlanta Linux Showcase 1999 (which I've still got), but clearly initrd obsoleted that split even before initramfs existed. (Linux 0.0.1 had kernel/blk_drv/ramdisk.c already.)

But I also did it because computer history is a hobby of mine and I learned backstory of _why_ /usr/bin and sbin and lib happened: Their first hard drive was only half a megabyte, they added a bigger but slower 2.5 megabyte RK05 disk pack on /usr for the home directories, but the root disk was so small it leaked into /usr, and when they got a third disk (another RK05 disk pack I think, I need to track down the references) they mounted it on /home and gave /usr over to the system. Keeping commonly used binaries in /bin and /sbin was because the first disk was _faster_.

All of this was an implementation detail of their original PDP-11 system circa 1972. It made perfect sense for Ken and Dennis to do, it _never_ applied to Linux running on PC hardware in any way. When I got taught it at ALS in 1999 I went "huh", because I am weird. (This is also why I'm so slow learning new tools: I question assumptions down to the bedrock on a regular basis, and am not COMFORTABLE unless I understand WHY we're doing stuff. Sometimes it's good, sometimes it's incredibly inconvenient. Oh well.)

Mirell's now informed me that I need to actually _write_ the computer history book I've been meaning to do forever, rather than just blogging about it. I have boxes of unscanned magazines, books to read, so many links to collate... I need to dig up the old proposed topic index I wrote up over 10 years ago. (Great thing about computer history: your old todo lists don't go stale. It's still history, now even more so.)

Why Linux on the Desktop will never happen, part eight thousand, four hundred, and seventy two:.

Finally rebooted my netbook today (instead of just suspend/resume) because too many things had stopped working. The network fails in low memory situations with a panic in dmesg saying it can't allocate memory. (Why the network card is trying to _allocate_ memory for each packet instead of having some static buffers is one of those unanswerable questions.) This fix is "sudo insmod -r iwlagn && sleep 1 && sudo insmod iwlagn" repeated several times because for the first couple dmesg says a watchdog timer dies after 4 seconds of trying to bring the card up.

The sound died too, but that's something like 15 separate modules with a dependency hierarchy I've never managed to work out, so I can't just rmmod and insmod that. So I did without sound for a while... until the network rmmod/insmod trick stopped working last night. Now it was giving me the timeout message _twice_ on each cycle, something was horked, so... reboot.

On reboot, it prompts me to log in, and won't let me. I note that I selected "shutdown" from the menu rather than just holding the button down; I was _nice_ to the system, which was probably my mistake.

So I ctrl-alt-F1 over to a text console, log in there and dig up .xsession-errors, and it says:

/etc/gdm/Xsession: Beginning session setup...
Setting IM through im-switch for locale=en_US.
Start IM through /etc/X11/xinit/xinput.d/all_ALL linked to /etc/X11/xinit/xinput.d/default.
/usr/bin/startxfce4: X server already running on display :0
<stdin>:1:3: error: invalid preprocessing directive #Those
<stdin>:2:3: error: invalid preprocessing directive #or
<stdin>:3:3: error: invalid preprocessing directive #Xft
<stdin>:4:3: error: invalid preprocessing directive #Xft
xrdb:  "Xft.hinting" on line 13 overrides entry on line 6
xrdb:  "Xft.hintstyle" on line 14 overrides entry on line 7
xfce4-session: Unable to access file /home/landley/.ICEauthority: Permission denied
XIO:  fatal IO error 104 (Connection reset by peer) on X server ":0.0"
      after 38 requests (37 known processed) with 0 events remaining.

Obviously the solution is "sudo mv .ICEauthority .ICEauthority.bak", we can totally expect any random end-user to know how to do that. Just as they'd know about ctrl-alt-F1 (which the kernel developers keep threatening to remove).

I note that even I have no idea what .ICEauthority is _for_. It's bound to be more of that unnecessary selinux/dbus/hal style crap Linux has been growing. Lennart Pottering recently published part 12 of his "systemd for administrators" series on LWN. Anything that requires a 12 part series to explain should not be on my system.

If you wonder why I'm not particularly worried about Android obsoleting vanilla Linux: from a usability perspective they honestly can't make it much worse.

The Obama administration wasn't content to take away Habeas Corups but recently removed the fifth amendment as well, which combined with the "oh yeah torture was fine" stuff means this is no longer funny.

This is why leaving war crimes unprosecuted was a bad idea: he didn't cauterize the wound, and so the infection continues to spread...

Banging on the long-delayed Aboriginal Linux release again, which is in part blocked by the fact that mips' network connection went away, so the emulated mips system thinks it can use distcc but fails when it tries (and then overloads because it hasn't got enough memory for a fully local -j 3 build without the OOM killer going off).The problem actually isn't the kernel upgrade, the problem turns out to be the QEMU upgrade. It worked in qemu 0.15.1, and didn't work in 1.0, and git bisect tracked that down to commit 5632ae46: "mips_malta: move i8259 initialization after piix4 initialization".

Not quite sure how to deal with that: "use qemu 1.0 for x86 because the emulator command name changed, but use 0.15.1 for mips because there's a blocking bug". Hmmm...

January 18, 2012

January 17, 2012

January 13, 2012

Resisted updating my notes.html symlink to point to the notes-2012.html file for a week and change because I'm writing an updated version of the python rss generator that'll also split the big file into individual files with prev/next links and an index. (To make linking to individual entries a lot easier.)

Alas, appreaching 2 weeks into the new year and not having finished it... time to set the symlink anyway.

I also need to get an aboriginal linux release out. And update the toybox todo. Maybe this weekend.

Day jobs. They are time consuming.

January 11, 2012