rss feed 2009 2008 2007 2006 2005 2004 2002 livejournal twitter


February 7, 2010

So, the dropbear and strace builds are working for all the arms (except armv4eb), i586 and i686, mips and mipsel, and x86_64. That means they're not working for armv4eb, m68k, powerpc/ppc440, sh4, and sparc.

Proper (non-coldfire) support for m68k still isn't in qemu, and I'm not convinced sh4 is even a real platform anymore. That leaves arm big endian, powerpc, and sparc.

The powerpc problem is that the qemu emulator I'm using only supports 2 drives, not 3, so when I feed in an hdc image it gets ignored. (Hmmm, actually the device tree is saying there's a third drive, but it's the same as the second drive. Under "/proc/device-tree/aliases", ide0, ide1, and ide2 are:

/pci/pci-ata/ata-1/cdrom
/pci/mac-io/ata-3/disk
/pci/mac-io/ata-3/disk

Huh. It seems to think the odd one out is a CDROM hanging off of a PCI bus? Do I have all the right drivers enabled? I think so, but it's not binding...


February 4, 2010

The trick to getting Hulu working is to clear all cookies in the browser every time it starts complaining. (Doesn't fix things like dailyshow.com which simply _never_ work, but oh well.)

Now that The Daily Show is in HD, it takes up 1/4 of my laptop screen. Makes it a touch harder to watch it in the backround while programming...

I need to fix the download.sh infrastructure in FWL, and I'm not sure how. It's a design problem.

Right now, there is a single download.sh script. That lists everything that gets downloaded, and it all goes into a single packages directory. The reason for this is so it can delete stale files out of that directory, because it gets a complete list of every package that should be updated and when it's done it can delete any file it hasn't seen yet. So any file that _isn't_ listed gets deleted.

This is why it's fiddly to have multiple download scripts, because either any files not listed in the current script get deleted, or else it leaks stuff as development goes on and you have to clean out the directory by hand.

This comes in when sources/more/native-static-build.sh needs to build dropbear and strace, two packages not used by the main build. They have to be downloaded somehow, and re-using the existing download infrastructure makes sense. But right now, those packages are in download.sh along with the stuff from the main build, due to the deletion stuff.

What this really means is you can't have multiple download scripts downloading into the same directory. I can have each one populate its own directory, but there's a clutter question. If I put it in build it gets blanked and re-downloaded, that stuff is transient. Having lots of top level directories is clutter. Moving the current "packages" to "packages/base" or something is a bit disruptive to people using the current workflow...

I suppose I could add a packages/native and then just have the cleanup logic look at files only (not directories). Hmmm, and it currently does.

Cool. Ok, a second script populating packages/native. That works.

Second problem: MIRROR_LIST is in download.sh. Hmmm... I guess that goes in sources/include.sh.

Next: the MANIFEST file is more or less hand rolled. (The list of packages is hardwired into do_readme, it vaguely seems generic at its call site but just plain isn't.) That one, I'm going to completely ignore right now.

Huh. Ok, clearing cookies, killall npviewer npviewer.bin, and the "clear private data" thing apparently aren't always enough to snap hulu out of whatever it's "this video is not currently available, and we will give you no HINT of a reason why" funk. Which is sad.

I'm in the process of ripping the native build stage of buildall.sh a new one...


February 3, 2010

I got a FWL release out yesterday, details on the FWL news page. Yeah, regressions. Fixing 'em.

I have been invited to speak at Flourish 2010 in Chicago, March 19 and 20th. (I feel I should track down a conference called "Blotts" and submit a paper to them, on general principles.)

I'm also speaking at CELF ELC, April 12-14 in San Francisco.


February 2, 2010

Fade got me an energy drink for my birthday. I am now about 2/3 of the way done with putting out a release (which is test building). These two facts are related.

The test build discovered that armv6 broke. (The armv5 target works, but armv6 breaks in uClibc with "libc/string/ffs.c:27:2: error: #error ffs needs rewriting!")

You know, I'm not holding up the release, I'm just LISTING REGRESSIONS IN THE RELEASE NOTES and I'll fix it up for next time.

The Drafthouse was showing Groundhog's Day today, which I was seriously tempted to go to even though A) I bought the DVD a couple months ago, B) I've watched it ~3 times in the past month.

The consulting "feast or famine" thing continues strange. CELF proposals are coming in, I just bid on this one which I'm pretty sure I can do. Mark also put me in touch with a friend of his who has a line on a quick consulting trip to California, might last a month.

You know, if it wasn't for the job hunting I'd love the consulting lifestyle. If there was just some web page I could pick contracts off of like a menu... (I'd like two days of enhancing busybox, two days of tweaking a toolchain, on a couple weeks of board bringup, and for dessert a three month contract doing a new board support package.)

Alas, consulting is constant job hunting, hence the desire for a permanent job I can keep doing without any more job hunting.


February 1, 2010

Happy Birthday To Me. I am officially old.

For my birthday, I would like for either the ongoing flirtation with the Ireland job (the identity of which is not exactly a secret), or else for the interview I did last tuesday at IBM, to turn into real full-time stable employment.

What I _got_ is a gift certificate to Red Lobster from my grandparents (which was excellent, as always).

Oh, and Eric and Mark called this morning to let me know they'd been sent forms to fill out as references for me. In month 7 of this process, we've progressed from getting approval from committees by attending meetings, to having third parties fill out forms. Outstanding.

I suppose it's my fault that I stopped looking for other work when (I thought) they got serious at the end of October. As a consultant, if I'm not constantly looking for work I rapidly become underemployed, and for the past 3 months I've only taken ultra-short-term jobs that wouldn't interfere with my availability to move overseas on a week or two notice. (Actually I just handed off to Mark some of the work I _did_ line up, because I've just been too stressed out by months of "hurry up and wait" to give it the attention it deserves, and it makes a nice weekend project for him. My Firmware Linux project is a bit neglected too, although that requires less concentration from me than coming up to speed on something new, and poking listlessly at it is something I do to relieve stress.)

Meaning so far this month I've billed less than one full-time week's worth of work, and have more or less run through the "liquid cash" portion of my savings and will soon have to either decide what retirement assets to cash out or else Get a Real Job (tm). Despite that, I'm not currently sending out more resumes since the Ireland thing got itself unblocked again. (Yes, I'm a sap, but I really DO want to work there.) But if the IBM position comes through first I'm not turning down a bird in the hand to continue waiting for a job I applied for 7 months ago, no matter how nice that job would have been or how close it seems to be to finishing.

Speaking of close to finishing, it turns out I've been more stressed than I needed to be because I've apparently misunderstood the interview process all along. The deadline I was waiting for last month wasn't the yes/no on an actual hiring decision. (Where did I get that idea?) That was just whether or not I'd passed the "technical committee". Silly me, the next step after that was asking for references. (Because obviously you don't ask for references until two months _after_ you fly somebody in for an in-person interview.) I pointed them at the author of The Cathedral and the Bazaar, the manager of Ubuntu Mobile, the chair of Ohio LinuxFest, the other co-founder of Impact Linux, and the maintainer of uClibc++. All of whom I've worked with in a professional paid capacity, _and_ on open source projects or conferences.

Yeah yeah, almost there. Oncoming train at the end of the tunnel, better to light a flamethrower than to curse the darkness, time flies like an arrow and fruit flies like an apple, any fool can criticize and most do... But closure would have been a _marvelous_ birthday present.

Anyway, enough whining about that. I'm way overdue for an FWL release.

So I tracked down the sparc thing, and now I'm fiddling with the code to make the uClibc and linux config files show up in usr/src again. (They used to, that bit-rotted during the great refactoring.)

Almost ready to cut a release. Hopefully tomorrow.

By the way, last week's hospital trip was probably some kind of stomach flu. (Either that or I've managed to give myself food poisoning again twice since then. Yeah, that's helped with the whole "concentrating on technical projects" stuff too.) Feeling better today, at least.


January 31, 2010

I used to wonder where Mystery Science Theatre 3000 got all those truly horrible movies. After looking at the movies currently available to watch on hulu, I think they were going easy on us.

In fact, "The brain that wouldn't die" is one of Hulu's current offerings, and the hover-text for it _IS_ from the mst3k version (mentioning Tom Servo by name)... but the movie they have isn't. Right.

I boggle at The Suck. Four different "Benji" movies. The Captain America movie from 1990. They have movies called "Rabid Grannies", "Chicken Chronicles", "Love is a Gun", and "Violent Cop". They have the 1997 remake of McHale's Navy and the 1999 rename of The Mod Squad. They have Karate Kid III (but not I and II). They have both the 1962 and 1998 verions of "Carnival of Souls" (horror movie about an evil clown, apparently). "Mirror Wars: Reflection One" ("this film was a huge box office smash in Russia.") American Virgin (Starring Bob Hoskins, the gumshoe character from Who Framed Roger Rabbit) and My Five Wives (starring Rodney Dangerfield, with the cover photo showing his face covered with lipstick kisses)...

I was tempted to watch "Pure Luck" _again_, even though I watched it off Hulu a month or two ago. Watched the first few minutes of the Mel Brooks disaster "Life Stinks", but when the flash plugin crashed again (I switched window focus, that kills it about 20% of the time, yes Linux on the desktop continues to suck) there was absolutely no reason to restart it. Wandered off to Youtube to see what they've got...

Wow. Lots of the same stuff. (National Lampoon Presents Cattle Call, Bad Girls from Mars, Hercules in New York, Cheerleader Ninjas... They're _taunting_ me with the Adam Sandler vehicle "Going Overboard" which is not the Goldie Hawn/Kurt Russel "Overboard" which I would happily watch right now. Maybe I should break down and watch Shirley Temple's "The Little Princess".)

Of course Youtube's "shows" tab has some new movies hulu didn't, if you can call "Slave of the Cannibal God", "Torture Chamber of Dr. Sadism", or "Rescue from Gilligan's Island" new.

Maybe I should try getting netflix "watch instantly" to work with Ubuntu again... Oh, right.

I need to buy a mac.


January 30, 2010

So, back to poking at a release.

The sparc and m68k architectures aren't building. (They haven't gotten much testing because they don't work under qemu, but they used to at least _build_.)

Hmmm, sparc got broken by the uClibc upgrade. Which is a pain to bisect because so many of the intermediate versions don't build, and the patches you need to beat sense out of it change as you go along. Sigh.


January 26, 2010

So the official position of the maintainer of the SuperH port of Linux is that any toolchain older than October 16th (the binutils 2.20 release) is too old to build the Linux kernel. With the corollary that that in the 5 years since the PC went 64 bit, apparently nobody ever actually tried to build anything for SuperH.

I'm trying to figure out if I should bother to fix gas, revert the kernel commit that broke the build and call it good, or just drop sh4 support as "not a real platform".

The fun part is the bug they're freaking out about in gas only triggers when you feed it hand-coded assembly that asks the assembler to do something it can't do. The compiler produces assembly that doesn't hit this, so as far as I can tell you'll A) never see this from C code, B) have to choose to trigger this from hand coded assembly.


January 25, 2010

Poking at current -git for the linux kernel. Mips is broken. Chance to use bisectinate.sh.

Wanna hook up bisectinate.sh to the cron job. First question, should I hook it up to buildall.sh, or to cronjob.sh? The one that knows where the local copies of the various repositories are is cronjob.sh, but the one that's doing this kind of work is buildall.sh. Decisions, decisions...

There are a few design issues to automating this:

Meanwhile, the mips problem bisected down to:

commit 1b93b3c3e94be2605759735a89fc935ba5f58dcf
Author: Wu Zhangjin 
Date:   Wed Oct 14 18:12:16 2009 +0800

    MIPS: Add support for GZIP / BZIP2 / LZMA compressed kernel images

Which means it's probably "new config symbol showed up" and I need to run oldconfig. I have a script for that already, migrate_kernel.sh, but that's not hooked up to the automated builds either. So that would be yet another bullet point above.

(On a side note, migrate_kernel.sh vs migrate-kernel.sh is one of those chronic issues with me. Yeah, I'm mostly standardized on dash instead of underline in FWL, but some things like x86_64 are forced by other contexts, and I occasionally forget.)

And Marc Andre Turner reminded me the project's overdue for a release. All of this mess can probably be deferred until after that. (And I need to finish converting the snapshots directory to Vladimir's new pretty version, right now it's on the floor in pieces.


January 23, 2010

Went to the hospital shortly after midnight, with dizziness, sweating, and chest pains. As with the last time this happened, it wasn't a cardiac thing, but better safe than sorry when it comes to chest pains. (And hey, I actually had health insurance this time. Dunno if I still will once they see that I actually went to the hospital, but oh well.)

Last time it was a sinus infection causing swelling that pressed on a nerve. (Reeeeeeeally annoying.) It might be something like that again. (I note the cedar pollen season's starting up again, count's up around 300 or so already).

But it could also have been food poisoning, which it started presenting as _after_ I went to the hospital and sat in the little room hooked up to all the electrodes for an hour and had a chest x-ray. (This means mild food poisoning can be more annoying than full blown food poisoning, because it's harder to diagnose. Sigh.)

All this lasted until around 4am, so I'm on a night schedule again. May need an energy drink for my in-person job interview tuesday afternoon.


January 22, 2010

Went to Mr. Notebook and got a new power supply for my laptop. (They had to check five of them before they found one that works, apparently Dell power supplies are flaky in general.) This one's 90 watts instead of 65, so it charges the battery reasonably fast even when I'm using the sucker.

Heard back from a 1-year AIX contract I applied for. Phone interview today, in-person interview on Tuesday. (Haven't given up on the Ireland job, I'm just not waiting around for it either.)


January 21, 2010

Why I hate git, part gazillion:

I want to grab the description for a commit. I know the "git show COMMIT" command which appends a patch to the commit, but in this instance I don't want the patch. If I say "git log COMMIT" it doesn't show that specific commit, it shows a bunch of commits.

So I've already found two ways to get the data, but I have to chop out the subset of it I want with "sed". It's non-orthogonal crap that goes out of its way to be hard to use in a script.

I looked at the "git help show" (recycled man page), and it has the --pretty option, but none of those prevent it from appending a patch. I went through eight screens full of the "git help log" without finding anything useful there. From previous expeditions I know I can do "git log COMMIT^1..COMMIT", which is ugly but works.

I think my real problem with git is it's not a unix program. You have to go out of your way to beat _simple_ functionality out of it, by writing shell scripts to drive it and by screen-scraping the overcomplicated crap it spits out. Alas, that's what the kernel went with, and uClibc and busybox both did the mindless "me too" thing. Sigh.

There's probably some magic "do this" command line option...

And hey, may laptop's power supply died. This is the third one for this laptop. It still provides _power_ just fine, but Dell's patented "YOU'RE PIRATING POWER SUPPLIES!!!!11!1!" circuitry is refusing to identify itself, meaning it won't charge the battery anymore, and the thing has clocked down to 800 mhz.

I remember my toshiba. The power cord it used was the same power cord the radio used, and an electric shaver I had, and just about every other device out there. It did not have any circuitry in it.

Note that it refuses to even charge the battery when the laptop is _off_. If it's worried about potentially not being able to draw enough power from the thing, why won't it charge when the laptop is POWERED DOWN? Only answer: Dell's being greedy and hoping to charge extra for its power adapters. (And prevent you from using third party ones.)

I'll have to see if Mr. Notebook has anything in the morning. They closed at 6...

Of course Ubuntu's taking this as an opportunity to act _extra_ nuts. (It's convinced that one of the processors is at 1.7 mhz and the other is at 800 mhz. And the iwl3945 network card had firmware errors and aborted after thirty seconds _twice_, so I'm using my cell phone internet for the moment. If there was a way to just smack the Dell bios and go NO REALLY, IT'S A PERFECTLY USABLE AC ADAPTER YOU'VE BEEN USING FOR A YEAR NOW... Sigh.


January 20, 2010

So I wrote a git bisect script that rebuilds a given FWL target and runs the native build under it, and a bisect "bad" is when it doesn't successfully produce a dropbear binary from the native build. (This catches everything from patches failing to apply during the source tarball extract all the way through something subtle going wrong with the native compiler.) Hooking this up to the nightly cron job build should make that much easier to keep up with.

Right now I'm trying to figure out two things:


January 19, 2010

Video interview with the guy in Zurich today, for the Ireland job. (I was 15 minutes late, couldn't find the building. Sigh. As the song goes, "Que sadilla, sadilla. It's mexican food, you see. Tortillas and refried beans, all covered with cheese...")

Since the the six month anniversary of my application for the Ireland job was last week, I started sending out fresh resumes to positions that look interesting. (Mark pointed me at indeed.com, which seems to work a bit better than Dice. I still feel weird about applying for positions advertised on linkedin, since I don't use that for anything. I only have an account there people keep trying to link to me. (I have sent out exactly one link request, which I believe was ignored. And I felt dirty afterwards.)

One recruiter already got back to me about an AIX job. (Well, not the one advertised but a different one that's still within operational parameters, as it were. Working on a different Unix variant might make a nice change, and I'd rather be paid to work on honestly propreitary code than GPLv3.) Local position, easy commute, Fade could stay in ACC through her prerequisites for her doctorate and we wouldn't have to try to get this condo on the market before spring break this time around.

At this point, it's "whoever comes through first with something acceptable". I'm tired of waiting.


January 18, 2010

And uClibc 0.9.30.2 dropped! Yay, the project isn't entirely dead. (I prefer to see this as a Princess Bride "mostly dead" situation rather than a Monty Python "He says he's not dead" situation.)

It broke Mips, of course. Builds fine, but booting dies trying to launch init. The static hello world works. I should write the automatic git bisect stuff to track this down...

Catching up on Doctor Who episodes. Silence in the Library through Midnight. For some reason, this entire DVD seems to consist of "episodes that were running short so we needed to insert an extra 15 seconds of repetition, reaction shots, or random atmospheric stuff", over and over. Fade says it's intentional. *shrug*


January 17, 2010

So I'm reading The MPEG "System" layer standard (ISO 13818-1). The System layer is essentially an archive format that bundles together "elementary streams". An elementary stream contains audio, video, or timing information.

Elementary stream data is stored in "PES packets", and a Packetized Elementary Stream is a trivial system layer that contains a single elementary stream (with some minimal timestamp and rate data in the Elementary Stream Clock Reference (ESCR) and Elementary Stream Rate (ES_Rate) fields of the PES packet headers).

The two main MPEG System layer archive formats are Transport Streams (TS) and Program Streams (PS), both of which bundle together audio and video into a single file, and keep them synchronized. These also contain PES packets, but they can contain several different groups of them bundled together representing multiple elementary streams. (Or they can contain just one, you can use a TS or PS stream to store an MP3 if you want to.)

Transport Streams are more seekable. They're designed to resume from an arbitrary point in the stream without having seen what came before, which lets you recover from lost data by skipping the damaged bits, and also lets you change channels on a television and start playing at an arbitrary point without having seen the firt half of the program. The format is implemented as a series of small constant-size packets (generally 188 bytes), each of which starts with the same byte (0x47). It's used for HDTV and video camera output and so on.

Program Streams offer better compression that Transport Streams, but you need access to the whole file in order to interpret it. This is generally what you find on DVDs and such, and it's common in downloadable video because the files are smaller.

Other complicationos are that each Transport stream can contain multiple "programs", meaning not only are you grabbing and synchronizing a video stream with an audio stream out of the passing data, but there could be more than one to choose from. Program Streams are designed for just one program each, although you can have different audio tracks to choose from. And in both types of system streams you can have other elementary streams than just audio and video. (Subtitles, for example.) You can convert a program between TS and PS, it looks a bit like converting between tar and zip.

What I'm trying to do is teach VLC to chop transport streams into different files every X seconds. The hard part is actually digging through all the weird VLC layering. (Note: VLC violates its own layering left and right, this design is kind of ugly. Plus it calls extensively to external libraries so you have to dig through three or four different projects to try to follow the control flow.) In theory, specifying the output (display, write it to a file, send it to a network) is orthogonal to what type of data is being generated. In practice, this is not the case, except when it is.

Tempted to just write a dumb little daemon to accept the network output and have _that_ write to the files, parsing the stream just enough to do the chopping...


January 16, 2010

If you log into justin.tv's chat thing with your twitter account, your chat things are limited to 61 bytes. The reason for this is every chat comment you make gets posted to your twitter account with an enormous advertising tag. Yeah. Revoked its access, deleted the spurious posts, apologized for the noise.

Getting that login to work involved three attempts, and in the middle of that I had to delete cookies for that site because the site was just coming up as blank for me, not even any "view source" content. (But it worked fine via wget.)

I thought "hey, maybe that'll get The Daily Show displaying again", but alas: no. It still displays an ad, and then hangs waiting for 174.129.6.226. (Leave it there for half an hour, nothing will ever display.) I notice that all these websites that have problems with this (Hulu, blip.tv, etc) work just FINE to display the ads, and then hang before displaying their actual content. It's not displaying video that has a problem, it's confirming I'm not DOWNLOADING the video. Alas, this means that untangling the guts of what flash is doing and trying to download the video myself is the only way to view it. Kind of self-defeating, eh?)

Alas, it wasn't all these websites that changed. It was letting the ubuntu update manager apply its updates. It updated firefox, and now it doesn't work. And I don't have the option to downgrade it back down to the one that did work. And it's been weeks without a fix. (Last Daily Show episode I managed to watch was December 15th.) Linux: smell the usability.

Up most of the night playing Sims 3 on Fade's computer. My little ghost girl is stuck in the floating mode where they're on their knees hovering a little above the ground whenever they go anywhere, meaning they move VERY VERY SLOWLY. (Unless they get on a bike or something, in which case they move at the same speed as anyone else.) Tried several things to fix it, to no avail. Not sure how to Google for it because I dunno what keywords to use. Wound up sending her back to the netherworld, hopefully reviving her again will fix it. (If not, hit the darn sim with ambrosia, although since she was born a ghost this would just be "surrect" instead of resurrect. I think.)

Oh, and I was watching The Daily Show in another window, because it plays just _fine_ on Fade's Mac, but plays only a commercial and then hangs before displaying actual program on Ubuntu. Same website, same network connect: Linux's flash support got subtly broken by one of the Ubuntu upgrades. Again.

(Yes, I spent the night fiddling with a game I broke while taking advantage of an environment in which a different bug wasn't manifesting. This is my experience with computers, always.)


January 13, 2010

What is it with me needing to address schedule slippage with cake?

I haven't uploaded my blog up to the website recently because of a looming job interview stressing me out, and thus being all I really wanted to blog about. I didn't want to comment on it while it was in progress. (This also means my project repositories haven't been synced up to the website, since it's the same rsync script. This blog is a file I edit in vi, with an rss feed generated via python. So I've been writing stuff in it, I just haven't been sending it anywhere.)

But today is the sixth month anniversary of me applying for the job, which I went through three phone interviews for before being flown to a full day in-person interview, and after an entire month of repeatedly rescheduling the decision resulting from that... I now have a new interview with somebody else on tuesday, this one via teleconference. Wheee.

Since I can no longer use the word "progress" to describe what this process is in (not with a straight face, anyway), I might as well get on with my life and rsync the past week up to the server. (January 6 was the third deadline for a decision since the in-person interview. I was silly enough to think it _meant_ something and was holding off posting this until I had _something_to_say_, one way or the other. Ah, the naievete of youth...)

The cake comes in because Fade is going back to college for her doctorate (well, at the moment taking prerequisite courses for the program at the local community college), and this looming schedule uncertainty has now caused her to miss her class registration deadline, because she put off registering until she knew whether or not we were going to move soon. We still don't, but today was the last day, and she forgot and missed the deadline by about an hour. That led to this, which of course led to me baking a cake. (I mean, we had the mix and the pan was clean and everything. I did have to wash a bowl.)

The thing is, I sent two cakes to Erik Andersen to unblock uClibc releases. (I also once arranged for a delivery of Mangoes to Neil Gaiman's assistant Lorraine for reasons that require some context to explain, but that's another story.) Going by historical precedent, what I might need to do is send a cake to the guys in Ireland.

In any case, there comes a point where I have to get on with my life, or at least run the darn rsync script.


January 12, 2010

Flying back to Austin. Loving the new laptop battery. Recharging (even though there's over 2/3 of the battery left) in Vegas. Trying to figure out why the strace build is breaking in the current system images, but worked fine in the last release.

One fun trick you can do if you have files that differ in two contexts that make diffing them problematic (such as two qemu instances) is "head -n filename 80 | sha1sum" on both files, and then vary the number until you find the first changed line via binary search. In this case, it's that linux/netlink.h isn't being detected via ./configure. (Insert standard autoconf-is-horrible rant here.)

The error autoconf is dying on is:

include/linux/netlink.h:35: error: expected specifier-qualifier-list before 'sa_family_t'

The problem turns out to be the linux kernel changed from 2.6.31 to 2.6.32 and that included git 9c501935a3cd. In the one that was working, linux/netlink.h includes linux/socket.h which defines sa_family_t. In the one that isn't working, linux/socket.h is largely an empty shell that had the relevant chunk removed by #unifdef because Ben Hutchings thought it was only still used by libc5.

Sigh. Email the kernel guys to ask why, patch strace to fix the immediate problem... If I wasn't on a plane right now I'd check to see if there was a current strace development project to send patches upstream to. (Last I checked, it was moribund.)


January 10, 2010

Still in Idaho, visiting relatives.

Belated christmas present opening yesterday. I got a new laptop battery! Woot!

Banging on FWL a bit. Alpha has the same missing inhibit_libc guards that sh4 did, and when you fix those the uClibc build dies with:

libm/e_exp.c: In function '__ieee754_exp':
libm/e_exp.c:144: internal compiler error: in emit_move_insn, at expr.c:3275

Isn't that special?

Also, the m68k and sparc builds were both broken by my futimes patch, because the last ever release of uClibc didn't include the INLINE_SYSCALL stuff that the other architectures I was actually testing did. (Neither of those worked properly under qemu, but they both _built_. The uneven nature of uClibc's support for them caused a build break on those architectures.) My resulting patch to fix this is AMAZINGLY ugly, but A) I'm fixing something that got fixed in the -devel branch months ago, B) what's the point of pushing things upstream if there's never going to be another uClibc release?

Since uClibc's so obviously moribund, I'm vaguely pondering poking at Android's "bionic" project instead, and maybe even their "toolbox" project. The first is their libc (uClibc replacement) and the second is their coreutils (busybox replacement). They wrote their own because they decided they want all the userspace stuff to be BSD licensed, no GPL in userspace. Before GPLv3 came out I would have objected to this, but now it seems kind of reasonable. (If the FSF is really _that_crazy_, then you don't want to get any of it on you.)

Yeah, traditionally BSD-licensed projects fork themselves to death. And I'm not even talking about free/net/open/dragonfly, I'm talking about developers being hired to work on propreitary forks. When BSD first worked up enough momentum to become interesting Sun hired away developers like Bill Joy to work on a proprietary fork in 1982, over the next decade it recovered enough for BSDI to hire the next generation of developers away to work on a different proprietary fork a decade later, and once it recovered from _that_ Apple hired Jordan Hubbard and friends away to work on MacOS X.

p>But Android is a corporate owned project like Darwin, Star Office, Mysql, QT, and so on. (And like Mozilla was before Netscape/AOL lost interest, and the whole first few years of the Fedora project before people stopped pretending it was anything other than Red Hat Enterprise Rawhide.) Corporate owned projects seldom get enough external contributions to be in much danger of forking, and never will. Hobbyists just aren't all that interested in something they can't affect the direction of, so just about all the project's developers will continue to work for the company that owns the copyright on the code. (Maybe it's only 1/10th as active as a real open source project, but that still puts it ahead of dying efforts like uClibc.)

Unfortunately, neither of Android's sub-projects have a mailing list. Instead they have a "google group", which (as I've mentioned before) is not a substitute for a mailing list. When I go to that Google Group, it starts with "Read the FAQs", which links to "What is Android", which says:

Android is a software stack for mobile devices that includes an operating system, middleware and key applications. The Android SDK provides the tools and APIs necessary to begin developing applications on the Android platform using the Java programming language.

That in a nutshell is why I've never had any real interest in Android. Java stopped being interesting as anything other than Cobol's successor back around 1999. It lost "the language of web browsers" to javascript (and then that lost about half of its share to flash's activescript). The famous "212% growth" of Linux in 1998 was all the Java developers LEAVING JAVA, for a platform that had no real java support (and thanks to Sun screwing over blackdown didn't for the next 5 years). The vast majority of us are not going back.

Come on guys, _java_? It's what replaced Cobol during Y2K when all the old stuff from the 1960's was being forcibly rewritten: it was the flavor of the month now graven in stone to serve the pointy-haired on their steam-powered mainframes, and outside of that niche it has all the appeal of a Pat Boone concert.

Aside from that, every platform in the world "also does java". It's a programming language. Crippling your platform so it _can't_ run stuff written in a language like that would be the hard part. Saying "the way to write code on this platform is to do it in perl" (or in python, or in ruby, or any other specific language other than C, which all those others are written _in_) means "this is not a general purpose programming environment", and we _have_ on of those already, it's called the web browser and it uses a totally unrelated language called javascript (which used to be called iscript but was renamed during the height of Java's flash-in-the-pan celebrity).

The fact I'm hearing people talk about bionic and toolbox now means the embedded community is finally starting to pay attention to the ability to program these devices in _C_, but they don't make it easy finding out how.

Alright, bite the bullet, just skip their pathological web pages, ignore the lack of mailing list archive to browse, and google for "android sdk download", which gets me to here which lists a linux tgz, right click save as and... it's trying to download an html file? Darn it, does it have one of those stupid broken sourceforge mirroring setups?

Nope, worse: it's a license agreement you have to agree to before they let you download the SDK.

Sigh. This is _so_ not an open source development project. (The result may be open source, but the development project is considerably _less_ friendly to hobbyists than the "Microsoft Developer Network".)

And numerous sections in the license mean "this SDK ain't open source" either:

3.3 Except to the extent required by applicable third party licenses, you may not copy (except for backup purposes), modify, adapt, redistribute, decompile, reverse engineer, disassemble, or create derivative works of the SDK or any part of the SDK. Except to the extent required by applicable third party licenses, you may not load any part of the SDK onto a mobile handset or any other hardware device except a personal computer, combine any part of the SDK with other software, or distribute any software or device incorporating a part of the SDK.

See also 4.2, 4.3, 4.4... about half the license really.

The other half of the license is Android treating us like infants or perhaps merely morons:

4.5 You agree that you are solely responsible for (and that Google has no responsibility to you or to any third party for) any data, content, or resources that you create, transmit or display through the Android platform and/or applications for the Android platform, and for the consequences of your actions (including any loss or damage which Google may suffer) by doing so.

Guys? Could you inform your lawyers that even the GPL is _not_ based on contract law, and you went OUT OF YOUR WAY to select BSD licensing instead? You insist on X/MIT-style licensing for your userspace and then you pull the "you must be 18 years old to press this button but obviously we don't think you really are" crap. Having a click-through license AT ALL is unnecessary, but what you put IT it is just _stupid_. (Also, could you inform your lawyers that if you're not recording WHO agreed to the darn contract then there is no privity of contract and thus NO CONTRACT UNDER CONTRACT LAW? Last I checked you couldn't legally HAVE a contract if you DON'T KNOW WHO IT'S WITH.)

A quick search for git repositories for bionic finds here and here in the first screen of hits, and I don't have to agree to anything to download those. Did the people who did that violate the "contract" terms in putting the code up online?

Alas, I've lost interest in dealing with android again. So far I've seen more input from lawyers and marketers than from programmers on this thing, and apparently I'm not the only one. I've been paid to work on far worse, but why would I spend my _hobby_ time messing with this clueless morass of pointy hair, java, and confused legal boilerplate? I wonder if they actively don't want to be bothered by developers like me, or if it simply never occurred to them that hobbyists might have something to contribute? (Obviously the only worthwhile open source comes from places like IBM. Nobody does this in their spare time, or at least nobody does anything _interesting_. That's the vibe this project exudes with a flamethrower. I wonder if it's intentional?)


January 8, 2010

So I heard back from the Ireland job recruiter again this morning:

Hi Rob,

These discussions are still ongoing. There are differing thoughts on the
interview feedback and if/where we could position your profile.

I should have more early next week once these discussions have taken place.
Apologies for the continued delay but you're still in the mix...

I have no idea how to respond to that. (I have a profile in need of positioning? There's a mix?) Fade's sister Lisa works in hollywood, and I asked her if she could translate that for me, but it still boils down to the fact that hiring me is still an "if", and a decision has not been made.

I'm starting to desire closure more than anything else out of this process. I've lost count of how many times the actual decision part has been postponed, and "at least it's not a no" is starting to lose its appeal as a mantra.

Oh well. One more set of wednesdays...


January 7, 2010

Flying to Boise today, to visit Fade's parents (who moved there from California a few years ago) and siblings (who are converging on their parents' house).

Fade got a spare key made and I dropped it in the mail to Mark so he can feed the kiggies. (And we left them with five bowls of food and two bowls of water, so even if they don't get fed again before Tuesday they should be ok.)

The first leg of the flight was the most unpleasant experience I've had on Southwest in as long as I can remember. Every seat full: check. Crying toddler in his father's lap in the seat to my left: check. Guy who somehow managed to get on the wrong plane (thinking this one's going to El Paso) so we have to taxi back off the runway and drop him off at the terminal: check.

Still, it wound up way ahead of either of my recent experiences with Alaska Airlines. We made our flight and arrived in time to make our connecting flight (we would have just made it even if said connecting flight hadn't also been delayed an hour). The flight attendants gave me four snacks and three drink refills and attempted to placate the baby with cheez-its (which as far as I can tell were laced with horse tranquilizers), and when we arrived sang the plane a song apologizing for arriving late.

In Idaho now. Have yet to see any potatoes. Tired.

Still no word on the Ireland job.


January 6, 2010

Stressed. Today is apparently the day I learn whether or not I get a really cool job that would involve moving to Ireland.

It's been a six month interview process. I applied back in July, heard back in October, and got two phone interviews in november. That apparently convinced the recruiter I was talking to that he wanted to hire me, so then I had to convince the rest of the company. Thus they flew me to california for an in-person interview at the start of December. That was just over a month ago.

While I was there they said they review interview write-ups and make decisions about them on Wednesdays, but when I was there they said the results from a friday interview probably wouldn't be in by the following Wednesday (in this case the 9th), so they'd probably do the review on the 16th instead. On the 17th, I got email that "the feedback took a little longer than expected to collate so you have been put into next Wednesday's review", and it was bumped to the 23rd. Then on the 22nd I got a phone call that people on the review committee were on vacation, so they bumped it two weeks to get past the holidays. That would be today.

In the past few months I've been invited to speak at a conference in Chicago this spring, and couldn't give 'em a clear answer because I don't know where they'd be flying me in _from_. Fade's only registered for one spring class because she doesn't know if we'll be moving. The real estate market near UT is intensely seasonal, if I'm going to sell the condo this year I should really have it on the market by spring break...

I'm supposed to be working on a VLC modification for a guy in Florida, but I'm just too stressed to focus. (And securitybreach went down yesterday, not sure why. That doesn't help.)

I also haven't been able to focus on FWL, which is having the weirdest problems with libgcc_s.so, because on arm eabi '__aeabi_ldivmod' is magic and evil. The more I look at that code the worse idea it was, maybe I should just hardware libgcc.a and _eh.a all the time...

When the host compiler from Ubuntu is building hello world, it's calling the linker via (lots of extraneous gorp removed):

ld crt1.o crti.o crtbegin.o hello.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed crtend.o crtn.o

The thing is, -lgcc asks for libgcc, and there's only libgcc.a (no .so). That means that libgcc_s.so can only replace libgcc_eh.a, which seems kind of pointless. But if I take out that -lgcc, then I get the __aeabi_ldivmod thing on armv5l, which is not straightforward.

I dunno, maybe this is a case where a new gcc release actually simplified something rather than making it worse. (There's a first time for everything.) But the new release is still GPLv3, so I still won't touch it.


January 5, 2010

I'm poking at toybox again. Added a trivial app (setsid) just to break the logjam, and also checked in some pending stuff I've had in my tree for... possibly a year?

I'm not sure what this really means, other than busybox is no fun for me to work on in its current state. I should look into what would be involved in porting largeish busybox applications to toybox commands. Bound to be easier than getting the busybox developers interested in redoing their infrastructure...


January 4, 2010

Learning about various things VLC can do. It's actually quite a powerful package, with documentation Google just can't find. Google finds the standard useless wiki, and some howtos with big warnings about how obsolete they are and which have a mixture of quite informative bits immediately followed by long sections about a windows-only wizard they apparently never bothered to implement on Linux

But what Google _doesn't_ find is any information on the command line options.

It turns out, what you do is "vlc -H" (which is not the same as "vlc --help", or even "vlc --help --advanced"), and that shows you all the actual options you've got. Apparently not all versions of vlc have that compiled in, but when it does it's quite informative. Why it's not online anywhere, I dunno...

January 3, 2010

Still weird to type 2010.

So I submitted another twitter help request for the same old problem. (Essentially boiling down to yet more whinging about Horrible Retweets being an abomination, but for a specific aspect of it that can be considered an actual bug rather than merely an epic design failure.)

This week's way to submit a twitter help request is to go to here and fill out the form. As far as I can tell, that link is no longer linked from anywhere on twitter's site, I found it by guesswork. Then when you post the form, it will go directly to "solved" status. The difference between "solved" and "closed" is you can still comment on it, and commenting will open it and put it in the actual queue for a human to look at, so you have to view your solved requests and then click on your new request and add a comment.

This post is pretty much to remind me how to do it. I'll have to update it when they add yet more hoops.


January 2, 2010

Blundering around like a bull in a china shop in my own code base. "This is broken, that's broken, that needs to go over there..." I'm back in "I can fix it!" mode, but I also should break this down into chunks and check it in in smallish pieces. (Or at least check it in soonish before it snowballs much more.)

(Several of these things are todo items, such as making ccwrap automatically detect whether or not gcc_s.so is in the library path instead of hardwiring that in at compile time.)


January 1, 2010

BWAHAHAHAH!

It took two weeks of sitting around moping and having writers block, and either making changes I reverted at the end of the day because it just wasn't improving anything or else feeling like banging on FWL was just no fun at all (so I didn't do it)... And then today while randomly poking at a reduced subset of the problem I'd been wrestling with to try to get SOMETHING done, I saw how to FIX IT.

And suddenly, it's fun again and I'm getting so much CRAP cleaned out of root-filesystem.sh.

The trick is to have root-filesystem.sh _never_ build a native toolchain. It should not be doing that at all. That incestuous breeding tangle of "this calls that but it's also called from over there AND called from over there too" that I just couldn't simplify? I don't need to do it in the first place.

Instead, it should check to see if build/native-toolchain-$ARCH exists, and if it does copy it verbatim into the new root filesystem. I was already building one, so just use _that_. (If it doesn't exist, then use path_search to copy the shared libraries out of the cross compiler that's building us, and just use those.)

I repeat: root-filesystem.sh should _never_ build its own C library. It should just use the one out of an existing target compiler. It seems counter-intuitive that we don't rebuild everything, but we already depend on the existence of a working binary C library for the target in the compiler or nothing works. So run with that.

I'll probably grab some variant of my old ldd root filesystem generation script I wrote for busybox's test suite years ago (see testing/testsuite.sh in the busybox source, functions mkchroot() and dochroot(). I'd link to the busybox source control system, but it appears horked at the moment. Submitted a trouble ticket to osuosl.org). Anyway, the point of using that is to grab just the shared libraries that are actually used by something, rather than all the shared libraries the compiler has access to. But that's an optimization for later.

The new code looks something like this (still debugging):

# If a native compiler exists for this target, use it.

if [ -d "$BUILD/native-compiler-$ARCH" ]
then
  cp -a "$BUILD/native-compiler-$ARCH/." "$ROOT_TOPDIR" || dienow
else if [ ! -z "$BUILD_STATIC" ]
then
  mkdir -p "$ROOT_TOPDIR/lib" &&
  path_search "$("$ARCH-cc" --print-search-dirs | sed 's/^libraries: =*//')" \
              "*.so*" 'cp -h "$DIR/$FILE" "$ROOT_TOPDIR/lib/$FILE"' &&
  # Since we're not installing a compiler, delete the example source code.
  rm -rf "$ROOT_TOPDIR/src/*.c*" || dienow
fi

And that replaces LOTS of old existing crap, _and_ it decouples this build from the toolchain (look, no more library path redirection, although we're not entirely out of the woods on that front yet because when the native toolchain _does_ exist it's supplying the shared libraries for the final system and there's currently nothing actually enforcing a match between them, but eh. Worry about that later. Possibly the else goes away and the cp should only happen if the target file doesn't exist yet, something like that. The ldd script from busybox above works in here somehow.

But I am unblocked! I am removing complexity! Ok, some of it's migrated into build.sh and that's bad and I still need to keep chasing it but PROGRESS. There is still mess but it's FLEXIBLE mess, it moves when I push on it, so I'll mold it into something reasonable eventually.

Sigh. Happy.


December 31, 2009

Happy new year. I haven't done anything worth blogging about it a week. (I've twittered but that's a service _designed_ for having very little to say. Sort of the point of the thing, really.)

I have to move the blog file and start a new one tomorrow. (Actually what I have to move is the "notes.html" symlink to point to the new file. And possibly I should start making the suckers so the date is a link to the individual post...)

Or just use a real blogging service. Can't quite bring myself to make my old livejournal important again now that the russian mafia owns it. But the span tags I've been putting in the entries don't show up in the rss feed, and every time I ponder doing something about that it always brings up the fact there's no comments and no individual entry URLs (beyond the hash tags)...

I dunno, what blogging services out there these days are worth using?


December 28, 2009

Wow, am I an agnostic.

I just added somebody to my spam filter over an argument about a tecnical topic, and not even because I particularly disagree with him (the approach he takes works fine for his use case). It's that he was arguing that what he'd chosen to do was a One True Way, conflating together a half-dozen different choices that are in reality orthogonal, insisting his personal preferences were universal truths... and I just don't want to argue with people like that anymore. It's no fun. Possibly I used up my current stock of patience in last week's LWN thread, dunno.

Christmas came and went, New Year's approaches, it's the holiday week between the two and I've been sleeping for 11 hours a night. Not quite sure why, just tired all the time. (Fade points out the time of year. If I do wind up in Dublin I'm going to need big flourescent lights, right next to my desk. Yeah, that's still an if.)


December 25, 2009

Merry Christmas! Watched a festive holiday movie (Groundhog's Day), opened presents, cooked an enormous roast...

I have been _massively_ slacking off due to the holidays. Haven't made any progress on any computer related things since last week. (In part because my current FWL thingy is trying to cut a gordian knot of complexity that I'm writing an explanation of for the mailing list, but mostly because it's christmas and I just want to relax and enjoy it.)

Wrote up a status report on FWL, and then moved it to the mailing list instead.


December 23, 2009

Sigh. Back in the 90's I found a marvelous orange bread recipe on Ye Olde Internet, which was accidentally vegan. (The whole "that carrot has been murdered, I must go straight edge on you now" aspect was incidental, I liked that it was actually _good_.) I dug the recipe up out of a box again years later and it was still marvelous, but alas it seems long lost since sometime before the move to Pittsburgh.

What I really remember about it is it was trivial to make. The main ingredients were flour and orange juice, with a bit of salt and baking powder/soda. (I still can't quite keep those straight. Possibly both.) Oh, and a teaspoon or so of lemon juice, optional. There might have been a little sugar, although the orange juice had rather a lot already. I do remember that the only liquid ingredient was the juice. (No eggs, butter, oil, shortening, or anything like that.) Also, no yeast. (It was important not to stir it too much, you just fold the mixture over once or twice until it's wet and then _stop_stirring_.)

Unfortunately, attempting to google for this recipe keeps bringing up pages and pages of irrelevant crap. No grated peel, no nuts, no bananas, no pumpkin, no cranberry. No milk, no chunks of fruit.

By the time I've put in all those filters, I think I broke Google's brain. I'm now getting a video recipe for "Frog's Eye Salad", a page about "Catsup Cake Flour" (?), A PDF from a state.tx.us page that claims to be a tax publication but apparently contains a grocery list, a recipe for "Orange Roughy Roll-ups with Creamy Dill Sauce" (if that has any relation to fruit roll-ups, I'm going to pass, thanks), fondue recipes, "Holiday Baking with Whole Grains", and a page from the North Dakota Wheat Commission. That's all in the first two pages of results.

I suspect my best option is to just fiddle around and try to recreate this stuff by trial and error. Except I don't currently have any orange juice.

Wow. After I stopped directly replying to His Specialness over on lwn (because it just wasn't accomplishing anything), and then later stopped posting to the lwn.net thread at all (well, except for this) when Jonathan Corbet (editor of Linux Weekly News) asked us to stop. (Then I fell behind on email and twitter for a couple days because it's christmas and I was busy, so I'm just catching up now.)

But other people didn't stop posting, and I still get emailed the replies to replies because I checked the little boxes, and apparenty His Specialness has decided that anyone who disagrees with him must really be me in disguise.

I find that hilarious. When have I ever been reluctant to insult him to his face? The only reason I've stopped using His Specialness's name is I think he's a glory hound on the order of those white house party crashers, balloon boy's parents, and that toupee that used to govern Illionis. Getting his name repeated seems to be what he wants, with context in which it's repeated a very distant secondary consideration. I suspect the only reason he hasn't done one of those reality television shows where you eat a bug on camera is they wouldn't take him.

LWN doesn't seem to have user profiles, so out of morbid curiosity I spent the 30 seconds to type "site:lwn.net syspig" into google, which found a comment from 2006 on its first page of hits. His Specialness apparently did not bother to do this.

I repeat my comment from the last big post I made to the thread before the moratorium: "I'm tired of arguing with people who don't bother to do their homework". (And I note yesterday's squirrel analogy on the difference between clever and smart.)


December 22, 2009

Yesterday at the mall there were tons of banners for this clear.com thing, and today I clicked on a web ad for 'em, because I'm all for any alternative to Time Warner (which needs to die on general principles because they keep trying to retroactively sneak usage limits into their contracts), and these guys look like they've got reasonable plans with no usage caps... if you can find them.

Unfortunately, their web site was designed by somebody who thinks that clicking through five pages to answer a simple question somehow improves matters, and that all pages should be generated by javascript based on cookies so that when you finally DO dredge an interesting piece of info out of their navigational morass, you can never actually get a URL directly to that page so you can send said URL to someone else. (Oh, and two of the pages I tried to visit had firefox bring up a "redirect loop detected" page, that was nice.)

It's a very _pretty_ website. Lovely plumage.

They had a "live chat" service which was answered by a very nice lady who was very polite, gave prompt replies... and didn't actually have any new information to give me. (She was looking stuff up on the website for me, and apparently she couldn't find some of it either.)

Hopefully if I come back in a month their teething troubles will have worked through, their support people will be more experienced, and maybe they'll have burned their existing website to the ground and replaced it with something that doesn't spend all its effort on being clever and none of it on being smart. (I can hope. When squirrels hide nuts so ingeniously they can't find them again, they're being clever but not smart. It's less cute in a large corporation.)


December 21, 2009

Ok, the FROM_ARCH, FROM_HOST, CROSS_HOST, and CROSS_TARGET stuff is once again TOO CONFUSING. It works, and it was worked out laboriously via extensive trial and error, but it's too nasty and tangled for me to keep it straight in my head, and I wrote it.

Let's see...

if CROSS_HOST is blank, it's $(uname -m)-walrus-linux
if CROSS_TARGET is blank, it's $ARCH-unknown-lilnux
  if CROSS_TARGET is set by an arch dir (such as powerpc-440fp),
  then FROM_HOST=$CROSS_TARGET
if FROM_ARCH is blank, FROM_ARCH=$ARCH
if FROM_HOST is blank, FROM_HOST=$FROM_ARCH-thingy-linux

Ok, FROM_ARCH defaults equal to $ARCH. (Probably $ARCH should just be called $TARGET these days.) This is blanked for the binutils/gcc/ccwrap stages when we're building a simple cross compile. It's set to the host for static builds (currently hardwired to i686), and then set equal to the target arch for native builds.

That's overcomplicated because the toolchain build and root filesystem stuff is glued together. Setting it to the toolchain host for cross and native builds makes sense, but the "default equal to $ARCH, except when it's blank" bit is crazy. There should be some way to signal we're building a simple compiler, but overloading FROM_ARCH seems a bit silly.

FROM_HOST is derived from FROM_ARCH, and overriden by CROSS_TARGET under magic circumstances. That's a mess, and I believe it's a mess for binutils. Vaguely recall much pain last time I fiddled with this, and this is just where I left off because it was SUCH a mess.

CROSS_HOST exists solely to humor the binutils and gcc builds, because the people who wrote that are crazy. It's always `uname -m`-walrus-linux. It's proof positive that autoconf is useless because it' can't figure out what the current system it's running the build on is, and must be told, and if I can't figure out how to remove it I think I'm going to hardwire it into the call sites.

Sigh. Attempting to clean up the multi-variable mess while separating the toolchain build from root-filesystem.sh (and unify it with cross-compiler.sh) turned into a large mess. I need to do this in stages, but it's all tangled together. Where to start...


Back to 2009