Rob's Blog rss feed old livejournal twitter

2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2002

January 12, 2020

Jeff's excited about a circuit design GUI thing and poked me into getting it built from source, then tried to teach me how to use it. I objected (FIVE open cans or worms already!) and that led to a longish design discussion about ASIC strategy (I.E. why we need to open a sixth can of worms in parallel). We REALLY need to do youtube videos about this stuff...

Wearing the wrist brace Fade got me. It itches, but my mildly sprained left wrist keeps twinging and I think it's trying to turn into RSI with all the typing I do. (It's in the wrong _place_ to be carpal tunnel, but it has high hopes. I guess these days that would be high hopes.)

January 11, 2020

More shopping for the tiny new office. We got a humidifier, a handheld whiteboard, a shelf, and a _second_ router. (Jeff reimagined the first one with openwrt, so of course it's now a brick. He says it's because he usually usses ddwrt, which doesn't support any hardware we can find in stores. I convinced him we're already behind a NAT here, so our second layer of NAT isn't really providing significant security, and we expect to outgrow this office this year anyway.)

January 10, 2020

We have an office! It's a "hello office" which is a Japanese company that rents out office space on the model of Public Storage units, only instead of an 8x10 concrete space behind a garage door, roughly the same amount of space in a corridor of similar doors, each with a little nameplate, in a building with 4 floors of this stuff; the first floor is a Curves gym and the basement is the G-19 subway station. The room has 4 desks, industrial carpet, flourescent lighting, an electrical outlet, and an ethernet jack with gigabit download speed (haven't benched the upload speed yet).

Of course we wasted hours trying to associate with the wifi until we figured out it's a "bring your own router" situation. (The orientation pamphlet didn't say.)

Also met with a lawyer who started filing the Visa Paperwork to get me permission to stay longer than 90 days/year.

Four bottles of milk tea (and a maple milk tea at Excelsior in Akihabra) was probably a bit more caffeine than I should have in a day, but it's SO GOOD.

January 9, 2020

I'm terrible about dating blog entries in Tokyo because I keep my laptop set to tokyo time, and it currently insists it's 5 PM on the 8th when it's 8 am on the 9th here. This was less of an issue when my blog entries were uploaded months behind where I was writing it, and I was mostly writing down "what _did_ I do on tuesday?" (check email, check twitter, slack, line, freenode backscroll, git log...) But now that I at least _intend_ to upload them after I complete each one (ok, currently hung up on finishing the Jan 3 entry because I left myself another "writeup this topic by integrating these 11 ideas into a coherent synthesis" and that's _WORK_...)

Anyway, it's morning in Tokyo. Ronald Regan is still dead. The TV is playing Japanese children's programming, on the theory I'm more likely to pick up snippets of that than the news. (And it's less likely to mention the GOP's _current_ Bastion of Senility, who is lashing out at the staff of his nursing home demanding to know who stole his teeth because he forgot where he put them so obviously they were stolen. Except this involves Iran, which was totally pacified when he took office.)

A duck/robot is singing a (surprisingly upbeat) chiptune song about having a pixelated picture of a broken heart on its chest. And now it's playing drums in a band where a purple/pink squid is on electric guitar and... (I think that's a dog costume?) is on bass. And now the person in the dog costume and... I don't know what the yellow blob with the helmet is supposed to be... are doing some kind of sports commentary. And now it's switched from people in mascot costumes to crayon drawing animation.

I believe I've completed the curly bracket expansion, or at least it's passing all my tests. Somewhere along the way I screwed up an allocation tracking and something's getting freed when it shouldn't, so "echo hello" is now barfing half the time saying it can't exec a string of line noise. (Washed through variable expansion and then freed before use I expect.) But that shouldn't be too hard to track down, and I bypassed the bracket expansion (which is a wrapper function at the start of variable resolution so that's easy to do) and that didn't fix it, so it was something else I did recently and didn't immediately notice (because use-after-free sometimes works, I should switch on a page poisoner) so it doesn't stop me from checking this in to pending.

January 8, 2020

The flight across the international dateline ate a day, so as far as I'm concerned January 8 started at 4pm in the airport Tully's (a coffee shop chain here). I caught an airport shuttle at 2:30 am and flew for SO LONG it's now the afternoon of the following day.

I'm very tired.

January 7, 2020

In airport. The ironic part is I had to stop binging "That time I got reincarnated as a slime" because my flight to Japan was taking off. (Crunchycrunch does not let you download episodes, and does not work in Japan at all due to region locked licenses.)

Despite much sleep dep, I got some debugging done on the flight. There are so many corner cases in curly bracket expansions, and I'm not 100% certain I'm testing them all yet. I've tried to be systematic about it, but keep hitting new weird ones. Wound up checking in a commit that's like 2/3 debug printfs by weight, but it's progress. (It passes TWO of the tests before segfaulting on the third.)

January 6, 2020

Called the doctor up in minnesota about my blood pressure medication not being refilled after a week, and they won't give me any more until I get my kidneys tested. (Because 20 years of caffeine being a stronger diuretic than the stuff they gave me clearly had no effect, but_this_ might.) And they're ok with me being off the blood pressure meds until I get that done, even though I explained I fly to Japan tomorrow for a month, and have already been off the pills since 3 days after I requested the refill a week ago, so they've basically taken me off blood pressure medication for the forseeable future, and they seem to be ok with that. (Oh well, if the medical professional doesn't care I guess that's ok then...)

John Rogers (the guy behind Leverage) linked to an excellent thread about money laundering, but I seriously think the Boomers dying is the only way the strangehold will break: a group of people who _already_ have single digit life expectancy aren't gonna be dissuaded by their decisions dooming us all. (And yes, that's a good photo.)

Scheduled to fly to Japan first thing in the morning, shuttle picking me up at 2:30 am. Trying to get toysh to a good stopping point. Debugging the bracket logic. There's a debugging failure mode I can get into where there's so many dprintf(2, "blah %p\n", varname); that you can't easily follow the _code_ anymore, but I don't want to remove any because as soon as I've answered _this_ question I want the context for the next run...

January 5, 2020

Two days to Japan flight. Racing to get toybox usable. Still wrestling with brace expansion.

Ok, there's 8 kinds of string segments you can output in brace expansion:

// span from start to {
// span from { to }
// span from { to {
// span from } to }
// span from } to {
// span from , to {
// span from } to ,
// span from } to end

Keeping in mind that other expansions haven't happened yet, ala "A= ABC=123; echo $A{BC,''}" outputs "123", we're just outputting literal string segments to the next stage of expansion. Except start->{ and ,->{ and {->{ are basically the same thing, and the same for the trailing three. Hmmm... [3 hours of staring at code before I remember to blog about it]. As always, I go through dozens of culdesacs where I go "wait, but this implies this and that means..." until I come up with something simple looking by process of elimination. It's somewhere between a jigsaw puzzle and untangling a small gordian knot.

January 4, 2020

The bash man page says there are 7 types of expansion, but there are actually 8 if you add quote expansion. And the problem is the order of operations they give, "brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and pathname expansion" does not _include_ quote expansion, and it... sort of goes in multiple places?

$ echo {~,~root}/pwd
/home/landley/pwd /root/pwd
$ echo \{~,~root}/pwd
$ echo ""{~,~root}/pwd
~/pwd ~root/pwd

The quotes affect {brace,expansion} but aren't _consumed_ before brace expansion, which implies they have to be parsed more than once, which I'd _really_ like to avoid. (My sh.c is 1796 lines and some of that's fluff that goes away when I'm done. I'm trying to keep the whole mess under 3000 if I can, 3500 if necessary.) Ah, I see it says quotes are removed at the end, but have been PARSED multiple times earlier, which means multiple passes over the string which seems gratuitous? Ok, that's how _they_ did it. Can I do better?

Grrr, this logic kinda wants to be a recursive call and I'm trying hard to do it with loops. But braces nest so it would need a stack.

Sigh, I have to care about $IFS. I hate $IFS. It's SUCH a hack.

I'm treating process substitution as a redirection, meaning "abc<(echo)def" does not become "abc/dev/fd/63def" because WHAT IS THE POINT OF DOING THAT? (Ok, if you've mounted devtmpfs someplace strange then MAYBE you'd want a prefix, but it would still have a hardwired /dev/ under that. And lots of stuff uses /dev/null and friends without caring, not much point special casing that.)

So, parsing {brace,expansion}, the easy thing to do is parse all the way through to the end and then iterate through each segment allocating a new extent a copying the relevant contents into it. I was doing an array of offset, length, and index (for traversing), but said array kind of wants a hardwired length and there isn't inherently a limit on how many of these you can have in an argument. Plus not every { results in one of these extends, you need a { with matching } _and_ a comma in between. With no comma, you leave {contents} with the curly brackets in the result. Or if there's no closing curly brackets, "echo {A,B,C" prints {A,B,C as-is. So if there _is_ a limit, it can get hung up on "echo {}{}{}{}{}{}{}{}{}..." exhausting said limit. (Yeah, unlikely, but not the right thing to do.)

Nope, better approach: make a struct, assemble a linked list of that struct, and pop completed candidates off onto a second list as they're completed, or dispose of them if there was no comma before they ended. (Which can get silly: {{{{A,B},C}D},E}

January 3, 2020

Part of the reason I used to get so far behind on blog entries is I would leave myself notes like this:

New david graeber talk, seems related to Mark whatsisface talk, and the australia guy (rutger bregman?). previous link, stitch together argument

Which means NOTHING to anyone but me. The new talk makes a bunch of good points I should make notes about (which means rewatching it), I put together a playlist of Mark Blyth talks (which I'm sure I tweeted about, but it seems to fall in the gap between the backscroll twitter shows and the last archive I downloaded before they screwed up the format so it was no longer CSV of the actual useful data but instead horrible bloated json you need some sort of tool to parse anything out of). I also tweeted about Rutger Bregman who is an eloquent proponent of basic income who I used to follow on twitter.

And I forget quite _which_ argument I found so compelling at the time, because I haven't rewatched the talk yet. I've combined Blyth's and Graeber's observations into a story before, and I've done my own research to show that 70% of the population was subsistence farmers 200 years ago and now it seriously looks like less than 10% of the population does ANYTHING related to actual human survival. Seriously, that link has numbers: if 90% of the population stopped working entirely the main impact is we might be less entertained (although AO3 fanfic, podcasts, and whatever would replace youtube if capitalism stopped strangling it could probably take over from Disney pretty smoothly).

I've written here before about how Graeber's BS Jobs argument is understated because he focuses on the 1/3 of people who say their jobs are _entirely_ useless, not the people who have 2 hours of real work each 40 hour week, or the dozens of support workers (HR, payroll, cafeteria, janitors, building maintenance, tech support...) every investment banker is propped up by, filling all the floors of the skyscraper below the penthouse by doing useful work in _service_ of some kind of financial scam.

So I left myself work to do. Again. My todo list runneth over: the more work I do, the more work is left to do.

January 2, 2020

I should do a toybox release before flying to japan. This means I should do it on the 6th because I have to leave for the airport something like 3am on the 7th.

I added MAYFORK to pwd, which replies on $PWD being set for -L to work right, which means I'm teaching the toysh "cd" builtin to set pwd, which means I'm implementing -L and -P and so on, which turns out to be a bit of infrastructure with fallout.

Converting "help" to work as a builtin too. Running it with no arguments should list the "usage:" lines of all the builtins, which is more new plumbing. (For one thing, "determine if you're running as a builtin", which is a new high optflag to go along with NODASH.

I should do a FAQ entry writeup that you can't build or test the standalone "help" or "install" through "make", you have to "scripts/ help" and "scripts/ help". (Flat namespace, and those have other meanings.)

Why is "help -au" showing "true" and "test" at the start of the alphabetical list (out of order)? It can't be "that's where all the NOHELP commands go" because sed and false are in alphabetical order. (Sed has to handle its own --help so it can say "this is not gnu sed 9.0" for lying to autoconf when it asks stupid questions.) Yes, I'm aware that show_help() is inefficient when showing a list of help text in order, probably I should do something with a callback. (The help strings are static to lib/help.c and the code parsing them is all localized there, because then I can gzip the help text at some point. For now I'm going "help text display is not something that needs performance optimization"...)

Next problem with cd expansion: the code I wrote assumes tilde expansion is implemented. Hmmm, how crappy can tilde expansion be? What does bash do for 'echo ""~landley'? Answer: it's a literal ~landley, not a path. Ok then, it's a prefix. Ended by : and / as far as I can tell. (Because / is a path segment and : can't be in a /etc/passwd format username field. Presumably \n terminates too, but that's kind of already covered. :)

I implemented the first actual variable expansion type, which means I'm adding the first allocation to the delete list... and double free dump. Which is misleading because what _actually_ happened is I was creating a struct double_list and then traversing it as a (singly linked) arg_list, using "prev" as "data", and freeing the wrong pointer. (Because I have a dlist_add() function so it's easy to create a doubly linked list, but singly linked lists I do by hand and in this case it's 4 steps, borderline "eh that should have a function"...)

I should probably have struct double_list be a structure with _just_ the prev and next pointers, and then have it be the first member of other structures. That would avoid all the typecasting I'm doing whenever I use those functions. (It's easy to use C in an object oriented manner, I just don't bother half the time because unnecessary layering is a variant of infrastructure in search of a user.) But the switchover is an intrusive change and my tree is not clean right now...

Speaking of all this environment variable fiddling, using the getenv/setenv stuff I've already got is actually wrong here, because it searches through the list each time. So it would search through the list doing lots of strncmp to find if it's an external variable, then search _again_ to set it if it is, otherwise search through the local list. Given how hot path variable resolution is, I kinda want that optimized...

January 1, 2020

Twenty years of copyright breakage unblocks today. Yay!

Tickets to Japan are booked, redeye flight on the 7th. Trying to get toysh to a good stopping point...

Back to 2019