I find myself typing the most absurd search strings (they read like lexical Tourette’s or XKCD passwords):
pseudocluster ubuntu precise pangolin cloudera cdh3
I spent a while getting my new laptop set up with a Cloudera CDH3 Hadoop pseudo-distributed cluster. But I really wish I’d had the following instructions, simplified off the web with some help from some of my friends.
I hope these are helpful to someone besides me.
To quote Wikipedia: an anti-pattern is:
a pattern used in social or business operations or software engineering that may be commonly used but is ineffective and/or counterproductive in practice. [emphasis mine]
I’ve been exploring patterns for actually working on software — not for designing it — and I realized that I myself spent a lot of time living inside one particular pattern, which we might call the Grad School collaboration anti-pattern.
Grad school — especially the process of writing a PhD — values three things, no matter your department or specialty:
- novelty – what you create must be different from what everybody else until now has done
- individual effort – what you create must be your own work, not something produced by a team
- completion over sustainability – sometimes called “PhinisheD”, or “the point of a PhD is to finish a PhD”.
Each of these targets is critical to the idea that a PhD is a work of heroic individual effort to expand the boundaries of science. This idea is a fiction, and — like so many useful fictions — is a useful fiction, though it’s rarely true in practice (my PhD, for example, was a product of my labmates and fellowship [ETA: and of course my own effort!]).
But each of these is actually a collaboration anti-pattern of its own:
- novelty can spiral off into Not Invented Here — and frequently does
- individual effort fosters “Colleague, pronounced as “competitor”
– many, many escape grad school (or not) with absolutely vicious attitudes towards others working on related projects
- completion over sustainability encourages the Just Ship It antipattern — most research code is so heavily grown into the bench that it cannot be run outside of the lab — often even the implementer him- or herself can no longer run it, by the time dissertation defense rolls around.
It sometimes astounds me that so many brilliant researchers survive PhD land — and it worries me: so many good software designers and implementers must be turned off by the dysfunction implied by each of these three.
Here’s my favorite collaboration pattern so far: the Bank Heist collaboration pattern. This pattern, which we know from The A-Team, Ocean’s 11 and Leverage, among others, shares many properties with an excellent developer team:
- You don’t have to like following orders to be on the team.
- Everybody’s a generalist, and an expert in one area (pickpocket, cat burglar, safe-cracker, grifter, etc) but nobody is an expert at everything.
- “Building the team” is part of the fun.
- There is – or should be – mutual respect for complementary skills.
- Everybody on the team needs to do their part and get out of the other people’s way.
- Prima donnas ruin the whole party.
- There’s even a role for management: the Nate Ford/Danny Ocean “mastermind” character is an ideal manager: he can do enough of all the other players’ roles to see how they can all work together and set up the whole job.
I don’t know if identifying this collaboration pattern is actually useful, or if it’s just entertaining, but it is undoubtedly attractive: most people I’ve shared this collaboration pattern with get very excited to work with a team that uses this collaboration pattern. If you or a team you’re on derives some benefit from this pattern, drop me a note.
Heist movies pick up the drama when the team starts to violate these prescriptions: when the grifter decides he’d be a better mastermind than the current leader, for example. This opens up two perspective games I like to play:
- heistify: take your boring office politics (“QA is dawdling because they were convinced the dev will botch it anyway”) and rewrite into a bank heist: “safe-cracker didn’t bother bringing his stethoscope because he figured the second-story man wouldn’t be able to kill the alarms”. Much more fun, isn’t it?
- shyster: make heist movies boring again by inverting the transformation above.
Software development is a fundamentally social process: it’s all about working together. We (software developers as a caste) have expressions like “programming by contract” and design patterns like “Delegation” that reflect how we humans work together – and we use these patterns to describe how we instruct our robot minions to function. We think about our programs with social metaphors because we’re social apes: we think well with social metaphors, and our software design patterns reflect how we think best.
But we rarely use these social metaphors to think about how we make software. We need patterns for collaboration that match our social creature wetware, the way “delegation” and “factory” and “handshake” patterns help design software. Continue reading
One of us recalls producing a “random” plot with only 11 planes, and being told by his computer center’s programming consultant that he had misused the random number generator: “We guarantee that each number is random individually, but we don’t guarantee that more than one of them is random.” Figure that out.
which in turn reminds me of this:
My brother Daniel introduced me to a new term he and his security-geek friends are trying to encourage the rest of us mere mortals to adopt: “TPC”, or “Trusted Physical Console”.
In short, it’s the sturdy, small laptop running a trusted operating system, to which you (and probably only you) have control of the tools available. (Probably not counting: laptop provided by your boss or school, especially if you don’t have root privileges.) It’s a nifty term — especially from the point-of-view of improving security and privacy culture — because it reminds us that “the cloud” is not necessarily trustworthy, even (perhaps especially) for technically savvy people.
I made a lazyweb request for recommendations for a TPC that I can use on the bus, and here’s my out-loud thoughts summarizing the responses (received on Twitter and elsewhere):
Two weeks I saw Argo [highly recommended!] and that made me remember the 1980 October Surprise controversies, which were hecka confusing to a five-year-old at the time. I have my own November — or possibly early December — surprise coming: I’m about to become a dad, and I’m, well, 80% enthusiastic and about 20% terrified. Reminders that most of the world does this, at half my age, are welcome.
Please send bottled sleep, if you have any around; I’m trying to stockpile.
I’ve been too busy with my not-so-new job (I’ve been there ten months, commuting all the way across greater Seattle) and other things [see below] to write anything longform here, but I swing by now and then to clean out the spamtraps. And — as today — to free-associate a little.
I believe I’ve changed the settings to no longer post twitter-logs here. If you miss my weekly twitter-log updates (why would you?) you can read my twitter feed directly or read a mirror of it on this blog.
- Corollary: "computational linguist: better at linguistic analysis than software engineers, better at programming than linguists." #
- SF peeps: last chance to help Bikes for San Francisco Youth on @indiegogo http://t.co/gSfaGytl (I just pushed it over $4k!) #
- TIL: What I know as Golden Hammer Syndrome ("to man with h–, world looks like nail") should be dubbed Maslow's Hammer (he of the Hierarchy) #
- Am reading: increasingly rude argument among corpus linguists over necessity of exclusive definition for "corpus" #physicianhealthyself #
- “Eva", 2, is obsessed with our house: she saw raccoons here once. I know she's still hooked: I can hear her mom calling her out of my yard. #
- OH: "how does a career in technical comedy END?" #
- "Burton Quim, founder of ex-gay ministry Overcome…" #aptronym #seriously #seriously #
- academy is strange. “live-tweeting at conferences is a form of neoliberalism…“ why NOT share your ideas? http://t.co/VOF650uu via @Yendi #
- At UW CSE department, waiting for Carlos Guestrin talk on GraphLab. Expected to see more peeps I know, but here's a few. #
- Eight years ago, went on mostly-blind date with @imtboo . Today, we await our son. She's Best Thing Ever for me, among many awesome things. #
- 7:35p C: full; grudging when pushy boarder (me) tells “yeah yer Seattleites & don't like talking to each other; MOVE BACK“ @westseattleblog #
- Gates: "measuring software in LOC is like measuring an airplane by weight." Today I removed 40Mb of code from master. #git #airborne #
- Co-worker just told me to change my business card to read 'Software Amputect'. #negativeLoC #
- C line completely jammed. Not even standing room at 2nd & Columbia at 6 pm. @kcmetrobus, we need roughly DOUBLE this capacity at rush hour. #
- Combined insights from @jim_adler & @imtboo into new insight (what probably only makes sense to me): "curiosity is the dual of play". #
- …I got a bad feeling about this. :-[ #
- Oh, man. I wish I could read THIS Fantastic Four. Sounds amazing & fun. (Also, Sue is the leader!) http://t.co/WxqKsKP1 #