«Knights of the Rainbow Table 2 Chapter Knights of the Rainbow Table —Cory Doctorow (doctorow T he day that our lawyer told us that ...»
Knights of the
Knights of the
—Cory Doctorow (firstname.lastname@example.org)
T he day that our lawyer told us that the DA had added a federal conspiracy
charge to the rap sheet, I immediately flashed on what Abbie Hoffman
said when they served papers on him for the Chicago 8 conspiracy trial:
“Conspiracy? Man, we can’t even agree on where to have dinner!” I would have
said it, but just after Moszkowski gave us the bad news, our waitress showed up and asked Sir Tristan what he wanted to order.
We Knights of the Rainbow Table had always had a problem with restaurants: that problem’s name was Sir Tristan Erkko, who was intolerant of lactose, processed carbohydrates, salt, vegans, forks with bent tines and people with poor grammar. Tristan never spoke above a whisper, and he affected a huge mustache that made it impossible for waitresses to read his lips, which made the lengthy negotiations even longer. Normally, the Knights of the Rainbow Table tried to find quiet restaurants where Sir Tristan could dicker at length without having to whisper directly into the server’s ear, but Moszkowski had chosen the place, a busy pizza parlor on Telegraph Hill filled with noisy Berkeley kids. I suppose he thought it would allow us to converse without being overheard. He was a paranoid old civil rights fixer, but Knights of the Rainbow Table could have taught him volumes about the practical limits of privacy.
The three of us stared uncomfortably at one another until Tristan had conveyed his exacting pizza parameters, Lady Tracey and Moszkowski took their turns, and then it was my turn.
8 Chapter 2: Knights of the Rainbow Table ■ 9 Here’s the truth: I might make fun of Sir Tristan’s ordering peccadilloes, I might sneer at Moszkowski’s privacy naiveté, I might turn up my nose at the pineapple and anchovies that Lady Tracey eats with such gusto, but the fact is, at least they can all *eat,* which is more than you can say for me.
“I’ll have a glass of water,” I said, and endured three pairs of eyes looking at me with that mixture of pity and disgust I’ve come to know so well. My stomach growled at me, a sound I felt to my eyebrows, and I touched my midriff with one bony finger, a feeling like a dried out drum-head, the skin stretched so tight and desiccated that it rustled.
The waitress reached out with a chewed fingernail and tapped the scratched sign on the table that said, “$10 minimum order per customer.” I’d already noticed the sign. I slid out the $10 bill I’d placed under my fork and knife and passed it to her. “It’s cool,” I said.
She rolled her eyes with youthful eloquence and walked off, leaving our conspiracy to get back to business.
“You know,” I said, “for a gang of supercriminals, we’re a pretty sorry bunch.” Moszkowski said, “I’m going to pretend you didn’t say that.” Sir Tristan reached across the table and took my fork and held it up to the light, inspecting the tines and comparing it to his fork. Wordlessly, Lady Tracey handed him her fork, too, and after some deliberation, he picked one and passed the other two back.
Lady Tracey waited until he’d carefully squared the fork up on his napkin, keeping it evenly spaced with his knife and framed by a uniform border of white paper napkin. Then she darted out one hand, snatched the fork he’d so carefully chosen, and licked all its tines, front and back, and returned it to its spot.
Sir Tristan glared at her and then set the fork aside and picked up the remaining three and chose the one with the second-most-uniform tines and once again set to squaring it up on his napkin. I saw the demonic glee light up Lady Tracey’s eyes as she made ready to lick this one, too, and I put out my hand and caught her wrist. “Don’t,” I said. “Come on.” My cracked skin rasped over Lady Tracey’s smooth fingers and she shuddered involuntarily.
“All right, children,” Moszkowski said. “Enough. You’re being arraigned tomorrow and I need you all to be on your best behavior. The Computer Fraud and Abuse charges are bad, add the conspiracy count and there’s a serious chance you’re going to end up in a cell until your trial. None of us want that, do we?” 10 ■ The Tomorrow Project Anthology: Conversations About the Future Here, at last, was something that all the storied Knights of the Rainbow Table could agree upon.
### Once upon a time, in the Duchy of Berkeley in the mythical kingdom of Bayarea, three brave knights did swear an oath to fight the forces of trolls, creeps, identity thieves, snoops, spooks and the whole motley army of Internet Evildoers, lo until they breathed their last breath.
We didn’t set out to become knights. We set out to test a cluster. A big, big cluster. Tristan got the idea after going to an art installation in which 1,000 antique PCs were networked across row on row of fake Metro wire shelving, each with a 3.5” floppy in its drive. The artist had done some clever mojo to span a single, redundant 1GB filesystem across the whole cluster, which was used to store and retrieve a constantly updating loop of video shot with a single camera fixed into the ceiling, looking down on the exhibition. The artist had hired about a dozen kids on roller-skates to ride around the racks, swapping out floppies as they went corrupt, the video on the big screen juddering every time this happened and the filesystem recovered.
“So my idea was why not build something you know useful out of all the old computer junk around here like a cluster or something?” Tristan always had the least punctuation of anyone I knew, but he made up for this by inserting extra excited spittle between his words as he talked.
“Where would we keep it?” I said. At the time, Tristan and I were sharing a one-bedroom apartment near campus. I slept in the living room. When we had houseguests (itinerant hackers, mostly), they slept on an air mattress on the kitchen floor.
“What about the electric bill?” Tracey said. She’d bossed a colo in Texas before she moved to Berkeley for college. She could quote the formula for calculating the net amperage-per-flop for the chillers from memory. “What about administration? What about backhaul?” Tristan shrugged and twirled up some of his mustache and stuck it in his mouth. Neither of us took any notice; he’d been doing that since freshman year, when we three had aced a joint project together that led to frequent hanging out and then roommatehood and even a short-lived and mercifully painless romance between Tracey and me. “I don’t know but what about a cluster? I mean a big one like a teraflop or bigger I just thought it’d be cool.” We admitted this was so and went on to the next thing and the next and several days went by before Tracey forwarded us both an email about the
Chapter 2: Knights of the Rainbow Table ■ 11engineering faculty inheriting a semi-derelict brick factory near the docklands as a bequest on the condition that they not sell it for 10 years. Tristan wrote most of the proposal (he was much better in print than he was in person), Tracey filled in the technical details, and I made the pitch to a prof we all liked and he passed it up the food chain and we ended up with 10,000 square feet— about one tenth of the available square footage.
We rigged the water-cooling ourselves, using sea-water. Our faculty supervisor carefully ensured that he didn’t know what we were doing. At first, there were a few other groups that applied for space in the Brick Shithouse (as the factory was instantly dubbed, thanks in part to the faint sewage smell that no amount of airing could get rid of ). But no one really wanted to haul ass out to the Shithouse and soon we had the place all to ourselves. Our 10,000 square feet quickly grew to nearly the whole building, row on row of PCs of every description salvaged from campus and nearby, wiped and enlinuxed, networked and left to wheeze for as long as they went on working. At any given time, about 15 percent of the cluster was nonfunctional, and we made good use of impressionable freshmen whom we sourced via Craigslist and threw at the problem machines, letting them keep whatever they could fix. This sounds zero-sum: if they took everything they fixed, wouldn’t that mean that the dead machines would disappear?
But fixing computers is like eating potato chips: most people can’t stop at just one. Each one is a perfect puzzle of vendor defects, material wear, capricious software ghouls, and emergent phenomena. The brain-reward for restoring a genuinely threadbare PC to active duty was more psychotropic than anything for sale in People’s Park.
How much power did we have? Less than Google; more than all the radio astronomers in Europe, combined. The number isn’t important: to buy an hour’s worth of as much CPU as we assembled in today’s cloud marketplace, you’d have to work for two hours at minimum wage. Our pizza waitress could blow us out of the water and still have enough left for rent, assuming she had a roommate.
What did we do with all that power? At first, we just entered various big computation projects, throwing our farm at Folding@Home to do some handily parallelizable fast Fourier transforms in order to help fight AIDS and cancer and such. This is extremely altruistic work, but it’s not very interesting from a research perspective. As interesting as it was to step into the frigid Shithouse and be engulfed in the white-noise jetwash of all the computers, we weren’t learning anything about our cluster or its individual components.
12 ■ The Tomorrow Project Anthology: Conversations About the Future
without gain. In order to gain access the hacker needs to know the password, not its hash. Thus a computer can be secured with a minimum of secrecy, and that’s good, because secrets are very hard to keep (more on this anon).
Think of hashed passwords as the language of a dead culture with no progeny, a remarkable, forgotten people who had an equivalent word for every word in every contemporary language and even for our non-words and nonsense and who left a key for translating our written phrases into theirs, but with no key by which we might go in the opposite direction. As these foreign words have no cognates with our own, there is no way to guess, merely by reading a word in Hashese, what its contemporary equivalent might be.
Enter the rainbow table. Imagine that you fed a dictionary into the hashing algorithm and made a table of the hashed equivalent of every word, and even non-words, such as words in combination, words with simple substitutions, slang, vernacular, foreign words, common phrases. Then, having captured a password’s hash, you could compare it with every one of those hashed words— look it up in your handy English-Hashese/Hashese-English dictionary–and out pops the secret password, no longer a secret. Now you have the password, which is all you need to successfully impersonate its owner to some naive computer, and mischief awaits.
“What would we do with rainbow tables I’m no crook,” Tristan muttered from under his mustache, as he idly punched the product ID off a potentially faulty network card into Google to see if anyone had written alternative drivers for it.
Tracey started to answer, but just then one of our undergrad technicians came to the door of the Shithouse’s little office, reporting a fault in the cooling system. The cooling system had a *lot* of faults, because water-cooling systems that use sea-water are just stupid, given that sea-water gradually eats everything that contains it, and what it doesn’t eat, it chokes with a sclerotic crust of dried salts. Not that we had any choice, because running traditional chillers would have clobbered the Shithouse’s annual budget in a matter of weeks, and without aggressive cooling, the place would spike at 60’ C in about five minutes, and then it would be a race to see whether the assembled computers fused and died before they could start a fire that killed us all.
Cooling took priority.
When Tracey came back–it was a minor pump problem that we’d all fixed a dozen times and it was mostly just a matter of knowing where to kick and how hard–she said, “Why rainbow tables? Two reasons: first, they are cool, and;
second, they parallelize.” 14 ■ The Tomorrow Project Anthology: Conversations About the Future That was the kicker, parallelization. A parallelizable problem is one that can be worked on in lots of different places because no step relies on the output from a previous step. Think of stuffing envelopes with friends: one group of friends can address envelopes, another can fill them, and a third can run them through the postage meter. That’s pretty parallizable–if you finish with the meter before the stuffing group is done, you can grab a stack of metered envelopes and start stuffing, too. In theory, one slow doofus won’t leave a roomful of people sitting around, fuming.
Now, compare that with a serial task, like assembling a jet engine on an assembly line. First, Alice screws on one dingus, then Bill attaches a dingus to that, and Carol attaches a dingus to *that,* and so on, all the way to Zeke, fitting the cowling at the other end of the assembly line. If Alice had a wild night last night and is dragging her ass as a result, Bob, Carol and the rest, yea, unto Zeke are going to be stuck with thumbs firmly planted up their alimentary canals.
It might be possible to redesign a jet engine factory to run with more parallel tasks–think of the famed Japanese car companies where small teams assemble whole cars, where one team’s dragassing doesn’t hold up the factory–but there’s a lot of computation where the next step depends on the previous one and the more of that there is, the less you can do with a big parallel cluster.
Of course, there’s always some extent to which problems are serial. If there’s one guy who’s really slow with the address labels, eventually, you’re all going to be sitting around waiting for him to finish off his sheet, looking meaningfully at your watches and trying to figure out when the last bus goes.