This was going to be an extended rant on how people use databases where people shouldn’t use databases, but the more I wrote the more I realized that this had been analyzed quite a bit by many in the systems research community and blogosphere at large, many members of which are far more knowledgeable than I. So I’ll summarize my rant in a paragraph and then move onto more philosophical, “meta”-type comments.

Twitter’s architecture (as much as they’ve shown us) is a Ruby on Rails app backed by a MySQL database. This combination is the Golden Hammer of Web 2.0. A frighteningly large number of web application developers seem to follow the mantra, “If I need to store data, use SQL as a Big-Ass Table (no, not that Big-Ass Table). Who needs high-speed middleware? I’ll write everything in Ruby!” The problem is that schema design is as close to alchemy as CS gets and tuning databases is tedious and hard to do right. If you are writing something that must process tens of thousands of messages a day, do not think you can write it in an interpreted language and have it frequently converse with a database. If you think this will work, you are living in a magical dream world. I’m talking directly to you, Twitter, you poor sad whipping boy of the Web 2.0 universe. Please, for your own sake, rewrite Starling in C or C++ and use a more suitable back-end.

That concludes the synopsis of my multi-page rant of doom. Now, for the meta: if I were to write an essay for NPR’s This I Believe, the following would be that essay.

I believe in telling systems what I want, not how to get it, and having them give it to me as quickly as possible. I believe that programmers are lazy, and that middleware should give them the ability to do the right thing the easy way. I believe in intrinsic scalability and building on sound principles. I believe that the disk is evil and writing to it should be avoided until you have no other choice. I believe in most of what databases do and in the potential of what their descendant systems can and will do.

I believe in the awesome potential of automatic everything.

If I want to download an update from Software Update today on my home computers, I have to do it three times - once for my Mac Mini (file server/backup server/media center), once for my laptop and once for my tower. The actual update binary is, in most cases, identical. If I wanted to only download the update once, I’d have to find where Software Update keeps the update’s installer file, copy it to the other machines and run it there. In some cases I have to download tens or hundreds of megabytes of file that could easily be transferred over my home network, saving both my time and the update provider’s money.

The thing that’s the most irritating about this is that it’s a completely solved problem. Blizzard, for example, distributes updates to World of Warcraft over Bittorrent. My roommate just started playing WoW again and had to install a patch (~2 GB) on two of his computers. He downloaded and installed the patch on the first computer, which took about an hour and a half. The download-and-install process for the second computer took all of about five minutes because the computer automatically recognized that a source for the update existed on its local network and downloaded the file peer-to-peer from the other machine.

Imagine if everyone interested in downloading the iPhone patch could download it not only from Apple but from each other. After the first few hundred downloads (which would have to pull directly from Apple) most of the remaining transfer would be peer-to-peer. If iTunes needs to authenticate the phone with Apple before installing, that’s fine; the load on the servers from authorization would be far lower and of a much shorter duration than the load from patch downloading. Security, of course, is an issue with Bittorrent-esque downloads, but there are relatively straightforward ways to deal with that.

I’m just saying it’s about time that someone did something about this, because it’s getting a little ridiculous.

Dear Sega:

Please stop making terrible Sonic games. You’ve been doing almost constantly since 1999 and it has to stop. A little part of my soul dies every time another of your unpolished, forgettable piece of crap games gets released with my childhood hero’s name on it. Thank you.

I’m of the opinion that “long tail” social networking – social networking for niche groups of people – is becoming increasingly irrelevant. Most online startups are designed with one thing in mind – giving users something they don’t know they need yet and making money off of it. The social networking demographic knows they need social networking. Facebook has rendered many social networking startup ideas stillborn with the recent release of its API. In the post-Facebook-API world, saying “I’m going to make a social networking site for golfers to compete with Facebook” is like saying “I’m going to make a blog engine for golfers to compete with WordPress”. Facebook isn’t just a social networking site anymore, it’s becoming a social networking platform. Making another social networking platform targeted at a specific audience is reinventing the wheel – and it’s tackling entirely the wrong problem. Companies like iLike have gained massive amounts of exposure by leveraging Facebook’s API to augment their existing service. Other groups have developed systems that operate purely in Facebook and have done quite well.

If you’re looking to reinvent the social networking site, that’s one thing. If you’re making a Facebook knockoff because that other twenty-something guy was successful at it, that’s quite another.

Microbes just not interesting enough for you? Now your e-mail is buxom, scantily-clad beach-goers frolicking in the sand! No really, it is. I really wish I were kidding. Watch the demo video – it’s priceless.

Now I’m all in favor of visualizing data in ways that makes it more intuitive for people to view and sort through, but am I alone in finding this a bit ridiculous? A common complaint from people at work is that they receive too much e-mail, spend too much time in their inboxes every day. I somehow doubt that having to round up and towel off my e-mail before reading it will make that any better.

Gross.

And the more I thought about it, the more the sad truth of it became apparent. I mean, look what happened to the Muslim community (and anyone whose ancestors were from the Middle East) in 2001. People are a bunch of idiots, apparently. I don’t remember the “white backlash” after Columbine, probably because there wasn’t one. The man who killed all those people on Monday was not acting on the part of Koreans everywhere and to imply that the Korean community or some sort of collective consciousness on their part is responsible for this tragedy is absolutely absurd. Yet people continue to do so, and it boggles my mind.

This is one more reason I think that judging, rewarding or punishing based on race or ethnicity, any race or ethnicity, is an absolute crock of crap. We live in a country that is culturally and socially myopic, and brother does it show these days. Perhaps if we stopped pointing fingers and arbitrarily assigning blame we could figure out how to prevent things like this from happening in the future. There’s only one person to blame here, and he’s dead. I have no sympathy for this man, but I do sympathize with the people who have yet to be wronged by his actions because of how he looked.

“Hi, I’d like to put my name in for call-ahead seating?”

“OK, what time would you like to put that in?”

What time? This is an unexpected question. Now would work fine? “Uhh, now I guess?”

“Well, you see sir, the best we can do is, like, put your name at the bottom of the list and we call you when we have free space.”

Thank you, Captain Obvious, of course you can’t just push me to the front of the list.

“… uhh, yeah, right, that’s exactly what I’d like to do. I’d like to put my name on the list. For 2, if that’s possible.”

“Oh, um OK. Bye!”

“Wait, aren’t you going to tell me how long the wait is going to be?”

“The wait? What wait?”

“I mean, normally I tell you my name and how many people, and you tell me about how long I can expect to wait. That’s how this works, right?”

Sure enough, this was the way things were, and the estimate I got cut my wait time down by a few minutes, but I have to ask the obvious question here. Am I wrong in assuming that the entire motivation behind call-ahead seating is to plan your arrival time so that you don’t have to wait? Is there some other idea here that I’m not understanding?

I’m not regretting my decision to go to graduate school instead of work. I think that it will ultimately be one of the better decisions I’ve ever made. The process of getting applications turned in is draining and seeing more and more of my friends give into the strain, give up and take jobs shows me an easy out that, although I don’t want to take it, is take-able. With things as hectic as they are that option is really tempting and resisting that temptation doesn’t really help my mood.

Our football team is going to a pretty mediocre bowl, but seeing the Big Game last Saturday (which, by the way, we won for the fifth year straight) made me realize that our team this year was sort of over hyped and that the Rose Bowl would have been an utter bloodbath even if we had gone. And the fact that we’re co-champs of the Pac 10 for the first time in a while definitely improves the overall shape of the season. Kudos to UCLA for making that happen, by the way. So yeah, I’m not as nearly disappointed as I thought I’d be as the season closes.

You probably won’t hear from me again until after this is all over. That said, don’t worry about me, I’m fine.

It’s things like this that remind me why I die a little inside whenever I realize that, damn, I could have gone there instead of grad school.