Automatic Everything

Posted: August 21st, 2008 | Author: | Filed under: Computers, Ranting | 1 Comment »

This was going to be an extended rant on how people use databases where people shouldn’t use databases, but the more I wrote the more I realized that this had been analyzed quite a bit by many in the systems research community and blogosphere at large, many members of which are far more knowledgeable than I. So I’ll summarize my rant in a paragraph and then move onto more philosophical, “meta”-type comments.

Twitter’s architecture (as much as they’ve shown us) is a Ruby on Rails app backed by a MySQL database. This combination is the Golden Hammer of Web 2.0. A frighteningly large number of web application developers seem to follow the mantra, “If I need to store data, use SQL as a Big-Ass Table (no, not that Big-Ass Table). Who needs high-speed middleware? I’ll write everything in Ruby!” The problem is that schema design is as close to alchemy as CS gets and tuning databases is tedious and hard to do right. If you are writing something that must process tens of thousands of messages a day, do not think you can write it in an interpreted language and have it frequently converse with a database. If you think this will work, you are living in a magical dream world. I’m talking directly to you, Twitter, you poor sad whipping boy of the Web 2.0 universe. Please, for your own sake, rewrite Starling in C or C++ and use a more suitable back-end.

That concludes the synopsis of my multi-page rant of doom. Now, for the meta: if I were to write an essay for NPR’s This I Believe, the following would be that essay.

I believe in telling systems what I want, not how to get it, and having them give it to me as quickly as possible. I believe that programmers are lazy, and that middleware should give them the ability to do the right thing the easy way. I believe in intrinsic scalability and building on sound principles. I believe that the disk is evil and writing to it should be avoided until you have no other choice. I believe in most of what databases do and in the potential of what their descendant systems can and will do.

I believe in the awesome potential of automatic everything.

No related posts.

One Comment on “Automatic Everything”

  1. 1 Nicholas Peterson said at 9:12 am on November 1st, 2008:

    I feel you, I don’t touch alchemy if I can help it. I had a project where we had to design advanced hash tables and I stared into the darkness for too long. I feel like the problem with a lot of web 2.0 stuff is that people automatically believe new is better. Ruby is relatively new, so it must be good. The problem is that when it comes down to it, C does pretty much everything you would ever need to do in programming only faster than interpreted languages. It’s just harder to conceptualize and do. I feel that the web is purposefully given these languages to give it’s developers some sanity after years of having to write programs for lackluster browsers. Pretty soon all this will be fixed and we’ll all be talking about how shit IE10 handles VRML2.0 because we needs digg posts in 3d.