SDCC’s split personality

SDCC seems to be spiritually divided into two separate conventions. There’s the part of the convention that’s about comic books. I’m talking about artists exhibiting their work or trying to get their portfolios noticed, collectors buying and selling and getting stuff appraised. I imagine that this aspect of the convention operates in much the same way it did 40 years ago when the convention was first getting started. There’s a part of it that seems static and ageless in a weird way.

On the other hand, there’s the part of the convention that’s about “popular art”, which is pretty much a euphemism for “stuff nerds like”. This is the part of the convention with toys and games and movies and celebrity signings and Star Wars, so much Star Wars.

I think that both of these sub-conventions are important parts of what comic con has become. The “popular art” portion has become significantly larger than the comics portion, however, and that separation is getting more and more pronounced by the year. The only place I was ever able to freely move around on the show floor was Artists’ Alley (where individual artists can show off their portfolios), because it was pretty much deserted. Similarly with the vendors with endless boxes of bagged-and-boarded single issues; there just wasn’t anybody there. I don’t view that intrinsically as a bad thing, but I fail to see why some of these artists and vendors keep coming to the convention given how expensive it must be to get space on the floor. Maybe there’s an industrial interaction happening there (vendor-to-vendor, or artist-to-artist) that I’m just not seeing.

The thing that’s special for me about conventions like comic con is that it gives you, as a fan, the chance to interact with the people who make the things that you love. Whether that’s comic books, video games, TV shows or movies, you can actually meet and support (either directly through your cash, or indirectly through your public, vocal enthusiasm) the creators themselves.

There’s always going to be the marketing side of the con – after all, people are there so that you can buy their stuff, otherwise they wouldn’t have bothered coming all this way. That said, there’s a big difference between marketing in support of the creatives and marketing in support of the faceless mega-corporations, and you can definitely see some of the latter sneaking their way in. For some reason I find it really hard to pin down exactly how I feel about this, but some (not many, but some) of the large vendors had this tangible feeling of insincerity to their entire presentation, like they were hyper-optimizing for comic-con’s stereotypical target demographic (socially-awkward men in their 20s and early 30s who like bacon, apparently?) and were hoping desperately that we’d fall for it.

My only real concern with SDCC’s growing popularity is that the cost of maintaining the convention will eventually price out everyone but the faceless mega-corporations. I think that day is a long way off, if it even happens at all, but it’s still a concern.

The trouble with passwords

I’m going to go out on a limb and say that all existing methods for dealing with passwords are awful.

Just look at the options you have. You can pick an easy-to-remember password, but then it’s easy to crack. You can pick a complicated password, but then it’s difficult to remember. You can use the same password everywhere, but if a thief grabs one password, then they’ve got the keys to the kingdom. You can use a different password everywhere, but then you either have to remember 100 different passwords or come up with some easy-to-remember password generating algorithm for remembering them all. Either of these approaches require weakening the password somewhat. In short, passwords that are hard to guess are hard to remember.

Then there are password managers. Give them one master password, and they’ll generate a ton of cryptographically strong passwords for you. The only problem is that they present a large target for hackers, and they only need to guess one password – the master – to have the keys to the kingdom.

Two-factor authentication makes things a little better. In addition to providing a password, you have to provide information generated by something that you have. Google’s two-factor authentication system uses an application on your phone, for example. The problem there is that if you find yourself without your second authentication factor – say it gets stolen or broken – you’re in serious trouble, because you need that second factor to authenticate. The same thing goes for your master password in a password manager. If you lose it, you lose access to everything.

What makes it even worse is that a lot of websites completely fail at password security. They store your passwords in plaintext and e-mail them to you (in plaintext) when you forget them. They don’t accept symbols or capital letters in their passwords. They impose a maximum password length limit. And unless they support some form of external authentication like OpenID, you are at their mercy.

Let’s not leave key loggers out here. If you’ve got a piece of malware on your system logging your keystrokes, it doesn’t matter how secure your password is; if you enter it by typing it, the key logger will see it. The more complicated your password is, the easier it will be for scripts to extract.

So, the whole notion of having passwords seems fundamentally inadequate. Of course, nothing’s going to be perfect – even biometric methods like fingerprint scans and voice recognition aren’t completely foolproof. The problem really comes down to verifying that you are who you say you are. And if it’s so easy to make and get fake IDs in the physical world, why should faking your identity in the digital world be any harder?

On (Lack of) Documentation

I am really starting to get irritated with the lack of documentation present in some “production-ready” open source projects. Issues related to lack of documentation have hamstrung me multiple times in the last few months and it’s really starting to get on my nerves.

If you’re writing a library, your documentation is just as important as your code. The simple fact is that your library, regardless of how elegant or fast or awesome it is, is completely useless unless it’s got decent documentation. Decent documentation falls under a number of categories – all of these categories are important.

Thorough, up-to-date user-facing documentation: This means tutorials and example code, but it also means things like wikis that can change as users start to expose common traps and pitfalls. The documentation should change as the code changes, which means it should be auto-generated whenever possible.

Helpful exceptions and assertions: Don’t just assert(b == "foo"); actually attach a meaningful message to the assertion so that I know why the assertion was made and what it means if it failed. If you can give me a permalink to a page telling me what I’m doing wrong, so much the better. And don’t just give me “b isn’t foo. Something’s wrong.” That doesn’t give me any information. Similarly with exceptions: throwing a GenericException without any accompanying message or stack trace makes me want to punch you (seriously, I’ve seen this happen many times and it’s really irritating).

Also, please give me a stack trace. If a failed assertion doesn’t give me a stack trace and indistinguishable copies of the same assertion appear in 20 different places, the only way I’m going to know which assertion just failed is to hook a debugger to the program and try to reproduce the error. That sucks, and sometimes it isn’t even possible (if the problem is non-deterministic or the situation that causes it to happen is rare).

In-code documentation: I don’t necessarily believe that you should have a line of comments for every line of code you write; requirements that rigid lead to a lot of “This line adds 2 and 7 together” comments that just make the code harder to read. If the code gets messy, write some inline comments explaining at a high level what the code is supposed to do. Your users will thank you and when Future You looks at the code that Past You wrote, he might have a chance of understanding what it was Past You was thinking.

First, be helpful: Many things about the usage of a library may seem perfectly obvious to you because you wrote the library. To a new user, some things may not be so clear. So many times in mailing lists and message boards I see threads that look like this:

User: “Here’s a code block; it’s throwing some random error. Anyone know why?”

Developer (in Comic Book Guy voice): “I do not understand why you users are so stupid. Clearly you must initialize the host key container before initializing the SSL session but after initializing the session transport. Worst. Users. Ever.”

Or this:

User: “Here’s a code block; it’s throwing some random error. Anyone know why?”

Developers: *years of silence*

This is a great way to lose existing users and discourage new ones from using your library.

Commit messages are a part of internal documentation: Documentation is just as useful for other library developers as it is for users. Commit messages are a great deal more important for developers than they are for users, but they’re part of your documentation nonetheless. I heard a great quote relating to this in a post on source control by Troy Hunt: “Write every commit message like the next person who reads it is an axe-wielding maniac who knows where you live”.

tl;dr: Your documentation will never be perfect. It will probably never even be great, unless you’ve got people dedicated to working on documentation. Despite this, small improvements can make a big difference. Popular libraries become popular because they effectively solve a problem that a lot of users have and because it’s easier for users to use that library than it is to solve the problem themselves. Good library design and talented programmers make the first part happen; the second part can’t happen without good documentation.

On the End of the Space Shuttle

Image from National Geographic

Earlier this week, the shuttle Discovery’s lifted off for the last time. Hopefully it will return safely to Earth on March 7th, and after that it will probably end up in a museum somewhere. The end of its flight will mark the effective end of the space shuttle program, and we don’t really have anything lined up to replace it.

Some people seem to think that this is a tragic loss for the United States. Personally, I don’t think the end of the shuttle program is such a horrible thing.

I’ll admit, I’m biased; my dad works for JPL, and JPL doesn’t really do any manned missions. That said, my opinions here are my own and should not be construed as his or anyone else’s but mine.

In order to ask why it’s bad that we’re losing the shuttles, we must first ask why we had the shuttles in the first place. Mainly, they’re for moving things from the ground into low-Earth orbit. Those things might be people, or satellites like the Hubble Space Telescope, or pieces of the International Space Station (ISS). People are usually along for the ride to help deploy the equipment aboard, or to put it together once they’re in microgravity, or do experiments and run tests either in the shuttle or aboard the ISS.

In my opinion, those satellites, that space station, those experiments and tests, are in the service of one overarching goal – to do good science. I don’t think that necessarily requires putting people in space, though.

This isn’t to discount the contributions of the manned space program; the fact that we’ve been able to keep people in space for long periods of time mere decades after first sending anyone into orbit is a remarkable testament to human achievement. We also know that, eventually, we’re going to have to leave the planet (if we wait long enough, after all, the sun is going to eventually burn out), and seeing how people react to time in space helps to figure out what challenges we’re going to face in our eventual exodus.

That said, it’s unclear how much of our unanswered questions can’t be answered on the ground. Suppose that the goal of the manned space program is to put a man on Mars. A successful manned mission to Mars depends on advances in materials science, propulsion technology, and the production of clean, plentiful energy, as well as studies of the psychological and physical effects of months of isolation with only your flight-mates for company. None of these things require shooting people into space. Asking what the loss of the shuttle fleet will do to our chances of getting to Mars first is like asking how your child’s first word will affect their chances of getting into college. It’s not even clear how we’ll get there yet, let alone when that’s going to happen. In all likelihood, any mission to Mars will have to be multi-national; it will simply be too expensive in terms of money and resources for any one country to handle alone.

If you’re worried about America’s supremacy in space, you shouldn’t worry all that much just yet. NASA remains the most advanced space program in the world. We paid for most of the ISS. We’ve got ground- and space-based telescopes and imagers and orbiters that have traveled to the outer edges of the solar system and explored the planets. We’ve got probes running around the surface of Mars. If the goal is to do good science, all these things are doing good science right now and in that respect, the US is second to none.

Next, let’s look at relative cost. According to nasa.gov, it costs about $450 million to launch a single space shuttle mission. The total budget for the shuttle program over its lifetime exceeds $160 billion. By contrast, the entire cost of the Mars Exploration Rovers – building, launching, landing and performing the primary mission – was $820 million (at least according to Wikipedia). The Cassini-Huygens mission to Saturn cost the US about $2.6 billion for the whole package (again, Wikipedia). These missions have been doing good science for years for the cost of shooting humans into space for a matter of weeks.

Another criticism of the shuttle fleet’s retirement is that it will discourage young people in the United States from becoming scientists and engineers. I have an admittedly cynical view here.

When I was a kid, the shuttle program didn’t make me want to be an engineer. It made me want to be an astronaut. If you see a fighter jet streak by overhead and you’re 10, you don’t think “I want to build jet engines!”, you think “I want to fly jets!”  What cultivates a desire to become an engineer is an innate fascination with how things work and a strong desire to either understand why things work, or build things that work, or both. What got me excited about engineering as a kid was my dad showing me up-close pictures of Jupiter and Saturn and knowing that he helped build the thing that was out there, really far away, taking those pictures.

People who weren’t as fortunate as I was to grow up in an environment where interest in science and engineering is cultivated can face the cold hard truth – that so-called STEM (science, technology, engineering and math) disciplines are where the money is and where the jobs are. In the current economic environment, that’s as much motivation as they’ll need if they’re even a little bit interested. Not that they should be solely motivated by money, but it’s a pretty good incentive.

If we’re only in the business of putting people into space for the sake of saying “America puts people into space all by ourselves, look how mighty we are”, without regard to why precisely we’re putting them there or what they expect to accomplish, we’re wasting our time and energy on a vanity project. In short, chill out – America is still doing stuff in space and we’re still doing good science. As long as our representatives in Washington understand that the space program isn’t just the space shuttle and don’t completely strip NASA of funding, that’s unlikely to change in the near term.

Automatic Everything

This was going to be an extended rant on how people use databases where people shouldn’t use databases, but the more I wrote the more I realized that this had been analyzed quite a bit by many in the systems research community and blogosphere at large, many members of which are far more knowledgeable than I. So I’ll summarize my rant in a paragraph and then move onto more philosophical, “meta”-type comments.

Twitter’s architecture (as much as they’ve shown us) is a Ruby on Rails app backed by a MySQL database. This combination is the Golden Hammer of Web 2.0. A frighteningly large number of web application developers seem to follow the mantra, “If I need to store data, use SQL as a Big-Ass Table (no, not that Big-Ass Table). Who needs high-speed middleware? I’ll write everything in Ruby!” The problem is that schema design is as close to alchemy as CS gets and tuning databases is tedious and hard to do right. If you are writing something that must process tens of thousands of messages a day, do not think you can write it in an interpreted language and have it frequently converse with a database. If you think this will work, you are living in a magical dream world. I’m talking directly to you, Twitter, you poor sad whipping boy of the Web 2.0 universe. Please, for your own sake, rewrite Starling in C or C++ and use a more suitable back-end.

That concludes the synopsis of my multi-page rant of doom. Now, for the meta: if I were to write an essay for NPR’s This I Believe, the following would be that essay.

I believe in telling systems what I want, not how to get it, and having them give it to me as quickly as possible. I believe that programmers are lazy, and that middleware should give them the ability to do the right thing the easy way. I believe in intrinsic scalability and building on sound principles. I believe that the disk is evil and writing to it should be avoided until you have no other choice. I believe in most of what databases do and in the potential of what their descendant systems can and will do.

I believe in the awesome potential of automatic everything.

This is the future – why are my updates still failing?

So anyone who has an iPhone or iPod Touch will be pretty aware that Apple’s update servers basically fell over in response to all the demand today due to the new iPhone firmware. Recently, Firefox’s update servers suffered exactly the same problem. Now I’m sure that these guys have a really expensive load balancer in front of their update server cluster, but why in the world are so many major companies still having all their users go to a single place for updates?

If I want to download an update from Software Update today on my home computers, I have to do it three times - once for my Mac Mini (file server/backup server/media center), once for my laptop and once for my tower. The actual update binary is, in most cases, identical. If I wanted to only download the update once, I’d have to find where Software Update keeps the update’s installer file, copy it to the other machines and run it there. In some cases I have to download tens or hundreds of megabytes of file that could easily be transferred over my home network, saving both my time and the update provider’s money.

The thing that’s the most irritating about this is that it’s a completely solved problem. Blizzard, for example, distributes updates to World of Warcraft over Bittorrent. My roommate just started playing WoW again and had to install a patch (~2 GB) on two of his computers. He downloaded and installed the patch on the first computer, which took about an hour and a half. The download-and-install process for the second computer took all of about five minutes because the computer automatically recognized that a source for the update existed on its local network and downloaded the file peer-to-peer from the other machine.

Imagine if everyone interested in downloading the iPhone patch could download it not only from Apple but from each other. After the first few hundred downloads (which would have to pull directly from Apple) most of the remaining transfer would be peer-to-peer. If iTunes needs to authenticate the phone with Apple before installing, that’s fine; the load on the servers from authorization would be far lower and of a much shorter duration than the load from patch downloading. Security, of course, is an issue with Bittorrent-esque downloads, but there are relatively straightforward ways to deal with that.

I’m just saying it’s about time that someone did something about this, because it’s getting a little ridiculous.

When will it end?

Dear Sega:

Please stop making terrible Sonic games. You’ve been doing almost constantly since 1999 and it has to stop. A little part of my soul dies every time another of your unpolished, forgettable piece of crap games gets released with my childhood hero’s name on it. Thank you.