SDCC’s split personality

SDCC seems to be spiritually divided into two separate conventions. There’s the part of the convention that’s about comic books. I’m talking about artists exhibiting their work or trying to get their portfolios noticed, collectors buying and selling and getting stuff appraised. I imagine that this aspect of the convention operates in much the same way it did 40 years ago when the convention was first getting started. There’s a part of it that seems static and ageless in a weird way.

On the other hand, there’s the part of the convention that’s about “popular art”, which is pretty much a euphemism for “stuff nerds like”. This is the part of the convention with toys and games and movies and celebrity signings and Star Wars, so much Star Wars.

I think that both of these sub-conventions are important parts of what comic con has become. The “popular art” portion has become significantly larger than the comics portion, however, and that separation is getting more and more pronounced by the year. The only place I was ever able to freely move around on the show floor was Artists’ Alley (where individual artists can show off their portfolios), because it was pretty much deserted. Similarly with the vendors with endless boxes of bagged-and-boarded single issues; there just wasn’t anybody there. I don’t view that intrinsically as a bad thing, but I fail to see why some of these artists and vendors keep coming to the convention given how expensive it must be to get space on the floor. Maybe there’s an industrial interaction happening there (vendor-to-vendor, or artist-to-artist) that I’m just not seeing.

The thing that’s special for me about conventions like comic con is that it gives you, as a fan, the chance to interact with the people who make the things that you love. Whether that’s comic books, video games, TV shows or movies, you can actually meet and support (either directly through your cash, or indirectly through your public, vocal enthusiasm) the creators themselves.

There’s always going to be the marketing side of the con – after all, people are there so that you can buy their stuff, otherwise they wouldn’t have bothered coming all this way. That said, there’s a big difference between marketing in support of the creatives and marketing in support of the faceless mega-corporations, and you can definitely see some of the latter sneaking their way in. For some reason I find it really hard to pin down exactly how I feel about this, but some (not many, but some) of the large vendors had this tangible feeling of insincerity to their entire presentation, like they were hyper-optimizing for comic-con’s stereotypical target demographic (socially-awkward men in their 20s and early 30s who like bacon, apparently?) and were hoping desperately that we’d fall for it.

My only real concern with SDCC’s growing popularity is that the cost of maintaining the convention will eventually price out everyone but the faceless mega-corporations. I think that day is a long way off, if it even happens at all, but it’s still a concern.

SDCC 2011

Last weekend was Comic-Con.  This was the third consecutive time I’ve been to SDCC. I really don’t have an excuse not to go, since I live pretty close to the convention center. As always, it was a lot of fun. It’s gotten me thinking about a couple of things that I’ll try to puzzle out here.

Holy crap, the lines. Somehow the lines seemed a lot worse than in previous years, although I’m sure they were probably comparable. We managed to get to the convention center pretty early, and we still waited almost two hours in line just to get our badges to get in (first world problems, right?). Thankfully they let you print all of your days’ badges at once rather than having to wait in line every day.

We were originally thinking of buying next year’s tickets at this year’s con (given how poorly the ticket buying experience went this year), but that was a non-starter. They released a certain number of tickets per day (2500 or 3000 I think), and those tickets sold out within 45 minutes of going on sale. People were camping out in front of the hotel where tickets were being sold. No matter how popular the event is, that’s kind of ridiculous.

The incredible lines (for just about everything) really make me think that the convention has outgrown the venue. I think they’re trying to artificially decrease demand by raising ticket prices (four day passes are $175 next year, which is kind of steep and a significant increase from last year) but I’m not sure that’s going to have any effect. If you’re flying in from out of town and staying in a hotel for four days, you’ve spent well in excess of the ticket price just to get to the door. It might stop a lot of locals from showing up just for kicks, though. The consensus among our little group seemed to be that they’d have much better luck putting it somewhere like Vegas that has the convention and hotel space. Somehow though, I don’t think it would ever be quite the same after that. Given that San Diego just renewed their contract to host the convention for the next several years, it sounds like the long lines are going to keep being a feature of the convention for some time to come.

What’s the deal with all the QR codes? Having a QR code (those square two-dimensional barcodes that you can read with your smartphone) somewhere on your booth or product seems to have been a requirement this year. I’ve never seen so many QR codes in one place. I’m wondering if this was a tie-in to the SDCC iPhone app or if there’s some larger QR code trend going on. Still, it was fun to actually be able to scan QR codes.

SDCC is really two conventions. I started writing about this aspect of the convention and realized that I had a lot more to say than would fit comfortably in one post, so I’ll post that particular screed tomorrow.

I’d definitely like to go back next year (assuming I can get tickets, of course), but I think I’ll be satisfied with only going for one or two days. The only reason I would imagine you’d want to go for all four days is to go to the panels, and while the ones I did get into were pretty cool, I’m not sure they were worth the hours standing in line.

The trouble with passwords

I’m going to go out on a limb and say that all existing methods for dealing with passwords are awful.

Just look at the options you have. You can pick an easy-to-remember password, but then it’s easy to crack. You can pick a complicated password, but then it’s difficult to remember. You can use the same password everywhere, but if a thief grabs one password, then they’ve got the keys to the kingdom. You can use a different password everywhere, but then you either have to remember 100 different passwords or come up with some easy-to-remember password generating algorithm for remembering them all. Either of these approaches require weakening the password somewhat. In short, passwords that are hard to guess are hard to remember.

Then there are password managers. Give them one master password, and they’ll generate a ton of cryptographically strong passwords for you. The only problem is that they present a large target for hackers, and they only need to guess one password – the master – to have the keys to the kingdom.

Two-factor authentication makes things a little better. In addition to providing a password, you have to provide information generated by something that you have. Google’s two-factor authentication system uses an application on your phone, for example. The problem there is that if you find yourself without your second authentication factor – say it gets stolen or broken – you’re in serious trouble, because you need that second factor to authenticate. The same thing goes for your master password in a password manager. If you lose it, you lose access to everything.

What makes it even worse is that a lot of websites completely fail at password security. They store your passwords in plaintext and e-mail them to you (in plaintext) when you forget them. They don’t accept symbols or capital letters in their passwords. They impose a maximum password length limit. And unless they support some form of external authentication like OpenID, you are at their mercy.

Let’s not leave key loggers out here. If you’ve got a piece of malware on your system logging your keystrokes, it doesn’t matter how secure your password is; if you enter it by typing it, the key logger will see it. The more complicated your password is, the easier it will be for scripts to extract.

So, the whole notion of having passwords seems fundamentally inadequate. Of course, nothing’s going to be perfect – even biometric methods like fingerprint scans and voice recognition aren’t completely foolproof. The problem really comes down to verifying that you are who you say you are. And if it’s so easy to make and get fake IDs in the physical world, why should faking your identity in the digital world be any harder?

Sonic Megamix – An Outstanding Fan-Made Sonic Game

Technically, Sonic Mega-Mix is a disassembly modification of Sonic CD. The modification is so extensive, however, that it’s pretty much an entirely new game. I’ve been disappointed with the mediocrity of official Sonic games for a long time, so games like this are a breath of fresh air. If you’ve got even a remote interest in re-living the glory days of Sonic’s 16-bit past, grab an emulator (the best one I’ve ever seen is Kega Fusion) and download the game here. It won’t disappoint.

Esoteric Tip #5: Finding Undocumented Files with Doxygen

In an effort to practice what I preach, I’ve been spending 15 minutes a day documenting code. I’ve been using Doxygen for all my C++ documentation. One thing that I wanted to do early on was easily identify all the files where I was missing documentation and, if possible, the particular places where documentation was missing. I think I’ve managed to do it (or at least come close).

In my Doxyfile, I’ve got the following lines:

WARNINGS = YES
WARN_IF_UNDOCUMENTED = YES
WARN_IF_DOC_ERROR = YES
WARN_LOGFILE = doxygen_warn.log

This means that Doxygen will generate warnings for undocumented members as well as if the documentation is incomplete or isn’t syntactically correct, and that it will log all warnings to the file doxygen_warn.log.

In order to get a list of undocumented files, I look for warnings about missing documentation, extract the filenames for each warning, and make sure that the list of files doesn’t contain any duplicates:

grep "is not documented" doxygen_warn.log | awk '{print $1}' | sed -E 's/:[0-9]+://g' |
    sort | uniq

In order to further expand that list to include the undocumented members in each file, I do something similar, except I extract the filename and information about the undocumented member instead of just the filename:

grep "is not documented" doxygen_warn.log | sed -E 's/is not documented.//g' |
    cut -d' ' -f 1,3- | sed -E 's/:[0-9]+:/ : /g' | sort

It’s kind of quick and dirty, but it seems to do the job. These commands run automatically whenever I generate documentation. The goal (unreasonable though it may be) is to bring the list of undocumented files down to zero.

Managing the Information Flood

We’re constantly being bombarded with information. Good or bad, the amount of information reaching us is only going to increase. We need to be able to filter it or we won’t have time for anything else. Unfortunately, this filtering is hard for computers to do well.

If you’re subscribed to even a handful of RSS feeds, you might get hundreds of new items from RSS feeds streaming into your feed reader every day. Add social networks into the mix and you’ve got hundreds of people streaming pieces of their lives to you constantly. And then there’s e-mail. Everybody seems to hate e-mail because of how much of it they get every day. Entire books have been written about how to effectively manage e-mail, but their tips and tricks all seem to be different forms of sophisticated filtration.

Filtration assumes that not all items you receive over these channels are equally important to you. The filter’s job is to sort items in decreasing order of importance and hide items that are completely unimportant. In the past, newspapers and magazines served as filters, but they were inherently static and didn’t update very frequently. The name of the game in the 21st century is ridiculously fine-grained customization in as close to real-time as possible. Making this work in a straightforward, automated way is really hard.

Say you only want to subscribe to RSS items from a site that covers a particular topic. If you’re really lucky, the site has many available RSS feeds categorized by topic. If you’re only slightly lucky, the site’s items will each have an easy-to-find list of categories or a well-defined system for titling items, either of which can be extracted and used to filter the feed by something like Yahoo Pipes. Unfortunately, anything beyond the most basic filtration with such systems isn’t really effective unless you know regular expressions, and even then it’s a bit limited. Also, you’re limited to filtering by those categories – if you want a custom subset of items from a given category, you’re stuck.

This problem stems from the fact that most of the information on the web has a really complicated structure. Some might even say that it’s “unstructured”, but that isn’t really true. Everything that people produce is at least partially structured, it’s just that the structure is really complicated and it’s really hard to get computers to make any sense of it. Being able to look at a block of text and accurately determine what it’s talking about is a huge challenge that forms the basis for entire areas of active research.

Fortunately, filtering information by importance on a source-by-source basis is a little easier. Google+’s circles are a step in the right direction here, allowing you to filter the social network fire-hose that comes in (and goes out) to include only people in a given context. Twitter’s lists are another example of a fairly low-effort information filtration system, although they only work on your incoming data. The problem with these mechanisms, however, is that you have to manage them manually. Friends must be moved into and out of lists or circles or whatever other mechanism. There aren’t a lot of good ways available to say “please infer how I know these people and prompt me if you’re not really sure”. Doing things like this is yet another active area of research.

Clearly we haven’t gotten this problem completely figured out, but it’s really important that we do. As a society, we’re already experiencing information overload, and I fear that we’ll eventually feel like we only have two choices: be overwhelmed by information or simply live without it.

Sort benchmark records!

My thesis work focuses on highly-efficient large-scale data processing. My group and I wrote a system called TritonSort that does large-scale sorting as a proof-of-concept for some of the techniques and technologies for developing more efficient data processing systems. In 2010, we broke two world sorting records with it. This year, we improved our own results in those two records by almost 50% and broke an additional three records.

Needless to say, we’re pretty excited about these results, but there’s still a lot of work left to be done. We’re currently working on gaining experience with and improving the general-purpose version of TritonSort; hopefully we’ll be able to publish some more details on that work sometime in the next few months. If you’re interested in learning more about the project, check out tritonsort.eng.ucsd.edu.

Software I Use Daily: Evernote

Few tools have proven more useful in my day-to-day life than Evernote has. Evernote’s design is pretty simple; you can make notes that can include pictures, sound or documents as attachments, search through your notes, bundle them up into folders or tag them with tags. Notes get synchronized between any device that runs an Evernote client. Any image that gets included in a note also gets passed through OCR so that any words that appear in the image are indexed and searchable as well.

I probably refer to or write a note in Evernote at least half a dozen times per day. Every time I feel like I’m going to need to look up a piece of information more than once, it goes into Evernote. If I’m trying to figure something out, any information I find on that topic goes into Evernote.  As I refine my understanding about something, I’ll turn that raw info dump into something more compact and digestible. The ability to jump back to previous versions of notes is really helpful for this refining phase.

What I use Evernote for most is daily research logs. I really wish that I had started writing daily research logs years ago. They’ve been immensely helpful for organizing my thoughts and preventing me from doing redundant work. They also provide an easy-to-read archive of the work I’ve been doing, which makes preparing for status report meetings a lot easier.

I’ve been really pleased with Evernote overall, especially since they gave their mobile app a much-needed UI redesign. If you’re looking for a place to dump all the stuff that won’t fit in your brain, Evernote is definitely worth a look.

Computer Science Goes to Hollywood

Update: Looks like Matt Welsh posted on the same topic while I was writing this. Check out his perspective, it’s a good read.

I just read an article in the New York Times titled, “Computer Studies Made Cool, on Film and Now on Campus” talking about the recent upswing in enrollment in computer science that, the article claims, is due in part to Hollywood’s favorable portrayal of the tech world in films like The Social Network. They also say that computer science programs are shifting away from a curriculum focused on theory toward one that integrates more applications and implementation in an effort to boost the major’s appeal and keep enrollment trending upward.

The fact that computer science enrollment is on the upswing is good; the US (and the world in general) needs more computer scientists. One missing piece of the puzzle that isn’t explored in great detail in the article, however, is these new students’ motivations for pursuing a degree in computer science. One thing that makes me anxious is the segment of new students who are becoming computer scientists to “become the next Mark Zuckerberg”. Spun one way, this means “create something that millions of people use every day”, which is great. Computer science is most exciting, in my opinion, because it enables you to create great things out of virtually nothing. Spun another way, this means “make billions of dollars and have a movie made about you that’s scored by that guy from Nine Inch Nails”. This is disconcerting. Becoming a computer scientist to become the next Mark Zuckerberg is like playing basketball to become the next Michael Jordan. It will happen to a handful of people, and takes a lot of hard work, dedication and luck, but the vast majority won’t even come close. If you’re not playing basketball mostly for the love of the game, you’re setting yourself up for a lifetime of frustration and disappointment. Same thing goes for CS.

I’m also not convinced that “banishing the perception of the computer scientist as a geek typing code in a basement” is entirely a good idea. I hate to break it to you, but most of computer science is geeks typing code in basements. Well, not necessarily in basements. Facebook was created and continues to be created by geeks typing code in office buildings. Apple was created by geeks typing code (and soldering stuff together) in a garage. Name a technology company whose products you use every day, and it started with a bunch of hackers cranking out code in a dumpy little room somewhere. This is like trying to banish the perception of the blacksmith as a big sweaty guy who spends most of his time in front of a furnace making things out of hot iron – sorry folks, but that’s just the way it is. Blacksmiths don’t just wave their arms and cause awesome swords to appear all day.

That’s not to say that our public image could use a bit of improvement. In the past, computer scientists were portrayed in film and television as a bunchy of pale, asocial man-children with an over-fondness for Mountain Dew and Cheetos. If we’re getting away from that stereotype, then that’s fantastic, but by and large I don’t think we are. I withhold judgment on The Social Network (I haven’t seen it yet, but it’s in my Netflix queue), but most of what I’ve seen has just shifted to portraying us as a bunch of ludicrously wealthy, pale, asocial man-children.

It’s true that the theoretical fundamentals of computer science are hard to make exciting without applications. However, if we swing in the other direction, over-emphasizing application at the expense of fundamentals just to enhance our appeal, we run the risk of over-narrowing the major’s focus and making it less useful and less educational. If that happens, I’m worried about the quality and quantity of computer scientists (who actually like what they do) that we’ll produce. I don’t think it will be a problem at places like UW, USC and Stanford, but the reality is that most people don’t go to those places.

I originally pursued computer science because I liked messing around with computers and I thought I wanted to make video games. If I had taken a major in game design in college, I might be working on the next Madden at Electronic Arts and hating my life. The breadth of my undergraduate education exposed me to areas of the field that I didn’t know existed. Now I sort things really fast every day, and I’m really enjoying myself. And I spend most of my time writing code, although I don’t do it in a basement.

Airfoil, the Whole-House Music Streaming Killer App

iTunes has had the ability to stream music to a remote set of speakers for a while now. At first it was just devices like the Airport Express, but now they’ve struck agreements with a bunch of different companies that make speaker docks and A/V units so that iTunes can stream to them too. Being able to play music out of a remote pair of speakers is great, but there are a few key limitations to AirPlay that have always left me less than sold on the idea.

First, AirPlay is all about iTunes streaming to something. Let’s face it, the only reason most people use iTunes is because it’s the application for synchronizing music to iPods. It works fine as long as your music library isn’t enormous, but it’s far from the most feature-rich music player out there and its support for things like Internet radio hasn’t improved much in 7 years. If you want to stream Pandora or Last.fm to remote speakers, AirPlay  just doesn’t fit the bill.

Second, the device set that recognizes AirPlay is still fairly limited and decidedly non-free. If you want an Airport Express, you’ll be paying $100 for what is essentially a wireless access point with an audio out jack. This has always rubbed me the wrong way. I already have a computer with an audio out jack and a network connection, why can’t I just stream to that? For that matter, why can’t my computers stream audio to each other?

Enter AirFoil.

Attach AirFoil to an application (pretty much any application), and it captures that application’s audio and sends it to one or more sets of speakers. AirFoil can stream to any AirPlay device as well as anything running the companion AirFoil Speakers application. The AirFoil server application runs on OS X or Windows, and there are versions of AirFoil Speakers for OS X, Windows, Linux and iOS (meaning it works on iPhones, iPads and iPod Touches too). The AirFoil server keeps all the audio streams to the various speakers magically in sync. AirFoil uses Bonjour service notification messages to find and advertise speakers, so AirFoil can see any speakers on the local network without the need for configuring anything.

When I want to listen to music from my iTunes library in my living room, I just fire up AirFoil on my desktop in the bedroom and stream through my media center PC in the living room. If I want an additional set of speakers in the kitchen (because hey, why not?) I can hook a pair of speakers to my phone and run AirFoil Speakers on it. This is something that would have cost me hundreds of dollars in additional, purpose-bought hardware to do without software like this. I’m extremely impressed by AirFoil; if any of this sounds remotely intriguing to you, I’d really recommend giving it a try.