Let’s Talk about Hard Drives

Last week, one of my external drives failed, and another indicated that it’s about to die by failing a read and causing my RAID volume to degrade. Neither of these failures were surprising; both drives were well outside of their warranty periods. The way these drives failed and the (sadly ongoing) quest to replace them has brought up a couple of things that I’ll talk about here.

Failed drives means shopping for replacements. When it comes to external hard drives, we seem to be presented with a multitude of choices, none of which are good. Judging by reviews on NewEgg, external consumer-grade hard drives are some combination of:

• Unreliable
• Slow
• Feature-poor
• Plagued with awful customer support

I was surprised at how many of the one- and two-star reviews for hard drives on NewEgg (and virtually everywhere else that sells drives) display some of the same common misconceptions. It’s a sad indicator that as an industry, we still haven’t figured out how to make computers anything less than magical and inscrutable to the average consumer. I’m going to lay out a couple of those misconceptions in the next couple of posts. They’ve doubtlessly been rehashed elsewhere, but these are things that deserve repeating.

The Bathtub Curve

If you were to plot failure rate of hard drives versus time on a graph, the graph would probably look like the blue line in the graph below (thanks, Wikipedia!):

This blue line is what’s referred to in reliability engineering as a bathtub curve, because its shape is evocative of a bathtub. In plain English, the bathtub curve basically says

• Things that are shipped with defects usually fail early.
• Things that work as designed still eventually wear out.
• In the middle, anything can happen, but failure is less likely.

Many one-star NewEgg reviews I came across were some variant of:

Drive fails after X days of use. What a piece of crap. I’m never buying from this company again.

These are people who have unfortunately hit the wrong end of the bathtub curve.

Why does this happen? Well, some of it has to do with manufacturing; with something this intricate there will inevitably be defects, regardless of how much quality assurance you put into it. Some of it might have to do with what happens to the drives during shipping. Sometimes there is actually a systemic defect in a particular model or production batch that goes undetected by quality assurance; this usually results in a class action lawsuit months or years down the road.

The best bet, as I’ve stated here several times in the past, is to never assume that the drive will last another day. I was shocked at the number of times I read a review like this:

Bought this drive and it died three days later. Now 50,000 photos of my cat Muffins are gone. I hate you, Seagate, and so does Muffins.

So you bought this drive, and copied your photos to it, and then … you deleted the originals?! I’ve said it before and I’ll say it again: if there’s only one copy, it is only a matter of time before you lose that data.

Next week: why the replacement for your failed drive is more likely to fail, and why hard drive manufacturers are lying to you.

The Joys of Markdown

OK, so I’m about 7 years late to the party on this one, but man oh man do I love Markdown.

I spend a lot of time dealing with text, but most of the time it’s text designed to be consumed by compilers and interpreters rather than people. When I write people-facing text, it’s almost always in LaTeX. In the process of dealing with these kinds of writing tasks, I’ve become really intolerant of WYSIWYG text editors. They’re just not precise enough.

Evernote’s a notorious culprit here. I tell it to bold a line, it bolds the next blank line too. I change the font, it gets changed back in weird places. Bulleted lists sometimes re-bullet or re-indent themselves in weird ways. It’s irritating. I’ve had similar problems with WordPress’ visual editor.

In short, I’m one of those weird people who doesn’t care what it looks like on-screen while I’m editing it as long as the finished product looks like I want it to look.

Markdown brings the kind of precision I’m used to in LaTeX to the realm of writing HTML. One thing that it loses is all the markup (hence the name, I suppose). For example:

**bold** produces bold text, *italicized* produces italicized text. Similarly terse, readable syntax for headers, links, images, and so on. All of the common stuff that you’re used to when writing HTML, without … well, the HTML.

Apparently Markdown got huge a few years ago, and obsessive programmers like me have integrated it into all sorts of things. There’s a Markdown plugin for WordPress (in which I’m editing this post). Of course, there’s a Markdown mode for Emacs. The one piece of my daily routine that lacks Markdown is Evernote, unfortunately. I think I might be able to get around that with a clever combination of evernote-mode and markdown-mode. If I can figure something out, I’ll post it here.

Power-Using GNU Screen

Ever since I started exploring the more advanced features of GNU Screen, I’ve been using it constantly. For years, I never really thought of it as much more than a way to keep a shell open when I switched machines. I was also turned off by its appropriation of ctrl-A for commands, since it overwrites “jump to beginning of line” in Bash and emacs. You can still get at it, but the shortcut is ctrl-A + A, which never felt anything less than completely awkward.

Thankfully, Keaton sent me a copy of his screenrc which contains, among other things, an escape-sequence remapping command. I changed the control code to ctrl-O, since I couldn’t find any command sequence that conflicted with it (by adding escape ^Oo to my .screenrc).

I’ve posted the rest of my .screenrc file as a GitHub Gist.

Screen’s Hidden Gems

I used to just use screen for attaching and detaching single terminal windows, if I needed to leave a process running in a shell for example. In the past couple of months, I’ve gotten more of a taste of its capabilities, which are numerous. Among its more awesome features:

• Multiple Terminal windows per session that can be attached and detached as a group.
• Horizontal and vertical screen splitting – just like iTerm and Terminator, but screen predates them both by years and allows you to attach and detach terminals from panes at will.
• Notification on activity and inactivity (that is, have screen notify you when a terminal starts or stops producing output).

I honestly wish that I’d started to use screen more heavily a long time ago.

Fixing Window Fragmentation

I typically have a lot of windows open in a screen session at a time. If you close and open windows a lot, you’ll find that the windows’ numbers start getting fragmented. Rather than having windows numbered 1,2,3,4, for example, you’ll have windows numbered 1,3,4,7. Maybe this pegs me as an obsessive, but that’s sort of irritating. Also, I’d like to have the windows sorted by name. Because, you know, I sort of have a thing for sorting.

There’s a Bash script that’s been floating around called screnum that renumbers windows within a session so that they’re sequential, but I could never really get it to work reliably; it had problems with windows that have the same name and was really slow. So I ported it to Python (because in my opinion, Bash scripts get gross once they get more complicated than short lists of straight-line commands), fixed the same-window-name bug, and dramatically sped up the window sorting routine.

I posted the script, screnum.py, to Github. If you’re interested in using it or have ideas for improving it, have at it.

Esoteric Tip #6: Dead Simple Screencasts in OS X

I was looking around for software for doing screencasts so that I could make a short tutorial video for my students. I was a little bewildered by the lack of good, free screencast software for OS X; surely there must be something that can capture audio and video from the screen. Turns out there is – good ol’ trusty QuickTime Player 10.

If you select New Screen Recording from the File menu, it will present you with a window with a prominently-featured “record” button. Simply configure your audio input and hit the button and you’re recording. It’s CPU intensive and doesn’t have a very high framerate, but it certainly worked well enough for my purposes.

Reflections on CS61A

Sometimes I am reminded of how long it’s been since I took my first programming course in college. I came across a post called “SICP is Under Attack” written by an undergrad from the class of 2015 named Vedant Kumar. In this post, he points out that CS61A (the introductory CS course at Berkeley) is switching from Scheme to Python, although they won’t be switching entirely away from the course textbook, Structure and Interpretation of Computer Programs (commonly abbreviated SICP), which (as Mr. Kumar points out) is an incredibly good introductory computer science textbook.

I won’t go into detail on why they’re making the switch. Brian Harvey, the prof who has taught the course for the last 25 years (although he didn’t teach it when I took it), has already written about this at length on his website. I reacted to news of 61A’s transition away from Scheme the same way that I reacted to finding out that the Shakey’s Pizza near my parents’ house where I went for team parties as a kid had been bulldozed and replaced by a Wallgreens. I recognize that things can’t stay the way we remember them forever, but it’s still a little sad. The news did get me reflecting on that course and that time in my life. If this thing had visuals, this is where the flashback ripple effect would kick in.

I took CS61A in the fall of 2003. Before that point, I had very little practical experience with programming (I had futzed around with HyperCard and Object Logo, and done a little bit of PHP programming, but hadn’t programmed anything “real” at that point). I remember there being talk among my fellow incoming freshmen about an entrance exam, in which you’d have to demonstrate your knowledge of this technique called recursion that I’d never heard of before. Almost everyone I met that week didn’t seem that concerned; they’d programmed tons before, or done well on the CS AP exam, or even taken the CS AP exam. I was convinced that I would get destroyed by the entrance exam and be forced to take some sort of remedial course and then everything would be ruined forever. This sort of thing happened to me a lot during that first year; that little voice that keeps saying “These people are way smarter than you and you don’t belong here” just would not shut up.

My friend Naren told me years later that when he met me that first day in 61A, I “looked super angry”. Far from it – I was terrified. It turns out that there was no entrance exam, much to my relief. It also turns out that the course was to be taught in a language that hardly anybody had ever heard of before. Some of the savvier students asked, “Why Scheme? Why not just use Common Lisp?” The point, the profs explained, was that they weren’t teaching you a programming language. They were going to teach you how to think like a computer scientist, and that meant teaching you recursion. Teaching you recursion is easier, they said, if the one of the only constructs the language supports is recursion.

61A was somewhat of a trial by fire. It was a lot of work and it was not by any means easy. At the time, I bitched and moaned about it a great deal. I missed our triple-overtime win against USC studying for a 61A midterm (current undergrads, learn from my mistakes!).However, it introduced me to a wide swatch of the fundamentals of computer programming and computer science – recursion, complexity analysis, debugging, logic languages, dynamic programming, the list goes on. It also helped me figure out that I wasn’t that much better or worse at this whole programming thing than anyone else there, and that there hadn’t been some horrible mistake in the admissions office.

I understand the desire to “modernize” the curriculum, and I am an enormous fan of Python, but for a course like 61A the choice of language doesn’t really matter all that much. The purpose of the course is to teach the fundamentals, and to give you an introduction to what computer science is all about so you can figure out if it’s the right major for you. As long as the course accomplishes those goals, nothing major will have changed. Regardless of what textbook they’re using by default, I think that SICP will remain an important volume for that course in the same way that The Art of Computer Programming is important, not for the language that the examples are written in but for its exceptional quality as a teaching, learning, and reference tool.