Wednesday, June 14, 2006

Competency and Layers

Larry Osterman has yet another great network programming post on his blog. To sum up, he declares his second rule of "making things go fast on the network":

You can't design your application protocol in a vacuum. You need to understand how the layers below your application work before you deploy it.

An excellent rule. Actually, I've often heard (and used) a more general form:

You can't be competent doing computer work at level N unless you have a good grasp of level N-1.

Programming is all about abstractions, and we as programmers are fond of thinking that our abstractions mean you "don't need to know" what's under the covers. But abstractions aren't perfect, and if you don't know what's under your current level of abstraction, then you're simply not competent.

For example, if you want to be a good MFC programmer, you need to have a decent grasp of Win32 API fundamentals. If you want to work in Python, you don't need to be a Python core hacker, but you'd better know enough about the implementation to know why, for example, repeated string concatenation is slow. And if you're using a object-relational mapper over top of a relational database, you still need to know your way around SQL.

I first heard this from Dr. Ralph Droms, one of my professors at Bucknell (who also invented DHCP). If I recall correctly, he was quoting one of the "elder statesmen" of computer science (Dijkstra, maybe?), but I can't recall just who it was.


Anonymous said...

JoelOnSoftware has commented on the issue of Leaky Abstractions.

Tim Lesher said...

You're absolutely correct; in fact, I thought I had linkified "aren't perfect" to point to Joel's essay, but I missed it.


Anonymous said...

Interestingly, in Cpython 2.4 they've optimized string concatenation when you use +=. Of course, that still points to the fact that you need to know how python is run internally (the += operations, in general, tend to be different).

I still usually resort to ''.join semantics unless there are only a few concatenations (in which case ''.join is actually slower).

Anonymous said...

I once looked into the string concatentation optimization in some detail, and it turns out that you need a fair amount of things to work out just right before any optimization kicks in. (One starting requirement is that the target string needs to already be longer than about 240 characters.)

The gory details are here for the interested.

Anonymous said...

Oops. My apology for the previous anonymous comment; somehow the setting for anonymous versus non-anonymous flipped partway through the preview process without me noticing.

(How interesting. Blogger seems to flip from 'Other' to 'anonymous' every time I preview, or at least every time I preview with JavaScript off.)