Thursday, December 30, 2004

Perforce does Python

At work, we're switching source control systems; we've settled on Perforce. I used Perforce years ago at a now-dead dot-com, but it's been a while, so I started reading the online docs to refresh my memory before the rollout. I was pleasantly surprised to find this in the standard client's command-line options:
-G: Causes all output (and batch input for form commands with (-i) to be formatted as marshalled Python dictionary objects. This is most often used when scripting.

Of course, I'm not the first blogger to write about it, and it looks as if there are a number of people using Perforce with Python scripts.

I've always been impressed with Perforce-the-company--they were pretty cool to deal with when we needed assistance, and they've done some very non-evil things for the programming community (including providing no-cost licenses for open source projects and open-sourcing their own Jam build tool). A little digging reveals they also sponsored PyCon in the past.

I'm glad to be supporting a company like this one.

Wednesday, December 22, 2004

Effbot on Bill Mill on Decorators

Followup: What was even more interesting than Bill Mill's decorator rant was a comment to Bill's post, by the inimitable Effbot:
I'm beginning to realize that the people who are the most enthusiastic about [recent Python additions] do not think in the same Python as I do..... I'm not even sure they think in Python; I suspect they think in something else, and translate that to Python when coding.
If I read him correctly, recent changes to Python seem to be causing fragmentation of the language--not into different ports a la the fragmentation of UNIX, but into different subsets of the language, with each subset preferred by a different subculture of the language. I haven't decided if this is good or bad yet.

I know from reading the PIL source, I don't seem to think in quite the same Python as Effbot (dammit, where are the docstrings?). But from past comp.lang.python posts, I think he and I are closer than most.

Bill Mill on decorators

Bill Mill chimes in on decorators:
Please leave the decorators to other people, and write your code without the sugar. Make the world just a little bit simpler.
I don't go as far as he did, and I suspect it's because we're each seeing a different part of the decorator elephant. It sounds like his reaction is more like my initial reaction (lost to the maelstrom of a PyDS bug) to metaclasses: "Leave metaclasses to the folks who need them."

And my reaction to his comment is the same one that someone made to me: "Who are the 'other people' to whom I should leave them?"

My only, weak answer was, "If you think you need one, you don't. If you know you need one, why are you taking advice from me?" The similarities are making me think there's something significant and common to both cases, but I haven't figured out what it is yet.

Two for the Price of One

The two Python IDEs I've been watching, DrPython and SPE, both got Christmas upgrades. The new DrPython release looks like mostly bugfixes, but the SPE release adds a neat new class diagramming feature.

My earlier characterisation still holds: I like the speed and stability, but dislike the unPythonic coding style and awful GUI layout of DrPython. I like the shiny features, but dislike the instability of SPE. And I'm still using SPE until PyChecker or another feature craps out, at which time I fall back to good old Emacs.

And I'm still fighting the urge to create yet another Python IDE.

Monday, December 20, 2004

Why Debian Sarge Is Late...

Because an issue like this turns into several hundred messages including this, this, and this, and culminates in a message like this.

Friday, December 17, 2004

From Linux to FreeBSD

Goodbye, Linux. Hello, FreeBSD.

About five years ago, when I was building out my home mailserver, I considered a couple of OS platforms. Windows 2000 was right out for resource reasons (the machine was a 90 MHz Pentium). I looked at the Debian, RedHat, and Mandrake distributions of Linux, as well as FreeBSD. Installing FreeBSD yielded a thoroughly broken system--probably my own mistake, but given the short amount of time I had to evaluate operating systems, I ended up settling on Mandrake.

Today I have a cheap, two-year-old laptop that dual-boots Windows XP and Fedora Core. But on this machine, Fedora Core has let me down more with every release:

Fedora Core 1
: It worked pretty well, but not completely. The ACPI support didn't like my laptop's broken ACPI implementation. In fact, the only part of it that worked was the power button (no case lid switch, battery status, fan status, or suspend functionality). To quote the Linux ACPI implementors: "We refuse to be bug-for-bug compatible with the other major implementation [Windows]." In other words, "pound sand, chump; buy a real machine". Well, gee, thanks, guys. I mentally thanked them again every time my laptop suddenly shut down because the battery had run down to nothing. On the other hand, Yum was great, almost as good as Debian's apt-get (although when I installed it on an older machine, it was close to unuseable, so I switched that one to using apt-get).

Fedora Core 2: ACPI still broken, and now the touchpad stopped working. I did the right thing and filed a bug against it with as much info as I would gather. Turns out there was a kernel change that broke the touchpad driver, and no one seemed terribly interested in fixing it. Eventually I found and compiled a third-party driver, which worked around the problem. They also moved the mount point for cd-rom drives, which broken a number of my backup scripts. Yum didn't work out of the box because of the heavy load on the default servers, but a quick Google found some mirrors that fixed that.

Fedora Core 3
: ACPI still broken, touchpad still broken, and now SE Linux doesn't like mod_python. OK, turn off SE Linux. They also moved the mount point for cd-rom drives... again. Re-fix the scripts. It doesn't show the boot menu by default ("why would you want to boot anything else?"), so you have to be quick if you want to dual-boot, or else tweak the boot prompt. The second time I ran yum to apply updates, it broke itself... irrevocably. OK, reinstall, re-fix touchpad, re-fix mod_python, re-fix scripts, re-fix boot prompt... why did I upgrade again?

Actually, I didn't intend to install FC3; last year I got back into Debian for work-related projects (after a several-year hiatus). This summer I had resolved to install Debian when Sarge was released in late September October November December next year.

This week the token OS/X geek at work was going on about the *BSD systems, and recommended NetBSD for my laptop. I dug around a bit, downloaded both NetBSD and FreeBSD, and decided to give FreeBSD a whirl.

It just works.

Right out of the box, ACPI works, the touchpad works, the X Windows system works, and the CD burner works--all without lifting a finger to configure them. I only found two minor problems so far: the sound driver wasn't loaded by default, and ACPI's level S3 suspend-to-RAM doesn't bring the display back. But unlike that of Linux, FreeBSD's ACPI documentation goes into great detail on how to diagnose the problem, how to report the problem, and even how to attempt to write the needed patch yourself.

I've been tinkering with Linux kernels for many years, and I'm right at home customizing them, but from the few hours I've played with FreeBSD, the kernel compilation and install process is cleaner and simpler, and more understandable when it goes wrong. The FreeBSD documentation is also much more consistent than that of Fedora Core.

So what are the downsides? Not many, so far. I was expecting a little more software on an initial full install (there's only one installation CD), the ports and packages seem to lag further behind the upstream revisions than they do in Fedora Core, and starting large apps like mozilla or emacs seems to take longer than on FC3 (possibly related to FC3's prelinking feature). But I can deal with those.

I'm going to be decommissioning that old "Little Pentium-90 That Could" mailserver later this winter. And it looks like the replacement will be running FreeBSD.

[Update: mea culpa. The long load times for mozilla and emacs were caused by misconfigured DNS--I installed while off the network. After I got the network settings squared away, I don't see a visible difference in startup speed.]

Tuesday, December 07, 2004

Going Back to Flint Axes

I had a little project to do last night: a small GUI application, only needed to run on Windows, with just a few bells and whistles--nothing too complicated. Just out of curiosity, I decided to go to the Wayback Machine and code it in straight, honest-to-Petzold Win32 and C. Not C++, not Microsoft's C-in-C++'s-clothing MFC, but real C, with slash-star comments and all.

Now, just for background, for a task like this, the nearest tool in my toolbox is usually Python with wxWidgets. The second tool in the toolbox would be either C++ with MFC (if I were really in a hurry) or C++ using ATL/WTL (if it needed to be clean and presentable). Third choice would be Java, but only if I needed to incorporate something that was already written in Java. I haven't done any straight C in about 8 years.

But C and Windows (3.1, actually) are where I cut my teeth, and recently we've been having some discussions at work about "the way things work" under the covers of the Windows CE operating system we're using. The constraints of CE (32 processes, 32MB address space, small display) have put me in mind of my first programming gig (Windows 3.1, 8MB of memory on a good day, 800x600x256 display, reboot three times a day for luck).

So back to C I went, just as an exercise to see how much C and Win32 I'd forgotten. And I was surprised by what I missed and didn't miss. What I missed most:
  1. Declaration of variables at point of use (C++), rather than at the top of a block (C), or not at all (Python). This was the big one. I've become spoiled.
  2. Default arguments (C++, Python) and function overloading (C++). This one really surprised me... I didn't realized I used them that much.
  3. Typed allocation and deallocation (C++). Casting malloc() just looks wrong now.
  4. Reasonably rich string handling (C++/STL or Python). string.split(), my kingdom for string.split()!
  5. Support for Windows Common Controls: I forgot how much inane drudgery is involved in creating a simple new-style toolbar and rich edit control when you have to do it by hand.
  6. Productivity, productivity, productivity. I put about 9 hours into the project, and I'm about 75% done. Most of the time was spent remembering all the minutiae of the Win32 API (that's right--you have to manually call LoadLibrary() on the RichEdit DLL, or else the OS doesn't realize it exists...). I would probably have been done in 2 hours had I used C++/MFC, or about 4 with either C++/ATL or Python/wxWidgets. Yes, usually Python is far more productive than using C++ with either toolkit, but what I was doing played right into the strengths of both MFC and Microsoft's tool support. On the other hand, it would have been about the same amount of time in Java, just because I've decided that next time I have to do a client in Java, I'm going to learn SWT.
But I was more surprised by what I didn't miss:
  1. Classes. Only once in the whole process did I really miss wrapping up data and code together.
  2. A message handling framework. Microsoft hasn't updated their message cracker support since 1999, according to the header file comments, but I think the resulting code is actually clearer than that of most object-oriented GUI frameworks.
  3. Safety features (type checking or garbage collection). I created only two bugs that would have been caught by either of these techniques--one was because I forgot that strncpy() doesn't always NULL-terminate strings; the other was the predictable C memory leak. It helped that I enabled maximum compiler warnings and used forward declarations, though.
And the biggest surprise? The #1 thing I missed (declaration of variables at point of use) forced me to write better-factored code. I broke things up into shorter, clearer functions just so I wouldn't have to look at massive declaration lists. That kept the functions to a well-defined, single purpose, which led to self-descriptive names and access to only the data they really needed. Nice.

Moreover, the C code I wrote last night was far and away better than the C code I wrote 8 years ago--all that exposure to industrial-strength, server-side C++, Java, and Python (as well as bits of experimenting with Lisp) has helped.

But one thing I can't deny: ending up with a 30k, statically-linked executable, dependent on no DLLs other than those of the OS, made me smile.

Friday, December 03, 2004

Back in the Day

I don't know why, but I've always loved reading old computer magazines and advertisements (old being ~10 years before whatever year it happens to be). One of my "one of these days" projects is to create a website dedicated to the unbridled optimism, fallen prophecies, and educated harrumphing of the "golden age" of personal computing (whenever I figure out when that is, was, or will be).

Great example, from (1990):
Toshiba has developed a digital still camera which can store pictures to memory card. The MC200 can handle up to 400,000 pixels creating ultra-clear images. Pictures taken can be stored on a PC. It costs a trifling £10000.
Two years ago, when I bought a Dell machine for home, I got a 1-megapixel digital camera as a free perk. I gave it to my three-year-old to play with.

Wednesday, December 01, 2004

"Aftermarket Pipes"

After the blog was up on for about three months, Googling for "aftermarket pipes" came up with my blog rather than the more-appropriate bike shops or manufacturer sites.

Enough people have asked me, "Why 'aftermarket pipes'?" that I have to have some kind of answer. Last year, when I was setting up the first incarnation of this blog (from my cubicle where I worked at the time), I got to the "enter the name of your blog" question on PyDS. I sat there for about ten minutes, unable to think of anything interesting but not twee. Two coworkers were discussing their bikes across the cube wall, and random phrases were seeping through my headphones:
"..wugga wugga modifications wugga wugga not too expensive wugga wugga aftermarket pipes wugga wugga..."
At the time, I was heavily into wxPython, and I was writing a few little classes to make certain things less cumbersome (not too much unlike what Hans Nowak did with Wax), and it struck me: my classes were to wxPython as aftermarket pipes were to bikes. So, "Aftermarket Pipes" it was.

And is.

Another beginning for the Pipes

After two unexplainable data losses, a handful of strange errors, and a week of site downtime, I'm moving Aftermarket Pipes to blogspot. I like the PyCS and PyDS folks, and the system they're creating, but I'll take stability for right now.

Now, if I can figure out a nice way to import the old blog entries, I'll be happy.

Thursday, November 11, 2004

Steam-driven gaming

[It's a shame Blogger doesn't do categories. Pretend this entry is in a "Gaming" category, and that you're not seeing it if you're just interested in the Python side of the 'Pipes. In any event, this is reposted from a backup I had from before I move Apipes to blogspot.]

I have a habit of coming up with an idea, writing it down, forgetting about it, and then reading that someone else has had the same idea and developed it into something really cool. It's happened again.

Early this year, I was inspired (by reading The Victorian Internet, The Guns of the South, and The CSS Hunley in quick succession, while at the same time re-playing MechCommander 2), to create a game or story setting for a "steampunk", Victorian- or slightly post-Victorian alternate history.

I started with steam-driven, clockwork tanks (called "armored coaches" or "steamcoaches") and U-boats, but it grew into steam-driven mechs, with Tesla and Edison as scientific advisors on opposite sides of a just-post-Victorian war.

Two possible scenarios for that war developed: a corporate war or a second US Civil war. The corp war (aka "The Steam Wars", or TSW) occured around 1915 between a united Westinghouse/Union Pacific conglomerate (with Edison as the technical advisor) and a TELM/Southern Railway/Central Pacific conglomerate jointly headed by Tesla himself and the head of the Southern Pacific railroad. The dates had to be fudged a bit, as I had a hard time nailing down which of the SP's executives would have had enough guts to pull it off--maybe Stanford.

The other option was to have the South win succession (via British and/or French assistance), then make a "land grab" for the Western states with the help of Tesla and the Western Railroads around 1918-ish (again, I cheated by making TELM a successful Colorado
company, and making the post-civil war railroad situation in the South less dismal that it was historically), thus causing The Second US Civil War, aka T2CW or, in the words of one rabid Northern journalist of the time, "The War of Southern Aggression".

In both cases, I had to remove the assassination of the Archduke of Austria-Hungary from the record books. I wanted the European situation to be simmering but not erupting, so as to free up US energies for internal conflict. I toyed with the idea of making the corporate war occur before 1914, and having its results feed into World War I, but I don't really know enough about European-American politics of that time to stitch it together, and the dates had to be pushed around enough as it was.

The historical (i.e., real) webs of international politics were really interesting but somewhat hard to get a handle on (particularly in the second case), outdone only by the web of inter-corporate controls, holdings, rivalries, etc. between the railroads during the same time. It made for some fun research, including a surprisingly large amount of data and bibliographic pointers from Wikipedia.

Anyway, I got pretty far with it in notes, even writing some vignettes, but it Mr. Blamire has gone much further. He's gone a completely different route, though:
  • In my milieu, Canada was a tangential player in the T2CW setting and unmentioned in the TSW setting; it looks like a Prussian invasion of Canada (?!) is his inciting incident.
  • Blamire's "mechs" are anthropomorphic. Interesting, but I haven't seen much to make me think that a late 19th-century inventor would take that direction, other than a rather fanciful web page I ran across while researching. Mine were of a more antlike design.
  • Blamire's "mechs" are the dominant force during his time period. Mine are just being designed and prototyped during the story arc (and thus form a major part of the story). The real combat is happening between "steamcoaches", horse cavalry (losing badly to the former), and infantry, with a few brave souls experimenting with airships.
  • Airplanes aren't a real factor in either of my settings, because Curtiss is involved in mech design for Edison. The Wrights were technically successful but haven't garnered the amount of attention they did in the Real World--from my research, it appears that much of their fame grew out of the Curtiss rivalry.
Aren't alternate histories fun?

Thursday, August 26, 2004

Can you fly that thing? Not yet...

[republished 1 Dec 2004 from]

Every once in a while I notice something that reminds me of a scene from a movie. Generally, if it's a movie that has made me think a bit or consider things in a new light (Fight Club, The Matrix, and Pulp Fiction are a few), it means there's something worth chewing on in the situation.

I'm no Matrix fanboy, but I had an experience that put me in mind of one scene. Neo asks Trinity if she can fly a helicopter. She answers, "not yet," calls up her "operator", and he transfer the appropriate skills to her virtual persona.

Today I wanted to check out something using the ctypes module, so I popped open an interpreter window and typed import ctypes. No dice--I didn't have it installed on my work machine. One quick download and install later, I uparrow+entered in the same interpreter (no restart here!),and had ctypes available. Not bad.

But this could get better. PyPI is gaining traction--it's not CPAN yet, but it seems to be turning a corner. Thinking in terms of yum and apt-get, what would be very cool is to turn this:

>>> import blahdb
Traceback (most recent call last):
File "", line 1, in ?
ImportError: No module named blahdb
[switch to web browser, hit Google, find the right page, find the download
for my version of Python, download, find installer, run installer,
switch back to console]
>>> import blahdb

into this:

>>> import blahdb
Traceback (most recent call last):
File "", line 1, in ?
ImportError: No module named blahdb
>>> install(blahdb)
PyPI: found blahdb (Library for accessing Blah databases)
PyPI: version 1.6 for Python 2.3
PyPI: <>: signature OK
PyPI: installed
>>> import blahdb

or even this:

>>> from __future___ import magic_imports
>>> import blahdb

Monday, August 16, 2004

The Contractual Obligation Post

It appears that, as it has "Python" in the tagline, this blog is legally required to comment on the current decorator controversy in the Python community.

I think Hans Nowak has the best and most thorough explanation of what the new decorator does (at least in its current implementation), and exploration of how you can use it in interesting ways. Certainly, it's been the explanation that fits my brain best.

Everything I've seen so far, though, has been "this is how it works", plus some "this is cool because it makes classmethod less of a special case". And it lets you extend the language, which is very good, in that Python-as-a-larval-Lisp way.

The next obvious question to me is, which interesting applications of decorators are going to be provided in the standard library? There are a number of "good ideas" (function attributes, synchronized, docstring replacement, PyProtocols implementation, magical hook insertion like atexit and sys.settrace, etc.)

My biggest concern: right now, everyone is caught up in the syntax for the feature, and to a lesser extent, the implementation corner cases. I haven't really seen anyone (and by "anyone" I mean the BDFL) talking about what decorators will exist as part of the standard Python distribution.

I fear that decorators will become Python's equivalent of the "everyone's own favorite homegrown string library" phenomenon we saw in the early days of C++. Everyone will have their own favorite and flawed implementation of @synchronized.