tag:blogger.com,1999:blog-94104492024-03-07T00:48:42.060-05:00Aftermarket PipesProgramming, Python, games, et al.Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.comBlogger74125tag:blogger.com,1999:blog-9410449.post-53064024356278501562018-10-16T08:19:00.001-04:002018-10-16T08:41:31.591-04:00RIP Paul Allen.<span style="background-color: white; font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif; font-size: 14px; white-space: pre-wrap;"><b>"So, what did I just buy?"</b></span><br />
<br style="background-color: white; font-family: roboto, robotodraft, helvetica, arial, sans-serif; font-size: 14px; white-space: pre-wrap;" />
<span style="background-color: white; font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif; font-size: 14px; white-space: pre-wrap;">That was my first impression of Paul Allen. I was a 20-year-old who had just gone from intern to tech lead when Paul bought the startup I was working at. The other programmers had departed, the founder cashed out to become an artist, and I was left to run the project--the first software video editor on Windows (before Premiere).</span><br />
<br style="background-color: white; font-family: roboto, robotodraft, helvetica, arial, sans-serif; font-size: 14px; white-space: pre-wrap;" />
<span style="background-color: white; font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif; font-size: 14px; white-space: pre-wrap;">He had called me into his office for a demo. I fumbled through, avoiding the parts that I knew tended to crash. He asked a lot of deep, technical questions, most of which I didn't have a great answer for (intern here!), but he did it with a knowing kindness, a kind of "older brother who knows a lot".</span><br />
<br style="background-color: white; font-family: roboto, robotodraft, helvetica, arial, sans-serif; font-size: 14px; white-space: pre-wrap;" />
<span style="background-color: white; font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif; font-size: 14px; white-space: pre-wrap;">Working for Asymetrix, I got to do a lot of things that were above my pay grade (and honestly above my ability). "I'm investing in this company--go and see if their tech is real." (Ok.) "Go to this CD pressing plant--see if they're actually capable of shipping ToolBook." (Umm. wha?)</span><br />
<br style="background-color: white; font-family: roboto, robotodraft, helvetica, arial, sans-serif; font-size: 14px; white-space: pre-wrap;" />
<span style="background-color: white; font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif; font-size: 14px; white-space: pre-wrap;">Given that he was running half a dozen companies and a basketball team at the time, my face time with him was limited. It was usually "Paul wants to look into... can you do that?" But when I ran into him casually at work, he always had real questions about the projects that revealed how closely he tracked things behind the scenes.</span><br />
<br style="-webkit-tap-highlight-color: transparent; background-color: white; font-family: Roboto, RobotoDraft, Helvetica, Arial, sans-serif; font-size: 14px; white-space: pre-wrap;" />
<span style="background-color: white;"><span style="font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif;"><span style="font-size: 14px; white-space: pre-wrap;">When I told my boss I was resigning to move back to Pennsylvania (my fiancée didn't like Seattle), I received an interesting "counter": Paul had a stake in Cardinal, a modem manufacturer in Lancaster. I could work for them (on paper) while still working on the video editor for Asymetrix. I took them up on it immediately. (I only ever set foot in the place to sign the paperwork--we jokingly called it "employee laundering.") I don't know how much direct influence he had on that arrangement, but it seems like the kind of thing he'd have done.</span></span></span><br />
<br style="background-color: white; font-family: roboto, robotodraft, helvetica, arial, sans-serif; font-size: 14px; white-space: pre-wrap;" />
<span style="background-color: white; font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif; font-size: 14px; white-space: pre-wrap;">I never saw him after that, but I think we could do with more people like Paul in tech. I'll miss him.</span><br />
<span style="background-color: white; color: rgba(0 , 0 , 0 , 0.87); font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif; font-size: 14px; white-space: pre-wrap;">
</span><img src="https://image.oregonlive.com/home/olive-media/width600/img/trending/photo/2018/02/08/1988-press-photo-paul-allen-owner-portland-trailblazers-bellevue-washington-49a9326d814e7b5a.jpg" /><br />
<span style="background-color: white; color: rgba(0 , 0 , 0 , 0.87); font-family: "roboto" , "robotodraft" , "helvetica" , "arial" , sans-serif; font-size: 14px; white-space: pre-wrap;"><br /></span>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-89903590760324192172017-10-11T12:09:00.002-04:002017-10-11T12:29:57.766-04:00This is why we don't put changelogs in source files any more.<div id="body:c142" jsname="uwfEUd" style="-webkit-tap-highlight-color: transparent; background-color: white; color: rgba(0, 0, 0, 0.87); font-family: Roboto, RobotoDraft, Helvetica, Arial, sans-serif;">
<div class="ELUvyf" style="-webkit-tap-highlight-color: transparent; font-size: 14px; line-height: 20px; margin: 0px 16px 16px; word-wrap: break-word;">
<div class="fu5e3b sMVRZe" data-th="180" jsname="ddUTqe" style="-webkit-tap-highlight-color: transparent; overflow: hidden;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEdEBHQytbz4z7gkyCmLXV_T30-thJR1ckdHLv7l5i3tQ7s_twBrRKD4cYQHxrH87QHF8ipG8hNww9Qy0T9Lm60HiyBSs23FxuTvqAluCG9AGHeDgwpRQOBh1Uzq5fZPQlf-G3/s1600/pianokey.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="360" data-original-width="640" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEdEBHQytbz4z7gkyCmLXV_T30-thJR1ckdHLv7l5i3tQ7s_twBrRKD4cYQHxrH87QHF8ipG8hNww9Qy0T9Lm60HiyBSs23FxuTvqAluCG9AGHeDgwpRQOBh1Uzq5fZPQlf-G3/s320/pianokey.jpg" width="320" /></a></div>
<div class="wftCae" dir="ltr" jsname="EjRJtf" style="-webkit-tap-highlight-color: transparent; white-space: pre-wrap;">
<a href="https://lh3.googleusercontent.com/-_AlattOPBMQ/Wd5Ai5Ru5eI/AAAAAAAAG1I/uNINnSGkmJYxi-oBRgBuB35xNcKR9nyYgCJoC/w530-h298-n-rw/pianokey.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /></a>
This is a piano key. It belongs to a piano that we inherited from my sister-in-law, who inherited it from the school where she worked, who inherited it from a church sometime in the 1940s. We think it's about 90 years old.<br />
The previous owners had the tuner sign and date this key when they maintained the piano. This is convenient, because you already have the piano disassembled, and you're looking right at the key anyway, so might as well. And then when you've got the piano disassembled because you're maintaining it, and you're looking at the key, you can see when you had it maintained last.<br />
However, the time when you really need to know "when did I get the piano maintained" isn't when you have the piano disassembled: it's when you're thinking about getting the piano maintained again. The real place you want it is in a calendar or a journal, outside the piano.<br />
Likewise, you generally care about the change history of your source when you're looking at the source control tool, trying to understand how it changed over time--not when you're down in the bowels of the code. <br />
I mean, it's neat to have something to read while you have the piano disassembled, but that's not really the best place to store the data.</div>
</div>
</div>
</div>
<div class="ELUvyf" style="-webkit-tap-highlight-color: transparent; background-color: white; color: rgba(0, 0, 0, 0.87); font-family: Roboto, RobotoDraft, Helvetica, Arial, sans-serif; font-size: 14px; line-height: 20px; margin: 0px 16px 16px; word-wrap: break-word;">
</div>
<div jsname="MTOxpb" style="-webkit-tap-highlight-color: transparent; background-color: white; color: rgba(0, 0, 0, 0.87); font-family: Roboto, RobotoDraft, Helvetica, Arial, sans-serif;">
<div jsaction="rcuQ6b:QNozjc;" jscontroller="T4Un2e" style="-webkit-tap-highlight-color: transparent;">
<div class="A2zxfe" id="c140" jsaction="click:KjsqPd;H5bEoe:KjsqPd;" jscontroller="J3oyWd" jsdata="LfLzDd;https://plus.google.com/photos/111632697325416564882/albums/6475684276173047729/6475684282440279522?authkey=CKCx5r376p2nowE;$333" jslog="10942; track:impression,click" jsmodel="bTCmc" style="-webkit-tap-highlight-color: transparent; cursor: pointer; position: relative;">
<div class="is2Add" style="-webkit-tap-highlight-color: transparent; margin: auto; position: relative;">
<div class="pQcgke" id="c141" style="-webkit-tap-highlight-color: transparent;">
<div class="E68jgf" style="-webkit-tap-highlight-color: transparent; height: 0px; overflow: hidden; padding-top: 261.359px; position: relative; width: 464.812px;">
</div>
</div>
</div>
</div>
</div>
</div>
Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-82379530944054232782012-05-16T07:26:00.000-04:002012-05-16T07:26:47.404-04:00It's probably been done.I started programming in junior high, writing <a href="http://en.wikipedia.org/wiki/Commodore_BASIC">BASIC </a>code on a <a href="http://en.wikipedia.org/wiki/Commodore_128">Commodore 128</a>. At the time the only learning resource I had was the owner's manual and some issues of <a href="http://en.wikipedia.org/wiki/COMPUTE!%27s_Gazette">Compute!'s Gazette</a>, so I was almost completely self-taught. I don't think I had actually heard the words "computer science".<br />
<br />
If you've never programmed in BASIC of that era, understand that:<br />
<br />
<ol>
<li>There were no "functions"; a program consisted of a list of steps, executed linearly, with some control structures sprinkled in and plenty of GOTO jumps</li>
<li>Variables were all global</li>
<li>Variable names were limited to two letters</li>
</ol>
<div>
There was a limited bit of "subroutine" support: GOSUB was like a GOTO that remembered where it came from, and a RETURN would start executing from the next line after the GOSUB. But since all variables were global, subroutines couldn't be reentrant.</div>
<div>
<br /></div>
<div>
So I got the clever idea that instead of using variables in a GOSUB routine, you could use arrays, and then have a variable that kept track of how many times you'd entered the subroutine. Sure, you had to define the maximum number of times you expected to GOSUB before you started returning, but at least you didn't have to remember whether or not the variable you were about to mutate was going to munge a previous call.</div>
<div>
<br /></div>
<div>
It was <i>brilliant</i>.</div>
<div>
<br /></div>
<div>
And when I got to college I learned that I'd "invented" the call stack and recursive functions. Rather a let-down, that.</div>
<div>
<br /></div>
<div>
So, grab your 8502 and load up a JMP 2011, when I wrote a blog post on "Stages of Competency":</div>
<blockquote class="tr_bq">
<span style="background-color: white; color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20px; text-align: left;">After doing "this programmer thing" for a few years now, I've noticed a pattern in how I acquire skills and techniques. It's surprisingly consistent, and consists of these stages:</span></blockquote>
I thought it was pretty remarkable how consistently I saw the progression from "awareness" to "familiarity" to "functional understanding" to "understanding" to "competence" in myself and in others. <br />
<br />
Pretty insightful, huh?<br />
<br />
Turns out that pattern is a simplified subset of an educational classification system called <a href="http://en.wikipedia.org/wiki/Bloom's_Taxonomy">Bloom's Taxonomy</a>.<br />
<br />
And it was first proposed in 1956.<br />
<br />
So, yeah, that insight you thought you just had? It's probably been done.<br />
<blockquote class="tr_bq">
</blockquote>
<div>
<br /></div>
<div>
<br /></div>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com2tag:blogger.com,1999:blog-9410449.post-2371718073150720482011-11-11T08:38:00.001-05:002011-11-11T08:57:37.080-05:003 Simple Rules That Will Make You a REAL Superstar Developer<br />
In my experience there are two kinds of "rock star" software developers. There's the Neil Peart rock star developer, who combines a natural blessing of talent and intelligence with a relentless work ethic and humble attitude, and over time becomes the developer that people not only want to hire, but <i>want to be</i>. And then there's the "prima donna" rock star developer, who combines a modicum of raw talent with sheer attitude and self-promotion into the programming equivalent of a hotel-room-trashing, sex-and-drugs-and-rock-and-roll tabloid icon (I won't name an equivalent musician--use your favorite example).<br />
<br />
Last year a tweet by Zed Shaw pointed me to a brilliant piece of satire called <a href="http://coderoom.wordpress.com/2010/01/28/3-simple-rules-that-will-make-you-a-superstar-developer/">3 Simple Rules That Will Make You a 'Superstar' Developer</a> that gave three simple rules and two deeper principles for becoming a hard-living, Type 2 rock star programmer. It's remarkably concise and accurate. But what struck me is how close the three rules and two principles are to rules and principles for becoming a real Professor on the Programming Drums.<br />
<br />
<b>"Prima Donna" Rule 1: Write lots of code.</b><br />
<br />
Have to fix a small bug in an area someone else has written? Don't waste time trying to understand it or their motivations for writing it that way. Just rewrite the lot as you think it ought to work. Call it refactoring if anyone asks.<br />
<br />
<b>"Neil Peart" Rule 1: Read lots of code.</b><br />
<br />
You will spend more of your career reading code than writing code. Learn how to do it well. That means doing it a lot. Read code even when you don't absolutely have to, and understand it deeply even when you think a shallow once-over will tell you all you need to know. Have a spare half hour? Read the last couple checkins from other people on the team, even if you don't need to. You will learn more about the system faster, you might find issues earlier, and you will probably learn something or see a technique you didn't know about. Have a spare afternoon? Find an open source project and start reading code. Copiously reading both awful code and good code will help hone your internal sense of the difference.<br />
<br />
<b>Prima Donna Rule 2: Write your code quickly. </b><br />
<br />
Touch lots of files, and include every one of them in the ChangeLog. Don't worry about accidentally introducing hard-to-find bugs; they'll actually help you later on, as long as they're actually hard to find. Avoid introducing trivial bugs.<br />
<br />
<b>Neil Peart Rule 2: Finish your code quickly. </b><br />
<br />
"Done" is a Boolean state, and work isn't done until you would be surprised to have to revisit it again in a few weeks. Minimize your personal work-in-progress.<br />
<br />
Don't let 90%-finished tasks rot outside source control in a local directory, because you will forget the details. Don't check something in, thinking "this will do for now; we'll get to hardening it later", because you *will* forget the details. And it's not finished until it's tested to your team's standards, documented to your team's standards, and understood well enough that if you get hit by a bus on your way home tonight, someone else can take your place.<br />
<br />
<b>Prima Donna Rule 3: Don't take time to document your code.</b><br />
<br />
And don't add little comments explaining potential pitfalls in modifying some of the less clear statements you've introduced. You don't need them--you <i>wrote </i>the code.<br />
<br />
<b>Neil Peart Rule 3: Document your code with a single-minded purpose.</b><br />
<br />
Obvious code with only as much documentation as is needed is the Holy Grail. Undocumented and unclear code is as bad, but <i>over-documented code can be worse</i> because it becomes a crutch ("so what if the code is ugly: that's why I commented it!").<br />
<br />
You already know that when you write code, you put your reputation on the line that it is correct. But when you <i>document</i> code, you put your reputation on the line that not only is it correct and sufficient now, but it<br />
will be correct and sufficient when someone looks at it down the road. So <i>minimize your risk of shame</i>. Boilerplate comments, rambling exposition, comments that duplicate the code, and commented-out code<br />
left in "just in case" are signs of laziness hiding behind the mantra of "comments are good; more must be better!"<br />
<br />
<b>Behold, the Underlying Principles</b><br />
<br />
Strikingly, each set of rules emerge from one technical principle and one social principle, and <i>each set of principles is a mirror image of the other</i>:<br />
<br />
<b>The Prima Donna Technical Principle: You're 10x as productive when you're working on code you wrote as on code you didn't write. </b><br />
<br />
So, maximize your opportunity to work in code that you wrote, no matter the consequences.<br />
<br />
<b>The Neil Peart Technical Principle: You're 10x as productive when you have full awareness and mastery of your environment. </b><br />
<br />
Yes, you can achieve that by always working on your own code. That will mean being so prolific that all the problems you're fixing are your own creations.<br />
<br />
Or you can achieve it by having a deep and total understanding of as much of your team's project and tool set as possible. And the cleaner a design or a development process is, the more of it you can fit in your head at once.<br />
<b><br /></b><br />
<b>The Prima Donna Social Principle<span style="font-family: inherit;">: </span></b><span style="background-color: white; color: #222222; line-height: 19px;"><b><span style="font-family: inherit;">You win The Game by improving your reputation to superstar guru levels.</span></b></span><br />
<br />
Your programming ability is judged by how much code you write, how quickly you finish features and fix critical bugs and how often your insights are necessary to solve problems.<br />
<br />
<b>The Neil Peart Social Principle: Optimize your life for <i>value</i>, not <i>perceived ability</i>. </b><br />
<br />
Your value to your project and your team is only partially related to your programming ability (perceived or real). It's <i>directly</i> proportional to your ability to add value to your project and your team. The more deeply you understand your project, your team, your code, and your tools, the more value you can add. Conversely, any technical debt you create will be repaid either by you or those who follow you, and servicing that debt reduces the mental "capital" you have available toadd value.<br />
<br />
In many ways, the "prima donna" and "Neil Peart" principles differ only subtly. Maybe that's why it's so easy to find yourself on one path, when you really think you're on the other.<br />
<div>
<br /></div>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com1tag:blogger.com,1999:blog-9410449.post-91338519148736458682011-09-06T12:29:00.013-04:002011-09-19T15:21:42.370-04:00Case Study: Python as Secret Weapon for C++ Windows ProgrammingOne of my favorite features of Python is its interactive shell. If you want to try something, you type in the code and try it immediately. For someone whose first coding environment was the equally-immediate Applesoft Basic, this is just as natural. But if your introduction to programming was C, C++, or Java, the benefits might not be apparent, especially if you're trying to do exploratory coding in one of those languages.<br /><br />So I'm going to walk through a recent experience as a case study.<br /><br /><font class="Apple-style-span" size="5"><b>The Problem</b></font><br /><br />At work we develop a Windows program that talks to certain devices via serial cables. Those devices also come in wireless Bluetooth flavors, and we connect to them via a "virtual serial port". To the program running, it looks as if the Bluetooth device is plugged into a real serial port, because all of the wireless connectivity is abstracted away by Windows. These devices are unidirectional--they transmit data to the Windows program, which passively reads it.<br /><br />If you power off and restart one of these wired devices, it will start chattering away at the Windows program with hardly a hiccup--our program never even sees a disconnect. However, we noticed that this didn't happen with the Bluetooth devices: powering down one of those requires the Windows app to reconnect. But the Windows app didn't even seem to get any notification that the device disconnected. So how do you solve this chicken and egg problem?<div><font class="Apple-style-span" size="5"><b><br /></b></font></div><div><font class="Apple-style-span" size="5"><b>Research</b></font></div><div><br /></div><div>It had been a while since I'd done any actual hardware serial programming, so I started with some documentation, and remembered that the <a href="http://en.wikipedia.org/wiki/RS-232">RS-232</a> serial spec included a line called DCD, or Data Carrier Detect (also called RLSD, for Receive Line Signal Detect). Back in the dinosaur days, this signal meant that your modem was connected to the remote modem, and was able to start communicating back and forth.</div><div><br /></div><div>Sure enough, a <a href="http://www.google.com/search?q=site%3Amsdn.microsoft.com+RLSD">search</a> brought up the right bit of <a href="http://msdn.microsoft.com/en-us/library/aa363193(v=vs.85).aspx">Win32 API documentation</a>, which told me how to detect an RLSD change on a physical serial port using the SetCommMask and WaitCommEvent calls. The question now became "does the Microsoft virtual serial port for Bluetooth support RLSD"?</div><div><br /></div><div><font class="Apple-style-span" size="5"><b>Exploration</b></font></div><div><br /></div><div>At this point I could have started up Visual Studio, created a scratch project, written a couple dozen lines of C++ code, compiled and linked, fixed the compile errors, compiled and linked again, run the program, fixed the inevitable errors that the compiler didn't catch, and then had my answer.</div><div><br /></div><div>But I'm too impatient to wait for Visual Studio to start up, too lazy to write C++ when I don't have to, and I have the hubris to think I can come up with something better than the obvious solution. <a href="http://en.wikipedia.org/wiki/Larry_Wall#Virtues_of_a_programmer">Programmers are funny like that</a>.</div><div><br /></div><div>So instead, I cranked up DreamPie.</div><div><br /></div><div><font class="Apple-style-span" size="4"><b>Secret Weapon #1: DreamPie</b></font></div><div><font class="Apple-style-span"><br /></font></div><div><font class="Apple-style-span"><font class="Apple-style-span" style="color: rgb(0, 0, 238); font-size: 16px; -webkit-text-decorations-in-effect: underline; "><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRJmY7HNLyiFmVKSG6p6OyRW557o6SUIlsxvCnrSxGUUcJUCGtuR5Y_mvlXdfTYSswGNKgTbbP6i8SEETGtuj1RlCNwuuFR9YqIYJDIVA3-t5i1Pcba5caI-POt8-sNl4XYMnP/s200/dreampie.PNG" border="0" alt="" id="BLOGGER_PHOTO_ID_5649294290886059314" style="float: left; margin-top: 0px; margin-right: 10px; margin-bottom: 10px; margin-left: 0px; cursor: pointer; width: 200px; height: 152px; "></font></font></div><div><font class="Apple-style-span" style="font-size: medium; "><a href="http://dreampie.sourceforge.net/">DreamPie</a> is, very simply, my favorite cross-platform interactive Python interpreter. It began life as a fork of Python's built-in IDLE command shell, and from there it's never looked back. It has excellent interactive completion for packages (so you can type "from sys import s<tab>" and get a list of "stdin, stdout, stderr"). </tab></font></div><div><font class="Apple-style-span" style="font-size: medium; "><tab><br /></tab></font></div><div><font class="Apple-style-span" style="font-size: medium; "><tab>Even better, it does completion when you're typing file paths in arbitrary strings. </tab></font><span class="Apple-style-span" style="font-size: medium; ">I use this a lot to get to modules I'm trying to test: <font class="Apple-style-span" face="'courier new'">"import os,sys; sys.path.append('c:/src/</font><tab><font class="Apple-style-span" face="'courier new'">'"</font> gives me a list of all the directories in in "c:/src".</tab></span></div><div><font class="Apple-style-span"><br /></font></div><div><font class="Apple-style-span">It also has a slick separation of (typed) input and (generated) output, and a neat "copy only code" feature that makes it perfect for "try this code interactively, and when it works the way I want it, yank it into the actual source file" exploration. </font></div><div><font class="Apple-style-span"><br /></font></div><div><font class="Apple-style-span">DreamPie works pretty much the same on both Linux and Windows systems. It's reputed to work well on Mac systems, too, but I don't use them for day-to-day development.</font></div><div><font class="Apple-style-span"><br /></font></div><div><font class="Apple-style-span">So where I'd normally crank up the Python command interpreter for interactive exploratory coding, I usually reach for DreamPie instead.</font></div><div><font class="Apple-style-span"><br /></font></div><div><font class="Apple-style-span">But what I needed to explore now was the Windows API as called from C++, not Python.</font></div><div><font class="Apple-style-span"><br /></font></div><div><font class="Apple-style-span" size="4"><b>Secret Weapon #2: ctypes</b></font></div><div><font class="Apple-style-span"><br /></font></div><div>ctypes is a "foreign function interface" (FFI) that's been part of Python since version 2.5. An FFI is just a way to call code that isn't written in your current programming language. In our case, the functions I wanted to call in order to test out serial port notification are in the kernel32.dll library, which is part of Windows. ctypes makes this really easy. Well, easy if you happen to have the Windows API documentation and all of the correct C header files handy, and if you know exactly what you're looking for:</div><div><br /></div><div><div><font class="Apple-style-span" face="'courier new'">>>> import ctypes</font></div><div><font class="Apple-style-span" face="'courier new'">... file_mode = 0x80000000 # GENERIC_READ from <winnt.h></font></div><div><font class="Apple-style-span" face="'courier new'">... open_existing = 3 # from <winbase.h></font></div><div><font class="Apple-style-span" face="'courier new'">... buffer = ctypes.create_string_buffer(100)</font></div><div><font class="Apple-style-span" face="'courier new'">... bytes_read = ctypes.c_ulong(0)</font></div><div><font class="Apple-style-span" face="'courier new'">... hfile = ctypes.windll.kernel32.CreateFileW(r'\\.\COM17', file_mode, 0, None, open_existing, 0, None)</font></div><div><font class="Apple-style-span" face="'courier new'">... ctypes.windll.kernel32.ReadFile(hfile, buffer, 100, ctypes.byref(bytes_read), None)</font></div><div><font class="Apple-style-span" face="'courier new'">... buffer.value</font></div><div><font class="Apple-style-span" face="'courier new'">0: b'\r\n052100746029\r\n'</font></div><div><font class="Apple-style-span" face="'courier new'">>>> </font></div></div><div><br /></div><div>Hooray. We can call the Win32 API functions to open the serial port and read from it, just like we would from C code.</div><div><br /></div><div>But... that's an awful lot of crap to remember and type. I had to know exactly the C code I wanted to write. I had to know the Windows API well enough to find the constants and the functions to call. I had to know the ctypes API well enough to wire up Python to the C return values via ctypes buffers. </div><div><br /></div><div>What a chore. Did I mention I'm lazy?</div><div><br /></div><div>ctypes is the universal adapter--it can connect Python code to anything. But if you're specifically looking to call the Windows API, there's an even better tool:</div><div><br /></div><div><font class="Apple-style-span" size="4"><b>Secret Weapon #3: PyWin32</b></font></div><div><br /></div><div><a href="http://sourceforge.net/projects/pywin32/">PyWin32</a> predates ctypes, but it has a similar goal: gluing Python to something else. In this case, something else is specifically the entire Win32 API. PyWin32 consists of about two dozen modules, for example, "win32print" for printing, or "win32gui" for window handling, which wrap a good portion of the Win32 API.</div><div><br /></div><div>The documentation is rather Spartan, but if you know the Win32 API side, you can map those calls to the PyWin32 modules without too much pain. The 8-line, hard-to-remember ctypes example above turns into just <b>four lines of simpler code </b>using PyWin32:</div><div><br /></div><div><div><font class="Apple-style-span" face="'courier new'">>>> import win32file # for CreateFile</font></div><div><font class="Apple-style-span" face="'courier new'">... import win32con # for constants</font></div><div><font class="Apple-style-span" face="'courier new'">... hfile = win32file.CreateFileW(r'\\.\COM17',</font></div><div><font class="Apple-style-span" face="'courier new'">... win32con.GENERIC_READ,</font></div><div><font class="Apple-style-span" face="'courier new'">... 0,</font></div><div><font class="Apple-style-span" face="'courier new'">... None,</font></div><div><font class="Apple-style-span" face="'courier new'">... win32con.OPEN_EXISTING,</font></div><div><font class="Apple-style-span" face="'courier new'">... 0,</font></div><div><font class="Apple-style-span" face="'courier new'">... None)</font></div><div><font class="Apple-style-span" face="'courier new'">... win32file.ReadFile(hfile, 50, None)</font></div></div><div><div><font class="Apple-style-span" face="'courier new'">0: (0, b'\r\n052100746029\r\n')</font></div></div><div><br /></div><div><font class="Apple-style-span" size="4"><b>The Final Secret Weapon</b></font></div><div><br /></div><div>My actual exploratory DreamPie session to see if Window's virtual Bluetooth serial port supported RLSD looked like this:</div><div><br /></div><div><div><font class="Apple-style-span" face="'courier new'">>>> import win32api, win32file, win32con</font></div><div><font class="Apple-style-span" face="'courier new'">>>> hfile = win32file.CreateFileW(r'\\.\COM17', win32con.GENERIC_READ | win32con.GENERIC_WRITE, 0, None, win32con.OPEN_EXISTING, 0, None)</font></div><div><font class="Apple-style-span" face="'courier new'">>>> win32file.GetCommMask(hfile)</font></div><div><font class="Apple-style-span" face="'courier new'">0: 0</font></div><div><font class="Apple-style-span" face="'courier new'">>>> win32file.SetCommMask(hfile, win32con.EV_RLSD)</font></div><div><font class="Apple-style-span" face="'courier new'">Traceback (most recent call last):</font></div><div><font class="Apple-style-span" face="'courier new'"> File "<pyshell#3>", line 1, in <module></module></pyshell#3></font></div><div><font class="Apple-style-span" face="'courier new'"> win32file.SetCommMask(hfile, win32con.EV_RLSD)</font></div><div><font class="Apple-style-span" face="'courier new'">AttributeError: 'module' object has no attribute 'EV_RLSD'</font></div><div><font class="Apple-style-span" face="'courier new'">>>> win32file.SetCommMask(hfile, win32file.EV_RLSD)</font></div><div><font class="Apple-style-span" face="'courier new'">>>> win32file.GetCommMask(hfile)</font></div><div><font class="Apple-style-span" face="'courier new'">1: 32</font></div><div><font class="Apple-style-span" face="'courier new'">>>> win32file.EV_RLSD</font></div><div><font class="Apple-style-span" face="'courier new'">2: 32</font></div><div><font class="Apple-style-span" face="'courier new'">>>> win32file.WaitCommEvent(hfile)</font></div><div><font class="Apple-style-span" face="'courier new'">3: (0, 32)</font></div><div><font class="Apple-style-span" face="'courier new'">>>> </font></div></div><div><br /></div><div>This is an actual copy of the DreamPie buffer from my test session, mistakes and all. This is what really happened when I tried to figure out if RLSD would work:</div><div><ol><li>I typed up the code to open the serial port, which I knew should succeed, and it did.</li><li>I looked up the Win32 API call to get the "event mask", or the set of events that were being watched on the serial port handle, and saw that it was "<a href="http://msdn.microsoft.com/en-us/library/aa363257(v=vs.85).aspx">GetCommMask</a>". I blindly typed "win32file.GetCo", and lo and behold, DreamPie brought up a list of completions, which assured me that GetCommMask was there.</li><li>The Win32 API said that GetCommMask returned its result in a buffer passed into the call. Knowing that PyWin32 usually does a pretty good job of hiding return buffers, I decided to just try calling it with the input parameter, and got back zero. That made sense, if the serial port wasn't being monitored for events.</li><li>So I decided to push my luck: if GetCommMask worked, SetCommMask should work, too. A quick peek at the <a href="http://msdn.microsoft.com/en-us/library/aa363435(v=vs.85).aspx">documentation</a>, and... hrm. win32con didn't contain the "EV_RLSD" constant I was looking for to monitor the RLSD signal.</li><li>Well, I <b>could</b> have just typed the exact value (0x020) from the Windows docs... or I could just retype the line and use PyWin's autocompletion to see if win32file has the constant. I typed "win32file.EV_<tab>", and I had my answer. Then a quick re-test of GetCommMask() showed that the value was set.</tab></li><li>The API docs claimed that WaitCommEvent should wait for one of the masked events to occur, and then return which one occurred. But the documentation showed that it took another of those return buffers. Thinking that PyWin32 might help me here, too: I typed "win32file.WaitCommEvent(hfile)", and the call appeared to block.</li><li>So I powered down the device, and within a few seconds, I was rewarded with the return value from WaitCommEvent: (0, 32). Aha. This meant that the Windows API version of WaitCommEvent returned 0 (for success), and that the return buffer contained 32, or EV_RLSD.</li></ol></div><div><br /></div><div>I included all the steps, including the mistakes, to show the last secret weapon: <b>flexibility</b>. Be willing to bounce back and forth between the documentation, the code you <i>think</i> should work, and the feedback you get both from the code under test and the tools you're using--and be willing to change your mental model based on that feedback.</div><div><br /></div><div>In reality, this whole test took <b>under five minutes</b> from "Hmm... I wonder if I can use the DCD signal" to "Aha, looks like it works! Time to test it in C++." To be honest, I didn't even type out the whole ctypes version while testing--I started on it, realized that I'd have to look up and type all the constants by hand, then restarted DreamPie to jump over to PyWin32. Remembering that you can switch tools on the fly keeps you from getting stuck in ratholes that aren't directly related to the task at hand.</div><div><br /></div><div>Flexibility is the key to fast and efficient exploratory coding. Using an interactive language like Python with a good set of support tools and libraries can be a secret weapon for speeding up exploratory coding--even when your target language is C++.</div><div><br /></div><div><br /></div>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com4tag:blogger.com,1999:blog-9410449.post-10214581174066354952011-08-10T11:54:00.004-04:002011-08-10T12:29:53.831-04:00Stages of CompetencyAfter doing "this programmer thing" for a few years now, I've noticed a pattern in how I acquire skills and techniques. It's surprisingly consistent, and consists of these stages:<div>
<br /></div><div><b><span class="Apple-style-span">0: Awareness</span></b></div><div>
<br /></div><div>I've heard of the technique and can regurgitate a definition and a couple of use cases. I can probably pass a really bad phone screen (and in my experience, most of them are).</div><div>
<br /></div><div><span class="Apple-style-span"><b>1: Familiarity</b></span></div><div>
<br /></div><div>It's intrigued me enough that I've read up on it. I've probably looked at some code that uses it, and I can pick it out of a crowd, but I still mentally skip over it when reading its code (a bad habit that makes it harder to get past this phase). </div><div>
<br /></div><div>At this point, if I were asked "what is X" or "how does X work" in an interview, I can probably pass the question, as long as there isn't a followup involving coding or something like "what are the pitfalls of using X over the long term", which is why I don't use questions like that in interviews anymore!</div><div>
<br /></div><div><span class="Apple-style-span"><b>2: Functional Understanding</b></span></div><div>
<br /></div><div>At this point I've either had to work with someone else's code that uses it, or else I've gone through an article that shows how to use it. I don't mentally skip over it anymore, and I can debug and modify it with some difficulty. Importantly, I can tell someone else what it's doing, but I will probably get embarrassed if I try to get into the details or (worse) debug it with them.</div><div>
<br /></div><div>But I can't usefully <i>synthesize</i> anything with it. I get to a point in code and think, "Ah, this is a good place to use X!". Two hours (or more) later, I have bruises on my forehead from bashing it into the desk, I'm thinking "THIS CAN NOT BE THAT HARD", and I start wondering why I don't stay with the subset of techniques I know like the back of my hand. That's really tempting.</div><div>
<br /></div><div>I get stuck in stage 2 a lot. I was there with C++ template metaprogramming for about five years, and I'm still there right now with Python metaclasses.</div><div>
<br /></div><div><span class="Apple-style-span"><b>3: Understanding</b></span></div><div>
<br /></div><div>After several frustrating episodes in stage 2, I do exactly the same thing in another context and... it makes sense. It works. I don't believe it, so I tweak things that should make it break, and it breaks in predictable ways. And I can reverse the tweaks and have it work again, predictably.</div><div>
<br /></div><div>At this point, I always have the same three internal questions: a) do I really understand this? b) how did I not really understand this before? and c) what am I missing? I get uncomfortable not knowing how I know something.</div><div>
<br /></div><div>Then all is well until I try to teach it to someone else, and we end up in another multi-hour WTF session.</div><div>
<br /></div><div>What I've really learned at this stage is a single "groove" that works. As long as I don't deviate too much from the way I've used the technique, everything is fine. I think that subconsciously I know the limitations of that "groove", so I don't tend to make the little changes that expose the rough corners of my understanding. When I'm working with someone else, they have different edges to their own understanding. That's when I get this "uh-oh" feeling that tells me I really don't know what's going to happen when we do this. </div><div>
<br /></div><div>Absent working with other people, I still <i>think</i> I understand it, which is a dangerous bit of self-delusion, and the biggest reason I'd rather work with a team than solo.</div><div>
<br /></div><div><span class="Apple-style-span"><b>4: Competence</b></span></div><div>
<br /></div><div>I don't know how I get here either, except maybe via repetitions of stage 3. In fact I don't usually notice even <i>getting</i> to this stage. The sign is usually that I'm having to do something outside the "groove" of my usual use of a technique, and that little "uh-oh" goes off, and then... it still works. Or else someone asks me about what would happen in a nasty corner case, and what comes out of my mouth is a better explanation of the details than I thought I could come up with.</div><div>
<br /></div><div>This is also the point at which I finally feel comfortable writing about the technique, showing someone else how to use it, or trying to extend or modify it. The irony of it is that unless I do those things earlier, when I don't feel competent to do so, I tend not to get to this stage.</div><div>
<br /></div><div><hr></div><div>
<br /></div><div>The funniest thing about this model is that if I look at code I've written in the past, I can usually pick out where I was on the scale when I wrote it. Again, I can't say exactly what the "tells" are, but when I get to stage 4 on something and look back at earlier code, I can think "ahh, ok, I was stuck in stage 2 at the time, and the places this code will break are probably X, Y, and Z."... and they usually are.</div><div>
<br /></div><div>Forget owner's manuals--I wish brains came with source code. This progression would make a lot more sense then.</div>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com2tag:blogger.com,1999:blog-9410449.post-36880609579375587282010-10-20T14:58:00.005-04:002010-10-20T15:50:39.961-04:00SwitchpyOne of the consequences of the 2.x-to-3.x Python changeover is that I need to keep both versions around for a while on my Windows dev workstation.<div><br /></div><div>Actually, strike that: I need to keep <i>many</i> versions around:<div><ul><li>2.5.4, because that's the earliest version we support at work for some internal tools</li><li>2.6.6, because one particular internal tool jumped the gun and started using the "with" statement before we migrated to...</li><li>2.7, because that's what we're migrating those internal tools to (slowly)</li><li>3.1.2, because that's what we're targeting for new development</li><li>A "special" 3.1.2, which mimics the version we've modified for use in our embedded devices</li><li>The most recent 3.2 alpha, for testing</li><li>A 3.2 trunk install, for testing patches</li></ul><div><a href="http://pypi.python.org/pypi/virtualenv">Virtualenv </a>doesn't exactly do what I want: you have to install it from within an already-installed version of Python, and it doesn't support Python 3 yet (although there is a <a href="http://bitbucket.org/brandon/virtualenv3/">fork </a>that does). Plus it doesn't handle anything other than environment variables--it doesn't understand Windows' defaults.</div></div></div><div><br /></div><div><a href="http://nedbatchelder.com/">Ned Batchelder</a> wrote a neat <a href="http://nedbatchelder.com/blog/200810/switching_python_versions_on_windows.html">script</a> that does some of that, but again, it doesn't handle everything.</div><div><br /></div><div>So starting from Ned's script, I came up with switchpy:</div><div><ul><li>Supports Windows Python versions from 2.5 up to 3.2</li><li>Changes the local PATH environment in the current shell (via the same batchfile trick as <a href="http://apipes.blogspot.com/2010/02/mpath-command-line-path-manipulation.html">mpath</a>)</li><li>Updates the Registry-based associations (via code from Ned's script)</li><li>Pings Explorer so that if you run "python.exe" from the Start | Run command, it notices the update</li><li>Automatically reads installed official versions from the Registry, so you can say "switchpy 31" instead of "switchpy c:\python31"</li></ul><div>So now, testing scripts in multiple versions of Python is as easy as:</div><br /><div style="margin-left:1em;"><br /><pre>C:\src\myscript>switchpy 25<br />Switching to Python at C:\Python25\...<br />Python is now C:\Python25\<br /><br />C:\src\myscript>py.test<br />============================= test session starts =============================<br />python: platform win32 -- Python 2.5.4 -- pytest-1.3.0<br />test object 1: C:\src\myscript<br /><br />myscript\tests\test_script.py ...<br /><br />========================== 3 passed in 0.03 seconds ===========================<br /><br />C:\src\myscript>switchpy 31<br />Switching to Python at C:\Python31\...<br />Python is now C:\Python31\<br /><br />C:\src\myscript>py.test<br />============================= test session starts =============================<br />platform win32 -- Python 3.1.2 -- pytest-1.3.1<br />test object 1: C:\src\myscript<br /><br />myscript\tests\test_script.py ...<br /><br />========================== 3 passed in 0.03 seconds ===========================<br /><br /><br /></pre></div><br /><br /><div>For now, you can find switchpy in the same <a href="http://bitbucket.org/tlesher/mpath/">bitbucket repo</a> as mpath; if I add any more scripts, I'll probably end up making it a more general repo.</div></div>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com4tag:blogger.com,1999:blog-9410449.post-1557983902847097532010-05-10T11:32:00.004-04:002010-05-10T12:17:03.362-04:00Language of the Year<strike><span class="Apple-style-span" style="font-size: medium;">Several</span></strike><span class="Apple-style-span" style="font-size: medium;"> </span><strike><span class="Apple-style-span" style="font-size: medium;">some</span></strike><span class="Apple-style-span" style="font-size: medium;"> </span><span style="border-bottom-style: dotted; border-bottom-width: 1px;" title="damn, I'm getting old"><span class="Apple-style-span" style="font-size: medium;">many</span></span><span class="Apple-style-span" style="font-size: medium;"> years ago, I decided to learn a new programming language a year. I don't subscribe to the "once you're a programmer, you can learn any new language in a few weeks" theory, so I didn't intend to get to guru status on any of them in 52 weekends and a few nights. Instead, the idea was to get good enough to be able to read code well, synthesize code at a decent level of competency, and understand the ideas behind the language, then move on.</span><div><span class="Apple-style-span" style="font-size: medium;"><br /></span><div><span class="Apple-style-span" style="font-size: medium;">If memory serves me correctly, I started with Java, then moved through REBOL, Python, Ruby, Scheme, and Lua before I dropped the practice. I tried getting back into it with Haskell a few years ago, but didn't take it to the "ok, I get it now" level of proficiency--mostly because of time constraints of changing jobs rather than any issue with the language.</span></div></div><div><span class="Apple-style-span" style="font-size: medium;"><br /></span></div><div><span class="Apple-style-span" style="font-size: medium;">I did learn something from each of these, which is the real point of the exercise. Now I'm thinking about starting a (belated) Language of the Year for 2010. The contenders are:</span></div><div><span class="Apple-style-span" style="font-size: medium;"><br /></span></div><div><span class="Apple-style-span"><b><span class="Apple-style-span" style="font-size: medium;">Erlang</span></b></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Pro: interesting concurrency (independent processes) and data (immutable) models.</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Con: might be too big to get my head around in nights and weekends</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;"><br /></span></span></div><div><b><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Io</span></span></b></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Pro: Small surface area; looks like "Lua with a prototype-based object model" so far; might be useful practically.</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Con: Not sure there's enough new there after doing Lua and REBOL.</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;"><br /></span></span></div><div><b><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Clojure</span></span></b></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Pro: Interesting transactional memory model. Also, </span><span title="I really do like LISP, from afar." style="border-bottom-style: dotted; border-bottom-width: 1px;"><span class="Apple-style-span" style="font-size: medium;">parentheses</span></span><span class="Apple-style-span" style="font-size: medium;">.</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Con: I'm wondering how much time I'll spend re-learning the Java environment vs. learning Clojure.</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;"><br /></span></span></div><div><b><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Go</span></span></b></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Pro: Interesting concurrency model; might be useful practically.</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Con: Not sure there's enough different there, compared to Clojure or Erlang.</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;"><br /></span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">Part of the problem is picking a language that different enough to make it worth learning, but not so different that I can't use it for small, practical tools that I can actually use day-to-day. The Haskell and Scheme experiences showed me that if I can't use what I'm practicing day-to-day, I find it hard to keep devoting time to the project.</span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;"><br /></span></span></div><div><span class="Apple-style-span"><span class="Apple-style-span" style="font-size: medium;">At the moment, it's looking like Clojure > Go > Erlang > Io. I'll probably pick up a distribution for each, get as far as "hello world", and then decide.</span></span></div><div><span class="Apple-style-span" style="font-size: medium;"><br /></span></div>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com1tag:blogger.com,1999:blog-9410449.post-44330725045134027402010-02-16T12:45:00.005-05:002010-11-19T13:28:28.837-05:00Mpath: command-line path manipulation for Windows<p> I'm a command line geek. Windows' style of installing everything in its own directory makes it easier to clean up after uninstallation, but it makes for very long PATH environment variables. If I put every directory containing command line tools in the system path, it gets too long for Windows to handle. So I usually end up doing "<tt>PATH=%PATH%;c:\somethingelse\bin</tt>" just before I use it. That also makes for long paths over long runtimes, especially when you use it in batch files (since you end up with <tt>PATH=c:\somethingelse\bin;c:\somethingelse\bin;c:\somethingelse\bin;[rest of path]</tt> after multiple invocations). </p><p> So I wrote <a href="http://bitbucket.org/tlesher/mpath/" target="_top">mpath</a>. Mpath is a combination batch file and Python script that takes advantage of some quirks of the Windows command shell, to let a child process alter the environment of a parent command shell process (something that you typically can't do in win32, but mpath gets around it by creating a temporary batch file that gets executed in the parent process). </p><p> Syntax: </p><p> </p><ul style="color: rgb(0, 0, 0);"><li> <code><b>mpath <em>pathname</em> :</b></code> prepends <em>pathname</em> to the current command shell's PATH, if it doesn't already exist. </li><li> <code><b>mpath + <em>pathname</em> :</b></code> appends <em>pathname</em> to the current command shell's PATH, if it doesn't already exist. </li><li> <code><b>mpath - <em>pathname</em> :</b></code> removes <em>pathname</em> from the current command shell's PATH, if it exists. </li></ul> <p> A quick demo: </p><div style="font-family: courier new;" class="codebox"> C:\> PATH=C:<br /><br />C:\> PATH<br />PATH=C:<br /><br />C:\> mpath c:\foo <em>(prepend c:\foo to the path)</em><br /><br />C:\> PATH<br />PATH=c:\foo;C:<br /><br />C:\> mpath - C:\FOO <em>(take it off the path--note case insensitivity)</em><br /><br />C:\> PATH<br />PATH=C:<br /><br />C:\> mpath + c:\foo <em>(append c:\foo to the path)</em><br /><br />C:\> PATH<br />PATH=C:\;c:\foo<br /><br />C:\> mpath c:\foo <em>(try to prepend it again--mpath knows it's already there)</em><br />c:\foo already in path.<br /><br />C:\> PATH=%PATH%;c:\foo <em>(silly user should have used mpath...)</em><br /><br />C:\> PATH<br />PATH=C:\;c:\foo;c:\foo <em>(now there are two copies!)</em><br /><br />C:\> mpath - C:\FOO <em>(but mpath takes care of that.)</em><br /><br />C:\> PATH<br />PATH=C:<br /></div><br /><br /><p><strike>I've tested Mpath with Windows XP running Python 2.5 and 2.6. I know it doesn't work on 3.x; I plan on fixing that at some point when I need it.</strike></p><br /><br /><p><em>Update: mpath is now tested on 2.5, 2.6, 2.7, and 3.1.</em></p>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com4tag:blogger.com,1999:blog-9410449.post-55636215603132710442009-12-07T10:12:00.005-05:002010-10-20T15:50:59.909-04:00Five Pycon 2010 Talks I Need to SeeFollowing the example of <a href="http://catherinedevlin.blogspot.com/2009/11/pycon-pre-favorites.html">Catherine Devlin</a> and <a href="http://pyright.blogspot.com/2009/11/pycon-2010-pre-favorites-carl-t-edition.html">Carl Trachte</a>, I thought I'd put together a list of the five <a href="http://us.pycon.org">Pycon </a>talks I need to see in 2010. But I couldn't--I struggled to get below a dozen. So here are the top five I need to see, plus the ones I'll probably kick myself for not seeing because they're undoubtedly going to be scheduled in the same slots as the top five:<br /><br />1. <a href="http://us.pycon.org/2010/conference/talks/#proposal_link_9">Import this, that, and the other thing: custom importers</a> (Brett Cannon)<br />This is an easy choice, because I'm about to be implementing one of these for work. Would have been be nicer if Pycon 2010 had been scheduled for September 2009, but I'll take what I can get.<br /><br />2. <a href="http://us.pycon.org/2010/conference/talks/#proposal_link_82">Understanding the Python GIL</a> (David Beazley)<br />Another easy choice. After reading lots of code and debugging thread issues in our embedded Python interpreter at work, I think have a decent grasp of the GIL implementation. Given David's <a href="http://www.dabeaz.com/coroutines/">mindbending generators tutorial</a> last year and his <a href="http://www.dabeaz.com/python/GIL.pdf">GIL presentation</a> from ChiPy, I expect this talk to be rich in things I will be disturbed to have learned.<br /><br />3. <a href="http://us.pycon.org/2010/conference/talks/#proposal_link_186">Powerful Pythonic Patterns</a> (Alex Martelli)<br />Alex's talk last year, Abstractions as Leverage, was curiously satisfying. He didn't present any facts I hadn't already heard or read, but his presentation made some new connections for me (in a "My God, it's full of stars!" way).<br /><br />4. <a href="http://us.pycon.org/2010/conference/talks/#proposal_link_187">Threading is Not a Model</a> (Joe Gregorio)<br />In the last few years, I've begun to see pervasive threading as a placebo more than a solution. To paraphrase JWZ, some people, when confronted with a problem, think, "I know, I'll spin up a new thread." Now they have two problems. In reality, they've usually created an unknown number of problems, bounded only at the lower end by the number two. I'm really interested in seeing what Joe brings to the discussion beyond the usual "<a href="http://en.wikipedia.org/wiki/Rock-paper-scissors">threads, select(), or fork()</a>" question.<br /><br />5. <a href="http://us.pycon.org/2010/conference/talks/#proposal_link_169">Turtles All The Way Down: Demystifying Deferreds, Decorators, and Declarations</a> (Glyf Lefkowitz)<br />I have a long history of utter contempt for the practice of using syntactic sugar to "re-define the language in order to provide a more concise, natural style" for a given purpose. Glyf says he "will try to convince you that all of this wonderful magic isn't all that weird". Sounds like a challenge. If you're not continually questioning your own biases, you're heading for a mental rut, so I'm going to try to attend this with an open mind (and probably leave with a thoroughly-bitten tongue).<br /><br />These are the ones I will move heaven, earth, and lunch plans to see. The others I really want to attend are:<br /><br /><ul><li>How Are Large Applications Embedding Python? (Peter Shinners). Totally relevant for work, but probably more elementary than I'd want.</li><li>What Every Developer Should Know About Database Scalability (Jonathan Ellis). Totally irrelevant for my current work, but I've had to work in this area in the past, so it's somewhat interesting, and I'm curious about what's changed lately.</li><li>Optimizations and Micro-Optimizations in CPython (Larry Hastings). Pure geeky personal interest.</li><li>New *and* Improved: Coming changes to unittest, the standard library test framework (Michael Foord). I'm not <span style="font-style: italic;">quite</span> a test-driven development zealot, but I'm about as close as you can get without applying for membership.</li><li>Python Metaprogramming (Nicolas Lara). More pure geeky goodness.</li><li>Eventlet: Asynchronous I/O with a Synchronous Interface (Donovan Preston). I can't quite decide whether this is applicable to work or not, and there's only one way to find out.</li><li>Seattle: A Python-based Platform for Easy Development and Deployment of Networked Systems and Applications (Ivan Beschastnikh). I was quite disappointed by last year's sandboxing talk (the description didn't really let on that it was all about PyPy), so I'm hoping I can pick up more from this one.</li><li>Tests and Testability (Ned Batchelder). Probably more elementary-level than I'd like, but might have some good discussion.</li><li>On the Subject of Source Code (Ian Bicking). Another blue-sky talk by Ian? Yes, please.</li><li>Python's Dusty Corners (Jack Diederich). I have a feeling this will be like Doug Hellman's PyModule of The Week: 80% of it is "yeah, yeah, I knew that," and 20% is "oh, wow, how did I <span style="font-style: italic;">not </span>know that?"</li></ul>I wasn't terribly impressed by the tutorial list (other than the compiled Python one), so I'll probably pass on them, but the talks look even better than last year. See you in Atlanta!Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-86074546395971999192009-10-17T22:20:00.006-04:002009-10-18T01:21:02.083-04:00Five Things I Hate About C++A few years ago, the "five things I hate about my favorite programming language" went around. I think it originated with <a href="http://use.perl.org/~brian_d_foy/journal/32556">Brian D. Foy's post on Perl</a>. I like his reasoning: if you can't think of five things you don't like about it, you probably don't know enough about it to advocate for it. <br /><br />Peter Siebel's <a href="http://gigamonkeys.com/blog/2009/10/16/coders-c++.html">recent post </a>about the opinions of the folks he interviewed for <em>Coders at Work</em> made me remember it again. While I often reach for <a href="http://www.python.org/">Python</a> as the top tool in my toolbox these days, I've been writing C++ for most of my career, so I thought I'd take a crack at C++ first. <br /><br />So:<br /><br /><span style="font-size:130%;"><strong>5. No consistent ABI</strong></span><br /><br />C++ doesn't define a standard application binary interface (a standard for how the binaries produced from source code are laid out or linked together). If you're writing code to link against a pre-built library, then unless you're using the same version of the same compiler, you can't guarantee that your code will work correctly. (Technically, C doesn't either, but for practical purposes, though, C is in much better shape, mostly because C++'s features provide far more opportunities for implementations to disagree.)<br /><br />The practical result is often that C linkage is considered "safe" and C++ linkage is considered "unsafe", which means that C linkage is the lingua franca for object-level interoperability, and no one really pushes for compatible C++ linkage--which in turn means that it doesn't happen. (While my experience is mostly in Windows programming, the situation seems to be better in the <a href="http://gcc.gnu.org/gcc-3.2/c++-abi.html">g++ and GNU/Linux world</a>--as is often the case.)<br /><br /><strong><span style="font-size:130%;">4. Sorta-kinda safety</span></strong><br /><br />The first benefit of C++ over C for me wasn't object-orientation. It was that C++ seemed to be much better at catching the kinds of low-level programmer errors I tended to make back then. Class member protection, type-checked function parameters, exceptions that (unlike return codes) can't accidentally be ignored, constructors and destructors that are guaranteed to be called at the right time, improved casting operations--what's not to like?<br /><br />The problem is that most of the safety features aren't really <em>safe</em>, they're just a little <em>safer</em>, often due to the desire for source code compatibility with C or concerns about run-time performance. You can probably argue that C++ is safer than C, but I believe it's "just safer enough" that C++ programmers get complacent.<br /><br />Plus the interaction language of features makes it much easier to commit horrible, higher-level design mistakes that are harder to see when reading the code, particularly with things like non-trivial constructors and destructors, misused (and overused!) inheritance, and non-obvious method overrides.<br /><br /><span style="font-size:130%;"><strong>3. Textual macros</strong></span><br /><br />The LISP world has had the "hygienic vs unhygienic macros" argument for a long time. In a (grossly oversimplified) nutshell, hygienic macros allow you to define new, reusable bits of language without worrying about the context in which they'll be evaluated. This makes for safer macro definitions, but precludes some very useful techniques that unhygienic macros allow--for example, enabling the code in the expanded macro use and affect variables in the context in which it's expanded.<br /><br />But the C/C++ macro implementation makes LISP's unhygienic macros look like an Intel cleanroom. That's because they're not even really part of the C language syntax: they're just a simple, dumb textual replacement done in a preprocessing step, before compilation even occurs. <br /><br />This feature inherited from C is so error prone that C++ added features like "inline" and namespacing to try to approximate the most common use cases for C macros, so that we wouldn't have to deal with them. It still didn't want to touch preprocessor macros for fear of breaking backwards compatibility, though, so now we have the worst of both worlds: a dangerous feature implemented outside the language syntax, with some of its bits duplicated in the language syntax, and <a href="http://www.research.att.com/~bs/bs_faq2.html#macro">guidance</a> that says "sorry about the mess--here's some partial replacements that don't quite cover the gamut, but that's all you get. Have a nice day."<br /><br /><strong><span style="font-size:130%;">2. Worst-of-both-worlds standardization</span></strong><br /><br />C was born as an in-house development language in an AT&T lab in the late 1960s, and was used in anger almost from day one (for reimplementing the UNIX operating system). By the time standardization started, the language feature set was fairly solid and well-proven, and implementators already had real-world knowledge of the features.<br /><br />C++, on the other hand, didn't go through this process. While C was designed as a language for implementing operating systems (and applications), C++ was designed as a language for implementing language features. It wasn't used (as far as I know) as the backbone of a single, well-known system in the way C was, so the language was free to evolve more divergently and more slowly.<br /><br />Worse, the development of the language seems to have been driven by the the design and evolution of the specification, rather than by things tried and lessons learned in implementation. In some cases, features were added to the language <em>specification</em> before they were even implemented, in the hopes that smart compiler vendors would figure something out. <br /><br />As a result, we have features that don't work like you'd expect (like std::vector<bool> or auto_ptr<>), features that don't interact well (like templates and class inheritance), and even features that, well, just <em>don't work</em> (like export, which was in the standard speculatively for years before its first attempted implementation, and which as far as I know has never been fully and correctly implemented by <em>anyone</em>).<br /><br />On the other hand, while C was standardized after it had mostly stabilized, the C++ standardization process started while the language was still very much in flux. As a result, the core language is full of weirdnesses that are explainable only when you know the political situation at the time. <br /><br />For example, the construction "virtual void foo() = 0;" is a pretty weird way to spell "pure virtual". In <em>The Design and Evolution of C++</em>, Bjarne Stroustrup reveals that the "=0" construction is there because he wanted to get pure virtual functions into the language specification, but a committee meeting was coming up soon, and he didn't think he could convince enough people to get behind adding a new "pure" keyword.<br /><br />This leads right into...<br /><br /><span style="font-size:130%;"><strong>1. C++ tries to be all things for all people</strong></span><br /><br />I think this one is the root of most of C++'s problems. C++ is and has always been a "more-is-better" language. If you like C, we'll make sure you like C++ by bending over backwards to make C code still work (except when it doesn't) and by making efficiency our top, err, one of our top-ten priorities. If you like object oriented programming, we've added classes and inheritance. Oh, multiple inheritance? Yep, we heard that works well, so we'll add it in there too. Parametric polymorphism? Multiple dispatch? Currying? Oh, hrm, we seem to have painted ourselves into a corner... but we can bodge most of that in with templates and partial template specialization. Oh, and guess what? We just figured out that you can use templates to do metaprogramming, so you get that feature for free! Free is good, right?<br /><br />This results in two, mostly-correct perceptions:<br /><br />1) C++ is a big grab-bag of language features, some of which are razor-sharp and don't really hang together coherently, but work great so long as you're really, really careful.<br />2) C++ is more-or-less better than C, so long as you stay with a "sane subset" of its features. <br /><br />But what is that sane subset? That depends entirely on who you talk to, and the subset that they choose usually reveals more about their own priorities and experience than anything about the language itself.<br /><br />All that being said, I still choose C++ (or my own trusted subset of it, at least) over C because of the convenience of constructors and destructors, the expressiveness of templates, and the confidence I get from RAII. I still choose it over Java because I don't need to worry about a runtime VM, because I can access platform-specific APIs and native libraries at will, and no checked-exception silliness. <br /><br />But that doesn't mean I don't sigh a little every time I burn multiple days chasing down an intermittent memory leak, or that I don't steal a surreptitious glance at <a href="http://www.digitalmars.com/d/">younger, better-looking languages</a> with less emotional baggage from time to time.Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com1tag:blogger.com,1999:blog-9410449.post-8396059608173002512009-10-15T12:49:00.016-04:002009-10-16T16:03:11.796-04:00Buying or Building... FurnitureMost software developers are familiar with the "<span style="font-weight: bold;">buy-or-build</span>" question: is it more effective to find existing software and try to make it work in your situation, or to build it to your exact specification and take on the burden of maintaining it? But sometimes it comes up in other contexts.<br /><br />Like office furniture.<br /><br />My current project at work is winding down, and I'm rolling over to a new one. As part of the transition, I'm moving from my old, two-person office into the new team's bullpen environment.<br /><br />It might seem like a poor trade, but this team <span style="font-style: italic;">chose </span>to trade in their fairly nice offices because they valued the higher conversational bandwidth they got in a bullpen. Yes, it's a bit noisier, but most of the noise is project-related, and results in quicker and more complete information dispersal both among developers and between developers and SQA engineers (who also share the space).<br /><br />The big win for me is that it reduces the barrier to pair-programming to the cost of mumbling, "Uh... can anyone take a look at this with me?" And we still have the offices for when we need to make a phone call or do an interview.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXutPp6VHFgKrGY6Dc7FE2qjPBf1rQ3LfLUeQwGlbAkooCXB6gi5n4lx9ZA0dLioWlD87mzlTD4FVnXUPDt2XLIj_EERcnUKI2MNNxgtgvTuj-bqszOQCvpA9UxmWNsAHubibA/s1600-h/curvy.jpg"><img style="margin: 0pt 10px 10px 0pt; float: right; cursor: pointer; width: 200px; height: 150px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXutPp6VHFgKrGY6Dc7FE2qjPBf1rQ3LfLUeQwGlbAkooCXB6gi5n4lx9ZA0dLioWlD87mzlTD4FVnXUPDt2XLIj_EERcnUKI2MNNxgtgvTuj-bqszOQCvpA9UxmWNsAHubibA/s200/curvy.jpg" alt="" id="BLOGGER_PHOTO_ID_5393269781954318850" border="0" /></a>One of the stipulations on building out the bullpen was that we had to use existing furniture. Unfortunately, while our current furniture is nice (and somewhat pricey, from what I'm told), it's optimized for a one-person or two-person office. We each get a curvy desk, a table with attached bookshelf that fits the curvy desk as an extension, and a funky rolling file cabinet. But the curvaceousness of the furniture means that it only fits well in a few prescribed configurations--none of which match a bullpen where you want to pair-program!<br /><br />So the current bullpen, built from curvy bits loosely jammed together, isn't big enough to hold more people. And naturally, the people who handle furniture and facilities wouldn't be terribly happy with us saying, "Oh, this expensive furniture is nice. Now would you mind finding some place in our already-filled building to store it, and buy us some additional expensive furniture just like it, but without curvy bits?"<br /><br />So our manager/Scrum Master, being the pragmatist that he is, decided we should build our own. From scratch.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWmJ0hsemV2bG7aYgDlbobYsFeE6sk0Ong5wqkk6Hs2o-bIM9j01rI3TB4DLtwZZR5aVSvA7mzygAPBTaIJedLle-3sK-WUfrPknHL0T1y7DMClPU-cIUfcmIzBYMmrZ76qUAy/s1600-h/construction.jpg"><img style="margin: 0pt 10px 10px 0pt; float: left; cursor: pointer; width: 200px; height: 150px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWmJ0hsemV2bG7aYgDlbobYsFeE6sk0Ong5wqkk6Hs2o-bIM9j01rI3TB4DLtwZZR5aVSvA7mzygAPBTaIJedLle-3sK-WUfrPknHL0T1y7DMClPU-cIUfcmIzBYMmrZ76qUAy/s200/construction.jpg" alt="" id="BLOGGER_PHOTO_ID_5393273605864650338" border="0" /></a>Actually, "scratch" in this case really means heavy, solid-core interior doors for tabletops, and prebuilt folding-table legs to hold them up. Assembly is trivial, the surfaces are generous, prefinished, and attractive, and the cost was just a fraction of what we'd have paid for non-curvy versions of our standard furniture (which keeps the facilities folks happy... or at least <span style="font-style: italic;">happier</span>).<br /><br />There are, of course, some drawbacks. Making single large pairing stations means that you have to choose a single table height. In our case, it was chosen for us by the height of the prefab table legs.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIPWzQA-1b-t2y4Kx_bzp0OxUC-caOpP5gKmGJp6ZftUUlptRV6NAepWtKnjYfTeelbGLD6kul9enrmkUIm8SObfjKTS87k1ZERFK_ryG8_p4C9xQfAl3nBsuraFRhNbxgosp7/s1600-h/height.jpg"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 200px; height: 150px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIPWzQA-1b-t2y4Kx_bzp0OxUC-caOpP5gKmGJp6ZftUUlptRV6NAepWtKnjYfTeelbGLD6kul9enrmkUIm8SObfjKTS87k1ZERFK_ryG8_p4C9xQfAl3nBsuraFRhNbxgosp7/s200/height.jpg" alt="" id="BLOGGER_PHOTO_ID_5393270089289606866" border="0" /></a>However, my current programming partner suffers from an unfortunate and tragic genetic defect that caused his growth to continue far beyond normal human levels (the medical term is, I believe, "freakishly tall"). I, on the other hand, boast a full 5'3" of height, which seems far more normal to me, all things being relative.<br /><br />So our alternative solution was to just tear the bookshelves off two small tables (again, that storage problem!), and then use one for each person, moving the tables around when we need to. The works great if the tables have cool adjustable legs like ours do.<br /><br />(Mine, of course, is the station on the right.)Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com1tag:blogger.com,1999:blog-9410449.post-61115773970312073402008-09-10T20:47:00.004-04:002008-09-10T21:03:24.790-04:00How to be the most awesome Dad everTo be the most awesome Dad ever, capable of carrying out feats of skill and mastery usually reserved for the likes of the <a href="http://en.wikipedia.org/wiki/Aang">Avatar</a>, <a href="http://en.wikipedia.org/wiki/James_Bond_(character)">James Bond</a>, or <a href="http://en.wikipedia.org/wiki/Doctor_(Doctor_Who)">the Doctor</a>, requires just a few common ingredients:<div><ol><li>The locked, most secret diary of a pre-adolescent daughter (who has lost the key),<br /></li><li>The knowledge that all such cheap locks are the same,<br /></li><li>A set of cheap luggage locks with keys,<br /></li><li>A frantic pre-adolescent daughter in possession of #1 but not #2 or #3, and<br /></li><li>A flair for the dramatic, with which one discloses that one knows how to pick locks, but it's a secret handed down from master spy to master spy, therefore the work must be done behind a locked door (which neatly conceals the fact that you're rummaging around in your bedroom drawer to find #3).<br /></li></ol></div>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com3tag:blogger.com,1999:blog-9410449.post-74175791831753841162008-06-03T13:49:00.007-04:002008-06-05T18:31:32.103-04:00Fun with itertoolsSometimes it's hard to shake old habits, especially when you've burned them into your brain as the "standard" way to do things. For example, I've been doing network programming with C and C++ for a very long time. One of the standard pieces of code I've written again and again is the "connect with backoff" pattern.<br /><br />If a program needs a continuous network connection, and that connection is lost, it should try to reconnect. On the one hand, you want to reconnect as quickly as possible; on the other hand, you don't want to keep retrying (and failing) in a tight loop. So you use a "backoff" timer: after each attempt, you wait longer (up to a maximum limit).<br /><br />As a C programmer, I would implement an algorithm that resembles this Python-like pseudocode:<br /><pre><br /># After the first failure wait half a second before retrying;<br /># double this each time up to eight seconds.<br />backoff_times = [.5, 1, 2, 4, 8]<br />cur_backoff = None<br /><br />while 1:<br /> try:<br /> # Try to connect<br /> connect()<br /> except ConnectionError:<br /> # Failed; update the backoff counter<br /> if cur_backoff is None:<br /> cur_backoff = 0<br /> else:<br /> cur_backoff = min(len(backoff_times)-1, cur_backoff+1)<br /> # Wait to retry<br /> time.sleep(backoff_times[cur_backoff])<br /> else:<br /> # Success; reset the backoff timer<br /> cur_backoff = None<br /></pre><br />But in Python the code to manage the current backoff timer looks out of place.<br /><br />In a high level language, <b>when the ratio of "code that says what I want" to "code that tells the language how to do what I want" gets too low, you're doing it wrong.</b> It means that you're spending too many mental cycles on the "how," and not enough on the "what".<br /><br />In this case, Python gives me a better way to tell it just "what" I want it to do: use an iterator.<br /><pre><br />import itertools<br /><br />def iter_pegged(seq):<br /> """Return an iterator that walks the sequence, and then 'pegs' on the last item."""<br /> return itertools.chain(seq, itertools.repeat(seq[-1]))<br /><br />backoff_times = [.5, 1, 2, 4, 8]<br />cur_backoff = iter_pegged(backoff_times)<br /><br />while 1:<br /> try:<br /> # Try to connect<br /> connect()<br /> except ConnectionError:<br /> # Wait to retry<br /> time.sleep(cur_backoff.next())<br /> else:<br /> # Success; reset the backoff timer<br /> cur_backoff = iter_pegged(backoff_times)<br /></pre><br />Other than the definition of iter_pegged, each line of code says what only <span style="font-style: italic;">what</span> it wants to do, not <span style="font-style: italic;">how</span> it wants to do it.<br /><br />And that's what coding in a high level language is all about, no?Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com3tag:blogger.com,1999:blog-9410449.post-16320145229077811882008-06-02T13:06:00.003-04:002008-06-02T13:25:16.521-04:00Git: what he said.About three months ago, I started writing a blog post on why my (development) life has changed for the better since I started using <a href="http://git.or.cz/">git</a> for version control. It's been stewing in my "Drafts" folder for lack of time and attention.<br /><br />My main point was that git is <span style="font-weight: bold;">optimized for messy reality</span>, as opposed to an idealized view of software development that never really happened that way, but looks better when you draw it on a white board.<br /><br />A few weeks ago, <a href="http://tomayko.com/">Ryan Tomayko</a> said most of what I intended to say, in <a href="http://tomayko.com/writings/the-thing-about-git">The Thing About Git</a>.<br /><br />So, "what he said". In particular:<br /><br /><ul><li>Git means never having to say, “<span style="font-style: italic;">you should have</span>”</li></ul><ul><li><code>git --rebase interactive</code> is “a bit like <code>git commit --amend</code> hopped up on acid and holding a chainsaw - completely insane and quite dangerous but capable of exposing entirely new states of mind.”</li></ul>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-39757198696040766402008-04-18T10:11:00.002-04:002008-04-18T10:16:01.028-04:00That Looks About Right<pre><br />tlesher@badwolf:~$ history | awk '{print $2}' | sort | uniq -c | sort -rn | head<br /> 109 ls<br /> 107 cd<br /> 78 git<br /> 26 sudo<br /> 14 rm<br /> 12 ssh<br /> 11 source<br /> 11 more<br /> 10 wget<br /> 9 ipython<br /></pre><br /><br />(via <a href="http://www.cwinters.com/news/display/3624">Chris</a> and <a href="http://blog.moertel.com/articles/2008/04/11/that-looks-about-right">Tom</a>)Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-69284681346774876592008-01-05T23:40:00.000-05:002008-01-31T12:58:08.369-05:00Oh, look! A startup!This December, I took a deep breath and jumped. I left a great job at a <a href="http://www.vocollect.com/">great company</a>, working on <a href="http://www.vocollect.com/global/web.php/en/offerings/vocollect_voice">cool projects</a> with fun people, and executed a perfect swan dive back into the tech startup blender. <br /><br />Now why would a thirty-something refugee from the tech bubble with a wife, two young kids, and a mortgage give up a good salary and stable, interesting job for the stress, uncertainty, and heavier workload of a new startup?<br /><br />For me, it's all about aligning outcomes with effort. In a very early stage startup, the link between "what is about to happen" and "what am I doing" is as close to direct as you can possibly get. The outcome for the company is directly tied to what you're doing and how well you're doing it: no excuses, and no wiggle room. Naturally, there are external risks (like being <a href="http://onstartups.com/home/tabid/3339/bid/115/Hindsight-2-0-Lessons-From-A-Failed-Web-2-0-Startup.aspx">run over</a> by a company so big they barely notice the bump), you see the results of your effort clearly.<br /><br />Working for a BigCo, Inc. is very different. Expending effort in a large company is like pulling on a bungie cord attached to a rock. Pull moderately, and nothing happens. Pull a little more, and the the rock comes along, but it tends to wiggle around in directions you don't intend. Yank <span style="font-weight: bold;">really</span> hard, and the rock has an annoying tendency to fly up and smack you in the head.<br /><br />Now replace the bungie cord with a stick attached to the weight. You pull a little, the weight moves a little. You pull a lot, the weight moves a lot. You push, and the weight moves the opposite direction. The outcome is directly tied to your effort.<br /><br />To be fair, my last company wasn't quite to the "bungie cord" stage. But when a former colleague approached me about his idea for a startup, I was drawn back in again.<br /><br />Of course, to <span style="font-style: italic;">(ahem)</span> stretch the analogy, sticks do tend to break more easily, but hey: that's the risk you take.Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com1tag:blogger.com,1999:blog-9410449.post-66598846151734276472007-12-20T11:25:00.000-05:002007-12-20T11:30:38.009-05:00Sad Mac<div align="center"><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEUcZ1wajdKzS6A2mB5xE8Bvpv8gpRtgEAxivsgfDh31B-mHmBl47dxXV6nalS_yYh5b0hXrqKdn18YzygfdSToBpX9gOSoIhIW7JG3RpTpqoJtu3efnlkEY-kGYEZZc7FgMpA/s1600-h/unhappy_mac_sm.jpg"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEUcZ1wajdKzS6A2mB5xE8Bvpv8gpRtgEAxivsgfDh31B-mHmBl47dxXV6nalS_yYh5b0hXrqKdn18YzygfdSToBpX9gOSoIhIW7JG3RpTpqoJtu3efnlkEY-kGYEZZc7FgMpA/s320/unhappy_mac_sm.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5146093173152821202" /></a><br /><br /><br />Now that, my friends, is a <a href="http://en.wikipedia.org/wiki/Sad_Mac">Sad Mac</a>.</div>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com2tag:blogger.com,1999:blog-9410449.post-72548741030992643672007-05-07T09:08:00.000-04:002007-05-07T09:25:47.442-04:0010 Things I Learned at MEDC<ol><li>In person, <a href="http://blogs.msdn.com/ce_base/archive/category/10306.aspx">Sue Loh</a> sounds exactly like you'd imagine from reading the <a href="http://blogs.msdn.com/ce_base/">CE blog</a>.</li><li>In person, <a href="http://www.bolingconsulting.com/">Doug Boling</a> sounds exactly like you'd imagine from reading <a href="http://www.amazon.com/Programming-Microsoft-Windows-Net-Third/dp/0735618844/ref=sr_1_1/103-4416368-8462238?ie=UTF8&s=books&qid=1178543892&sr=8-1">his book</a> (except his humor comes off better in person).</li><li>Windows Mobile 6 isn't as big a deal as it first seemed (unless you happen to be a managed code developer, which I'm not).</li><li><a href="http://www.vegasexposure.com/tao_nightclub.htm">The Tao</a> is worth seeing (if you can either get in on your own coolness, or else persuade a <a href="http://www.microsoft.com">multi-billion-dollar company</a> to slip the owner some cash).</li><li>You never know which of your mild-mannered colleagues will turn out to be a raving, iPod-dancing, Viva-Las-Vegas-singing Elvis fan.</li><li>Managed code (C#) actually runs on "real" embedded devices that are too small for even CE.</li><li>No matter how awesome <a href="http://venetian.com">your hotel</a> looks, you still need to provide running water to your guests, or they get cranky.</li><li>When attending a conference, make sure you introduce yourself and describe your company to as many people as you can: you never know which one will suggest a very cool opportunity a few days later.</li><li>Windows Smartphones are now outselling Blackberry devices, and 90% of them are being sold to consumers rather than businesses.</li><li>Despite pushing most of the cool new features onto Windows Mobile first, Microsoft still isn't abandoning CE devices just yet. Thanks for that.<br /></li></ol>Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-13079512783363887322007-05-02T18:11:00.000-04:002007-05-02T18:12:23.820-04:00Ok, I give in.Just call me a <a href="http://twitter.com/tlesher">sheep</a>.Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-62881593811253664702007-05-01T14:23:00.000-04:002007-05-01T21:36:06.144-04:00O Sole Mio...Ok, so I've missed <a href="http://us.pycon.org/TX2007/HomePage">PyCon</a> for the past three years running because of work conflicts. But at least this year they've been nice enough to send me to the <a href="http://medc2007.com/">Microsoft Mobile and Embedded DevCon (MEDC)</a> at <a href="http://www.venetian.com/">The Venetian</a> in Las Vegas.<br /><br />No "<a href="http://en.wikipedia.org/wiki/Stevenote">one more thing</a>" announcements from today's keynote, but we did get an earful of how Microsoft wants to position the Windows Mobile family: "it's not just for business anymore". Robbie Bach, the president of Microsoft's Entertainment & Devices Division, claims that Win Mobile-based smart phones are outselling Blackberry devices, and that 90% of those sales are actually to consumers (not businesses).<br /><br />That sounds great, but the massive emphasis on "all things smartphone" makes me a bit nervous. Microsoft is a smart optimizer when it comes to business strategies: what happens when their CE OEMs (like us) become 2% of their developer base, and their Windows Mobile Pro/Standard OEMs are 98%?<br /><br />Won't someone think of the poor headless CE devices? Just try using your fancy QVGA display in a freezer for a few hours...Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-72951006728232990462007-04-27T11:15:00.000-04:002007-04-27T12:09:19.044-04:00Your Memex is here. Are you using it?Microsoft Research has put significant effort into implementing a near-literal version of Vannevar Bush's "<a href="http://en.wikipedia.org/wiki/Memex">memex</a>" in its MyLifeBits project. I think we have the memex already: we just don't realize it.<br /><br />In 1945, the Atlantic Journal published "<a href="http://www.theatlantic.com/doc/194507/bush">As We May Think</a>", in which Vannevar Bush speculated that in the future, a machine--the "memex," or "memory extender"--would assist researchers by storing, indexing, and retrieving every piece of information they could possibly need. A user could also add his own text, images, or recordings, and could record notes and comments on the content. And it all fit within a large desk.<br /><br />This was strong stuff for the time: understand that the state of the art was the <a href="http://en.wikipedia.org/wiki/Harvard_Mark_I">Harvard Mark I</a>: a 50-foot-long, 10,000-pound, four-function calculator that could divide at the blinding speed of four operations per minute. To put it in perspective, Bush's prediction was made when my grandparents were not yet old enough to drive a car.<br /><br />Since 2002, Microsoft Research has been working on implementing <a href="http://research.microsoft.com/barc/mediapresence/MyLifeBits.aspx">MyLifeBits</a>, their version of the memex. And after five years of effort, they now have a <a href="http://www.sciam.com/article.cfm?chanID=sa006&colID=1&articleID=CC50D7BF-E7F2-99DF-34DA5FF0B0A22B50">one-user prototype</a> to show for their efforts. So don't expect to be shelling out for the Microsoft Memex anytime soon.<br /><br />But a few weeks ago, I had a realization. I went back to the original, 60-year-old article, and read over the description of the memex again:<br /><br /><blockquote>A memex is a device in which an individual stores all his <a href="http://books.google.com/">books</a>, <a href="http://www.google.com/notebook">records</a>, and <a href="http://mail.google.com/">communications</a>, and which is mechanized so that it may be consulted with exceeding speed and flexibility....<br /><br />It consists of a <a href="http://www.apple.com/imac/">desk</a>, and while it can presumably be operated <a href="http://www.tightvnc.com/">from a distance</a>, it is primarily the piece of furniture at which he works....<br /><br />In one end is the <a href="http://www.google.com/base">stored material</a>. The matter of bulk is well taken care of by <a href="http://www.tomshardware.com/2007/04/17/hitachi_7k1000_terabyte_hard_drive/">improved microfilm</a>. ...<br /><br />Most of the memex contents are purchased on microfilm ready for insertion. <a href="http://www.amazon.com/">Books </a>of all sorts, <a href="http://images.google.com/">pictures</a>, <a href="http://news.google.com/">current periodicals</a>, <a href="http://www.nytimes.com/">newspapers</a>, are thus obtained and dropped into place....<br /><br />All this is conventional, except for the projection forward of present-day mechanisms and gadgetry. It affords an immediate step, however, to <a href="http://del.icio.us/">associative indexing</a>, the basic idea of which is a provision whereby any item may be caused at will to select immediately and automatically another....It is exactly as though the physical items had been gathered together from widely separated sources and bound together to form a new book. It is more than this, for any item can be joined into numerous trails.... And <a href="http://googleblog.blogspot.com/2007/04/your-slice-of-web.html">his trails do not fade.</a><br /><br /></blockquote>There are more parallels, but that's a good start. <br /><br />So on the one hand, we have a research project to create a literal implementation of the memex that might exist sometime in the future, or a distributed, chaotic, mashup of individual technologies that together get about 90% of the way there today.<br /><br />Any bets on which will get there first?<br /><br />More importantly, what are you waiting for that isn't there yet?Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com1tag:blogger.com,1999:blog-9410449.post-54678645348495887582007-04-23T08:57:00.000-04:002007-04-23T09:00:33.484-04:00Slightly less perpetually behindAdam Rifkin says, "<a href="http://ifindkarma.com/blog/">Having a blog means feeling perpetually behind</a>." Well, today I'm slightly less behind. The Pipes have been blocked for a few months (you might notice that this happens every time a project at <a href="http://vocollect.com">work </a>goes into the home stretch), but now that pressure is off, and I'll be able to (finally) polish and post some items that have been languishing in the Drafts folder for a while.Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com0tag:blogger.com,1999:blog-9410449.post-37495808479059265602006-09-15T10:25:00.000-04:002006-09-15T10:59:03.502-04:00TurboGears decorator madness: linkifyOne of the neat features in TurboGears is the @jsonify decorator. It uses RuleDispatch to define generic functions to convert data model objects into JSON notation for use in AJAXish applications. For example, TurboGears provides this default converter for the User identity class:<br /><br /><pre>@jsonify.when('isinstance(obj, User)')<br />def jsonify_user(obj):<br /> result = jsonify_sqlobject( obj )<br /> del result['password']<br /> result["groups"] = [g.group_name for g in obj.groups]<br /> result["permissions"] = [p.permission_name for p in obj.permissions]<br /> return result</pre><br /><br />The first line lets the default JSONifier rules handle the object; after that, it removes the "password" field for security reasons, and then adds support for fields that the default rules can't handle (like joins). The @jsonify.when decorator handles mapping the default jsonify() function to the type-specific version, so when you want to return a User object converted to JSON, you just return "jsonify(myUser)" and you're done.<br /><br />This approach can be used for other purposes. For example, in one project, I kept running across is the need to render references to objects as links to view that object. For example, say you have a app that renders the text<br /><br />Last updated at 12:00 by Joe<br /><br />with the template snippet:<br /><br /><pre><p>Last updated at ${thing.last_update_time} by ${thing.update_user.display_name}</p></pre><br /><br />Easy and straightforward. But if you want to link "Joe" to Joe's user profile page, then every time you want to do this, you end up writing something like:<br /><br /><pre><p>Last updated at ${thing.last_update_time} by <br /> <a href="${'/users/%d' % thing.update_user.id}" <br /> title="User profile for ${thing.update_user.display_name}"<br /> ${thing.update_user.display_name}<br /> </a><br /></p></pre><br /><br />Then hours later you kick yourself because you find one place out of 20 where you made a typo in this monstrosity (see if you can find the one in the example above!).<br /><br />So I "borrowed" jsonify's approach and created linkify.py for the project:<br /><br /><pre>import dispatch<br />import model<br />import types<br /><br />from elementtree import ElementTree<br /><br /># Linkify generic methods... modeled after jsonify<br /><br />@dispatch.generic()<br />def linkify(obj):<br /> raise NotImplementedError<br /><br />@linkify.when('isinstance(obj, model.User)')<br />def linkify_user(user):<br /> link = ElementTree.Element('a',<br /> href='/user/%d' % user.id,<br /> title='User Profile for "%s"' % user.display_name)<br /> link.text = user.display_name<br /> return link</pre><br /><br />Then, in your controllers.py, you can make this available to templates:<br /><br /><pre># Add linkify to tg namespace<br />import linkify<br />def provide_linkify(vars):<br /> vars['linkify'] = linkify.linkify<br />turbogears.view.variable_providers.append(provide_linkify)</pre><br /><br />And now, in your template, you just write:<br /><br /><pre><p>Last updated at ${thing.last_update_time} by ${tg.linkify(thing.update_user)}</p></pre><br /><br />Much, much nicer.Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com2tag:blogger.com,1999:blog-9410449.post-1149854371709460642006-08-02T07:59:00.000-04:002006-08-02T13:50:06.943-04:00Yes, games ARE different.In the late 1990s, the game industry was starting to tackle "best practices". Basic techniques common in the rest of the software world (and mostly taken for granted now) weren't getting much traction in the game business. I often heard fellow game developers complain that those practices wouldn't work for games, because the industry was "just different." A decade later, I think they were right--but not in the way they thought they were.<br /><br />When will <a href="http://www.3drealms.com/duke4/">Duke Nukem Forever</a> be released? <span style="font-style: italic;">When it's done</span>. DNF is the most popular whipping boy for a game development schedule gone horribly wrong, but there have been plenty of similar examples (including the original Unreal). Game developers are notorious for scope creep and gold-plating, as well as massive mid-stream changes in design and architecture. Why is that?<br /><br />I've become convinced it's not <span style="font-style: italic;">solely</span> due to "unprofessionalism" or lack of effective project management. You see, if you're creating a business application, or a device driver, or control software, you can specify the requirements, from which you can create a design, from which you can create code, which you can trace back to requirements. It's not hard to write effective, measurable requirements for this type of software, which means it's not hard to justify or drop a given feature with objectivity.<br /><br />But games have a damnable, hidden, unwritten <span style="font-weight:bold;">Requirement Zero</span>:<br /><br /><blockquote><span style="font-weight: bold;">Thy Game Shall Be Fun</span>.</blockquote><br /><br />Requirement Zero is a real killer. Because it's not objectively measurable, it's not easy to manage. You can pick on Daikatana because of its schedule slips, or because you don't like its designer, but the bottom line was it just wasn't a fun game, and for more reasons than the obvious "sidekicks get squished by doors" bugs.<br /><br />I once had a conversation with Danielle Bunten Berry (of M.U.L.E. fame) and a colleague about a particular game. It was gorgeous, free of technical glitches, pushed the envelope on technology, and had a decent storyline. But my colleague summed it up this way: "There's a big hole where the fun should be".<br /><br />You see, users don't expect their word processor, or their spreadsheet, or the software that dispenses their soda at the local mini-mart to be fun. It's sufficient for it to be "fun-neutral": so long as it isn't so badly designed, defective, or poorly-performing as to be "un-fun", it's acceptable.<br /><br />You can create a game that completely satisfies the spec, is totally free of bugs, finishes on-time, stays within its budget, and has a flawless marketing program--and it can still be a miserable failure if it isn't fun.<br /><br />Yes, games ARE different.Tim Lesherhttp://www.blogger.com/profile/02469159188193434861noreply@blogger.com5