Are tweets the new .plan?

Back in the good old days, there was the .plan file. You could use the finger command to ping a server and it would return the contents of the .plan file. The awesome folks at id Software were big users of this service.

It was so very cool to get a behind the scenes peek at what was going on with a group of people making a top notch game. Other developers began doing this also, and soon there was a nice little ecosystem of plan updates flying around. This was all pre-RSS.

Over time the .plan file fad kinda fizzled out and died. But now with the rise of twitter, it seems like it could see a come back. True, 140 characters is not a lot of room to explain certain technical issues, but John Carmack’s tweets are a step in the right direction. I’m glad he’s back.

RSS Extension for Chrome

I started using Chrome as my web browser a few months ago. Firefox has been good to me, but Chrome has that hunger to take over the browser market so I figured I would give it a spin. Especially since there are now good extensions for important things like flash blockers.

But my one major gripe has been the seeming lack of RSS support in Chrome. In Firefox, Safari, and almost every other browser out today if you visit a page that has an RSS feed for it, you will see a little RSS icon (usually in the address bar). Clicking on it will take you to your preferred RSS reader where you can add in that feed.

However in Chrome, there did not seem to be any native support for doing this. It turns out its an extension you have to install.

Once you install the RSS Subscription Extension you will get that RSS icon in the address bar and be able to subscribe to the feed for that site.

Mercurial talks to Git (and has for a while now)

This is really awesome: A mercurial plugin that allows you to interact with git repositories

It has apparently been around for a while, here’s a blog post explaining how to use hg-git

I see lots of stuff going on with git, there’s a lot of projects using it and github is a hotbed of activity. It has some really neat tools like gist, and all of the visualizations of the project stats and I think that is a large part of its appeal. As a mercurial user the closest site we have is bitbucket which is nice, but lacks some of those features.

This hg-git bridge finally pushed me to sign up for a github account. I honestly don’t know why I didn’t get one before, if nothing else it is a great thing to have because of the gist feature. I have some ideas for some small python projects, maybe I’ll give my github account a test drive with one of those.

I’m sure I’ll stay with my bitbucket repositories for a while, but it is nice to know that there are options out there. Especially from CVS which I have come to dread working with again. Svn really doesn’t bother me that bad, but the flexibility of DVCS systems like hg and git are beginning to spoil me…

Python function overloading via multimethods

One of my favorite features of OO languages like Java is the ability to overload a method, which is where you have several methods with the same name, but they take in different types in their arguments.

I love the python language, but as I blogged about a while ago, not having overloaded methods in python is kind of a bummer. But do you have to trade your dynamic freedom in for static types to gain this? It turns out that with a little ingenuity, you can get that functionality using the generic dispatch pattern and python’s decorator syntax.

The example on the above wiki page is nice and concise, but I think that Alex Gaynor’s blog post about multimethods is the best example of how to implement this useful idea. In leiu of an explanation of how it works, Alex gives us a unit test that exercises the mutimethod class and the dispatch/overloading idea that it brings to life. As long as you know what a decorator is the code does make sense.

There are other examples of using a dispatch, but I feel that Alex’s is more clear. These are of course no substitute for having function overloading built into the python language, but having the functionality wrapped up in a nice neat class like this is a very good thing.

How to identify what to measure

My last post about metrics really got me thinking. Most GTD programs don’t really talk about improvements from a dead stop, they are more concerned about changing your process. But altering how you implement your process can have a huge impact.

Here is a suggestion on how to identify what to measure:

1. What is causing you pain?
2. What causes the pain to occur?

Step 1 is all about prioritization. Pain is bad. If something is causing you pain, you want to stop it. You need to stop it. So pick the thing/item/problem that is causing you the most pain. This is what you need to improve or fix.

Step 2 is looking at what is causing the pain to occur in the first place. Once you know the factors that are contributing to the pain, you can try to change them. If you are doing the right thing, the factors contribute less, and the pain should go down.

For example: Let us say I want to write more python code. My day job doesn’t involve much python, so I feel like I’m being left behind because all of the cool kids on reddit are talking about the fun they are having writing python code.

The pain: Not being able to write in the language I want to. The cause: Day job doesn’t use python.

Addressing the cause will affect the “pain”. If you get a different job (e.g. one that involves writing python code) then you have solved both problems.

But what if you can’t just switch jobs? Then you try experimenting with writing python code for your own personal use. Try writing it to solve scripting problems on your PC at work. If that makes you feel better, then you are moving in the right direction. If you try writing open source code in python in your after work hours, but it winds up stressing you out, then you are moving in the wrong direction. (Or you’ve identified that there might be a different problem than what you originally though there was!)

By tackling the sources of your pains, and applying small changes there, you can get a measure of where your efforts are going. Sometimes these measures are going to be cold hard numbers, and sometimes they are just going to be “feelings”. But either way, they are indicators of how you should change your processes and habits in order to become more successful.

Metrics: The kick in the ass that is the key to productivity

What you can measure, you can manage.

I just watched a new video from Giles where he talks about how you can improve your programming productivity. Its a really good short video that hits the nail on the head. If you want to make a change, guessing about what to fix won’t cut it. You need to measure what you are doing, and then adjust accordingly.

The quote at the top was in the video, and I have to say I don’t think I had ever heard it before. But as soon as I heard it I knew it was 100% correct. Think about it: everyone who is in charge of things in you life (bosses, teachers, parents, etc.) track what you do to some extent. When you step out of line, they know about it and are able to let you know. Why? Because they are “measuring” you progress via grades, work done, chores completed, etc.

So if it works for them, why can’t it work for you? Track the things that are important to you, and see if you can make a positive change.

I’ve been hemming and hawing lately about tracking things like my programming projects or seeing if my neighborhood association really is increasing the house values. The time for action is now.

My first step: Putting widget on this blog to track my BitBucket RSS feed. If I’m going to work on a project out in the open why not let everyone know about it? This way if I’m not being productive, it will be pretty visible.

(As a side note, this is something that Giles has mentioned before that I really believe in: If you are a programmer, you should have some project out in the public eye. Open source is a good thing. Contributing to open source is a great thing. Being known as a programmer who contributes to open source software is the best thing.)

My next step: Start treating time tracking on my projects as a first class citizen. I’m starting a new sprint tomorrow: I’m going to track my time better. Also, I’m going to add some tasks to my Satellite Tracking project so I can make sure I’m on task when I’m working on it.

Thanks Giles, that video was a good kick in the ass.

Stargate Universe: I’m not the only one not watching

I’m a huge fan of SG-1.

I own all 10 seasons on DVD. Even the last one, which *really* wasn’t the greatest.

When Stargate Universe was announced I was skeptical. I wasn’t big on Stargate Atlantis, but I decided that the new series deserved a fair shake.

So I watched the first half of the season. It went from “Not bad” to “Ehhh” to “*sigh*” to “Why bother” pretty quickly for me. Since it was taking a 4 month break between the first and second half combined with me not really caring about the major characters, I never watched the second half.

Today on the most excellent Stargate site, Gateworld.net, they released some ratings numbers. It looks like I wasn’t alone in my decision to tune out. Check out this nifty plot I made of the ratings over time:

*sigh*…..

Linking against Python on the Mac

Today while working with swig I kept running into problems trying to get python to read the generated output files. The error I finally wound up with was:

Fatal Python error: Interpreter not initialized (version mismatch?)

Abort trap

Fatal Python error: Interpreter not initialized (version mismatch?)

Abort trap

gcc myfile.c -lpython -shared -o mylib.so

gcc myfile.c -framework python -shared -o mylib.so

Subversion and Mercurial in the same place

In a previous post I talked about trying to use Mercurial (hg) in a Subversion (svn) world. Its been almost 2 months, and I’ve learned quite a bit about hg and how to use it in a svn based environment.

At the time it seemed like the obvious choice was to get something (hgsubversion) that would let hg talk directly to the svn server, but still allow me to explore DVCS workflows. It turns out that this isn’t the case. Luckily there are several different approaches that can be used.

I coulda been a contender…

The idea behind hgsubversion is to use hg as your version control tool. This extension lets it “talk” to svn servers, so you only need one tool. It sounds like a really great idea, but I ran into some stumbling blocks that while not necessarily show stoppers, they did cause me to look around for alternatives (more on that later). Here’s the issues, good and not-so-good, that I had with this extension:

• Poor support in cygwin – Installing this on cygwin was a bitch-and-half. Getting all of the pieces to compile and install was a major act that I just couldn’t pull off. Which is a bummer, I do love me some command line. However, TortoiseHg to the rescue!
• It works great on the mac – I usually have the most trouble installing new stuff on my mac, but hgsubversion works like a champ with no problems.
• TortoiseHg rocks - This program is amazing. It worked correctly right out of the box, and took no effort to get up and running on my windows machine. It really let me get started on this little experiment.

Overall I really like using hg. Making it work with svn has been a big bonus because now I can get the best of both worlds without having to abandon svn. (That’s a big bonus as most day jobs rely on svn!)

The iPad and the German Tank Problem

Based on order numbers the day the iPad was made available for pre-order, several sites were speculating that Apple was selling about 50,000 iPads per hour.

This reminded me of something I heard about back when the iPhone was just getting started: The German Tank Problem.

Its basically a way of statistically estimating the maximum number of “items” that have been produced based on the serial numbers of the item. This method works when the serial numbers are in sequential order, and basically allows you to produce a somewhat realistic estimation (as opposed to a wild-ass guess).

Estimation is one of those skills that most people (myself included) could stand to improve a little bit. In the wikipedia link above, this is shown by the intelligence estimates the Allies had for the number of tanks the Germans were producing in the middle of WWII. The initial estimates were really high (1,000 per month), but using statistics based on the serial numbers of crankcases from captured or destroyed German tanks showed that the number might be lower (around 200 per month, or 1/5th of the original. After the war when the factory records were looked at, the true number was a lot closer to 200 than 1,000.

Math FTW

The formula is pretty easy:

N=m + (m/k) -1

Where m is the largest serial number observed, and k is the number of serial numbers seen. The variance is roughly equal to:

(N^2)/(k^2)

Which means that the standard deviation is roughly:

N/k

So, how does this apply to the iPad’s initial orders? Based on the data points of 2 known orders spaced 50,000 numbers apart (keeping in mind these order numbers also probably included orders for items other than iPads). So plugging those two numbers into the equation we get N=

>>> m=50000
>>> k=2
>>> N = m + (m/k)-1
>>> N
74999
>>>

Wow, almost 75,000… That seems like a really big number. The question we should then ask ourselves is “How realistic is this number?” Using the standard deviation and variance we could find out how spread out our numbers are:

>>> (N**2)/(k**2)
1406212500L
>>> N/k
37499
>>>

Wow. Those numbers huge. And that is a very bad thing. The bigger the standard deviation and variance, the more less accurate the estimation. Another way to approach this analysis is to look at the confidence interval and see how big it is. The wikipedia article has a handy formula for finding the confidence interval which leads us to the estimation that there are (based on k=2 and m=50,000) between 50,000 and 225,000 iPads ordered!

Those numbers are lying

Why? There’s two reasons: The main one is that our sample size (of 2 orders) is waaaaaaaay too small. Its like trying to guess how big your grocery bill is by averaging the price of two items, and then multiplying it times the number of things you bought. That estimation will be way off.

As an example, if there were 20 orders to work with (k=20), the high end of the confidence interval would shrink to 58,000. But that leads to the second reason why we can’t trust these numbers:

We don’t know the lower bound.

In other words, yes, there could have been 50,000 orders between the first and last data points. But what if only half of them were for iPads? That would mean that m is actually 25,000 which would drastically skew the numbers down. Remember that N that was almost 75,000? With m at 25,000 (and keeping k = 2) N drops to 37,499 which is half the original estimate!

So how many were sold?

That’s a really good question. Knowing Apple and how people love their products, I bet they sold a TON of iPads. But based off these rough numbers we see in the news, we can’t really draw a good conclusion. We can get a couple of estimates which are better than nothing, but they are so numerically shaky (huge standard deviation and enormously questionable confidence interval) that they strain believability. More data points will help establish a upper bound (i.e. the estimated maximum number sold), but without a lower bound to keep us grounded the numbers will still look really huge.