The last time I did this, the version of Django was 0.9 and today’s version is 1.3.1. As a I started work on the site I discovered some things here and there that had changed in the years in-between those version. The one that bit me the hardest was this: I could view the main page (/), but any other page (including index) would give me a 404 error.

This was really weird because the mappings existed in the urls.py file, but a 404 error means it couldn’t be found. If it was a programmatic error you would expect a 500 error, but I never saw one.

Slowly but surely I tore the system apart looking to see what could cause mapped urls to disappear in Django. Eventually I discovered the root cause was how static resources are handled.

One of the changes in the new versions of Django was to add functionality for handling static resources. The biggest change (at least as far as my ancient code was concerned) was in introduction of the STATIC_URL variable. According the documentation this is the prefix for static resources (like css, javascript) that are referenced by the web page that is being built by Django.

In my laziness many years ago, I just plopped the main CSS file (and everything else) in the root of the web directory so that in the html pages didn’t have to have paths on them. So when I was adding the STATIC_URL variable to make sure my CSS file would actually load, I set the variable this way:

STATIC_URL = ‘/’

…because it wouldn’t let me put an empty string of ”. Well, it turns out that this was wrong wrong wrong. Django was looking at the incoming requests as I tried to visit the pages of the site and checking the static directory to see if there was anything there (which there wasn’t, just my CSS file).

Once I realized this, I changed it to:

STATIC_URL = ‘/static/’

…which basically forces you to create a directory to hold your static files. Which is right right right right. Basically Django was forcing me to clean up my act and code the site in a much cleaner manner. As soon as I did that all of the 404 errors went away and I was able to hit every page in my urls.py file.

I was pretty pleased with myself that I knew a lot of the things listed, and I was really happy when I ran into some nuggets that I knew nothing about. For example, I did not know that you could do this in Python:

1 < x <10

To me, that is really awesome because not only is what you would write in a mathematical sense, but it also is much more concise and what you would see in a pseudo-code representation of a program. That is the power of Python, taking an idea (a computer program) and presenting it in a readable manner. Being able to use the above type of expression just takes us further down that path of readability and that is a good thing. And yes, you can write crappy code in Python, but at least you have the choice: many languages simply have one way to do things. Choices are good.

And speaking of choices, I thought this feature was very cool, even if I would never use it:

from __future__ import braces

That allows the use of {} instead of white space in a Python program. I know some people can’t stand the indention rules of the language and feel more comfortable when they are “braced” (for impact?), but the fact the language has features to be able to turn this on? That is 1 million pounds of awesome in  a 5 pound sack.

There is a ton of other neat tricks there, go and read up and see what you can do to make your code work more for you! Choice is a great thing. With Python, you have lots of great choices you can make. And if you decide to make no choices, you still wind up with pretty readable code which is always a good thing.

If you are going to go down the Linux route with AWS (and honestly why wouldn’t you?), then you are in luck since python ships on most modern distributions. But, while python is “batteries included” there are some really nice projects out there that it is good to take advantage of. Installing these projects is where you start to find out there are differences in the EC2 AMI’s.

My first attempt was with the default Amazon AMI. It seemed like a good choice, but the more I played with it, the more I realized that it was pretty bare-bones. easy_install is included, but when running it to try and install something you run into permissions problems.

Because this is supposed to be a nice experiment, I only played around with it for a hour or so. My patience exhausted, I went looking for an alternative and ran right into the perfect match: ActiveState has an AMI specifically for python developers

This is awesome because ActiveState puts out a really nice Windows package for Python (and other lesser languages). I fired up their AMI instance and have found it to be really nice. I put my python project out there earlier this month and it ran without a problem… until the code crashed because I forgot to put a try catch block in place.

Seriously though, I really like this AMI. Not only is a nice python distribution there, but they also included gcc so if you need to compile something, you don’t have to go far. The easy_install script runs like a dream, and I haven’t had any problems with it at all. In fact, my whole experiment (which seems to getting attention mostly from blog aggregators at the moment) has only cost me a total of $0.19. Yes, only 19 cents! And 1 cent of that was from the ActiveState Python AMI (the other 18 cents was from my playing around with the Amazon AMI that wasn’t that great).

Now to get back to my ideas, and sling some python code out into the cloud…

But of course, I’m not the best or most interesting developer on twitter. Here’s a lit of developers I’m follow because they post really interesting stuff:

• John Carmack – id Software, creator of Doom, Quake, and lots of other awesome stuff
• Ryan C. Gordon (icculus) – Awesome dude, ports a lot of games to linux
• Ted Dziuba – He’s got opinions, and he isn’t shy about sharing them
• Zed Shaw – An old-school hacker, he’s always up to something interesting

There’s a ton more out there, go and check it out!

It is pretty easy to do! First clone the project to your local system:

hg clone http://hg.example.com/coolproject

You now have a local copy. Then go and log into your bitbucket account. Create a new repository, for this example lets call it coolproject-fork. To push your local copy to BitBucket you can do this:

hg push http://bitbucket.org/your_username/coolproject-fork

Then your local copy will be pushed out to BitBucket. But… what do you do if you want to pull from the original and push to your private fork? Doing an hg out (from your local copy) will show that it is trying to push to http://hg.example.com, not BitBucket. To fix this, you have to modify the hgrc in the project and tell it the out path is different than the in path.

Go into the .hg directory in the coolproject directory and open up the hgrc file. Make these changes:

[paths]

default = http://hg.example.com/coolproject

default-push = http://bitbucket.org/your_username/coolproject-fork

Now you can do hg pull to get updates from the original project (on example.com) and when you do hg push it will push your changes back to BitBucket!

It has taken me a while, but I have learned through experience that just because something is new, it doesn’t mean you should jump into it.

A lot of people don’t realize the value that an established process/technology provides, that stability can be priceless. For example, people who jumped whole-hog into JavaFX are probably starting to regret that choice.

Now having said that, there is a break even point, the older a project is, the more one should look at specific fixes for specific problems. Switching from Java to .NET? Not important. Switching from COBOL to Java/.NET/Ruby? Important. Changing as few variables at one time as possible? Vitally important!

Version Control really seems to be finally getting the attention it deserves lately and that is a great thing. Disciplined open source projects have been setting a great example for developers and businesses alike for some time. As a result, a lot of them are using “older” technologies like CVS. A few years ago there was a flood of people breaking away from CVS to use Subversion, and these days there’s a push to go to distributed version control solutions like Git or Mercurial.

This great article talks about one group who made that decision and the work they did to ensure their history (and build-able version of older releases) were ported over into the brave new world of Git.

I am very impressed with the level of maturity the project leaders showed with that conversion: They did not just drink the git-workflow kool-aide, instead they decided to change only one thing: the version control system.

Side note: One of the major turn offs for me about the Git community was their instance on adopting their workflow model when using the git tool. While I welcome new points of view and ideas, a while ago it seemed like it was “git’s way or the highway” when it came to using that system. Mercurial on the other hand seemed a little more open to the typical svn/cvs way of doing development. That is one of the reasons why I personally chose to work with Mercurial. In the time since then my personal workflow has changed a little bit to be more like the DVCS approach. In time I might become less of a svn developer and more of a DVCS developer. Having said that, I still don’t care for the Git staging area.

My hat is off to them for keeping their project under their control while modernizing a significant piece of infrastructure. My hope is that Postgres will stand as an example of how to grow without starting over. Once the developers, testers, users, and other stakeholders get used to the new version control system, then they can make the next move (whatever that may be).

I think that down the road a few years this will be seen as an important decision by the project leadership, in the event of a bug being found, they will be able to look at the history and see when it entered the system and at the  same time be able to reap the benefits of the DVCS revolution.

Specifically, I’m taking an existing code base (the AIAA SGP4 code which is written in C/C++), wrapping it with python, and then eventually putting a web front end on it. The end goal is that astronomers will be able to use this code/website to check their observations to see if there was a satellite in their field of view when they took their measurements.

Since I’m using a known library in to create something new, this seemed like a really great opportunity to use some Test Driven Development (TDD) to ensure that as I build up the parts of the system, the numbers stay true and that the code stays honest.

This is working out really well. Combining python’s unit test module with nosetests allows me to quickly bang out a test to make sure that the positional numbers that the SGP4 code generates is what the documentation says it should be. By using TDD I can map out where the code needs to go a little better: As I find something that is missing I can write out a quick test that will fail. When I run nosetests, I have that failure glaring at me telling me where more code is needed.

For example, I have one set of test data that gives the observer’s location in standard latitude and longitude coordinates. My code isn’t quite expecting this, the MPC format is a little bit different. This is a problem that is easy to fix, but I have a plethora of problems at the moment and I need to make sure this one gets taken care of properly. So I opened up the python file that tests the Observatory class and added in a failing test that called testLatLongToXYZ(). When I run the tests I now see that this is a failing method which gives me the mental kick in the pants to go and implement that method, even if it is just a stub.

Why do I say just a stub? Because putting in the stub lets  me do two things: 1) I’ve now got something there, and 2) I can now update the test to call that stub. The secret is that the test should now call the stub and expect it to pass, but the stub should always return something (None, False, -1, etc.) to indicate that it failed.

This acts as a pointer to an area that needs improvement in the code. This is what TDD is all about. You keep repeating this process until you converge on a “correct” answer which is a method that does just what it needs to do. In theory, and so far my practice confirms this, the resulting code should be smaller and more accurate.

For me this is a great thing because it encourages code that is more honest. What is honest code? To me, it is code that does what it says and nothing else. The shorter the code is, the more focused it is. The python language is very expressive and allows you to do a lot without saying a lot. This power can lead you to try and do too much in one function. Once you have a function that not only converts Lat and Long, but also converts polar coordinates and writes satellite data to the database, can you really honestly call that function convertLatLongToXYZ()?

This is especially valuable in the scientific context of this astronomy related code. The code is open source, and in the event there is a problem with it, as people dig into the code they need to have confidence that the code is relatively well written. At a high level the tests give them the numerical sense of ease that it is working when the tests run and show the proper numbers being returned. At a low level, seeing that the code is broken up into small logical sections gives the confidence that once  problem is found it can be corrected, tested, and shown that there are minimal side effects in other parts of the program.

As the lone developer on this project, this type of check-and-balance gives me a lot of hope that I’ll be able to produce something that is accurate and useful to the community at large. Openness and honesty help build confidence and trust. If you are interested in the project, be sure to check it out:

ObsSatId: A python astronomy project to wrap the SGP4 code to check for satellites

It was so very cool to get a behind the scenes peek at what was going on with a group of people making a top notch game. Other developers began doing this also, and soon there was a nice little ecosystem of plan updates flying around. This was all pre-RSS.

Over time the .plan file fad kinda fizzled out and died. But now with the rise of twitter, it seems like it could see a come back. True, 140 characters is not a lot of room to explain certain technical issues, but John Carmack’s tweets are a step in the right direction. I’m glad he’s back.

It has apparently been around for a while, here’s a blog post explaining how to use hg-git

I see lots of stuff going on with git, there’s a lot of projects using it and github is a hotbed of activity. It has some really neat tools like gist, and all of the visualizations of the project stats and I think that is a large part of its appeal. As a mercurial user the closest site we have is bitbucket which is nice, but lacks some of those features.

This hg-git bridge finally pushed me to sign up for a github account. I honestly don’t know why I didn’t get one before, if nothing else it is a great thing to have because of the gist feature. I have some ideas for some small python projects, maybe I’ll give my github account a test drive with one of those.

I’m sure I’ll stay with my bitbucket repositories for a while, but it is nice to know that there are options out there. Especially from CVS which I have come to dread working with again. Svn really doesn’t bother me that bad, but the flexibility of DVCS systems like hg and git are beginning to spoil me…

I love the python language, but as I blogged about a while ago, not having overloaded methods in python is kind of a bummer. But do you have to trade your dynamic freedom in for static types to gain this? It turns out that with a little ingenuity, you can get that functionality using the generic dispatch pattern and python’s decorator syntax.

The example on the above wiki page is nice and concise, but I think that Alex Gaynor’s blog post about multimethods is the best example of how to implement this useful idea. In leiu of an explanation of how it works, Alex gives us a unit test that exercises the mutimethod class and the dispatch/overloading idea that it brings to life. As long as you know what a decorator is the code does make sense.

There are other examples of using a dispatch, but I feel that Alex’s is more clear. These are of course no substitute for having function overloading built into the python language, but having the functionality wrapped up in a nice neat class like this is a very good thing.