2018 Review

Another year done, so let’s review! At a high level 2018 was a pretty great year for me. To despite a pretty big set back early in the year, by taking consistent action I finished pretty strong and managed to live up to a lot of my potential.

For 2018 I decided to make the theme “Action”. I chose this because in the past I felt like I spent too much time in analysis mode, and that prevented me from actually doing.

So this year ever project that I started was viewed from the lens of “how do I make this move forward”. The results were pretty good!

The bad stuff first

I don’t like to focus on the negative aspects of the past year. It is such a cliched and unproductive thing to do, yet everyone on social media will be the first to tell you how bad the year was.

Whatever. Bad shit happens to everyone from time to time. How did you respond to it? That’s what matters.

For me, the year started off rocky due to being laid off from my day job at the beginning of February. Not an ideal way to get things going, but it did open some doors I didn’t expect…

New projects, new ideas

As I started a job search it occurred to me that I should make a list of what my dream job would look like. Doing this lead me to (re)discover the idea of remote working.

I absolutely despise sitting in traffic. For me, it induces stress like no other activity can. Why should I do this if it causes me so much non-joy?

So I began to look for remote-first jobs. To help me do this I wrote a small web app that I quickly realized could be expanded to help others in a similar situation. Thus Remote Matcher was born!

Side note: I eventually found the perfect remote job for me! Sadly I did not find it though Remote Matcher. That would have been too perfect! 🙂

I continued to take action throughout the year and launched a few more projects. None of them turned out the way I wanted them to, but as was pointed out to me by a good friend: I produced a lot of things this year and learned a ton in the process.


As a family we decided that we needed to do a little more traveling now that the kids are older and can appreciate things more. As a result, there was a ton (for me at least) of travel over the summer. 3 major trips, and several smaller one helped us broaden our horizons a little bit.

This coming year will probably see more local/regional travel.

The goals for 2018

I started off the year by making several goals/objectives for me to achieve. As the year progressed I visited these often and in one case refined the goal a little bit to make it more actionable and measureable.

Here’s a quick run down of what this looked like:

Business revenue

One goal was to increase business revenue to the point that it was a stable secondary source of income. For a variety of reasons, this didn’t happen.

I learned a lot, but overall I am disappointed by my lack of progress here. Every day is a new chance, and I am determined to reverse this in 2019.


My health is becoming something I am paying more attention to as I get older. There’s a million benefits to this that I’m not going to go into other than to say this: Its easier to keep good health than to get it back.

To this end I significantly improved my exercising routine over last year. By the end of the year it was clear how I could take it to the next level, so that made setting a 2019 goal pretty straightforward.

(Of course right after I did that I caught the flu and was not able to do anything physical for the last few days of 2018. So 2019 hasn’t started as strongly as I would like, but hey that’s life. Adapt and move on!)

I also made a goal to meditate on a regular basis. This is one of those habits that gets dropped in time crunches, and after a year of reflecting on this I see it needs to be a priority.

Again, there’s a lot of reasons to do it, and for me they are pretty solid. Already this year I have been making the change to get mindfulness into my daily routine.

Quality time

One of the main reasons I chose to work remotely was to increase my time spent with my family. My kids are at the age where a little bit of guidance and presence will go a long way in shaping the people that they become as adults.

I’m happy to say that while I didn’t hit the metric I set for this goal, I did come really close. And this is one of those things where any work is good work, so overall I am really happy with this.


Overall I did pretty well with my goals. I didn’t hit the numbers, but I was pretty consistent throughout the year in working on them.

Also, having the theme of “Action” really helped me. A few times a month I would look at my goals and see the word “Action” at the top.

As corny as it sounds, seeing that and thinking about it got me to take action more times than no.

And let’s face it, action is what we need if we are going to accomplish anything meaningful in life.

Looking ahead

In 2019 my goals are still focused on the same areas, just with more specifics on the actions I need to take to achieve them.

I did add one more area, reading.

I have come to the realization that what I read makes a huge difference in my mood and motivation. So this year I am adopting the habit of reading for 20 minutes per day of non-junk information. My outcome for this is to become more curious and knowledgeable about the world around me.

So this year’s theme for me is: unstoppable.

In the past I have let things get in my way or even cause me to backtrack. This year, as I face challenges, I will tell myself that I am unstoppable.

Much like the theme of “action”, I look forward to being “Unstoppable” as I progress further on my journey to a better me.

Wrapping up: Take action

2018 was a great year. Don’t let others negative opinions infect your view of your world.

2019 is going to be a great year for me. I’ve already decided this, and I’m not going to let external voices change my mind.

Hacking on Hacktoberfest

Hacking on Hacktoberfest

Have you ever wanted to contribute to Open Source software but weren’t sure where to start?

Do you like winning T-Shirts?

Then I have something for you!

What is it?

Hacktoberfest is a month long celebration of open source software and specifically about contributing to the packages we all use.

It is a great chance to jump into open source and contribute back to it! It is a great way to improve projects or to learn a new skill. The best part is you don’t have to be an expert or a developer to make a contribution.

How can I help out?

It is super easy!

One misconception is that you have to be a developer in order to contribute. This isn’t the case! There’s plenty of need for documentation, testing, and just plain discussions.

This year I decided to try and learn a new language by working through an issue (or two) for Hacktoberfest. I have been really curious about
and JWT. I noticed that the Clojure library for JWT had some features that were missing.

As it happened, I also was able to do some Pull Requests for some work related libraries which was a really nice bonus!

How do I find things to work on?

If there’s any programs or libraries you use on a regular basis, I would encourage you to go check those out first. The issues tab and see if there’s anything there that looks interesting!

Some project will mark issues with special tags like “hacktoberfest” or “beginner” and those are great issues to look at first. They typically are things that would really help the project so if you can jump in there it would make the world a little bit better.

One of the lesser known things about GitHub is the powerful search engine. It can be used to find a lot of different things such as specific code chunks, or conversations about a topic, or even specific types of files!

We can use this to find all kinds of interesting and easy-to-work-on problems.

For example you can search for issues labeled “hacktoberfest” to see things that maintainers have specifically marked as being a good for contributing.

Searching pro-tips

Here’s a good base search to help get you started:


This search will look for any open issues that are labeled “hacktoberfest”. As I type this there are over 29,000 open issues!

To help narrow it down, try selecting a language from the list on the left side of the screen. This will narrow the list significantly to only the language you are interested in.

I used that to look for Clojure and Python issues which helped me get to interesting problems faster.

Bonus tip: If you are nervous about contributing code, many project also welcome documentation updates! Try searching for commonly misspelled words and you will find tons of repositories that you could make a pull request to.

Wrapping up

Open Source software is awesome. If you know it or not, you are probably using it right now.

Why not try and contribute to it? Its easy, and can be a lot of fun!

And winning a free t-shirt is always awesome. 🙂

A lite refactoring with PyCharm

Inspired by @levels I decided to try adding a Telegram integration to my current project RemoteMatcher. After seeing it work in production I decided to expand on the idea a little bit and do some refactoring with PyCharm.

Let me show you how this all evolved and how easy it was to make this into a reusable bit of code.

TL;DR; Here’s a video of me using PyCharm’s refactoring tool

Just getting it working

The initial challenge was to get my app to send messages to a private Telegram chat. Telegram has a reputation for being easy to integrate with because of its HTTP based interface.

For me this was pretty ideal: All I want is to post a message once my daily task runs. Since we can talk to Telegram via HTTP, there wasn’t really a need to use a full fledged library (and increase the dependencies for the project).

The hardest part of all this was getting the bot to post to the private room. Once I got that figured out, I was able to post things just using cURL. This is a good sign: It means we can use the regular urllib code from the Python stdlib.

Here’s a rough outline of the steps I took to make this happen: * Create a telegram account for myself * Create a private chat * Create a bot account * Add the bot as an administrator for the private channel * Use cURL to confirm the bot can post things to the channel * Translate the cURL code in to Python

Working first, then refactor

The part of my app I’m interested in runs only once a day, so after testing I needed to wait overnight to see if the results were what I was looking for.

Of course things weren’t 100% the way I liked them the first time, so I did some small tweaks (mostly formatting of the text) for the next run. After 2 days I was pretty happy with results!

My objective for this task was complete. I could now check the status of last night’s jobs without having to log into the GCloud logging interface.

Having this nice consoladated information delivered to my phone got me thinking, where else could I do this type of reporting? I quickly thought of 2 more places: new subscribers and unsubscribes!

But to do this… Did I really want to copy and past the same 6 lines over and over? Hell no!

PyCharm to the rescue

When I worked in Java one of the things I really liked about the Java ecosystem was the strong support for refactoring tools. IntelliJ has great support for this essential Software Engineering task, and thankfully PyCharm inherited many of these capabilities.

I was able to use the “Extract Method” feature to pull out that code into a function, and with a quick modification to the parameters I made a very reusable function.

I also used the “Refactor->Move” feature to rearrange the code a little bit and put my new send_to_telegram function to the util.py module. This cleanup move resulted in a cleaner looking module that is more focused on one task. (SOLID anyone?)

Showing, not just telling

The end result of all this?

Showing the result of refactoring with pycharm: a nice status report in telegram

The next morning I woke up to a telegram alert showing me that some people got emails with job leads last night!

It also gave me an alert that someone had unsubscribed after getting that email.


Although this was a pretty small refactoring in the grand scheme of things, it was great to see the PyCharm tools in action. Honestly, the hardest part of this whole thing was figuring out how to get the bot to post to the private channel. Everything else was really easy!

Here’s a video I made of me doing this fun little task:

Here’s a video of me using PyCharm to tackle this fun little task:

Wrapping up

I hope you liked this little peak into making a quick-and-dirty telegram integration to improve reporting!

If you are looking for a remote programming job, check out my free aggregator: Remote Matcher

Using functools.partial to refactor repetitive code

The other day a friend made a comment about iterative development and it got me thinking. In some situations this is a good approach to get things going, but there is a dark side to it: Crufty nasty code. Functions that we are fear to touch. Code that screams out to for a refactoring.

It got me thinking about the code I hacked together for Remote Matcher. It’s shiny and new, but does it have a dark side?

How bad could it be?

For this project I developed “iteratively”, and I decided I needed to stop and see what shape the code was in. In my views.py file, it definitely needed some attention, and not just because there were todo comments saying “THIS IS TERRIBLE. PLEASE CLEAN IT UP”. (I literally put that in the code. Twice.)

Here’s a quick enumeration of the sins of this code:

  • Repeated strings (like, we check for a string, then go and use that string again on the next line)
  • A bunch of elif statements that grows every time a new data source is added
  • There are several long constants that get import‘ed (and they will grow every time a source is added)
  • The same 2 functions are called over and over, but with slightly differing parameters
  • Although this file is called views.py it sure looks like there’s business logic that’s starting to leak into the functions… even though we have a dedicated module that is supposed to handle that logic!

And that’s in just 25 lines of code.


Clearly things need to change. I have 2 new sources I want to add to the system and the thought of that causing that function to grow at least 4 lines really made me mad.

The strings could be consolidated, but that wouldn’t help with the leaking of logic, or the growth of the if statement. Usually I’m ok with a little bit of repetition in code, but at this point we clearly spiraling out of control. I kept thinking if I could get this code into a dict and then do a lookup I could probably help get this code under control.

As I thought more about this I had a flash of insight: I could use Python’s functools module to help with the function invocation!

I decided to take a swing with the approach and it worked! Rather than try to explain what I did, I made a video showing my approach. Here’s me walking and talking my way through this refactoring:

Parting thoughts

Although the total line count didn’t go down tremendously in the video, the code in the views.py file is on the path to getting more streamlined and having less of business logic laced into it.

The root cause of this was me hacking on it to “just get it working”. Since I knew I was going to have 2 similar but different data sources I didn’t put a lot of thought into “correct” software architecture principals early on. Thankfully I revisited this code before it got too nasty.

So, the moral of the story: revisit your code and look for opportunities to simplify and consolidate things. That and Python’s functools module is pretty awesome! A lot of things like partials sound like magic, but when you need them they work perfectly.

Testing AppEngine cron jobs locally

Lately I’ve been doing a lot with Google AppEngine. It has a lot of great features, but to get those you need to give up a few things. Sadly I discovered that included the ability to locally run “protected” API endpoints. At least until I discovered this one strange trick to make everything work…

The setup

So AppEngine applications need an app.yaml file that defines a lot of things needed to run the code. It is also defines the routing for the app’s endpoints, and who is allowed to access them. (Basically either administrators, or the whole world)

My app is making use of the cron.yaml file to periodically ping certain endpoints in the app. The catch is that I don’t want just anyone hitting those endpoints, a bad actor could hammer that sensitive endpoint and kill my API access.

Did someone say "Bad Actor"?
Did someone say “Bad Actor”?

Thankfully, Google recognized this and allows you to setup endpoints in the app.yaml file with a login: parameter. Setting this to “admin” tells AppEngine that only logged in users who have admin rights to the domain are allowed to hit that end point.

Yay! I don’t have to write any custom login/user management code. But….

The problem

If you are running the code locally, say doing development, you are probably going to need to hit those end points to make sure the damn thing is working. Right?

Well, the dev_appserver.py script doesn’t know about who is and isn’t logged into Google… because it is only running on localhost! Therefore having the login set to “admin” means you will never be able to access that endpoint.

Boo Hoo, HTTP 302 for you.

So, what do we do? Commenting out the login: field will let you access it locally, but what if you accidently deploy that into production? (Spoiler alert: You are :screwed:)

Run to the console

Although dev_appserver.py is the cause of our problems, it also turns out to be the solution too!

When dev_appserver.py boots, it not only starts your app, but it also starts a lightweight admin app too. This app by default runs on localhost:8000 and provides all kinds of useful tools like a DataStore viewer and… a cron utility!

Going to localhost:8000/cron brings up a page that lists all of the (AppEngine application) registered cron jobs, what schedule they are setup to run on, and…. wait for it… a…. button to kick off that job!

Yes, by clicking on that button the admin console will trigger your cron job for you so that you can run and see the results locally! Yay for debugging locally not in production!

Other tricks

The admin console is pretty awesome and has lots of other useful tricks up it sleeves. Here’s some of what I use it for:
* Doing quick checks on entities stored in the DataStore
* Faking incoming XMPP and SMTP messages (I’ve never tried this, but it looks pretty cool for one off testing)
* A memcache viewer/editor
* An interactive console

That last one is pretty sweet. Since I can’t seem to startup an IPython terminal AND connect it up to my app, this is the next best thing. From the webpage it will let you type in some Python code and it will execute it for you.

Perfect for those times when you just want to delete all of your entries because you had a horrible misspelling in one of the field names.

Not that I’ve ever done that.

If you are curious to see the app I built using AppEngine, check out RemoteMatcher! It is a remote job aggregator that scans a bunch of job sites and only emails you the ones that match your interests. No more scanning tons of boards, instead just check your inbox for the best matches.

Running a daily mailing list with Python and MailChimp

So I’m a really big fan of Stoic philosophy. I really like the way it prepares us for troubles in life, and I thought it would be really cool to have a daily email to go out and give you a shot of Stoic inspiration for the day. And since I liked it, why not start a mailing list and share this others?

The first step was to go to MailChimp and setup a mailing list. Getting people on to your mailing list is a huge topic and I won’t really go into detail here but if you’re interested to learn more tweet me at @nloadholtes and let me know and I’ll whip up a post for you. (Here’s the list if you want to join it)

The next thing that was needed was to organize these quotes into a way that was usable. I’m using an a Google spreadsheet because it’s just really easy to put stuff there. Simpler to maintain than a database, this choice turned out to be a pretty good move! There are python libraries that can easily manipulate these spreadsheets.

My (basic) Workflow

Every Sunday evening I would go and go through my list of quotes in the spreadsheet. I usually just did a sort on the “date_used” column and then I choose a quote that I have not used in a long time and set that into an email template that would go out on a given day.

Doing this is an extremely manual process. In the beginning I could be done fairly quickly, taking about 20 minutes. But after a while that got very old and there were a few days where I actually missed setting up the emails because I just didn’t have the time or energy on Sunday night.

Another problem that I ran into was a human error. When you are copying and pasting from a spreadsheet into a separate window of a web browser, it’s very easy to lose track of which quote you’re working on and what day is supposed to go out. Additionally there was a weird mental stress that popped up, but more on that later.

Putting this process into a script seem like a very obvious way to make my life easier.

Python + MailChimp API = <3

The script I wound up writing randomly chooses 5 older quotes that have not been in the past 90 days. It takes those quotes and generates an API call to MailChimp for each one to create a email campaign, one for each weekday. As it chooses a quote, it updates the “date_used” cell with the date we are going to publish the quote on. Here are the things you need to make this happen:

  • A MailChimp account (free is fine)
  • A google spreadsheet with quotes (See this example sheet, and make a copy!)
    • An API key for access to that spreadsheet. (You will need read and write access, see this documentation for details)
    • The “key” id for the spreadsheet (this is the long string in the URL of the spreadsheet, 
  • `pip install mailchimp3 gspread` to get the Mailchimp library and the GSpread library

With those pieces, you are ready to rock! Here’s what my code looks like:

This code is little hacky because I threw it together slowly over several months. At first, I was just getting the quotes and printing them to the screen. Then eventually I modified it to start posting them to MailChimp. The most recent change makes a dump of the quote data into a JSON file that I then feed into another script that handles posting to Facebook. (Let me know if you want a post about that)

How it works

The MailChimp and Google credentials are read from environment variables, but the spreadsheet key and a few other things are hardcoded. This is just how I did it, ideally those hardcoded things should also be parameters or env variables. (Translation: Don’t do what I did there!)

The main method gets a “start” date parameter from the command line. (This script is assuming it will generate 5 days worth of quotes at a time, which is my normal cadence.)

That date is then passed into the get_quotes() function which eventually returns a list of dicts containing the quotes for the week.

That list is then serialized for another script to use, or if I need to do a re-run of this with the same data. The list is then iterated through, and each “day” in the list is fed into the create_campaign() function which generates the email.

The final step is having the email scheduled for delivery on the appropriate day.

After this runs you can log into your MailChimp account and see the emails all scheduled for delivery:

And at this point, everything is set! I have found MailChimp to be very reliable and the scheduled emails have gone out without a hitch for over a year.

Some numbers

As of this writing I have 109 people on the mailing list. MailChimp has some very generous quotas for the free level, I have yet to bump into them with this list.

Another thing I like about MailChimp: The reporting page is pretty nice and straightforward for these types of campaigns. Your numbers may vary, but this is what I see when I look at these reports:

Considering this is a small list on a very niche topic, and is running on a free plan, this is pretty nice!

Earlier I mentioned the time savings. Before my script, the MailChimp portion of this was taking about 20 minutes to do manually. Now that it is automated, all I have to do is type in the correct date and run the script. That normally takes about 30 seconds to complete. 🙂 At this point I should probably create a cronjob and just use that to kick off the process automatically every Sunday.

Another interesting thought: before I used to stress a little about picking the “right” quote for the day. By handing this responsibility to Python’s random.sample() function, I no longer worry about this. Instead I too get the pleasant surprise of seeing a random quote every weekday.

Quick Note: I haven’t done the cronjob YET because I still haven’t fully automated the Facebook script that cross posts these quotes. Once I get that “fixed” the whole process will become hands off.