BuildBotIcon - Python Style (and it's got tests this time!)

If you've read earlier posts here, you might know that I'm behind BuildBotIcon, a small Java application that sits in the system tray, polling one or more BuildBot servers and reports any changes. It also plays sounds and turns lava lamps on or off.

I've decided to do a version in Python using PyQt instead. It's mostly as a fun exercise swim in the ecosystem of python (setuptools, unittest, mocking, coverage, yaml, etc) but it might come out better than the original.

PyQs nice, and the new-style signals-slots makes it a breeze to do what you want if, assuming you know your way around Qt. (Not that it's hard to learn, I just knew Qt pretty well before embarking on this project.)

Writing unittests are a breeze, and I think that it sure helps knowing that the code works, instead of knowing it only compiles. Coming from C++ this is actually pretty nice change. However, you should write with testing from the beginning. Grafting unit-testing later on is difficult and might not be a net win for all components, especially in C++ . (Mostly since you've run them for a while and the obvious (and thus easy-to-test-for) bugs have already been ironed out.)

However, the greatest fun so far has been using Mock, yet another mock object module for python that takes a nice and intuitive approach to mocking, IMHO. Instead of record/replay, or setup/run/verify, it's more like mock/run/assert. Read Mocking, Patching, Stubbing: all that Stuff (and it's link to Mocks aren't Stubs) for more information on this.

The ability of substituting any function or object in the APIs you're running makes unittesting immensely powerful and thorough. If you trust the framework (Qt in my case) it's very easy to verify that play() is called on the right QSound (without making noises) or that get() is run with the right URL to access the network.
Of course, more complex behaviours are trickier to mock properly, so it helps to refactor your app into testable parts (which you should do anyway).

Also, I've used YAML for the settings file this time (using PyYAML). All is good and well, except that it turns out that writing the settings file first and implementing the app second had probably been a better idea. TDD again.

It has all fallen out pretty nicely so far, except that, similar to the Java version, I've bungled together the backend that does networking and parsing with the UI. Since I currently have three ways to show build state (tray icon, sound and lava lamps) I should really had put a Listener interface in there. Fixing that now means refactoring the unit tests too, which is less fun. :-|

Nevertheless, all of this is experience is bound to pay back in my ThrustRacer-project (should I get it off the ground) or at work, where we're about to embark on a year of much Python coding. I don't expect either to get it right from the start, one never does. :)

More on HTML & JavaScript Client Apps

In a recent post I wrote about the new log window for our Qt apps at work. Essentially, it delivers the log via AJAX to a web browser, with the html & javascript embedded in and served statically by the app. Works well enough.

However, there are two more ships on the horizon flying similar flags.

The first is my (secret) plan to build a debugger for a State Machine for a few years. Since Qt now has a state machine as of 4.6, and it seems both solid, well documented and behaves mostly correct (observing that we've yet to learn all ins and outs of a full state chart machine) is seems a good target for this. I've already written a QStateMachine to GraphViz translator, which outputs an SVG which can (with some good faith and heavy squinting) be interpreted as a UML state chart (cough cough).

Now, SVG is an XML document which has a DOM, which you can manipulate with Javascript in most modern browsers. Yay! However,  this did not, at the time of having the idea, include Google Chrome (my favorite), since SVG-DOM updates didn't redraw the image immediately.... They have fixed it since, but I am pretty busy with other things so this small show stopper held me up. I hit a bump, I move in another direction, especially when I'm just going somewhere to see where I end up. Will have to get back on this one ... :)

Anyway, for the final product, I think a nice event log and some dynamic colouring/resizes of states and transitions would work wonders for debugging the app. Breakpoints via mouse-click events in the graph is then the second step. Then there are some question marks, but the last step is, as always, profit. ;)

The problem is always to package it so that other people can re-use it. I have to try to get better at that, but it's sadly not on top of the priority pile when working at a startup.

Secondly, it seems as the BuildBot project is planning to boot my Jinja webstatus implementation that replaced the html-string-pasting in python. (I never blogged about that.. oh well.. fun times it was anyway) Anyways, sad as that can be, the new plan is for... a html/javascript-only client! The content can then be served statically (via twisted or your local "real" webserver, for more oomph) and the webpage then communicates with the buildmaster using relatively small AJAX requests.

I think the idea is pretty neat, and has much in common with the things above, so I might go ahead and tackle that as well. Or procastrinate until someone else does it and steal^h^h^hborrow ideas from that place. Anyways, the jinja-rewrite made it much easier to see the actual HTML, so the new javascript (preferrably using jQuery) client could build on top of the existing layout.

Small steps in the right direction makes the world a better place. :)

Also, some day in March, if things go as they should, the household will gain a new member. One very small and in need of almost-constant care. I estimate I will not be able to contribute to open source for a few years starting then. Best do what I can before that. ;)

Teh build is fail!

Just to complete the gallery:


The responsible one has left the building but it's been pretty effective so far in getting people to fix the build a.s.a.p. :)

Also, the latest version of BuildBotIcon has optional sound, and currently we have Homer Simpson going 'DOH!' or 'Woohoo!' (rather loud). This makes everyone aware of it instantly.

(And yes, I've been reading lolcats too much lately..)

Lava lamps - up and running!

One picture, 1k words, etc:



We've also added a timer to turn them off during the night, along with the status monitor (which you can't see), to be nice to the environment.

Again, the USB board can be bought from Active Robots who were very friendly and helpful shipping the package to me, in spite of the efforts of our local post office. ;)

Lava lamp USB board assembly pictures

Here's some pictures of the assembly process. It went reasonably well, but making sure you have big enough drills beforehand helps a bit. :)

Here's how the board looks before assembly:



First, check that the circuit board fix in plastic box. Tight fit. :)



Drill holes for 230V mains (in/out) and USB cable: (Had to use a knife to make the hole for the USB-B connector fit, as it is slightly asymmetric. Luckily, the plastic was very easy to work with.)



Drilled for circuit board with USB connector in place (avoids misalignment):



Prepare cables:



Do the wiring (and remember to pull the cables through the holes in the box first):



Fit the wired circuit board in the box, make sure all cables go inside and put tie-wraps on the cables to secure against mechanical stress:



Here's the box with the lid on:



Ta-daaa. :)

As you can see, I currently only use one relay. It's of course possible to use all four, but that requires more drilling and wiring, and more lava lamp shelf space. :)

Always leave room for upgrades, I say. ;)

Lava lamp - 99.5%

Here's a tech-porn picture. :)

It's connected at office and it works, but I didn't have time for lavalamps.

Instead, I connected it to my office stereo system and set the computer to constantly play something from Tribute to Vengaboys when the build has failed.

Forget the carrot, it's all about the stick. :)

Lava lamps - 99% there

The usb-board and wiring is finally connected and safely encased inside in a box!

I'll go shop for lamps tomorrow and have pics up in a week or so. (Got much to do this weekend, as it's Midsummer holiday in Sweden.)

Lava lamps - soon

I just added Lava Lamp support to BuildbotIcon using the Phidgets USB interface kit. (Bought from Active-Robots UK)

Can't wait to have it going in the office. I've got the code running, but I need some casing and wiring for it to be safe to connect to the 220V outlet.

If you like it, click here:



Of course, it's nothing compared to Last.FM's setup but you have to start somewhere. :)

My BuildBotIcon app now hosted on BitBucket

I've uploaded my BuildBotIcon app to BitBucket.org!

It's a small java app that polls a buildmaster and changes the color of an icon in the java tray depending on the status. It also alerts when statuses change so you notice it a bit more.

I wrote it 1,5 years ago and published the jar-file on BuildBot's mail-list, but I decided to put it up as a proper project tonight, just for fun.

Bitbucket and Github are both greats hosting sites for open source projects. Very easy to use, very fast (compared to SourceForge) and they use DVCS:es both. I like Mercurial more, so I opted for Bitbucket.

On a related note, my work with patches for BuildBot on Github is apparently going fairly well, as I was just promoted to contributor with commit rights to the current 'official' BuildBot repository. (Dustin Mitchell's master branch).

On a related related note, Github's network feature is cool, if you like graphs. See mine here.

Oh, I also hang out a bit at StackOverflow.com.

Mercurial and python 2.6! :)

This is a follow up on my previous post 'Distributed Version Control Systems on Windows' where I just want to note that I've made some progress with Mercurial by using Python 2.6 instead of 2.5.2.

The good thing is that Python 2.6 uses MSVS 2008, which I do have available. So with that, I can use setuptools to compile python libs. Including Mercurial!

Compiling and install Mercurial into my Python 2.6 64-bit installation was dead easy, by the books all the way! Yay!

That will probably help with running buildbot/mercurial/trac on a pure windows-based server (as in no cygwin). But I'm considering using a linux box for this anyway, since it's smoother and more easy to add misc minor stuff to. (Not disregarding setting up ssh-accounts for mercurial users, which is what I've done currently.)

I'm currently using it to try to assist the Mercurial situation on BuildBot, which is lacking a bit, especially when it comes to in-repo branches. There are patches on the buildbot trac, but they haven't been applied and the crucial one won't be added until it gets a proper test-case.

So, I'm working on that. There's some issue with line-endings preventing try-patches to apply cleanly in the tests. (It works in real life, as I did use the try-functionality at my last job. Never got it ready for every developer though...)