I develop a bunch of tools for my own use that usually run in the terminal, for the simple reason that this maximizes their usability to me. It’s also true that usually a terminal program takes much less code and effort to write than a gui program.

But occasionally I want to make something of mine usable to the so-called “end users” that we keep hearing about. So what I was wondering about was: What would it be like to add a gui to an existing python program and run it on .NET?

How do you find out? You try it. This is something that I’ve been meaning to try for a while anyway.

Pros:

• 2.3mb download

Cons:

• gtk# dependency
• sluggish gui

Gtk

This is the first time I’ve used gtk. I’ve written one program in the past using winforms and it was quite painful, so I wanted to try gtk.

To build the gui I naturally use glade, which is the best gui design experience ever. I then have to find out how to connect the signals, write the handlers and all that stuff.

Mono’s api docs on gtk seem quite complete, so you can usually find out what you need to know. I seem to remember looking at these in the past seeing nothing but “Documentation for this section has not yet been entered”, so perhaps they have improved this recently. I have to say they do use the beloved javadoc model of having the left frame list all the classes, which makes navigation a pain. Api doc presentation is not really known to be a cutting edge field, it really could use work. Then again, the presentation of the docs can’t help reflect the organization of the api to a large extent and if your api is organized as a kingdom of nouns rather than thematic modules then there’s probably little magic the api doc guy can wield.

Gtk is not as discoverable as it could be, due to the fact that they have invented some terminology. A window resize event is not a ResizeEvent, it’s an ExposeEvent. When I tried to discover this one I tried connecting all the signals that mention resizing in the api docs and none of them worked. Maybe this one belongs to a base class that wasn’t listed in the api doc I was looking at? Could be. (You see how api docs matter?)

In general though, I get the impression that the more you know about gtk the more it makes sense, it seems a really well engineered toolkit. This, of course, in stark contrast to winforms, where the more you know about it the worse it gets. I don’t know what gtk is like in c, but at least under .NET it doesn’t have the manual memory management “feature” of winforms.

Gtk is hugely superior in many ways and that I don’t think is even debatable. The layouting is great, internationalization support is painless, pango rendering is lovely and lets you use markup etc. One advantage that winforms has is the way it appears under Windows. But this I think is more a theming issue than a toolkit issue. Winforms looks nice under Windows and gtk looks mediocre. On the other hand, if I boot Mono’s livecd with the latest mono on it, and I run the samples there, gtk looks lovely and winforms looks atrocious, even though this is ostensibly supposed to demonstrate how well both of them run on the latest mono. So I take it from this that simply not enough people care about gtk on Windows to make it look better.

For the python coder, you can apply the api docs for c# directly, because the binding exposes the same names to python. This is very helpful. There are also quite a few pygtk examples around the web you can find if you need to discover how to do something in gtk.

IronPython

Let’s start with the positives. If you give it an existing python program and tell it where the standard library is, it will run your program. In this case I had to work around some platform nastiness and patch some code that’s using os (throwing false positive OSErrors on file renames) that works fine on CPython/Windows but doesn’t on IronPython/Windows. But in general it seems to work quite well.

Development experience

But if all you’re going to do is write code using the standard library, there is absolutely no reason to use IronPython. How do you access .NET apis from Python? No, seriously, that’s not a rhetorical question. IronPython ships with a number of assemblies, so where are the api docs for these? Are they browsable on the web? Are they viewable locally? The only thing I can see documentation wise is, next to IronPython.dll, something called IronPython.xml. It seems to be an aggregate of all the inline api comments in the source code. Is this it? Is this how you’re supposed to view the api doc? IronPython 2.7 does something interesting. It ships a file called IronPython.chm, but only the .msi installer package includes this. As I’m sure the IronPython guys are well aware, the installer is Windows only and the .chm file too is Windows only (although I think there is something called kchmviewer that could read it).

In fact, the .chm file is basically the original Python 2.7 documentation (written in .rst, which compiles to html), renamed to IronPython 2.7, and includes a few additional pages about IronPython. Except that instead of throwing the html files on the web or including them in the download, they have compiled them into a .chm which they only give to the Windows people. And this is supposed to be a platform independent runtime? Is this a prank?

IronPython ships some examples, but it’s not enough to figure out how to use the apis. To write a veritable IronPython gui program you need to produce an assembly, which has to be compiled with -target:winexe so that it does not show a terminal window on Windows. The simplest way seems to be to write a wrapper in c#, using the runtime hosting api, and execute your python in there. That’s what the Pyjama project uses (they also use gtk#), so I was able to look that up. But when I needed to set up sys.argv and sys.version using the hosting apis before launching the python I had no idea how. How are you supposed to find this out without documentation?

It turns out that you need to do something like this:

ScriptScope scope = Python.ImportModule(runtime, “sys”);
scope.SetVariable(”version”, “ironpython”);

Easy, right? Yeah, when you know it, of course it is. But what if you didn’t? How would you know that there is a function ImportModule in the Python namespace?

And how do you set sys.argv?

IronPython.Runtime.List lst = (IronPython.Runtime.List) scope.GetVariable(”argv”);

How would you know that IronPython.Runtime.List is the class that models a python list? You can use it like a python list:

lst.append(”program.py”);

In the end your best bet might actually be dir(). So how do you find out what’s in the Python namespace when coding c#? Use python:

clr.AddReference(”IronPython”)
import IronPython
print dir(IronPython.Hosting.Python)

You could actually use this method to recursively dir() the assemblies and produce a map of what’s in there, generating some kind of api doc in the end, but this is getting quite out of hand now.

At this point I was going to mention msdn and how that ought to host all the .NET related api docs you could ever want, but I see that the site is even worse than it used to be and I don’t have the palest idea of where to find anything anymore. I can only imagine that if you install the latest Visual Studio Ultimate Premium Professional it will enhance your hard drive with untold gigabytes of xml compiled to a proprietary binary file format worth of api docs, which you can only browse in the proprietary api doc reader, which internally just renders html anyway. But actually, since IronPython was ousted by Microsoft and since it has the stigma of being an open source project, it could well be that it would be considered too tainted to include its api docs in the Visual Studio-installed doc browser.

As your final recourse, I suggest you warm up your grep, git clone this IronLanguages repo and hope for the best.

Debugging

When I try this somewhere in my hosted python program:

v = None + 1

this is what I get:

Unhandled Exception: Microsoft.Scripting.ArgumentTypeException: unsupported operand type(s) for +: ‘NoneType’ and ‘int’

And the rest of the stack trace doesn’t mention the python code at all. Better than “Segmentation fault”, yes, but not by a whole lot. No filename, no line number.

This being the case I would strongly recommend developing your python with CPython first and then, once it’s in good shape, build the gui so you can do most of your debugging under CPython.

Upgrade cycle

I mentioned IronPython 2.7 earlier, but funny thing, I’ve never actually tried it. The 2.7 release is compiled against .NET 4.0, which is not supported by the mono packages in Ubuntu. I thought that since I have everything working with .NET 2.0/IronPython 2.6 I could just throw in the newer assemblies, run my makefile and compile it against .NET 4.0, on a mono release recent enough to support 4.0 (such as the mono livecd I mentioned before). But no deal. And if you just install .NET 4.0 under Windows it doesn’t come with a compiler or anything, so there’s no way to try it.

IronPython + Gtk

So how responsive is an ironpython/gtk application, even a tiny one like this?

Start up speed on my Ubuntu system is something like 4s, which is just about fast enough not to notice that it runs on IronPython instead of CPython. But Ubuntu uses gtk natively, so the libraries are in memory. On Windows it varies from a warm start of 6s to 20s+, for a cold start. In the worst case the first hello message from python appears after maybe 5s, so the rest must be accounted for by loading gtk.

Once it’s running, it can be a bit sluggish, in particular halting the program on Ubuntu sometimes seems to freeze the gui for a few seconds before it goes away. On Windows the issue (as usual) seems to be io. This program scans the filesystem whenever the input parameters change, which produces a list of files that is eventually displayed in the gui. But I’m not convinced that it’s any slower than CPython/Windows.

• no error on transmission failure
• no failure on invalid recipient
• no assurance of delivery to temporarily disconnected recipient

All of which amounts to the result that when you do send a text you have no idea if the number is correct, if the message went through, and in the event the person’s phone is turned off if he’ll ever see it.

Now, I really don’t care if the flaws are in the protocol, the phones or the services we pay for; what I do know is that it all makes for an appalling technology whose reliability imitates Microsoft’s infamous msn messenger where messages constantly go missing without notice. I hear all the time about messages I never saw and apparently when my phone is off for a few days there is zero chance of getting messages sent in that interval.

Compare it to email, which doesn’t have any of these flaws, and it seems incredible that we are actually paying to use this.

And that’s to say nothing of the horrendous user interface for typing messages on a phone to begin with.

This entry belongs in a comprehensive tome I have been writing in my head for a decade, detailing just how much cell phones suck. But the field is so vast that it overwhelms my ability to articulate it.

Ruby is a great language, but one thing it needs is process. And what seems to suffer most from this is documentation.

1. Ruby’s not ready
2. Ruby 1.9.1 released

So I’m researching .NET/Mono for an application I might be doing for Windows. There are two important constraints:

• The application shall not require installation privileges. (Or users in locked down environments might not be able to run it.)
• The application shall be maximally robust with respect to runtime environment. (So that I can leave it behind with the best possible hope that it will work for years.)

The first rules out shipping an install bundle of any sort, and the runtime environment is presumed to be a vanilla Windows install. At most I can hope for a .NET runtime since it comes with XP-SP2 (I think) and certainly Vista and onwards. As for the second, I’ve considered things like py2exe, which bundles code with .dlls for Qt and whatever else the python code actually uses. While this seems to produce a self sufficient bundle (without requiring an install), how do I know if py2exe is going to be around in a few years when I may need to rebuild? And the Qt libs for that matter, they depend on OS-level libs that may change over time, that I have no control over.

So .NET seems to be the best only choice. Now, I obviously want to be able to run it on Mono as well, if for no other reason than to run it myself. So I have to stay clear of .NET-only bits.

That means hello Windows Forms (WPF is out, Mono doesn’t support it). Now, I should say first off the bat that I’m not a gui coder. I don’t like gui, it takes too much work to produce a really good gui, and it’s way more hassle than I like to put up with. Nevertheless, I’ve seen a few of them. And Windows Forms… it’s something else. On the face of it, the .NET library seems to be stocked reasonably, with classes for gui controls (System.Windows.Forms) and for drawing primitives (System.Drawing).

But then you look at some example code and you have to pinch yourself cause it’s so unbelievable. I should preempt here by saying that all the gui toolkits I have seen in recent history handle layouting the same way: with layouts. You create a container that is, say, a grid, and put some widgets in it. The layout takes care of distributing the widgets in the container in a particular way. This takes care of resizing, because the widgets scale to the size of the container (and the containers, in turn, to the size of the gui). Containers can be nested to produce panels and so on.

Far be it for me to say that I like coding gui this way. It’s a huge pain to set all this up. And it doesn’t make resizing work well either, you have to put in a lot of extra effort just for that part. But at least it’s a reasonably clean solution, and it’s how Swing, WxWidgets, and I’m pretty sure Gtk and Qt as well, work.

Not Windows Forms. In Windows Forms you have containers, but there are no layouts. Everything is positioned with coordinates. As in here is a button, put it 5 pixels from the edge of the container; here’s another button, put it in this location I have computed manually based on the first button. This has implications:

• positioning is pixel based,
• dimensioning is absolute – no resizing possible,
• there is no rendering.

As it turns out, the gui form is basically a canvas you can paint on. Widgets and drawing mix freely on the same pane. But remember that Drawing api I mention? It’s pixel based, no Qt or Cairo rendering for shapes or fonts. Neither does .NET have any kind of svg-based rendering that I know of.

The whole thing makes me feel like a time traveler. Remember that Windows Forms isn’t a deprecated library, it’s still actively used (and Mono just recently got done implementing it).

So why bring this up? It’s good to remember sometimes that just because something is hyped a lot doesn’t make it good. Remember, .NET 3 only appeared recently, in 2006. That makes Windows Forms the designated gui toolkit right up to that point. Swing, ugly as it may be to look at (and sluggish as it’s always been) came out with Java 1.2 in 1998. (As a matter of fact, I think even AWT had layout managers in 1995.)

Audio in linux, how to put it into words? How about: oss, alsa, pulseaudio, esound, arts, portaudio, jack, gstreamer, phonon. Did I miss any? Embarrassment of riches? Or just embarrassment?

I will not rehash history any more than to say that between buggy/incomplete drivers for sound cards and the wonderful world of alsa I’ve never been able to understand how the hell audio works beyond getting output and, sporadically, input. I am the quintessential dumb user of linux audio, even though I have tried to figure it out.

But let past be past. Ubuntu 8.10 Intrepid. Pulseaudio ready and everything, since 8.04. Why Ubuntu decided to plug in pulseaudio without setting up any gui controls for it is beyond me. All I know is that occasionally sound output will stop working (I get 2 seconds of output and then it stops) and then “pkill pulse” cures it.

Let’s run through the list.

alsamixer

I’ve learned to use alsamixer to fix low level audio problems. It is reliable in that it gives me all the channels on my sound card, so I always use it to mute/unmute my microphone and to untangle the master/pcm/headphones/front speakers settings caused by mixers that only allow me to control the volume of one channel and apps that output on god knows what channel.

That’s alsamixer on a Ubuntu 8.10 desktop with all updates installed as per today. Here’s the same thing on my laptop.

Same distro, same version, all updates installed. And I have made no conscious choice for this to come about, all I did was install the pulsaudio gui tools. All of a sudden, alsamixer is useless as it only has access to a single “master” channel. It would appear now that pulseaudio is sitting between the sound card and alsa, but does that sound right?

pulseaudio gui

pulseaudio is here, might as well use it, right? Ubuntu ships a bunch of pa* packages that are gui tools for pulseaudio (not installed by default). This one is called pavucontrol. It lets you set volume per audio stream. I love this, I’ve wanted to have it since forever.

But then it comes to output and what do I see?

A single output device. Where is my master, my pcm, all my channels? What’s more bizzarre is that pulseaudio says it’s connecting to alsa while alsamixer says it’s connecting to pulsaudio. Surely that way madness lies?

What’s worse, pavucontrol is a gui app without any kind of systray integration, so much as I would like to use it as an easy-to-reach mixer, I can’t.

kmix

I run kmix in my tray, it lets me set the volume by scrolling the mousewheel over the icon, which is exactly what I want. It’s not a terribly impressive mixer, as it only lets me select one channel to control, whereas what I really would like is to be able to lock several output channels into one, otherwise I’m never really in control. So the best I can do is set the volume to max on master, and tell kmix to show me pcm. That seems to work well enough.

It appears I have two identical channels, but the leftmost is misnamed, that’s actually Master.

Now, let’s see. alsamixer doesn’t show any of these channels, because it delegates to pulseaudio. So kmix connects to… god knows what, but at least I can access these channels.

gnome-volume-control

Gnome’s volume manager is an interesting one. It has a combo box for devices. I’m showing here all the devices related to output and how channels are connected.

The good news is that the channels in the default mixer correspond to those in kmix (sweet sanity!). But then there is that bottom mixer with a channel of the same name as the single output channel in pavucontrol. And yes, they are the same (and the same as the single output in alsamixer).

But this channel appears to be some kind of composite output channel that has no bearing on any of the output channels in the top mixer (or in kmix). (Ie. while it might appear to be a composite pipe that aggregates the flow of all these standard channels, it’s actually a pipe that sits after all of them. If pcm volume is 0, this channel won’t receive any input, and will be mute no matter its volume setting.)

So neither can I effect the settings of master/pcm/etc with this channel nor vice versa. That means if I want to use this single channel as my master volume control, I have to make sure that absolutely no application can mess with the volume for any of the standard master/pcm/etc channels. Good luck with that.

A sane spec

I know this sounds crazy, but what’s it gonna take to get a single server+mixer applet that

• captures every possible audio stream, no matter the audio server/api it’s on,
• prevents applications from messing with master volume controls (eg. mplayer),
• has a single virtual channel controlling all the underlying output channels, so that I can have a single master volume slider,
• has per stream/app volume controls pulseaudio style,
• integrates well into the systray,
• integrates with laptop hotkeys

?

UPDATE: Ubuntu users can now vote on a proposal to unify sound systems on ubuntu brainstorm. Hopefully this idea and the related ones can gain some traction with Canonical to attack the problem.