Archive for the ‘reviews’ Category

things i would want to know about erlang

August 10th, 2013

Earlier this year I spent a few weeks playing with Erlang. I wanted to make something out of it, but despite an encouraging start I found it too frustrating to use.

I got excited about Erlang because a lot of interesting things have been done in Erlang. Like CouchDB, RabbitMQ, Riak and so forth. Besides that, Erlang is a dynamic language and I generally find those quite nice to use.

I won't recap Erlang's selling points here - I assume you've heard them. These are my discoveries.

  • The language feels very static. Dynamic type checking and code reloading are about the only things that seem dynamic about it. There is no introspection. The build system is make. You can't change the structure of a record once the program is running. There is no meta programming. And so on and so forth. It really isn't a dynamic language at all in the sense of Python, Ruby or Javascript.
  • ...some of these static restrictions are plain weird. If you want to call a function in the same module you don't have to do anything special, but if you're going to export the function you need to declare it as an export as func/2 (where 2 is its arity), and maintaining consistency between the function definition and the export declaration is another source of bugs. The question is: what's the point of having this? The arity is a kind of statically enforced type, but you can still call the function with any arguments you want - you can even give all your functions a single parameter and then call it with a tuple and Erlang has nothing to say about what that tuple contains.
  • The data abstraction is very weak. There is no object system - all you have is records and they are compile time defined and provided to client code as header files, like in c. This is very rigid.
    If you don't like records you can just use tuples. Tuples don't have to be declared in a header file and compiled in, just start using them freely. But now you have a container with positional arguments and if you want to change the structure of the tuple you have to update all your code, because any code using this tuple has to pattern match against all of its fields. That's even worse.
  • The syntax is Prolog and that's not a good thing. I didn't even realize how much it is Prolog until I read a Prolog tutorial a few weeks ago. It may be alright for Prolog, but it has definitely not been extended in an elegant way by Erlang. The tuple syntax is probably the worst part of it, but control flow structures too are so easy to get wrong with so many different line termination characters in use.
  • Strings are just lists, and there's no way to detect whether a list merely contains integers or whether it's supposed to be a string. If you wanted to be very generous you could call this sloppy.
  • Single assignment may be a nice idea, but it makes code ugly. At first I was naming my variables things like Something = func(OriginalSomething), but then I realized I had to plan very carefully what I would name the variable to always have a descriptive name. So I abandoned that and started using Houses2 = func(Houses) and so on. That is a bit more flexible at least, but now it's like I'm maintaining a list in sorted order and if I discover that I need to add a step in between Houses3 = func(Houses4) I can either introduce Houses7 or I have to update all the indexes. This plain sucks. (And no: Haskell's solution of adding apostrophes is just a different numbering syntax, it doesn't make it any better.)
  • Dynamic typing + pattern matching can make a mess. I was looking at ibrowse, which is a very feature rich and mature library. But some of its function definitions are 3 pages long. Instead of splitting it up into several helper functions, it's simply defining the same function with many different clauses, each of which has a different signature (obviously), but also a different arity and sometimes expecting different types for those positional arguments.
    This is of no concern to the caller, because I just call the top level clause of this function and I don't see what happens behind the scenes. But what you actually end up with in the background is a call graph of essentially different functions (but with the same name), heavily recursive, that is hell to try to make sense of. And because there is no static type information, well good luck.
  • OTP is a straightjacket. OTP kind of assumes that you will be using a supervisor, that you will be using gen_server and that you will be using OTP code patterns, OTP packaging and an OTP directory layout. I found it quite hard to write a program in a single module just to try out an idea first. I would later have migrated it to OTP style, but it was hard to get the benefits of OTP without going all the way in. What is the point of a supervisor that supervises a single process without restart behavior? It's pure overhead. Yet in OTP everything is supervised, whether or not that's useful. Also, the lack of a proper data abstraction is nowhere more painful than in OTP where you're constantly passing nested tuples around and trying to figure out which part is used by the function you're calling directly, and which part is propagated as part of the message payload. If you happen to know the OTP API like the back of your hand then this is probably a non-issue, but you can say that about any awful API.
  • The tooling is low quality. It makes you think noone has made a real effort to improve it in a good 10-15 years. Like the REPL. There is one, but it sucks. It doesn't support readline. It has tab completion, but it doesn't complete all function names. You think you can explore the libraries this way, but some modules don't show up at all in the completer, or some of their functions are missing. The same goes for all the process introspection tools. They exist, but they've been programmed against Motif or something and they're terrible to use. And stack traces are formatted in a weird way that makes it hard to see exactly what the failing thing is. And if you use sasl then the actual output you care about it drowned out in a bunch of other output that you don't care about.

So yes, several very interesting ideas in Erlang, but a poor development experience. Unfortunately, most of these problems are deep in the language and the culture of Erlang and not likely to ever change. Someone could write a better REPL, but it would take a lot of community work to improve records or the syntax. There are other problems in Erlang that are probably more tractable, like a certain amount of duplication of functionality in the standard library (several competing dictionary implementations), but not even that seems to be worked on.

UPDATE: Erik Ridderby has written a response that includes a fascinating piece of history painting a nice picture of where Erlang came from and what sort of development environments it was used in.

ironpython + gtk

April 9th, 2011

nametrans_about_box I develop a bunch of tools for my own use that usually run in the terminal, for the simple reason that this maximizes their usability to me. It's also true that usually a terminal program takes much less code and effort to write than a gui program.

But occasionally I want to make something of mine usable to the so-called "end users" that we keep hearing about. So what I was wondering about was: What would it be like to add a gui to an existing python program and run it on .NET?

How do you find out? You try it. This is something that I've been meaning to try for a while anyway.

Pros:

  • 2.3mb download

Cons:

  • gtk# dependency
  • sluggish gui

Gtk

This is the first time I've used gtk. I've written one program in the past using winforms and it was quite painful, so I wanted to try gtk.

To build the gui I naturally use glade, which is the best gui design experience ever. I then have to find out how to connect the signals, write the handlers and all that stuff.

Mono's api docs on gtk seem quite complete, so you can usually find out what you need to know. I seem to remember looking at these in the past seeing nothing but "Documentation for this section has not yet been entered", so perhaps they have improved this recently. I have to say they do use the beloved javadoc model of having the left frame list all the classes, which makes navigation a pain. Api doc presentation is not really known to be a cutting edge field, it really could use work. Then again, the presentation of the docs can't help reflect the organization of the api to a large extent and if your api is organized as a kingdom of nouns rather than thematic modules then there's probably little magic the api doc guy can wield.

Gtk is not as discoverable as it could be, due to the fact that they have invented some terminology. A window resize event is not a ResizeEvent, it's an ExposeEvent. When I tried to discover this one I tried connecting all the signals that mention resizing in the api docs and none of them worked. Maybe this one belongs to a base class that wasn't listed in the api doc I was looking at? Could be. (You see how api docs matter?)

In general though, I get the impression that the more you know about gtk the more it makes sense, it seems a really well engineered toolkit. This, of course, in stark contrast to winforms, where the more you know about it the worse it gets. I don't know what gtk is like in c, but at least under .NET it doesn't have the manual memory management "feature" of winforms.

Gtk is hugely superior in many ways and that I don't think is even debatable. The layouting is great, internationalization support is painless, pango rendering is lovely and lets you use markup etc. One advantage that winforms has is the way it appears under Windows. But this I think is more a theming issue than a toolkit issue. Winforms looks nice under Windows and gtk looks mediocre. On the other hand, if I boot Mono's livecd with the latest mono on it, and I run the samples there, gtk looks lovely and winforms looks atrocious, even though this is ostensibly supposed to demonstrate how well both of them run on the latest mono. So I take it from this that simply not enough people care about gtk on Windows to make it look better.

For the python coder, you can apply the api docs for c# directly, because the binding exposes the same names to python. This is very helpful. There are also quite a few pygtk examples around the web you can find if you need to discover how to do something in gtk.

IronPython

Let's start with the positives. If you give it an existing python program and tell it where the standard library is, it will run your program. In this case I had to work around some platform nastiness and patch some code that's using os (throwing false positive OSErrors on file renames) that works fine on CPython/Windows but doesn't on IronPython/Windows. But in general it seems to work quite well.

Development experience

But if all you're going to do is write code using the standard library, there is absolutely no reason to use IronPython. How do you access .NET apis from Python? No, seriously, that's not a rhetorical question. IronPython ships with a number of assemblies, so where are the api docs for these? Are they browsable on the web? Are they viewable locally? The only thing I can see documentation wise is, next to IronPython.dll, something called IronPython.xml. It seems to be an aggregate of all the inline api comments in the source code. Is this it? Is this how you're supposed to view the api doc? IronPython 2.7 does something interesting. It ships a file called IronPython.chm, but only the .msi installer package includes this. As I'm sure the IronPython guys are well aware, the installer is Windows only and the .chm file too is Windows only (although I think there is something called kchmviewer that could read it).

In fact, the .chm file is basically the original Python 2.7 documentation (written in .rst, which compiles to html), renamed to IronPython 2.7, and includes a few additional pages about IronPython. Except that instead of throwing the html files on the web or including them in the download, they have compiled them into a .chm which they only give to the Windows people. And this is supposed to be a platform independent runtime? Is this a prank?

IronPython ships some examples, but it's not enough to figure out how to use the apis. To write a veritable IronPython gui program you need to produce an assembly, which has to be compiled with -target:winexe so that it does not show a terminal window on Windows. The simplest way seems to be to write a wrapper in c#, using the runtime hosting api, and execute your python in there. That's what the Pyjama project uses (they also use gtk#), so I was able to look that up. But when I needed to set up sys.argv and sys.version using the hosting apis before launching the python I had no idea how. How are you supposed to find this out without documentation?

It turns out that you need to do something like this:

ScriptScope scope = Python.ImportModule(runtime, "sys");
scope.SetVariable("version", "ironpython");

Easy, right? Yeah, when you know it, of course it is. But what if you didn't? How would you know that there is a function ImportModule in the Python namespace?

And how do you set sys.argv?

IronPython.Runtime.List lst = (IronPython.Runtime.List) scope.GetVariable("argv");

How would you know that IronPython.Runtime.List is the class that models a python list? You can use it like a python list:

lst.append("program.py");

In the end your best bet might actually be dir(). So how do you find out what's in the Python namespace when coding c#? Use python:

clr.AddReference("IronPython")
import IronPython
print dir(IronPython.Hosting.Python)

You could actually use this method to recursively dir() the assemblies and produce a map of what's in there, generating some kind of api doc in the end, but this is getting quite out of hand now.

At this point I was going to mention msdn and how that ought to host all the .NET related api docs you could ever want, but I see that the site is even worse than it used to be and I don't have the palest idea of where to find anything anymore. I can only imagine that if you install the latest Visual Studio Ultimate Premium Professional it will enhance your hard drive with untold gigabytes of xml compiled to a proprietary binary file format worth of api docs, which you can only browse in the proprietary api doc reader, which internally just renders html anyway. But actually, since IronPython was ousted by Microsoft and since it has the stigma of being an open source project, it could well be that it would be considered too tainted to include its api docs in the Visual Studio-installed doc browser.

As your final recourse, I suggest you warm up your grep, git clone this IronLanguages repo and hope for the best.

Debugging

When I try this somewhere in my hosted python program:

v = None + 1

this is what I get:

Unhandled Exception: Microsoft.Scripting.ArgumentTypeException: unsupported operand type(s) for +: 'NoneType' and 'int'

And the rest of the stack trace doesn't mention the python code at all. Better than "Segmentation fault", yes, but not by a whole lot. No filename, no line number.

This being the case I would strongly recommend developing your python with CPython first and then, once it's in good shape, build the gui so you can do most of your debugging under CPython.

Upgrade cycle

I mentioned IronPython 2.7 earlier, but funny thing, I've never actually tried it. The 2.7 release is compiled against .NET 4.0, which is not supported by the mono packages in Ubuntu. I thought that since I have everything working with .NET 2.0/IronPython 2.6 I could just throw in the newer assemblies, run my makefile and compile it against .NET 4.0, on a mono release recent enough to support 4.0 (such as the mono livecd I mentioned before). But no deal. And if you just install .NET 4.0 under Windows it doesn't come with a compiler or anything, so there's no way to try it.

IronPython + Gtk

So how responsive is an ironpython/gtk application, even a tiny one like this?

Start up speed on my Ubuntu system is something like 4s, which is just about fast enough not to notice that it runs on IronPython instead of CPython. But Ubuntu uses gtk natively, so the libraries are in memory. On Windows it varies from a warm start of 6s to 20s+, for a cold start. In the worst case the first hello message from python appears after maybe 5s, so the rest must be accounted for by loading gtk.

Once it's running, it can be a bit sluggish, in particular halting the program on Ubuntu sometimes seems to freeze the gui for a few seconds before it goes away. On Windows the issue (as usual) seems to be io. This program scans the filesystem whenever the input parameters change, which produces a list of files that is eventually displayed in the gui. But I'm not convinced that it's any slower than CPython/Windows.

sms is a garbage protocol

August 8th, 2009
  • no error on transmission failure
  • no failure on invalid recipient
  • no assurance of delivery to temporarily disconnected recipient

All of which amounts to the result that when you do send a text you have no idea if the number is correct, if the message went through, and in the event the person's phone is turned off if he'll ever see it.

Now, I really don't care if the flaws are in the protocol, the phones or the services we pay for; what I do know is that it all makes for an appalling technology whose reliability imitates Microsoft's infamous msn messenger where messages constantly go missing without notice. I hear all the time about messages I never saw and apparently when my phone is off for a few days there is zero chance of getting messages sent in that interval.

Compare it to email, which doesn't have any of these flaws, and it seems incredible that we are actually paying to use this.

And that's to say nothing of the horrendous user interface for typing messages on a phone to begin with.

This entry belongs in a comprehensive tome I have been writing in my head for a decade, detailing just how much cell phones suck. But the field is so vast that it overwhelms my ability to articulate it.

that thing about ruby

January 31st, 2009

Ruby is a great language, but one thing it needs is process. And what seems to suffer most from this is documentation.

  1. Ruby’s not ready
  2. Ruby 1.9.1 released

Hello .NET

January 8th, 2009

So I'm researching .NET/Mono for an application I might be doing for Windows. There are two important constraints:

  • The application shall not require installation privileges. (Or users in locked down environments might not be able to run it.)
  • The application shall be maximally robust with respect to runtime environment. (So that I can leave it behind with the best possible hope that it will work for years.)

The first rules out shipping an install bundle of any sort, and the runtime environment is presumed to be a vanilla Windows install. At most I can hope for a .NET runtime since it comes with XP-SP2 (I think) and certainly Vista and onwards. As for the second, I've considered things like py2exe, which bundles code with .dlls for Qt and whatever else the python code actually uses. While this seems to produce a self sufficient bundle (without requiring an install), how do I know if py2exe is going to be around in a few years when I may need to rebuild? And the Qt libs for that matter, they depend on OS-level libs that may change over time, that I have no control over.

So .NET seems to be the best only choice. Now, I obviously want to be able to run it on Mono as well, if for no other reason than to run it myself. So I have to stay clear of .NET-only bits.

That means hello Windows Forms (WPF is out, Mono doesn't support it). Now, I should say first off the bat that I'm not a gui coder. I don't like gui, it takes too much work to produce a really good gui, and it's way more hassle than I like to put up with. Nevertheless, I've seen a few of them. And Windows Forms... it's something else. On the face of it, the .NET library seems to be stocked reasonably, with classes for gui controls (System.Windows.Forms) and for drawing primitives (System.Drawing).

But then you look at some example code and you have to pinch yourself cause it's so unbelievable. I should preempt here by saying that all the gui toolkits I have seen in recent history handle layouting the same way: with layouts. You create a container that is, say, a grid, and put some widgets in it. The layout takes care of distributing the widgets in the container in a particular way. This takes care of resizing, because the widgets scale to the size of the container (and the containers, in turn, to the size of the gui). Containers can be nested to produce panels and so on.

Far be it for me to say that I like coding gui this way. It's a huge pain to set all this up. And it doesn't make resizing work well either, you have to put in a lot of extra effort just for that part. But at least it's a reasonably clean solution, and it's how Swing, WxWidgets, and I'm pretty sure Gtk and Qt as well, work.

Not Windows Forms. In Windows Forms you have containers, but there are no layouts. Everything is positioned with coordinates. As in here is a button, put it 5 pixels from the edge of the container; here's another button, put it in this location I have computed manually based on the first button. This has implications:

  • positioning is pixel based,
  • dimensioning is absolute - no resizing possible,
  • there is no rendering.

As it turns out, the gui form is basically a canvas you can paint on. Widgets and drawing mix freely on the same pane. But remember that Drawing api I mention? It's pixel based, no Qt or Cairo rendering for shapes or fonts. Neither does .NET have any kind of svg-based rendering that I know of.

The whole thing makes me feel like a time traveler. Remember that Windows Forms isn't a deprecated library, it's still actively used (and Mono just recently got done implementing it).

So why bring this up? It's good to remember sometimes that just because something is hyped a lot doesn't make it good. Remember, .NET 3 only appeared recently, in 2006. That makes Windows Forms the designated gui toolkit right up to that point. Swing, ugly as it may be to look at (and sluggish as it's always been) came out with Java 1.2 in 1998. (As a matter of fact, I think even AWT had layout managers in 1995.)