dynamic or lexical, that is the scope

October 2nd, 2008

Apologies for the awful pun on a 16th century action movie.

Do you know how in the movies, when someone has to testify they first pin his hand on a Bible and make him recite that I swear to the tell the truth, the whole truth, and nothing but the truth, so help me God litany? Presumably, the god they're talking about is the god in the book, that's why the book is there (I bet polytheists find this very helpful). I guess they think it's harder for people to lie after taking a pledge while handling a Bible. (Do we have any statistics on whether that works?)

Anyway, in a dynamic scope, there is witness called Python. He will make his pledge based on the book that they happened to shove under his hand that day. One day it could be the Bible, a week later it could be The Gospel of the Flying Spaghetti Monster. So that means the pledge will be somehow relative to the god in that particular book. Uneasy about one god, very comfortable with another one.

In a lexical scope, there is a witness called Perl. He is very emotional about his first time as a witness. And even though they give him a new book every time, he just can't seem to notice. He makes his pledge based on the very first book they slipped him.

And now for a short digression into the world of programming languages. You have two scopes, one is the innermost, the other is the outer scope. There is a variable in the inner scope, but it's bound in the outer scope. How do you evaluate this variable? There are two answers to this question.

Under dynamic scoping the variable gets the value that it has in the outer scope at the time of evalution. Under lexical scoping the variable gets the value that it has in the outer scope at the time of declaration.

That didn't explain anything, did it? I know, read on.

Who cares?

This is an important question, and people rarely seem to ask it. Functions care. Named functions and unnamed functions like lambdas, blocks, closures (different languages have different names for the same thing). Anything that has a scope and can be passed around, so that's only functions.

So all the blah about lexical scoping really just boils down to one little detail that has to do with how functions are evaluated. Hardly seems worth the effort, does it?

Dynamic, baby

Dynamic scoping is the more intuitive one. You grab the value of the variable that it has when you need to use it. That's what's dynamic about it, today this value, tomorrow another.

Consider this Python program. It prints the same string in three different colors. output is the function responsible for the actual printing. output has an inner helper function colorize, which performs the markup with shell escapes. Now, since colorize is defined in the scope of output, we can just reuse those bindings. I pass the string explicitly, but I don't bother passing the color index variable. (A variable gets interpolated where there is an %s).

def output(color_id, s):

    def colorize(s):
        return "\033[1;3%sm%s\033[0m" % (color_id, s)

    print colorize(s)


for e in range(1, 4):
    output(e, "sudo make me a sandwich")

Lexical ftw

Lexiwhat? If you recall from last time, "lexical" is a pretentious way of saying "where it was written".

What this implies is that the outer binding is evaluated only the first time. After that, whatever scope the function finds itself being evaluated in, it doesn't matter, the variable with an outer binding doesn't change value.

Consider this Perl code which is exactly the same as before.

sub output {
    my ($color_id, $s) = @_;

    sub colorize {
        my ($s) = @_;
        return "\033[1;3${color_id}m${s}\033[0m";
    }

    print colorize("$s\n");
}


for (my $e = 1; $e < 4; $e++) {
    output($e, "sudo make me a sandwich");
}

How do you think it evaluates?

Oh no, it's broken! Why? Because the first time colorize is evaluated, the value of ${color_id} is recorded and stored for all eternity. The term lexical in this example isn't helpful at all, because the function is *always* evaluated where it was declared, it's not passed to some other place where the value of ${color_id} could have been decided by someone other than output. 'pedia says lexical scoping is also called static scoping, which makes more sense here.

Interestingly, in the language of tweaks that is Perl you can replace my with local on line 2 and you got yourself dynamic scoping! The code will run as expected now.

Which is better?

I don't know. I don't have any conclusions yet. I got into the habit of writing inner functions in Python without passing in all the arguments, it's useful sometimes when you have a lot of variables in scope. And then I got in trouble for doing the same thing in Perl.

In languages without assignment, they will obviously pick lexical, because it reinforces the rule of referential transparency. A variable assigned always keeps the same value.

You need lexical scoping to have closures. A function being defined has to be able to capture and store the values of its unbound variables. Otherwise you could pass it to some other scope that doesn't have bindings for variables with those names, and then what?

But you know what? Python has closures anyway. Here, colorize is defined inside output, but it's not called. It passes back to the loop, and it's called there. But that scope doesn't have a binding for color_id! And yet it still works.

def output(color_id):

    def colorize(s):
        return "\033[1;3%sm%s\033[0m" % (color_id, s)

    return colorize


for e in range(1, 4):
    f = output(e)
    print f("sudo make me a sandwich")

If you try the same thing with Perl and local in place of my, and set $color_id to $e, it works too.

So at least for Python and Perl, you can't reasonably say "dynamic scoping" or "lexical scoping". They do a bit of both. So why is that? Are the concepts dynamic and lexical just too simplistic to use in the "real world"?

:: random entries in this category ::

7 Responses to "dynamic or lexical, that is the scope"

  1. [...] dynamic or lexical, that is the scope By numerodix Consider this Perl code which is exactly the same as before. sub output { my ($color_id, $s) = @_; sub colorize { my ($s) = @_; return "33[1;3${color_id}m${s}33[0m"; } print colorize("$s\n"); } for (my $e = 1; $e < 4; … numerodix blog - http://www.matusiak.eu/numerodix/blog [...]

  2. Brian says:

    Hmm, I'm not sure Perl's named subs are really lexically scoped. Subs get stuffed into the global or package namespace when the code is parsed, before any of it even starts to run.

    foo();
    sub foo { print "foo\n" }
    foo();
    sub foo { print "another foo\n" }
    foo();

    This prints "another foo" three times (and throws a warning about redefining `foo`, if you `use warnings`). You can call a sub before it's lexically defined in the code. I don't know what the proper term for this is. Package-scoped? Un-scoped? If it was lexically scoped, I'd expect the first to throw an error and the second to print "foo" if we ever reached it.

    Nested subs in Perl aren't actually nested in any sense; they still get stuffed into the same outer namespace.

    sub foo {
    sub bar {
    print "bar"
    }
    }

    bar();

    That still prints "bar", but if bar() was lexical it should've gone out of scope.

  3. numerodix says:

    I can't say much about Perl, I just recently started looking at it.

    But are you saying that if the language can resolve names in a package so that you can call a function earlier in the code than you declare it, then it's not lexically scoped? I mean compiled languages obviously do this so.. and from the impression I got, Perl compiles modules before running them, so that would fit with the model, even if it's a "scripting language".

  4. Brian says:

    Maybe you could call it lexically scoped. Now that I googled a bit I've seen different definitions of the term. One is "Lexical scope means scope can be determined at compile-time by lexical analysis of source code", which is more general than what I was thinking of, and would apply to Perl subs.

    Another definition which I had in mind is "Lexical scope means scope is limited to the immediate surrounding lexical block containing a declaration", which is more specific and rules out nested subs in Perl, because nested subs reach outside the immediate surrounding block. The more general definition may actually be the better one though.

  5. This isn't a scoping issue. It's a binding issue.

  6. dleverton says:

    > Are the concepts dynamic and lexical just too simplistic to use in the “real world”?

    No, you just have absolutely no clue whatsoever what you're talking about.

  7. Eevee says:

    Bit late, but whoa, this is totally wrong.

    What you have there is a closure. The Python example is still using lexical scoping.

    The difference is that, in Perl, named subs are evaluated only when the code is COMPILED, not run. You can put a named sub declaration in a loop and run it as many times as you want, but Perl will only create the sub when it first compiles the file.

    In Python, function creation is treated like any other block, only done when the code is executed. You can replicated this in Perl with anonymous subs; try my $colorize = sub { ... };.

    This issue has absolutely nothing to do with scoping, or even with variables at all.

    Ironically, Perl DOES have dynamic scoping (see local()), and Python does not, although I'm sure it's possible to hack in somehow.

    (Dynamic scoping, by the way, means that variable lookup is done up the call stack.)