In this day and age, with the consumer being treated like dirt, whenever a company launches an offer that is actually good, it's such a revelation. One such revelation (of which there are few), I discovered today, is allofmp3.com. I'm actually in awe of these Russian entrepreneurs. So let me tell you why that is.

1. They let you preview a track before buying in low quality (this is actually disappointing, because you have no idea how it really sounds).
2. They let YOU pick the encoding you want, among all the most widely used formats. How amazing is that? :cool:
3. All content is usable (no DRM).
4. The price of a track depends on the the sound quality you pick (ie. you pay by quality), but the prices are generally low.
5. Their collection is wide enough to carry albums that I actually want.

To give you a concrete example.. The other day I was raving about "Chevaliers de Sangreal" from the Da Vinci Code movie. I search for the album on allofmp3, they have it. The price for this track (at 192kbps mp3) is $0.18. I put it in my basket, I set the encoding to FLAC lossless compression, the calculated charge is $0.71. Meanwhile, Apple's iTunes Store, as far as I know, charges $0.99 per track (I cannot verify this as the store is only available through iTunes, which of course, has no linux support).

Granted, their payment system is a bit of a pain. They make you transfer a minimum of $10 into your account before you can buy anything. And I had to do this through a different site, even had to use my cell phone to retrieve a pin code they sent me. But, if you're a frequent customer, just transfer $100 in one go and you won't have to do this for a long time again.

The one drawback I've seen is that while they keep track of the tracks you've bought (no pun intended), they won't let you re-download them. So once you download, keep it in a safe place (like backup to CD/DVD).

The average album on allofmp3 costs about $2.50, that is 11x, E-L-E-V-E-N T-I-M-E-S, less than the average album in a music store in Norway, at 180kr ~ $27 (perhaps Holland is a bit cheaper, I rarely go to music stores anymore, I wouldn't know). And for that you get tracks at 192kbps (which is fine for most music, soundtracks and classical is more demanding music, I might get that in higher bitrates or FLAC), and you can buy per track. Not to mention that I never use CDs, because mp3s are so much more practical (and even if I did I could burn the CDs myself). Give me one good reason why I should ever buy another CD again.

How many times have you turned on CNN only to see American flags being burnt on the streets somewhere? What always makes me wonder is.. where do they get these flags?? Are there flag stores on every corner in these cities? Is someone getting rich selling flags? Is there a flag millionaire mansion somewhere? Or are they home made? Would someone really labor over a flag for weeks, cutting the fabric, measuring, sewing (if you've ever sewn anything, you know how much work it is) only to set it on fire? Even making a big banner is less work than sewing a flag. All you need is some big chunk of paper and a magic marker. But a flag requires fabric in every color of the flag, it requires careful measurement, it requires a sewing machine (otherwise it *really* takes ages to assemble).

But either way it's such a huge waste. If you buy the flag and burn it, that's just like fireworks, a complete waste of money for a moment of fun. And if you produce it yourself, well that's even worse. Hours of labor wasted in a few seconds. The whole idea is lost on me.

What's amazing to me is that for every technical problem, there's already been lots of people who've thought about it and tried to solve it. Given the world population is at some 6 billion, that's not really surprising, nevertheless it's very satisfying. Like lately I've been having thoughts about writing a small application using the Qt library. I haven't even begun designing it, I've just been doing preliminary research. My concern is that the program should only be a single binary and it should be as small as possible. The reason for this aim is that I want to make it as accessible as possible - it should require only a small download, and no installation necessary. So if it's a single binary, that's the easiest way to accomplish this.

But, of course, using any library at all already adds filesize in the shape of dependencies. Since I only want a single binary, I'm looking to compile statically, which will include all the library code that I'm using. Qt in particular, is a huge library. It probably adds up to about 15mb of library objects, and I don't want all of that in my "little" binary.

Let's do this by example. A few years ago I was dealing with High Dynamic Range (HDR) images, I even wrote a tutorial on how to produce them starting with pictures taken with a digital camera. I used Greg Ward's hdrgen utility for this. Greg's program is a single static binary. Now, if Greg had the same goal as I do, there are a couple of things he could have done.

$ ls -lh hdrgen
-rwxr-xr-x 1 alex users 8.7M Oct 24 2003 hdrgen

So the file is over 8mb in size, is there any way we can shrink it?

$ file hdrgen
hdrgen: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.0.0, statically linked, for GNU/Linux 2.0.0, not stripped

The information we're looking for is shown here in emphasis. The file is not stripped, which means it contains a bunch of symbols that aren't strictly necessary for it to run. Symbols that ease debugging or relocating the binary. Also note that the binary is statically linked, which means it does not depend on any libraries to run.

The first thing we can do it strip the binary. Stripping removes symbols and leaves only the bare essentials.

$ strip -s hdrgen
$ file hdrgen
hdrgen: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.0.0, statically linked, for GNU/Linux 2.0.0, stripped
$ ls -lh hdrgen
-rwxr-xr-x 1 alex users 1.9M Sep 19 22:59 hdrgen

As expected, the binary is now stripped. Notice also that the filesize has been reduced from 8.7mb to just 1.9mb! That's pretty sweet.

But it doesn't end here. A further way to reduce filesize (for static binaries only!!), is to compress them. UPX is a way to compress binaries to reduce their size further. It is a lossless compression method (otherwise it would be useless, of course), which bundles the compressed binary, and everything needed to uncompress it, in a single file.

$ upx -9 hdrgen
Ultimate Packer for eXecutables
Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004
UPX 1.25 Markus F.X.J. Oberhumer & Laszlo Molnar Jun 29th 2004
File size Ratio Format Name
-------------------- ------ ----------- -----------
1889480 -> 694035 36.73% linux/386 hdrgen
Packed 1 file.
$ file hdrgen
hdrgen: ELF 32-bit LSB executable, Intel 80386, version 1, statically linked, corrupted section header size
$ ls -lh hdrgen
-rwxr-xr-x 1 alex users 678K Sep 19 22:59 hdrgen

The binary is now compressed. As you can see, the file utility is having some problems understanding what it is, because of the added compression. But the binary is definitely smaller, down to just 678kb!

Tell me if you've been in this situation. You're with a group of people and this one person has taken upon him/herself to "entertain" the group. For some reason or another, it's usually guys who do this. So while a conversation is in progress, this guy keeps shooting in with really bad jokes. And I mean bad, they're obvious, they're stupid, you see the punch line a mile away and you just wish you could excuse yourself, which you can't, or zap the guy into a pile of dust, preferably. So the guy is telling these jokes, and some people are laughing, but not everyone is. You're not laughing, because you're appalled with what can pass for a joke with this guy. So the guy notices this and he says to you, in front of the group, "hey lighten up, have a sense of humor". Have you had this happened to you? Of course, everyone has.

But what does it mean to tell a bad joke? When I say bad, I realize it can mean more than one thing. A bad joke can be one you don't laugh at, because it's too obvious. Or you don't laugh, because you don't like making fun of whatever is the butt of the joke. Or some other reason. In this case, I'm sticking with the first definition. So the joke is too obvious. Now, if you'll indulge me for a minute, the following metaphor I borrow from one Jerry Seinfeld. A joke is a canyon between two cliffs. If you get the joke, you jump across the canyon, onto the other side. If you don't get the joke, you don't make it to the other side. The satisfaction from making it, is the laugh. Now, if the cliffs are too far apart, you won't make it, it's too far away. If they are too close, you make it, but it was easy, so there's no satisfaction from making the leap. So in other words, the object of a comedian is to set the cliffs far enough apart so that the leap will be big, but not too far apart, so that the audience can still make the leap.

But let's get back to telling bad jokes. What is a bad joke really? No, what is a joke really? A joke is a sequence of statements which are logically coherent with each other. If A, then B. If B, then C. And finally, at the end of this chain, there is a contradiction. Something which doesn't add up with everything that has been told up to this point. And that is the punch line. Why is this funny? I've no idea, but this is the anatomy of a joke. Now, the degree to which a joke is funny, is the lack of coherence between the punch line and the story the precedes it. If you can see that the punch line contradicts the story, then it is funny. If you do not see the contradiction, it means you don't get the joke. Once again, a bad joke is one where the contradiction between the story and the punch line is obvious.

So, a joke is a test (yes, a test!) of reasoning ability to see whether you can see the contradiction. You pass the test, then you laugh (or you don't laugh because it was too obvious). Now, a bad joke is simply a test which is too easy. So what does it mean when you tell a joke, what does it say about you? It says that you are giving this test to me, and you expect me to pass it. You would like me to pass it, with some effort, so that I will laugh at the joke. Now, if you make this test too easy, it basically means that you believe this easy test will not be trivial to me. It means that by giving me this easy test, you expect me to struggle a bit.

So let's say that the test is easy, the contradiction in the joke is obvious. How can I interpret this? Well, there are two obvious ways. 1) You think this joke is clever, and you think I will appreciate that. 2) You realize this joke is obvious, but you still think it won't be obvious to me. In the first instance, it doesn't make you look good, it makes you look a little stupid. In the second case, it's an insult to me, to think that I wouldn't find this obvious.

So there it is, telling a bad joke is one of two things. Exposing your lack of intellectual ability (and ambition). Or just being patronizing.

One of the first things that pique my curiosity when I find myself in a new network environment is "what's around me?". To me it feels a bit like waking up from a dream and not remembering where I am or how I got there, so I want to look around a bit. I wrote a little script for this, and I can't say that it was terribly effective. It was based on using ping to send packets to every possible host on the current network (ie. the one I'm connected to presently). The scan was sequential, so it would ping 10.0.0.1, then ping 10.0.0.2 and so on. Most of these addresses had no hosts bound to them, so the scan would take forever for the ping to time out and move on to the next host. It would actually take so long (10min+) that in a wireless network, clients would come and go between the start and the end of the scan.

I didn't use ping because it was such a great choice for this problem, just that it was the first thing that occurred to me. I did get the script to run a bit faster by parallelling the pings, but this is a very silly thing to do, because with a Class C network, there are now 254 instances of ping running on the system. This would often drown out the packets from the hosts which were connected and the script would fail to report any hosts at all. I'm not sure why that is, but I improved the situation a bit by pausing for one second before starting every new thread.

Just the other day I stumbled upon a mention of using nmap to do this same thing. Sure enough, nmap was *designed* for this, so it should be the obvious choice. Somehow that never occurred to me. :lala: So I rewrote my little script to use nmap in place of ping. nmap does essentially the same thing as my script did, it pings hosts in parallell, but it does so without forking itself 254 times and it has some clever algorithms that monitor the state of the network to get best throughput at least congestion. To put that in plain English, here's a little comparison for a scan across 254 IP addresses:

• parallell nmap: 0m 5.868s
• parallell ping: 4m 15.912s

In other words, the ping method is absolutely rubbish. But, while I always have nmap available on my laptop, it's not an application that is installed by default on every system (unlike ping), so perhaps it would be handy to be able to fall back on the ping method, as lame as it is, if that's all we have.

Another small refinement is checking ifconfig for network info, so the user doesn't have to supply this manually. Again, this could fail (no priviliges, no ifconfig), so it's made to be an option, not a requirement.

#!/usr/bin/env python
#
# Author: Martin Matusiak <numerodix@gmail.com>
# Licensed under the GNU Public License, version 2.
#
# revision 2 - add hostname lookup


import os, string, re, sys, time, thread


def main():
	network = None
	try:
		netinfo = check_network()
		(ip, mask) = netinfo
		network = ip + "/" + mask
	except:
		print "Warning: No network connection found, scan may fail."

	if len(sys.argv) > 1:
		network = sys.argv[1]

	if not network:
		print "Error: No network range given."
		print "Usage:\t" + sys.argv[0] + " 10.0.0.0/24"
		sys.exit(1)


	if cmd_exists("nmap"):
		nmap_scan(network)
	else:
		print "Warning: nmap not found, falling back on failsafe ping scan method."
		ping_scan(network)


def nmap_scan(network):
	try:
		print "Using network: " + network
		cmd = 'nmap -n -sP -T4 ' + network + ' 2>&1'
		res = invoke(cmd)
		lines = res.split('\n')
		for i in lines:
			m = find('Host\s+\(?([0-9\.]+)\)?\s+appears to be up.', i)
			if m:
				print m, "\t", nslookup(m)
	except: pass


def ping_scan(network):
	iprange = find('(\w+\.\w+\.\w+)', network)
	print "Using network: " + iprange + ".0/24"
	for i in range(1,254):
		host = iprange + '.' + str(i)
		thread.start_new_thread(ping, (host, None))
		time.sleep(1)


def ping(host, dummy):
	try:
		cmd = 'ping -c3 -n -w300 ' + host + ' 2>&1'
		res = invoke(cmd)
		if "bytes from" in res: print host, "\t", nslookup(host)
	except: pass


def nslookup(ip):
	if cmd_exists("host"):
		cmd = 'host ' + ip + ' 2>&1'
		res = invoke(cmd)
		if "domain name pointer" in res:
			return res.split(" ")[4][:-2]
	return ""


def check_network():
	cmd = "/sbin/ifconfig"
	res = invoke(cmd)

	iface, ip, mask = None, None, None
	lines = res.split('\n')
	for i in lines:
		
		# find interface
		m = find('^(\w+)\s+', i)
		if m: iface = m
		
		# ignore loopback interface
		if iface and iface != "lo":
			
			# find ip address
			m = find('inet addr:([0-9\.]+)\s+', i)
			if m: ip = m
			
			# find net mask
			m = find('Mask:([0-9\.]+)$', i)
			if m: mask = m

	if ip and mask:
		mask = mask_numerical(mask)
		return (ip, mask)


def mask_numerical(mask):
	segs = find('(\w+)\.(\w+)\.(\w+)\.(\w+)', mask)
	mask = 0
	adds = (0, 128, 192, 224, 240, 248, 252, 254, 255)
	for i in segs:
		for j in range(0, len(adds)):
			if int(i) == adds[j]:
				mask += j
	return str( mask )


def find(needle, haystack):
	try:
		match = re.search(needle, haystack)
		if len(match.groups()) > 1:
			return match.groups()
		else: 
			return match.groups()[0]
	except: pass


def invoke(cmd):
	(sin, sout) = os.popen2(cmd)
	return sout.read()


def cmd_exists(cmd):
	if invoke("which " + cmd + " 2>&1").find("no " + cmd) == -1:
		return True
	return False



if __name__ == "__main__":
	main()

The output looks like this:

Using network: 192.168.2.119/24
192.168.2.1
192.168.2.119	james.home.lan

The first host listed, whose address ends in a 1, is often a router. Then there's the host transmitting the scan, that is localhost. At the time of the scan there were no other hosts connected on the network. Of course, beyond finding hosts, there's a lot more one can find out about them using.. *drumroll*.. nmap.

Update: I added a name lookup feature so that if there is a nameserver on the network, you not only get ip addresses, but hostnames as well. :)