Archive for September 13th, 2006

fixing greedy emoticon matching in kopete

Wednesday, September 13th, 2006

I have a lot of admiration for the KDE project. The way that things come together and integrate into a common desktop with KDE is quite extraordinary. And all the time there are people interested in improving just about every bit of it. Now, of course, it’s all about KDE4, the long awaited upgrade will come at some point in 2006, I guess the date hasn’t been set yet.

Anyway, the beauty of free software is that if there’s a bug that gets to you, you can fix it yourself. And one such bug irks me in Kopete. I’ve been testing the xtorg emoticon theme and with a fairly rich set of emoticons (82 images, 117 replacement strings), it’s quite a good testset and exposes certain problems. The emoticon theme comes with a file called test_suite.txt, which just lists all the emoticon replacement strings, so that you can paste them into a chat client and see if they come up correctly. The special thing about the xtorg theme is that I’ve made sure to include the most common Msn Messenger strings, so that Windows people can reuse the ones they’re used to already. In Kopete 0.12.2, using the test suite gives this result.

kopete_bugged.png

So evidently, Kopete’s parsing of emoticons is not as good as it could be. I have examined the issue and found that the problem lies in non-greedy matching. This means that if : s and : s t a r : are both defined as emoticon strings, and : s just happens to appear before : s t a r : in Kopete’s internal list of emoticons, : s t a r : will be parsed as [: s] t a r :, not as [: s t a r :]. This is not what the user expects, having defined a list of replacement strings, the user expects all of them to work.

This is not the kind of bug that will affect a lot of users, because the average user does not use big emoticon styles like this one (and will probably never encounter the error). Thus if I were to report the bug, it’s not likely to be very high priority. Meanwhile it does bother *me*, so I thought I would try and fix it myself. So after a little hacking, I wrote a patch for Kopete, and it now does this.

kopete_fixed.png

I’ve reported this on KDE Bugzilla and Kopete developers willing, the fix will find itself into Kopete at some point. In the meantime, the patch is attached below.

diff -Naur kopete-0.12.2/kopete/libkopete/private/kopeteemoticons.cpp kopete-changed/kopete/libkopete/private/kopeteemoticons.cpp
--- kopete-0.12.2/kopete/libkopete/private/kopeteemoticons.cpp	2006-08-12 02:51:47.000000000 +0200
+++ kopete-changed/kopete/libkopete/private/kopeteemoticons.cpp	2006-09-13 07:20:28.000000000 +0200
@@ -48,6 +48,8 @@
 struct Emoticons::Emoticon
 {
 	Emoticon(){}
+	/* sort by longest to shortest matchText */
+	bool operator< (const Emoticon &e){ return matchText.length() > e.matchText.length(); }
 	QString matchText;
 	QString matchTextEscaped;
 	QString	picPath;
@@ -424,6 +426,7 @@
 		node = node.nextSibling();
 	}
 	mapFile.close();
+	sortEmoticons();
 }

@@ -492,9 +495,24 @@
 		node = node.nextSibling();
 	}
 	mapFile.close();
+	sortEmoticons();
 }

+void Emoticons::sortEmoticons()
+{
+	/* sort strings in order of longest to shortest to provide convenient input for
+		greedy matching in the tokenizer */
+	QValueList<QChar> keys = d->emoticonMap.keys();
+	for ( QValueList<QChar>::const_iterator it = keys.begin(); it != keys.end(); ++it )
+	{
+		QChar key = (*it);
+		QValueList<Emoticon> keyValues = d->emoticonMap[key];
+ 		qHeapSort(keyValues.begin(), keyValues.end());
+ 		d->emoticonMap[key] = keyValues;
+	}
+}
+

diff -Naur kopete-0.12.2/kopete/libkopete/private/kopeteemoticons.h kopete-changed/kopete/libkopete/private/kopeteemoticons.h
--- kopete-0.12.2/kopete/libkopete/private/kopeteemoticons.h	2006-08-12 02:51:47.000000000 +0200
+++ kopete-changed/kopete/libkopete/private/kopeteemoticons.h	2006-09-13 07:19:17.000000000 +0200
@@ -156,6 +156,12 @@
 	 * @see initEmoticons
 	 */
 	void initEmoticon_JEP0038( const QString & filename);
+
+	/**
+	 * sorts emoticons for convenient parsing, which yields greedy matching on
+	 * matchText
+	 */
+	void sortEmoticons();

 	struct Emoticon;
 

Download this code: kopete-012_2-greedy_emoticon_parsing_patch

EDIT: The original conclusion of this entry was that Gaim has parsing bugs too. This seems to be incorrect. A fresh new screenshot shows that Gaim handles the xtorg theme just fine.

gaim_emoticons.png

UPDATE: The patch was accepted verbatim into kopete svn, so the next release (kde-3.5.5), whenever it will be, should have this problem fixed. :)

what a waste

Wednesday, September 13th, 2006

I guess it was Ash’s entry that gave me this impulse to sort of wrap up this issue in my mind, as I’m done thinking about it.

So I moved to Utrecht in February and I was a little anxious about how I would handle this socially (with good reason, not one of my strengths). One of the first people I met, in an introductory class to the Dutch language, was this girl Zoe. In a situation like this, arriving in a new place, starting everything from scratch, there’s a lot of people to meet and greet. But there was something special about Zoe, it was as if we spoke ‘the same language’. So often with people I meet, there’s a language barrier, there’s accents to deal with, a difference of background, culture, expectations etc. It’s not that easy to understand where the person is coming from. And indeed this is the reason why I’ve never had a lot of real life friends, I just don’t meet a lot of people who ‘get me’ and whom I ‘get’. Coming to this new place, it was a concern for me to make friends and I wanted to make the best out of it.

Curiously, there seemed to be somewhat of an understanding between us on this point, she seemed pretty keen on staying in touch with me too. This is pretty rare, so often people come at this from different angles, so often I either have to do most of the work myself, with the other person not seeming very interested, whereas sometimes it’s the opposite – the other person wants to stay in touch, with me being fairly lukewarm at the idea. It’s rare that a balance is found, especially when you’re just getting to know the person, but this seemed to be one of those times. Seemed to anyway.

Just what a person really is thinking I’ll never know, but I certainly thought she was being sincere when she seemed happy to see me and eager to talk. Why would someone fake that? Trouble is, we only got to talk during that language course, or just standing outside the building after it was over. I wanted to ask, but I’m awkward with these things, I try to make the transition as smooth as possible, just to naturally fit it into the conversation, and I was not succeeding at all. But before the “I should get going”s were exchanged, she finished my thought “so maybe you can give me your email address and we’ll get together sometime?”. I don’t recall the exact words exchanged.

Lesson number one. Never give your email to someone and expect them to do something with it. The vast majority of people (I’ve had a chance to establish this fact over the years), even with the best intentions, will never use your email for anything. And if you actually take down someone’s email and send them a message, you are way above average.

So a week flew by, no nothing. I saw her again in class, she seemed pleased to see me, and apologetic for the email thing. Accidents happen, right? I won’t bother hashing out the exact record of events, I don’t even remember it to detail. But I didn’t always see her in class, at some point I did get her email address off a list of student data being passed around by the teacher. I tried to set up something, anything, by email – she was always cheerful, but busy. I have to say I was a little attracted to her as well, but frankly making friends with her was far more important to me, I needed a friend in this town.

Eventually I got very fed up with the “I’m busy” responses, which always included some form of pseudo apology for my inconvenience. This went on for about a month. So my careful diplomacy was completely failing, and the fact that we seemed to get along well didn’t seem to help at all. At this point, I was very frustrated. I hadn’t met anyone else who would ‘get me’ and I really invested a lot of thought and intent into this potential friendship. There was nothing wrong with what was being said either, just that nothing was happening.

So my patience was about to run out. I finally lost it and said something as if to imply that I didn’t believe she was actually that busy and that she was just humoring me. Wow, that was incredibly stupid. I was kicking myself for about 2 weeks for saying that. I can’t believe I said that, but I was really frustrated.

Lesson number two. Whatever the person’s story is, and whatever you believe or suspect the truth to be in relation to the story, never, ever, ever breathe a word of doubt over the story. What’s on the record is like evidence in a trial, it’s irrefutable. The only thing you can do is maneuver the person into admitting it themselves.

So I said that, and of course she was offended, saying she’s doing this and she’s doing that and she’s really completely booked. I felt like such an idiot, I tried to say something by way of an apology, but I couldn’t even convince myself that I could explain it or smooth it out. Basically, I had just shot myself in the foot and I was painfully (pun intended) aware of it.

So I tried to apologize and despite this screw up, she still seemed interested in ‘how I was’, which I thought was big of her. And that’s pretty much where it ended. The language class ended, I never saw her again. I wanted to say something, I just didn’t know what. I had basically tried every which way to do something with her, but I wasn’t getting anywhere. And on top of that I was disgusted with myself for being so completely tactless.

It preyed on my mind for a long time. I thought about this case as my best opportunity to make a friend, possibly a very good one, and I had failed so spectacularly. It just wouldn’t stop, I kept kicking myself for it and wishing something had been different. But eventually, I stopped. As I was coming back from vacation, I thought I would give it one last try. Time wipes the slate clean, right? So I sent an email just about the day I arrived, well over a week ago. And… nothing. I don’t even know if she’s in town anymore, but I’m done with this now. Aside from the little trip down memory lane that is this entry, which stirs up some momentary sadness over what happened, I’m at peace with this now.