a critical look at paludis

January 23rd, 2008

I've been meaning to scribble something about paludis for a while now. I was tempted to do that right after I started using it, but then I thought it would be better to get some perspective and that would also cover issues that may come up some months into actual use.

So the moment has come. I installed paludis sometime in mid-November 2007. Long before that it was announced portage compatible, so it should be safe enough. Migration is neither all that long nor that complicated, it's mostly a matter of getting used to paludis's different philosophy. One of the odd things is setting up repositories (overlays) in /etc/paludis/repositories, but it's easy enough.

From a portage user's perspective configuring paludis is not the most pleasant experience. The documentation is quite complete, but it really demands that you know exactly what you want. There aren't any texts to read, it's generally just a FAQ. As far as user guides go it's not the most friendly one:

Non-Problem: There's no PORTAGE_NICENESS equivalent.

Rationale: Learn how to use nice. There's no GCC_NICENESS or VIM_NICENESS either.

To me personally (although I'm sure I'm not alone in this), it is portage's "rounded corners" that made it such a great package manager to use from the beginning. It had all this built-in convenience, like PORTAGE_NICENESS, like color output, like output that is verbose enough to be informative, but not overly verbose, like make.conf where you could set a range of optional, useful settings, like having emerge --ask which I would use all the time etc. Contrast that with something like apt-get and there's absolutely no doubt what the nicer tool is. Perhaps this bling also undermines portage's conceptual integrity, eventually turning it into an unmanageable codebase. But it's also what made me choose gentoo: the fact that it had, as it aspires to, the best tool.

Now paludis is more puritanical about this. What that means in practice is that it pushes that burden onto you, the user. We don't want it in paludis, so it's now your problem. As evidenced by advice like this:

Non-Problem: Paludis doesn't restore the xterm title on exit.

Rationale: Neither does anything else. Some programs do set it to a guessed value based upon a default prompt for certain distributions, but they don't restore it. You should be using PROMPT_COMMAND to do that yourself -- see the bash documentation.

So since paludis won't do this for me, it's now my problem to set in place the proper infrastructure for this, and to maintain it. It ceases to be a configuration option, it becomes a user environment issue. And I have to maintain this environment across machines, because it's no longer part of the application. Paludis is a tool that is technically superior, but inferior on user friendliness.

Not having FEATURES also means that I have to set all these things on the command line:

$ type ipal
ipal is aliased to `paludis -i --dl-reinstall if-use-changed --debug-build none --log-level warning --continue-on-failure if-independent'

And I'm still not sure if I'm setting all the optimal options, because there's tons of them. (Yes, there is PALUDIS_OPTIONS, but I wonder if it's useful to have different options on install, query etc.)

One serious usability problem is that paludis is ridiculously verbose. I wonder what kind of giant monitors the paludis developers own, but for my part paludis = lots of scrolling. Even running paludis --help has to be piped to less to refresh my memory on the most useful switches. What's more, the most important output is always at the top, so I always have to scroll the longest distance. If you want to flood the screen, put the crucial bits at the bottom, that's common sense. Case in point, if I'm installing packages and I do a paludis --pretend, I have to scroll up through all the use flags, then I come to the list of packages, but each package entry is several lines long, so by the time I get to the top I'm quite annoyed. The verbosity of paludis has to be easily 3-4 times that of portage.

Furthermore, default settings matter a great deal, and as a developer I think you are entirely culpable for setting poor defaults. On my first day with paludis I reached a show stopping bug when openssl refused to compile. As it turns out, it was the test suite that was broken, and since paludis by default runs the tests (or did, anyway), it just wouldn't install. SKIP_FUNCTIONS="test" fixes this, but that was clearly a misguided default setting. But there are many examples of this. --debug-build is enabled by default, which I think is wrong because most gentoo users don't actively debug *every* package they install. Meanwhile, these files take up quite a bit of space. Furthermore, coming back to the verbosity problem, --log-level is set to qa. This means that I, as a user, have seen this message every time I invoked paludis for the last two months:

paludis@1200787359: [QA] In program inquisitio -s perl:
... When performing query action from command line:
... When handling query 'php':
... When fetching versions of 'dev-lang/php' in gentoo:
... When loading versions for 'dev-lang/php' in gentoo:
... When extracting version from '/usr/portage/dev-lang/php/php-5.2.4_pre200708051230-r2.ebuild':
... When parsing version spec '5.2.4_pre200708051230-r2':
... Number part '200708051230' exceeds 8 digit limit permitted by the Package Manager Specification (Paludis supports arbitrary lengths, but other package managers do not)

Not only is that a trifle of a bug, how exactly am I the user served by seeing this warning? I'm not a developer, so I couldn't fix it if I wanted to. Furthermore, the irony of it all is that apparently paludis, which gracefully handles this problem, is the one emitting the error, whereas portage, which may actually be affected by it, doesn't. A pure exercise in futility.

Installing packages also outputs chunks of lines that never seem to change with any package, output I have no need to see:

>>> Running ebuild phase prepare as root:root...
>>> Starting builtin_prepare
>>> Done builtin_prepare
>>> Completed ebuild phase prepare
>>> Running ebuild phases init saveenv as root:root...
>>> Starting builtin_init
>>> Done builtin_init
>>> Starting builtin_saveenv
>>> Done builtin_saveenv
>>> Completed ebuild phases init saveenv
>>> Running ebuild phases loadenv setup saveenv as root:root...
>>> Starting builtin_loadenv
>>> Done builtin_loadenv

Is this supposed to be useful information for the user? I don't even know what it means.

Clearly, there are a few glaring problems. And maybe that's not so shocking from a development team that seems dead focused on technical issues. No surprise then, perhaps, that from a technical standpoint paludis fires on all cylinders. It took me a few days to get used to paludis, but since then it hasn't done anything weird or unexpected, it hasn't crashed, it has been rock solid. And those "issues" that may come up in the fullness of time? They never came up.

And this you already knew: paludis is fast.

:: random entries in this category ::

11 Responses to "a critical look at paludis"

  1. benizi says:

    After getting sick of Portage spinning on "Checking dependencies for ______... /|\-/|\-/|\-/|\-/|\-" (etc.) for at least 3 minutes for installing any package whatsoever, I, too, took the Paludis plunge. It's only been a week or so, and I agree with virtually everything you said above. I've added PALUDIS_OPTIONS="--log-level warning --dl-upgrade as-needed" to my .zshenv.

    To add to the list of woes, I had a helluva time getting paludis to *not* rebuild emacs every time *any* package was installed. Again, this is a "Paludis is more correct than Portage" problem (There's some circular [or at least elliptical] dependency on app-editors/emacs from virtual/emacs or vice versa.) And I struggled to get the permissions on everything set up just so. (I use umask 077, which makes things harder sometimes.)

    But, the one thing I disagree with in your writeup is:

    SKIP_FUNCTIONS="test" fixes this, but that was clearly a misguided default setting.

    I don't want all the ebuild QA output. (That PHP one bugs me every time, too.) Ebuilds are metadata, not essential to the operation of the software you're installing. But, test suites *should* work. The whole point of running them is to uncover potential problems with software. It's much more important to highlight these problems so they can get back to the developers - even if it's quite possibly a faulty test.

    </rant>

    Paludis's better use-flag and keyword support is amazing. I did prefer that Portage defaults "category/app" to "category/app ~whateverArchYou'reOn" in /etc/portage/keywords. But, with wildcarding, I can live with that difference. I've added "dev-perl/* ~x86", "app-emulation/* ~x86", and "app-paludis/* ~x86". Even "*/*::paludis-extras ~x86".

    And the speed. Wow. Never going back.

  2. numerodix says:

    Re: test suites
    A package manager is for installing applications, not for testing them. By the time ebuilds hit the tree they are supposed to be somewhat well tested. I don't mind running test suites (although I would for something huge like OpenOffice), but when they fail and it's an important dependency that means you're stuck because you can't install the packages you actually care about (not this dependency), and that's definitely a bad setting. It's nice to be strict about things as long as they work, but if they basically disable your system, that's no good.

  3. benizi says:

    A package manager is for installing applications, not for testing them.

    I'd agree with that if most ebuilds were binary packages. With RPM/YUM/(Package Manager), the general interaction is:
    User: Here is a set of files I want installed on my system.PM: Okay. I put those files where the package told me to and tweaked a config file.
    And, you also get the "PM: Hmm, are you sure you don't want to install these other packages, too?" But, with Ports-like systems, there's so much more to the process that's variable on a per-system basis, that it seems like tests are necessary to ensure that things aren't broken out-of-the-box.

    By the time ebuilds hit the tree they are supposed to be somewhat well tested.

    Isn't that what ~arch keywords are for? By the time a package gets to 'arch' rather than '~arch', I'd think part of the idea would be that lots of people had successfully compiled and tested the package.
    I think the best argument against my position is actually this: Even without tests, builds fail some percentage of the time. And usually if it'll build, it'll run. So, why increase that percentage by adding test-suite failures?
    In any case, thanks again for the interesting writeup. I'll repeat that I agree with the vast majority of it.

  4. Zurk says:

    Just wondering... Did you stick with Paludis? Or did you go back?

  5. numerodix says:

    Yes, I did stick with paludis.

  6. poopludis says:

    Well turns out paludis isn't that fast and it was all FUD/Propoganda

    Seems out of the 3 package managers (Portage,Pkgcore,Paludis) Paludis is the slowest
    http://gentooexperimental.org/~patrick/weblog/archives/2008.03.html March 29th,March 30th, March30 blog entries

    All numbers so not just this induviduals views. All numbers that have been seen by others and anyone can try (although the paludis lot tried to wiggle out of said benchmark and also skew the results)

    Likewise it seems once yr system is paludis-ified it aint really portage-friendly and if you decide to stop using paludis it can become hard and if you have been using it for quite some time, virtually impossible

    So it seems Paludis's FUD that portage is slow and Paludis is fast turns out to be...

    slow python code (portage) is faster then fast C++ (paludis) code

  7. g2g591 says:

    "Likewise it seems once yr system is paludis-ified it aint really portage-friendly and if you decide to stop using paludis it can become hard and if you have been using it for quite some time, virtually impossible"
    now this is blatent FUD if I ever saw it I use both paludis AND portage on the same system and they work fine together.

    as for the actual writeup, I totally agree, it pushes too much debug/meaningless output and there does need to be a better way to control it. As for Pkgcore, don't bother, it crashes so much, it seems its primary function is to produce indecipherable python backtraces.

  8. me says:

    that may sound stupid, but one thing bothering me with paludis is the name. heck, the names of all the softwares included :

    paludis, contrarius, importare, inquisitio, reconcilio - What the hell is that ?

    typing paludis to install program feels like typing AIDS to get a manpage - it's just weird ...

    I know the problem is solved by setting up aliases, but it feels like choosing such command names is clearly stating "we are not here to be user-friendly, because the technical part is what matters. And just in case you didn't get our point, we are going to be frankly user-unfriendly".

  9. Coyote says:

    Well, I've started with paludis today, since I've got stuck with emerge (I've installed overlays that have packages with the same name and version as the one's in the gentoo tree). And after I heard that paludis was made out of C++, I've decided to change. The script to change the configuration from emerge to paludis worked just fine for me together with the documentation, the overlays were also easier to install thanks to playman (I emerged paludis with the ruby use flag). And the verbose output I don't mind, as it was one reason for the switch, since it can be usefull to find out something that bug me (e.g. why emerge wanted to install gcc 3.3.2 when I already have gcc 4.2.1, and many more like that).
    And being an programmer of both python and C/C++ (along with others), C++ has one thing that python will never have and that is usefull for this things, and that is proper thread support without an GIL (global interpreter lock) messing around. On overral I'm happy with the change so far.

  10. paludified says:

    Paludis is an amazing peice of software!

  11. benizi says:

    I'm not sure what the FUD comment is about. I've seen it with my own eyes. On two different systems, I've installed paludis and seen an apparent ~70% reduction in time taken to resolve dependencies. It's too bad the benchmarks are gone. I'd bet good money they were on almost completely fresh installations. It might even have been my own broken-ness (emerge -C'ing things) that caused it. But, still.

    There are definitely things that still bother me about paludis: some of the defaults (--debug-build split just got me today on a new system), and the refusal to create the .cache directories. (I made a small utility script to set my paludis permissions properly.)

    I got burned a long time ago by the 3.x line of MySQL silently corrupting data via some varchar() truncating thing, and non-working but present foreign keys (<-- I think). So I switched to PostgreSQL after being lured by psql's better default settings and have been a happy PostgreSQL'er since. Even if MySQL has righted some of its wrongs, or gained some spiffy features, it long ago lost my trust.

    Same's happened with paludis. paludis's category-wide keywords and use flags (dev-perl/* ~x86) attracted me away from an 'emerge' that had lost my trust. The speed and correctness kept me there, and now I'll never go back.