But undvd depends on a bunch of other tools, so what you really want is to let your package manager deal with satisfying the dependencies, and let you upgrade easily when a new version comes out.

Where can I find undvd?

I package undvd in three flavors myself: Gentoo, Fedora and Ubuntu. I run Gentoo and Ubuntu on my boxen, so those are the most well tested. In addition, ripdvd’s author wedge does the honors for Slackware.

But there are some unofficial packages out there as well, so when you add it all up undvd is pretty well supported at this point.

• Arch :: aur
• Fedora :: sf.net
• Gentoo :: portage / sf.net
• Slackware :: sf.net
• Suse :: packman.links2linux.org
• Ubuntu :: ppa / sf.net

Making friends with posix

Recently, a user expressed an interest in running undvd on a Mac, which is a whole new angle. I guess it never occurred to me that someone would use undvd outside linux, but why not. The recent switch to perl has obsoleted the many external tools that undvd previously relied on, which is a big [if unplanned] step towards portability. I ran some tests that revealed there were still a couple linuxisms left, so I’ve gotten rid of them now.

undvd-0.7.1 should in principle run on a Posix system with perl-5.8.8. The big thing isn’t undvd, it’s getting mplayer and the other binaries, cause they have to be built specifically for that platform. And outside linux, there’s sometimes no standard package manager, which doesn’t help at all.

At the end of the day, there is no escaping the fact that undvd on Posix is a fish out of water (true not only of undvd but probably most free software) – the environment just isn’t as well supported as it is in a linux distribution that ships all the tools. There are no packages for any of these, but you can just extract the tarball and run undvd from whatever location.

DesktopBSD 1.6 (FreeBSD Ubuntufied)

It runs (except mencoder can’t find x264, so I used xvid). The dvd device is called /dev/cdrom or /dev/acd0. Cloning with dd fails, vobcopy works.

OpenSolaris 2008.05

It runs (except mencoder can’t find x264, so I used xvid). I have no idea how the device naming works, but when you put in a cd/dvd it should automount and you can look up the dvd device in the mount table. Packages for mplayer are available from Blastwave. Playback with mplayer worked, encoding segfaulted mencoder.

Mac OS X Leopard

It runs. The dvd device is called something like /dev/rdisk1. Mac people seem to be obsessed with guis, so any builds I could find only had mplayer playback, no encoding. Your best bet is probably MacPorts, modeled loosely on FreeBSD’s ports. You’ll want to build mplayer with the codecs you want selected as variants (think Gentoo USE flags). Disc cloning doesn’t seem to work at all.

To install globally you’ll want something like DESTDIR=/opt/local sudo make install.

Cygwin

It runs, and you can get builds of mplayer from the official site. The dvd device is called d: (or some other drive letter). Set PATH=/cygdrive/c/mplayer:$PATH and make sure c:\mplayer doesn’t contain both a directory mplayer/ and a binary mplayer.exe, this makes cygwin very confused. On the whole it seems to work quite well, the process even successfully sets the cpu priority (nice).

To date, undvd has been using various tricks to get around this. Let the function echo the value and capture this in the caller. But then what if you have a failing condition? Well, you can echo the value to stdout and echo the error to stderr, so it doesn’t get captured as the result of the function. And then kill $$ to force an exit (you can’t just exit cause that is equivalent to a return from the function).

That kind of works, but eventually you break down when you have to return more than one string that may contain whitespace. Sure, you could quote them, let the caller find both strings based on where the quotes are, then chop off the quotes and voila. But all this just for a function call? It’s too much, and it’s unacceptable from a maintenance standpoint.

Bash’s overall weak support for other features of a typical programming language makes it a challenge to write structured programs. undvd-0.6 is therefore pretty much a dead end from a development standpoint. It works well enough, but it’s hard to get anything more out of bash. In order to keep evolving, undvd needs a new language.

Another substantial problem with bash is that you’re executing commands in the shell, in other words you build execution strings. There is a lot of potential for quoting bugs when you’re dealing with filenames that have spaces and quotes in them. And not just when feeding them as arguments to executables, but on every “function call” just the same. I’ve spent a lot of time trying to safeguard against this, but all it takes is one instance where the strings aren’t quoted correctly, and you have a fatal parsing error.

So it’s time to think about porting to another language. A language that is close to the shell. A language that lets you run a subprocess by passing in the arguments as a list, not a string. A language that has basic programming constructs, like functions. That has good string handling. That can do simple floating point arithmetic. That is as widely available as possible. A language like… perl.

It sounds absurd, doesn’t it? Porting to perl in the name of maintainability. But when you’re in bash and most of what you’re doing is string manipulation and calls to other executables, it’s the right choice. And I bet you have perl on your box.

Not that it hasn’t been fun. Bash was the right place to start, and I’ve learned a lot of things about bash on the way. I’ve also learned that you have to do obscene things like echo strings to bc to do simple floating point math.

The port

It’s a straight port. undvd 0.7 runs on perl, but the way it was written was to reproduce 0.6 exactly. The code is completely new, obviously, but the functionality is the same.

As a result of running in perl, all the string/numerical processing logic has been internalized, and all the calls to awk, sed, bc and so on are gone. This makes it run faster, scandvd is especially noticeable. This isn’t a big impact, since most of the work is done in mencoder, and that is still the same. Nevertheless, it’s a welcome side effect. It also makes me happier, since it’s less dependent on all these outside tools.

In terms of size the code is about the same. The perl code is actually 5% bigger.

What this means for you

• 0.6 and 0.7 are functionally equivalent.
• If you find a bug in 0.6, it’s probably also in 0.7.
• If you find a bug in 0.7, try 0.6.
• Please report bugs.

* I’m using the term “language” loosely here. I’m talking about both the language, and the implementation, and the execution environment (ie. standard libraries, or in bash’s case the gnu userland). Often we just pile all of this under “language”, because it’s easier to talk about it that way.

Writing undvd was a decent challenge in itself, because of all the complexity involved in the process. I had to find out all the stuff about video encoding that I didn’t really care about, but I thought if I put it into undvd, and make sure it works, then I can safely forget all about it and just use my encoder from that point on. When you start a project you really have no idea of where it’s going to end up. undvd has evolved far beyond anything I originally set out to build. That’s just what happens when you add a little piece here and another piece there. It adds up.

It’s been about 20 months. undvd is quite well tested and has been “stable” (meaning I don’t find bugs in it myself anymore) for over a year. One of the by products is a tool called vidstat for checking properties of videos. I wrote that one just so I could easily check the video files undvd was producing. But it turns out to be useful and I use it all the time now (way more than undvd). In the beginning I was overwhelmed by the number of variables that go into video encoding, and I wanted to keep as many of them as I could under tight control. I have since backtracked on a number of features I initially thought would be a really bad idea for encoding stability. But that’s just the way code matures, you start with something simple and when you’ve given it enough thought and enough tests, you can afford to build a little more complexity into the code.

Codec selection landed just recently. And once I was done scratching my head and trying to decide which ones to allow and/or suggest, I suddenly realized that with this last piece of the puzzle I was a stone’s throw away from opening up undvd to general video conversion. Urgently needed? Not really. But since it’s so easy to do at this point, why not empower?

The new tool is called encvid. It works just like undvd, stripped of everything dvd specific. It also doesn’t scale the video by default (generally in conversion you don’t want that). So if you’ve figured out how to use undvd, you already know how to use encvid, you dig?

Demo time

Suppose you want to watch a talk from this year’s Fosdem (which incidentally, you can fetch with spiderfetch if you’re so inclined). You get the video and play it. But what’s this? Seeking doesn’t work, mplayer seems to think the video stream is 21 hours long, that’s obviously not correct (incidentally, I heard a rumor that ffmpeg svn finally fixed this venerable bug). It seems a little heavy handed, but if you want to fix a problem like this, one obvious option is to transcode. If the source video is good quality, at least from my observations so far, the conversion won’t noticeably degrade it.

So there you go, a conversion with the default options. You can also set the codecs and container to your heart’s content.

You can also use encvid (or undvd for that matter) to cut some segment of a video with the --start and --end options.

I’m sold, where can I buy it?

• Gentoo: undvd-0.6.0 is already in portage
• Ubuntu: use my personal package archive (small apt repository)
• Rpm/others: grab it from sourceforge

The great thing about standards is there are so many to choose from.
- Someone

undvd 0.5.0 introduced a new option to choose the codec and container for the rip. The only problem is that you have to know which ones to choose. mencoder supports a staggering number of codecs and containers, most of which are now exposed also in undvd. The resulting rip can also be remuxed to a couple of other popular containers with additional tools.

But I wasn’t content with solving a problem by introducing a new problem. Now, it’s not so easy to say exactly which combinations are good and bad, but if at least you knew which ones definitely do not work, that would be a start, wouldn’t it? Then at least you can rescue the user from phase one of the Monte Carlo method in getting something that actually works.

The methodology is like this:

1. Rip 5 seconds of the dvd using undvd with a given container/video codec/audio codec combination.
2. Attempt playback with mplayer.

This is what codectest does. The result is either a text file showing line by line whether or not the given combination successfully produced a rip, or a pretty matrix picture. This gives you an idea of what you can expect to use. If you run this on your system, it’s also a tip off if you see something that should work but doesn’t.

I must stress that if the given combination of codecs does produce a file, this is no guarantee that the file is to be considered a good rip. It may not play on other media players, it may not even play on mplayer (incidentally, this is something akin to a fuzzer, I’ve discovered that some combinations really aren’t expected ). So if codectest says it works, verify that you get a working video file out of it!

The standard set looks something like this:

It’s also possible to run it on the full combination of all codecs and containers that are now exposed in undvd. You’ll need a few hours to do it:

But at the end of the day it’s a question of priorities, and having scratched all the important technical itches by now, if anything this is the right time for it. I got some user feedback recently that set me onto this path. The user was having trouble playing the files encoded in the classical avi+h264+mp3 format on other platforms, and that’s when I asked myself how important is it really to have a single format? As long as the default still works well, what’s the harm in offering a little customization?

Testing is a huge problem, which is why this new feature is considered to be experimental. The most common seems to be bad a/v sync. There is just no way to account for all the possible combinations of codecs and containers, and to maintain an up-to-date document for this as things evolve. So the burden of testing is squarely on the user here (which is quite unfortunate).

The new functionality is available in undvd 0.5 and up. Here’s a shot of the new goodness. All these files were encoded from the same dvd title. A 22 minute title was ripped with different containers (represented with different filenames). The audio codec is mostly the same in all cases (mad = mp3), except for 1.mp4 (faad = aac). The video codec is also mostly the same (h264 = avc1), except for 1.flv. The only variation here is the container being set to different values, all the other settings are defaults. You can also witness that some containers are more wasteful than others (given the same a/v streams), but not by a huge amount. (The audio bitrates shown are actually misleading, mplayer seems to give the lowest bitrate in a vbr setting.)

This demo is by no means exhaustive of the full collection of codecs that can be used, for that see the user guide. There is also an option to use the copy codec, which just copies the audio/video stream as is.

]]> http://www.matusiak.eu/numerodix/blog/index.php/2008/09/08/of-codecs-and-containers/feed/ 3 http://www.matusiak.eu/numerodix/blog/index.php/2008/03/09/undvd-gets-smarter-scaling/ http://www.matusiak.eu/numerodix/blog/index.php/2008/03/09/undvd-gets-smarter-scaling/#comments Sat, 08 Mar 2008 22:51:28 +0000 numerodix http://www.matusiak.eu/numerodix/blog/index.php/2008/03/09/undvd-gets-smarter-scaling/ The big new change in undvd 0.4.0 was a dynamically set bitrate based on the bits-per-pixel (bpp) value. That addition not only completely changed the way undvd deals with bitrates, it also made possible further improvements.

The guiding principle of undvd remains be smart if possible, don’t bother the user. A brand new feature is a smarter way to determine how to scale the video. This is another under-the-hood improvement which improves the program without introducing any more bloat or changing the user interface.

How do we scale?

To motivate the issue, let’s consider an example. Until now, undvd scales to 2/3 of the original width.

For a 720×576 (5:4) dvd movie this gives target dimensions 480×384, for a total of 184,320 pixels. For a 2 hour movie (25 fps), this gives results in a video 771mb (plus the size of the audio, some 140mb).

480 * 384 * 25 * 0.195 * 7200 / 1024² / 8 = 771mb

But for a movie of different dimensions, a constant scale factor would yield a smaller number of pixels per frame. Suppose another movie has dimensions 720×480 (3:2). This would give target dimensions 480×320, a total of 153,600 pixels. The video size would, of course, be proportionally smaller as well.

480 * 320 * 25 * 0.195 * 7200 / 1024² / 8 = 642mb

This is probably not what you want. You give up the same proportion of pixels in both cases, but for a movie of smaller dimensions it would be nice to get the same frame size, ie. keep a greater portion of the frame. In the extreme case, for a movie with dimensions 720×224 you would probably want to keep the frame intact (161,280 pixels) and you would still get a smaller file.

Scaling by frame size, not width

So the obvious solution is to consider the number of pixels in a frame, not just along the width. As with the bpp in the previous increment, what used to be the default becomes the starting point for interpolation. So 720×576 will still scale to 480×384. But 720×480 becomes 528×352 (185,856 pixels) and 720×224 would not be downscaled at all.

What complicates the issue somewhat is the need to pick dimensions such that it comes to an even number of 16×16 pixel blocks (I’m not sure how crucial this actually is, but the absolute of majority of video files obey this rule). This reduces the number of possible dimensions for a movie of a given aspect ratio.

For instance, take dimensions 720×272. This gives a frame size just over the desired 184,320 pixels. But the next possible option is 256×96, which is tiny. Here undvd does the “smart” thing and picks the dimensions closest to the target size (in this case leaving it intact).

The nitty gritty

Deciding on a scaling factor is quite simple. Starting from the equations to scale the width and height separately,

width * factor = scaled_width

height * factor = scaled_height

we obtain a formula to find the number of pixels in a frame.

width * height * factor² = framesize

Assuming we know how many pixels we want in a frame irrespective of the dimensions, we can derive a formula to find the scaling factor.

factor = sqrt( framesize / (width * height) )

To reuse the example of the 720×480 movie, we find

0.730 = sqrt( 184,320 / (720 * 480) )

Scaling by this factor we get the target dimensions of the video.

720 * 0.730 = 525.813

480 * 0.730 = 350.542

It may not surprise you to know that the new dimensions do not divide evenly by 16. The nearest dimensions satisfying the multiples-of-16 rule are 480×320 and 528×352, but the latter is closer to our starting point.

Multiples of 16

Finding the width and height is a bit more hairy because we have to find two values that correspond. I don’t think there is a formulaic solution to the problem because it amounts to solving this set of equations:

width * height = framesize

width = ratio * height (where ratio is known)

width = a * 16

height = b * 16

framesize = c * 16

The last three equations do not help, because they do not relate any two of the variables (width, height, framesize) to each other. And so we have the top two equations of three unknowns, which is not going to work. Intuitively, we know this is true, because we expect there are many sets of (width, height, framesize) that are possible solutions, so this set of equations will not give us a solution.

But it can be solved with an algorithm:

local ratio="$orig_height/$orig_width"
 
step=-1
unset completed
while [ ! "$completed" ]; do
	step=$(( $step + 1 ))
 
	local up_step=$(( $width + ($step * 16) ))
	local down_step=$(( $width - ($step * 16) ))
	for x_step in $down_step $up_step; do
		local x_width=$(( $x_step - ($x_step % 16) ))
		local x_height=$( echo "scale=0; $x_width*$ratio/1" | bc )
		if (( ($x_width % 16) + ($x_height % 16) == 0 )); then
			completed="y"
			width=$x_width
			height=$x_height
		fi
	done
done

Download this code: undvd_scale16

Here step is the distance from our starting point, each time used to set the values up_step and down_step to successive upwards and downwards increments of 16 respectively (525+16, 525-16). For each of these we run the inner loop, setting x_width to the value that divides 16 within the range [x_step, x_step-16]. We now have a valid value for the width. We now use the ratio to find the corresponding height while preserving the aspect ratio, stored in x_height. If both x_width and x_height are multiples of 16, we have found dimensions for the video.

Building 80% of the product takes 20% of the effort.

It’s not surprising. A little code goes a long way. Typically problems we attack have a wide frontier we can cover with little effort. That is why we choose them. And while no software really can be said to be 100% complete in a mathematical sense, it certainly takes a great deal of effort to “finish it off” by handling all those special circumstances and corner cases.

What is wrong with 80% completion anyway? Isn’t that good enough? Often it is. But there is that little voice in your head that says since you’ve already done “almost all of it”, why not finish it? After all, there is a sense of accomplishment at stake, and since you’ve already attacked the problem, there is that urge to solve it completely.

undvd was 80% complete by version 0.2.2, with packages for Gentoo, Ubuntu and Fedora. Version 0.3.x added things I didn’t plan to include at all when I started this project. The new features made it possible to set an output size (and thus indirectly set the bitrate), set 2 pass encoding, and set scaling. I was wary about compromising the key principle: undvd should be obvious and easy to use. But the new goodies were tucked into an “advanced” menu and so I don’t think they did a lot of damage.

Since 0.3.0 there has been a host of tweaks and small fixes for small problems. One such problem was that undvd worked well on terminals with a black/dark background, but was not tested on a white/light background. That is unfortunate, given how gnome-terminal and konsole both use a light background by default. These kinds of fixes are enjoyable in that they improve the overall user experience in subtle ways, but without changing anything in the way you have to use the software.

Preparing to leap again

Having said that, one technical aspect I wasn’t entirely pleased with was how undvd sets the bitrate. This is something of a black art, and I had done pretty intensive testing before deciding that 900kbps seemed ideal for h264 video. But debating the ideal bitrate is like debating the ideal size for a jpeg image. At the very least, it matters what the dimensions are. I had been scaling the image to 480 pixels wide in my tests, and so with greater dimensions the result wouldn’t be as good.

I came across a very helpful article that deals with bitrates and also contrasts the h264 and xvid codecs. Discussing bitrate is pointless, because bitrate isn’t a constant, it’s a function of other quantities. What we need is a constant to standardize on.

Going back to the jpeg example, let’s start with a frame of video. 480×384 gives 184,320 pixels, if we use one bit to store each pixel we come up with a quantity that is bits per pixel, abbreviated to bpp. So we’re using 184,320 bits (180 kilobits, or 22 kilobytes) to store this one frame.

Let’s take the single frame and another however many frames that fit into one second of video (25 is a common number). So now we have 25 of those 480×384 frames. If we now use one bit to store one pixel we need 25 bits for the job.

To complete the illustration, say we take our 480×384 frame 25 times per second, for a total of 2 hours (7200 seconds). If we used one bit per pixel, we used 25 pixels for 7200 seconds, which gives 180,000 bits per pixel in the movie. Multiply that by the number of pixels in one frame and we find 33,177,600,000 bits (30 gigabits, or 3.8 gigabytes).

Where does the bitrate come in? The bitrate is bits per seconds, so now that we have the number of bits, we divide by the number of seconds and find 4,608,000 (4500 kbps). If it isn’t clear yet why the bitrate isn’t a useful measure, it soon will be.

Getting it right

To formalize our deductions we draw up some equations. First, as we said, bpp is bits per pixel:

bpp = bits / pixel

To calculate the size (in bits) of the movie we took the bpp, the framerate (frames per second), the length (in seconds) and the dimensions (in pixels).

size = bpp * fps * length * width * height

And the bitrate (bits per second) is now the size divided by the length of the movie:

bitrate = size / length

We can also solve this equation for size, and substitute the formula into the previous one, which gives a different equation for the bitrate:

bitrate = bpp * fps * width * height

It should now be clear what the problem is with the bitrate as a measure of quality. It not only encapsulates the bpp, it also subsumes the dimensions and the frame rate of the movie!

Still not clear? Let’s solve the equation for bpp.

bpp = bitrate / (width * height * fps)

Now let’s assume we use the bitrate as a measure of quality. As mentioned, undvd up to version 0.3.x sets a constant bitrate of 900kbps (921,600 bits). Let’s try that on a couple of examples.

0.200 = 921600 / (480 * 384 * 25)

When we plug the bitrate into the equation we find the bpp is 0.2.

But now suppose the user does not want any video scaling and the dimensions of the output video are 720×576.

0.088 = 921600 / (720 * 576 * 25)

Now we have less than half the space to store one pixel than we did in the first case, because we have more than twice the number of pixels in every frame. Clearly, the same number of bits to store this movie, spread across more pixels, will allow less for each pixel, and thus lower the quality.

Therefore, the bitrate needs to vary depending on the dimensions and the length of the movie. It is thus a function, not a constant! On the other hand, bpp is a constant, and so if we want a measure of quality we should use that instead.

Getting practical

The equations are easy. The question at hand is what value to set for the bpp. As shown in the example, 900kbps at the dimensions frequently used gives about 0.2 bpp, which is known to work well. Thanassis Tsiodras suggests that 0.2 bpp is enough for xvid, while h264 can be encoded at just 0.125 bpp. But considering that undvd is a one-size-fits-all solution, and therefore should encode any video well (even at the cost of filesize in cases where it’s unnecessary), I would rather overshoot the target size a bit and play it safe. Therefore, based on my results, the default bpp is set to 0.195. In case of two pass encoding, we trade processing time for filesize, and the value is set to 0.150. For xvid the corresponding values are 0.250 and 0.200.

As before, these defaults can be overridden by setting the output filesize.

The best thing about this new functionality is that it doesn’t invalidate any existing assumptions. For a default rip the result will be the same as always, which is supposed to be very good. But now it scales to custom video dimensions as well.

Something missing?

Pleased as I was that undvd now has a mathematical underpinning for deciding the bitrate, I was still not quite satisfied. It’s nice and all that we now use a constant to determine the quality, but what is bpp anyway? How do we relate to it? And how do we know what a good value of bpp is?

To make this more tangible, I decided it can’t just be a value in a calculation, it has to be made visible. undvd now has a “dry run” option -D, which shows all the parameters of the video to be encoded. The bpp value is highlighted, because that is our measure of quality. This image shows title 01 is chosen for ripping. The dimensions will be scaled to 480×384, the frame rate is 25 fps, the length is 148 minutes, the bpp is .195, the bitrate is thus calculated to 877kbps, we are using one pass encoding, and the h264 codec. The output size is estimated at 1122mb.

Short of showing the bpp for a title we are about to encode, it would also be useful to inspect videos we have encoded in the past, wouldn’t it? That would give us an idea of what to expect.

For this there is a new tool vidstat, which displays the same information about any file or dvd title (much like stat displays information about a file on the filesystem) .

Here we see the first 60 seconds of title 01 (from above) being ripped. Then we run vidstat on the output file to examine it. Here we see that mencoder decided to use 809kbps instead of 877 as it was instructed to.

The bpp value is highlighted in yellow, red or green depending on whether undvd thinks it’s unnecessarily high (above the single pass bpp default) or low (below the two pass bpp default), or just right (within the two).

It’s actually a bit of a dilemma what to do with these small projects. A big project just looks neglected without a website, complete documentation, packages for several distributions, up-to-date entries on various directory sites etc. I did the full scale follow-up with galleryforge, and that now speaks for itself, it presents itself as a proper project. But for one thing it was my first time hosting with one of these open source providers, so I wanted to explore the options quite deeply. And secondly, you don’t really know what kind of a future a project will have when you’re starting it, but galleryforge isn’t used much, not even by me. So it was fine to do then, but it’s some effort to put in for every small piece of code like this. (Not so much doing it then, but the burden of maintenance is the key issue.)

undvd is even smaller, and isn’t expected to grow much (beyond feature complete by now). It started out as a blog entry and a link on my code page. Then I posted it on opendesktop, and it got seen by a handful of people who happened to be looking at the frontpage when it was added. There was also a regression bug posted there, which spurred me to start tracking it with git to avoid these things in the future. Then I decided I wanted to package it up, because that makes it organized and easy to use. But then I couldn’t figure out how to write the url (in the ebuild) for the tarball hosted on opendesktop, because their file hosting is a little sucky. So I registered on sourceforge. The only thing I’m using there right now is the file release function. There’s no website, and I don’t really see the call for one. Meanwhile, I’ve been trying to maintain the user guide (which is shipped with the code) clear and up-to-date, to make that the one source of information. I think that’s enough for now.

There are people out there who are proper movie buffs. People who can quote lines out of a movie and identify passages to extreme detail. I’ve known a bunch of people like that, but I’ve never been like that myself. To me watching movies was never a really important thing, it was kind of a second tier interest. But knowing people like this has its advantages, you can borrow their movies. I remember how we used to lug around VHS tapes at school.

So my lack of outright passion about movies is really the reason why I never took after the dvd ripping crowd. When DVDs were fairly new, people were so excited about divx. My friends at school used to discuss bitrates and filesizes. I didn’t follow any of that, I wasn’t really interested. I didn’t have a DVD drive, nor did I think paying €38 for a movie was such a good deal.

But admittedly, ripping the occasional movie is useful. DVDs in general are just a hassle and I don’t like to put up with them. Back in those days (a few years later, when I had bought my first dvd drive for my first custom built computer) I would try it a few times. I didn’t mirror the excitement of my friends. It took a really long time to rip a movie, and the result wasn’t that good. I don’t know what I was doing wrong. We used the Gordian Knot suite and followed the guides on Doom9. But while the program gave me every possibility to tweak the settings, I didn’t know what they mean, I didn’t *want* to know what they mean, I just wanted to rip it and move on.

And that is precisely the point where undvd comes in. I’ve tried a bunch of tools for ripping DVDs. dvd::rip might be the most merited linux tool. There’s also acidrip and a bunch of others I don’t even remember anymore. All these tools had the same two flaws:

• They present a lot of options that I don’t know anything about, and I don’t care about. And presumably, since my results were never very good, these are crucial for good results.
• It takes anything from 15 to 45 minutes to actually set up all the options, inputs, outputs, filters, cropping, the whole shebang. When something fails, like the disc can’t be read from, the tool sometimes crashes. If it does, all your set-up work is lost. But even if you have a working setup from before and you want to rip another movie you still have to double check all these settings, because ripping takes a long time and if you mis set something by accident you may only discover it far along the way.

Therefore, I think it’s fair to say that we have the Photoshop of DVD ripping in place. Personally I want to use Paint for this activity, because I have no deep interest in all the details.

Making it work

So if there is something complicated you have to do again and again, and every time you worry that you’ll get it wrong and even if you get it right next time you probably don’t remember what you did again. Faced with that, what do you do? In the middle ages monks would copy books by hand, at extraordinary effort. Not only would they transcribe the text, they would also copy and enhance the illustrations so that the copy, while not being the original, would be no less worthy. This was their calling, transcribing was an art form. Then came Gutenberg. Now granted, building a printing press took a lot more effort than transcribing a book. But once you had done that, copying books was a no brainer.

To me ripping a DVD is not an art form, it’s a chore. And if I can figure out how to do it once and never have to worry about it again, then it’s well worth the effort. So I set out to figure it out, I looked up the details and read the fine print. Eventually I had a fairly good idea of how to do it, and the results were coming along.

undvd became the wrapper around that nastiness I didn’t want to look at or remember. And for my money it worked well. It also encompasses certain design decisions that were important to me.

Why no gui?

The Photoshop rippers all come with a gui interface. That’s hardly a surprise, the plethora of settings they allow you to fiddle with would look like mplayer’s list of options if they weren’t gui apps. It’s practically impossible *not* to have a gui for those guys.

But DVD ripping is not inherently a gui centric activity. You need the gui to write your missile launch plan, but once you start ripping the gui doesn’t serve any purpose anymore. It just sits there in memory, and you could just as well remove it while ripping and then bring it back once it’s over. So 95% of the time the gui isn’t being used.

Apart from the simple fact that coding a gui is a lot more work, I consider a gui a hindrance in this case. There is nothing a gui would do for undvd that would improve how the program is being used.

Why not interactive?

There are some non-gui rippers/encoders out there, like h264enc and ripdvd (and probably a bunch of others). Common to them is that they run interactively. They ask you a series of questions to determine everything they need to know about ripping the DVD before they can start doing the work.

Unfortunately, interactive mode suffers from the same problems that a gui interface does. You have to answer all the questions whether you know what to answer or you don’t. And more importantly, it’s hard to reproduce the result, because you don’t remember what you answered the last time.

And crucially, an interactive program won’t let you just run it and go do something else, you have to stick around to do your part before the app will do its part. With a ripper this isn’t such a big deal, because all the interaction happens at the beginning, but it’s still something I like to do without.

Why bash?

The standard way of building a ripper is to wrap an interface around the scary mencoder command line interface. Whether this is a gui or not has no impact on how the ripper interacts with mencoder. There is no programmatic interface to mencoder, so you’re stuck running it from a shell one way or the other.

Taking this into account, a bash script is pretty much the easiest way to handle shell commands. (If you’re a perl nut, perhaps that would suit you better, but I’m not.) I’ve tried running commands from python (and using it to capture the exit code, stdout, stderr etc), and it’s far easier just to use plain bash.

As a self imposed restriction, it will also keep the program from becoming too big. Bash is not exactly the greatest abstraction language and takes quite a bit of discipline to keep it from getting totally convoluted, which is why I would not use it for anything bigger than a simple application.

Feature creep

Unfortunately, every program that evolves will face the issue of new features. Personally I think undvd does what it does well enough, I’m satisfied. But any one-size-fits-all strategy is bound to be somewhat constrained. The question is where to stop?

Two things came up that weren’t intended for undvd:

Scaling

To start off with the first issue, undvd makes the assumption that the output video will be smaller than the original. This is fairly sensible, given that most rips are done this way, and considering that 6x compression does demand a lot from your encoder unless you compromise a bit on dimensions. Crucially, even if you watch a movie full screen the software/hardware scaling does a good enough job of stretching the image without clobbering it. Having said that, undvd’s decision to scale to 2/3 of the original size is arbitrary and I accept that it’s not a well justified constraint.

So scaling will be added.

Bitrate and 2-pass encoding

The bitrate issue, on the other hand, is hairy. Very hairy. I kept thinking about a meaningful way to set the bitrate. The obvious thing is to make it an integer parameter, like say 1000kbps. But what is that? It’s just a number, a meaningless number. The “right” bitrate differs from movie to movie, from codec to codec, perhaps even from encoder to encoder. We are back in Photoshop land.

So I follow the convention of setting a file output size instead. If you say you want this title ripped to a file of 700mb, the bitrate is calculated automatically. This method is flawed, however, because the size we can accommodate for the video depends on how much space the audio requires, and there’s no way to know this exactly. (Currently the mean audio bitrate is presumed 160kbps.) So the output size tends to come within 10% of the user-specified size.

The thing is, if you start messing with the bitrate you should also consider 2-pass encoding, because you can mess up your video quite badly that way. undvd tries to do “the right thing” by picking 2-pass encoding if the calculated bitrate is below the default. But you can always override to 1-pass or 2-pass whatever the case is.

In any event, if you don’t use these new options then undvd does exactly what it always has.

And now for some screenshots

So that’s what I set out to do. It took me *a lot* of testing and playing with the settings to find a recipe that both gives great quality and doesn’t take too long. And still there may be, and probably will be, cases where the results aren’t great. But for my own use, it works very well. My main goal was to hide as many details as possible from the user, turning the complicated maze that is mencoder into a single button to push. As it turns out, however, it’s really hard to abstract away everything completely, so even with undvd there is a (hopefully modest) learning curve.

undvd is a collection of a couple bash scripts, which I decided to base on lsdvd and mencoder, part of mplayer. In doing so, I wanted to use the disc as little as possible, considering all the problems I’ve had with reading dvds in the past. I also found out that by extracting the vob, some of the information about audio/subtitles is lost, so I first clone the disc with dd, and then go to work on it. The script starts off by making an image of the disc, whereupon the disc is no longer needed.

First, to see what’s on the disc, run scandvd.sh.

At this point, you have to decide on which title(s) to rip. If you don’t know what they are, scandvd.sh suggests using mplayer to find out. Once you know what to rip, you run undvd.sh with the chosen options. Just keep in mind that the files will be created in the directory you run undvd.sh from, so make sure you have enough disk space.

What is worth noting here is everything that you don’t see. mencoder is run in the background, with a host of complicated settings, but you don’t see the horrifying output. You only see the status of what is happening, the settings you chose, and mencoder’s estimated time to completion. Sure, the full log is there if you want it, just say the word. But unless something goes wrong, you don’t need to see it, this will do just fine.

After ripping is finished, what you’ll have is the files shown. 01.avi and 02.avi are the titles. disc.iso is the image of the dvd, which you can use to rip more titles still, or just delete. And then there’s logs that you won’t bother even looking at unless something went haywire.

And that is dvd ripping reduced to one line of output for every title. Simple, isn’t it?

Get undvd from opendesktop.org.

A technical note

Make sure you have lsdvd and mplayer installed (with support for encoding, x264, xvid, and mp3/mad).