The big new change in undvd 0.4.0 was a dynamically set bitrate based on the bits-per-pixel (bpp) value. That addition not only completely changed the way undvd deals with bitrates, it also made possible further improvements.

The guiding principle of undvd remains be smart if possible, don't bother the user. A brand new feature is a smarter way to determine how to scale the video. This is another under-the-hood improvement which improves the program without introducing any more bloat or changing the user interface.

How do we scale?

To motivate the issue, let's consider an example. Until now, undvd scales to 2/3 of the original width.

For a 720x576 (5:4) dvd movie this gives target dimensions 480x384, for a total of 184,320 pixels. For a 2 hour movie (25 fps), this gives results in a video 771mb (plus the size of the audio, some 140mb).

480 * 384 * 25 * 0.195 * 7200 / 1024² / 8 = 771mb

But for a movie of different dimensions, a constant scale factor would yield a smaller number of pixels per frame. Suppose another movie has dimensions 720x480 (3:2). This would give target dimensions 480x320, a total of 153,600 pixels. The video size would, of course, be proportionally smaller as well.

480 * 320 * 25 * 0.195 * 7200 / 1024² / 8 = 642mb

This is probably not what you want. You give up the same proportion of pixels in both cases, but for a movie of smaller dimensions it would be nice to get the same frame size, ie. keep a greater portion of the frame. In the extreme case, for a movie with dimensions 720x224 you would probably want to keep the frame intact (161,280 pixels) and you would still get a smaller file.

Scaling by frame size, not width

So the obvious solution is to consider the number of pixels in a frame, not just along the width. As with the bpp in the previous increment, what used to be the default becomes the starting point for interpolation. So 720x576 will still scale to 480x384. But 720x480 becomes 528x352 (185,856 pixels) and 720x224 would not be downscaled at all.

What complicates the issue somewhat is the need to pick dimensions such that it comes to an even number of 16x16 pixel blocks (I'm not sure how crucial this actually is, but the absolute of majority of video files obey this rule). This reduces the number of possible dimensions for a movie of a given aspect ratio.

For instance, take dimensions 720x272. This gives a frame size just over the desired 184,320 pixels. But the next possible option is 256x96, which is tiny. Here undvd does the "smart" thing and picks the dimensions closest to the target size (in this case leaving it intact).

The nitty gritty

Deciding on a scaling factor is quite simple. Starting from the equations to scale the width and height separately,

width * factor = scaled_width

height * factor = scaled_height

we obtain a formula to find the number of pixels in a frame.

width * height * factor² = framesize

Assuming we know how many pixels we want in a frame irrespective of the dimensions, we can derive a formula to find the scaling factor.

factor = sqrt( framesize / (width * height) )

To reuse the example of the 720×480 movie, we find

0.730 = sqrt( 184,320 / (720 * 480) )

Scaling by this factor we get the target dimensions of the video.

720 * 0.730 = 525.813

480 * 0.730 = 350.542

It may not surprise you to know that the new dimensions do not divide evenly by 16. The nearest dimensions satisfying the multiples-of-16 rule are 480x320 and 528x352, but the latter is closer to our starting point.

Multiples of 16

Finding the width and height is a bit more hairy because we have to find two values that correspond. I don't think there is a formulaic solution to the problem because it amounts to solving this set of equations:

width * height = framesize

width = ratio * height (where ratio is known)

width = a * 16

height = b * 16

framesize = c * 16

The last three equations do not help, because they do not relate any two of the variables (width, height, framesize) to each other. And so we have the top two equations of three unknowns, which is not going to work. Intuitively, we know this is true, because we expect there are many sets of (width, height, framesize) that are possible solutions, so this set of equations will not give us a solution.

But it can be solved with an algorithm:

local ratio="$orig_height/$orig_width"

step=-1
unset completed
while [ ! "$completed" ]; do
	step=$(( $step + 1 ))

	local up_step=$(( $width + ($step * 16) ))
	local down_step=$(( $width - ($step * 16) ))
	for x_step in $down_step $up_step; do
		local x_width=$(( $x_step - ($x_step % 16) ))
		local x_height=$( echo "scale=0; $x_width*$ratio/1" | bc )
		if (( ($x_width % 16) + ($x_height % 16) == 0 )); then
			completed="y"
			width=$x_width
			height=$x_height
		fi
	done
done

Here step is the distance from our starting point, each time used to set the values up_step and down_step to successive upwards and downwards increments of 16 respectively (525+16, 525-16). For each of these we run the inner loop, setting x_width to the value that divides 16 within the range [x_step, x_step-16]. We now have a valid value for the width. We now use the ratio to find the corresponding height while preserving the aspect ratio, stored in x_height. If both x_width and x_height are multiples of 16, we have found dimensions for the video.

If we were to apply the Pareto principle to fit the world of software development, I would suggest the following translation.

Building 80% of the product takes 20% of the effort.

It's not surprising. A little code goes a long way. Typically problems we attack have a wide frontier we can cover with little effort. That is why we choose them. And while no software really can be said to be 100% complete in a mathematical sense, it certainly takes a great deal of effort to "finish it off" by handling all those special circumstances and corner cases.

What is wrong with 80% completion anyway? Isn't that good enough? Often it is. But there is that little voice in your head that says since you've already done "almost all of it", why not finish it? After all, there is a sense of accomplishment at stake, and since you've already attacked the problem, there is that urge to solve it completely.

undvd was 80% complete by version 0.2.2, with packages for Gentoo, Ubuntu and Fedora. Version 0.3.x added things I didn't plan to include at all when I started this project. The new features made it possible to set an output size (and thus indirectly set the bitrate), set 2 pass encoding, and set scaling. I was wary about compromising the key principle: undvd should be obvious and easy to use. But the new goodies were tucked into an "advanced" menu and so I don't think they did a lot of damage.

Since 0.3.0 there has been a host of tweaks and small fixes for small problems. One such problem was that undvd worked well on terminals with a black/dark background, but was not tested on a white/light background. That is unfortunate, given how gnome-terminal and konsole both use a light background by default. These kinds of fixes are enjoyable in that they improve the overall user experience in subtle ways, but without changing anything in the way you have to use the software.

Preparing to leap again

Having said that, one technical aspect I wasn't entirely pleased with was how undvd sets the bitrate. This is something of a black art, and I had done pretty intensive testing before deciding that 900kbps seemed ideal for h264 video. But debating the ideal bitrate is like debating the ideal size for a jpeg image. At the very least, it matters what the dimensions are. I had been scaling the image to 480 pixels wide in my tests, and so with greater dimensions the result wouldn't be as good.

I came across a very helpful article that deals with bitrates and also contrasts the h264 and xvid codecs. Discussing bitrate is pointless, because bitrate isn't a constant, it's a function of other quantities. What we need is a constant to standardize on.

Going back to the jpeg example, let's start with a frame of video. 480x384 gives 184,320 pixels, if we use one bit to store each pixel we come up with a quantity that is bits per pixel, abbreviated to bpp. So we're using 184,320 bits (180 kilobits, or 22 kilobytes) to store this one frame.

Let's take the single frame and another however many frames that fit into one second of video (25 is a common number). So now we have 25 of those 480x384 frames. If we now use one bit to store one pixel we need 25 bits for the job.

To complete the illustration, say we take our 480x384 frame 25 times per second, for a total of 2 hours (7200 seconds). If we used one bit per pixel, we used 25 pixels for 7200 seconds, which gives 180,000 bits per pixel in the movie. Multiply that by the number of pixels in one frame and we find 33,177,600,000 bits (30 gigabits, or 3.8 gigabytes).

Where does the bitrate come in? The bitrate is bits per seconds, so now that we have the number of bits, we divide by the number of seconds and find 4,608,000 (4500 kbps). If it isn't clear yet why the bitrate isn't a useful measure, it soon will be.

Getting it right

To formalize our deductions we draw up some equations. First, as we said, bpp is bits per pixel:

bpp = bits / pixel

To calculate the size (in bits) of the movie we took the bpp, the framerate (frames per second), the length (in seconds) and the dimensions (in pixels).

size = bpp * fps * length * width * height

And the bitrate (bits per second) is now the size divided by the length of the movie:

bitrate = size / length

We can also solve this equation for size, and substitute the formula into the previous one, which gives a different equation for the bitrate:

bitrate = bpp * fps * width * height

It should now be clear what the problem is with the bitrate as a measure of quality. It not only encapsulates the bpp, it also subsumes the dimensions and the frame rate of the movie!

Still not clear? Let's solve the equation for bpp.

bpp = bitrate / (width * height * fps)

Now let's assume we use the bitrate as a measure of quality. As mentioned, undvd up to version 0.3.x sets a constant bitrate of 900kbps (921,600 bits). Let's try that on a couple of examples.

0.200 = 921600 / (480 * 384 * 25)

When we plug the bitrate into the equation we find the bpp is 0.2.

But now suppose the user does not want any video scaling and the dimensions of the output video are 720x576.

0.088 = 921600 / (720 * 576 * 25)

Now we have less than half the space to store one pixel than we did in the first case, because we have more than twice the number of pixels in every frame. Clearly, the same number of bits to store this movie, spread across more pixels, will allow less for each pixel, and thus lower the quality.

Therefore, the bitrate needs to vary depending on the dimensions and the length of the movie. It is thus a function, not a constant! On the other hand, bpp is a constant, and so if we want a measure of quality we should use that instead.

Getting practical

The equations are easy. The question at hand is what value to set for the bpp. As shown in the example, 900kbps at the dimensions frequently used gives about 0.2 bpp, which is known to work well. Thanassis Tsiodras suggests that 0.2 bpp is enough for xvid, while h264 can be encoded at just 0.125 bpp. But considering that undvd is a one-size-fits-all solution, and therefore should encode any video well (even at the cost of filesize in cases where it's unnecessary), I would rather overshoot the target size a bit and play it safe. Therefore, based on my results, the default bpp is set to 0.195. In case of two pass encoding, we trade processing time for filesize, and the value is set to 0.150. For xvid the corresponding values are 0.250 and 0.200.

As before, these defaults can be overridden by setting the output filesize.

The best thing about this new functionality is that it doesn't invalidate any existing assumptions. For a default rip the result will be the same as always, which is supposed to be very good. But now it scales to custom video dimensions as well.

Something missing?

Pleased as I was that undvd now has a mathematical underpinning for deciding the bitrate, I was still not quite satisfied. It's nice and all that we now use a constant to determine the quality, but what is bpp anyway? How do we relate to it? And how do we know what a good value of bpp is?

To make this more tangible, I decided it can't just be a value in a calculation, it has to be made visible. undvd now has a "dry run" option -D, which shows all the parameters of the video to be encoded. The bpp value is highlighted, because that is our measure of quality. This image shows title 01 is chosen for ripping. The dimensions will be scaled to 480x384, the frame rate is 25 fps, the length is 148 minutes, the bpp is .195, the bitrate is thus calculated to 877kbps, we are using one pass encoding, and the h264 codec. The output size is estimated at 1122mb.

Short of showing the bpp for a title we are about to encode, it would also be useful to inspect videos we have encoded in the past, wouldn't it? That would give us an idea of what to expect.

For this there is a new tool vidstat, which displays the same information about any file or dvd title (much like stat displays information about a file on the filesystem) .

Here we see the first 60 seconds of title 01 (from above) being ripped. Then we run vidstat on the output file to examine it. Here we see that mencoder decided to use 809kbps instead of 877 as it was instructed to.

The bpp value is highlighted in yellow, red or green depending on whether undvd thinks it's unnecessarily high (above the single pass bpp default) or low (below the two pass bpp default), or just right (within the two).

I was pleasantly surprised today to discover that undvd has made its way into portage. :cool: It does now have three releases on sourceforge with packages for gentoo (ebuild), ubuntu and fedora. But I haven't made any requests for it to be included, so apparently the powers that be decided it's not completely useless. :party:

It's actually a bit of a dilemma what to do with these small projects. A big project just looks neglected without a website, complete documentation, packages for several distributions, up-to-date entries on various directory sites etc. I did the full scale follow-up with galleryforge, and that now speaks for itself, it presents itself as a proper project. But for one thing it was my first time hosting with one of these open source providers, so I wanted to explore the options quite deeply. And secondly, you don't really know what kind of a future a project will have when you're starting it, but galleryforge isn't used much, not even by me. So it was fine to do then, but it's some effort to put in for every small piece of code like this. (Not so much doing it then, but the burden of maintenance is the key issue.)

undvd is even smaller, and isn't expected to grow much (beyond feature complete by now). It started out as a blog entry and a link on my code page. Then I posted it on opendesktop, and it got seen by a handful of people who happened to be looking at the frontpage when it was added. There was also a regression bug posted there, which spurred me to start tracking it with git to avoid these things in the future. Then I decided I wanted to package it up, because that makes it organized and easy to use. But then I couldn't figure out how to write the url (in the ebuild) for the tarball hosted on opendesktop, because their file hosting is a little sucky. So I registered on sourceforge. The only thing I'm using there right now is the file release function. There's no website, and I don't really see the call for one. Meanwhile, I've been trying to maintain the user guide (which is shipped with the code) clear and up-to-date, to make that the one source of information. I think that's enough for now.

When you decide to code an application in a problem space that already has a rich bouquet of solutions the typical questions come to mind immediately. What is this going to accomplish? Why not just use an existing application? Why reinvent the wheel? Perfectly valid concerns, and yet every application has some reason to be, or it wouldn't exist in the first place. Sometimes it's a question of timing - better solutions did not exist at inception. Sometimes it's about quality: that one over there works but doesn't do it well enough. And sometimes it's something different altogether. That one does too much and I don't understand it.

There are people out there who are proper movie buffs. People who can quote lines out of a movie and identify passages to extreme detail. I've known a bunch of people like that, but I've never been like that myself. To me watching movies was never a really important thing, it was kind of a second tier interest. But knowing people like this has its advantages, you can borrow their movies. :cool: I remember how we used to lug around VHS tapes at school.

So my lack of outright passion about movies is really the reason why I never took after the dvd ripping crowd. When DVDs were fairly new, people were so excited about divx. My friends at school used to discuss bitrates and filesizes. I didn't follow any of that, I wasn't really interested. I didn't have a DVD drive, nor did I think paying €38 for a movie was such a good deal.

But admittedly, ripping the occasional movie is useful. DVDs in general are just a hassle and I don't like to put up with them. Back in those days (a few years later, when I had bought my first dvd drive for my first custom built computer) I would try it a few times. I didn't mirror the excitement of my friends. It took a really long time to rip a movie, and the result wasn't that good. I don't know what I was doing wrong. We used the Gordian Knot suite and followed the guides on Doom9. But while the program gave me every possibility to tweak the settings, I didn't know what they mean, I didn't *want* to know what they mean, I just wanted to rip it and move on.

And that is precisely the point where undvd comes in. I've tried a bunch of tools for ripping DVDs. dvd::rip might be the most merited linux tool. There's also acidrip and a bunch of others I don't even remember anymore. All these tools had the same two flaws:

• They present a lot of options that I don't know anything about, and I don't care about. And presumably, since my results were never very good, these are crucial for good results.
• It takes anything from 15 to 45 minutes to actually set up all the options, inputs, outputs, filters, cropping, the whole shebang. When something fails, like the disc can't be read from, the tool sometimes crashes. If it does, all your set-up work is lost. But even if you have a working setup from before and you want to rip another movie you still have to double check all these settings, because ripping takes a long time and if you mis set something by accident you may only discover it far along the way.

Therefore, I think it's fair to say that we have the Photoshop of DVD ripping in place. Personally I want to use Paint for this activity, because I have no deep interest in all the details.

Making it work

So if there is something complicated you have to do again and again, and every time you worry that you'll get it wrong and even if you get it right next time you probably don't remember what you did again. Faced with that, what do you do? In the middle ages monks would copy books by hand, at extraordinary effort. Not only would they transcribe the text, they would also copy and enhance the illustrations so that the copy, while not being the original, would be no less worthy. This was their calling, transcribing was an art form. Then came Gutenberg. Now granted, building a printing press took a lot more effort than transcribing a book. But once you had done that, copying books was a no brainer.

To me ripping a DVD is not an art form, it's a chore. And if I can figure out how to do it once and never have to worry about it again, then it's well worth the effort. So I set out to figure it out, I looked up the details and read the fine print. Eventually I had a fairly good idea of how to do it, and the results were coming along.

undvd became the wrapper around that nastiness I didn't want to look at or remember. And for my money it worked well. It also encompasses certain design decisions that were important to me.

Why no gui?

The Photoshop rippers all come with a gui interface. That's hardly a surprise, the plethora of settings they allow you to fiddle with would look like mplayer's list of options if they weren't gui apps. It's practically impossible *not* to have a gui for those guys.

But DVD ripping is not inherently a gui centric activity. You need the gui to write your missile launch plan, but once you start ripping the gui doesn't serve any purpose anymore. It just sits there in memory, and you could just as well remove it while ripping and then bring it back once it's over. So 95% of the time the gui isn't being used.

Apart from the simple fact that coding a gui is a lot more work, I consider a gui a hindrance in this case. There is nothing a gui would do for undvd that would improve how the program is being used.

Why not interactive?

There are some non-gui rippers/encoders out there, like h264enc and ripdvd (and probably a bunch of others). Common to them is that they run interactively. They ask you a series of questions to determine everything they need to know about ripping the DVD before they can start doing the work.

Unfortunately, interactive mode suffers from the same problems that a gui interface does. You have to answer all the questions whether you know what to answer or you don't. And more importantly, it's hard to reproduce the result, because you don't remember what you answered the last time.

And crucially, an interactive program won't let you just run it and go do something else, you have to stick around to do your part before the app will do its part. With a ripper this isn't such a big deal, because all the interaction happens at the beginning, but it's still something I like to do without.

Why bash?

The standard way of building a ripper is to wrap an interface around the scary mencoder command line interface. Whether this is a gui or not has no impact on how the ripper interacts with mencoder. There is no programmatic interface to mencoder, so you're stuck running it from a shell one way or the other.

Taking this into account, a bash script is pretty much the easiest way to handle shell commands. (If you're a perl nut, perhaps that would suit you better, but I'm not.) I've tried running commands from python (and using it to capture the exit code, stdout, stderr etc), and it's far easier just to use plain bash.

As a self imposed restriction, it will also keep the program from becoming too big. Bash is not exactly the greatest abstraction language and takes quite a bit of discipline to keep it from getting totally convoluted, which is why I would not use it for anything bigger than a simple application.

Feature creep

Unfortunately, every program that evolves will face the issue of new features. Personally I think undvd does what it does well enough, I'm satisfied. But any one-size-fits-all strategy is bound to be somewhat constrained. The question is where to stop?

Two things came up that weren't intended for undvd:

Scaling

To start off with the first issue, undvd makes the assumption that the output video will be smaller than the original. This is fairly sensible, given that most rips are done this way, and considering that 6x compression does demand a lot from your encoder unless you compromise a bit on dimensions. Crucially, even if you watch a movie full screen the software/hardware scaling does a good enough job of stretching the image without clobbering it. Having said that, undvd's decision to scale to 2/3 of the original size is arbitrary and I accept that it's not a well justified constraint.

So scaling will be added.

Bitrate and 2-pass encoding

The bitrate issue, on the other hand, is hairy. Very hairy. I kept thinking about a meaningful way to set the bitrate. The obvious thing is to make it an integer parameter, like say 1000kbps. But what is that? It's just a number, a meaningless number. The "right" bitrate differs from movie to movie, from codec to codec, perhaps even from encoder to encoder. We are back in Photoshop land.

So I follow the convention of setting a file output size instead. If you say you want this title ripped to a file of 700mb, the bitrate is calculated automatically. This method is flawed, however, because the size we can accommodate for the video depends on how much space the audio requires, and there's no way to know this exactly. (Currently the mean audio bitrate is presumed 160kbps.) So the output size tends to come within 10% of the user-specified size.

The thing is, if you start messing with the bitrate you should also consider 2-pass encoding, because you can mess up your video quite badly that way. undvd tries to do "the right thing" by picking 2-pass encoding if the calculated bitrate is below the default. But you can always override to 1-pass or 2-pass whatever the case is.

In any event, if you don't use these new options then undvd does exactly what it always has.

And now for some screenshots

I always found ripping dvds to be a huge pain, because of how complex the process is. There is a million ways to convert a dvd into avi format, a myriad of settings to play with, options to tune for performance, for size etc. That's great if you want to tinker. But it's much more difficult to give a straight answer to the question "how do I rip a dvd?" without going into all these details. I for one would like a simple way that would work on any dvd everytime.

So that's what I set out to do. It took me *a lot* of testing and playing with the settings to find a recipe that both gives great quality and doesn't take too long. And still there may be, and probably will be, cases where the results aren't great. But for my own use, it works very well. My main goal was to hide as many details as possible from the user, turning the complicated maze that is mencoder into a single button to push. As it turns out, however, it's really hard to abstract away everything completely, so even with undvd there is a (hopefully modest) learning curve.

undvd is a collection of a couple bash scripts, which I decided to base on lsdvd and mencoder, part of mplayer. In doing so, I wanted to use the disc as little as possible, considering all the problems I've had with reading dvds in the past. I also found out that by extracting the vob, some of the information about audio/subtitles is lost, so I first clone the disc with dd, and then go to work on it. The script starts off by making an image of the disc, whereupon the disc is no longer needed.

First, to see what's on the disc, run scandvd.sh.

At this point, you have to decide on which title(s) to rip. If you don't know what they are, scandvd.sh suggests using mplayer to find out. Once you know what to rip, you run undvd.sh with the chosen options. Just keep in mind that the files will be created in the directory you run undvd.sh from, so make sure you have enough disk space.

What is worth noting here is everything that you don't see. mencoder is run in the background, with a host of complicated settings, but you don't see the horrifying output. You only see the status of what is happening, the settings you chose, and mencoder's estimated time to completion. Sure, the full log is there if you want it, just say the word. But unless something goes wrong, you don't need to see it, this will do just fine.

After ripping is finished, what you'll have is the files shown. 01.avi and 02.avi are the titles. disc.iso is the image of the dvd, which you can use to rip more titles still, or just delete. And then there's logs that you won't bother even looking at unless something went haywire.

And that is dvd ripping reduced to one line of output for every title. Simple, isn't it? :)

Get undvd from opendesktop.org.

A technical note

Make sure you have lsdvd and mplayer installed (with support for encoding, x264, xvid, and mp3/mad).