BHUGA WOOGA!

Yum, like all of the package managers, is seriously lacking a 'diff' function, i.e. making a machine's list of installed packages match the list of installed packages on another machine.

This is actually a hard problem, so I don't blame the tools for not having a built-in method. It's rare you want an *exactly* matching package list, anyway. Much more problematic is that Red Hat has package dependencies I would consider buggy. Xen, for example, depends on python-virtinst, some python libraries for creating virtual machines via libvirt, Red Hat's vain attempt to keep you from using the same stuff as everybody else. By adding a layer of python-based abstraction, your system can become less useful and proprietary at the same time, thus meeting several of Red Hat's business needs in one fell swoop!

It sounds like a perfect system, but Xen depends on python-virtinst, which depends on virt-viewer, which depends on gtk2.0. So installing Xen the Red Hat way means you'll install all of X-windows, and you'll get 4 weekly, very important security vulnerabilities about local privilege escalation attacks based on mouse drivers. Thus, Doing The Right Thing will probably result in packages you'd rather skip, which is why the generic solution would be very difficult.

I spent a few minutes in #yum on freenet, where a kindly fellow named Geppeto pointed me to the clear building block for any such solution, 'yum shell'. Yum shell is yum, but you can type transaction commands one at a time and attempt to run it, then modify the set if there was a failure; it's a lot easier than the command line. It's also happy to accept standard input. Thus the solution I'm writing up below.

First, a warning: you need to know what you're doing for this to be a realistic operation. You might have to override a dependency, and some of the components that might have problems could be utterly worthless or absolutely key to the operation of your system. If you're not careful, you'll destroy things. If any of the steps below seemed unintuitive to you, you probably don't want to be doing this.

First we'll need a list of packages on the good machine. Make sure to get the architecture in the package name if you're using CentOS or Red Hat, both of which have ridiculous dependency strings resulting in most of the OS being installed twice.

correct-host$ rpm --queryformat="%{n}.%{arch}\n" -qa | sort | uniq > uniq_good_packages.txt

good_packages.txt will look like this:

acl.x86_64
acpid.x86_64
alsa-lib.x86_64
amtu.x86_64
anacron.x86_64
apr-util.x86_64
apr.x86_64
.
.
.

We'll do the same thing on the target host:

[ben@nps1 ~] NPS PRODUCTION $ sudo rpm --queryformat="%{n}.%{arch}\n" -qa | sort | uniq > uniq_existing_packages.txt

Now we're into some tricks. We've got a list of packages and we need to make this a transaction set. Diff will get us 99% of what we need in terms of creating a transaction set, but there will be a few mistakes. It's also formatted incorrectly. The formatting is fixed easily with ruby or perl; here's what I used, saved as yum_diff.rb:

#!/usr/bin/env ruby
while line = gets do
  next unless line =~ /^(<|>)/
  line.sub!(/^(<)/,'remove ')
  line.sub!(/^(>)/,'install ')
  puts line
end
puts "run"

Give it a go, unless you have your yum configured not to verify before making a transaction. I keep such warnings on my system, so that I can be as sloppy as this solution is.

[ben@nps1 ~] NPS PRODUCTION $ diff uniq_existing_packages.txt uniq_good_packages.txt  | ./yum_diff.rb | sudo yum shell

You'll likely have issues. For example, my source system does not have python-iniparse installed, and I'm not sure if that's an OS version difference or what. The reason for the discrepancy doesn't matter: python-iniparse is required for *yum*, which would be pretty tragic to remove. Conflicts look like this:

---> Package python-iniparse.noarch 0:0.2.3-4.el5 set to be erased
--> Processing Dependency: python-iniparse for package: yum

Dependency transactions also appear at the end of the transaction to-do list, which will conveniently tell you all of the packages yum's planning to add/remove that you don't want it to.

Other conflicts may arise. I don't have 'device-mapper-event' on my source system, and not having it would have led to the removal of my kernel, via the dependency path of kernel requires mkinitrd requires lvm2 requires devivce-mapper-event. This is the part where you really have to know what you're doing, knowing what all of these packages do is key. Fortunately, we're doing a unixy solution, so the fix here is easy:

[ben@nps1 ~] NPS PRODUCTION $ diff uniq_existing_packages.txt uniq_good_packages.txt  | ./yum_diff.rb | grep -v python-iniparse | grep -v device-mapper-event | sudo yum shell

Now for the exciting part. Red Hat has what I would consider a bug in its dependencies, as mentioned at the beginning of this post. Xen requires python-virtinst requires virt-viewer requires gtk2.0 requires about the whole install DVD. Yuck. You REALLY need to know what you are doing here, as you need to know what python-virtinst does and what virt-viewer does; you can break your system doing this kind of stuff. Neither matter if you are not using libvirtd, which I do not (in fact, I've been unable to make it work with bridging at all. Who decides this stuff is the way to go?). So the solution is:

[ben@nps1 ~] NPS PRODUCTION $ sudo rpm -e --nodeps python-virtinst

The final command?

[ben@nps1 ~] NPS PRODUCTION $ diff uniq_existing_packages.txt uniq_good_packages.txt  | ./yum_diff.rb | grep -v python-iniparse | grep -v device-mapper-event | sudo yum shell -y

The result?

Install     13 Package(s)         
Update       1 Package(s)         
Remove     710 Package(s)
.
.
.
  Erasing   : at-spi-devel                 ##################### [722/724]
 
Removed: GConf2.i386 0:2.14.0-9.el5 GConf2.x86_64 0:2.14.0-9.el5 GConf2-devel.i386 0:2.14.0-9.el5 GConf2-devel.x86_64 0:2.14.0-9.el5 ImageMagick.x86_64 0:6.2.8.0-4.el5_1.1 ImageMagick.i386 0:6.2.8.0-4.el5_1.1 # 700 more packages  

This:

[ben@nps1 ~] NPS PRODUCTION $ rpm -qa | wc -l
1299

Has become:

[ben@nps1 ~] NPS PRODUCTION $ rpm -qa | wc -l
577

Beautiful!

The US today explicitly backed all of Frannie's debt. All of it. Sovereign wealth funds in Asia are quite happy. I guess they would be, since this will delay the riots that should rightly occur in protest of the business model of taxing people so that they can invest their money in fraudulent American mortgage debt.

I, on the other hand, take a more grim view. Fannie and Freddie have outstanding bonds worth more than 5 trillion. How much of that is actually covered by their performing loans? It's utterly impossible to say, and the treasury, while admitting that there were not-quite-illegal accounting tricks to delay reporting losses, has neglected to mention how bad the picture really is. I wonder if they really know.

In case you're wondering what it all means, every American taxpayer is now on the hook for some $47,000. An undetermined amount of that will be covered by Frannie income. That amount does not include social security or medicare. Fully one-third of that $47k are these ridiculous Frannie MBS's. The percentage of that we can expect to be covered can't be estimated , since these now-public-property entities don't seem to have any meaningful reporting requirements.

Consider: My niece was born in April (woohoo!) She was born on American soil. This means she's American, and, adjusting for inflation, she can expect to be personally on the hook for more than $65,000 in federal debt the day she starts her first high school McJob at 14. Naturally, this applies whether she lives in the US or not. All of the things this money bought--bridges, roads, dead Iraqi children and, as of today, overpriced McMansions 110 miles outside of Phoenix--will all be crumbling into disrepair by the time she might want them (although I'm rather hoping she never really wants any dead Iraqi children; I never did).

My niece was born into Original Debt. If Original Sin is what dooms us to death, Original Debt is what dooms us to taxes--why shouldn't the two universal constants both begin at birth, after all? It worked for feudalism, after all.

Original Debt, like Original Sin, has its origins in the foolish actions of mankind, tempted by low-hanging fruit. Fortunately, it has no particular misogynistic connotations: today's batch of fools, for example, are balding treasury technocrats.

Original Debt is stickier than Original Sin. Original Sin requires an hour a week to fix, or to simply stop being catholic. My niece's Original Debt will require from 1/4th to 1/3rd of her every working hour, for life. And there's no converting: Americans who renounce citizenship are on the hook for taxes for 10 further years, and gaining citizenship elsewhere is not easy.

And just like Original Sin, Original Debt is not not a legal liability, but a moral responsibility. Many religions believe that intent is not as important as action. In that sense, my niece, at her first job, would be culpable for almost everything America has done in the last 40 years. She'll pay for wars that were fought before she was born. She'll pay for the uncomfortable retirement of, statistically, her very own 1.3 or so retired boomers. She'll pay for levees in New Orleans and bridges to nowhere, prisons to hold drug offenders and university research facilities, nuclear bombs and fusion research, drug enforcement helicopters in Columbia and, of course, speculator's homes in half-built subdivisions in Ohio. She'll pay the interest on the debt that financed all of these things, and her moral choices will, in sense, be thusly made for her.

It's arbitrary. It's unfair. Ethics aside, it's completely unrealistic to expect her or anyone else her age to pay off the debts we are running up today. We'll see how that goes, I guess.

I evidently have much to learn on Drupal views. The books feed is http://bhuga.net/books-feed instead of the previously-mistaken location.