“It’s not the end of the world, but I’ve had better days.”

– Better Days – The Floating Men

yum results

March 20, 2008

A lot of work has been done on yum in the past few months. A whole lot. We’ve kept api compliance to yum 3.0.X (yay). But a lot of speed ups have gone in. I was getting ready to upgrade my laptop to rawhide and I decided to do a few tests.

I cleared out my cache and then I run yum makecache so everything was waiting for me and I wouldn’t get weird results due to network issues. I ran this command on both yum 3.2.8 (the current released version in fedora 7 and fedora 8 ) and on 3.2.13 which just came out today:

time echo ‘n’ | sudo ./yummain.py –disablerepo=’*’ \

–enablerepo=’development’ \

–enablerepo=’livna-development’ \

update

I’m echoing ‘n’ so the transaction doesn’t run. I just want it to setup, depsolve and exit, so I can measure memory use and time.

Yum 3.2.8: start->end of the above takes 2m11s. Max mem consumed 144MB. This is on my 1.2ghz (speed stepped to 800mhz) i686 laptop

Yum 3.2.13: takes 56s. Max mem consumed 106MB.

Same results in both cases but more than double the speed and 2/3rd the memory.

The actual dep-solve time in 3.2.13 is 28s. We didn’t have this information outputted separately for yum 3.2.8, so, I don’t have that number.

Some pretty good improvements, though.

Thanks to Florian Festi, James Antill and Tim Lauridisen for a lot of the good results we’ve gotten out of yum.

This is worth the cost of the paid account:

“Dpkg, of course, is the low-level package management tool used by Debian-based distributions; it is the direct counterpart to the RPM tool used by many other systems. Like RPM, it is a crucial component in that it determines how systems will be managed – and how much hair administrators will lose in the process. And, like RPM, it apparently causes a certain sort of instability in those who work with it for too long.” – Jonathan Corbet from: http://lwn.net/Articles/273714/

Just so we’re clear: I’d just like to say fedora had the problem of various forks of our package management system long before ubuntu did. Once again, fedora is ahead of all the rest.

woo hoo – water use

March 18, 2008

We got our new water bill today. Eunice and I got our water use down to 18 gallons per person per day for the last 2 months!

We rock. 🙂

Playing with various items to see how they impacted yum memory use today and we came up with a few interesting items:

Here’s the command we were running with more or less the same number of pkgs installed:

yum –disablerepo=’*’ –enablerepo=’development’ update

on an i686 box the memory use during the depsolve stage was 112M

on an x86_64 box the memory use during the depsolve stage was 409M

Now the number of pkgs available on the x86_64 box was more b/c x86_64 repos have about 25% more packages. But 25% does not account for a spike in memory use of 4x.

After a fair bit of messing around and eliminating variables I found this:

“Changed in 2.5: The number of extra bytes allocated is 4*sizeof(size_t). Before it was 16 on all boxes, reflecting that Python couldn’t make use of allocations >= 2**32 bytes even on 64-bit boxes before 2.5.”

From here: http://svn.python.org/projects/peps/trunk/pep-0353.txt

From what I can tell, we end up with a 2x to 3x bump on x86_64.

This makes me sad.

Anyone know of a way to beat/mitigate this other than ‘use fewer python objects’?

I wanted to respond to this comment by Brian Pepple. Everyone on the board has gotten uncomfortable with the idea that inclusion of the default set of items in codeina means we are encouraging the sale and use of closed source software.

We think codeina is very useful. As long as it is distributing only open source software then it is fine. It has not been doing that and we are all worried and concerned about it. We discussed it and decided an easy solution was to patch out the closed-source items that are advertised/offered in the xml file that ships with codeina.

I cannot think of a tenet more central to fedora than ONLY AND ALWAYS FREE SOFTWARE.  I think the decision of the board is consistent with that tenet.

bug day success

March 14, 2008

James, Tim, James, Paul, Florian, Jeremy, Jeff and I killed lots of bugs today. We went from 91 open yum bugs to 45 bugs left open. And I believe every open bug got commented on and updated if it couldn’t be solved and closed. That’s pretty rocking. yum-utils went from 36 open bugs to 13. Also pretty great. We checked in an enormous number of patches from various people and patches we put together to fix things here and there.

Bug day was full of productivity and that’s very good.

Toward the end of the day James A. hit me in the head with a not-so-obvious bug but 2 hours later we found it, squashed it and even corrected the test case. Victory is best served warm with a layer of Vietnamese food around it and maybe a bubble tea.

I think we might try to have bug days a bit more regularly for yum and maybe I’ll see about organizing some other events.