yum benchmarks

June 18, 2009

About 30dozen people sent me to the test comparing yum to apt-deb. Always a fun comparison. It’d be even more fun if any of the numbers seemed accurate.

I put a short script together for myself – just to see how yum is doing.

Here’s the very simple script:

http://skvidal.fedorapeople.org/misc/yum-test.sh.txt

I ran it like this:

sudo ./yum-test.sh >> /tmp/simple-yum-results.txt 2>&1

Here are my results on my x61s laptop running F11:

http://skvidal.fedorapeople.org/misc/simple-yum-results.txt

Full disclosure. I had already updated the metadata b/c I didn’t feel like testing the network speed in this coffee shop. I also downloaded the pkgs that it installed already by pulling them into the cache with a yumdownloader. Again, I wanted to test yum (and rpm) not the bandwidth of  the coffee shop. I’m running yum 3.2.23-3.fc11 on a 686 install of f11.

Overall the numbers look pretty good to me. Run a test for yourself and point out your results below.

Update enabled repos for the benchmarks

sudo yum repolist
repo id             repo name                                 status
fedora              Fedora 11 – i386                          enabled: 13,289
updates            Fedora 11 – i386 – Updates                enabled:  1,988
repolist: 15,277

16 Responses to “yum benchmarks”

  1. Ray Says:

    Good stuff Seth. Yum is quite fast these days. Still though, the #1 thing I hear from my Ubuntu-ite friends who try Fedora every now and then is that apt-get is sooooooooooo much faster.

    Is the Debian/Ubuntu metadata somehow kept much smaller? I do admit, yum itself is fast and most of my waiting is done when metadata has to be pulled in for filelists or other reasons.

    But I would assume this happens on the apt-get side as well?

  2. skvidal Says:

    debian does not have file-based-dependencies. So the complete filelists are never necessary for depsolving. So, in short, no, debs do not have this metadata.

    If you’d like to lead the charge to forever get rid of file deps in fedora/rpm please go for it.

    • Ray Says:

      Makes sense. I wonder how large of an undertaking this would be?

      Definitely didn’t mean to come across as overly critical btw. I’m perfectly happy with yum and thumb my nose at those silly Ubuntu users 😉

      • skvidal Says:

        No sweat. If you want to see how many file deps there are try this command:

        repoquery -a –requires | grep ^/ | sort |uniq

        just as an example

  3. jef spaleta Says:

    For information which repositories were enabled during that run and what was the number of total package count across repositories? Even duplicates..as duplicates sitting in multiple repository branches still show up as metadata that yum needs to parse..or am I wrong about that? I’d like to understand the benchmarks as a function of repository/metadata size if possible.

    Your comment about network speed I think is an important point. My real concern with comparative benchmarks associated with yum and apt has always been about methodology concerning the testing of comparable operations. I think people make bad assumptions as to when and where yum and apt are doing network io.

    I’d like to see a methodology described that ensures a comparative benchmark for comparable operations and excluding network io for both applications.

    -jef

  4. cornel panceac Says:

    well, on my dual core am2/2800, the times from your script were all very small. however, i’ve just seen a bunch of updates coming down the tubes, so i timed the yum update on my x40 notebook:

    real 16m30.296s
    user 3m57.293s
    sys 0m33.789s

    the first part, downloading included, was

    Total 239 kB/s | 52 MB 03:41

    so how is that a 33 seconds process ends in a 16.5 minutes time wasted?

    i confess this really is above my understanding in this very moment 🙂

    • skvidal Says:

      Did you capture the output from running that script? If so, please post it – it’ll help in showing where time is being used. Though with a number like 16m I suspect it’s all network time.

      • cornel panceac Says:

        no, it was a simplified version, just

        time yum -y –disablepresto update

        the output was this:

        Loaded plugins: fastestmirror, presto, refresh-packagekit
        Loading mirror speeds from cached hostfile
        * fedora: mirror.arlug.ro
        * rpmfusion-free: http://ftp.nb.lug.ro
        * rpmfusion-free-updates: http://ftp.nb.lug.ro
        * rpmfusion-nonfree: http://ftp.nb.lug.ro
        * rpmfusion-nonfree-updates: http://ftp.nb.lug.ro
        * updates: mirror.andreas-mueller.com
        Setting up Update Process
        Resolving Dependencies
        –> Running transaction check
        —> Package GConf2.i586 0:2.26.2-1.fc11 set to be updated
        —> Package GConf2-gtk.i586 0:2.26.2-1.fc11 set to be updated
        —> Package PackageKit.i586 0:0.4.8-1.fc11 set to be updated
        —> Package PackageKit-glib.i586 0:0.4.8-1.fc11 set to be updated
        —> Package PackageKit-gstreamer-plugin.i586 0:0.4.8-1.fc11 set to be updated
        —> Package PackageKit-gtk-module.i586 0:0.4.8-1.fc11 set to be updated
        —> Package PackageKit-yum.i586 0:0.4.8-1.fc11 set to be updated
        —> Package PackageKit-yum-plugin.i586 0:0.4.8-1.fc11 set to be updated
        —> Package anacron.i586 0:2.3-75.fc11 set to be updated
        —> Package brasero.i586 0:2.26.2-1.fc11 set to be updated
        —> Package brasero-libs.i586 0:2.26.2-1.fc11 set to be updated
        —> Package brasero-nautilus.i586 0:2.26.2-1.fc11 set to be updated
        —> Package cheese.i586 0:2.26.2-1.fc11 set to be updated
        —> Package cronie.i586 0:1.3-1.fc11 set to be updated
        —> Package empathy-libs.i586 0:2.26.2-1.fc11 set to be updated
        —> Package eog.i586 0:2.26.2-1.fc11 set to be updated
        —> Package evince.i586 0:2.26.2-1.fc11 set to be updated
        —> Package fipscheck-lib.i586 0:1.2.0-1a.fc11 set to be updated
        –> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.2.0-1a.fc11.i586
        —> Package gcalctool.i586 0:5.26.2-1.fc11 set to be updated
        —> Package gdb.i586 0:6.8.50.20090302-27.fc11 set to be updated
        —> Package gedit.i586 1:2.26.2-1.fc11 set to be updated
        —> Package glib2.i586 0:2.20.3-1.fc11 set to be updated
        —> Package gnome-applets.i586 1:2.26.2-1.fc11 set to be updated
        —> Package gnome-desktop.i586 0:2.26.2-1.fc11 set to be updated
        —> Package gnome-panel.i586 0:2.26.2-1.fc11 set to be updated
        —> Package gnome-panel-libs.i586 0:2.26.2-1.fc11 set to be updated
        —> Package gnome-settings-daemon.i586 0:2.26.1-7.fc11 set to be updated
        —> Package gnome-themes.noarch 0:2.26.2-1.fc11 set to be updated
        —> Package gstreamer.i586 0:0.10.23-2.fc11 set to be updated
        —> Package gstreamer-tools.i586 0:0.10.23-2.fc11 set to be updated
        —> Package gtk2.i586 0:2.16.2-1.fc11 set to be updated
        —> Package gtk2-engines.i586 0:2.18.2-1.fc11 set to be updated
        —> Package gtksourceview2.i586 0:2.6.2-1.fc11 set to be updated
        —> Package gucharmap.i586 0:2.26.2-1.fc11 set to be updated
        —> Package libpng.i586 2:1.2.37-1.fc11 set to be updated
        —> Package libsoup.i586 0:2.26.2-1.fc11 set to be updated
        —> Package mousetweaks.i586 0:2.26.2-1.fc11 set to be updated
        —> Package orca.i586 0:2.26.2-2.fc11 set to be updated
        —> Package pango.i586 0:1.24.2-1.fc11 set to be updated
        —> Package pygtksourceview.i586 0:2.6.0-1.fc11 set to be updated
        —> Package vlgothic-fonts.noarch 0:20090612-1.fc11 set to be updated
        —> Package vlgothic-fonts-common.noarch 0:20090612-1.fc11 set to be updated
        –> Running transaction check
        —> Package fipscheck.i586 0:1.2.0-1a.fc11 set to be updated
        –> Finished Dependency Resolution

        Dependencies Resolved

        ================================================================================
        Package Arch Version Repository
        Size
        ================================================================================
        Installing:
        fipscheck-lib i586 1.2.0-1a.fc11 updates 7.7 k
        replacing fipscheck.i586 1.1.1-1.fc11

        Updating:
        GConf2 i586 2.26.2-1.fc11 updates 1.7 M
        GConf2-gtk i586 2.26.2-1.fc11 updates 22 k
        PackageKit i586 0.4.8-1.fc11 updates 458 k
        PackageKit-glib i586 0.4.8-1.fc11 updates 136 k
        PackageKit-gstreamer-plugin i586 0.4.8-1.fc11 updates 73 k
        PackageKit-gtk-module i586 0.4.8-1.fc11 updates 72 k
        PackageKit-yum i586 0.4.8-1.fc11 updates 132 k
        PackageKit-yum-plugin i586 0.4.8-1.fc11 updates 69 k
        anacron i586 2.3-75.fc11 updates 42 k
        brasero i586 2.26.2-1.fc11 updates 3.6 M
        brasero-libs i586 2.26.2-1.fc11 updates 65 k
        brasero-nautilus i586 2.26.2-1.fc11 updates 19 k
        cheese i586 2.26.2-1.fc11 updates 2.7 M
        cronie i586 1.3-1.fc11 updates 74 k
        empathy-libs i586 2.26.2-1.fc11 updates 256 k
        eog i586 2.26.2-1.fc11 updates 2.0 M
        evince i586 2.26.2-1.fc11 updates 1.4 M
        gcalctool i586 5.26.2-1.fc11 updates 1.7 M
        gdb i586 6.8.50.20090302-27.fc11 updates 2.2 M
        gedit i586 1:2.26.2-1.fc11 updates 4.4 M
        glib2 i586 2.20.3-1.fc11 updates 1.5 M
        gnome-applets i586 1:2.26.2-1.fc11 updates 7.3 M
        gnome-desktop i586 2.26.2-1.fc11 updates 1.1 M
        gnome-panel i586 2.26.2-1.fc11 updates 2.8 M
        gnome-panel-libs i586 2.26.2-1.fc11 updates 62 k
        gnome-settings-daemon i586 2.26.1-7.fc11 updates 517 k
        gnome-themes noarch 2.26.2-1.fc11 updates 1.7 M
        gstreamer i586 0.10.23-2.fc11 updates 812 k
        gstreamer-tools i586 0.10.23-2.fc11 updates 22 k
        gtk2 i586 2.16.2-1.fc11 updates 4.3 M
        gtk2-engines i586 2.18.2-1.fc11 updates 383 k
        gtksourceview2 i586 2.6.2-1.fc11 updates 759 k
        gucharmap i586 2.26.2-1.fc11 updates 2.5 M
        libpng i586 2:1.2.37-1.fc11 updates 256 k
        libsoup i586 2.26.2-1.fc11 updates 166 k
        mousetweaks i586 2.26.2-1.fc11 updates 1.1 M
        orca i586 2.26.2-2.fc11 updates 2.4 M
        pango i586 1.24.2-1.fc11 updates 400 k
        pygtksourceview i586 2.6.0-1.fc11 updates 35 k
        vlgothic-fonts noarch 20090612-1.fc11 updates 2.3 M
        vlgothic-fonts-common noarch 20090612-1.fc11 updates 16 k
        Updating for dependencies:
        fipscheck i586 1.2.0-1a.fc11 updates 14 k

        Transaction Summary
        ================================================================================
        Install 1 Package(s)
        Update 42 Package(s)
        Remove 0 Package(s)

        Total download size: 52 M
        Downloading Packages:
        ——————————————————————————–
        Total 239 kB/s | 52 MB 03:41
        Running rpm_check_debug
        Running Transaction Test
        Finished Transaction Test
        Transaction Test Succeeded
        Running Transaction
        Updating : vlgothic-fonts-common-20090612-1.fc11.noarch 1/85
        Updating : vlgothic-fonts-20090612-1.fc11.noarch 2/85
        Updating : glib2-2.20.3-1.fc11.i586 3/85
        Updating : pango-1.24.2-1.fc11.i586 4/85
        Updating : GConf2-2.26.2-1.fc11.i586 5/85
        Updating : libsoup-2.26.2-1.fc11.i586 6/85
        Updating : 2:libpng-1.2.37-1.fc11.i586 7/85
        Updating : gtk2-2.16.2-1.fc11.i586 8/85
        Updating : gnome-desktop-2.26.2-1.fc11.i586 9/85
        Updating : gnome-panel-libs-2.26.2-1.fc11.i586 10/85
        Updating : brasero-libs-2.26.2-1.fc11.i586 11/85
        Updating : gtksourceview2-2.6.2-1.fc11.i586 12/85
        Updating : gucharmap-2.26.2-1.fc11.i586 13/85
        Updating : pygtksourceview-2.6.0-1.fc11.i586 14/85
        Updating : gnome-panel-2.26.2-1.fc11.i586 15/85
        Updating : gtk2-engines-2.18.2-1.fc11.i586 16/85
        Updating : gstreamer-tools-0.10.23-2.fc11.i586 17/85
        Updating : gstreamer-0.10.23-2.fc11.i586 18/85
        Updating : brasero-2.26.2-1.fc11.i586 19/85
        Updating : empathy-libs-2.26.2-1.fc11.i586 20/85
        Updating : evince-2.26.2-1.fc11.i586 21/85
        Updating : brasero-nautilus-2.26.2-1.fc11.i586 22/85
        Updating : cheese-2.26.2-1.fc11.i586 23/85
        Updating : 1:gnome-applets-2.26.2-1.fc11.i586 24/85
        Updating : 1:gedit-2.26.2-1.fc11.i586 25/85
        Updating : mousetweaks-2.26.2-1.fc11.i586 26/85
        Updating : gnome-settings-daemon-2.26.1-7.fc11.i586 27/85
        Updating : eog-2.26.2-1.fc11.i586 28/85
        Updating : gcalctool-5.26.2-1.fc11.i586 29/85
        Updating : GConf2-gtk-2.26.2-1.fc11.i586 30/85
        Updating : orca-2.26.2-2.fc11.i586 31/85
        Updating : gdb-6.8.50.20090302-27.fc11.i586 32/85
        Updating : anacron-2.3-75.fc11.i586 33/85
        Updating : cronie-1.3-1.fc11.i586 34/85
        Updating : gnome-themes-2.26.2-1.fc11.noarch 35/85
        Updating : PackageKit-glib-0.4.8-1.fc11.i586 36/85
        Installing : fipscheck-lib-1.2.0-1a.fc11.i586 37/85
        Updating : PackageKit-gtk-module-0.4.8-1.fc11.i586 38/85
        Updating : fipscheck-1.2.0-1a.fc11.i586 39/85
        Updating : PackageKit-gstreamer-plugin-0.4.8-1.fc11.i586 40/85
        Updating : PackageKit-yum-0.4.8-1.fc11.i586 41/85
        Updating : PackageKit-yum-plugin-0.4.8-1.fc11.i586 42/85
        Updating : PackageKit-0.4.8-1.fc11.i586 43/85
        Cleanup : fipscheck-1.1.1-1.fc11.i586 44/85
        Cleanup : gnome-desktop-2.26.1-4.fc11.i586 45/85
        Cleanup : orca-2.26.1-1.fc11.i586 46/85
        Cleanup : vlgothic-fonts-20090422-1.fc11.noarch 47/85
        Cleanup : cronie-1.2-7.fc11.i586 48/85
        Cleanup : gucharmap-2.26.1-1.fc11.i586 49/85
        Cleanup : PackageKit-glib-0.4.6-8.fc11.i586 50/85
        Cleanup : PackageKit-yum-0.4.6-8.fc11.i586 51/85
        Cleanup : anacron-2.3-74.fc11.i586 52/85
        Cleanup : PackageKit-yum-plugin-0.4.6-8.fc11.i586 53/85
        Cleanup : 1:gedit-2.26.1-1.fc11.i586 54/85
        Cleanup : brasero-2.26.1-3.fc11.i586 55/85
        Cleanup : PackageKit-gtk-module-0.4.6-8.fc11.i586 56/85
        Cleanup : PackageKit-gstreamer-plugin-0.4.6-8.fc11.i586 57/85
        Cleanup : gnome-themes-2.26.1-1.fc11.noarch 58/85
        Cleanup : glib2-2.20.1-1.fc11.i586 59/85
        Cleanup : gdb-6.8.50.20090302-26.fc11.i586 60/85
        Cleanup : libsoup-2.26.1-1.fc11.i586 61/85
        Cleanup : empathy-libs-2.26.1-1.fc11.i586 62/85
        Cleanup : gnome-settings-daemon-2.26.1-5.fc11.i586 63/85
        Cleanup : GConf2-2.26.0-2.fc11.i586 64/85
        Cleanup : gtk2-2.16.1-4.fc11.i586 65/85
        Cleanup : pygtksourceview-2.4.0-3.fc11.i586 66/85
        Cleanup : PackageKit-0.4.6-8.fc11.i586 67/85
        Cleanup : mousetweaks-2.26.0-1.fc11.i586 68/85
        Cleanup : gnome-panel-2.26.1-2.fc11.i586 69/85
        Cleanup : gstreamer-tools-0.10.23-1.fc11.i586 70/85
        Cleanup : gcalctool-5.26.1-1.fc11.i586 71/85
        Cleanup : gnome-panel-libs-2.26.1-2.fc11.i586 72/85
        Cleanup : pango-1.24.1-1.fc11.i586 73/85
        Cleanup : eog-2.26.1-1.fc11.i586 74/85
        Cleanup : GConf2-gtk-2.26.0-2.fc11.i586 75/85
        Cleanup : gstreamer-0.10.23-1.fc11.i586 76/85
        Cleanup : brasero-libs-2.26.1-3.fc11.i586 77/85
        Cleanup : gtksourceview2-2.6.1-1.fc11.i586 78/85
        Cleanup : evince-2.26.1-1.fc11.i586 79/85
        Cleanup : gtk2-engines-2.18.1-1.fc11.i586 80/85
        Cleanup : 2:libpng-1.2.35-1.fc11.i586 81/85
        Cleanup : vlgothic-fonts-common-20090422-1.fc11.noarch 82/85
        Cleanup : cheese-2.26.0-1.fc11.i586 83/85
        Cleanup : brasero-nautilus-2.26.1-3.fc11.i586 84/85
        Cleanup : 1:gnome-applets-2.26.1-3.fc11.i586 85/85

        Installed:
        fipscheck-lib.i586 0:1.2.0-1a.fc11

        Updated:
        GConf2.i586 0:2.26.2-1.fc11
        GConf2-gtk.i586 0:2.26.2-1.fc11
        PackageKit.i586 0:0.4.8-1.fc11
        PackageKit-glib.i586 0:0.4.8-1.fc11
        PackageKit-gstreamer-plugin.i586 0:0.4.8-1.fc11
        PackageKit-gtk-module.i586 0:0.4.8-1.fc11
        PackageKit-yum.i586 0:0.4.8-1.fc11
        PackageKit-yum-plugin.i586 0:0.4.8-1.fc11
        anacron.i586 0:2.3-75.fc11
        brasero.i586 0:2.26.2-1.fc11
        brasero-libs.i586 0:2.26.2-1.fc11
        brasero-nautilus.i586 0:2.26.2-1.fc11
        cheese.i586 0:2.26.2-1.fc11
        cronie.i586 0:1.3-1.fc11
        empathy-libs.i586 0:2.26.2-1.fc11
        eog.i586 0:2.26.2-1.fc11
        evince.i586 0:2.26.2-1.fc11
        gcalctool.i586 0:5.26.2-1.fc11
        gdb.i586 0:6.8.50.20090302-27.fc11
        gedit.i586 1:2.26.2-1.fc11
        glib2.i586 0:2.20.3-1.fc11
        gnome-applets.i586 1:2.26.2-1.fc11
        gnome-desktop.i586 0:2.26.2-1.fc11
        gnome-panel.i586 0:2.26.2-1.fc11
        gnome-panel-libs.i586 0:2.26.2-1.fc11
        gnome-settings-daemon.i586 0:2.26.1-7.fc11
        gnome-themes.noarch 0:2.26.2-1.fc11
        gstreamer.i586 0:0.10.23-2.fc11
        gstreamer-tools.i586 0:0.10.23-2.fc11
        gtk2.i586 0:2.16.2-1.fc11
        gtk2-engines.i586 0:2.18.2-1.fc11
        gtksourceview2.i586 0:2.6.2-1.fc11
        gucharmap.i586 0:2.26.2-1.fc11
        libpng.i586 2:1.2.37-1.fc11
        libsoup.i586 0:2.26.2-1.fc11
        mousetweaks.i586 0:2.26.2-1.fc11
        orca.i586 0:2.26.2-2.fc11
        pango.i586 0:1.24.2-1.fc11
        pygtksourceview.i586 0:2.6.0-1.fc11
        vlgothic-fonts.noarch 0:20090612-1.fc11
        vlgothic-fonts-common.noarch 0:20090612-1.fc11

        Dependency Updated:
        fipscheck.i586 0:1.2.0-1a.fc11

        Replaced:
        fipscheck.i586 0:1.1.1-1.fc11

        Complete!

        real 16m30.296s
        user 3m57.293s
        sys 0m33.789s

  5. skvidal Says:

    so if 3:41 was spent downloading and you had 16m total I’d bet about 11:30 of the remaining 12ish minutes was in the rpm transaction. Feel free to confirm them by running:

    yum -d 3 update

    the next time you have an update to run.

    but if you get a chance – run the actual test I posted, not something else entirely.

  6. skvidal Says:

    A couple of more. These are both the 2nd or 3rd runs after downloading the pkgs:

    http://jkeating.fedorapeople.org/yum.out.txt

    http://mmcgrath.fedorapeople.org/yum2.out


  7. These comparisons are always interesting. 🙂 Good stuff.

    As an aside, I always think one thing that gets overlooked in the yum vs. apt discussion is just how nice, clean, and informative yum’s output is compared to apt’s.

    Thanks for all the hard work, Seth.

    • skvidal Says:

      It hasn’t been just my hardwork for quite a while other folks who deserve ENORMOUS thanks for making things so much better:

      James Antill
      Florian Festi
      Jeremy Katz
      Tim Lauridisen
      Panu Matilainen

  8. Chris Smart Says:

    I’ll run this test when I get a chance and post the results, too.

    I would also be interested to see if the results change if you reboot between each test?

    -c

    • skvidal Says:

      I’m sure there are benefits from having a hot rpm cache – but to replicate actual uses of yum rebooting between each test isn’t helpful. People don’t run: yum list updates; then reboot then run yum update. They run them together.


Leave a comment