Home Home > 2007 > 08 > 26 > Sneak Peeks at openSUSE 10.3: New Package Management
Sign up | Login

Sneak Peeks at openSUSE 10.3: New Package Management

August 26th, 2007 by

openSUSE 10.3 is set to contain a new, significantly improved and more mature package management stack by default. ZMD, the package management component causing problems in SUSE Linux 10.1 and to a lesser extent in openSUSE 10.2, has been completely removed and is now replaced by the new libzypp and its tools. Today we’ll be taking a look at the new package management and talking to Duncan Mac-Vicar Prett, one of the central libzypp developers.

ZMD and Package Management Woes

ZENworks Management Daemon (ZMD) was part of the enterprise package management selection that was imported into SUSE Linux 10.1, and it used extra helpers in order to interact and synchronise with libzypp. ZMD offered many advanced features and provided rug, a powerful command-line tool for package management. Nevertheless, the integration of ZMD into openSUSE became problematic and it was eventually decided that the given features were not so necessary for openSUSE, but that it would remain on the SUSE Linux Enterprise products.

Looking Forward

So the ZYpp team have been working hard on improving libzypp, and there is a lot to show for it by this stage. It is reliable, more mature, and an awful lot faster. There is no more refreshing and parsing during startup, greater compatibility with tools like yum and smart, and increased speed for the most common use-case: installing a package.

Among other changes, one of the most important improvements is the new binary cache, which significantly improves start-up time and package installation. Originally caching the metadata is slow, but this is only done once and the data is then stored preparsed in the cache, meaning that you can read all repositories in a matter of a few seconds.

Repository handling has also changed. All repositories have a unique alias in the system, and adding a repository is simply a matter of copying a repo file, just like the ones used in the openSUSE Build Service, into the /etc/zypp/repos.d directory.

New Tools

In addition to this, two new tools were also created in order to work with the new libzypp:

  • zypper, an advanced, featureful command-line tool. Below are a couple of usage examples to see zypper in action:

Displaying your repository list:

opensuse:~ # zypper repos
# | Enabled | Refresh | Type | Name | URI
--+---------+---------+--------+---------------+--------------------------------------------------------------------------------
1 | Yes | Yes | yast2 | 10.3 - Main Repository (NON-OSS) | http://download.opensuse.org/distribution/SL-Factory-non-oss/inst-source-extra/
2 | Yes | Yes | rpm-md | KDE:Community | http://download.opensuse.org/repositories/KDE:/Community/openSUSE_Factory
3 | Yes | Yes | yast2 | 10.3 - Main Repository (OSS) | http://download.opensuse.org/distribution/SL-OSS-factory/inst-source/

Installing a package:

opensuse:/home/francis # zypper install filelight
* Reading repository '10.3 - Main Repository (NON-OSS)' cache
* Reading repository 'KDE:Community' cache
* Reading repository '10.3 - Main Repository (OSS)' cache
* Reading installed packages [100%]
The following NEW package is going to be installed:
filelight
Overall download size: 619.3 K. After the operation, additional 1.0 M will be used.
Continue? [y/n]: y
Downloading package filelight-1.0-6.1.i586, 619.3 K (1.0 M unpacked)
* Installing: filelight-1.0-6.1 [100%]

  • openSUSE Updater, a software updater applet that notifies you about software updates. The new one contains a small dialog to notify you of its progress:

opensuse-updater

The application interface of the updater has also improved, giving you more options and allowing you to select patches to be applied from directly within the application instead of YaST:

openSUSE updater in 10.3

Of course you can still continue to use YaST for all your package management requirements, and the interface itself has not undergone any significant changes through this release. You can help test and try out all of these tools by installing the recently released openSUSE 10.3 Beta 2.

 

Talk with Duncan Mac-Vicar Prett

I caught up with the man himself to find out a little more.

What have been the main tasks with the new package management stack in openSUSE 10.3?

We can summarize the project in this way:

For 10.1 we got YaST and Zenworks Linux Management working together, but with some rough edges. The resulting libZYpp library is shared between YaST and ZLM and brings the best of both worlds. ZLM got YaST flexibility regarding repository types (YaST, rpm metadata) and delivery formats (CDs, DVDs, nfs, smb, etc) while YaST took advantage of the redcarpet dependency resolver. However, YaST being mainly a GUI application vs. ZLM’s approach of a daemon (ZMD) with many small helpers didn’t really fit.

For 10.2 we focused on fixing the top issues, and I think we delivered a much more usable version which worked but suffered from the same conceptual issues.

For 10.3 we sat down and discussed how to improve the situation, in a reasonable time-frame. What we did was concentrate mostly on the 20% that caused 80% of the unhappiness, both for the users and for the developers. The task involved lot of profiling (with lot of help from other teams), designing and trying prototypes and proof of concepts. Then we branched and created a zypp2 library where we started to re-create all parsers, the cache engine, downloaders, etc from scratch, and unit tests for them. Later everything was integrated with the main library and zypp2 was removed. Then the next step was to port zypper and the YaST glue.

Who else has been working on it?

A lot of people have been involved in the project. Testsuite and solver development is lead by Stefan Schubert and Stefan Haas and recently also by Arvin Schnell. Jan Kupec and Martin Vidner hack on zypper. Thomas Göttlicher and Jörg Kress have fun with the notification applets. The YaST glue is maintaned by Ladislav Slezak. Katarina Machalkova, Stefan Hundhammer and Michael Meeks adapted the YaST user interfaces.

Michael Andres and I were in charge of ZYpp library development. Dirk Müller gave us a hand with profiling and contributed some code. Benjamin Weber contributed the one-click-install feature. Jiri Srain, Edith Parzefall, Klaus Kämpf and Stanislav Visnovsky coordinated human and technical aspects so we could deliver. Those are the people who worked directly in the refactoring project, but still the success always depends on more actors, project managers, quality assurance, etc.

What kind of advantages does this package management have over the old, pre SUSE Linux 10.1, package management?

The pre-10.1 stack, while mature, was showing its limitations. With the ZYpp based stacks we got:

  • A better resolver than before
    • Predictable behavior supported by a testsuite of upgrade and install scenarios
    • More information about why a package is installed or no solution is found
  • A better integration of all those feature that were added over the years to our package manager.
  • A common handling of packages *and* patches together
  • Dependency handling for update packages
  • A better way to handle selections (we call them “patterns” now)
  • More flexibility in handling of different repositories, e.g. it is possible to have additional patterns for each repository.
  • Additional dependencies based on language (for fonts, translations, etc.) or hardware (for drivers)

And now in 10.3 you will get:

  • Cleaner separation of different tasks like repository management, repository refresh, dependency resolution, package download and package installation. No more refresh and parsing during startup.
  • More compatibility with tools like yum and smart, we use the standard .repo files to list known repositories. The same format smart and yum uses and the same files you can find in the openSUSE build service repositories.
  • More speed for the common use case: install a package, search from the command line or upgrade your system.
  • More user-frendly notification applets.
  • A saner policy for 3rd party packages. They are not locked by default.

Upgrade candidates are considered from the same vendor, so you don’t jump automatically between feature-sets and vendors as new versions are available.

Any plans for the future?

Yes.

  • Keep improving the speed. Get rid of bottlenecks like the rpm database when searching.
  • Make zypper more useful for system administrator and advanced users.
  • Grab more community around our tools.
  • Innovate more around the assets we already have: Build Service. Community developments like the one-click-install to make openSUSE a distribution where installing, distributing and trying software is fun and a piece-of-cake.

Where can I find out more?

The project communication channels are:

* http://en.opensuse.org/Libzypp
* #yast on Freenode

Mailing lists:
* zypp-devel@opensuse.org (subscribe) for the core package management stack development
* software-mgmt@opensuse.org (subscribe) for openSUSE software management discussions, standards and ideas
* yast-devel@opensuse.org (subscribe) for general YaST development, which has a lot to do with ZYpp.

Thanks!

Both comments and pings are currently closed.

30 Responses to “Sneak Peeks at openSUSE 10.3: New Package Management”

  1. openSUSE

    Indeed, YaST is fast again! :D

  2. ra100

    to openSUSE:do you mean “software management” of YaST? Seriously? Are you sure?
    Becasue i wouldn´t say it is faster.I´d say it is slower than 10.2 even :(
    Longer searching of packages, longer checking of dependency, longer switching/changing over among repositories…
    I´m curious when or since which Beta it will be faster.So far i´m really disappointed

    • apokryphos

      Are you on openSUSE 10.3 Beta 2? There’s absolutely _no way_ it’s as slow as 10.2; it’s so, so much faster. It’s an awful lot faster; taking a look at some zypper times is evident enough of this fact. For example:

      desktop:/home/francis # time zypper repos
      ……
      real 0m0.084s
      user 0m0.060s
      sys 0m0.012s

      desktop:/home/francis # time zypper se kde
      ……
      real 0m3.795s
      user 0m3.160s
      sys 0m0.352s

      YaST’s Software Management is also really quick!

      • openSUSE

        Similar times here, I have three repos at the moment : oss, non-oss and updates :

        time zypper repos

        real 0m0.078s
        user 0m0.040s
        sys 0m0.012s

        time zypper se kde

        real 0m2.451s
        user 0m1.988s
        sys 0m0.276s

  3. openSUSE

    I meant both, YaST as YaST and as Software Manager. It is much faster than in 10.1 and 10.2. I would even say that it is fastest YaST/Package Manager so far! You said “Longer searching of packages, longer checking of dependency, longer switching/changing over among repositories…” but I have exactly opposite experience. Everything is just really fast. OTOH, beta2 is still too buggy for daily use.

  4. openSUSE

    And also new updater GUI looks awesome. But still kind of buggy ;) It said I have 0 security and 1 recommended update. But it didn’t say what is this recommended update, and now it is downloading something, I don’t know what it is, is it any good for me? Something like “show advanced information” option would be useful. And I think it freezed :D

    http://img59.imageshack.us/my.php?image=screenshotff9.png

    • Bart

      “Something like “show advanced information” option would be useful.”

      In KDE you can set that option if you rightclick on the icon and choose Configure Applet (or some like that). Maybe the Gnome-boys wanted to keep is ‘easy’ meaning the option is not available in Gnome.

      • Beineri

        The GNOME version still lacks some parts like the “Detail” dialog which would tell you more about the updates.

  5. User

    Smart hasn’t patterns, dependencies based on language or hardware, and upgrades can change vendor (vendor == repository?). But I am still curious about how the dependency resolver taken from redcarpet compares to the one from Smart. The “Case Studies” from http://svn.labix.org/smart/trunk/README look pretty impresive. Has libzypp the same problem that APT: “This clearly shows that the APT algorithm marks a single version as candidate, and when this is not the wanted version for some operation, the whole operation is compromised.” ?

  6. With SUSE 10.3 B2 we get a segmentation fault error message when trying to add software in YaST.

  7. ra100

    to apokryphos and openSUSE:
    So you are lucky boys ;)
    Yes i´m on Beta2.I haven´t still tried time zypper.I don´t usually use it in command line.But in GUI(i use KDE)it seems slower.Yes, it is faster than 10.1, but not faster than 10.2

  8. Marius

    I use the command line a lot(especially o the server side, i have 14 servers) and now i’m playing with 10.3b2 on my laptop. An “zypper patch-check” it seems to look up on all available repositories defined(mine where defined in the gui interface), but will show only results from the “suse update repositorie”(if defined) and also the “zypper update” will only update those updates(if any available). Will ignore other source like packman, guru or whatever, even if they are new packages available in them.
    The new GUI looks a lot better and is a lot faster but, from the “software management” interface, how do you find out if they are new packages available? the newer packages are not colored in blue(like where before) and their is no “upgrade if newer available” like it was in the old interface.

    • openSUSE

      “and also the “zypper update” will only update those updates(if any available). Will ignore other source like packman, guru or whatever, even if they are new packages available in them.”

      I remember that there was option you had to add to zypper to check updates from all sources, not only updates. But I forgot which option is it :rolleyes:

    • That won’t work because zypper update defaults to patches only. If you want to upgrade all your _packages_, you have to do something like:
      zypper update -t package
      As far as I know new packages should be still showing in blue; you could search the bugzilla to see if that’s a known issue.

  9. CzP

    About speed: my experience is, that it got faster of fast machine (2Ghz+ dualcore), about the same on a single core 2.4Ghz P4, and a lot slower on anything below.

    • ra100

      to CzP:
      Could it be true?.I have Athlon XP 2500+ (it´s about 1.8GHz) and i´ve even increased RAM from 768MB to 1.2GB.
      I don´t think it would be a good step, because everybody doesn´t have such a good hardware.

  10. Marius

    Yep, you are right, sorry. With a “zypper update -h| i got this:

    -t, –type Type of resolvable (package, patch, pattern, product) (default: patch)

    Sooo, we change the default of “patch” with “package” like: “zypper update -t package” and that does the trick.
    hmm, that’s kind of cute. Now i can use zypper instead of smart.

  11. Robin

    You can even configure the updater applet to update packages where no patch files are provided, so i.e. packman will be updated by the updater, too. (it’s an “expert” option)
    Btw, libzypp is even vendor aware, which means it won’t update from suse to packman to guru packages (whatever is the latest version).

    Looking forward :)

    • Robin

      oh, just saw that this vendor awareness was already mentioned in the article (I read it some days ago)

  12. Stefan

    I’m not convinced of the “better dependency checking” since I had a hard time to update from 10.3-a7 to 10.3-b2.
    There were dependency conflicts with involved packages cvs, tclsh, libblt and others.
    This could’nt be resolves by selecting thesse packages for deletion.
    At some point yast pretended that there are still dependency conflicts, but the list was empty!

  13. heksys

    It’s about time they got rid off that damm ZEN-UPDATER. I can’t wait for 10.3 to get here it’s going to be nice…

  14. lws

    Thank you very much by the article!
    A very good work!!

  15. Richard

    Is there any reason why the openSUSE Updater doesn’t work. I had this problem with it on 10.2, and in 10.3 beta 1, 2, and 3. All the repositories are correctly listed, but if I check for updates with the Updater applet, it says there are none. If I go into Yast’s software management I can see that there are a lot of updates.

    Any idea’s on what I am doing wrong?

    Thanks

  16. sina

    Every version upgrade in suse introduces a new annoying piece of software.
    In 10.1 and 10.2 ZMD used to take up 99% of CPU time for several minutes on every single start up, and now wow it is zypper!
    Every time I boot my machine this zypper thing is running for one to two minutes eating up CPU cycles and spinning Hard Drive like crazy.

    And who said SuSE boot time is twice faster in this version? yes the login screen comes faster but the time when the system is really at your service
    is same as before.

    • Anonymous

      zypper is the tool called by your desktop online update notification applet, you can configure how often/if it shall do update checks.

  17. peeved

    just write a yast plugin for yum and be done with it …

    Sick of .. YoU won’t do this, zypper wont do that, Software Management falls into a memory hole…. whilst big money in development terms seems to be spent on compiz /beryl and glorified eye candy the “Meat and Veg” of the disto like package management/system stability/repository clarity is falling by the way side. You could probably get a few good C programmers if you traded in a few of your gui dev people……

    I’ts been flakey since suse 9.2, so far I’m up to 10.3 GM … yeah GM. Greatly Mangled, your touting 11 in 60 something days when 10.3 is still actually beta qality. Darwin ports is starting to look effective compared this dogs breakfast.Its cool to want to surf, but earn the cred first, learn to paddle.