Home Home > 2008 > 06 > 06 > Sneak Peeks at openSUSE 11.0: Package Management, with Duncan Mac-Vicar
Sign up | Login

Sneak Peeks at openSUSE 11.0: Package Management, with Duncan Mac-Vicar

June 6th, 2008 by

In this article we will be covering all of the changes in and around the package management stack in the upcoming openSUSE 11.0. There have been a plethora of both visual and behind-the-scenes changes. We’ll also be talking to Duncan Mac-Vicar, YaST team lead, ZYpp and KDE developer, to find out a little more later.


Behind the Scenes

New Metadata

One of the major changes resulting in the lightning-fast package management of openSUSE 11.0 are the new SOLV files used now for metadata. While the classic RPM-MD (YUM) metadata in XML format is nicely readable, it results in significantly larger files and takes much longer to parse than is needed. The new dictionary-based SOLV format for repositories are up to 1/3 of the size and can be parsed in virtually an instant.

New Solver

The old solver had several problems. It was extremely slow in some cases, had a few bad design decisions, and provided bad diagnostics and suggestions if a particular case was unsolvable.

Faster

The new SAT solver by Michael Schroeder is based on expressing package dependencies as a boolean satisfiability problem. This in itself brings huge advantages as it is a well-researched problem (many example solvers available), it’s incredibly fast, and there is no need for complex algorithms. Indeed, package solving complexity is extremely low in comparison to other areas where SAT solvers are used.

To see a demonstration of just how much faster it is, see Duncan’s video comparison of the old and new Zypper.

Performs Better

Furthermore, these changes with SOLV files and the new solver have resulted in significantly better performance, with particularly reduced memory usage when compared to both Smart and YUM:

Smarter

One of the prized features of the Smart Package Manager was its ability to make smarter decisions with package management where APT and YUM fail. In particular, a few cases were proposed in Smart’s README where Smart behaves very well. So how does the new ZYpp stack do with these cases? It passes them all.

One of the other surprising yet handy features of the new package management stack is that it can be involved in hardware recommendation of packages. Want to get your webcam working? Plug it in and run zypper up for example (or with YaST) and it will try to grab all of the drivers from the online repository!

Interoperability

Patches and Patterns

One of the main advantages of openSUSE package management has been the availability of patches and patterns. Patches are small updates to fix a problem (used in the official update repository), and patterns are intelligent groups of packages which can recommend, require and suggest packages in order to make certain functionality available, without being too strict in the specific packages to install (so the more troublesome metapackage solution is not needed).

Fedora’s update metadata uses a yum plugin and a updateinfo.xml description; metadata for deltarpm availability is handled via the yum-presto plugin. In openSUSE 11.0, the package management stack reads patches from this file too! This means that you can use the yum stack out of the box, and you can generate patches using existing Fedora tools as well. Furthermore, there are continued efforts to build ZYpp and YaST on Fedora.

PackageKit

PackageKit is a D-Bus abstraction layer that allows the session user to manage packages in a secure way using a cross-distro, cross-architecture API. openSUSE 11.0 is fully PackageKit-enabled, meaning that all upstream tools across distributions using PackageKit will work perfectly on openSUSE.

See Duncan’s blog post on this for more information.

New Features

YaST

Both the Qt (KDE) and GTK (GNOME) versions of YaST have seen several changes, and in particular there are improvements to both package manager front-ends. Integration with PackageKit now means that there is a clearer view of all the package groups, with icons to distinguish them quickly:

Package Groups

The patterns view has also been improved:

Patterns

The GTK front-end is now based on an entirely new, clean design:

GTK YaST

Repository management can now also take place from directly inside the package manager. Just head over to Repositories -> Repository Manager. The Community Repositories module has also been integrated into here, so you can still easily select from a list of popular community repositories to add.

Updater Applets

Integration with PackageKit is tightened, with GNOME in openSUSE now using the PackageKit updater applet for all official update handling:

GNOME Updater Applet

The KDE updater applet has now also been ported to KDE 4, and has an optional PackageKit backend.

Zypper

While Zypper is significantly faster as the result of all the previous package management changes, there have been a lot of new features added to Zypper, including:

Install remote and local RPMs seamlessly:
root:~ # zypper install http://download.opensuse.org/repositories/KDE:/Community/openSUSE_11.0/i586/filelight-1.0-7.3.i586.rpm
Reading installed packages...
 
The following NEW package is going to be installed:
filelight
 
Overall download size: 582.0 K. After the operation, additional 1.2 M will be used.
Continue? [YES/no]: y
Downloading package filelight-1.0-7.3.i586 (1/1), 582.0 K (1.2 M unpacked)
Installing: filelight-1.0-7.3 [done]
 
root:~ # zypper install ./banshee-0.13.2-79.i586.rpm
Reading installed packages...
The following NEW packages are going to be installed:
 
tango-icon-theme taglib-sharp gnome-themes gnome-audio yast2-control-center-gnome podsleuth nautilus-cd-burner nautilus metacity libssui0 libssui libgweather1
libgweather libgtop-2_0-7 libgtop libgnomesu0 libgnomesu libgnomeprintui libgnomeprint libgnomekbd libgnomecups libgnome-menu2 libgnome-desktop-2-2
libexempi3 libeel-2-2 gnome-vfs-sharp2 gnome-sharp2 gnome-settings-daemon gnome-panel gnome-mount gnome-main-menu gnome-desktop gnome-control-center
glade-sharp2 gconf-sharp2 evolution-data-server eel banshee-plugins-extra banshee-plugins-default banshee-engine-gst banshee art-sharp2 PolicyKit-gnome-libs
  
Overall download size: 17.0 M. After the operation, additional 70.4 M will be used.
Continue? [YES/no]:

Support for wildcards:
root:~ # zypper install *ktouch
Reading installed packages...
 
The following NEW package is going to be installed:
kde4-ktouch
  
Overall download size: 1.4 M. After the operation, additional 3.2 M will be used.
Continue? [YES/no]:

Stay tuned to opensuse.org/Zypper/Changes/11.0 for a more complete list.

Talk with Duncan Mac-Vicar

What have been the main challenges with changing major components of the package management again?

During openSUSE 10.3 we invested a lot of effort into restructuring the libzypp API so that we could actually change things. For 11.0 this paid off. We did not change any API! (just add some, like locks, and other to allow access to low level stuff, like the SAT namespace). So the challenge was to modify the classes so they act like a thin wrapper over the sat solver library. Michael Andres and Stefan Schubert did a grat job in this part. Once this was done, almost everything worked out of the box.

I would say the biggest obstacles where changing things where the basic concept also changed, like not installing product, patterns and patches anymore but use the satisfied concept. Ths pays off as a pure-rpm system, but we have to still mature the details.

There was also a lot of work for the sat solver team, who got this fast and awesome C library. The ZYpp team did a great job integrating it without many changes, but they had to add lot of features to provide all of the functionality that we had before.

What other shining features have been implemented behind the scenes?

The changes in PackageKit, which mean you can use any PackageKit application and it will use our package management engine and you won’t see the difference. Also, Delta RPMs are no longer tied to patches, they are just extra metadata in a repository, and libzypp calculates which ones it can use. This means that we could start offering delta RPMs for factory updates, for example, at any time. Also the format is compatible with yum-presto.

Our patches metadata is the same as the one used by yum, updateinfo.xml, which supports the build service strategy of building for multiple distributions. If deltarpms or update metadata is ever provided by the build service, there should be no difference if you use fedora or suse, yum or zypper. Also, if you have internal company infrastructure for generating your own updates you do not need to have two variations of this tools.

What are the plans for the future?

For the first time I think we are on the track to where we would like to be. Plans for the future include polishing, like more PackageKit work, enable user features like hardware recommends in the user interface (these features are there for years, but are not very visible), build service integration, adding semantic data, etc

Both comments and pings are currently closed.

32 Responses to “Sneak Peeks at openSUSE 11.0: Package Management, with Duncan Mac-Vicar”

  1. Thanks Duncan and openSUSE team! I am using RC1 and the package manager really flies :)
    Now it is time to polish it, I would suggest the following small changes for 11.0 release:

    – save search combobox entries between invokations
    – ability to hide new “debugsource” packages (and save the “hide” menu settings)
    – save order and width of package list header columns

    And for future 11.1 release I would like to see an option to scan for unused packages (packages that have no executable bin files, and are not required by other packages), so I could clean up the mess of libraries etc. that I never use. Or better: an advanced search/filter option, where you could specify multiple search criteria, e.g. “find all installed packages that are bigger than 500kB, do not contain any bin executables, and depend on “libqt-mt”.

    I used Synaptic on a friends Ubuntu machine, and felt like trapped in a cave, you could not even search for packages that depend on a specific library… but I felt a bit embrassed to see how fast it was. No longer, yast is now faster :) And not only this, RC1 feels faster than 10.3 or Ubuntu 8.04 as a whole!

  2. porcus

    Woohoo!

    It’s very cool. I realy wait new openSUSE release.

  3. jano

    Actually, ‘up’ will not grab them, but ‘dup’ or ‘inr’ will :O)

    • jano

      Ah, wanted to quote “Want to get your webcam working? Plug it in and run zypper up for example (or with YaST) and it will try to grab all of the drivers from the online repository!” in my reply. Apparently i did not manage to do it correctly and there’s no preview to check :O(

  4. well…

    there are great improvements and i have to congratulate everyone on the Yast team… it’s simply great!!!

    However it still ain’t completely intuitive for newbies to remove an application and everything related to it… :(

    if only there was already something to reverse the “1-Click Install” changes as easy as the “1-click install” itself… 8(

  5. R. J.

    I think this is one of the main things OpenSUSE has been lacking, and now we have it, just makes it even stronger as the best choice

  6. PeterPac

    I am working on a uninstaller but out side of the community for SuSe using the GoboLinux file system. Once I get it working without problems I will intro it into the community. We are working with a completely revamped Linux File Manager and File system and it is looking great so far. We just past the one year mark and have managed to get most of the bugs out while consolidating the file system to where a newbie can understand it but still have a ways to go.

    I joined GoboLinux as I was curious to see how their new file system works and to develop a uninstaller for SuSe since they have a pretty stable uninstaller presently but it will not work with the SuSe file system as of yet.

    • R. J.

      I can’t see what is wrong with the present way of uninstalling programs from Suse. I found using the software manager very easy when I switched from Windows to Linux.

      • I’m sure you don’t deal with newbies everyday as I do…

        Can, for example, an older person such as your mother or grandma remove software in Linux… at least as easy as they do it on windows???

        I can assure you one thing that is keeping some clients i have to move from windows to Linux is exactly this problem…

        • R. J.

          John asks can an older person remove programs in linux. My mother is in her 60’s. she moved from windows to linux and has no problem removing programs.

        • then, i’m sure your mother use to play whit computers (windows) a lot!!!

          And she probably knows more about computers then most of the people…

          I have friends around the 28 Years old that deal well whit windows and refuse to use Linux because they think ( Installing (before 1-click install)/)removing software a hard task!!!

          I can remove software well… and i’m not asking this for me!!!

          Just for those who don’t get this so well…

  7. Jeff

    I remember reading about the package management improvements a while back. Just after Beta 3 hit, I installed it on my laptop. It was faster than 10.3, but it didn’t really feel too impressive.

    Then I realized that YaST was looking at 13 repositories instead of 4. The four I wanted (OSS, non-OSS, Updates, Packman) were in there three times each, with at least two instances of each set to “automatically refresh”. And the DVD was still enabled as a repo.

    WOW! It flies now. It’s not as fast as what Debian users have at their disposal, but it makes up for it by having more functional tools. Unless I know exactly what I’m looking for, I feel trapped working with most Debian package manager tools – not so with YaST.

  8. Grósz Dániel from Hungary

    Is there a way to view package groups in a tree as before? Unless that, I wouldn’t call that part an improvement, and it is quite useless when there are patterns.

    • No.

      • Grósz Dániel

        Then IMHO it’s quite a stupid and inexplicable modification (while the package manager is greatly improved in other aspects). At the current complexity of a Linux distro with a two-level selection interface it is impossible to look through all the software to select those I might need to use (without reading the description of thousands of packages that I will never use). It’s also more difficult to find a software I need – there is search but it’s less reliable (the keywords might show up in the description of packages that have nothing to do with what I want and they might not appear in the description of the software I need). This is why there are the rpm groups and why most distributions include a least three-level package selection interface.

        • Duncan Mac-Vicar P.

          Till now, I have got only bad feedback for such change from you (you were also the one commenting in my blog). All the other people I asked found it better (as it allows to enhance it with other nice virtual searches and views, just like the recommended packages view).

          May be for 11.1 we could do something, I agreed removing it completely won’t make all users happy, but it is impossible to do that.

          Nobody could ever explain me how rpm groups made any sense at all. Normally trees don’t make sense at all, because in these times it is pretty hard to classify something under ONE category, like, should Amarok be under KDE applications, or in Music Players or under MP3 Tag editor? The packagekit grouplist is very basic just because that.

          The tree was really complex, and the zzzAll thing is a joke for a user interface.

          We will look into making something that could fulfill all tastes though. I do agree that we went to close with patterns, and now it could be hard to tell the difference for people not knowing that groups are provided by the package and patterns by distributors.

        • Grósz Dániel

          Maybe I’m alone with that I like to install all the software I might want to use at the installation, and so go through all the available programs – but not read the descriptions of the programs in groups that I won’t use.

          Yes, it’s a problem that a tree cannot solve everything – but this interface does not solve those problems. A tree in which a package can appear more than once (e. g. at Misic players and MP3 tag editors) could solve this.

          The zzzAll really looked like a joke. I used it to one thing: to review the changes I made (together with sorting by the status) but this got a separate view a few releases before.

          What I can’t see is the drawback of having a tree. In the tree-view interface one can click on a top-level category and view all the packages in it. So if this interface had sub-trees, users who don’t want a tree could click on the top-level categories the same way, even without noticing the “+” signs at the categories (provided that the sub-trees are closed by default).

        • > “A tree in which a package can appear more than once (e. g. at Misic players and MP3 tag editors) could solve this.”

          I totally agree with you!!!

          There should be an easier way to remove software… and a Tree View could solve this easily!!!

          I came up with a few ideas i posted on

          opensuse.awardspace.com

          They are all intended to solve the newbies problem to remove software… and i really think a tree view could help… so… if my comment count’s for something… here a vote for the tree view ;)

        • Blaine Simpson

          I agree completely with Grosz. I don’t install all of my software during OS install, but it is extremely useful to work with narrowed lists under Office… if I only want to add Office apps, etc. Though with Smart, it’s very difficult to filter by size, platform, etc., it has excellent ability to select the view tree, including ability to sort by software groups underneath source channel at the highest level.

          I really hoped Zypper would approach Smart speed, interdependency handling, and end-user convenience finally, but I see they’ve only succeeded with the first two of these. I’ve never had any serious wrapped-around-the-axle problems with Smart (which happened constantly until now with Zypp when mixing RPMs from various sources or versions); and I suspect Smart will come out with a version with performance which leap-frogs Zypp (they’ve had no performance competition for years).

        • Duncan Mac-Vicar

          What do you think guys about something like a tag cloud for 11.1 ?

          http://en.wikipedia.org/wiki/Tag_cloud

          There packages could appear under various tags, we could add tags on runtime based on different criterias (coming from keywords, groups, etc)

        • Blaine Simpson

          I don’t see how the tag values could be populated automatically and consistently. A brief look at tag cloud looks like their values are derived based on web site use cases.

          More importantly… why? There is already a RPM Spec field “Group” specifically for categorizing the rpm in a tree, and it has worked just fine for me for 15 years. As I think somebody else said in this thread, it’s great to be able to expand and collapse branches. Additional information already in the RPMs (like platform, size, dependencies) can be derived from the RPMs without a new abstraction.

  9. Grósz Dániel

    Have the dependency error messages improved? In 10.3 zypper sends quite cryptic messages and the YaST interface is not very nice for it.

  10. Tom

    At the end of the day, package manager speed will also include the amount of DOWNLOADED DATA in a repo refresh task. How will/does openSUSE 11.0 compare in that department to Ubuntu, Fedora, Mandriva, etc.?

    I am currently running openSUSE 10.3 and using SMART package manager for general system updates. I find the repo refresh load is quite hefty. Regardless of whether my installed packages have updates, refresh will occur if there are new packages in the repos and the current repo refresh download for 10.3 updates is 12+ MB and 10.2+ MB for Packman. Almost 25MB download and I may not need to update even one package, HOW SMART IS THAT? :-)

    • cjk

      Well, that’s yum’s metadata format smartness all the way.

    • Duncan Mac-Vicar

      For 11.1 the idea is to generate the solv files in the server and avoid downloading xml if the solv files are there.

      We just did not make it in time for 11.0 with the huge amount of things with higher priority :-)

  11. Ryan

    Duncan’s idea that this could further integrate the community (i.e. make if more interoperable) is a fantastic idea. But perhaps I’m not understanding how this package manager works; not that I’m wanting more explanation. I’m hoping that my computer, at some future time, will avoid the foreseeable consequence of downloading a patch from Fedora that will fix something on their end and break something on mine. After all, I downloaded Kiba, and I know it’s a toy, but it broke another toy – Compiz. Waa!, pacify me. It probably just moved a pointer assuming my window manager was Metacity and was probably something easy to fix. But, instead of looking into it, I just 1-Click installed Compiz again. Please don’t misunderstand me, Duncan knows much more than I do about the technology, so I’m letting the doctor handle the scalpel and I (the layman) will stop peeking over his shoulder. Anyway, eagerly awaiting the final release!!

    • Duncan Mac-Vicar

      Our interest is not that you use openSUSE and get patches from fedora.

      Our interest is that if in your company you generate custom patches for fedora and suse machines, you need one set of tools to generate them (still two repos)
      You can use fedora tools on openSUSE (with openSUSE update repos)
      We could get openSUSE tools (even YaST) on Fedora, and it will work for them (using Fedora update repos)

      :-)

  12. anon

    Out of interest. Just looking at the screen shots. How come the Gnome Package Selection has more detail and really looks much nicer than the KDE one?

    • Grósz Dániel

      From the screenshot it seems to me that the KDE package selection interface gives the same, if not more information than the GTK one, expect the package tree. In other aspects, I would say that the KDE installer is more usable.

    • Duncan Mac-Vicar

      Some people have the taste in the wrong place. Others can’t figure out there are the same widgets in a different layout.

      Sorry, we can’t do anything for you. Just keep using your favorite one.

  13. cristian diaz riveros

    Muy bien trabajo Duncan se nota que estan haciendo las cosas muy bien allá en novell y por sobre todo en opensuse, ya he probado opensuse 11 aca en Chile donde tu sabes la banda ancha no es muy rapida como en otros paises y el cambio desde las versiones de de opensuse 10.3 a la 11 rc1 son notables en cuanto a rapidez de actualizaciones y gestion de paquetes principalemente via yast2. felicitaciones por el trabajo.

    Una cosa que opensuse ya se merecia ya que por un tiempo, emigre a otras distibuciones como ubuntu por su facil y rapido manejo de paquete pero ahora me encuentro muy satisfecho con lo que se ha logrado hasta el momento

    bueno chao duncan un saludo desde Chile