zypper-1.4.2: rewritten package selection code

Zypper has undergone a major refactoring of its package selection code recently. This was mainly to enable automatic testing and to make it easier and safer to add new features.
I'll submit zypper 1.4.2 including this change to Factory soon. It seems to work fine for the most part, but if you spot something that used to work for you in 1.4.1, please let us know via bugzilla. Don't forget to attach both zypper.log and solver test case.

The new version already enables use of unified package arguments (in install, update, and remove commands) in the form of:

[+/-][repo:][type:]name[.arch][OPevr[.arch]]

± (or ~|!)                        ; install/remove modifier
repo  =                           ; repo alias, number, or name
type  = patch|pattern|product     ; if not specified 'package' isimplied
name                              ; can even be a glob
OP    = -|=|>=|<=|>|<             ; version operator
evr   = [epoch:]version[-release] ; edition (version)

E.g. $ zypper in packman:xine-ui-0.99.5cvs20091115-0.pm.1.4 (don't forget to quote the args, if they contain ?/*/</> or spaces).

More about all this (and about a few more features added in the last months) later ...

There is still some work to do to fully support this (especially don't try foo.arch :O), but the main focus now is to make sure that all that used to work before works also now. For future plans see roadmap and comments in the SolverRequester_test code. Don't hesitate to send feedback to or discuss this stuff on zypp-devel@opensuse.org or comment this blog post.

Improved Installation Summary in Zypper

And the best thing last (for now) :O) This was one of the most wanted features for a year now (bnc #389128). Here's how it currently (zypper 1.2.0) looks.

If the architecture or the vendor of an installed package is about to be changed, the old and the new arch/vendor is shown next to the package name:

The following packages are going to change vendor:
  kde4-akonadi                openSUSE -> openSUSE Build Service
  kde4-knemo                  http://packman.links2linux.de -> openSUSE Build Service
  oxygen-icon-theme           openSUSE -> openSUSE Build Service
  oxygen-icon-theme-scalable  openSUSE -> openSUSE Build Service

Other information, like version or repository is not shown by default, like it was before (i argue that it would be too much info for the default output):

The following NEW packages are going to be installed:
  dmidecode libpkcs11-helper1 libstorage libx86emu1

The following packages are going to be upgraded:
  ConsoleKit ConsoleKit-32bit ConsoleKit-x11
...
Continue? [y/n/?] (y):

But you can toggle display of such info in the 'Continue?' dialog. Enter '?' to show all the available options:

Continue? [y/n/?] (y): ?

y - Yes, accept the summary and proceed with installation/removal of packages.
n - No, cancel the operation.
v - Toggle display of package versions.
a - Toggle display of package architectures.
r - Toggle display of repositories from which the packages will be installed.
m - Toggle display of package vendor names.
d - Toggle between showing all details and as few details as possible.
g - View the summary in pager.

[y/n/?] (y):

Enter 'v' to see the versions:

[y/n/?] (y): v
The following NEW packages are going to be installed:
  dmidecode          2.10-3.4
  libpkcs11-helper1  1.07-1.7
  libstorage         2.18.16-1.1
  libx86emu1         1.1-3.1

The following packages are going to be upgraded:
  ConsoleKit        0.3.0-1.5 -> 0.3.0-1.6
  ConsoleKit-32bit  0.3.0-1.5 -> 0.3.0-1.6
  ConsoleKit-x11    0.3.0-1.5 -> 0.3.0-1.6

If the list is too long, enter 'g' to view the result in pager.

Also, the package counts are shown at the end:

1230 packages to upgrade, 2 to downgrade, 5 new, 2 to remove, 4  to change vendor.
Overall download size: 1.02 GiB. After the operation, 175.0 KiB will be freed.

If you have colors enabled the numbers will be highlighted and the ones to-be-removed and to-change-vendor are shown in red to catch your attention.

Enjoy, and let us know what you think!

todo: Add an option to show the dependency graph of the listed packages. That will answer the questions why zypper wants to install or remove a particular package.
todo: Add an option to view detailed package info.
todo: Add an option to view the changelog (esp. of the to-be-upgraded packages).

Zypper Dialogs

I was puzzled by how to make good user dialogs (or prompts) in Zypper for quite some time. My idea was that they should:

• be translatable, including the answers (or options)
• have default option, that would be chosen with a simple Enter key stroke (or automatically in non-interactive mode)
• have the default option visibly marked
• have help
• be simple (have expert options hidden from default view)

This has gradually evolved into the following:

Download (curl) error for 'http://foo.bar/beer/content':
Error code: Connection failed
Error message: Couldn't resolve host 'foo.bar'

Abort, retry, ignore? [a/r/i/?] (a): ?

a - Skip retrieval of the file and abort current operation.
r - Try to retrieve the file again.
i - Skip retrieval of the file and try to continue with the operation without the file.
u - Change current base URI and try retrieving the file again.

[a/r/i/?] (a):

And a translated version:

Ukončiť, znova, ignorovať? [k/z/i/?] (k): ?

k - Preskoč súčasný súbor a ukonči operáciu.
z - Skús stiahnuť súbor znova.
i - Preskoč súčasný súbor a pokračuj v operácii bez neho.
u - Zmeň základnú URI adresu a skús súbor stiahnuť znova.

[k/z/i/?] (k):

• the prompt options are made of a string like "a/r/i/u", which is marked for translation. Translators are advised in the source comment about the meaning of the options and that they
  should use lowercase letters to translate them (but even a longer string would work).


• The default option is chosen by the programmer, using its position (0 in the above example). On Enter, this option would be used as an answer.


• The default option is visibly shown in separate parentheses. First i thought highlighting it with some color would be nice, but then my colleagues convinced me that showing different output in color/non-color mode was not a good idea.


• Whenever there's help or more options available, a question mark (?) is shown as one of the possible options. When entered, the help would be shown, as you can see in the examples above.


• Programmer can choose how many options should be shown, the rest will be hidden. Only the most important options should be shown, the rest will be listed after entering the '?'.

It's surely not perfect, but hopefully quite usable. Debian's apt does something similar, but i'm not sure about the translations and other details.

todo: Allow to pass arguments together with the option as part of the answer. This would allow to e.g. remove particular package from the list of to-be-installed packages when seeing the summary. Or show detailed info (e.g. changelog) of a package that caught your attention in the summary before starting the installation.

(oh, did i mention that we'd be glad to receive patches implementing any of the todo's i mention in my blog posts? :O) If you like zypper and would enjoy some coding in C++, don't hesitate to contact us).

Configure Your Zypper

So i mentioned zypper.conf in my previous post? Yes, Zypper now (since 1.2.0) reads configuration file from /etc/zypp/zypper.conf and from your $HOME/.zypper.conf. It does not have too much to offer right now, appart from the colors, but the point is now we can easily add new ones there, whenever there's a need.

If there is Zypper option that you write too often, it should have it's configuration file counterpart as well, so that it saves you typing. I actually have a list of few, collected over time from users, so i'll be adding them soon. You'll always find the latest list of supported config options here, so stay tuned (and let us know if you need something that's not there yet).

So which value will be used after all, the one from the config file, or the one passed via command line? Here is the override priority order, highest to lowest.

1. command line options
2. options from $HOME/.zypper.conf
3. options from /etc/zypp/zypper.conf
4. and the lowest priority have the ones from zypp.conf, so that you can override libzypp's settings in zypper, if zypper allows it.

You can use an alternative config file via --config custom/zypper.conf global option. If you do, the other config files (item 2 and 3 above) will be ignored.

todo: Add command for manipulating the configuration files as an alternative to manual editing. The command would list available options with current values and descriptions and would allow to set new values, or restore the defaults.

Colors in Zypper

Zypper 1.2.0 is coming to Factory soon, bringing a few new features i will briefly describe in a few posts.

My last post was about colors in terminal applications, so i'll start with colors in zypper. There is new [color] section in zypper.conf (oh, i should have started with zypper.conf.. nevermind :O), where you can enable or disable colorization of the output, tell zypper whether you use dark or light terminal background, and, finally, select your own color for each kind of output, if you don't like the defaults.

By default, zypper will print progress messages from ongoing operation in white, finished stuff in grey, result in white, errors in red (surprise :O), and apart from this general stuff there are some special things highlighted, like the package counts in the installation summary. More is yet to come (but i don't want to overcolorify it, of course).



Here are the config options. Those who do not want to wait for the new package to reach Factory can get it from zypp::Head OBS repo.

So, play around, enjoy, and let me know what you think.

zypper's wiki updated

openSUSE 11.0 is out so i synced zypper's usage page with it. This changes page may be of particular interest for those already familiar with zypper. I also took a while to reorganize the main page a bit.

If there is something wrong or something you'd like to see in those pages, let me know (or just add/change it). Feedback and help is welcome.

(Writing docs is quite some work, it took me whole week to update those pages and the man page (which still has some glitches). Didn't i already tell to myself that i must make sure to update them as soon as any change happens? Well, but that's one week less for development... :O) oh...)

More options to modifyrepo

In 10.3, zypper modifyrepo (mr) only allowed you to enable/disable one repository and enable/disable autorefresh for it. In 11.0 the options have grown in number a bit. Did i mention the nice help texts?

$ zypper help mr
modifyrepo (mr) <options> <alias|#|uri>
modifyrepo (mr) <options> <--all|--remote|--local|--medium-type>

Modify properties of repositories specified by alias, number or URI
or by the '--all, --remote, --local, --medium-type' aggregate options.

Command options:
-d, --disable             Disable the repository (but don't remove it).
-e, --enable              Enable a disabled repository.
-r, --refresh             Enable auto-refresh of the repository.
-R, --no-refresh          Disable auto-refresh of the repository.
-n, --name                Set a descriptive name for the repository.
-p, --priority <1-99>     Set priority of the repository.
-k, --keep-packages       Enable RPM files caching.
-K, --no-keep-packages    Disable RPM files caching.
-a, --all                 Apply changes to all repositories.
-l, --local               Apply changes to all local repositories.
-t, --remote              Apply changes to all remote repositories.
-m, --medium-type <type>  Apply changes to repositories of specified type.

This is what you will see in zypper 0.11.3. The aggregate options (and the -k options) were added just recently by our new colleague Josef Reidinger. Those familiar with 10.3 zypper, note that the --enable-autorefresh/-a and --disable-autorefresh options are now deprecated (-a shorthand even had to be removed because of conflict with --all) in favor of --refresh and --no-refresh options. This change was introduced for the sake of consistency with other commands' options. All the options will be described in more detail in the man page soon.

Few examples

To disable autorefresh for all repositories, do:

$ zypper mr -Ra
Nothing to change for repository 'openSUSE-DVD 11.0'.
Autorefresh has been disabled for repository 'fate'.
Nothing to change for repository 'factory'.
Nothing to change for repository 'packman'.
Autorefresh has been disabled for repository 'factory-nonoss'.
Autorefresh has been disabled for repository 'factory-debug'.
Nothing to change for repository 'vbox'.
Autorefresh has been disabled for repository 'zypp:svn'.

To disable all cd/dvd repositories, do:

$ zypper mr -d -m cd -m dvd
Repository 'openSUSE-DVD 11.0' has been sucessfully disabled.

To enable RPM caching of RPM files for all remote (http/https/ftp) repositories, do:

$ zypper mr -kt
Nothing to change for repository 'factory'.
RPM files caching has been enabled for repository 'factory-debug'.
RPM files caching has been enabled for repository 'factory-nonoss'.
RPM files caching has been enabled for repository 'fate'.
Nothing to change for repository 'packman'.
RPM files caching has been enabled for repository 'vbox'.
RPM files caching has been enabled for repository 'zypp:svn'.

Of course don't forget to use zypper clean when you run out of free disk space then :O)

zypper search is back

I said i would write about zypper's XML output almost two months ago. Hm... obviously other stuff gained higher priority (i will get back to the XML output later). In the openSUSE Updater applets, the focus has shifted from the zypp back-end to PackageKit and i focused on other zypper/libzypp features and fixes. I will write about them in the following few posts.

Just recently i re-enabled all the original zypper search functionality (except --match-all which will be hopefully comming soon). Apart from the original features, there is one frequently requested change in the presenation of the search results (and thanx to the zypp and sat-solver guys it is also fast :O).

The old zypper listed all available instances of a matching package, all versions from all repositories. But people are more often interested in what the package is about than what versions are available from each respository. Hence, since 0.11.1 you will see a list of packages grouped by their names and types with a summary column instead of repository and version column:

$ zypper search browser
Reading installed packages...

S | Name                | Summary                          | Type
--+---------------------+----------------------------------+--------
i | kim-browser         | CIM Browser for CIM-XML Protocol | package
  | mysql-query-browser | A Graphical MySQL Query Shell    | package
  | tvbrowser           | digital TV guide                 | package

This output caught a few users unprepared. I hope learning that --details/-s can turn on a detailed list similar to the old one satisfied most of them:

$ zypper search -s browser
Reading installed packages...

S | Name                | Type    | Version    | Arch   | Repository
--+---------------------+---------+------------+--------+-----------
i | kim-browser         | package | 0.3-208    | x86_64 | factory
v | kim-browser         | package | 0.3-208    | i586   | factory
  | mysql-query-browser | package | 5.0r12-183 | x86_64 | factory
  | mysql-query-browser | package | 5.0r12-183 | i586   | factory
  | tvbrowser           | package | 2.6.3-22   | noarch | factory

Since zypper 0.11.2 the summaries will be abbreviated as needed to fit the width of your console screen. This is how it looks on an 80 character wide console:

$ search music
Reading installed packages...

S | Name                  | Summary                                 | Type
--+-----------------------+-----------------------------------------+-----------
  | asc-music             | Music for Advanced Strategic Command--> | package
  | libmusicbrainz        | Library That Provides Access to the M-> | srcpackage
  | libmusicbrainz-devel  | Include Files and Libraries Mandatory-> | package
  | libmusicbrainz3       | Library That Provides Access to the M-> | srcpackage
  | libmusicbrainz3-6     | Library That Provides Access to the M-> | package
  | libmusicbrainz3-devel | Library That Provides Access to the M-> | package
i | libmusicbrainz4       | Library That Provides Access to the M-> | package
  | texlive-bin-musictex  | MusiXTeX and MusicTeX                   | package
  | texlive-musictex      | MusiXTeX and MusicTeX                   | package

Of course you can use --no-abbrev/-A global option to turn the abbreviation off - the output will be messy, but you can read the whole summaries. Another way to obtain this info is to use zypper info.

...and don't forget to try zypper help search, zypper has nice help texts :O) I'll be fixing zypper update and writing a lot of documentation in the meantime.

bash completion for zypper

Version 0.10.3 of zypper will include a bash completion script written by Marek Stopka during Hack Week 2. The script uses zypper help [command] output to retrieve the list of commands and command options in order to feed it to the command line. Given the growing number of zypper's commands and options i believe this will be a pleasant help for many users.

Thanx to Marek for doing this!

Currently the script provides completion of commands and command options. This can be extended in the future to include package name completion for the install, remove and info commands, or even repository aliases for the repo handling commands and the --repo option.