Wednesday, March 4, 2009

Augeas for libzypp and zypper?

The need for a configuration file for zypper (plus my wish for a nice CLI for handling it) and a visit to Raphaël Pinson's presentation at FOSDEM made me look at Augeas closely.

Augeas enables you to read and edit your config files in a way which preserves comments and formatting details, allowing humans and programs coexist peacefully when it comes to editing configuration files. More than that, you can map some special structures of the file to special parts of the augeas tree, rather than using the generic INI file parser. See the zypp.conf for example (zypper.conf will have the same structure):

## Option description.
## Default: etc.
# disabledoption = value

## Another option description

another.option = value

With Augeas you can write a lens (the file contents description) which will map such file into a tree like this:

/files/etc/zypp/zypp.conf/main/1/description[1] = "Option description."
/files/etc/zypp/zypp.conf/main/1/description[2] = "Default: etc."
/files/etc/zypp/zypp.conf/main/1/disabledoption = "value"
/files/etc/zypp/zypp.conf/main/2/description[1] = "Another option description."
/files/etc/zypp/zypp.conf/main/2/another.option = "value"

Note the commented node. Augeas has an API for looking for the nodes, getting values from them, setting the values, and finally saving the tree back to the config file.

What does this mean for zypper? Implementing zypper conf --list is a piece of cake, as well as zypper conf --set another.option value. Commenting or uncommenting the option is a matter of adding or removing the commented node. Getting the option description for zypper conf --help some.option is easy (and maybe it is possible to map the multi-line ## description to a single 'description' node!).

Here is a draft of lens for zypper (it will need comments and further improvements). To try it out you can crete a /etc/zypp/zypper.conf file with contents like the above example, and use the augtool like this:

$ augtool -I /the/dir/with/the/zypper.aug/file
augtool> print /files/etc/zypp/zypper.conf/
/files/etc/zypp/zypper.conf/anon/description[1] = "Configuration file for zypper"
/files/etc/zypp/zypper.conf/anon/description[3] = "Boolean values are 0 1 yes no on off true false"
/files/etc/zypp/zypper.conf/main/1/description[1] = "Whether to use colors"
/files/etc/zypp/zypper.conf/main/1/description[3] = "Default value: autodetected"
/files/etc/zypp/zypper.conf/main/1/colors = "yes"

Type help for other commands, use TAB key for completion, and check also the Augeas home page for more info.

What does this mean for libzypp? Basically the above, with respect to libzypp's files. Currently, you either edit your repos.d/*.repo, locks, credentials, and other files by hand or you leave it to libzypp's frontends, or you at least take care not to write comments in those files, because they will all get removed upon the next change done by libzypp (what we do there is to rewrite the entire files from in-memory data upon saving). This can be easily avoided using Augeas and we can add an editing API for zypp.conf. Plus the bonuses described above.

The library package (libaugeas0) would add 300 kiB of installed size to libzypp's/zypper's dependencies, (which is not little), plus an optional 200 kiB of the official lenses (augeas-lenses), if we decide to use them.

todo: Look at Augeas' error reporting. The user can of course break the structure of the file in a way that the lens can't map it to the tree anymore. If parsing of the file will fail, we need to report why, so that the user can fix it.


Martin Vidner said...

I thought augeas is overkill and surely there must be a smaller and simpler library for your purposes. I found these, but I am not sure they can handle the current file format.

libconfig (in Packman as libconfigduo)

jano said...

Yeah, i guess. I should have looked at the size first. But is the 220/300k (32bit,64bit) really too much? I mean readline has 300k on 64bit, too :O) But, yeah, it's already in the base system.

I tend to agree it's too much, i just want to give it a good thought before giving up on Augeas.

jano said...

Glance on libconfuse and libconfig: the former is 75k, the latter 140k on 64bit. Better. But it also enforces one config file format (plus add some other features).

Which might be OK for zypper... OTOH it would mean saving 225k or 160k of installed size over losing some nice features of Augeas.

And if we later decide to do something about the problems of writing zypp files, libconfuse and libconfig are not usable, while Augeas is.

jano said...

Another thought: zypper could also do with the existing zypp's INI parser and stick only with reading the config file. Editing it would be up on the user. But this solution is not that nice.