Mancoosi Weblog

Ralf just got back from his trip to Boulder, Colorado, where he attended the 14th International ACM SIGSOFT Symposium on Component Based Software Engineering to present an article written by Pietro Abate, Roberto Di Cosmo, Ralf Treinen and Stefano Zacchiroli, titled MPM: a modular package manager. Guess what, in his suitcase (finally recovered after being lost in the air for a few days), he brought back an ACM SIGSOFTDistinguished Paper Award, received during the conference, that lasted from 21 to 23 June, 2011.

It's nice to see our work recognised, and we hope this is only the beginning of a fruitful collaboration with the CBSE community.

Sune Vuorela has added support for the Debian Weather service in the KDE Plasma Dataengine (here is the change).

Starting from the next KDE release, you'll be able to use the nice applet you can see in the above figure to monitor the amount of not installable packages in a Debian/suite architecture of your choice ... and avoid attempting upgrades during heavy rain!, such as the recent Perl 5.12 transition in Debian unstable.

In order to better support multiple package architectures, we have just changed the layout of the Mancoosi APT repository hosted at http://mancoosi.debian.net.

The new APT lines for your /etc/apt/sources.list are as follows:

  deb http://mancoosi.debian.net/debian/ unstable/all/
  deb http://mancoosi.debian.net/debian/ unstable/$(ARCH)/
  deb-src http://mancoosi.debian.net/debian/ unstable/source/

(don't worry about "$(ARCH)", it's a magic variable that gets filled by APT automatically)

Enjoy!

During the question time of my FOSDEM 2010 talk about cross-distro dependency resolution, several people, mostly interested sysadms, asked me how they could contribute upgrade scenarios to the Mancoosi project. The idea is simple: we are working on improving dependency resolution techniques, algorithms, and tools; to that end we need a corpus of upgrade scenarios where shortcomings of state of the art tools are evident. We already have quite some of such upgrade (partly provided by members of the Mancoosi project, partly generated, etc.), but we could use more.

So, I'm hereby happy to announce the first public release of the mancoosi-contest package, for Debian-based distributions. mancoosi-contest essentially offers a wrapper, called dudf-save, that you can pre-pend to your usual invocations of package managers such as apt-get and aptitude. What it does is to capture all package meta-data that concerns your request, as well as the request itself, and the package manager output. All these data, which as you can imagine can be quite big, are collected in a DUDF document which is then bzipped and (optionally) uploaded to a DUDF collector service. All in all it is very similar to the popularity-contest architecture but instead of collecting installed packages is collecting specific upgrade attempts.

Of course participation is completely optional and the package is currently being provided unofficially by the Mancoosi project. Nevertheless, if you feel like having stumbled upon a challenging upgrade scenario, please consider contributing it using mancoosi-contest. We, for once, would be grateful :-) , and you will be helping out in improving the state of the art in dependency resolutions. Of course, the most interesting upgrade scenarios will also be considered as material for the forthcoming MISC competition, which we've discussed already on this blog.

Technically, dudf-save is still quite hackish and while apt-get support is OK, aptitude support is sub-optimal. In particular it will work properly only for "completely batch" aptitude sessions (or, better, it will not catch all intermediate solver scenarios that aptitude goes through when interactively asking questions to the user).

You can get mancoosi-contenst from mancoosi.debian.net

After the first test runs of the solver competition, at that time still called the 'Mancoosi internal Solver Competition (MiSC)', and the lessons we learned from that experience, the first Mancoosi International Solver Competition (MISC) will take place during the summer of 2010. The results will be announced at the LoCoCo 2010 workshop which will take place on July 10 as part of the FLoC conference.

More about the competion can be learned from the MISC web page. This page is intended for readers who are not yet familiar with the Mancoosi project; those who know the project already might want to jump to the bottom of that page where they will find pointers to the technical details. Participants of the internal MiSC competition beware: there are some changes to the rules, in particular we made the 'trendy' track more difficult by taking into account Recommends, that is non-mandatory package dependencies. Other significant changes include taking into account time. However, the total time used by a participating solver is only taken into account when breaking ties between solvers that otherwise are ranked equal, and furthermore only to determine the winner.

Let the games begin :-)

In the occasion of the Nice 2010 Mancoosi meeting, we organised a first Mancoosi internal Solver Competition (aka MiSC, notice the lowercase i!) to get ready for the Mancoosi International Solver Competition (aka MISC, notice the uppercase i!) which will take place in conjunction with the LoCoCo workshop later this year.

Read the rest

Everybody who is interested in applying logical methods to problems of configuring components should come to Edinburgh on July 10, 2010, and attend the Workshop on Logics for Component Configuration (LoCoCo 2010). This workshop will be part of the 2010 Federated Logic Conference, a large gathering of scientists working on logics in computer science which takes place every 3 to 4 years, and which federates the major international conferences in the field, together with some 50 workshops. The LoCoCo workshop, in fact, is in this context affiliated with the SAT conference, that is the conference about one of the technologies that are employed in the mancoosi project to attack package upgrade problems.

The problems we are currently trying to solve in mancoosi are only a small part of the scope of LoCoCo. The workshop aims more largely and attempts to bring together researchers working on different aspects of software component configuration, as long as they are using logic for that purpose. We are happy that Carsten Sinz from Karlsruhe agreed to give an invited talk at our workshop. The LoCoCo workshop will also be the stage for the first Mancoosi International Solver Competition (MISC), more details about this will be posted soon.

Original research papers accepted for the workshop will - in the spirit of Free and Open Source - be published under an open access licence by EPTCS, and will be freely available online. The deadline for submissions is March, 26. More details can be found on the LoCoCo website, and more specifically in the call for papers.

The 2010 edition of the Free and Open Source Developers European Meeting (FOSDEM 2010) will take place on the weekend of February 6 and 7 in Brussels, Belgium. This is probably the biggest F/OSS event in Europe, with expected more than 4000 participating F/OSS developers from all over the world, and mancoosi will of course be there.

This year will see for the first time the Distribution DevRoom with presentations from ten different FOSS distributions and projects. The mancoosi project is one of the officially participating projects (and the only one who is not a distribution editor itself). There will be three talks from mancoosi in the Distribution DevRoom on sunday afternoon:

• Cross-distro dependency resolution, by Stefano Zacchiroli
• Transactionally Protected Package Management, by Jeff Johnson
• Transactional roll-back, by John Thomson

Besides the participation in the Distribution DevRoom, mancoosi also contributes a lightning talk by Boriss Mejias on BeerNet.

Further details about mancoosi at fosdem 2010 can be found on the Mancoosi wiki. For exact times and rooms please check the FOSDEM schedule as these are still subject to modification.

Since this morning, it is possible to follow on Twitter the svn commits on the Mancoosi forge.

The log of every committ is sent to the user mancoosi (channels #mancoosi #svn)

Thanks to the sysadmin at UCL for setting this up for us.

As Boriss points out, there is also a RSS feeder for the people who don't follow tweets, and for those days when the Fail Whale appears too often.

https://gforge.info.ucl.ac.be/rss/mancoosi.xml https://gforge.info.ucl.ac.be/rss/mancoosi-admin.xml

The Mancoosi team met in Nice on January 7th and 8th 2010.

This was a full immersion project meeting, which mixed plenary tracks presenting advances in the different areas of the project, and a wealth of small team working group sessions devoted to important subtopics related to the Domain Specific Languages, rollback strategies, and package metadata debunking. It was also the occasion to run a first, internal solver competition on real upgrade data coming from our industrial partners: more on this in a following post.

Our project has now a nice leaflet, that you can see here, and even download at high resolution.

It presents the project in a nutshell, higlhighting the virtuous circles we are creating among users, developers, distribution editors and researchers.

Stay tuned for exciting news from Mancoosi over the next months!

Thanks to Giuseppe "denever" Martino, we now have a new, full-fledged Debian weather GNOME applet.

What the applet offers is a GNOME panel applet that interfaces with the Debian weather service, which has been provided since very long time by the EDOS (now Mancoosi) project. In short, the service tells you how many packages are not installable in a given Debian suite, day-by-day. The interest of monitoring that is to avoid attempting package upgrades when a lot of packages are not installable (that would be a bad/stormy day).

We used to have a debian-weather-applet package before, but it was a tad buggy and was not really up to date with respect to GNOME applet current guidelines.

Recently, Giuseppe "denever" Martino has contacted us offering a complete rewrite of the applet, in a few lines of Python code. It works as a dream and it is very nice too, I'm personally using it day-to-day :-)

You can get the applet from the Mancoosi Debian package repository, package name is debian-weather-applet.

If you like it, please join us in thanking Giuseppe for his work!

Due to the relevance of CUDF in the workflow of Mancoosi, we have worked a bit in improving the documentation of CUDF and its visibility on the web. The result is a CUDF homepage which explains in very simple terms what CUDF is about.

Additionally, to help implementors in getting started with CUDF, we have also written a brief CUDF primer that explains the main CUDF concepts with realistic examples.

If you are interested in CUDF and related technologies: have a read!, and let us know how can we improve it.

PS by the way, we also have a nice CUDF logo now :-)

We are please to announce that we have just released version 2.0 of the CUDF specification.

Let me very briefly remind what CUDF is: a document format to describe upgrade scenarios as those we face everyday when we ask a package manager (synaptic, aptitude, urpmi, ..) to add/remove/upgrade a given package. Most peculiarly, CUDF permits to encode such scenarios in a way which is both distribution- and package-manager-independent, offering a lingua franca where upgrade scenarios coming from different distribution can be compared. Finally, CUDF is also a rigorous mathematical model which understand dependencies, features, conflicts, ... so that a clear semantics exists to check whether the response given by a package manager to the user request is correct or not.

According to Mancoosi's goals, CUDF will be mainly used to collect upgrade scenarios submitted by distribution users (a-la popcon) and then to run a solver competition on top of them to find the "best" dependency solver ever (don't worry, you will soon hear from us more on this topic ...). But we believe CUDF will be more than that: for instance it has been shown, by Mandriva and Caixa Magica distributions, to be a good device to collect user problems during upgrades and to spot bugs in package manager. Even more, it can be used to factorize out of package managers the actual dependency solving code, which can then be re-used across different package managers.

Now, going back to CUDF 2.0, the advancement with respect to 1.0 are significant, and we hope/believe to have now reached a rather stable point in the evolution of the format. In particular, in this release we have added support for custom package properties that can now be declared and appropriately typed. Such properties constitute they key ingredient on top of which we will be building a powerful user-preference language that will enable user to encode upgrade policies (e.g. minimize the total installed size among all possible solutions) and fine-grained preferences (e.g. blacklist packages from a given maintainer that I do not trust).

CUDF 2.0 specifications are available from our technical reports page and supersede the former project deliverable about CUDF 1.0. The new specs come with a handy Changelog appendix which highlights all differences with respect to CUDF 1.0.

If you are a developer, you might be interested in testing the pre-release of libCUDF 0.6 (version 0.5.92), which already implements CUDF 2.0. The library can be used from C and OCaml (Java bindings are being worked on). You can download it from our software page.

On November 17th and 18th 2009, the first conference on Free and Open Source Software and Academia (Fossa) took place in Grenoble, and it was a nice occasion for presenting the work done in Mancoosi in a talk in the first day. Grenoble is a very interesting place for high-tech and software: it was possible to discuss interesting issues with people from Sun, Eclipse, and other groups. It appears that our work is definitely interesting for OpenSolaris, and we look forward to collaboration with people from Sun Microsystems on this.

Release of D3.2 - First version of the DSL based on the model developed in WP2. Authored by Davide Di Ruscio, John Thomson, Patrizio Pelliccione, and Alfonso Pierantonio. Reviewed by Jeff Johnson and David Lutterkort.

Read the rest

The Mancoosi team met in Lisbon on October 29th and 30th, at the very nice premises of the Instututo Superior Tecnico. This was a very fruitful project meeting, which allowed to see the state of advancement on both tracks of the project, on the Domain Specific Languages for rolling back unwanted upgrades, and on the advanced solver algorithms for planning upgrades. We could also see Mancoosi mentioned in the Portuguese press (courtesy of Caixa Magica), and the team took a moment to relax after the hard work by discovering the virtues of a certain amount of Portuguese wines.

Last week, I presented a MANCOOSI article at the ESEM 2009 conference in Lake Buena Vista, Florida (better known as Disneyworld).

It's a nice experience going from a Paris autumn to a Florida one (net difference: something like 20 degrees Celsius); and taking the train from Miami to Orlando definitely raised my opinion about American railways (there might not be as many of them as in Europe, but the ones they do have are absolutely wonderful). And when the conference venue looks like this:

Not bad at all.

The conference itself was interesting as well; ESEM stands for Empirical Software Engineering and Measurement, a field that I didn't really know much about, but there were some good presentations that expanded my horizons, so to speak.

The paper I presented was written by Pietro Abate, Roberto Di Cosmo, Stefano Zacchiroli and myself: Strong Dependencies between Software Components, about our ideas for using strong dependencies to measure several properties of component-based systems. The paper does not seem to be online yet, but a technical report version can be found on HAL.

An overview of the Mancoosi project will be given in the framework of the Open World Forum. See you there at the Mezzanine on Friday October 2nd 2009, at 1pm.

Please join us in welcoming the new Mancoosi logo (reported above) and the new layout of our beloved website.

Cheers!

A modified urpmi version has been released for Mandriva 2009.1

It is available here : http://www.mancoosi.org/software/

This version of urpmi permits the installation of urpmi-dudf (which is a plugin for urpmi). This plugin allows urpmi to generate and upload dudf error files (to Mancoosi’s servers). Others packages are the same as the Mandriva 2009.1 original ones.

To install urpmi-dudf, you juste have to upgrade your installed packages with these versions, and install urpmi-dudf.

This version is a backport of the one included into Mandriva 2010.0, out in october 2009.

Once urpmi-dudf installed, if an error occurs during an installation, urpmi will ask the user if he wants to generate a dudf file and upload it to our servers.

The user can force this generation/upload with a new urpmi parameter : "--force-dudf" (even if no error occurs).

More about Mancoosi : http://www.mancoosi.org

More about Mandriva : http://www.mandriva.com

I've attended DebConf9 (well, I'm still attending, but it will be over in a few hours), the annual Debian developers conference. There, I've delivered a talk where I've presented a status update about the Mancoosi project. The talk has also addressed some new usages we are doing of EDOS tools (currently maintained by Mancoosi project members, hello Jaap) for Quality Assurance purposes.

As always happen at DebConf, people find out hacks interesting new ways of using "old" tools. Last year, starting from an idea from Ralf, we developed edos-builddebcheck; it consists of a wrapper around edos-buildcheck which is able to detect packages whose build dependencies are not satisfiable in a given distribution. That, per se, is already very useful because truly free software is achieved only when its users are able to rebuild it starting from sources. ... but edos-builddebcheck can be made even more useful, by integrating it into build daemons (those pieces of every distribution infrastructure which automatically re-compile binary packages on all architectures supported by the distribution). And that is precisely the idea put forward, and implemented, by Joachim Breitner here at DebConf.

Starting from today the Debian build daemon scheduler, called wanna-build, will verify build dependencies using edos-builddebcheck before attempting to build a package. If they are not satisfiable, the package will be put into a new waiting state instead of trying build it. Very cool!, ... and a huge time saving for buildd maintainers that thus far needed to recognize build failures of this kind among tons of other, more serious, build failures.

A Mancoosi related paper has been presented at the 21st International Joint Conference on Artificial Intelligence (IJCAI'09) which took place on July 11-17th in Pasadena, CA. In our paper we propose a new method for dealing with the optimal integration of new versions of software packages, dubbed Multilevel Boolean Optimization. More details can be obtained from reading the paper.

On September 7-11 2009, in Oxford UK, will take place the XtreemOS Summer School. XtreemOS is a Linux-based operating systems that includes Grid functionalities, developed by an international consortium with which we share Edge-IT as a partner.

On June 4th and 5th 2009, the Mancoosi meeting originally planned in L'Aquila was organized in Rome in the premises of the Sapienza University of Rome, thanks to the efforts of our colleagues in L'Aquila despite the difficult circumstances. This was the opportunity of another fruitful project meeting (see agenda) with presentations of work carried on lately, close collaboration and discussions on the future of the project.

To be continued!

Support ideas for L'Aquila