Autobuilder network

The autobuilder network is a Debian development that manages package compilations for all the architectures Debian currently supports. This network is made up of several machines that use a specific software package called buildd to pick up packages from the Debian archive and rebuild them for the target architecture.

Why is the autobuilder network needed?

The autobuilder network provides a secure package building functionality for all supported architectures. The Debian distribution supports quite a few architectures, and the package maintainers often do not have access to all machines with needed architectures. On the other hand, Debian now requires ordinary binary packages to be generated from the source code in a controlled build environment (via the source-only upload requirement) to avoid the introduction of maliciously-crafted binary packages by human developers. The autobuilder network takes the package source code and build binary packages automatically, once for each supported hardware architecture. Failures are tracked in the autobuilder database.

The autobuilder network also eases the work of Debian Ports maintainers. When Debian/m68k (the first non-Intel port) started, developers for it had to watch out for new versions of packages and recompile them if they wanted to stay up-to-date with the Intel distribution. All this was done manually: developers watched the upload mailing list for new packages and took some of them for building. Coordination that no package is built twice by different people was done by announcing on a mailing list. It's obvious that this procedure is error-prone and time-consuming. This was, however, the usual way for keeping non-i386 distributions current for a long time.

The build daemon system automates most of this process. It consists of a set of scripts (written in Perl and Python) that have evolved over time to help porters with various tasks. They have finally developed into a system that is able to keep Debian distributions up-to-date nearly automatically. The security updates are built on the same set of machines to ensure their timely availability.

How does buildd work?

Buildd is the name usually given to the software used by the autobuilder network, but it's really made of different parts:

wanna-build
a tool that helps coordinate package (re)building through a database that keeps a list of packages and their status. There is one central database per architecture that stores package states, versions, and some other information. It's fed with Sources and Packages files retrieved from the various package archives Debian has (e.g. ftp-master and security-master).
buildd
a daemon that periodically checks the database maintained by wanna-build and calls sbuild to build the packages. After the build log was acknowledged by the buildd administrator it uploads the package to the appropriate archive.
sbuild
is responsible for the actual compilation of packages in isolated chroots. It ensures that all needed source dependencies are installed into the chroot before building and then calls standard Debian tools to start the build process. Build logs are submitted to the build log database.

All these parts operate together in order to make the builder network work.

What does a Debian developer need to do?

Actually, an average Debian developer does not need to explicitly use the buildd network. Whenever he uploads a package to the archive (binary compiled to a given architecture) it will be added to the database for all architectures (in state Needs-Build). Build machines will query the build database for packages in this state, and will routinely take packages from that list. The list is prioritized by previous compilation state (either out-of-date or uncompiled), priority, section and package name. Furthermore, to prevent some packages from starving at the end of the queue, the priorities are dynamically adjusted with increasing waiting time in the queue.

If the build succeeds in all architectures, the maintainer will not need to do anything. All those binary packages will be uploaded to the corresponding archive. If the build does not succeed the package will enter special states (Build-Attempted for build failures that were not reviewed, Failed for reviewed and reported bugs in the package or Dep-Wait, if they depend on specific build dependencies which are not available). The autobuilder administrators will review packages which fail to build and will report back to the maintainer, usually, opening up a bug in the Bug Tracking System.

Sometimes a package takes a long time to build for a given architecture and that holds the package from entering testing. If a package holds up a transition build priorities are usually adjusted upon request by the Release Team. Other requests will not be accepted as the increased waiting time in the queue will lead to a higher build priority automatically.

You can check that status of the different buildds attempt of the packages that belong to any given maintainer by checking the buildd logs. These logs are also linked from the Packages' Maintainer Overview.

For more information on the different states a package can be please read wanna-build-states.

Where can I find additional information?

Of course, both the documentation and the source code available for these different tools are the best way to find out how the buildd network works. Additionally, the Porting and being ported section of the Debian Developers Reference provides complementary information on how does it work and it also provides some information on package builders and porting tools which are involved in the process of both setting up and maintaining the buildd network.

There are some statistics available for the autobuilder network at the buildd stats page.

How can I setup my own auto-builder node?

There are several reasons why a developer (or user) might want to setup and run an autobuilder:

You can read more information on how you can setup an autobuilder.

Contacting the buildd admins

The primary contact address for the wanna-build maintainers is [email protected] (Mailing list)

The admins responsible for buildd's for a particular arch can be reached at [email protected], for example [email protected].

This introduction to the autobuilder network was written with bits and pieces provided by Roman Hodek, Christian T. Steigies, Wouter Verhelst, Andreas Barth, Francesco Paolo Lovergine, Javier Fernández-Sanguino and Philipp Kern.