I’m looking to be able to effectively audit builds related to Ubuntu that are built using Launchpad. Launchpad is the web application and server software the Ubuntu uses for:

The use case is an Ubuntu user who would like to confirm the built packages that Launchpad produces by replicating the build locally and comparing the output executables.

Personal background

I’m relatively new to the build and packaging game and have a cursory knowledge of the tools and processes, but before diving into Launchpad I want to write down what I do know about it, especially the Debian and Ubuntu processes and tools, so I have something to refer back to.

Prior to this year my experience with building was twofold, one was writing C code and using make, the other was building gcc, perl, emacs, and other open source software before the age of binary distributions.

I wrote batch-oriented scientific code in the early 90s while doing research for my Ph.D. This was at first using the IBM xlC1 compiler on AIX2. AIX used the XCOFF3 executable file format. The structure of an executable file is important to understand when doing practical analysis of reproducible builds because if you haven’t got to the point of producing bitwise identicle outputs you can still distinguish between trivial differences and significant differences within the sections of the executable file.

Later I wrote batch-oriented C code that used Posix threads4 to allow for parallel execution in a single memory space on a COMA5 machine, the KSR-16, in contrast to similar early experimental NUMA7 machines. Other users of the machine were trying to research algorithms that would work with such an architecture while I was just trying to calculate solutions to physics problems using numerical methods and trying to optimize what I could.

In parallel I did have to learn about open source building by learning to use the early versions of what is now GNU Autotools8. Just understanding the concept of the following sequence:

  1. n=1, compile gccn=gcc1 using xlC
  2. Compile gcc(n+1) using gccn
  3. If gcc(n+1) == gccn, install gccn, else n++ and loop back to step 2.

is a fundamental example of the need for reproducable builds.

In any event my experience was rather limited to using make and for builds and learning the XCOFF format.

Note about debbuild

My first Debian-style build was using the basics, debbuild. I’m going to just bullet-point a bunch of stuff I learned without trying to go deep into this since I want to, like I’ve said, not be precious about being complete in the short term in favour of getting stuff out of my head.

What I recall:

Note about schroot

I’ve already done a bit of that investigative work by doing builds of a couple Ubuntu packages (qemu-* and libvirt-*) using the schroot tool. But I still don’t have a good understanding of the layering of tools from the low-level Makefiles and compilers to the debian/* subdirectory and associated build tools (debbuild, …?) up to schroot, sbuild, and I mean, I have a hand-wavy understanding but not a detailed understanding, and that makes me feel completely unmoored.

Note about sbuild

I have yet to really investigate sbuild other than seeing it in part of the Launchpad installation and setup and seeing that it does relate to and possibly use schroot.


I’ve installed Launchpad using these resources but have not got far into it. I was going to review that here, today, but I got bogged down in the above. I will continue tomorrow, hopefully.

Article series

  1. xlC is IBM’s proprietary C compiler ↩︎

  2. AIX is IBM’s proprietary UNIX operating system ↩︎

  3. XCOFF is IBM’s extension of the SysV object file format COFF ↩︎

  4. pthreads allow for parallel execution with, for example, mutexes ↩︎

  5. Cache-only memory architecture ↩︎

  6. Kendall Square Research was a supercomputer company ↩︎

  7. Non-uniform memory access ↩︎

  8. GNU Autotools ↩︎

  9. [Quilt (software)]( ↩︎