[Reproducible-builds] [presentations] 01/01: Add first version of the DebConf14 presentation
Jérémy Bobbio
lunar at moszumanska.debian.org
Mon Aug 25 15:19:32 UTC 2014
This is an automated email from the git hooks/post-receive script.
lunar pushed a commit to branch master
in repository presentations.
commit bbc1a2007f6d759dbcb4fd931fab93d1335b7d19
Author: Jérémy Bobbio <lunar at debian.org>
Date: Mon Aug 25 15:19:16 2014 +0000
Add first version of the DebConf14 presentation
---
2014-08-26-DebConf14/.gitignore | 1 +
.../2014-08-26-DebConf14-header.tex | 29 ++
2014-08-26-DebConf14/2014-08-26-DebConf14.mdwn | 422 +++++++++++++++++++++
2014-08-26-DebConf14/Makefile | 30 ++
2014-08-26-DebConf14/images/openlogo-nd.pdf | Bin 0 -> 3168 bytes
2014-08-26-DebConf14/images/swirl-lightest.pdf | Bin 0 -> 3540 bytes
2014-08-26-DebConf14/images/tor-blog.png | Bin 0 -> 135486 bytes
7 files changed, 482 insertions(+)
diff --git a/2014-08-26-DebConf14/.gitignore b/2014-08-26-DebConf14/.gitignore
new file mode 100644
index 0000000..329d5fd
--- /dev/null
+++ b/2014-08-26-DebConf14/.gitignore
@@ -0,0 +1 @@
+2014-08-26-DebConf14.pdf
diff --git a/2014-08-26-DebConf14/2014-08-26-DebConf14-header.tex b/2014-08-26-DebConf14/2014-08-26-DebConf14-header.tex
new file mode 100644
index 0000000..f95eabb
--- /dev/null
+++ b/2014-08-26-DebConf14/2014-08-26-DebConf14-header.tex
@@ -0,0 +1,29 @@
+% Thanks Richard Darst on how to get a nice Beamer theme.
+% See http://rkd.zgib.net/wiki/DebianBeamerThemes
+
+\usebackgroundtemplate{\includegraphics[width=\paperwidth]{images/swirl-lightest.pdf}}
+\logo{\includegraphics[viewport=274 335 360 440,width=1cm]{images/openlogo-nd.pdf}}
+
+\definecolor{debianred}{rgb}{.780,.000,.211} % 199,0,54
+\definecolor{debianblue}{rgb}{0,.208,.780} % 0,53,199
+\definecolor{debianlightbackgroundblue}{rgb}{.941,.941,.957} % 240,240,244
+\definecolor{debianbackgroundblue}{rgb}{.776,.784,.878} % 198,200,224
+
+\usecolortheme[named=debianbackgroundblue]{structure}
+\setbeamercolor{normal text}{fg=debianred}
+\setbeamercolor{titlelike}{fg=debianblue}
+\setbeamercolor{sidebar}{fg=debianred,bg=debianbackgroundblue}
+
+\setbeamercolor{palette sidebar primary}{fg=debianred}
+\setbeamercolor{palette sidebar secondary}{fg=debianred}
+\setbeamercolor{palette sidebar tertiary}{fg=debianred}
+\setbeamercolor{palette sidebar quaternary}{fg=debianred}
+
+\setbeamercolor{section in toc}{fg=debianred}
+\setbeamercolor{subsection in toc}{parent=debianred}
+
+\setbeamercolor{item}{fg=debianred}
+
+\setbeamercolor{block title}{fg=debianblue}
+
+\usetheme{Boadilla}
diff --git a/2014-08-26-DebConf14/2014-08-26-DebConf14.mdwn b/2014-08-26-DebConf14/2014-08-26-DebConf14.mdwn
new file mode 100644
index 0000000..d181fc8
--- /dev/null
+++ b/2014-08-26-DebConf14/2014-08-26-DebConf14.mdwn
@@ -0,0 +1,422 @@
+% Reproducible Builds for Debian, a year later
+% Lunar \<lunar at debian.org\>
+% 2014-08-26 DebConf14
+
+What are reproducible builds?
+-----------------------------
+
+\begin{center}
+\Large
+“reproducible” builds
+enable anyone to reproduce the exact same
+binary packages from a given source
+\end{center}
+
+Why?
+----
+
+ * Prevent targeted attacks
+ * Debugging: ensure known source; create missing debug symbols
+ * Ensure packages can be built from source
+ * Help building `Multi-Arch: same` packages (Debian specific)
+ * Similar `.deb`: deduplication, small deltas
+ * Different build profiles, same common packages
+
+How did this start?
+-------------------
+
+\begin{center}
+\includegraphics[width=0.8\textwidth]{images/tor-blog}
+\end{center}
+
+Nothing new
+-----------
+
+\texttt{\footnotesize%
+From: Martin Uecker <muecker at gmx.de> \\
+Cc: debian-devel at lists.debian.org \\
+Date: Sun, 23 Sep {\large 2007} 23:32:59 +0200 \\
+}
+
+\textit{%
+I think it would be really cool if the Debian policy required
+that packages could be rebuild bit-identical from source.
+At the moment, it is impossible to independly verify the
+integricity of binary packages.
+}
+
+\begin{flushright}
+\tiny
+\url{https://lists.debian.org/debian-devel/2007/09/msg00746.html}
+\end{flushright}
+
+Although, reactions were not enthusiastic
+-----------------------------------------
+
+\texttt{\footnotesize%
+From: Neil Williams <codehelp at debian.org> \\
+To: debian-devel at lists.debian.org \\
+Date: Mon, 24 Sep 2007 07:22:30 +0100 \\
+}
+
+\textit{%
+> Then third parties can recreate the binaries \\
+> and publish recreated hashes.
+}
+
+\textit{%
+Why? I see no benefit.
+}
+
+\begin{flushright}
+\tiny
+\url{https://lists.debian.org/debian-devel/2007/09/msg00747.html}
+\end{flushright}
+
+Although, reactions were not enthusiastic
+-----------------------------------------
+
+\texttt{\footnotesize%
+From: Manoj Srivastava <srivasta at debian.org> \\
+To: debian-devel at lists.debian.org \\
+Date: Sun, 23 Sep 2007 23:25:16 -0500 \\
+}
+
+\textit{%
+I, for one, think this technically infeasible, but hey, I'll be
+happy to be proved wrong.
+}
+
+\begin{flushright}
+\tiny
+\url{https://lists.debian.org/debian-devel/2007/09/msg00760.html}
+\end{flushright}
+
+BoF during DebConf13
+--------------------
+
+ * Planned at the last minute
+ * 30 attendees
+ * Kicked off
+ `wiki.debian.org/ReproducibleBuilds`
+
+How?
+----
+
+ * Record the build environment
+ * Reproduce the build environment
+ * Eliminate unneeded variations
+
+Record the build environment
+----------------------------
+
+Record which versions of the build dependencies (and their dependencies) are
+installed.
+
+Reproduce the build environment
+-------------------------------
+
+`snapshot.debian.org`
+
+Source of variations
+--------------------
+
+ * Timestamps
+ * Build paths
+ * File order
+ * Locale
+ * …
+
+Timestamps
+----------
+
+`gzip` stores a timestamp.
+
+\tiny
+
+ $ file README.txt.gz
+ README.txt.gz: gzip compressed data, was "README.txt", from Unix,
+ last modified: Mon Mar 5 00:05:49 2012, max compression
+
+Timestamps
+----------
+
+`ar`, `tar`, `zip`, `jar`… store timestamps.
+
+\tiny
+
+ $ tar ztvf copyright-format.xml.tar.gz
+ -rw-r--r-- pbuilder/pbuilder 473 2012-03-05 00:02 Makefile
+ -rw-r--r-- pbuilder/pbuilder 56918 2012-03-05 00:05 copyright-format-1.0.html
+ -rw-r--r-- pbuilder/pbuilder 37218 2012-03-05 00:05 copyright-format-1.0.txt
+ -rw-r--r-- pbuilder/pbuilder 10007 2012-03-05 00:05 copyright-format-1.0.txt.gz
+ -rw-r--r-- pbuilder/pbuilder 53917 2012-03-05 00:02 copyright-format-1.0.xml
+ -rw-r--r-- pbuilder/pbuilder 808 2012-03-05 00:02 html.dsl
+ -rw-r--r-- pbuilder/pbuilder 97 2012-03-05 00:05 version.xml
+
+Timestamps
+----------
+
+`javadoc` writes timestamps:
+
+\tiny
+
+ $ head -n 5 /usr/share/doc/libjaxe-java-doc/api/serialized-form.html
+ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+ <!-- NewPage -->
+ <html lang="en">
+ <head>
+ <!-- Generated by javadoc (version 1.6.0_27) on Sat Jul 13 17:27:51 UTC 2013 -->
+
+Build paths
+-----------
+
+Build path is embedded in debug symbols:
+
+\tiny
+
+ $ readelf -w /usr/lib/debug/usr/bin/pidgin | grep '/tmp/build' | head -n 4
+ <11> DW_AT_name : /tmp/buildd/pidgin-2.10.6/./pidgin/pidginstock.c
+ <15> DW_AT_comp_dir : /tmp/buildd/pidgin-2.10.6/build/pidgin
+ <402d> DW_AT_name : /tmp/buildd/pidgin-2.10.6/./pidgin/gtkaccount.c
+ <4031> DW_AT_comp_dir : /tmp/buildd/pidgin-2.10.6/build/pidgin
+
+File order
+----------
+
+`readdir()` returns file in the order of the file system.
+
+Locale
+------
+
+Behaviour can change depending on configured locale:
+
+\tiny
+
+ $ printf 'a\nà\nb\n' | LC_ALL=C.UTF-8 sort
+ a
+ b
+ à
+
+ $ printf 'a\nà\nb\n' | LC_ALL=fr_FR.UTF-8 sort
+ a
+ à
+ b
+
+Misc.
+-----
+
+ * Hostname
+ * Uname output
+ * Username
+
+Cheat
+-----
+
+ * Use a VM: same kernel, same user, same build path
+ * `libfaketime`
+
+The hard path
+-------------
+
+ * Configure the toolchain:
+ binutils `--enable-deterministic-archives`
+ * Add missing options:
+ `javadoc --no-timestamps`
+ * Patch build systems:
+ `gzip -n`
+
+Experiment
+----------
+
+ * Build and rebuild of many source packages
+ * Using EC2 VM instances from Amazon Web Services
+ * Many thanks David Suárez!
+
+Experiment
+----------
+
+ * Build packages twice
+ * Setup clean chroot, unpack source code, install build-deps, build
+ * And again…
+ * Pass the timestamp of the first build to dpkg through environment variable
+
+Experiment
+----------
+
+Variations in this context:
+
+ * Time
+ * Build path
+
+No changes in hostname, username, uname, file order, locale…
+
+Experiment
+----------
+
+Modified packages for the January 2014 experiment:
+
+ * dpkg: use single timestamp in the archives
+ * dpkg: re-use timestamp from environment if given
+ * dpkg: stable file order in the archives
+ * debhelper: dh_strip calls `debugedit`
+ * dpkg: pass `-fno-merge-debug-strings` through `dpkg-buildflags`
+ * binutils: built with `--enable-deterministic-archives`
+
+Experiment
+----------
+
+ * Upon 5151 source packages
+ * 3196 produced identical binary packages
+
+Experiment
+----------
+
+\begin{center}
+\Huge 62\%
+\end{center}
+
+Waow.
+
+Already reproducible
+--------------------
+
+ source name popcon insts
+ --------------------- ------------
+ findutils 164641
+ wget 164512
+ klibc 163312
+ busybox 161494
+ installation-report 157494
+ laptop-detect 157352
+ python-support 155075
+ netkit-ftp 145548
+
+Failures in the remaining packages
+----------------------------------
+
+ 1017 build-id-mismatch
+ 295 unknown
+ 108 jar-file
+ 106 haskell-prof
+ 103 haskell-dev
+ 101 php-registry
+ 101 html-mismatch
+ 63 same-depends-different-order
+ 62 r-rds
+ 52 gzip-timestamp
+ 46 kde-doc-index
+
+Failures in the remaining packages
+----------------------------------
+
+ 45 mono
+ 35 specific
+ 33 docbook-to-man-timestamp
+ 23 do-not-use-dpkg-buildflags
+ 21 debugedit-not-run-or-failed
+ 16 puredata
+ 13 perl-manpage
+ 11 rdoc-timestamp
+ 10 zip-file
+ 8 ocaml-md5sums
+ 7 fonts
+ 7 erlang
+
+Further research
+----------------
+
+Still no good solution for the build ID issue.
+
+Further research
+----------------
+
+ * How about deciding on a canonical build path?
+
+ `/usr/src/debian/hello-2.9`
+
+ * `proot` can fake the current directory like `fakeroot`
+ fakes uid.
+ * `gdb` would be able to easily look up source code.
+
+Thanks to Stéphane Glondu for working the idea.
+
+Further research
+----------------
+
+Should `dpkg-buildpackage` export `GZIP=-n`?
+
+More experiments
+----------------
+
+ * New `dpkg-buildpackage` patch that will call `proot`.
+ * Unfortunately, David Suárez was too busy this time to
+ help with archive-wide experimentation.
+
+Other distributions
+-------------------
+
+ * Fedora
+ <http://securityblog.redhat.com/2013/09/18/reproducible-builds-for-fedora/>
+ * OpenSUSE build-compare
+ <https://build.opensuse.org/package/show/openSUSE:Factory/build-compare>
+ * NixOS
+ <http://lists.science.uu.nl/pipermail/nix-dev/2013-June/011357.html>
+
+Want to help?
+-------------
+
+Triage:
+
+ * Let's make a new archive-wide rebuild and sort the result.
+
+Want to help?
+-------------
+
+Specify:
+
+ * Think about the best way to record the environment.
+
+Want to help?
+-------------
+
+Code:
+
+ * Add “no timestamps” option to `jar`, `javadoc`, `epydoc`…
+ * Write a script to rebuild a package from a .changes file
+ and a recorded environment.
+
+Want to help?
+-------------
+
+Project management:
+
+ * Coordinate the baby steps needed to move this forward.
+
+Want to help?
+-------------
+
+Stay in touch:
+
+ * Subscribe to the `ReproducibleBuilds` wiki page.
+ * Subscribe to the `reproducible-builds at l.a.d.o` mailing list.
+
+BoF
+---
+
+\begin{center}
+BoF to discuss technical solutions at 19:00 in room 329
+\end{center}
+
+Questions? Comments?
+--------------------
+
+\begin{center}
+\Huge
+?
+\end{center}
+
+\begin{center}
+\vspace{3em}
+\url{wiki.debian.org/ReproducibleBuilds}
+\end{center}
diff --git a/2014-08-26-DebConf14/Makefile b/2014-08-26-DebConf14/Makefile
new file mode 100644
index 0000000..bbe7add
--- /dev/null
+++ b/2014-08-26-DebConf14/Makefile
@@ -0,0 +1,30 @@
+.PHONY: all source
+
+PRESENTATION = 2014-08-26-DebConf14
+
+all: $(PRESENTATION).pdf
+
+source: $(PRESENTATION)-src.tar.gz
+
+IMGS = \
+ images/swirl-lightest.pdf \
+ images/openlogo-nd.pdf \
+ $(shell echo $$(sed -n -e 's/^[^%]*\\includegraphics\([^{]*\)\?{\([^}]*\)}.*$$/\2.*/p' $(PRESENTATION).mdwn | sed -e 's/\.svg$$/\.pdf$$/' | sort -u))
+
+$(PRESENTATION).pdf: $(PRESENTATION).mdwn $(PRESENTATION)-header.tex $(IMGS)
+ pandoc -t beamer \
+ --include-in-header=$(PRESENTATION)-header.tex \
+ --variable=fontsize=14pt \
+ --latex-engine=lualatex -o $@ $<
+
+%.pdf: %.svg
+ inkscape --export-pdf=$@ --export-dpi=600 $<
+
+SRCS = \
+ $(shell find . -maxdepth 1 '(' -name '$(PRESENTATION).GNUMakefile' -o -name 'Makefile' ')' -printf '%P\n') \
+ $(PRESENTATION).mdwn \
+ $(PRESENTATION)-header.tex \
+ $(IMGS)
+
+$(PRESENTATION)-src.tar.gz: $(SRCS)
+ tar -zcvf $@ --transform 's,$(PRESENTATION)\.GNUMakefile,Makefile,;s,^,$(PRESENTATION)/,' $(SRCS)
diff --git a/2014-08-26-DebConf14/images/openlogo-nd.pdf b/2014-08-26-DebConf14/images/openlogo-nd.pdf
new file mode 100644
index 0000000..fed3d93
Binary files /dev/null and b/2014-08-26-DebConf14/images/openlogo-nd.pdf differ
diff --git a/2014-08-26-DebConf14/images/swirl-lightest.pdf b/2014-08-26-DebConf14/images/swirl-lightest.pdf
new file mode 100644
index 0000000..1c8ffd2
Binary files /dev/null and b/2014-08-26-DebConf14/images/swirl-lightest.pdf differ
diff --git a/2014-08-26-DebConf14/images/tor-blog.png b/2014-08-26-DebConf14/images/tor-blog.png
new file mode 100644
index 0000000..9bac1cb
Binary files /dev/null and b/2014-08-26-DebConf14/images/tor-blog.png differ
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/reproducible/presentations.git
More information about the Reproducible-builds
mailing list