[Secure-testing-commits] r14624 - doc

[Florian Weimer]

Author: fw
Date: 2010-05-07 18:51:15 +0000 (Fri, 07 May 2010)
New Revision: 14624

Added:
   doc/python-format.txt
Log:
Add documentation for the upcoming external data formats


Added: doc/python-format.txt
===================================================================
--- doc/python-format.txt	                        (rev 0)
+++ doc/python-format.txt	2010-05-07 18:51:15 UTC (rev 14624)
@@ -0,0 +1,116 @@
+NOTE: THIS DOES NOT DESCRIBE THE CURRENT IMPLEMENTATION
+
+# Layout of major internal data structures
+
+Most data structures use named tuples, as provided by
+xcollections.namedtuples (they are not available in Python 2.5, but
+the implementation from Python 2.6 works on Python 2.5, too).
+
+Due to the way unpickling works, you need to import the "parsers"
+package.
+
+The data structures described here are supposed to be fairly stable,
+except for the addition of additional attributes and changes in the
+internal order of named tuples (so you really should not rely on
+that).
+
+# Individual bug information
+
+The data/*/list files are parsed as lists of bugs.  A line which does
+not start with whitespace is called a "header", and the following
+intended lines are called "annotations".
+
+The top-level named tuple contains two elements:
+
+* list: the list of bug objects (see below)
+
+* messages: the list of messages from the parser (see below)
+
+All lists are sorted by file position of the contained objects.
+
+## bug objects
+
+* bug.file: path to the file containing this bug
+
+* bug.header: header object (see below)
+
+* bug.annotations: list of all annotations of this bug (see below)
+
+## header objects
+
+* header.line: line number
+
+* header.name: bug name (auto-generated for temporary issues)
+
+* header.description: string, can be empty or None
+
+## message objects
+
+* msg.file: file name
+
+* msg.line: line number
+
+* msg.level: "error" or "warning"
+
+* msg.contents: free-text message
+
+## Errors 
+
+## annotation objects
+
+All annotation objects have these fields:
+
+* ann.line: the line number
+
+* ann.type: code value to determine the  structure
+
+Additional fields are described below, depending on the ann.type
+value.
+
+### types "NOT-FOR-US", "NOTE", "TODO"
+
+* ann.description: user-supplied string
+
+### types "RESERVED", "REJECTED"
+
+These act just as flags; no additional data is present.
+
+### type "xref"
+
+* ann.bugs: list of bugs being referenced
+
+### type "package"
+
+* ann.release: applicable release, or None for unstable
+
+* ann.package: name of the package
+
+* ann.kind: one of "fixed" (version was supplied), "unfixed", "removed",
+  "itp", "no-dsa", "not-affected", "undetermined"
+
+* ann.version: fixed version number, or "None" for unfixed/not applicable
+
+* ann.urgency: one of None, undetermined, low, medium, high
+
+* ann.debian_bugs: set of numbers of Debian bugs
+
+* ann.description: free-text information, or None if not applicable
+
+# Derived vulnerability information
+
+These are contained in a list of info objects:
+
+* info.bug: name of the bug (potentially auto-generated)
+
+* info.package: name of the package
+
+* info.fixed: fixed version in unstable (a string), or None (no fix
+  available) or True (all versions fixed)
+
+* info.fixed_other: a tuple, containing other fixed versions (which
+  are less than the unfixed unstable version, but nevertheless known
+  not to be vulnerable)
+
+In itself, this data is not very illuminating, but comparision with
+other information sources can be used to detect vulnerable installed
+packages, generate bug and distribution overview pages etc.