Security Metrics

Discussion of what should be given attention for generating statistics and metrics for security in Ubuntu.

NOTE: Adjusted to jdstrand as the assignee since Kees left and the reports are not correct.

Agenda:
 * quick review of existing data and outputs
  * https://code.edge.launchpad.net/~ubuntu-security/ubuntu-cve-tracker/master
  * http://people.canonical.com/~ubuntu-security/graphs/
   * supported packages
   * fixed CVEs, USNs
   * outstanding CVEs
   * exposure timeframes
 * comparison to RedHat metrics
  * https://www.redhat.com/security/data/metrics/
 * OVAL output format
  * http://oval.mitre.org/oval/about/faqs.html

Work items:
[jdstrand] define set of desired data visualizations: DONE
[jdstrand] delegate data visualization tasks: DONE
[jdstrand] create web page putting metrics into context: DONE
[kees] create initial metric web page with basic formatting: DONE
[kees] export raw data as well as graphs: DONE
[kees] CVEs * USNs * source packages per month (what work we did, where the 'USN' factors in the releases), all releases, all time: DONE
[kees] CVEs * USNs * source packages per month (what work we did in last year, where the 'USN' factors in the releases), all releases, rolling 12 months: DONE
[kees] CVEs * USNs * source packages per month (what work we did, where the 'USN' factors in the releases), per release, all time: DONE
[kees] CVEs * source packages per month (what are the pending work trends), all releases, rolling 12 months: DONE
[kees] CVEs * source packages per month (what are the pending work trends), per release, rolling 12 months: DONE
[kees] regressions published per month, all releases, all time (what are the regression trends): DONE
[kees] regressions published per month, per release, all time (what are the regression trends, for (esp older releases): DONE
[kees] CVEs fixed per month, all releases, all time: DONE
[kees] CVEs fixed per month, all releases, rolling 12 months: DONE
[kees] CVEs fixed per month, per release, all time: DONE
[kees] USNs published per month, all releases, all time: DONE
[kees] USNs published per month, all releases, rolling 12 months: DONE
[kees] USNs published per month, per release, all time: DONE
[kees] source packages fixed per month, all releases, all time: DONE
[kees] source packages fixed per month, per release, all time: DONE
[kees] CVEs to be fixed per month, all releases, rolling 12 months: POSTPONED
[kees] CVEs to be fixed per month, per release, rolling 12 months: POSTPONED
[kees] script to generate raw data for number of incoming CVEs: DONE
[kees] graph number of incoming CVEs: POSTPONED

These possible future work items have been moved to the Roadmap:
[kees] produce response time summaries, similar to existing RH metrics
[kees] produce "first 6 months" exposure graphs instead of existing "on-going exposure"

Natty work items:
[kees] define and implement a programmatic u-c-t tag for "proactivity helped us": DONE
[mdeslaur] select security whitepaper topic: DONE
[mdeslaur] write security whitepaper: POSTPONED