Skip to content
This repository has been archived by the owner on Jul 6, 2024. It is now read-only.

Design

Jump to bottom
Michael Scovetta edited this page Dec 28, 2020 · 5 revisions

Security Metrics Design

High-Level Architecture

The project will consist of a set of data collectors, and API endpoint, a database for storing the data collected, and an application for displaying that data in interesting way.

Data Collectors

For the initial (MVP) release, the following collectors will be implemented:

Source code for these collectors is available on TBD, but will be run from a trusted virtual machine.

OpenSSF Scorecard

We'll use public data collected by the OpenSSF Scorecard project, refreshed periodically (daily? weekly?)

Security Reviews

We'll leverage the Security Reviews project, cloned and refreshed nightly.

API Endpoint

The API endpoint currently serves one purpose -- to abstract away database details from the collectors. One route is available, called AddMetric. It must be called (via POST) with JSON content that conforms to the following format:

{
  "package_url": "pkg:npm/left-pad",
  "key": "openssf.metadata.homepage",
  "operation": "replace",
  "values": [
    {
      "value": "https://example.com",
      "timestamp": "2020-12-28 07:06:06.745358"
      "properties": {
        "source": "libraries.io"
      }
    }
  ]
}

The operation field can either be replace or insert. If it is replace, then existing rows (for the given package_url and key) are removed prior to the new set of rows being added.

The values array will often contain only a single structure, but for cases like openssf.version, it would contain one structure for each version published.

This API call will return JSON that will convey the result of the operation.

Authentication and Authorization

For the MVP, the endpoint will require a secret key to be passed in along with the request. This will be improved post-MVP.

Database

The database is a "plain" PostgreSQL database with a single table called metrics:

CREATE TABLE public.metrics
(
    id bigserial,
    package_url text NOT NULL,
    key text NOT NULL,
    value text,
    "timestamp" timestamp without time zone NOT NULL,
    properties jsonb,
    CONSTRAINT metrics_pkey PRIMARY KEY (id)
)

Column: package_url

The package_url column contains a valid Package URL, such as pkg:npm/left-pad or pkg:github/madler/zlib. By convention, these Package URLs will be without a version.

Column: key

The key column corresponds to the type of metric in this row. Currently used keys are:

  • openssf.metadata.homepage - The home page of a project
  • openssf.metadata.description - The description of a project
  • openssf.version - A released version of a component (from libraries.io)
  • openssf.version.github.release - A GitHub release
  • openssf.version.github.tag - A GitHub tag
  • openssf.scorecard.raw.* - Metrics coming from the OpenSSF Scorecard. The last part is taken directly from the Scorecard output (CheckName, converted to lowercase).
    • openssf.scorecard.raw.active
    • openssf.scorecard.raw.ci-tests
    • openssf.scorecard.raw.cii-best-practices
    • openssf.scorecard.raw.code-review
    • openssf.scorecard.raw.contributors
    • openssf.scorecard.raw.frozen-deps
    • openssf.scorecard.raw.fuzzing
    • openssf.scorecard.raw.pull-requests
    • openssf.scorecard.raw.sast
    • openssf.scorecard.raw.security-policy
    • openssf.scorecard.raw.signed-releases
    • openssf.scorecard.raw.signed-tags
  • metrics.development.contributors.unique[90d] - Number of unique code contributors in the past 90 days
  • metrics.development.contributors.unique[365d] - Number of unique code contributors in the past 365 days
  • security-review - Content of a security review
  • openssf.bestpractice.raw.* - Metrics coming from the OpenSSF Best Practices Criteria. The full list can be found in the JSON output of the Best Practices API](https://github.com/coreinfrastructure/best-practices-badge/blob/master/doc/api.md).

Column: value

The value column is TEXT, and contains the primary quality indicator for the corresponding key.

Column: timestamp

The timestamp column should contain the most relevant date/time that the metric reflects.

Column: properties

The properties column is a JSON (technically jsonb) structure that can contain arbitrary data.

Dashboard Application

The dashboard application will use the open source version of Grafana. Anonymous viewer access will be enabled, and editor access will be available to members of the OpenSSF that will work on this project.

A development version of this application is currently available at: https://openssf-security-dashboard-dev1.westus2.cloudapp.azure.com/

Clone this wiki locally