GSD Analysis

The primary goal of this analysis is to understand the data sources, schema, and counts of the Global Security Database (GSD).

The below analysis was completed on the GSD timestamp: 2022-05-20T05:00:05:000. To replicate the below results checkout the gsd-database at commit SHA d8ce33d48de2f00130e821e9828c3e04b9a4b520.

The gsd_analysis.py can be used to replicate the below results, note it takes approximately 1 hour and 20 minutes to complete, the goal here wasn't optimization as I don't plan to run this script often.

GSD Counts

GSD Complete Schema

The schema was generated using the GenSON package. Each GSD JSON was opened and added to the GenSON schema object to create an overall schema for all potential entries within GSD.

Below is a condensed schema after loading all GSD entries.

{
    "GSD": {"type":  "object"},
    "OSV": {"type":  "object"},
    "namespaces": {
        "properties": {
            "cisa.gov": {"type":  "object"},
            "cve.org": {"type":  "object"},
            "gitlab.com": {"type":  "object"},
            "nvd.nist.gov": {"type":  "object"},
            "github.com/kurtseifried:582211": {"type":  "object"}
        }
    },
    "overlay": {
        "properties": {
            "cve.org": {"type":  "object"}
        }
    }
}

Complete schema: ./data/schemas/gsd_complete_schema.json

Data Sources

From the above schema a set of data sources/keys can be identified:

GSD:

Ideally, every entry should have a GSD identifier.

GSD Schema:

Full schema can be seen here: ADD LINK / FILE

{
    "affected": {"type":  "array"},
    "affected_component": {"type":  "string"},
    "affects": {"type":  "object"},
    "alias": {"type":  "string"},
    "attack_vector": {"type":  "string"},
    "credit": {"type":  "string"},
    "data_format": {"type":  "string"},
    "data_type": {"type":  "string"},
    "data_version": {"type":  "string"},
    "description": {"type":  "string"},
    "details": {"type":  "string"},
    "duplicate": {"type":  "string"},
    "extended_references": {"type":  "array"},
    "id": {"type":  "string"},
    "impact": {"type":  "object"},
    "modified":{"type":  "string"},
    "notes": {"type":  "string"},
    "product_name": {"type": "object"},
    "product_version": {"type":  "string"},
    "published": {"type":  "string"},
    "references": {"type":  "array"},
    "reporter": {"type":  "string"},
    "reporter_id": {"type":  "integer"},
    "summary": {"type":  "string"},
    "vendor_name": {"type":  "string"},
    "vulnerability_type": {"type": "object"},
    "withdrawn": {"type":  "string"}
}

Complete GSD object schema: ./data/schemas/schema_gsd_object.json

GSD Outliers:

Two entries are missing GSD objects, both come from GitLab advisories:
- https://raw.globalsecuritydatabase.org/GSD-2016-1000249
- https://raw.globalsecuritydatabase.org/GSD-2016-1000253
18 entries only contain GSD keys:
- 15 of these are very short, example format:
```
{
    "GSD": {
        "alias": "CVE-YYYY-XXXXX",
        "id": "GSD-YYYY-XXXXX",
        "references": ["string"]
    }
}
```
- Two entries come from "kurtseifried" as the reporter and are much more detailed:
  - https://raw.globalsecuritydatabase.org/GSD-2022-1000000
  - https://raw.globalsecuritydatabase.org/GSD-2022-1000006
- One is a GSD duplicate: https://raw.globalsecuritydatabase.org/GSD-2022-1000292

OSV:

The actual OSV format: https://ossf.github.io/osv-schema/

Examples from GSD with an OSV object:

OSV Schema

{
    "properties": {
        "affected": {"type":  "array"},
        "alias": {"type":  "string"},
        "details": {"type":  "string"}, 
        "id": {"type":  "string"},
        "modified": {"type":  "string"},
        "published": {"type":  "string"},
        "references": {"type":  "array"},
        "summary": {"type":  "string"}
    },
    "required": [
        "affected",
        "details",
        "id",
        "modified",
        "published",
        "summary"
    ],
    "type": "object"
}

Complete OSV object schema: ./data/schemas/schema_osv.json

cisa.gov:

Examples:

cisa.gov schema:

{
    "properties": {
        "cveID": {"type": "string"},
        "dateAdded": {"type": "string"},
        "dueDate": {"type": "string"},
        "product": {"type": "string"},
        "requiredAction": {"type": "string"},
        "shortDescription": {"type": "string"},
        "vendorProject": {"type": "string"},
        "vulnerabilityName": {"type": "string"}
    },
    "required": [
        "cveID",
        "dateAdded",
        "dueDate",
        "product",
        "requiredAction",
        "shortDescription",
        "vendorProject",
        "vulnerabilityName"
    ],
    "type": "object"
}

Complete cisa.gov object schema: ./data/schemas/schema_cisa.json

cve.org:

Examples:

cve.org schema:

{
    "properties": {
        "CVE_data_meta": {"type":  "object"},
        "affects": {"type":  "object"},
        "configuration": {"type":  "object"},
        "credit": {"type":  "object"},
        "cve_id": {"type":  "string"},
        "data_format": {"type":  "string"},
        "data_type": {"type":  "string"},
        "data_version": {"type":  "string"},
        "description": {"type":  "object"},
        "discoverer": {"type":  "string"},
        "exploit": {"type":  "object"},
        "generator": {"type":  "object"},
        "impact": {"type":  "object"},
        "problemtype": {"type":  "object"},
        "references": {"type":  "object"},
        "solution": {"type":  "object"},
        "source": {"type":  "object"},
        "timeline": {"type":  "array"},
        "work_around": {"type":  "array"},
        "workaround": {"type":  "object"},
        "x_advisoryEoL": {"type":  "boolean"},
        "x_affectedList": {"type":  "array"},
        "x_likelyAffectedList": {"type":  "array"}
    },
    "required": [
        "CVE_data_meta",
        "data_format",
        "data_type",
        "data_version",
        "description"
    ],
    "type": "object"
}

Complete cve.org object schema: ./data/schemas/schema_cve_org.json

gitlab.com:

Examples:

gitlab.com schema:

{
    "properties": {
        "advisories": {
            "items": {
                "properties": {
                    "_git_import_path": {"type":  "string"},
                    "affected_range": {"type":  "string"},
                    "affected_versions": {"type":  "string"},
                    "credit": {"type":  "string"},
                    "cvss_v2": {"type":  "string"},
                    "cvss_v3": {"type":  "string"},
                    "cwe_ids": {"type":  "array"},
                    "date": {"type":  "string"},
                    "description": {"type":  "string"},
                    "fixed_versions": {"type":  "array"},
                    "identifier": {"type":  "string"},
                    "identifiers": {"type":  "array"},
                    "not_impacted": {"type":  "string"},
                    "package_slug": {"type":  "string"},
                    "pubdate": {"type":  "string"},
                    "solution": {"type":  "string"},
                    "title": {"type":  "string"},
                    "urls": {"type":  "array"},
                    "uuid": {"type":  "string"},
                    "versions": {"type":  "array"}
                },
                "required": [
                    "affected_range",
                    "affected_versions",
                    "cwe_ids",
                    "date",
                    "description",
                    "fixed_versions",
                    "identifier",
                    "identifiers",
                    "package_slug",
                    "pubdate",
                    "title",
                    "urls",
                    "uuid"
                ],
                "type": "object"
            },
            "type": "array"
        }
    },
    "required": [
        "advisories"
    ],
    "type": "object"
}

Complete gitlab.com object schema: ./data/schemas/schema_gitlab.json

nvd.nist.gov:

Examples:

nvd.nist.gov schema:

{
    "properties": {
        "configurations": {"type":  "object"},
        "cve": {"type":  "object"},
        "impact": {"type":  "object"},
        "lastModifiedDate": {"type":  "string"},
        "publishedDate": {"type":  "string"}
    },
    "required": [
        "configurations",
        "cve",
        "impact",
        "lastModifiedDate",
        "publishedDate"
    ],
    "type": "object"
}

Complete nvd.nist.gov object schema: ./data/schemas/schema_nvd.json

github.com/kurtseifried:582211:

Eight entries contain a kurtseifried:582211 key:

overlay:

Two entries have an overlay key:
- https://raw.globalsecuritydatabase.org/GSD-2021-1002352
- https://raw.globalsecuritydatabase.org/GSD-2021-1002353

Name		Name	Last commit message	Last commit date
Latest commit History 15 Commits
data		data
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
gsd_analysis.py		gsd_analysis.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

GSD Analysis

GSD Counts

GSD Complete Schema

Data Sources

GSD:

GSD Schema:

GSD Outliers:

OSV:

OSV Schema

cisa.gov:

cisa.gov schema:

cve.org:

cve.org schema:

gitlab.com:

nvd.nist.gov:

github.com/kurtseifried:582211:

overlay:

About

Releases

Packages

Languages

License

tdunlap607/gsd-analysis

Folders and files

Latest commit

History

Repository files navigation

GSD Analysis

GSD Counts

GSD Complete Schema

Data Sources

GSD:

GSD Schema:

GSD Outliers:

OSV:

OSV Schema

cisa.gov:

cisa.gov schema:

cve.org:

cve.org schema:

gitlab.com:

nvd.nist.gov:

github.com/kurtseifried:582211:

overlay:

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages