The primary goal of this analysis is to understand the data sources, schema, and counts of the Global Security Database (GSD).
The below analysis was completed on the GSD timestamp: 2022-05-20T05:00:05:000. To replicate the below results checkout the gsd-database at commit SHA d8ce33d48de2f00130e821e9828c3e04b9a4b520.
The gsd_analysis.py can be used to replicate the below results, note it takes approximately 1 hour and 20 minutes to complete, the goal here wasn't optimization as I don't plan to run this script often.
The schema was generated using the GenSON package. Each GSD JSON was opened and added to the GenSON schema object to create an overall schema for all potential entries within GSD.
Below is a condensed schema after loading all GSD entries.
{
"GSD": {"type": "object"},
"OSV": {"type": "object"},
"namespaces": {
"properties": {
"cisa.gov": {"type": "object"},
"cve.org": {"type": "object"},
"gitlab.com": {"type": "object"},
"nvd.nist.gov": {"type": "object"},
"github.com/kurtseifried:582211": {"type": "object"}
}
},
"overlay": {
"properties": {
"cve.org": {"type": "object"}
}
}
}Complete schema: ./data/schemas/gsd_complete_schema.json
From the above schema a set of data sources/keys can be identified:
- Ideally, every entry should have a GSD identifier.
Full schema can be seen here: ADD LINK / FILE
{
"affected": {"type": "array"},
"affected_component": {"type": "string"},
"affects": {"type": "object"},
"alias": {"type": "string"},
"attack_vector": {"type": "string"},
"credit": {"type": "string"},
"data_format": {"type": "string"},
"data_type": {"type": "string"},
"data_version": {"type": "string"},
"description": {"type": "string"},
"details": {"type": "string"},
"duplicate": {"type": "string"},
"extended_references": {"type": "array"},
"id": {"type": "string"},
"impact": {"type": "object"},
"modified":{"type": "string"},
"notes": {"type": "string"},
"product_name": {"type": "object"},
"product_version": {"type": "string"},
"published": {"type": "string"},
"references": {"type": "array"},
"reporter": {"type": "string"},
"reporter_id": {"type": "integer"},
"summary": {"type": "string"},
"vendor_name": {"type": "string"},
"vulnerability_type": {"type": "object"},
"withdrawn": {"type": "string"}
}Complete GSD object schema: ./data/schemas/schema_gsd_object.json
-
Two entries are missing GSD objects, both come from GitLab advisories:
-
18 entries only contain GSD keys:
-
15 of these are very short, example format:
{ "GSD": { "alias": "CVE-YYYY-XXXXX", "id": "GSD-YYYY-XXXXX", "references": ["string"] } }- https://raw.globalsecuritydatabase.org/GSD-2009-5515
- https://raw.globalsecuritydatabase.org/GSD-2010-13616
- https://raw.globalsecuritydatabase.org/GSD-2010-26432
- https://raw.globalsecuritydatabase.org/GSD-2014-1197
- https://raw.globalsecuritydatabase.org/GSD-2015-9679
- https://raw.globalsecuritydatabase.org/GSD-2016-1000212
- https://raw.globalsecuritydatabase.org/GSD-2016-1000247
- https://raw.globalsecuritydatabase.org/GSD-2017-171479
- https://raw.globalsecuritydatabase.org/GSD-2019-1000029
- https://raw.globalsecuritydatabase.org/GSD-2019-1002162
- https://raw.globalsecuritydatabase.org/GSD-2019-14560
- https://raw.globalsecuritydatabase.org/GSD-2019-15167
- https://raw.globalsecuritydatabase.org/GSD-2019-15690
- https://raw.globalsecuritydatabase.org/GSD-2021-3513
- https://raw.globalsecuritydatabase.org/GSD-2021-47527
-
Two entries come from "kurtseifried" as the reporter and are much more detailed:
-
One is a GSD duplicate: https://raw.globalsecuritydatabase.org/GSD-2022-1000292
-
The actual OSV format: https://ossf.github.io/osv-schema/
Examples from GSD with an OSV object:
- https://raw.globalsecuritydatabase.org/GSD-2021-1001247
- https://raw.globalsecuritydatabase.org/GSD-2022-1000988
{
"properties": {
"affected": {"type": "array"},
"alias": {"type": "string"},
"details": {"type": "string"},
"id": {"type": "string"},
"modified": {"type": "string"},
"published": {"type": "string"},
"references": {"type": "array"},
"summary": {"type": "string"}
},
"required": [
"affected",
"details",
"id",
"modified",
"published",
"summary"
],
"type": "object"
}Complete OSV object schema: ./data/schemas/schema_osv.json
Examples:
- https://raw.globalsecuritydatabase.org/GSD-2014-6332
- https://raw.globalsecuritydatabase.org/GSD-2021-27085
{
"properties": {
"cveID": {"type": "string"},
"dateAdded": {"type": "string"},
"dueDate": {"type": "string"},
"product": {"type": "string"},
"requiredAction": {"type": "string"},
"shortDescription": {"type": "string"},
"vendorProject": {"type": "string"},
"vulnerabilityName": {"type": "string"}
},
"required": [
"cveID",
"dateAdded",
"dueDate",
"product",
"requiredAction",
"shortDescription",
"vendorProject",
"vulnerabilityName"
],
"type": "object"
}Complete cisa.gov object schema: ./data/schemas/schema_cisa.json
Examples:
- https://raw.globalsecuritydatabase.org/GSD-2018-3367
- https://raw.globalsecuritydatabase.org/GSD-2020-4008
{
"properties": {
"CVE_data_meta": {"type": "object"},
"affects": {"type": "object"},
"configuration": {"type": "object"},
"credit": {"type": "object"},
"cve_id": {"type": "string"},
"data_format": {"type": "string"},
"data_type": {"type": "string"},
"data_version": {"type": "string"},
"description": {"type": "object"},
"discoverer": {"type": "string"},
"exploit": {"type": "object"},
"generator": {"type": "object"},
"impact": {"type": "object"},
"problemtype": {"type": "object"},
"references": {"type": "object"},
"solution": {"type": "object"},
"source": {"type": "object"},
"timeline": {"type": "array"},
"work_around": {"type": "array"},
"workaround": {"type": "object"},
"x_advisoryEoL": {"type": "boolean"},
"x_affectedList": {"type": "array"},
"x_likelyAffectedList": {"type": "array"}
},
"required": [
"CVE_data_meta",
"data_format",
"data_type",
"data_version",
"description"
],
"type": "object"
}Complete cve.org object schema: ./data/schemas/schema_cve_org.json
Examples:
- https://raw.globalsecuritydatabase.org/GSD-2018-1000805
- https://raw.globalsecuritydatabase.org/GSD-2020-5301
gitlab.com schema:
{
"properties": {
"advisories": {
"items": {
"properties": {
"_git_import_path": {"type": "string"},
"affected_range": {"type": "string"},
"affected_versions": {"type": "string"},
"credit": {"type": "string"},
"cvss_v2": {"type": "string"},
"cvss_v3": {"type": "string"},
"cwe_ids": {"type": "array"},
"date": {"type": "string"},
"description": {"type": "string"},
"fixed_versions": {"type": "array"},
"identifier": {"type": "string"},
"identifiers": {"type": "array"},
"not_impacted": {"type": "string"},
"package_slug": {"type": "string"},
"pubdate": {"type": "string"},
"solution": {"type": "string"},
"title": {"type": "string"},
"urls": {"type": "array"},
"uuid": {"type": "string"},
"versions": {"type": "array"}
},
"required": [
"affected_range",
"affected_versions",
"cwe_ids",
"date",
"description",
"fixed_versions",
"identifier",
"identifiers",
"package_slug",
"pubdate",
"title",
"urls",
"uuid"
],
"type": "object"
},
"type": "array"
}
},
"required": [
"advisories"
],
"type": "object"
}Complete gitlab.com object schema: ./data/schemas/schema_gitlab.json
Examples:
- https://raw.globalsecuritydatabase.org/GSD-2017-12452
- https://raw.globalsecuritydatabase.org/GSD-2021-21956
nvd.nist.gov schema:
{
"properties": {
"configurations": {"type": "object"},
"cve": {"type": "object"},
"impact": {"type": "object"},
"lastModifiedDate": {"type": "string"},
"publishedDate": {"type": "string"}
},
"required": [
"configurations",
"cve",
"impact",
"lastModifiedDate",
"publishedDate"
],
"type": "object"
}Complete nvd.nist.gov object schema: ./data/schemas/schema_nvd.json
- Eight entries contain a kurtseifried:582211 key:
- https://raw.globalsecuritydatabase.org/GSD-2022-1000001
- https://raw.globalsecuritydatabase.org/GSD-2022-1000002
- https://raw.globalsecuritydatabase.org/GSD-2022-1000003
- https://raw.globalsecuritydatabase.org/GSD-2022-1000004
- https://raw.globalsecuritydatabase.org/GSD-2022-1000005
- https://raw.globalsecuritydatabase.org/GSD-2022-1000066
- https://raw.globalsecuritydatabase.org/GSD-2022-1000067
- https://raw.globalsecuritydatabase.org/GSD-2022-1000068
- Two entries have an overlay key: