Inline refs (#526)

* inline reference link

* replace numbered table reference

* formatting

* inline refs

docs/topics/asset_management.md

@@ -31,7 +31,7 @@ Once the organization remediates or mitigates all the high-priority vulnerabilit
 
 Asset management and risk management also drive some of the up-front work an organization would need to do to gather some of the necessary information.
 This situation is not new; an asset owner cannot prioritize which fixes to deploy to its assets if it does not have an accurate inventory of its assets.
-The organization can pick its choice of tools; there are about 200 asset management tools on the market [@captera].
+The organization can pick its choice of tools; there are [hundreds of asset management tools on the market](https://www.capterra.com/it-asset-management-software/).
 Emerging standards like the [Software Bill of Materials](https://www.cisa.gov/sbom) (SBOM) would likely reduce the burden on asset management, and organizations should prefer systems which make such information available.
 If an organization does not have an asset management or risk management
 (see also [Gathering Information About Mission Impact](../reference/decision_points/mission_impact.md))

docs/topics/evaluation_of_draft_trees.md

@@ -91,37 +91,31 @@ Third, the pilot provides a proof of concept method and metric that any vulnerab
 
 The vulnerabilities used as case studies are as follows. All quotes are from the [National Vulnerability Database (NVD)](https://nvd.nist.gov/) and are illustrative of the vulnerability; however, during the study each vulnerability was evaluated according to information analogous to that in the scenario table above.
 
-### Safety-Critical Cases
+!!! example "Safety-Critical Cases"
 
-  - [CVE-2015-5374](https://nvd.nist.gov/vuln/detail/CVE-2015-5374): “Vulnerability … in \[Siemens\] Firmware variant PROFINET IO for EN100 Ethernet module… Specially crafted packets sent to port 50000/UDP could cause a denial-of-service of the affected device…”
+    - [CVE-2015-5374](https://nvd.nist.gov/vuln/detail/CVE-2015-5374): “Vulnerability … in \[Siemens\] Firmware variant PROFINET IO for EN100 Ethernet module… Specially crafted packets sent to port 50000/UDP could cause a denial-of-service of the affected device…”
+    - [CVE-2014-0751](https://nvd.nist.gov/vuln/detail/CVE-2014-0751): “Directory traversal vulnerability in … GE Intelligent Platforms Proficy HMI/SCADA - CIMPLICITY before 8.2 SIM 24, and Proficy Process Systems with CIMPLICITY, allows remote attackers to execute arbitrary code via a crafted message to TCP port 10212, aka ZDI-CAN-1623.”
+    - [CVE-2015-1014](https://nvd.nist.gov/vuln/detail/CVE-2015-1014): “A successful exploit of these vulnerabilities requires the local user to load a crafted DLL file in the system directory on servers running Schneider Electric OFS v3.5 with version v7.40 of SCADA Expert Vijeo Citect/CitectSCADA, OFS v3.5 with version v7.30 of Vijeo Citect/CitectSCADA, and OFS v3.5 with version v7.20 of Vijeo Citect/CitectSCADA. If the application attempts to open that file, the application could crash or allow the attacker to execute arbitrary code.”
 
-  - [CVE-2014-0751](https://nvd.nist.gov/vuln/detail/CVE-2014-0751): “Directory traversal vulnerability in … GE Intelligent Platforms Proficy HMI/SCADA - CIMPLICITY before 8.2 SIM 24, and Proficy Process Systems with CIMPLICITY, allows remote attackers to execute arbitrary code via a crafted message to TCP port 10212, aka ZDI-CAN-1623.”
+!!! example "Regulated Systems Cases"
 
-  - [CVE-2015-1014](https://nvd.nist.gov/vuln/detail/CVE-2015-1014): “A successful exploit of these vulnerabilities requires the local user to load a crafted DLL file in the system directory on servers running Schneider Electric OFS v3.5 with version v7.40 of SCADA Expert Vijeo Citect/CitectSCADA, OFS v3.5 with version v7.30 of Vijeo Citect/CitectSCADA, and OFS v3.5 with version v7.20 of Vijeo Citect/CitectSCADA. If the application attempts to open that file, the application could crash or allow the attacker to execute arbitrary code.”
+    - [CVE-2018-14781](https://nvd.nist.gov/vuln/detail/CVE-2018-14781): “Medtronic insulin pump \[specific versions\] when paired with a remote controller and having the “easy bolus” and “remote bolus” options enabled (non-default), are vulnerable to a capture-replay attack. An attacker can … cause an insulin (bolus) delivery.”
+    - [CVE-2017-9590](https://nvd.nist.gov/vuln/detail/CVE-2017-9590): “The State Bank of Waterloo Mobile … app 3.0.2 … for iOS does not verify X.509 certificates from SSL servers, which allows man-in-the-middle attackers to spoof servers and obtain sensitive information via a crafted certificate.”
+    - [CVE-2017-3183](https://nvd.nist.gov/vuln/detail/CVE-2017-3183): “Sage XRT Treasury, version 3, fails to properly restrict database access to authorized users, which may enable any authenticated user to gain full access to privileged database functions. Sage XRT Treasury is a business finance management application. …”
 
-### Regulated Systems Cases
+!!! example "General Computing Cases"
 
-  - [CVE-2018-14781](https://nvd.nist.gov/vuln/detail/CVE-2018-14781): “Medtronic insulin pump \[specific versions\] when paired with a remote controller and having the “easy bolus” and “remote bolus” options enabled (non-default), are vulnerable to a capture-replay attack. An attacker can … cause an insulin (bolus) delivery.”
-
-  - [CVE-2017-9590](https://nvd.nist.gov/vuln/detail/CVE-2017-9590): “The State Bank of Waterloo Mobile … app 3.0.2 … for iOS does not verify X.509 certificates from SSL servers, which allows man-in-the-middle attackers to spoof servers and obtain sensitive information via a crafted certificate.”
-
-  - [CVE-2017-3183](https://nvd.nist.gov/vuln/detail/CVE-2017-3183): “Sage XRT Treasury, version 3, fails to properly restrict database access to authorized users, which may enable any authenticated user to gain full access to privileged database functions. Sage XRT Treasury is a business finance management application. …”
-
-### General Computing Cases
-
-  - [CVE-2019-2691](https://nvd.nist.gov/vuln/detail/CVE-2019-2691): “Vulnerability in the MySQL Server component of Oracle MySQL (subcomponent: Server: Security: Roles). Supported versions that are affected are 8.0.15 and prior. Easily exploitable vulnerability allows high privileged attacker with network access via multiple protocols to … complete DoS of MySQL Server.”
-
-  - [CVE-2019-9042](https://nvd.nist.gov/vuln/detail/CVE-2019-9042): “\[I\]n Sitemagic CMS v4.4… the user can upload a .php file to execute arbitrary code, as demonstrated by 404.php. This can only occur if the administrator neglects to set FileExtensionFilter and there are untrusted user accounts. …”
-
-  - [CVE-2017-5638](https://nvd.nist.gov/vuln/detail/CVE-2017-5638): “The Jakarta Multipart parser in Apache Struts 2 2.3.x before 2.3.32 and 2.5.x before 2.5.10.1 has incorrect exception handling and error-message generation during file-upload attempts, which allows remote attackers to execute arbitrary commands via crafted \[specific headers\], as exploited in the wild in March 2017…”
+    - [CVE-2019-2691](https://nvd.nist.gov/vuln/detail/CVE-2019-2691): “Vulnerability in the MySQL Server component of Oracle MySQL (subcomponent: Server: Security: Roles). Supported versions that are affected are 8.0.15 and prior. Easily exploitable vulnerability allows high privileged attacker with network access via multiple protocols to … complete DoS of MySQL Server.”
+    - [CVE-2019-9042](https://nvd.nist.gov/vuln/detail/CVE-2019-9042): “\[I\]n Sitemagic CMS v4.4… the user can upload a .php file to execute arbitrary code, as demonstrated by 404.php. This can only occur if the administrator neglects to set FileExtensionFilter and there are untrusted user accounts. …”
+    - [CVE-2017-5638](https://nvd.nist.gov/vuln/detail/CVE-2017-5638): “The Jakarta Multipart parser in Apache Struts 2 2.3.x before 2.3.32 and 2.5.x before 2.5.10.1 has incorrect exception handling and error-message generation during file-upload attempts, which allows remote attackers to execute arbitrary commands via crafted \[specific headers\], as exploited in the wild in March 2017…”
 
 ## Pilot Results
 
 For each of the nine CVEs, six analysts rated the priority of the vulnerability as both a supplier and deployer. The table below summarizes the results by reporting the inter-rater agreement for each decision point. For all measures, agreement (*k*) is above zero, which is generally interpreted as some agreement among analysts. Below zero is interpreted as noise or discord. Closer to 1 indicates more or stronger agreement.
 
 How close *k* should be to 1 before agreement can be considered strong enough or reliable enough is a matter of some debate. The value certainly depends on the number of options among which analysts select. For those decision points with five options (mission and safety impact), agreement is lowest. Although portfolio value has a higher *k* than mission or safety impact, it may not actually have higher agreement because portfolio value only has two options. The results for portfolio value are nearly indistinguishable as far as level of statistical agreement from mission impact and safety impact. The statistical community does not have hard and fast rules for cut lines on adequate agreement. We treat *k* as a descriptive statistic rather than a test statistic.
 
-The following table is encouraging, though not conclusive. *k*\<0 is a strong sign of discordance. Although it is unclear how close to 1 is success, *k*\<0 would be clear sign of failure. In some ways, these results may be undercounting the agreement for SSVC as presented. These results are for SSVC prior to the improvements documented in [Improvement Instigated by the Pilot](#improvements-instigated-by-the-pilot), which are implemented in SSVC version 1. On the other hand, the participant demographics may inflate the inter-rater agreement based on shared tacit understanding through the process of authorship. The one participant who was not an author surfaced two places where this was the case, but we expect the organizational homogeneity of the participants has inflated the agreement somewhat. The anecdotal feedback from vulnerability managers at several organizations (including VMware [@akbar2020ssvc] and McAfee) is about refinement and tweaks, not gross disagreement. Therefore, while further refinement is necessary, this evidence suggests the results have some transferability to other organizations and are not a total artifact of the participant organization demographics.
+The following table is encouraging, though not conclusive. *k*<0 is a strong sign of discordance. Although it is unclear how close to 1 is success, *k*<0 would be clear sign of failure. In some ways, these results may be undercounting the agreement for SSVC as presented. These results are for SSVC prior to the improvements documented in [Improvement Instigated by the Pilot](#improvements-instigated-by-the-pilot), which are implemented in SSVC version 1. On the other hand, the participant demographics may inflate the inter-rater agreement based on shared tacit understanding through the process of authorship. The one participant who was not an author surfaced two places where this was the case, but we expect the organizational homogeneity of the participants has inflated the agreement somewhat. The anecdotal feedback from vulnerability managers at several organizations (including VMware [@akbar2020ssvc] and McAfee) is about refinement and tweaks, not gross disagreement. Therefore, while further refinement is necessary, this evidence suggests the results have some transferability to other organizations and are not a total artifact of the participant organization demographics.
 
 Table: Inter-Rater Agreement for Decision Points
 
@@ -132,7 +126,7 @@ Table: Inter-Rater Agreement for Decision Points
 
 For all decision points, the presumed goal is for *k* to be close or equal to 1. The statistics literature has identified some limited cases in which Fleiss’ k behaves strangely—for example it is lower than expected when raters are split between 2 of q ratings when q\>2 [@falotico2015fleiss]. This paradox may apply to the safety and mission impact values, in particular. The paradox would bite hardest if the rating for each vulnerability was clustered on the same two values, for example, minor and major. Falotico and Quatto’s proposed solution is to permute the columns, which is safe with unordered categorical data. Since the nine vulnerabilities do not have the same answers as each other (that is, the answers are not clustered on the same two values), we happen to avoid the worst of this paradox, but the results for safety impact and mission impact should be interpreted with some care.
 
-This solution identifies another difficulty of Fleiss’ kappa, namely that it does not preserve any order; none and catastrophic are considered the same level of disagreement as none and minor. The table above displays a sense of the range of disagreement to complement this weakness. This value is the largest distance between rater selections on a single vulnerability out of the maximum possible distance. So, for safety impact, the most two raters disagreed was by two steps (none to major, minor to hazardous, or major to catastrophic) out of the four possible steps (none to catastrophic). The only values of *k* that are reliably comparable are those with the same number of options (that is, the same maximum distance). In other cases, closer to 1 is better, but how close is close enough to be considered “good” changes. In all but one case, if raters differed by two steps then there were raters who selected the central option between them. The exception was mission impact for CVE-201814781; it is unclear whether this discrepancy should be localized to a poor test scenario description, or to SSVC’s mission impact definition. Given it is an isolated occurrence, we expect the scenario description at least partly.
+This solution identifies another difficulty of Fleiss’ kappa, namely that it does not preserve any order; none and catastrophic are considered the same level of disagreement as none and minor. The table above displays a sense of the range of disagreement to complement this weakness. This value is the largest distance between rater selections on a single vulnerability out of the maximum possible distance. So, for safety impact, the most two raters disagreed was by two steps (none to major, minor to hazardous, or major to catastrophic) out of the four possible steps (none to catastrophic). The only values of *k* that are reliably comparable are those with the same number of options (that is, the same maximum distance). In other cases, closer to 1 is better, but how close is close enough to be considered “good” changes. In all but one case, if raters differed by two steps then there were raters who selected the central option between them. The exception was mission impact for [CVE-2018-14781](https://nvd.nist.gov/vuln/detail/CVE-2018-14781); it is unclear whether this discrepancy should be localized to a poor test scenario description, or to SSVC’s mission impact definition. Given it is an isolated occurrence, we expect the scenario description at least partly.
 
 Nonetheless, *k* provides some way to measure improvement on this a conceptual engineering task. The pilot evaluation can be repeated, with more diverse groups of stakeholders after the descriptions have been refined by stakeholder input, to measure fit to this goal. For a standard to be reliably applied across different analyst backgrounds, skill sets, and cultures, a set of decision point descriptions should ideally achieve *k* of 1 for each item in multiple studies with diverse participants. Such a high level of agreement would be difficult to achieve, but it would ensure that when two analysts assign a priority with the system that they get the same answer. Such agreement is not the norm with CVSS currently [@allodi2018effect].
 

docs/topics/related_systems.md

@@ -2,8 +2,8 @@
 # Related Vulnerability Management Systems
 
 There are several other bodies of work that are used in practice to assist vulnerability managers in making decisions.
-Three relevant systems are CVSS [@cvss_v3-1], EPSS [@jacobs2021epss], and Tenable's Vulnerability Priority Rating ([VPR](https://www.tenable.com/blog/what-is-vpr-and-how-is-it-different-from-cvss)).
-There are other systems derived from CVSS, such as RVSS for robots [@vilches2018towards] and MITRE's effort to adapt CVSS to medical devices [@mitre2019medical].
+Three relevant systems are [CVSS](https://www.first.org/cvss/v3.1/specification-document), [EPSS](https://dl.acm.org/doi/10.1145/3436242), and Tenable's Vulnerability Priority Rating ([VPR](https://www.tenable.com/blog/what-is-vpr-and-how-is-it-different-from-cvss)).
+There are other systems derived from CVSS, such as RVSS for robots [@vilches2018towards] and MITRE's [Rubric for Applying CVSS to Medical Devices](https://www.mitre.org/news-insights/publication/rubric-applying-cvss-medical-devices).
 There are also other nascent efforts to automate aspects of the decision making process, such as [vPrioritizer](https://github.com/varchashva/vPrioritizer).
 This section discusses the relationship between these various systems and SSVC.
 

docs/topics/worked_example.md

@@ -81,7 +81,7 @@ Depending on the details of the hospital’s contingency plans and its monitorin
 If there is no way to tell whether the insulin pumps are misbehaving, for example, then exploitation could go on for
 some time, leading to a [*catastrophic*](../reference/decision_points/safety_impact.md) [*Safety Impact*](../reference/decision_points/safety_impact.md).
 The pilot information is inadequate in this regard, which is the likely source of disagreement about
-[*Safety Impact*](../reference/decision_points/safety_impact.md) in Table 13.
+[*Safety Impact*](../reference/decision_points/safety_impact.md) in our evaluation of [inter-rater agreement](evaluation_of_draft_trees.md).
 For the purposes of this example, imagine that after gathering that information, the monitoring situation is adequate,
 and select [*hazardous*](../reference/decision_points/safety_impact.md).
 Therefore, mitigate this vulnerability *out-of-cycle*, meaning that it should be addressed quickly, ahead of the usual