A set of Bash scripts to support creating PDF reports using the Black Duck API.
Json-to-pdf (https://github.com/yogthos/json-to-pdf) is used to generate PDF files allowing configuration of the PDF layout, content and format in template json files. Also uses jq to format JSON data from API calls (must be preinstalled).
List of scripts included in this package:
- project_bom_report.sh: Produces an overall report showing alphabetical list of components, including bar charts of license, vulnerability and operational risk (example report here)
- project_license_report.sh: Produces a license report showing list of components sorted by license risk, including a bar chart of license risk (example report here)
- project_security_report.sh: Produces a security report showing list of components sorted by vulnerability counts, including bar charts of security risk by component and overall (example report here)
- project_custom_report.sh: Generate a general report of components using a custom template which can include API fields selected in the template (example report here)
Linux & MacOS (bash required)
JQ must be pre-installed (https://stedolan.github.io/jq/) - can be installed using yum/brew etc.
- Extract the project files to a chosen folder.
- Ensure script files
scripts/*.shhave execute permission (chmod +x) - Add the scripts folder to the path (e.g.
export PATH=$PATH:/user/myuser/BDReporting/scripts) - Create an API code (user access token) in the BD interface (use Username-->My Profile-->User Access Token)
- Update the HUBURL and APICODE values in the conf/bdreport.env file to represent your BD server and API code.
To produce an overall project report, run the command:
project_bom_report.sh projname projversion
where:
- projname is a full project name string (use quotes if it includes spaces)
- projversion is a full version name string (use quotes if it includes spaces)
Upon successful execution, a PDF file with the name report_overall_projname_projversion.pdf will be created in the invocation folder.
If the projname string does not match a full project then a list of matching project names will be shown and the script will terminate. For example running the command:
project_bom_report.sh duck
Would result in the following example output:
Matching projects found:
[
"black-duck-detect",
"BlackduckNugetInspector",
"blackducksoftware_blackduck-webapp",
"Duck",
"Duck Hub Demo Jenkins Sandbox",
"duck-hub",
"JenkinsDucky"
]
Please rerun selecting specific Project and supply Version
ERROR: Unable to find project + version
This can be repeated for the projversion string provided a valid projectname is provided.
It is also possible to specify the PDF filename in the command, for example:
project_bom_report.sh projname projversion PDFfile
The script project_bom_report.sh will produce an overall summary project report including 3 bar charts showing vulnerabiity, license and operational risk (count of components) and an alphabetical table of components with the following headings:
- Component
- Version
- Usage
- Security Risk
- License Risk
- Operational Risk
- Reviewed
- Policy Violation
The script uses the template file template_bom.json, and produces an output PDF file report_overall_projname_projversion.pdf by default.
You can find an example overall project report here.
Use the command:
project_bom_report.sh projname projversion [pdffile]
where:
- projname (required) is a full project name string (use quotes if it includes spaces)
- projversion (required) is a full version name string (use quotes if it includes spaces)
- pdffile (optional) is an alternate output PDF file name.
The template file template_bom.json is used to define the content and layout of the generated PDF file. To learn more about the json-to-pdf format see here. Note that it is not possible to remove columns in the table by deleting them from the template alone; you will need to modify the script logic to delete columns.
Several values will be replaced within the template as follows:
- __PROJECTNAME__ - Project name string
- __VERSIONNAME__ - Version name string
- __SEC_HIGH__ - Count of components with high risk vulnerabilities
- __SEC_MEDIUM__ - Count of components with medium risk vulnerabilities
- __SEC_LOW__ - Count of components with low risk vulnerabilities
- __SEC_NONE__ - Count of components with no vulnerabilities
- __LIC_HIGH__ - Count of components with high risk licenses
- __LIC_MEDIUM__ - Count of components with medium risk licenses
- __LIC_LOW__ - Count of components with low risk licenses
- __LIC_NONE__ - Count of components with no risk licenses
- __OP_HIGH__ - Count of components with high operational risk
- __OP_MEDIUM__ - Count of components with medium operational risk
- __OP_LOW__ - Count of components with low operational risk
- __OP_NONE__ - Count of components with no operational risk
The script project_license_report.sh will produce a license project report including a bar chart showing license risk (count of components) and an alphabetical table of components with the following headings:
- Component
- Version
- License
- Risk
The script uses the template file template/template_license.json, and produces an output PDF file report_licensing_projname_projversion.pdf by default.
You can find an example report here.
Use the command:
project_license_report.sh projname projversion [pdffile]
where:
- projname (required) is a full project name string (use quotes if it includes spaces)
- projversion (required) is a full version name string (use quotes if it includes spaces)
- pdffile (optional) is an alternate output PDF file name.
The template file template/template_license.json is used to define the content and layout of the generated PDF file. To learn more about the json-to-pdf format see here. Note that it is not possible to remove columns in the table by deleting them from the template alone; you will need to modify the script logic to delete columns.
Several values will be replaced within the template as follows:
- __PROJECTNAME__ - Project name string
- __VERSIONNAME__ - Version name string
- __COUNT_HIGH__ - Count of components with high risk licenses
- __COUNT_MEDIUM__ - Count of components with medium risk licenses
- __COUNT_LOW__ - Count of components with low risk licenses
- __COUNT_NONE__ - Count of components with no risk licenses
The script project_security_report.sh will produce a license project report including 2 bar charts showing security risk (count of components), total count of vulnerabilities and a table of components sorted by vulnerability counts with the following headings:
- Component
- Version
- High
- Medium
- Low
The script uses the template file template/template_security.json, and produces an output PDF file report_security_projname_projversion.pdf by default.
You can find an example report here.
Use the command:
project_security_report.sh projname projversion [pdffile]
where:
- projname (required) is a full project name string (use quotes if it includes spaces)
- projversion (required) is a full version name string (use quotes if it includes spaces)
- pdffile (optional) is an alternate output PDF file name.
The template file template/template_security.json is used to define the content and layout of the generated PDF file. To learn more about the json-to-pdf format see here. Note that it is not possible to remove columns in the table by deleting them from the template alone; you will need to modify the script logic to delete columns.
Multiple values will be replaced within the template as follows:
- __PROJECTNAME__ - Project name string
- __VERSIONNAME__ - Version name string
- __COMP_HIGH__ - Count of components with high vulnerability risk
- __COMP_MEDIUM__ - Count of components with medium vulnerability risk
- __COMP_LOW__ - Count of components with low vulnerability risk
- __COMP_NONE__ - Count of components with no vulnerability risk
- __VULNS_HIGH__ - Total count of high risk vulnerabilities
- __VULNS_MEDIUM__ - Total count of medium risk vulnerabilities
- __VULNS_LOW__ - Total count of low risk vulnerabilities
The script project_custom_report.sh will produce a custom project report including, by default, 3 bar charts showing security risk (count of components), total count of vulnerabilities and an alphabetical table of components with custom headings.
A template file must be specified as the first argument; an example is provided in template/template_security.json. The output PDF file is report_security_projname_projversion.pdf by default.
You can find an example report here.
Use the command:
project_custom_report.sh template_custom.json projname projversion [pdffile]
where:
- template_custom.json (required) is a json template file in the template folder (file name only required - not full path).
- projname (required) is a full project name string (use quotes if it includes spaces)
- projversion (required) is a full version name string (use quotes if it includes spaces)
- pdffile (optional) is an alternate output PDF file name.
Create a template file if you want to modify the content and layout of the generated PDF file (default example provided in template/template_custom.json).
The file contains the json-to-pdf formatting for the start of the document with some fields which will be replaced by values by the script as follows:
- __PROJECTNAME__ - Project name string
- __VERSIONNAME__ - Version name string
- __SEC_HIGH__ - Count of components with high risk vulnerabilities
- __SEC_MEDIUM__ - Count of components with medium risk vulnerabilities
- __SEC_LOW__ - Count of components with low risk vulnerabilities
- __SEC_NONE__ - Count of components with no vulnerabilities
- __LIC_HIGH__ - Count of components with high risk licenses
- __LIC_MEDIUM__ - Count of components with medium risk licenses
- __LIC_LOW__ - Count of components with low risk licenses
- __LIC_NONE__ - Count of components with no risk licenses
- __OP_HIGH__ - Count of components with high operational risk
- __OP_MEDIUM__ - Count of components with medium operational risk
- __OP_LOW__ - Count of components with low operational risk
- __OP_NONE__ - Count of components with no operational risk
Two lines starting with the strings __TABLE_ROWS__ and __REPLACEMENTS__ are used to define the layout and content of the table.
The '__TABLE_ROWS__:' line defines the columns to be shown in the table comprising the fields to be included. The fields are space delimited and are either in 'jq search format' defining fields to be extracted from the JSON API response or one of the strings __SEC_TEXT__, __LIC_TEXT__ or __OP_TEXT__. More information about jq search format strings can be found here. The JSON API response includes an array of components called items[] with multiple fields. An example JSON API response is provided in the file examples/API_components.json.
An example line format is shown below:
__TABLE_ROWS__:[.items[].componentName] [.items[].componentVersionName] [.items[].usages[0]] [.items[].reviewStatus] [.items[].policyStatus] __SEC_TEXT__
The '__REPLACEMENTS__:' line is used to replace text strings in the table for output formatting. The replacement pairs are separated by '|' and use ';' to separate the original text with the replacement. To explain why this is required, the '[.items[].reviewStatus]' column will return one of the following values 'DYNAMICALLY_LINKED,STATICALLY_LINKED,DEV_\TOOL_EXCLUDED' which can be replaced with strings suited to display in the PDF.
An example replacements line is shown below:
__REPLACEMENTS__:NOT_REVIEWED;No|REVIEWED;Yes|NOT_IN_VIOLATION;Yes|IN_VIOLATION;Yes|DYNAMICALLY_LINKED;Dynamically Linked|STATICALLY_LINKED;Statically Linked|DEV_TOOL_EXCLUDED;Dev Tool/Excluded
Json-to-pdf is documented at https://github.com/yogthos/json-to-pdf.
An example json file is available [here](https://github.com/matthewb66/blackduckPDFreporting/blob/master/examples/json-to-pdf_example.json} which shows several of the additional capabilities to format PDF files.