Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Use osv-scalibr SBOM extractors #1380

Draft
wants to merge 26 commits into
base: v2
Choose a base branch
from

Commits on Oct 24, 2024

  1. docs: update documentation about Maven registry support (google#1340)

    google#1286 adds support for Maven
    registry during resolution.
    
    As a follow up, this PR updates the documentation for transitive
    scanning about specifying data source during resolution as well as
    specifying Maven registry.
    
    This PR also corrects the deps.dev API version we are using.
    
    We also need to update the documentation in
    google#1181.
    cuixq authored Oct 24, 2024
    Configuration menu
    Copy the full SHA
    24aca23 View commit details
    Browse the repository at this point in the history
  2. feat: add --experimental-offline-vulnerabilities and `--experimenta…

    …l-no-resolve` flags (google#1342)
    
    Closes google#1339 and closes google#1121 
    Adds flags to use offline mode for vulnerabilities
    (`--experimental-offline-vulnerabilities`) and transitive resolution
    separately (`--experimental-no-resolve`)
    
    The original `--experimental-offline` flag retains the same behaviour by
    functionally setting both of these flags.
    michaelkedar authored Oct 24, 2024
    Configuration menu
    Copy the full SHA
    0c43a0e View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. chore: Also trigger workflow when merging into v2 (google#1343)

    Currently we can't reliably merge into v2 because checks don't
    automatically trigger
    another-rex authored Oct 25, 2024
    Configuration menu
    Copy the full SHA
    5e0e196 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. test: update snapshot (google#1354)

    Another day, another snapshot change 😄
    G-Rath authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    ce6950e View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2024

  1. chore: remove unused fixture file (google#1353)

    I'm not sure exactly where this file came from but we're not using it so
    it can go
    G-Rath authored Oct 30, 2024
    Configuration menu
    Copy the full SHA
    ff81dcd View commit details
    Browse the repository at this point in the history
  2. chore(deps-dev): bump rexml from 3.3.8 to 3.3.9 in /docs in the bundl…

    …er group (google#1349)
    
    Bumps the bundler group in /docs with 1 update:
    [rexml](https://github.com/ruby/rexml).
    
    Updates `rexml` from 3.3.8 to 3.3.9
    <details>
    <summary>Release notes</summary>
    <p><em>Sourced from <a
    href="https://github.com/ruby/rexml/releases">rexml's
    releases</a>.</em></p>
    <blockquote>
    <h2>REXML 3.3.9 - 2024-10-24</h2>
    <h3>Improvements</h3>
    <ul>
    <li>Improved performance.
    <ul>
    <li><a
    href="https://redirect.github.com/ruby/rexml/issues/210">GH-210</a></li>
    <li>Patch by NAITOH Jun.</li>
    </ul>
    </li>
    </ul>
    <h3>Fixes</h3>
    <ul>
    <li>
    <p>Fixed a parse bug for text only invalid XML.</p>
    <ul>
    <li><a
    href="https://redirect.github.com/ruby/rexml/issues/215">GH-215</a></li>
    <li>Patch by NAITOH Jun.</li>
    </ul>
    </li>
    <li>
    <p>Fixed a parse bug that <code>&amp;#0x...;</code> is accepted as a
    character
    reference.</p>
    </li>
    </ul>
    <h3>Thanks</h3>
    <ul>
    <li>NAITOH Jun</li>
    </ul>
    </blockquote>
    </details>
    <details>
    <summary>Changelog</summary>
    <p><em>Sourced from <a
    href="https://github.com/ruby/rexml/blob/master/NEWS.md">rexml's
    changelog</a>.</em></p>
    <blockquote>
    <h2>3.3.9 - 2024-10-24 {#version-3-3-9}</h2>
    <h3>Improvements</h3>
    <ul>
    <li>Improved performance.
    <ul>
    <li><a
    href="https://redirect.github.com/ruby/rexml/issues/210">GH-210</a></li>
    <li>Patch by NAITOH Jun.</li>
    </ul>
    </li>
    </ul>
    <h3>Fixes</h3>
    <ul>
    <li>
    <p>Fixed a parse bug for text only invalid XML.</p>
    <ul>
    <li><a
    href="https://redirect.github.com/ruby/rexml/issues/215">GH-215</a></li>
    <li>Patch by NAITOH Jun.</li>
    </ul>
    </li>
    <li>
    <p>Fixed a parse bug that <code>&amp;#0x...;</code> is accepted as a
    character
    reference.</p>
    </li>
    </ul>
    <h3>Thanks</h3>
    <ul>
    <li>NAITOH Jun</li>
    </ul>
    </blockquote>
    </details>
    <details>
    <summary>Commits</summary>
    <ul>
    <li><a
    href="https://github.com/ruby/rexml/commit/38eaa86ac7abe0d31cf49d8df57ad239fdeb80e9"><code>38eaa86</code></a>
    Add 3.3.9 entry</li>
    <li><a
    href="https://github.com/ruby/rexml/commit/ce59f2eb1aeb371fe1643414f06618dbe031979f"><code>ce59f2e</code></a>
    parser: fix a bug that &amp;#0x...; is accepted as a character
    reference</li>
    <li><a
    href="https://github.com/ruby/rexml/commit/a09646d395a07399cbf9bc3bc8d6d8bb1d13ecea"><code>a09646d</code></a>
    test: fix indent</li>
    <li><a
    href="https://github.com/ruby/rexml/commit/cf0fb9c9ca3dc0d725c8e4644aa0e728025f42ce"><code>cf0fb9c</code></a>
    Fix <code>IOSource#readline</code> for <code>@pending_buffer</code> (<a
    href="https://redirect.github.com/ruby/rexml/issues/215">#215</a>)</li>
    <li><a
    href="https://github.com/ruby/rexml/commit/1d0c362526f6e25e2abcd13e2fcefcc718c20e78"><code>1d0c362</code></a>
    Optimize <code>IOSource#read_until</code> method (<a
    href="https://redirect.github.com/ruby/rexml/issues/210">#210</a>)</li>
    <li><a
    href="https://github.com/ruby/rexml/commit/622011f25ac1519fd553d6c56da52d7eba14a787"><code>622011f</code></a>
    Bump version</li>
    <li>See full diff in <a
    href="https://github.com/ruby/rexml/compare/v3.3.8...v3.3.9">compare
    view</a></li>
    </ul>
    </details>
    <br />
    
    
    [![Dependabot compatibility
    score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=rexml&package-manager=bundler&previous-version=3.3.8&new-version=3.3.9)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
    
    Dependabot will resolve any conflicts with this PR as long as you don't
    alter it yourself. You can also trigger a rebase manually by commenting
    `@dependabot rebase`.
    
    [//]: # (dependabot-automerge-start)
    [//]: # (dependabot-automerge-end)
    
    ---
    
    <details>
    <summary>Dependabot commands and options</summary>
    <br />
    
    You can trigger Dependabot actions by commenting on this PR:
    - `@dependabot rebase` will rebase this PR
    - `@dependabot recreate` will recreate this PR, overwriting any edits
    that have been made to it
    - `@dependabot merge` will merge this PR after your CI passes on it
    - `@dependabot squash and merge` will squash and merge this PR after
    your CI passes on it
    - `@dependabot cancel merge` will cancel a previously requested merge
    and block automerging
    - `@dependabot reopen` will reopen this PR if it is closed
    - `@dependabot close` will close this PR and stop Dependabot recreating
    it. You can achieve the same result by closing it manually
    - `@dependabot show <dependency name> ignore conditions` will show all
    of the ignore conditions of the specified dependency
    - `@dependabot ignore <dependency name> major version` will close this
    group update PR and stop Dependabot creating any more for the specific
    dependency's major version (unless you unignore this specific
    dependency's major version or upgrade to it yourself)
    - `@dependabot ignore <dependency name> minor version` will close this
    group update PR and stop Dependabot creating any more for the specific
    dependency's minor version (unless you unignore this specific
    dependency's minor version or upgrade to it yourself)
    - `@dependabot ignore <dependency name>` will close this group update PR
    and stop Dependabot creating any more for the specific dependency
    (unless you unignore this specific dependency or upgrade to it yourself)
    - `@dependabot unignore <dependency name>` will remove all of the ignore
    conditions of the specified dependency
    - `@dependabot unignore <dependency name> <ignore condition>` will
    remove the ignore condition of the specified dependency and ignore
    conditions
    You can disable automated security fix PRs for this repo from the
    [Security Alerts
    page](https://github.com/google/osv-scanner/network/alerts).
    
    </details>
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    Co-authored-by: Xueqin Cui <[email protected]>
    dependabot[bot] and cuixq authored Oct 30, 2024
    Configuration menu
    Copy the full SHA
    1f69d4a View commit details
    Browse the repository at this point in the history
  3. docs: update usage references (google#1351)

    # PR Summary
    Small PR - adjusts the sources to use the correct `usage.md` file.
    
    Signed-off-by: Emmanuel Ferdman <[email protected]>
    Co-authored-by: Rex P <[email protected]>
    emmanuel-ferdman and another-rex authored Oct 30, 2024
    Configuration menu
    Copy the full SHA
    8af6458 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b13f37e View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. fix: parsing crash on malformed pnpm lockfile (google#1327)

    Scalibr is crashing with a SIGSEGV while trying to parse this directory:
    
    
    https://github.com/semgrep/semgrep/tree/develop/cli/tests/default/e2e/targets/dependency_aware
    
    This is due to the lack of an array length check after the
    dependencyPath split.
    
    Added failing lockfile as a test.
    
    ## Steps to reproduce it
    
    ```
    $ git clone https://github.com/semgrep/semgrep
    $ osv-scanner semgrep/cli/tests/default/e2e/targets/dependency_aware/pnpm-error-key/pnpm-lock.yaml
    
    panic: runtime error: index out of range [0] with length 0
    
    goroutine 1 [running]:
    github.com/google/osv-scanner/pkg/lockfile.extractPnpmPackageNameAndVersion({0xc000328fd0?, 0xb?}, 0x401599999999999a?)
    	/tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:95 +0x34f
    github.com/google/osv-scanner/pkg/lockfile.parsePnpmLock({0xc000814230?, 0xc0004a6fc0?})
    	/tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:141 +0x145
    github.com/google/osv-scanner/pkg/lockfile.PnpmLockExtractor.Extract({}, {0x7f545125f268, 0xc00015de40})
    	/tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:208 +0x25b
    github.com/google/osv-scanner/pkg/lockfile.ExtractDeps({0x7f545125f268, 0xc00015de40}, {0x0, 0x0})
    	/tmp/osv-scanner/pkg/lockfile/extract.go:61 +0x24a
    github.com/google/osv-scanner/pkg/osvscanner.scanLockfile({0x19374a8, 0xc000359e00}, {0xc00037e540, 0x69}, {0x0, 0x0}, 0x0)
    	/tmp/osv-scanner/pkg/osvscanner/osvscanner.go:371 +0x8aa
    github.com/google/osv-scanner/pkg/osvscanner.scanDir.func1({0x7ffd822dfd35?, 0x100?}, {0x1932558, 0xc0004a4b90}, {0x0?, 0x0?})
    	/tmp/osv-scanner/pkg/osvscanner/osvscanner.go:168 +0x745
    path/filepath.walkDir({0x7ffd822dfd35, 0x54}, {0x1932558, 0xc0004a4b90}, 0xc000814970)
    	/usr/lib/golang/src/path/filepath/path.go:443 +0x50
    path/filepath.WalkDir({0x7ffd822dfd35, 0x54}, 0xc000814970)
    	/usr/lib/golang/src/path/filepath/path.go:533 +0x7b
    github.com/google/osv-scanner/pkg/osvscanner.scanDir({0x19374a8?, 0xc000359e00?}, {0x7ffd822dfd35?, 0x54?}, 0x0?, 0x0?, 0x1?, 0x0?)
    	/tmp/osv-scanner/pkg/osvscanner/osvscanner.go:126 +0x212
    github.com/google/osv-scanner/pkg/osvscanner.DoScan({{0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, {0xc0004a4a80, 0x1, 0x1}, {0x0, ...}, ...}, ...)
    	/tmp/osv-scanner/pkg/osvscanner/osvscanner.go:906 +0xf73
    github.com/google/osv-scanner/cmd/osv-scanner/scan.action(0xc000359500, {0x19255e0, 0xc0000f6028}, {0x19255e0, 0xc0000f6030})
    	/tmp/osv-scanner/cmd/osv-scanner/scan/main.go:208 +0xca8
    github.com/google/osv-scanner/cmd/osv-scanner/scan.Command.func2(0xc000815ab8?)
    	/tmp/osv-scanner/cmd/osv-scanner/scan/main.go:145 +0x2c
    github.com/urfave/cli/v2.(*Command).Run(0xc0005d2160, 0xc000359500, {0xc000688080, 0x2, 0x2})
    	GODIR/pkg/mod/github.com/urfave/cli/[email protected]/command.go:276 +0x97d
    github.com/urfave/cli/v2.(*Command).Run(0xc0005d2840, 0xc0003593c0, {0xc0004a6060, 0x3, 0x3})
    	GODIR/pkg/mod/github.com/urfave/cli/[email protected]/command.go:269 +0xbb7
    github.com/urfave/cli/v2.(*App).RunContext(0xc00024c600, {0x1932328, 0x2397500}, {0xc0004a6060, 0x3, 0x3})
    	GODIR/pkg/mod/github.com/urfave/cli/[email protected]/app.go:333 +0x5a5
    github.com/urfave/cli/v2.(*App).Run(...)
    	GODIR/pkg/mod/github.com/urfave/cli/[email protected]/app.go:307
    main.run({0xc0000400a0, 0x2, 0x2}, {0x19255e0, 0xc0000f6028}, {0x19255e0, 0xc0000f6030})
    	/tmp/osv-scanner/cmd/osv-scanner/main.go:52 +0x678
    main.main()
    	/tmp/osv-scanner/cmd/osv-scanner/main.go:126 +0x45
    ```
    
    ---------
    
    Co-authored-by: Gareth Jones <[email protected]>
    Co-authored-by: Xueqin Cui <[email protected]>
    3 people authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    bef97ac View commit details
    Browse the repository at this point in the history
  2. refactor(semantic): sort ecosystems by name (google#1363)

    Having ecosystems sorted by their name makes it easier to review this
    section of code
    G-Rath authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    fce42e1 View commit details
    Browse the repository at this point in the history
  3. fix(semantic): support parsing versions without a numeric component (g…

    …oogle#1365)
    
    While I'm pretty these are technically invalid, they're easy to support
    and without this Alpine is the only comparator that panics when parsing
    an empty string which I think is a little sad.
    G-Rath authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    be307de View commit details
    Browse the repository at this point in the history
  4. refactor(semantic): remove unneeded logic in parsing semver-like vers…

    …ions (google#1360)
    
    While I'm a little reluctant to do this as I'm sure I included these for
    a reason, they're apparently not covered by any tests and with
    `semantic` expected to go public soon, I think it's a bit nicer to have
    them gone until someone can prove they're needed as some of these could
    arguably be a breaking change to remove.
    
    I'm pretty sure the bulk of this was present as part of having
    `semantic` rebuild the parsed version for debugging when I was writing
    the implementation, but that's not actually a feature so we don't
    explicitly need to be doing it - this won't stop us from reintroducing
    the logic in future if we decide we want it
    G-Rath authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    00cdb36 View commit details
    Browse the repository at this point in the history
  5. refactor(semantic): simplify comparing of RubyGem version components (g…

    …oogle#1361)
    
    I realised that I have overly complicated the act of returning the
    result of comparing components in this function, and that the last two
    conditions will never be true as we loop over the largest number of
    components 😅
    G-Rath authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    cc702c8 View commit details
    Browse the repository at this point in the history
  6. refactor(semantic): remove unneeded condition in PyPI version compara…

    …tor (google#1362)
    
    Currently this return is not covered because we're explicitly doing all
    three possible comparisons, so we might as well move the equality check
    to the end as the default
    G-Rath authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    ffd2eb2 View commit details
    Browse the repository at this point in the history
  7. refactor(semantic): simplify comparing of "pre" letters in PyPI versi…

    …ons (google#1366)
    
    These letters naturally compare in the right order, so we can just
    compare them directly which saves us a loop and an untestable `panic`.
    
    (this will conflict with google#1362)
    G-Rath authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    998461f View commit details
    Browse the repository at this point in the history
  8. test: update snapshots (google#1368)

    🤖
    G-Rath authored Oct 31, 2024
    Configuration menu
    Copy the full SHA
    94c12b5 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2024

  1. chore: remove deprecated internal functions (google#1369)

    There's no reason for us to retain these in the internal packages, so
    they can just go
    
    Co-authored-by: Rex P <[email protected]>
    G-Rath and another-rex authored Nov 1, 2024
    Configuration menu
    Copy the full SHA
    11600e7 View commit details
    Browse the repository at this point in the history
  2. refactor: rename internal struct to avoid stuttering (google#1370)

    Since this is an internal package it's not a breaking change to rename
    this
    
    Co-authored-by: Rex P <[email protected]>
    G-Rath and another-rex authored Nov 1, 2024
    Configuration menu
    Copy the full SHA
    c20dd9f View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2024

  1. test(semantic): include ecosystems not supported by lockfile (googl…

    …e#1364)
    
    Notably this adds coverage over the Ubuntu entry, since that reuses
    Debian so we don't have any explicit tests for that in the comparison
    tests.
    
    We also no longer need to be manually adding `CRAN` since `lockfile`
    supports it
    G-Rath authored Nov 4, 2024
    Configuration menu
    Copy the full SHA
    f9ac170 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8c1beae View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c3a61d2 View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2024

  1. Configuration menu
    Copy the full SHA
    8509b99 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0cc32c7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    99e2974 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    a919048 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    7ac9097 View commit details
    Browse the repository at this point in the history