Update Gitleaks to 8.18.1 and custom rules

[markdboyd]

Changes proposed in this pull request:

caulking was flagging lines such as this as secrets because of the word keyword:

https://github.com/cloud-gov/logsearch-for-cloudfoundry/blob/develop/jobs/upload-kibana-objects/templates/kibana-objects/index-pattern/logs.json.erb#L10

I began exploring how to update our local.toml to ignore these false positives. Initially, I planned to leverage the regexTarget: "line feature added in v8.16.0 to ignore any lines with "type":"keyword", so I updated to Gitleaks v8.18.1.

But I was having issues with this repo's tests after upgrading to 8.18.1. It turns out that you need an id for every rule and that without that property, you get unexpected results.

I was adding id attributes to all of our rules when I also discovered that you can have your custom config extend the default config that comes with gitleaks. By doing this, we can eliminate all of the rules from gitleaks that we copied into our custom config as of gitleaks version 4.1.1 and instead just inherit them from gitleaks core. We also benefit from new rules that have been added to gitleaks core.

Specifically, I removed the Generic Credential rule that now exists as generic-api-key in gitleaks core. Removing this rule and relying on the defaults from gitleaks resolved my initial issues.

I did preserve some of our custom rules that didn't seem like they are covered in the upstream gitleaks core.

• update to Gitleaks version 8.18.1
• update test expectation for plain file to match new output from Gitleaks
• update config to extend default gitleaks config. This means caulking will inherit all of the rules that are maintained upstream in gitleaks (https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml), rather than us having to manually copy in those rules
• remove non-custom rules maintained upstream
• re-add custom generic username rule
• re-add allowlist rules
• fix allow lists
• update email rule
• add rule for suspicious filenames
• add suspicious filenames rule
• re-add generic credential rule

security considerations

This is a pretty significant refactor of our configuration, but with definite security benefits:

• no need to separately maintain rules from upstream gitleaks core
• less custom configuration to maintain
• getting more/new rules from upstream gitleaks core

make audit is passing with these changes, demonstrating that at least as far the tests demonstrate, these changes provide the same security guarantees.

[pburkholder]

This looks awesome.

I was able to update to this version with:

git checkout update-gitleaks-version-config
make clean
make install

I think only make clean install will be needed by our team to update once we merge.

The make audit command works as expected (it fails #14 because I'm not on main and the latest commit, so that's expected.)

However, bats development.bats fails 9 of the 33 tests. I haven't dug into why yet. We'll need to clean those up before merging. A great start though!

[pburkholder]

Passes local tests for me.