[cscli] We added patterns_dir option to config.yaml but isnt used in hubtest

[LaurenceJJones]

What happened?

When setting a custom patterns_dir in config.yaml and using cscli hubtest run --all the patterns dir is not used

Running test 'zoneminder-bf' failed: unable to copy 'patterns' from '/etc/crowdsec/patterns'to '/tmp/crowdsec-v1.6.2/hub/.tests/zoneminder-bf/runtime/patterns': open /etc/crowdsec/patterns: no such file or directory

config.yaml

config_paths:
  config_dir: ./config
  data_dir: ./data/
  notification_dir: ./config/notifications/
  plugin_dir: ./plugins/
  pattern_dir: ./config/patterns/

This is used within the test_env.sh script to not need crowdsec to be installed.

What did you expect to happen?

The pattern_dir options to be respected

How can we reproduce it (as minimally and precisely as possible)?

Prep the test folder:

cd $(mktemp -d)
wget https://github.com/crowdsecurity/crowdsec/releases/download/v1.6.2/crowdsec-release.tgz
tar xvzf crowdsec-release.tgz
cd crowdsec-v1.6.2
./test_env.sh
alias csdev="$PWD/cmd/crowdsec-cli/cscli -c $PWD/config/dev.yaml"
git clone https://github.com/crowdsecurity/hub

Edit config/dev.yaml and add the following:

config_paths:
  pattern_dir: ./config/patterns/

Anything else we need to know?

No response

Crowdsec version

$ cscli version
# paste output here

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Enabled collections and parsers

$ cscli hub list -o raw
# paste output here

Acquisition config

```console # On Linux: $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* # paste output here

On Windows:

C:> Get-Content C:\ProgramData\CrowdSec\config\acquis.yaml

paste output here

Config show

$ cscli config show
# paste output here

Prometheus metrics

$ cscli metrics
# paste output here

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

[github-actions]

@LaurenceJJones: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[simonmicro]

@buixor Please reconsider this value assignment. If anybody plans to follow https://docs.crowdsec.net/docs/parsers/create and create their own parser, they will end up frustrated here. Alternatively, as IIRC you need to run hubtest for local development, is there any workaround known?

[LaurenceJJones]

To aid with debugging it these codes right here:

crowdsec/pkg/hubtest/hubtest_item.go

Lines 598 to 603 in d8bc17b

	crowdsecPatternsFolder := csconfig.DefaultConfigPath("patterns")
	// copy template patterns folder to runtime folder
	if err = CopyDir(crowdsecPatternsFolder, t.RuntimePatternsPath); err != nil {
	return fmt.Errorf("unable to copy 'patterns' from '%s' to '%s': %w", crowdsecPatternsFolder, t.RuntimePatternsPath, err)
	}

crowdsec/pkg/hubtest/hubtest_item.go

Lines 419 to 424 in d8bc17b

	crowdsecPatternsFolder := csconfig.DefaultConfigPath("patterns")
	// copy template patterns folder to runtime folder
	if err = CopyDir(crowdsecPatternsFolder, t.RuntimePatternsPath); err != nil {
	return fmt.Errorf("unable to copy 'patterns' from '%s' to '%s': %w", crowdsecPatternsFolder, t.RuntimePatternsPath, err)
	}

It only get the default values and doesnt look at the parsed configuration.