Skip to content

Commit

Permalink
[Alienvault, CrowdStrike, Phishunt, ThreatFox, URLHaus] added the abi…
Browse files Browse the repository at this point in the history 
…lity to set x_opencti_score for select connectors (OpenCTI-Platform#2554)
  • Loading branch information
@brett-fitz
brett-fitz authored Sep 27, 2024
1 parent df65892 commit 71c398a
Show file tree
Hide file tree
Showing 27 changed files with 506 additions and 97 deletions.
42 changes: 25 additions & 17 deletions external-import/alienvault/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,28 @@ Below are the parameters you'll need to set for running the connector properly:

Below are the parameters you'll need to set for AlienVault connector:

| Parameter `AlienVault` | config.yml | Docker environment variable | Default | Mandatory | Description |
|----------------------------------|------------------------------------|-----------------------------------------------|-------------------------------|-----------|--------------------------------------------------------------------------------------------------------------------------------|
| Base Url | `base_url` | `ALIENVAULT_BASE_URL` | `https://otx.alienvault.com` | Yes | The base URL for the OTX DirectConnect API. |
| Api Key | `api_key` | `ALIENVAULT_API_KEY` | `ChangeMe` | No | The OTX Key. |
| TLP | `tlp` | `ALIENVAULT_TLP` | `White` | Yes | The default TLP marking used if the Pulse does not define TLP. |
| Create Observables | `create_observables` | `ALIENVAULT_CREATE_OBSERVABLES` | `True` | No | If true then observables will be created from Pulse indicators and added to the report. |
| Create Indicators | `create_indicators` | `ALIENVAULT_CREATE_INDICATORS` | `True` | No | If true then indicators will be created from Pulse indicators and added to the report. |
| Pulse Start Timestamp | `pulse_start_timestamp` | `ALIENVAULT_PULSE_START_TIMESTAMP` | `2020-05-01T00:00:00` | Yes | The Pulses modified after this timestamp will be imported. Timestamp in ISO 8601 format, UTC. |
| Report Status | `report_status` | `ALIENVAULT_REPORT_STATUS` | `New` | Yes | The status of imported reports in the OpenCTI. |
| Report Type | `report_type` | `ALIENVAULT_REPORT_TYPE` | `threat-report` | No | The type of imported reports in the OpenCTI. |
| Guess Malware | `guess_malware` | `ALIENVAULT_GUESS_MALWARE` | `False` | Yes | The Pulse tags are used to guess (queries malwares in the OpenCTI) malwares related to the given Pulse. |
| Guess CVE | `guess_cve` | `ALIENVAULT_GUESS_CVE` | `False` | Yes | The Pulse tags are used to guess (checks whether tag matches (CVE-\d{4}-\d{4,7})) vulnerabilities. |
| Excluded Pulse Indicator Types | `excluded_pulse_indicator_types` | `ALIENVAULT_EXCLUDED_PULSE_INDICATOR_TYPES` | `FileHash-MD5,FileHash-SHA1` | Yes | The Pulse indicator types that will be excluded from the import. |
| Enable Relationships | `enable_relationships` | `ALIENVAULT_ENABLE_RELATIONSHIPS` | `True` | No | If true then the relationships will be created between SDOs. |
| Enable Attack Patterns Indicates | `enable_attack_patterns_indicates` | `ALIENVAULT_ENABLE_ATTACK_PATTERNS_INDICATES` | `True` | No | If true then the relationships `indicates` will be created between indicators and attack patterns. |
| Filter Indicators | `filter_indicators` | `ALIENVAULT_FILTER_INDICATORS` | `True` | No | This boolean filters out indicators created before the latest pulse datetime, ensuring only recent indicators are processed. |

| Parameter `AlienVault` | config.yml | Docker environment variable | Default | Mandatory | Description |
|---------------------------------------|------------------------------------|-----------------------------------------------|-------------------------------|-----------|--------------------------------------------------------------------------------------------------------------------------------|
| Base Url | `base_url` | `ALIENVAULT_BASE_URL` | `https://otx.alienvault.com` | Yes | The base URL for the OTX DirectConnect API. |
| Api Key | `api_key` | `ALIENVAULT_API_KEY` | `ChangeMe` | No | The OTX Key. |
| TLP | `tlp` | `ALIENVAULT_TLP` | `White` | Yes | The default TLP marking used if the Pulse does not define TLP. |
| Create Observables | `create_observables` | `ALIENVAULT_CREATE_OBSERVABLES` | `True` | No | If true then observables will be created from Pulse indicators and added to the report. |
| Create Indicators | `create_indicators` | `ALIENVAULT_CREATE_INDICATORS` | `True` | No | If true then indicators will be created from Pulse indicators and added to the report. |
| Pulse Start Timestamp | `pulse_start_timestamp` | `ALIENVAULT_PULSE_START_TIMESTAMP` | `2020-05-01T00:00:00` | Yes | The Pulses modified after this timestamp will be imported. Timestamp in ISO 8601 format, UTC. |
| Report Status | `report_status` | `ALIENVAULT_REPORT_STATUS` | `New` | Yes | The status of imported reports in the OpenCTI. |
| Report Type | `report_type` | `ALIENVAULT_REPORT_TYPE` | `threat-report` | No | The type of imported reports in the OpenCTI. |
| Guess Malware | `guess_malware` | `ALIENVAULT_GUESS_MALWARE` | `False` | Yes | The Pulse tags are used to guess (queries malwares in the OpenCTI) malwares related to the given Pulse. |
| Guess CVE | `guess_cve` | `ALIENVAULT_GUESS_CVE` | `False` | Yes | The Pulse tags are used to guess (checks whether tag matches (CVE-\d{4}-\d{4,7})) vulnerabilities. |
| Excluded Pulse Indicator Types | `excluded_pulse_indicator_types` | `ALIENVAULT_EXCLUDED_PULSE_INDICATOR_TYPES` | `FileHash-MD5,FileHash-SHA1` | Yes | The Pulse indicator types that will be excluded from the import. |
| Enable Relationships | `enable_relationships` | `ALIENVAULT_ENABLE_RELATIONSHIPS` | `True` | No | If true then the relationships will be created between SDOs. |
| Enable Attack Patterns Indicates | `enable_attack_patterns_indicates` | `ALIENVAULT_ENABLE_ATTACK_PATTERNS_INDICATES` | `True` | No | If true then the relationships `indicates` will be created between indicators and attack patterns. |
| Filter Indicators | `filter_indicators` | `ALIENVAULT_FILTER_INDICATORS` | `True` | No | This boolean filters out indicators created before the latest pulse datetime, ensuring only recent indicators are processed. |
| default_x_opencti_score | `default_x_opencti_score` | `ALIENVAULT_DEFAULT_X_OPENCTI_SCORE` | `50` | No | The default x_opencti_score to use for indicators. If a per indicator type score is not set, this is used. |
| x_opencti_score_ip | `x_opencti_score_ip` | `ALIENVAULT_X_OPENCTI_SCORE_IP` | `50` | No | The x_opencti_score to use for IP indicators. If not set, the default value is `default_x_opencti_score`. |
| x_opencti_score_domain | `x_opencti_score_domain` | `ALIENVAULT_X_OPENCTI_SCORE_DOMAIN` | `50` | No | The x_opencti_score to use for Domain indicators. If not set, the default value is `default_x_opencti_score`. |
| x_opencti_score_hostname | `x_opencti_score_hostname` | `ALIENVAULT_X_OPENCTI_SCORE_HOSTNAME` | `50` | No | The x_opencti_score to use for Hostname indicators. If not set, the default value is `default_x_opencti_score`. |
| x_opencti_score_email | `x_opencti_score_email` | `ALIENVAULT_X_OPENCTI_SCORE_EMAIL` | `50` | No | The x_opencti_score to use for Email indicators. If not set, the default value is `default_x_opencti_score`. |
| x_opencti_score_file | `x_opencti_score_file` | `ALIENVAULT_X_OPENCTI_SCORE_FILE` | `50` | No | The x_opencti_score to use for StixFile indicators. If not set, the default value is `default_x_opencti_score`. |
| x_opencti_score_url | `x_opencti_score_url` | `ALIENVAULT_X_OPENCTI_SCORE_URL` | `50` | No | The x_opencti_score to use for URL indicators. If not set, the default value is `default_x_opencti_score`. |
| x_opencti_score_mutex | `x_opencti_score_mutex` | `ALIENVAULT_X_OPENCTI_SCORE_MUTEX` | `50` | No | The x_opencti_score to use for Mutex indicators. If not set, the default value is `default_x_opencti_score`. |
| x_opencti_score_cryptocurrency_wallet | `x_opencti_score_cryptocurrency_wallet` | `ALIENVAULT_X_OPENCTI_SCORE_CRYPTOCURRENCY_WALLET` | `50` | No | The x_opencti_score to use for Cryptocurrency Wallet indicators. If not set, the default value is `default_x_opencti_score`. |
10 changes: 10 additions & 0 deletions external-import/alienvault/docker-compose.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,14 @@ services:
- ALIENVAULT_EXCLUDED_PULSE_INDICATOR_TYPES=FileHash-MD5,FileHash-SHA1 # Excluded Pulse indicator types.
- ALIENVAULT_ENABLE_RELATIONSHIPS=true # Enable/Disable relationship creation between SDOs.
- ALIENVAULT_ENABLE_ATTACK_PATTERNS_INDICATES=false # Enable/Disable "indicates" relationships between indicators and attack patterns
- ALIENVAULT_INTERVAL_SEC=1800
- ALIENVAULT_DEFAULT_X_OPENCTI_SCORE=50
- ALIENVAULT_X_OPENCTI_SCORE_IP=60
- ALIENVAULT_X_OPENCTI_SCORE_DOMAIN=70
- ALIENVAULT_X_OPENCTI_SCORE_HOSTNAME=75
- ALIENVAULT_X_OPENCTI_SCORE_EMAIL=70
- ALIENVAULT_X_OPENCTI_SCORE_FILE=85
- ALIENVAULT_X_OPENCTI_SCORE_URL=80
- ALIENVAULT_X_OPENCTI_SCORE_MUTEX=60
- ALIENVAULT_X_OPENCTI_SCORE_CRYPTOCURRENCY_WALLET=80
restart: always
46 changes: 43 additions & 3 deletions external-import/alienvault/src/alienvault/builder.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,15 @@ class PulseBundleBuilderConfig(NamedTuple):
excluded_pulse_indicator_types: Set[str]
enable_relationships: bool
enable_attack_patterns_indicates: bool
x_opencti_score: int
x_opencti_score_ip: int
x_opencti_score_domain: int
x_opencti_score_hostname: int
x_opencti_score_email: int
x_opencti_score_file: int
x_opencti_score_url: int
x_opencti_score_mutex: int
x_opencti_score_cryptocurrency_wallet: int


class PulseBundleBuilder:
Expand Down Expand Up @@ -136,6 +145,31 @@ def __init__(
self.excluded_pulse_indicator_types = config.excluded_pulse_indicator_types
self.enable_relationships = config.enable_relationships
self.enable_attack_patterns_indicates = config.enable_attack_patterns_indicates
self.x_opencti_score = {
"default": config.x_opencti_score,
"IPv4": config.x_opencti_score_ip,
"IPv4-Addr": config.x_opencti_score_ip,
"IPv6": config.x_opencti_score_ip,
"IPv6-Addr": config.x_opencti_score_ip,
"CIDR": config.x_opencti_score_ip,
"domain": config.x_opencti_score_domain,
"Domain-Name": config.x_opencti_score_domain,
"hostname": config.x_opencti_score_hostname,
"Hostname": config.x_opencti_score_hostname,
"email": config.x_opencti_score_email,
"Email-addr": config.x_opencti_score_email,
"FilePath": config.x_opencti_score_file,
"FileHash-MD5": config.x_opencti_score_file,
"FileHash-SHA1": config.x_opencti_score_file,
"FileHash-SHA256": config.x_opencti_score_file,
"StixFile": config.x_opencti_score_file,
"URL": config.x_opencti_score_url,
"URI": config.x_opencti_score_url,
"Url": config.x_opencti_score_url,
"Mutex": config.x_opencti_score_mutex,
"BitcoinAddress": config.x_opencti_score_cryptocurrency_wallet,
"Cryptocurrency-Wallet": config.x_opencti_score_cryptocurrency_wallet,
}

def _no_relationships(self) -> bool:
return not self.enable_relationships
Expand Down Expand Up @@ -369,7 +403,9 @@ def _create_observations(

if self.create_observables:
observable_properties = self._create_observable_properties(
pulse_indicator_value, labels
value=pulse_indicator_value,
labels=labels,
x_opencti_score=self.x_opencti_score.get(pulse_indicator_type),
)
observable = factory.create_observable(observable_properties)

Expand Down Expand Up @@ -430,10 +466,10 @@ def _exclude_pulse_indicator_types_filter(
)

def _create_observable_properties(
self, value: str, labels: List[str]
self, value: str, labels: List[str], x_opencti_score: int
) -> ObservableProperties:
return ObservableProperties(
value, self.pulse_author, labels, self.object_markings
value, self.pulse_author, labels, self.object_markings, x_opencti_score
)

def _create_indicator(
Expand All @@ -456,6 +492,10 @@ def _create_indicator(
labels=labels,
confidence=self.confidence_level,
object_markings=self.object_markings,
x_opencti_score=(
self.x_opencti_score.get(main_observable_type)
or self.x_opencti_score.get("default")
),
x_opencti_main_observable_type=main_observable_type,
)

Expand Down
Loading

0 comments on commit 71c398a

Please sign in to comment.