Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ QUESTION ] Get-FalconAlert -Detailed -All produces 500: Internal Server Error at 10,000 results #423

Open
datorr2 opened this issue Sep 10, 2024 · 9 comments
Assignees
Labels
question Further information is requested

Comments

@datorr2
Copy link
Contributor

datorr2 commented Sep 10, 2024

Describe the bug
When using PSFalcon 2.2.7, Get-FalconAlert with parameter -All results in an HTTP 500 response.

To Reproduce
Get-FalconAlert -All

Expected behavior
API endpoint I believe has a limit of 1000 items, so would expect the function to get all Falcon Alerts in pages of 1000 and combine them into a single collection to return.

Environment (please complete the following information):

  • PSVersion: 7.4.5
  • PSEdition: Core
  • GitCommitId: 7.4.5
  • OS: Microsoft Windows 10.0.19045
  • Platform: Win32NT

Transcript content

**********************
PowerShell transcript start
Start time: 20240910125028
Username: [REDACTED]
RunAs User: [REDACTED]
Configuration Name: 
Machine: [REDACTED] (Microsoft Windows NT 10.0.19045.0)
Host Application: C:\Program Files\PowerShell\7\pwsh.dll -NoP
Process ID: 22864
PSVersion: 7.4.5
PSEdition: Core
GitCommitId: 7.4.5
OS: Microsoft Windows 10.0.19045
Platform: Win32NT
PSCompatibleVersions: 1.0, 2.0, 3.0, 4.0, 5.0, 5.1, 6.0, 7.0
PSRemotingProtocolVersion: 2.3
SerializationVersion: 1.1.0.1
WSManStackVersion: 3.0
**********************
Transcript started, output file is C:\Temp\PSFalcon.txt
PS C:\> Import-Module PSFalcon -RequiredVersion 2.2.7
PS C:\> Request-FalconToken [REDACTED]
PS C:\> Test-FalconToken | Format-Table -Property Token,Hostname

Token Hostname
----- --------
 True https://api.laggar.gcw.crowdstrike.com

PS C:\> $Alerts = Get-FalconAlert
PS C:\> $Alerts.Count
100
PS C:\> $Alerts = Get-FalconAlert -All
Write-Result: C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.7\private\Private.ps1:687
Line |
 687 |          Write-Result $Object
     |          ~~~~~~~~~~~~~~~~~~~~
     | {"code":500,"message":"Internal Server Error: Please provide trace-id='54bfe1d2-f4b3-410c-b89f-d6bbc3e45aad' to support"}
Write-Result: C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.7\private\Private.ps1:687
Line |
 687 |          Write-Result $Object
     |          ~~~~~~~~~~~~~~~~~~~~
     | {"code":500,"message":"Internal Server Error: Please provide trace-id='54bfe1d2-f4b3-410c-b89f-d6bbc3e45aad' to support"}

PS C:\> $Alerts = Get-FalconAlert -Detailed
PS C:\> $Alerts.Count
100
PS C:\> $Alerts = Get-FalconAlert -All -Detailed
Write-Result: C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.7\private\Private.ps1:684
Line |
 684 |          $Output = Write-Result $Object
     |                    ~~~~~~~~~~~~~~~~~~~~
     | {"code":500,"message":"Internal Server Error: Please provide trace-id='d02b8340-fc83-4f77-9965-3a6a2378a96c' to support"}
Write-Result: C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.7\private\Private.ps1:684
Line |
 684 |          $Output = Write-Result $Object
     |                    ~~~~~~~~~~~~~~~~~~~~
     | {"code":500,"message":"Internal Server Error: Please provide trace-id='d02b8340-fc83-4f77-9965-3a6a2378a96c' to support"}

PS C:\> Remove-Module PSFalcon
PS C:\> Import-Module PSFalcon -RequiredVersion 2.2.6
PS C:\> Request-FalconToken [REDACTED]
PS C:\> Test-FalconToken | Format-Table -Property Token,Hostname

Token Hostname
----- --------
 True https://api.laggar.gcw.crowdstrike.com

PS C:\> $Alerts = Get-FalconAlert
PS C:\> $Alerts.Count
100
PS C:\> $Alerts = Get-FalconAlert -All
PS C:\> $Alerts.Count
8732
PS C:\> $Alerts = Get-FalconAlert -Detailed
PS C:\> $Alerts.Count
100
PS C:\> $Alerts = Get-FalconAlert -All -Detailed
Write-Result: C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.6\private\Private.ps1:663
Line |
 663 |          Write-Result $Object
     |          ~~~~~~~~~~~~~~~~~~~~
     | {"code":413,"message":"request too large"}
Write-Result: C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.6\private\Private.ps1:663
Line |
 663 |          Write-Result $Object
     |          ~~~~~~~~~~~~~~~~~~~~
     | {"code":413,"message":"request too large"}

PS C:\> # Gather Alerts
$FQL = "data_domains:'Endpoint'"
$Alerts = [System.Collections.ArrayList]@()
$Total = (Get-FalconAlert -All -Filter $FQL).Count
$Counter = 0
while ($Counter -lt $Total) {
  $Response = Get-FalconAlert -Detailed -Limit 1000 -Filter $FQL -Offset $Counter
  try {
    $Alerts.AddRange($Response)
  } catch [System.Management.Automation.PSInvalidCastException] {
    $null = $Alerts.Add($Response)
  } catch [System.ArgumentNullException] {}
  $Counter += $Response.Count
}
"Gathered {0} Alerts" -f $Alerts.Count | Out-Host
Gathered 1385 Alerts
PS C:\> $Alerts.Count
1385
PS C:\> Stop-Transcript
**********************
PowerShell transcript end
End time: 20240910125617
**********************

I can provide an unabridged transcript complete with $VerbosePreference = 'Continue' directly to you if you require.

@datorr2 datorr2 added the bug Something isn't working label Sep 10, 2024
@bk-cs
Copy link
Collaborator

bk-cs commented Sep 11, 2024

Can you re-run your test with $VerbosePreference=2?

Get-FalconAlert is automatically grouping 1,000 composite_ids values per request in my installation of v2.2.7. If you don't see this line in your verbose output...

[Get-ParamSet] Creating groups of 1000 'composite_ids' values

Try re-installing your module:

Uninstall-Module -Name PSFalcon -AllVersions
Install-Module -Name PSFalcon -Scope CurrentUser

If it is showing up, there might be a difference between the maximum number of composite_ids values allowed by POST /alerts/entities/alerts/v2 in your cloud.

@bk-cs bk-cs changed the title [ BUG ] Get-FalconAlert -All broken in 2.2.7 [ BUG ] Get-FalconAlert -All produces 500: Internal Server Error Sep 11, 2024
@bk-cs bk-cs changed the title [ BUG ] Get-FalconAlert -All produces 500: Internal Server Error [ BUG ] Get-FalconAlert -Detailed -All produces 500: Internal Server Error Sep 11, 2024
@datorr2
Copy link
Contributor Author

datorr2 commented Sep 11, 2024

I removed all versions of PSFalcon and reinstalled only 2.2.7 and then tried:

$Alerts = Get-FalconAlert -All -Detailed -Verbose
VERBOSE: 13:16:57 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 13:16:57 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=10000
VERBOSE: 13:16:57 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:16:59 [ApiClient.Invoke] 200: OK
VERBOSE: 13:16:59 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 17:16:59 GMT, Transfer-Encoding=chunked, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=c6925450-f92c-46dc-8aef-7f7c3ba92576, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5999
VERBOSE: 13:16:59 [Write-Result] query_time=0.776686841, pagination.offset=0, pagination.limit=10000, pagination.total=179635, writes=, powered_by=detectsapi, trace_id=c6925450-f92c-46dc-8aef-7f7c3ba92576
VERBOSE: 13:16:59 [Get-ParamSet] Creating groups of 1000 'composite_ids' values
VERBOSE: 13:16:59 [Get-FalconAlert] /alerts/entities/alerts/v2:post
VERBOSE: 13:16:59 [ApiClient.Invoke] POST https://api.laggar.gcw.crowdstrike.com/alerts/entities/alerts/v2
VERBOSE: 13:16:59 [ApiClient.Invoke] ContentType=application/json, Accept=application/json
VERBOSE: 13:16:59 [ApiClient.Invoke] {"composite_ids":[ << 1000 composite_ids >> ]}
VERBOSE: 13:19:41 [ApiClient.Invoke] 200: OK
VERBOSE: 13:19:41 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 17:19:41 GMT, Transfer-Encoding=chunked, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=777ece9a-9416-40de-9f33-17c39e408b7f, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5989
VERBOSE: 13:19:43 [Write-Result] query_time=1.011670096, writes=, powered_by=detectsapi, trace_id=777ece9a-9416-40de-9f33-17c39e408b7f
VERBOSE: 13:19:43 [Get-FalconAlert] Retrieved 10000 of 179636
VERBOSE: 13:19:43 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=10000&offset=10000
VERBOSE: 13:19:43 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:19:43 [ApiClient.Invoke] 500: InternalServerError
VERBOSE: 13:19:43 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 17:19:43 GMT, Connection=keep-alive, X-Content-Type-Options=nosniff, X-Cs-Traceid=f07a1d2e-0ac7-41b7-b790-ea36ce1d7a7c, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5988, Strict-Transport-Security=max-age=31536000; includeSubDomains
VERBOSE: 13:19:43 [Write-Result] query_time=1.62E-07, powered_by=crowdstrike-api-gateway, trace_id=f07a1d2e-0ac7-41b7-b790-ea36ce1d7a7c
Write-Result: C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.7\private\Private.ps1:684
Line |
 684 |          $Output = Write-Result $Object
     |                    ~~~~~~~~~~~~~~~~~~~~
     | {"code":500,"message":"Internal Server Error: Please provide trace-id='f07a1d2e-0ac7-41b7-b790-ea36ce1d7a7c' to support"}

The 500 may be because it reached the end of 10000, but there's 161,000 alerts.

I tried limiting the number of alerts using FQL, and it still says the pagination.total is 161,000, which doesn't seem right to me, since the total should be less than 10000.

@bk-cs
Copy link
Collaborator

bk-cs commented Sep 11, 2024

Like most of the Falcon APIs, GET /alerts/queries/alerts/v2 has a maximum limit of 10,000 results, so unfortunately, the 500 error is expected. There should be a more descriptive error produced, but that's up to the API team.

The way around this is to break your results into filtered searches of less than 10,000 results (i.e. ~16 groups of searches) and then use -All to retrieve all of those groups of results.

@bk-cs bk-cs added question Further information is requested and removed bug Something isn't working labels Sep 11, 2024
@bk-cs bk-cs changed the title [ BUG ] Get-FalconAlert -Detailed -All produces 500: Internal Server Error [ QUESTION ] Get-FalconAlert -Detailed -All produces 500: Internal Server Error at 10,000 results Sep 11, 2024
@datorr2
Copy link
Contributor Author

datorr2 commented Sep 11, 2024

The odd thing is, if querying only Endpoint alerts, there shouldn't be 160k alerts.

With 2.2.6, if I specify data_domains:'Endpoint' in an FQL -Filter, I only get ~1400 alerts:

PS> # Gather Alerts
>> $FQL = "data_domains:'Endpoint'"
>> $Alerts = [System.Collections.ArrayList]@()
>> $Total = (Get-FalconAlert -All -Filter $FQL).Count
>> $Counter = 0
>> while ($Counter -lt $Total) {
>>   $Response = Get-FalconAlert -Detailed -Limit 1000 -Filter $FQL -Offset $Counter
>>   try {
>>     $Alerts.AddRange($Response)
>>   } catch [System.Management.Automation.PSInvalidCastException] {
>>     $null = $Alerts.Add($Response)
>>   } catch [System.ArgumentNullException] {}
>>   $Counter += $Response.Count
>> }
>> "Gathered {0} Alerts" -f $Alerts.Count | Out-Host
Gathered 1435 Alerts
PS> $Alerts.Count
1435

With 2.2.7, it doesn't seem to filter correctly:

$FQL = "data_domains:'Endpoint'"
>> $Alerts = Get-FalconAlert -All -Filter $FQL -Verbose
VERBOSE: 16:00:40 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 16:00:40 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=10000&filter=data_domains%3A'Endpoint'
VERBOSE: 16:00:40 [ApiClient.Invoke] Accept=application/json
VERBOSE: 16:00:41 [ApiClient.Invoke] 200: OK
VERBOSE: 16:00:41 [ApiClient.Invoke] Transfer-Encoding=chunked, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=779bc051-706b-48e6-9d25-cdaeb4a2d649, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5998,
Date=Wed, 11 Sep 2024 20:00:41 GMT, Server=nginx
VERBOSE: 16:00:41 [Write-Result] query_time=0.797458313, pagination.offset=0, pagination.limit=10000, pagination.total=172250, writes=, powered_by=detectsapi, trace_id=779bc051-706b-48e6-9d25-cdaeb4a2d649
VERBOSE: 16:00:41 [Get-FalconAlert] Retrieved 10000 of 172250
VERBOSE: 16:00:41 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=10000&filter=data_domains%3A'Endpoint'&offset=10000
VERBOSE: 16:00:41 [ApiClient.Invoke] Accept=application/json
VERBOSE: 16:00:42 [ApiClient.Invoke] 500: InternalServerError
VERBOSE: 16:00:42 [ApiClient.Invoke] Connection=keep-alive, X-Content-Type-Options=nosniff, X-Cs-Traceid=9b820f69-91a9-4f68-a3e2-d08dfaf8e58b, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5997, Strict-Transport-Security=max-age=31536000; includeSubDomains, Date=Wed, 11 Sep 2024 20:00:42 GMT, Server=nginx
VERBOSE: 16:00:42 [Write-Result] query_time=1.33E-07, powered_by=crowdstrike-api-gateway, trace_id=9b820f69-91a9-4f68-a3e2-d08dfaf8e58b
Write-Result : {"code":500,"message":"Internal Server Error: Please provide trace-id=\u00279b820f69-91a9-4f68-a3e2-d08dfaf8e58b\u0027 to support"}
At C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.7\private\Private.ps1:687 char:9
+         Write-Result $Object
+         ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidResult: (@{resources=System.Object[]}:PSObject) [Write-Result], Exception
    + FullyQualifiedErrorId : Write-Result

Could this be a bug with the API itself?

@bk-cs
Copy link
Collaborator

bk-cs commented Sep 11, 2024

I can't reproduce that in either US-1 or US-GOV-1. Both seem to be passing my filter properly. I thought maybe it was a PowerShell difference, but it works in both 5.1 and 7.4.5.

US-1

PS C:\Users\brend> Get-FalconAlert -Total
VERBOSE: 13:40:23 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 13:40:23 [ApiClient.Invoke] GET https://api.crowdstrike.com/alerts/queries/alerts/v2
VERBOSE: 13:40:23 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:40:23 [ApiClient.Invoke] 200: OK
VERBOSE: 13:40:23 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:40:24 GMT, Transfer-Encoding=chunked, Connection=keep-alive, Strict-Transport-Security=max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-1, X-Cs-Traceid=bdb1392e-8fdf-4629-b56a-f3112c3a3dfd, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5962
61009
PS C:\Users\brend> Get-FalconAlert -Filter "data_domains:'Endpoint'" -Total
VERBOSE: 13:40:32 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 13:40:32 [ApiClient.Invoke] GET https://api.crowdstrike.com/alerts/queries/alerts/v2?filter=data_domains%3A%27Endpoint%27
VERBOSE: 13:40:32 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:40:32 [ApiClient.Invoke] 200: OK
VERBOSE: 13:40:32 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:40:33 GMT, Transfer-Encoding=chunked, Connection=keep-alive, Strict-Transport-Security=max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-1, X-Cs-Traceid=0399b586-9690-4d28-91df-bc6552615b74, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5958
12968
PS C:\Users\brend> $Id = Get-FalconAlert -Filter "data_domains:'Endpoint'" -All
VERBOSE: 13:40:50 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 13:40:50 [ApiClient.Invoke] GET https://api.crowdstrike.com/alerts/queries/alerts/v2?limit=10000&filter=data_domains%3A%27Endpoint%27
VERBOSE: 13:40:50 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:40:52 [ApiClient.Invoke] 200: OK
VERBOSE: 13:40:52 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:40:53 GMT, Transfer-Encoding=chunked, Connection=keep-alive, Strict-Transport-Security=max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-1, X-Cs-Traceid=183beaca-ed2d-48a2-8189-7cad39f5b12b, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5958
VERBOSE: 13:40:52 [Write-Result] query_time=1.882501376, pagination.offset=0, pagination.limit=10000, pagination.total=12968, writes=, powered_by=detectsapi, trace_id=183beaca-ed2d-48a2-8189-7cad39f5b12b
VERBOSE: 13:40:52 [Get-FalconAlert] Retrieved 10000 of 12968
VERBOSE: 13:40:52 [ApiClient.Invoke] GET https://api.crowdstrike.com/alerts/queries/alerts/v2?limit=10000&filter=data_domains%3A%27Endpoint%27&offset=10000
VERBOSE: 13:40:52 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:40:53 [ApiClient.Invoke] 500: InternalServerError
VERBOSE: 13:40:53 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:40:54 GMT, Connection=keep-alive, X-Content-Type-Options=nosniff, X-Cs-Traceid=48834745-daf1-4164-af28-4e972a52d6c2, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5958, Strict-Transport-Security=max-age=31536000; includeSubDomains
VERBOSE: 13:40:53 [Write-Result] query_time=1.73E-07, powered_by=crowdstrike-api-gateway, trace_id=48834745-daf1-4164-af28-4e972a52d6c2
Write-Result: C:\git\CrowdStrike\PSFalcon\private\Private.ps1:687
Line |
 687 |          Write-Result $Object
     |          ~~~~~~~~~~~~~~~~~~~~
     | {"code":500,"message":"Internal Server Error: Please provide trace-id='48834745-daf1-4164-af28-4e972a52d6c2' to
     | support"}
PS C:\Users\brend> $Id.count
10000

US-GOV-1:

PS C:\Users\brend> Get-FalconAlert -Total
VERBOSE: 13:41:41 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 13:41:41 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2
VERBOSE: 13:41:41 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:41:41 [ApiClient.Invoke] 200: OK
VERBOSE: 13:41:41 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:41:42 GMT, Transfer-Encoding=chunked, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=f2a04a74-f8bb-4008-82df-57a9d3160347, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5999
303
PS C:\Users\brend> Get-FalconAlert -Filter "data_domains:'Endpoint'" -Total
VERBOSE: 13:41:46 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 13:41:46 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?filter=data_domains%3A%27Endpoint%27
VERBOSE: 13:41:46 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:41:46 [ApiClient.Invoke] 200: OK
VERBOSE: 13:41:46 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:41:47 GMT, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=32d0e566-9f98-4cbc-8d87-b6e9d8e0ff15, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5998
72
PS C:\Users\brend> $Id = Get-FalconAlert -Filter "data_domains:'Endpoint'" -Limit 30 -All
VERBOSE: 13:41:56 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 13:41:56 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=30&filter=data_domains%3A%27Endpoint%27
VERBOSE: 13:41:56 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:41:57 [ApiClient.Invoke] 200: OK
VERBOSE: 13:41:57 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:41:57 GMT, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=88108f07-c282-4dfb-afe2-40ba3668877f, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5997
VERBOSE: 13:41:57 [Write-Result] query_time=0.035144181, pagination.offset=0, pagination.limit=30, pagination.total=72, writes=, powered_by=detectsapi, trace_id=88108f07-c282-4dfb-afe2-40ba3668877f
VERBOSE: 13:41:57 [Get-FalconAlert] Retrieved 30 of 72
VERBOSE: 13:41:57 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=30&filter=data_domains%3A%27Endpoint%27&offset=30
VERBOSE: 13:41:57 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:41:57 [ApiClient.Invoke] 200: OK
VERBOSE: 13:41:57 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:41:57 GMT, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=42c08c1a-6fcf-449e-b66a-db2aea1ad891, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5996
VERBOSE: 13:41:57 [Write-Result] query_time=0.008939484, pagination.offset=30, pagination.limit=30, pagination.total=72, writes=, powered_by=detectsapi, trace_id=42c08c1a-6fcf-449e-b66a-db2aea1ad891
VERBOSE: 13:41:57 [Get-FalconAlert] Retrieved 60 of 72
VERBOSE: 13:41:57 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=30&filter=data_domains%3A%27Endpoint%27&offset=60
VERBOSE: 13:41:57 [ApiClient.Invoke] Accept=application/json
VERBOSE: 13:41:57 [ApiClient.Invoke] 200: OK
VERBOSE: 13:41:57 [ApiClient.Invoke] Server=nginx, Date=Wed, 11 Sep 2024 20:41:57 GMT, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=d3bba755-78d3-4a97-9fcd-3505e0579e91, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5995
VERBOSE: 13:41:57 [Write-Result] query_time=0.009242527, pagination.offset=60, pagination.limit=30, pagination.total=72, writes=, powered_by=detectsapi, trace_id=d3bba755-78d3-4a97-9fcd-3505e0579e91
VERBOSE: 13:41:57 [Get-FalconAlert] Retrieved 72 of 72
PS C:\Users\brend> $Id.Count
72

The [Get-FalconAlert] Retrieved X of Y verbose messages match the expected -Total values... but yours isn't doing that. I'm a bit stumped.

I recommend opening a ticket with CrowdStrike support for the filtering not being applied. That doesn't seem to be PSFalcon related since it is appearing in the request that's being sent, and it doesn't happen when I try to reproduce it.

@bk-cs
Copy link
Collaborator

bk-cs commented Sep 11, 2024

There's an encoding difference in the URL string when you send it

filter=data_domains%3A'Endpoint'

versus when I send it

filter=data_domains%3A%27Endpoint%27

PSFalcon encodes the query string with [System.Uri]::EscapeDataString() in the private Build-Content function (under the Build-Query sub-function). What happens when you do this in PowerShell?

PS C:\Users\brend> [System.Uri]::EscapeDataString("data_domains:'Endpoint'")
data_domains%3A%27Endpoint%27

@datorr2
Copy link
Contributor Author

datorr2 commented Sep 11, 2024

PSFalcon encodes the query string with [System.Uri]::EscapeDataString() in the private Build-Content function (under the Build-Query sub-function). What happens when you do this in PowerShell?

PS C:\Users\brend> [System.Uri]::EscapeDataString("data_domains:'Endpoint'")
data_domains%3A%27Endpoint%27

This is what I get:

[System.Uri]::EscapeDataString("data_domains:'Endpoint'")
data_domains%3A'Endpoint'

However, with [System.Web.HttpUtility]::UrlEncode():

PS> [System.Web.HttpUtility]::UrlEncode("data_domains:'Endpoint'")
data_domains%3a%27Endpoint%27

@bk-cs
Copy link
Collaborator

bk-cs commented Sep 12, 2024

Any difference if you modify private\Private.ps1 to use your [System.Web.HttpUtility] method?

Try changing line 194 from:

          ,($Field,([System.Uri]::EscapeDataString($Value)) -join '=')

To:

          ,($Field,([System.Web.HttpUtility]::UrlEncode($Value)) -join '=')

Given that PSFalcon did not encode quotation marks previously, I don't think that it's the problem... but it's worth a shot.

@datorr2
Copy link
Contributor Author

datorr2 commented Sep 12, 2024

Any difference if you modify private\Private.ps1 to use your [System.Web.HttpUtility] method?

The encoding is different in the Verbose output, but still getting 500 error:

$Alerts = Get-FalconAlert -All -Filter "data_domains:'Endpoint'"
VERBOSE: 16:05:45 [Get-FalconAlert] /alerts/queries/alerts/v2:get
VERBOSE: 16:05:45 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=10000&filter=data_domains%3a%27Endpoint%27
VERBOSE: 16:05:45 [ApiClient.Invoke] Accept=application/json
VERBOSE: 16:05:46 [ApiClient.Invoke] 200: OK
VERBOSE: 16:05:46 [ApiClient.Invoke] Transfer-Encoding=chunked, Connection=keep-alive, Strict-Transport-Security=max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains, X-Cs-Region=us-gov-1, X-Cs-Traceid=6db46330-faef-492e-8bda-cbde2b3c1eb1, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5996,
Date=Thu, 12 Sep 2024 20:05:46 GMT, Server=nginx
VERBOSE: 16:05:46 [Write-Result] query_time=0.725056636, pagination.offset=0, pagination.limit=10000, pagination.total=172372, writes=, powered_by=detectsapi, trace_id=6db46330-faef-492e-8bda-cbde2b3c1eb1
VERBOSE: 16:05:46 [Get-FalconAlert] Retrieved 10000 of 172372
VERBOSE: 16:05:46 [ApiClient.Invoke] GET https://api.laggar.gcw.crowdstrike.com/alerts/queries/alerts/v2?limit=10000&filter=data_domains%3a%27Endpoint%27&offset=10000
VERBOSE: 16:05:46 [ApiClient.Invoke] Accept=application/json
VERBOSE: 16:05:46 [ApiClient.Invoke] 500: InternalServerError
VERBOSE: 16:05:46 [ApiClient.Invoke] Connection=keep-alive, X-Content-Type-Options=nosniff, X-Cs-Traceid=a0a92b8b-65c1-4f16-baf0-9903b586ce31, X-Ratelimit-Limit=6000, X-Ratelimit-Remaining=5995, Strict-Transport-Security=max-age=31536000; includeSubDomains, Date=Thu, 12 Sep 2024 20:05:46 GMT, Server=nginx
VERBOSE: 16:05:46 [Write-Result] query_time=1.57E-07, powered_by=crowdstrike-api-gateway, trace_id=a0a92b8b-65c1-4f16-baf0-9903b586ce31
Write-Result : {"code":500,"message":"Internal Server Error: Please provide trace-id=\u0027a0a92b8b-65c1-4f16-baf0-9903b586ce31\u0027 to support"}
At C:\Program Files\WindowsPowerShell\Modules\PSFalcon\2.2.7\private\Private.ps1:688 char:9
+         Write-Result $Object
+         ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidResult: (@{resources=System.Object[]}:PSObject) [Write-Result], Exception
    + FullyQualifiedErrorId : Write-Result
 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants