Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Virustotal Get-VirusTotalIPReport exeeded quota #11560

Closed
Dr1nS0menOp opened this issue Dec 13, 2024 · 5 comments
Closed

Virustotal Get-VirusTotalIPReport exeeded quota #11560

Dr1nS0menOp opened this issue Dec 13, 2024 · 5 comments
Assignees
Labels
Playbook Playbook specialty review needed

Comments

@Dr1nS0menOp
Copy link

Dr1nS0menOp commented Dec 13, 2024

Describe the bug
When I run my predefined playbook I get the error "429". The raw details are as following in the body of the "Get an IP report":
{ "error": { "code": "QuotaExceededError", "message": "Quota exceeded" } }

To Reproduce
Steps to reproduce the behavior:

  1. Go to from https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/VirusTotal/Playbooks/Get-VirusTotalIPReport and use the playbook when a incident is created.

  2. Assign the permissions as described on the github page.

  3. Connect The log analytics space if not done already, connect Virustotal with the API from https://www.virustotal.com/gui/user//apikey

  4. Save the playbook and make an automation rule as following:
    { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "workspace": { "type": "String" } }, "resources": [ { "id": "[concat(resourceId('Microsoft.OperationalInsights/workspaces/providers', parameters('workspace'), 'Microsoft.SecurityInsights'),'/AutomationRules/cf5d03de-******************************')]", "name": "[concat(parameters('workspace'),'/Microsoft.SecurityInsights/cf5d03de-******************************2a5afc')]", "type": "Microsoft.OperationalInsights/workspaces/providers/AutomationRules", "apiVersion": "2024-04-01-preview", "properties": { "displayName": "EnrichIncidentWithVirusTotalData+TeamsMessage", "order": 2, "triggeringLogic": { "isEnabled": true, "triggersOn": "Incidents", "triggersWhen": "Created", "conditions": [ { "conditionType": "Property", "conditionProperties": { "propertyName": "IncidentRelatedAnalyticRuleIds", "operator": "Contains", "propertyValues": [ "[concat(resourceId('Microsoft.OperationalInsights/workspaces/providers', parameters('workspace'), 'Microsoft.SecurityInsights'),'/alertRules/', '0903fc19-74b4-4f54-8cb4***************')]" ] } } ] }, "actions": [ { "order": 1, "actionType": "RunPlaybook", "actionConfiguration": { "logicAppResourceId": "/subscriptions/a44a40cd-******************************/resourceGroups/SIEMPOC/providers/Microsoft.Logic/workflows/VIRUSTOTAL-GITHUBCOPY", "tenantId": "9ce66fc*******************************88******" } } ] } } ] }

  5. Then make a scheduled analytics rule with the following query to run every hour and grap the ip address of the ExtractedIP value:
    let ThreatIPs = ThreatIntelligenceIndicator | where isnotempty(NetworkDestinationIP) | project NetworkDestinationIP, Description; ApacheHTTPServer_CL | extend ExtractedIP = extract(@"^(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})", 1, RawData) | where isnotempty(ExtractedIP) // timefilter set to the last 100 years | where TimeGenerated >= datetime(1924-01-01) and TimeGenerated <= now() | join kind=inner (ThreatIPs) on $left.ExtractedIP == $right.NetworkDestinationIP | summarize Count = count(), RawDataList = make_list(RawData), DescriptionList = make_list(Description) by ExtractedIP, _ResourceId | where ExtractedIP == "57.***.***.***" // Filter towards a specific address to always get one result for testing purposes | project _ResourceId, DescriptionList, ExtractedIP, RawDataList

  6. test if the incident is created (It works for me!)

  7. Since the automation rule was created it needs to be connected to the incident.

  8. Test if the playbook has been run after the incident creation

  9. Test if I got enriched data and a teams message.

Expected behavior
I expect that the playbook utpates the incident with the scoring and data and it sends a nice teams message with a lot of Virustotal info.
But it did not it throws the error I

Screenshots
The playbook:
Image

The errors:
Image
Image
Image

Current quota:
Image

I even tried to do a timeout for 60 seconds for every request, but it does not work. I tested that Virustotal receives a IP address, that works. To make the problem even more difficult, I let the playbook run for every hour also in the night and some runs worked (I changed nothing). So sometimes it works sometimes it does not. Is this the fault of VirusTotal?

Additional context
My complete Playbook:
{ "definition": { "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#", "contentVersion": "1.0.0.0", "triggers": { "Microsoft_Sentinel_incident": { "type": "ApiConnectionWebhook", "inputs": { "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } }, "body": { "callback_url": "@{listCallbackUrl()}" }, "path": "/incident-creation" } } }, "actions": { "Entities_-_Get_IPs": { "runAfter": {}, "type": "ApiConnection", "inputs": { "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } }, "method": "post", "body": "@triggerBody()?['object']?['properties']?['relatedEntities']", "path": "/entities/ip" } }, "For_each": { "foreach": "@body('Entities_-_Get_IPs')?['IPs']", "actions": { "Condition": { "actions": { "Add_comment_to_incident_(V3)": { "runAfter": { "Send_Data": [ "Succeeded" ] }, "type": "ApiConnection", "inputs": { "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } }, "method": "post", "body": { "incidentArmId": "@triggerBody()?['object']?['id']", "message": "<p>Virus Total IP Report found for @{items('For_each')?['Address']}<br>\nReputation is: @{body('Get_an_IP_report')?['data']?['attributes']?['reputation']} which indicates likely harmless<br>\nQuery: VTIPReport_CL | where id_s == \"@{items('For_each')?['Address']}\"</p>" }, "path": "/Incidents/Comment" } }, "Send_Data": { "type": "ApiConnection", "inputs": { "host": { "connection": { "name": "@parameters('$connections')['azureloganalyticsdatacollector-1']['connectionId']" } }, "method": "post", "body": "@{body('Get_an_IP_report')?['data']}", "headers": { "Log-Type": "VTIPReport", "time-generated-field": "@{utcNow()}" }, "path": "/api/logs" } }, "Post_message_in_a_chat_or_channel": { "runAfter": { "Add_comment_to_incident_(V3)": [ "Succeeded" ] }, "type": "ApiConnection", "inputs": { "host": { "connection": { "name": "@parameters('$connections')['teams']['connectionId']" } }, "method": "post", "body": { "recipient": { "groupId": "**********************************", "channelId": "19:1**********************************[email protected]" }, "messageBody": "<p class=\"editor-paragraph\">**VirusTotal IP Analyse**<br><br>🔍 Geanalyseerd IP-adres: @{item()?['Address']}
Hostname(s): @{triggerBody()?['object']?['properties']?['relatedEntities']}<br><br>📊 Reputatiegegevens:<br>- Reputatiescore: @{body('Get_an_IP_report')?['data']?['attributes']?['reputation']}<br>&gt; *Een positieve score betekent een lage dreiging; negatieve waarden betekenen een verhoogd risico.<br>- Malicious engines: @{body('Get_an_IP_report')?['data']?['attributes']?['last_analysis_stats']?['malicious']}<br>- Suspicious engines: @{body('Get_an_IP_report')?['data']?['attributes']?['last_analysis_stats']?['suspicious']}<br>- Harmless engines:

📅 Laatste analyse:
- Datum: <br><br>**Netwerkgegevens:**<br>- ASN:
- ISP: <br>- Land:
-

🔗 Meer informatie:
https://www.virustotal.com/gui/ip-address/@{item()?['Address']}


"
},
"path": "/beta/teams/conversation/message/poster/@{encodeURIComponent('User')}/location/@{encodeURIComponent('Channel')}"
}
}
},
"runAfter": {
"Get_an_IP_report": [
"Succeeded"
]
},
"else": {
"actions": {
"Add_comment_to_incident_(V3)3": {
"runAfter": {
"Send_Data_2": [
"Succeeded"
]
},
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "@parameters('$connections')['azuresentinel']['connectionId']"
}
},
"method": "post",
"body": {
"incidentArmId": "@triggerBody()?['object']?['id']",
"message": "

Virus Total IP Report found for @{items('For_each')?['Address']}
\nReputation is: @{body('Get_an_IP_report')?['data']?['attributes']?['reputation']} which indicates likely malicous
\nQuery: VTIPReport_CL | where id_s == "@{items('For_each')?['Address']}"

"
},
"path": "/Incidents/Comment"
}
},
"Send_Data_2": {
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "@parameters('$connections')['azureloganalyticsdatacollector-1']['connectionId']"
}
},
"method": "post",
"body": "@{body('Get_an_IP_report')?['data']}",
"headers": {
"Log-Type": "VTIPReport",
"time-generated-field": "@{utcNow()}"
},
"path": "/api/logs"
}
}
}
},
"expression": {
"and": [
{
"greaterOrEquals": [
"@Body('Get_an_IP_report')?['data']?['attributes']?['reputation']",
0
]
}
]
},
"type": "If"
},
"Get_an_IP_report": {
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "@parameters('$connections')['virustotal-1']['connectionId']"
}
},
"method": "get",
"path": "/api/v3/ip_addresses/@{encodeURIComponent(items('For_each')?['Address'])}"
}
}
},
"runAfter": {
"Entities
-_Get_IPs": [
"Succeeded"
]
},
"type": "Foreach"
}
},
"outputs": {},
"parameters": {
"$connections": {
"type": "Object",
"defaultValue": {}
}
}
},
"parameters": {
"$connections": {
"value": {
"azuresentinel": {
"id": "/subscriptions//providers/Microsoft.Web/locations/westeurope/managedApis/azuresentinel",
"connectionId": "/subscriptions/
/resourceGroups//providers/Microsoft.Web/connections/azuresentinel-VIRUSTOTAL-GITHUBCOPY",
"connectionName": "azuresentinel-VIRUSTOTAL-GITHUBCOPY",
"connectionProperties": {
"authentication": {
"type": "ManagedServiceIdentity"
}
}
},
"azureloganalyticsdatacollector-1": {
"id": "/subscriptions/a44a40cd-b439-4582-a4f1-
/providers/Microsoft.Web/locations/westeurope/managedApis/azureloganalyticsdatacollector",
"connectionId": "/subscriptions/a44a40cd-b439-4582-a4f1-/resourceGroups//providers/Microsoft.Web/connections/azureloganalyticsdatacollector-1",
"connectionName": "azureloganalyticsdatacollector-1"
},
"virustotal-1": {
"id": "/subscriptions/***************************/providers/Microsoft.Web/locations/westeurope/managedApis/virustotal",
"connectionId": "/subscriptions/a44a40cd-b439-4582-a4f1-/resourceGroups//providers/Microsoft.Web/connections/virustotal",
"connectionName": "virustotal"
},
"teams": {
"id": "/subscriptions/a44a40cd-b439-4**************/providers/Microsoft.Web/locations/westeurope/managedApis/teams",
"connectionId": "/subscriptions/a44a40cd-b439-4582-a4f1-a**************/resourceGroups/
/providers/Microsoft.Web/connections/teams-1",
"connectionName": "teams-1"
}
}
}
}
}
`

@v-sudkharat v-sudkharat added the Playbook Playbook specialty review needed label Dec 13, 2024
@v-visodadasi v-visodadasi removed their assignment Dec 18, 2024
@v-shukore
Copy link
Contributor

Hi @Dr1nS0menOp, Thanks for flagging this issue, we will investigate this issue and get back to you with some updates. Thanks!

@v-shukore
Copy link
Contributor

Hi @Dr1nS0menOp,

Thanks for your patience,

As checked, we have determined that you are using a public API, which has a limit of 4 requests per minute. As a result, you are encountering the 'quota exceeded' error.
Image

you can refer this document for more clarification - https://docs.virustotal.com/reference/public-vs-premium-api.

@v-shukore
Copy link
Contributor

Hi @Dr1nS0menOp,
I hope you are doing well,
As mentioned in the previous comment, you are encountering this error. Please refer to the provided document for further details.
we are closing your issue #11560. If you still need support for this issue, feel free to re-open at any time. Thank you for your co-operation.

@Dr1nS0menOp
Copy link
Author

Dr1nS0menOp commented Jan 27, 2025

Hi @Dr1nS0menOp, I hope you are doing well, As mentioned in the previous comment, you are encountering this error. Please refer to the provided document for further details. we are closing your issue #11560. If you still need support for this issue, feel free to re-open at any time. Thank you for your co-operation.

Hi @v-shukore,

This is not the issue in my opinion because I see no traffic coming into Virustotal at the times it is queried. And at the same time it sometimes succeeds. Is there any explanation for that?

@v-shukore
Copy link
Contributor

Hi @Dr1nS0menOp,
Thanks for your response.
There might be issues on the VirusTotal server side that are causing these intermittent failures. This could result in some queries not reaching VirusTotal while others do. Additionally, you could reach out to VirusTotal support for further assistance, or we kindly request you to raise a support ticket in the Azure portal, so our support team can look into it and connect with you if required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Playbook Playbook specialty review needed
Projects
None yet
Development

No branches or pull requests

4 participants