Schema Changes in AzureDiagnostics Table Affecting WAF analytic rules

[paloaltosensei]

Describe the bug

We identified an issue with the schema of the AzureDiagnostics table, specifically affecting following analytic rules: "Application Gateway WAF - SQLi Detection" and "Application Gateway WAF - XSS Detection". Previously, the fields transactionId_g and hostname_s were available as top-level columns, but they are now nested within the AdditionalFields column as transactionId and hostname. Additionally, the fields details_message_s and details_data_s have been removed from the schema entirely.

Due to this schema change, the analytic rules relying on these fields to detect and analyze WAF events, such as SQL injection attacks, will be inefficient and require significant modifications. Could you please help us adjust these detection rules to align with the new schema changes of the AzureDiagnostics table?

Many thanks!

[v-visodadasi]

Hi @paloaltosensei , Thanks for reporting this issue, we are checking on it with team and get back to you with some update. Thanks!

[v-visodadasi]

Hi @paloaltosensei, Could you please share the AzureDiagnostics table schema with us?

[paloaltosensei]

TenantId	0	System.String	string
TimeGenerated	1	System.DateTime	datetime
ResourceId	2	System.String	string
Category	3	System.String	string
ResourceGroup	4	System.String	string
SubscriptionId	5	System.String	string
ResourceProvider	6	System.String	string
vnetResourceGuid_g	7	System.String	string
ResourceType	8	System.String	string
OperationName	9	System.String	string
ResultType	10	System.String	string
CorrelationId	11	System.String	string
ResultDescription	12	System.String	string
Tenant_g	13	System.String	string
JobId_g	14	System.String	string
RunbookName_s	15	System.String	string
StreamType_s	16	System.String	string
Caller_s	17	System.String	string
requestUri_s	18	System.String	string
Level	19	System.String	string
DurationMs	20	System.Int64	long
CallerIPAddress	21	System.String	string
OperationVersion	22	System.String	string
ResultSignature	23	System.String	string
id_s	24	System.String	string
status_s	25	System.String	string
LogicalServerName_s	26	System.String	string
Message	27	System.String	string
clientInfo_s	28	System.String	string
httpStatusCode_d	29	System.Double	real
identity_claim_appid_g	30	System.String	string
identity_claim_http_schemas_microsoft_com_identity_claims_objectidentifier_g	31	System.String	string
userAgent_s	32	System.String	string
ruleName_s	33	System.String	string
identity_claim_http_schemas_xmlsoap_org_ws_2005_05_identity_claims_upn_s	34	System.String	string
systemId_g	35	System.String	string
isAccessPolicyMatch_b	36	System.SByte	bool
EventName_s	37	System.String	string
httpMethod_s	38	System.String	string
subnetId_s	39	System.String	string
type_s	40	System.String	string
instanceId_s	41	System.String	string
macAddress_s	42	System.String	string
vnetResourceGuid_g	43	System.String	string
direction_s	44	System.String	string
subnetPrefix_s	45	System.String	string
primaryIPv4Address_s	46	System.String	string
conditions_sourcePortRange_s	47	System.String	string
priority_d	48	System.Double	real
conditions_destinationPortRange_s	49	System.String	string
conditions_destinationIP_s	50	System.String	string
conditions_None_s	51	System.String	string
conditions_sourceIP_s	52	System.String	string
httpVersion_s	53	System.String	string
matchedConnections_d	54	System.Double	real
startTime_t	55	System.DateTime	datetime
endTime_t	56	System.DateTime	datetime
DatabaseName_s	57	System.String	string
clientIP_s	58	System.String	string
host_s	59	System.String	string
requestQuery_s	60	System.String	string
sslEnabled_s	61	System.String	string
clientPort_d	62	System.Double	real
httpStatus_d	63	System.Double	real
receivedBytes_d	64	System.Double	real
sentBytes_d	65	System.Double	real
timeTaken_d	66	System.Double	real
resultDescription_ErrorJobs_s	67	System.String	string
resultDescription_ChildJobs_s	68	System.String	string
identity_claim_http_schemas_microsoft_com_identity_claims_scope_s	69	System.String	string
workflowId_s	70	System.String	string
resource_location_s	71	System.String	string
resource_workflowId_g	72	System.String	string
resource_resourceGroupName_s	73	System.String	string
resource_subscriptionId_g	74	System.String	string
resource_runId_s	75	System.String	string
resource_workflowName_s	76	System.String	string
_schema_s	77	System.String	string
correlation_clientTrackingId_s	78	System.String	string
properties_sku_Family_s	79	System.String	string
properties_sku_Name_s	80	System.String	string
properties_tenantId_g	81	System.String	string
properties_enabledForDeployment_b	82	System.SByte	bool
code_s	83	System.String	string
resultDescription_Summary_MachineId_s	84	System.String	string
resultDescription_Summary_ScheduleName_s	85	System.String	string
resultDescription_Summary_Status_s	86	System.String	string
resultDescription_Summary_StatusDescription_s	87	System.String	string
resultDescription_Summary_MachineName_s	88	System.String	string
resultDescription_Summary_TotalUpdatesInstalled_d	89	System.Double	real
resultDescription_Summary_RebootRequired_b	90	System.SByte	bool
resultDescription_Summary_TotalUpdatesFailed_d	91	System.Double	real
resultDescription_Summary_InstallPercentage_d	92	System.Double	real
resultDescription_Summary_StartDateTimeUtc_t	93	System.DateTime	datetime
resource_triggerName_s	94	System.String	string
resultDescription_Summary_InitialRequiredUpdatesCount_d	95	System.Double	real
properties_enabledForTemplateDeployment_b	96	System.SByte	bool
resultDescription_Summary_EndDateTimeUtc_s	97	System.String	string
resultDescription_Summary_DurationInMinutes_s	98	System.String	string
resource_originRunId_s	99	System.String	string
properties_enabledForDiskEncryption_b	100	System.SByte	bool
resource_actionName_s	101	System.String	string
correlation_actionTrackingId_g	102	System.String	string
resultDescription_Summary_EndDateTimeUtc_t	103	System.DateTime	datetime
resultDescription_Summary_DurationInMinutes_d	104	System.Double	real
conditions_protocols_s	105	System.String	string
identity_claim_ipaddr_s	106	System.String	string
ElasticPoolName_s	107	System.String	string
identity_claim_http_schemas_microsoft_com_claims_authnmethodsreferences_s	108	System.String	string
RunOn_s	109	System.String	string
query_hash_s	110	System.String	string
SourceSystem	111	System.String	string
MG	112	System.String	string
ManagementGroupName	113	System.String	string
Computer	114	System.String	string
RawData	115	System.String	string
error_code_s	116	System.String	string
error_message_s	117	System.String	string
fired_b	118	System.SByte	bool
tags_LogicAppsCategory_s	119	System.String	string
tags_hidden_SentinelTemplateName_s	120	System.String	string
tags_hidden_SentinelTemplateVersion_s	121	System.String	string
tags_hidden_SentinelWorkspaceId_s	122	System.String	string
isV2Threshold_b	123	System.SByte	bool
metadataOverflowContentLength_d	124	System.Double	real
actionCount_d	125	System.Double	real
location_s	126	System.String	string
executionClusterType_s	127	System.String	string
AdditionalFields	128	System.Object	dynamic
Type	129	System.String	string
_ResourceId	130	System.String	string