Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

atlassian_{bitbucket,confluence,jira}: improve pipeline robustness to empty strings #12977

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

atlassian_{bitbucket,confluence,jira}: improve pipeline robustness to empty strings #12977

wants to merge 1 commit into from

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Mar 6, 2025 •
edited
Loading

Proposed commit message

atlassian_{bitbucket,confluence,jira}: improve pipeline robustness to empty strings

Atlassian will send empty string values rather than omit fields, so null
out all empty string values at the start of processing. Also improve
error reporting and script robustness.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 added enhancement New feature or request Integration:atlassian_jira Atlassian Jira Integration:atlassian_confluence Atlassian Confluence Integration:atlassian_bitbucket Atlassian Bitbucket bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations] labels Mar 6, 2025
@efd6 efd6 self-assigned this Mar 6, 2025
@efd6 efd6 force-pushed the s5722-atlassian branch 2 times, most recently from cda573b to 4cf6aeb Compare March 6, 2025 01:32
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Mar 6, 2025 •
edited
Loading

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@efd6 efd6 marked this pull request as ready for review March 6, 2025 01:55
@efd6 efd6 requested a review from a team as a code owner March 6, 2025 01:55
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

efd6
efd6 commented Mar 6, 2025
View reviewed changes
packages/atlassian_bitbucket/data_stream/audit/_dev/test/pipeline/test-audit-api.log
{"timestamp":"2021-11-27T17:26:25.045Z","author":{"name":"System","type":"system","id":"-1","avatarUri":""},"type":{"categoryI18nKey":"bitbucket.service.audit.category.usersandgroups","category":"Users and groups","actionI18nKey":"bitbucket.service.user.audit.action.directorycreated","action":"User directory created"},"affectedObjects":[],"changedValues":[],"node":"8767044c-1b98-4d64-82db-ef29af8c3792","method":"System","extraAttributes":[{"nameI18nKey":"bitbucket.service.user.audit.attribute.directory.name","name":"Directory name","value":"Bitbucket Internal Directory"},{"nameI18nKey":"bitbucket.audit.attribute.legacy.target","name":"target","value":"Bitbucket Internal Directory"}]}
{"timestamp":"2021-11-27T17:38:53.360Z","author":{"name":"admin","type":"NORMAL","id":"2","uri":"http://bitbucket.internal:7990/users/admin","avatarUri":""},"type":{"categoryI18nKey":"bitbucket.service.audit.category.usersandgroups","category":"Users and groups","actionI18nKey":"bitbucket.service.user.audit.action.userrenamed","action":"Username changed"},"affectedObjects":[{"name":"test.user","type":"USER","id":"3"}],"changedValues":[{"key":"Username","i18nKey":"bitbucket.service.user.audit.attribute.user.name","from":"test","to":"test.user"}],"source":"","system":"http://bitbucket.internal:7990","node":"8767044c-1b98-4d64-82db-ef29af8c3792","method":"Browser","extraAttributes":[{"nameI18nKey":"bitbucket.audit.attribute.legacy.details","name":"details","value":"{\"oldUsername\":\"test\"}"},{"nameI18nKey":"bitbucket.audit.attribute.legacy.target","name":"target","value":"test.user"}]}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"source":""

packages/atlassian_confluence/data_stream/audit/_dev/test/pipeline/test-audit-api.log
{"timestamp":"2021-11-28T17:05:37.158Z","author":{"name":"Joe Bob","type":"user","id":"2c9680837d4a3682017d4a375a280000","uri":"http://confluence.internal:8090/admin/users/viewuser.action?username=admin123","avatarUri":""},"type":{"categoryI18nKey":"audit.logging.category.user.management","category":"Users and groups","actionI18nKey":"audit.logging.summary.user.updated","action":"User details updated"},"affectedObjects":[{"name":"asdf asdfasdf","type":"User","uri":"http://confluence.internal:8090/admin/users/viewuser.action?username=asdf123","id":"2c9680837d4a3682017d67821e520003"}],"changedValues":[{"key":"Display name","i18nKey":"Display name","from":"asdf","to":"asdf asdfasdf"}],"source":"10.100.100.2","system":"http://confluence.internal:8090","method":"Browser","extraAttributes":[]}
{"timestamp":"2021-11-28T17:05:37.158Z","author":{"name":"Joe Bob","type":"user","id":"2c9680837d4a3682017d4a375a280000","uri":"http://confluence.internal:8090/admin/users/viewuser.action?username=admin123","avatarUri":""},"type":{"categoryI18nKey":"audit.logging.category.user.management","category":"Users and groups","actionI18nKey":"audit.logging.summary.user.updated","action":"User details updated"},"affectedObjects":[{"name":"asdf asdfasdf","type":"User","uri":"http://confluence.internal:8090/admin/users/viewuser.action?username=asdf123","id":"2c9680837d4a3682017d67821e520003"}],"changedValues":[{"key":"Display name","i18nKey":"Display name","from":"asdf","to":"asdf asdfasdf"}],"source":"","system":"http://confluence.internal:8090","method":"Browser","extraAttributes":[]}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"source":""

packages/atlassian_jira/data_stream/audit/_dev/test/pipeline/test-audit-api.log
{"timestamp":"2021-11-28T18:23:13.741Z","author":{"name":"admin.user1","type":"ApplicationUser","id":"10000","uri":"http://jira.internal:8088/secure/ViewProfile.jspa?name=admin.user1","avatarUri":""},"type":{"categoryI18nKey":"jira.auditing.category.usermanagement","category":"user management","actionI18nKey":"jira.auditing.user.updated","action":"User updated"},"affectedObjects":[{"name":"admin.user1","type":"USER","uri":"http://jira.internal:8088/secure/ViewProfile.jspa?name=admin.user1","id":"JIRAUSER10000"}],"changedValues":[{"key":"Full name","i18nKey":"common.words.fullname","from":"Admin User","to":"Admin User1"}],"source":"10.100.100.2","system":"http://jira.internal:8088","method":"Browser","extraAttributes":[]}
{"timestamp":"2021-11-28T18:23:13.741Z","author":{"name":"admin.user1","type":"ApplicationUser","id":"10000","uri":"http://jira.internal:8088/secure/ViewProfile.jspa?name=admin.user1","avatarUri":""},"type":{"categoryI18nKey":"jira.auditing.category.usermanagement","category":"user management","actionI18nKey":"jira.auditing.user.updated","action":"User updated"},"affectedObjects":[{"name":"admin.user1","type":"USER","uri":"http://jira.internal:8088/secure/ViewProfile.jspa?name=admin.user1","id":"JIRAUSER10000"}],"changedValues":[{"key":"Full name","i18nKey":"common.words.fullname","from":"Admin User","to":"Admin User1"}],"source":"","system":"http://jira.internal:8088","method":"Browser","extraAttributes":[]}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"source":""

kcreddy
kcreddy reviewed Mar 6, 2025
View reviewed changes
packages/atlassian_bitbucket/changelog.yml
Comment on lines +4 to +6
- description: Improve error reporting.
type: enhancement
link: https://github.com/elastic/integrations/pull/12977
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In bitbucket changes, looks like improved error reporting is missing.

Copy link
Contributor Author

@efd6 efd6 Mar 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Added.

@efd6
atlassian_{bitbucket,confluence,jira}: improve pipeline robustness to…
3747c71 
… empty strings

Atlassian will send empty string values rather than omit fields, so null
out all empty string values at the start of processing. Also improve
error reporting and script robustness.
@efd6 efd6 force-pushed the s5722-atlassian branch from 4cf6aeb to 3747c71 Compare March 6, 2025 07:00
@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate failed Quality Gate failed

Failed conditions
71.8% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @efd6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kcreddy kcreddy kcreddy approved these changes

Assignees

@efd6 efd6

Labels
bugfix Pull request that fixes a bug issue enhancement New feature or request Integration:atlassian_bitbucket Atlassian Bitbucket Integration:atlassian_confluence Atlassian Confluence Integration:atlassian_jira Atlassian Jira Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations]
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@efd6 @elasticmachine @kcreddy