Consider flush_all_streams for schema updates

[mvgijssel]

Problem

Due to a high number of schema messages due to handling TOAST messages in transferwise/pipelinewise-tap-postgres#146 all the records are constantly flushed resulting in very slow performance.

Proposed changes

This changes ensures the flush_all_streams configuration setting is honored when a new schema message arrives in the target. If this is not set, then only the records for the current stream are flushed instead for all rows.

Types of changes

What types of changes does your code introduce to PipelineWise?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update (if none of the other choices apply)

Checklist

• Description above provides context of the change
• I have added tests that prove my fix is effective or that my feature works
• Unit tests for changes (not needed for documentation changes)
• CI checks pass with my changes
• Bumping version in setup.py is an individual PR and not mixed with feature or bugfix PRs
• Commit message/PR title starts with [AP-NNNN] (if applicable. AP-NNNN = JIRA ID)
• Branch name starts with AP-NNN (if applicable. AP-NNN = JIRA ID)
• Commits follow "How to write a good git commit message"
• Relevant documentation is updated including usage instructions

[Samira-El]

Thanks @mvgijssel, this makes sense! not sure why it was like this in the first place 😅 .

[Samira-El]

Can you rebase the branch so I can merge?

[mvgijssel]

Can you rebase the branch so I can merge?

Will do!

[mvgijssel]

GitHub shows me

First-time contributors need a maintainer to approve running workflows.

So once you approve the workflow run it should be good :)