Feature description

Incremental will remove duplicating rows (only with the same cursor field value, read the docs) based on content hash or primary key. This is not expected behavior when merge key is present - destination should be allowed to merge changes coming from the new records.
We want to disable "deduplication" in Incremental and expose destination to all the rows from data source.

Are you a dlt user?

Yes, I'm already a dlt user.

Use case

Please see #971 (comment)

Proposed solution

1. • disable deduplication by setting primary_key of Incremental to () (disable dedup) when resource has "merge" write disposition. we already propagate resource key to Incremental key so change should be easy
2. • keep explicitly set Incremental primary key (should already work like that)
3. • observe apply_hints changes in the resource (_set_hints) and if primary
4. • make sure that using with_hints marker that is executed during resource execution is able to apply write disposition and disable the deduplication
5. • also test for transformers that bind() the incremental late
6. • warn if dedup state grows too big ie. > 200 entries

Related issues

No response