[Bug]: non-stop repeated segment compaction

[zhengbuqian]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version: master

Current Behavior

Used scripts in https://pastebin.com/8KLQ8WyT, to simply insert 200M data. After compaction, 5 segments are left, each with ~400k rows. Then those 5 segments compact indefinitely.

See birdwatcher segment results here: https://pastebin.com/aZ6M7Y2U

Segment A, B, C, D, E was compacted inito A1, B1, C1, D1, E1, then A2, B2, C2, D2, E2, ... Each time compaction happens the old index was dropped and the new index was recreated.

Likely due to: the compaction trigger thought A+B can be merged into a larger segment say F, but the actual compactor thought F would be too large and simply writes A and B to A1 and B1 unchanged.

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[XuanYang-cn]

/assign

[xiaofan-luan]

@XuanYang-cn
should we add another restriction.
if a segment did a single compaction in like last 10 mins it shouldn't be triggered again.
So we can protect some corner case? the size calculation might not be very accurate

[XuanYang-cn]

@XuanYang-cn should we add another restriction. if a segment did a single compaction in like last 10 mins it shouldn't be triggered again. So we can protect some corner case? the size calculation might not be very accurate

This is not a single compaction, it's m -> m, actually it's a critical bug of mix compactor, that DN can't compact segment to targeted size.

I'll add more strict tests to cover the output target size. No previous tests have tested it before, hence the changes in storage could easily broke the output of mix compactor.

[XuanYang-cn]

/assign @zhengbuqian
Should be fix, please help verify

[XuanYang-cn]

/unassign