Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Make check expired message interval configurable to reduce the consuming thread block time #9019

Open
humkum opened this issue Dec 4, 2024 · 3 comments · May be fixed by #9068
Open

Comments

@humkum
Copy link
Contributor

humkum commented Dec 4, 2024

Is Your Feature Request Related to a Problem?

The consumer would be block more than consume timeout time, because that the clean expired messages schedule task's interval was set as consumeTimeout.
When consumeTimeout was set to 15 minutes, the consuming thread may be block 15~30 minutes actually.
image
image

Describe the Solution You'd Like

Make the cleaning expired message schedule interval configurable.

Describe Alternatives You've Considered

None.

Additional Context

No response

@lizhimins
Copy link
Member

队列模型里这个是预期内的,cache 中的 min offset 没有提交,导致看起来延迟很高,运行中实际上不影响后续消息消费,如果叠加负载均衡的影响,会有大量的重复。

@humkum
Copy link
Contributor Author

humkum commented Dec 19, 2024

队列模型里这个是预期内的,cache 中的 min offset 没有提交,导致看起来延迟很高,运行中实际上不影响后续消息消费,如果叠加负载均衡的影响,会有大量的重复。

是的,这里是预期内的。这里其实是想改一下检测消息消费超时的定时任务的时间间隔,目前配置的间隔和消息超时时间是一致的。会导致比如我配置的超时时间是 1h,那么有可能存在第一次定时任务检测时发现没有过期,需要再等 1h 才能判断是否过期的情况,因此实际超时时间成了 (1h, 2h]

@humkum
Copy link
Contributor Author

humkum commented Dec 19, 2024

队列模型里这个是预期内的,cache 中的 min offset 没有提交,导致看起来延迟很高,运行中实际上不影响后续消息消费,如果叠加负载均衡的影响,会有大量的重复。

是的,这里是预期内的。这里其实是想改一下检测消息消费超时的定时任务的时间间隔,目前配置的间隔和消息超时时间是一致的。会导致比如我配置的超时时间是 1h,那么有可能存在第一次定时任务检测时发现没有过期,需要再等 1h 才能判断是否过期的情况,因此实际超时时间成了 (1h, 2h]

这里如果我们将消费超时时间配置为 1h, 检测间隔配置为 15min,实际过期时间最多只有 1h + 15min。消费就不会卡那么久了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment