multiple clients

[jney]

Hi. Probably a misunderstanding, but I thought that if I had multiple consumers, they will read distinct messages from queue. But it appears that are reading same messages.
How could I achieve this?

[harlow]

Correct the library currently doesn’t support consumer groups (that auto balance shards)

It’s supports two strategies:

1. Consumer scans all shards
2. Consumer scans single shard (you must provide the shard ID)

[anton-pfc]

@harlow Do I understand correctly that it's impossible to consume a message only once if I'm running 10 instances? Currently what I observe in my tests - if I run 10 instances with the same appName - every message is consumed 10 times. Do you know a workaround for that? How did you implement "at-least-once" semantic in you system?

[harlow]

@anton-pfc sorry if there is confusion here; there isn't a way to run 10 consumer apps all with Scan func and not have them all consume the same records.

If you want to run 10 instances you'll need to pass the shardID to each of the individual consumers with the ScanShard func

the Scan func will automatically scan all shards from the given checkpoint

[thestephenstanton]

@harlow Out of curiosity, was there a reason you didn't implement a way for this thing to be able to run multiple clients looking at the same stream? And basically "gossip" via the store to see who has what shard? I know it is easier said than done, just curious if there was really more to it and it wasn't worth it.

[ckatsaras-godaddy]

Curious if anyone has an update on this issue . @thestephenstanton, did you happen to figure out a solution to this problem?

[thestephenstanton]

@ckatsaras-godaddy we ended up switching to Kafka and inherited a new set of problems lol. So I never figure out how to do this.

[ckatsaras-godaddy]

Ah, I see! Thanks for getting back to me so quickly! 😄