Add example of a producer that correlates posts with acks/nacks #43
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The BlazingMQ Python SDK and the BlazingMQ C++ SDKs use two different mechanisms for correlating a message that a producer posts with the ack or nack that the broker sends back. The BlazingMQ C++ SDK features a “correlation id”, which is a value that a producer can provide along with a posted message and that the broker will return with that message’s ack or nack. This concept is quite conspicuous in the C++ API and is featured in the advanced producer tutorial that is provided along with the SDK. In the Python SDK, on the other hand, there is no corresponding concept; we do not expose the C++ correlation id to clients using the Python SDK. In fact, internally, we use the correlation ID to store the pointer to the `on_ack` callback that the user provides with each posted message. Several users have asked how to correlate a message’s `post` call with its ack or nack. The expected idiom to do this in Python is to pass a different closure as the `on_ack` callback for each message you want to correlate. These closures can store arbitrary data that can be used in the `on_ack` callback, allowing producers to store an identifier by which they can correlate posts and acks/nacks. This patch includes a new example producer, `producer_with_correlation` that demonstrates this idiom, posting ten messages and providing each one with a distinct `on_ack` closure, constructed using the light-weight `functools.partial`. The description for this test is included in the examples documentation, and to make sure that this producer works as expected, we also add it to the `test_examples.py` integration test. Signed-off-by: Patrick M. Niedzielski <[email protected]>
This patch extends the `test_examples.py` integration test for the simple producer example with two additional checks inspired by the new producer-with-correlation example test. First, we make sure that the producer process completes successfully. Second, we verify that it only sent one message to the consumer while it was alive. Signed-off-by: Patrick M. Niedzielski <[email protected]>
Depending how your system is configured, running the `test_deadlock_detection.py` integration test on OSX may result in a dialog window warning you about a crashed or aborted Python process. This crash is intentional, as the test introduces a deadlock and tests whether it can be detected. This patch documents this behavior in both the build documentation and the test itself. Signed-off-by: Patrick M. Niedzielski <[email protected]>
pniedzielski
force-pushed
the
document-correlation-id
branch
from
bded8ff to
6608649
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We've had several users ask us how to correlate a message's post with its ack or nack. The way we intend is to provide a different closure for each message's
on_ack, including any information that that callback may need to correlate the message. This PR introduces an example documenting this.There are three patches in this PR:
Add post/ack correlation example. Here, we add a simple example of a producer that correlates posts with acks or nacks. We also add this to the test suite, to ensure that the example works as intended.
Extend the existing producer test with additional checks. The new producer example test has some additional checks that also make sense in the existing producer example test, so we also add them there.
Document an expected crash dialog on OSX when running integration tests. This test induces a deadlock so that it can verify that the library detects that deadlock. On Linux machines, the deadlock crash is silent as well as intentional. However, on OSX machines, depending on their configuration, each crash causes a dialog box to appear, telling the user that a crash occurred. This is annoying, but it is a product of the intention of the test.
Closes: #42