Add docs best effort responses #3865

oggy-dfin · 2024-12-06T16:42:45Z

DO NOT MERGE YET - this should be merged only once best-effort responses go live on mainnet.

This PR contains the documentation for the new best-effort response feature. In the process, I noticed that we were lacking a good introductory page on inter-canister calls, addressing topics such as:

how do I find which endpoints to call?
what's the role of Candid in inter-canister calls?
why do I care about the desugaring of async/await
attaching cycles to calls
Follows the developer docs style guide.
Follows the best practices and guidelines.
New documentation pages include document tags.
New documentation pages include SEO keywords.
New documents are in the .mdx file format to support the previous two components.
New documents are registered in /sidebars.js, otherwise, it will not appear in the
side navigation bar.
Make sure that the .github/CODEOWNERS file is
filled with new documents that you added. This way we can ensure that future Pull Requests are reviewed by the right people.

We were missing quite a lot of relevant topics: how to discover canister endpoints, guidance on how to perform calls, attaching cycles, no explanation of why the async/await desugaring was important etc. I also now changed the tag to beginner, as the Rust/Motoko language-specific documents are also rated as "beginner". The previous discussion was not really motivated (why should the user care about desugaring?) and quite language dependent, so I removed most of it - the language dependent stuff should go into the language specific docs. Now added all this and also added the difference between best-effort and guaranteed response calls.

github-actions · 2024-12-06T16:43:31Z

🤖 Here's your preview: https://tai3p-vaaaa-aaaam-abema-cai.icp0.io

alin-at-dfinity

Only looked at message-execution-properties.mdx for now and added some nitpicks.

However, I can't help thinking that the topic could be explained more clearly. E.g. start with calls/tasks vs. message executions as we already do, but before we list any properties; just a couple of pictures with simplified sequence diagrams. That way, the difference between call/task and message execution would be immediately clear. It would also immediately make it clear why two requests Req1 and Req2 delivered in that order may produce two responses Rep2 and Rep1, in reverse order (even before the protocol may decide to deliver them in an arbitrary order).

Very likely not for this PR, though.

docs/references/message-execution-properties.mdx

derlerd-dfinity

I left feedback on everything but message-execution-properties.md as Alin already made a pass over this one. I'll make a pass once you've addressed Alin's comments.

derlerd-dfinity · 2024-12-10T07:46:22Z

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx


-## Overview of language runtime for asynchronous code


Re "true" response from the callee: Strictly speaking, it is also not guaranteed in the current messaging model that the response will come from the callee. The response could also be generated by the system if the request couldn't be delivered to the callee.

The property the current messaging model provides is that the response is globally unique, i.e., if the system produces a reject one can conclude that the callee never saw the request.

Re "true" response from the callee: Strictly speaking, it is also not guaranteed in the current messaging model that the response will come from the callee. The response could also be generated by the system if the request couldn't be delivered to the callee.

True, I dropped the "from the callee" part.

The property the current messaging model provides is that the response is globally unique, i.e., if the system produces a reject one can conclude that the callee never saw the request.

Well technically a CANISTER_ERROR is also produced by the system but it means that the callee has seen the request (whether any mutation was made on the callee side is anyone's guess, though)

True, I dropped the "from the callee" part.

FWIW, I still see

in this mode the "true" response from the callee is not guaranteed to be delivered

Maybe you just didn't push your change yet.

I have pushed the change, I see below:

but in this mode the "true" response is not guaranteed to be delivered, and can be masked by a system generated error response instead.

Not sure what's going on 🤷

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx

docs/developer-docs/smart-contracts/best-practices/idempotency.mdx

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx

docs/developer-docs/smart-contracts/best-practices/idempotency.mdx

docs/references/message-execution-properties.mdx

Co-authored-by: Jessie Mongeon <[email protected]>

Co-authored-by: Alin Sinpalean <[email protected]>

docs/references/message-execution-properties.mdx

alin-at-dfinity · 2024-12-13T16:11:50Z

docs/references/message-execution-properties.mdx

 :::

 :::info
-Note that if the code does not `await` the response, the code after the callback is executed in the same message, until the next inter-canister call is triggered using `await`.
+Note that if the code does not `await` the response, the code after the callback is executed in the same message execution, until the next inter-canister call is triggered using `await`.


I guess I just find it awkward that we talk about a call or a callback without an await. If there's no await, nothing is called, you've merely created a future even though it's not obvious from the syntax.

IOW, leaving it at "the code after the await is executed as a separate message execution (callback)" (which the paragraph above already says) is IMHO cleaner than adding "but if there's no await, there's no callback". I would rather change this paragraph to say "note that it is actually the await, not the thing that looks like a function call that triggers the canister call and marks the separation point between message executions". Or something to that effect.

docs/references/message-execution-properties.mdx

docs/developer-docs/security/security-best-practices/inter-canister-calls.mdx

alin-at-dfinity · 2024-12-13T16:52:52Z

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx


-## Overview of language runtime for asynchronous code


True, I dropped the "from the callee" part.

FWIW, I still see

in this mode the "true" response from the callee is not guaranteed to be delivered

Maybe you just didn't push your change yet.

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx

docs/developer-docs/smart-contracts/best-practices/idempotency.mdx

…de.mdx Co-authored-by: Alin Sinpalean <[email protected]>

andrew-lee-work

LGTM, left some comments.

docs/developer-docs/security/security-best-practices/inter-canister-calls.mdx

docs/references/message-execution-properties.mdx

Co-authored-by: David Derler <[email protected]>

docs/developer-docs/backend/rust/intercanister.mdx

stiegerc · 2025-02-11T07:56:45Z

docs/developer-docs/backend/rust/intercanister.mdx


 ```bash
-dfx canister call subscriber setup_subscribe '(principal "<INSERT_PUBLISHER_PRINCIPAL_HERE>", "Apples")'
+$ dfx canister call caller stubborn_set '(42)'


Out of curiosity. Can you make a bounded_wait or unbounded_wait call specifically in dfx?

I'm not exactly sure what you mean, since dfx calls get their results by polling the ingress

docs/developer-docs/backend/rust/intercanister.mdx

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx

stiegerc · 2025-02-11T08:14:18Z

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx

+
+Best-effort calls address both of these problems. However, they can require more effort on the part of the caller to handle the additional error case. Here are some guidelines how to choose between the two types of calls:
+
+* Always prefer best-effort response calls for calls that don't change the state of the callee, i.e., reads.


reads -> queries? Or is there a difference I'm not aware of?

You can always mark a read as a query (though you don't have to)

docs/developer-docs/backend/rust/intercanister.mdx

derlerd-dfinity · 2025-02-14T13:54:08Z

One thing I just noticed: It seems we updated icrc1_ledger_usage but not the page right next to it: ICP ledger usage.

In this context we should also provide some pointers back and forth (to and from the tutorial page) in terms of how to interpret the related "unclean" errors as the ledger also uses panics as a tool for controlledly rolling back.

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx

Co-authored-by: Linwei Shang <[email protected]>

…de.mdx Co-authored-by: David Derler <[email protected]>

…de.mdx Co-authored-by: stiegerc <[email protected]>

mbjorkqvist · 2025-02-18T12:57:06Z

docs/developer-docs/defi/tokens/ledger/usage/icrc1_ledger_usage.mdx

+
+    let no_unknowns = true;
+    loop {
+        match Call::new(ledger, "icrc1_transfer")


Should this be Call::unbounded_wait instead of Call::new? If not, what's the difference between the two?

Yes it should be! Sorry for the confusion (Call:new is what the CDK alpha version at hand was actually using, so I was going back and forth when checking whether the call compiles, and this slipped through)

docs/developer-docs/backend/rust/intercanister.mdx

jessiemongeon1 · 2025-02-18T19:13:12Z

docs/developer-docs/backend/rust/intercanister.mdx


-A common problem in both distributed and decentralized systems is keeping separate services (or canisters) synchronized with one another. While there are many potential solutions to this problem, a popular one is the **publisher/subscriber** pattern, or "PubSub". PubSub is an especially valuable pattern on ICP as its primary drawback, message delivery failures, does not apply.
+We assume that you have the Internet Computer [developer tools](/docs/current/developer-docs/getting-started/install) installed. To follow along, clone the [Git repository](https://github.com/oggy-dfin/icc_rust_docs) of this tutorial. You can find the completed code in the `caller` package. Check out the Git repository of the tutorial. Then, in the tutorial directory, start a local IC instance in the background and install the counter canister.


Suggested change

We assume that you have the Internet Computer [developer tools](/docs/current/developer-docs/getting-started/install) installed. To follow along, clone the [Git repository](https://github.com/oggy-dfin/icc_rust_docs) of this tutorial. You can find the completed code in the `caller` package. Check out the Git repository of the tutorial. Then, in the tutorial directory, start a local IC instance in the background and install the counter canister.

To follow along, clone the [Git repository](https://github.com/oggy-dfin/icc_rust_docs) of this tutorial. You can find the completed code in the `caller` package.

Check out the Git repository of the tutorial. Then, in the tutorial directory, start a local IC instance in the background and install the counter canister:

I noticed that all the Rust docs have a pointer to the dev tools installation; are we getting rid of this?

jessiemongeon1 · 2025-02-18T19:14:01Z

docs/developer-docs/backend/rust/intercanister.mdx

-Before getting started, assure you have set up your developer environment according to the instructions in the [developer environment guide](./dev-env.mdx).
+## Basic example: getting and setting the counter value
+
+For the first example, we'll build a `call_get_and_set` method on the caller canister, which just invokes the `get_and_set` method on the counter project and forwards the result. This method sets the counter to the new value, and returns the previous counter value. The finished code for the project is already available in `src/caller`. You can clear the contents of `src/caller/src/lib.rs`, and paste the following code. The explanation is in the comments.


Suggested change

For the first example, we'll build a `call_get_and_set` method on the caller canister, which just invokes the `get_and_set` method on the counter project and forwards the result. This method sets the counter to the new value, and returns the previous counter value. The finished code for the project is already available in `src/caller`. You can clear the contents of `src/caller/src/lib.rs`, and paste the following code. The explanation is in the comments.

Our first example is a `call_get_and_set` method on the caller canister, which simply invokes the `get_and_set` method on the counter project and forwards the result. This method sets the counter to the new value and returns the previous counter value.

The finished code for the project is already available in `src/caller`. You can clear the contents of `src/caller/src/lib.rs`, and paste the following code. The explanation is within the comments.

Reworded the suggestion slightly

We do not use "we" or "our" in the Dev Docs. This is a stylistic choice from leadership.

I understand the general intent, but I don't see why we'd need to enforce this blindly. Clearly here it's "our" as "yours, the learner's, and mine, the teacher's", and not as "DFINITY's and definitely not yours". So I'd push back because I don't see the point of why we'd hurt readability, though it's certainly not a hill I'll die on.

From my understanding, this word choice was enforced due to legal concerns, so I'd like to adhere to it.

jessiemongeon1 · 2025-02-18T19:14:22Z

docs/developer-docs/backend/rust/intercanister.mdx

+
+For the first example, we'll build a `call_get_and_set` method on the caller canister, which just invokes the `get_and_set` method on the counter project and forwards the result. This method sets the counter to the new value, and returns the previous counter value. The finished code for the project is already available in `src/caller`. You can clear the contents of `src/caller/src/lib.rs`, and paste the following code. The explanation is in the comments.
+
+```rust


Can this example reference a GItHub link rather than inserting the code manually?

Yes, but it turns out that GH links are a pain to work with while writing the docs, both for the author and the reviewers. It's especially bad for the reviewers, because they can't see the actual code in the PR, and they can't comment on it unless the author has another PR on whatever repo the GH source is, plus they have to context switch. They're also a pain for the author, because to update an example you need to make a push to another branch somewhere, view the commit in GH, select the appropriate line numbers, and paste back.

I'm aware that I pushed for testable code samples myself :) I still think this is more important than author/reviewer convenience because documentation rots so quickly otherwise, and thanks for taking the time to make the GH plugin work! But given the current experience, my ideal solution would be to have the samples in the SAME repository as the portal, and ideally be able to just reference them using relative paths instead of having to deal with commits. Basically what this plugin claims to provide.

In the absence of the ideal solution, my current idea is to stick all the samples directly in the PR during the review process, and then replace them with links to a GH repo before committing.

We can reference GH files directly in the repo if they are pulled in as submodules, such as how we reference the files from the @dfinity/examples repo. In a perfect world, every example we have lives in that repo, but that is probably not realistic (or at least not currently).

As long as the plan is to replace the code with links before the PR is merged, that is fine :) Thank you!

If I referenced this through a submodule, the reviewers still wouldn't get to see the diff of the changes in the example, right?

No, but they would have a corresponding Example repo PR to reference that showed just those changes.

docs/developer-docs/backend/rust/intercanister.mdx

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx

Co-authored-by: Jessie Mongeon <[email protected]>

oggy-dfin added 6 commits December 6, 2024 17:07

WIP: adjust the docs to best-effort responses

f61fe5a

Adjust the safe retries doc

5f89f7e

Adjust security best practices

44806de

Improve the terminology in the message execution properties doc

410e042

Fix some typos

0613d32

oggy-dfin requested review from a team as code owners December 6, 2024 16:42

github-actions bot added the documentation Changes to Developer Docs label Dec 6, 2024

alin-at-dfinity reviewed Dec 9, 2024

View reviewed changes

derlerd-dfinity reviewed Dec 10, 2024

View reviewed changes

Some more improvements

d344cd9

jessiemongeon1 reviewed Dec 10, 2024

View reviewed changes

oggy-dfin and others added 10 commits December 11, 2024 14:43

Apply suggestions from code review

ddf5818

Co-authored-by: Jessie Mongeon <[email protected]>

Address David's and Jessie's comments

e213314

Update docs/references/message-execution-properties.mdx

7d82156

Co-authored-by: Alin Sinpalean <[email protected]>

Update docs/references/message-execution-properties.mdx

d545ab8

Co-authored-by: Alin Sinpalean <[email protected]>

Update docs/references/message-execution-properties.mdx

0132c93

Co-authored-by: Alin Sinpalean <[email protected]>

Update docs/references/message-execution-properties.mdx

9eb0ce2

Co-authored-by: Alin Sinpalean <[email protected]>

Update docs/references/message-execution-properties.mdx

7f4f09b

Co-authored-by: Alin Sinpalean <[email protected]>

Address Alin's comments (WIP)

d626bb4

Address Alin's comments

7da5a43

Minor improvements

dacf8bc

alin-at-dfinity approved these changes Dec 13, 2024

View reviewed changes

Update docs/developer-docs/smart-contracts/advanced-features/async-co…

7a4f424

…de.mdx Co-authored-by: Alin Sinpalean <[email protected]>

andrew-lee-work approved these changes Dec 13, 2024

View reviewed changes

oggy-dfin added 2 commits December 23, 2024 18:09

Andy's comments

2cd1f77

Merge branch 'master' into oggy/best-effort-responses

bf6cb13

oggy-dfin and others added 7 commits February 4, 2025 15:21

Update docs/developer-docs/backend/rust/intercanister.mdx

349a221

Co-authored-by: David Derler <[email protected]>

Update docs/developer-docs/backend/rust/intercanister.mdx

6971edd

Co-authored-by: David Derler <[email protected]>

Update intercanister.mdx

17c59d3

Revamp the inter-canister messaging tutorial completely

f18c6fb

Add the cycles example

022c1e2

Touch up some text

0d7cd05

Start the tutorial with a very basic example.

8a0f049

stiegerc reviewed Feb 11, 2025

View reviewed changes

Move to the new proposed CDK call error API and restructure again

5e40227

derlerd-dfinity reviewed Feb 12, 2025

View reviewed changes

docs/developer-docs/backend/rust/intercanister.mdx Outdated Show resolved Hide resolved

oggy-dfin added 4 commits February 12, 2025 17:56

Fix the page (stop from crashing)

419649a

Add summaries in info boxes

d18f034

Update ICRC-1 example and the inter-cansiter calls explanation

f59bb96

Fix the ICRC1 example

6ee732b

lwshang reviewed Feb 13, 2025

View reviewed changes

docs/developer-docs/backend/rust/intercanister.mdx Outdated Show resolved Hide resolved

Consistent naming (use (un)bounded-wait calls)

069994f

derlerd-dfinity reviewed Feb 14, 2025

View reviewed changes

docs/developer-docs/smart-contracts/advanced-features/async-code.mdx Outdated Show resolved Hide resolved

oggy-dfin and others added 5 commits February 17, 2025 11:11

Update docs/developer-docs/backend/rust/intercanister.mdx

9adbd3b

Co-authored-by: Linwei Shang <[email protected]>

Update docs/developer-docs/smart-contracts/advanced-features/async-co…

dd3c9b8

…de.mdx Co-authored-by: David Derler <[email protected]>

Improve wording

f337117

Touch up the documentation

1165a6f

Update docs/developer-docs/smart-contracts/advanced-features/async-co…

b437d6d

…de.mdx Co-authored-by: stiegerc <[email protected]>

mbjorkqvist reviewed Feb 18, 2025

View reviewed changes

jessiemongeon1 reviewed Feb 18, 2025

View reviewed changes

oggy-dfin and others added 5 commits February 19, 2025 15:12

Apply suggestions from code review

8cc7ca0

Co-authored-by: Jessie Mongeon <[email protected]>

Apply suggestions from code review

a063097

Co-authored-by: Jessie Mongeon <[email protected]>

Some minor language/keyword changes

3998d1a

Apply suggestions from code review

66f2bf6

Co-authored-by: Jessie Mongeon <[email protected]>

Update code examples to the new CDK

22db96a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add docs best effort responses #3865

Add docs best effort responses #3865

oggy-dfin commented Dec 6, 2024

github-actions bot commented Dec 6, 2024 •

edited

Loading

alin-at-dfinity left a comment

derlerd-dfinity left a comment

derlerd-dfinity Dec 10, 2024

oggy-dfin Dec 10, 2024

alin-at-dfinity Dec 13, 2024

oggy-dfin Dec 13, 2024

alin-at-dfinity Dec 13, 2024

alin-at-dfinity Dec 13, 2024

andrew-lee-work left a comment

stiegerc Feb 11, 2025

oggy-dfin Feb 12, 2025

stiegerc Feb 11, 2025

oggy-dfin Feb 17, 2025

derlerd-dfinity commented Feb 14, 2025

mbjorkqvist Feb 18, 2025

oggy-dfin Feb 19, 2025

jessiemongeon1 Feb 18, 2025

oggy-dfin Feb 19, 2025

jessiemongeon1 Feb 19, 2025

jessiemongeon1 Feb 18, 2025 •

edited by oggy-dfin

Loading

oggy-dfin Feb 19, 2025

jessiemongeon1 Feb 19, 2025

oggy-dfin Feb 20, 2025

jessiemongeon1 Feb 20, 2025

jessiemongeon1 Feb 18, 2025

oggy-dfin Feb 19, 2025

jessiemongeon1 Feb 19, 2025

oggy-dfin Feb 20, 2025

jessiemongeon1 Feb 20, 2025


		Best-effort calls address both of these problems. However, they can require more effort on the part of the caller to handle the additional error case. Here are some guidelines how to choose between the two types of calls:

		* Always prefer best-effort response calls for calls that don't change the state of the callee, i.e., reads.


		A common problem in both distributed and decentralized systems is keeping separate services (or canisters) synchronized with one another. While there are many potential solutions to this problem, a popular one is the publisher/subscriber pattern, or "PubSub". PubSub is an especially valuable pattern on ICP as its primary drawback, message delivery failures, does not apply.
		We assume that you have the Internet Computer [developer tools](/docs/current/developer-docs/getting-started/install) installed. To follow along, clone the [Git repository](https://github.com/oggy-dfin/icc_rust_docs) of this tutorial. You can find the completed code in the `caller` package. Check out the Git repository of the tutorial. Then, in the tutorial directory, start a local IC instance in the background and install the counter canister.

-For the first example, we'll build a `call_get_and_set` method on the caller canister, which just invokes the `get_and_set` method on the counter project and forwards the result. This method sets the counter to the new value, and returns the previous counter value. The finished code for the project is already available in `src/caller`. You can clear the contents of `src/caller/src/lib.rs`, and paste the following code. The explanation is in the comments.
+Our first example is a `call_get_and_set` method on the caller canister, which simply invokes the `get_and_set` method on the counter project and forwards the result. This method sets the counter to the new value and returns the previous counter value.
+The finished code for the project is already available in `src/caller`. You can clear the contents of `src/caller/src/lib.rs`, and paste the following code. The explanation is within the comments.


		For the first example, we'll build a `call_get_and_set` method on the caller canister, which just invokes the `get_and_set` method on the counter project and forwards the result. This method sets the counter to the new value, and returns the previous counter value. The finished code for the project is already available in `src/caller`. You can clear the contents of `src/caller/src/lib.rs`, and paste the following code. The explanation is in the comments.

		```rust

Add docs best effort responses #3865

Are you sure you want to change the base?

Add docs best effort responses #3865

Conversation

oggy-dfin commented Dec 6, 2024

github-actions bot commented Dec 6, 2024 • edited Loading

alin-at-dfinity left a comment

Choose a reason for hiding this comment

derlerd-dfinity left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrew-lee-work left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

derlerd-dfinity commented Feb 14, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessiemongeon1 Feb 18, 2025 • edited by oggy-dfin Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Dec 6, 2024 •

edited

Loading

jessiemongeon1 Feb 18, 2025 •

edited by oggy-dfin

Loading