-
Notifications
You must be signed in to change notification settings - Fork 117
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARC-46: Add end of the dynamic tuple to the head value #212
Changes from 2 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
--- | ||
arc: 39 | ||
title: Dynamic Tuple End In Head | ||
description: Append the end of a dynamic tuple to the head values | ||
author: Joe Polny (@joe-p) | ||
discussions-to: https://github.com/algorandfoundation/ARCs/issues/212 | ||
status: Draft | ||
type: Standards Track | ||
category: ARC | ||
created: 2023-07-15 | ||
requires: 4 | ||
--- | ||
|
||
## Abstract | ||
According to [ARC-0004](./arc-0004.md), when encoding dynamic types in a tuple the tuple consists of a `head` containing all of the offsets of the actual values in the `tail`. An additional value, the byte offset of the end of the tuple, should be added to the end of the `head` value for more efficient element reading. | ||
|
||
## Motivation | ||
When reading a value in a dynamic tuple, the head offset of the element must be extracted as a `uint16` to get the start of the element. To get the end of the element, which is necessary to properly read the element, the following head value can be extracted. This, however, fails to work when the element being read is the last element in its respective dynamic array or tuple. When the element is the last in the array or tuple, one must compute the length of the element. For static types, this is trivial and for dynamic arrays of static types this is also trivial because the length prefix can be used. For more complex nested dynamic types, however, this could involve many levels of extracting lengths and offsets, thus reading an element is `O(N)`, which `N` being the depth of the nested dynamic types. If there was an additional head value containing the end of the dynamic tuple, readnig any element, regardless of its type or position, would be `O(1)` since one can always just extract the following head value. | ||
|
||
## Specification | ||
The key words "**MUST**", "**MUST NOT**", "**REQUIRED**", "**SHALL**", "**SHALL NOT**", "**SHOULD**", "**SHOULD NOT**", "**RECOMMENDED**", "**MAY**", and "**OPTIONAL**" in this document are to be interpreted as described in <a href="https://www.ietf.org/rfc/rfc2119.txt">RFC-2119</a>. | ||
|
||
Tuples **MUST** be encoded with the following rules: | ||
|
||
* If `x` is a tuple of `N` types, `(T1,T2,...,TN)`, where `x[i]` is the value at index `i`, starting at 1 and `x[i]` is dynamic: | ||
* `head(x[i]) = enc(2 + len( head(x[1]) ... head(x[N]) tail(x[1]) ... tail(x[i-1]) ))` | ||
* `tail(x[i]) = enc(x[i])` | ||
* If `x` is a tuple of `N` types, `(T1,T2,...,TN)`, and any type is dynamic | ||
* `x = enc( head(x[i]) ... head(x[N]) + ( 2 + len( head(x[i]) ... head(x[N]) ) + len( tail(x[i]) ... tail(x[N]) ) ) + tail(x[i]) ... tail(x[N]) )` | ||
|
||
|
||
## Rationale | ||
This will make reading values from an ABI encoded tuple much more efficient, which is especially important for on-chain logic. | ||
|
||
## Backwards Compatibility | ||
[ARC-0004](./arc-0004.md) clients will be able to read tuples encoded according to this ARC, but they will not be able to write them. | ||
|
||
## Test Cases | ||
N/A | ||
|
||
## Reference Implementation | ||
|
||
Encoding of `["Hello", "World"]` as `string[]` | ||
|
||
[ARC-0004](./arc-0004.md): `0x0002 0004 000b 0005 48656c6c6f 0005 576f726c64` | ||
|
||
This ARC: `0x0002 0006 000d 0014 0005 48656c6c6f 0005 576f726c64` | ||
|
||
Note the addition of the `0014` signifying the end of the tuple/array and `head(x[1])` and `head[x[2]]` being incremented by two to account for this additional value. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If I understand correctly, this is not a great example because Can you provide an example where the benefit of including this additional information is clearer? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah this example was chosen to demonstrate the encoding rules, but it doesn't really show the rationale. Take a slightly more complex example:
To get the end of In this example, it exists but if there was further levels of nesting you would need to check if the parent element is the last element in it's parent and repeat until the next head is found. This means you need to spend opcodes checking the accessor against the prefix for every dynamic parent until you find the first parent tuple that has a dynamic element proceeding the accessed element. In the context of TEAL compilers, the compiler has no way of knowing whether the accessed element is last or not without using up those opcodes, unless only literal array access is supported. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Alternatively, you use the length prefix to calculate the length of the accessed element, but if that element is a nested dynamic type then you run into similar complexity of needing to calculate the total length of the last element each level. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why are you trying to get to the end of If your goal is to start decoding If you are not trying to reach You say it's hard "To get the end of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The goal is to get the value of
Which is great when There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I explained why substr'ing out the value is not necessary.
Surely it's even fewer opcodes to grab the suffix than to obtain the end (in any manner, whether by calculation or by pulling out the phantom length you want to add). |
||
## Security Considerations | ||
None | ||
|
||
## Copyright | ||
Copyright and related rights waived via <a href="https://creativecommons.org/publicdomain/zero/1.0/">CCO</a>. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see why this is true. Grab the suffix of the input, beginning at the start of the value. The value itself contains everything you need to read the right amount, starting there. There's no particular reason to worry, ahead of time, where the value ends, so just pass the entire suffix to whatever needs to decode it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the value is dynamic it doesn't contain "everything you need". If the value is a dynamic array the length prefix will tell you how many elements are in the array but not the length of those elements. You need to read the last value of the array and get the length for that. And if that value is a dynamic array, you must do the same thing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even if the elements are dynamic, their heads are statically sized, and laid out first. You know how many there are from the length. And, if they are dynamic, they will have pointers to their dynamically sized tails.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But in the case of nested dynamic types you have to continuously extract the head and calculate length. If we knew the end of the tuple it'd be much easier to use that to extract/substring.
To be clear, the desired goal here is to be able to efficiently extract a single value from an encoded nested dynamic type.