| Internet-Draft | +QUIC Acknowledgment Frequency | +November 2023 | +
| Iyengar, et al. | +Expires 2 June 2024 | +[Page] | +
This document describes a QUIC extension for an endpoint to control its peer's +delaying of acknowledgments.¶
+Discussion of this draft takes place on the QUIC working group mailing list +(quic@ietf.org), which is archived at +https://mailarchive.ietf.org/arch/search/?email_list=quic. Source +code and issues list for this draft can be found at +https://github.com/quicwg/ack-frequency.¶
+Working Group information can be found at https://github.com/quicwg.¶
++ This Internet-Draft is submitted in full conformance with the + provisions of BCP 78 and BCP 79.¶
++ Internet-Drafts are working documents of the Internet Engineering Task + Force (IETF). Note that other groups may also distribute working + documents as Internet-Drafts. The list of current Internet-Drafts is + at https://datatracker.ietf.org/drafts/current/.¶
++ Internet-Drafts are draft documents valid for a maximum of six months + and may be updated, replaced, or obsoleted by other documents at any + time. It is inappropriate to use Internet-Drafts as reference + material or to cite them other than as "work in progress."¶
++ This Internet-Draft will expire on 2 June 2024.¶
++ Copyright (c) 2023 IETF Trust and the persons identified as the + document authors. All rights reserved.¶
++ This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (https://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with + respect to this document. Code Components extracted from this + document must include Revised BSD License text as described in + Section 4.e of the Trust Legal Provisions and are provided without + warranty as described in the Revised BSD License.¶
+This document describes a QUIC extension for an endpoint to control its peer's +delaying of acknowledgments.¶
+The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this +document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] +when, and only when, they appear in all capitals, as shown here.¶
+In the rest of this document, "sender" refers to a QUIC data sender (and +acknowledgment receiver). Similarly, "receiver" refers to a QUIC data receiver +(and acknowledgment sender).¶
+This document uses terms, definitions, and notational conventions described in +Section 1.2 and Section 1.3 of [QUIC-TRANSPORT].¶
+A receiver acknowledges received packets, but it can delay sending these +acknowledgments. Delaying acknowledgments can impact connection +throughput, loss detection and congestion controller performance at a data +sender, and CPU utilization at both a data sender and a data receiver.¶
+Reducing the frequency of acknowledgments can improve connection and +endpoint performance in the following ways:¶
+Sending UDP packets can be very CPU intensive on some platforms. Reducing +the number of packets that only contain acknowledgments reduces the CPU +consumed at a data receiver. Experience shows that this reduction can be +critical for high bandwidth connections.¶
+Similarly, receiving and processing UDP packets can also be CPU intensive, and +reducing acknowledgment frequency reduces this cost at a data sender.¶
+For asymmetric link technologies, such as DOCSIS, LTE, and satellite, +connection throughput in the forward path can become constrained +when the reverse path is filled by acknowledgment packets [RFC3449]. +When traversing such links, reducing the number of acknowledgments can achieve +higher connection throughput, lower the impact on other flows or optimise the +overall use of transmission resources [Cus22].¶
+The rate of acknowledgment packets can reduce link efficiency, including +transmission opportunities or battery life, as well as transmission +opportunities available to other flows sharing the same link.¶
+As discussed in Section 8 however, there can be undesirable consequences +to congestion control and loss recovery if a receiver unilitaerally reduces the +acknowledgment frequency. A sender's constraints on the acknowledgment +frequency need to be taken into account to maximize congestion controller and +loss recovery performance.¶
+[QUIC-TRANSPORT] specifies a simple delayed acknowledgment mechanism that a +receiver can use: send an acknowledgment for every other packet, and for every +packet that is received out of order (Section 13.2.1 of [QUIC-TRANSPORT]). +This does not allow a sender to signal its preferences or constraints. This +extension provides a mechanism to solve this problem.¶
+Endpoints advertise their support of the extension described in this document by +sending the following transport parameter (Section 7.2 of [QUIC-TRANSPORT]):¶
+A variable-length integer representing the minimum amount of time in +microseconds by which the endpoint that is sending this value is able to +delay an acknowledgment. This limit could be based on the receiver's clock +or timer granularity. 'min_ack_delay' is used by the peer to avoid requesting +too small a value in the Request Max Ack Delay field of the ACK_FREQUENCY +frame.¶
+An endpoint's min_ack_delay MUST NOT be greater than its max_ack_delay. +Endpoints that support this extension MUST treat receipt of a min_ack_delay that +is greater than the received max_ack_delay as a connection error of type +TRANSPORT_PARAMETER_ERROR. Note that while the endpoint's max_ack_delay +transport parameter is in milliseconds (Section 18.2 of [QUIC-TRANSPORT]), +min_ack_delay is specified in microseconds.¶
+The min_ack_delay transport parameter is a unilateral indication of support for +receiving ACK_FREQUENCY frames. If an endpoint sends the transport parameter, +the peer is allowed to send ACK_FREQUENCY frames independent of whether it also +sends the min_ack_delay transport parameter or not.¶
+Until an ACK_FREQUENCY frame is received, sending the min_ack_delay transport +parameter does not cause the endpoint to change its acknowledgment behavior.¶
+Endpoints MUST NOT remember the value of the min_ack_delay transport parameter +they received for use in a subsequent connection. Consequently, ACK_FREQUENCY +frames cannot be sent in 0-RTT packets, as per Section 7.4.1 of [QUIC-TRANSPORT].¶
+This Transport Parameter is encoded as per Section 18 of [QUIC-TRANSPORT].¶
+Delaying acknowledgments as much as possible reduces both work done by the +endpoints and network load. An endpoint's loss detection and congestion control +mechanisms however need to be tolerant of this delay at the peer. An endpoint +signals the frequency it wants to receive ACK frames to its peer using an +ACK_FREQUENCY frame, shown below:¶
+
+ACK_FREQUENCY Frame {
+ Type (i) = 0xaf,
+ Sequence Number (i),
+ Ack-Eliciting Threshold (i),
+ Request Max Ack Delay (i),
+ Reordering Threshold (i),
+}
+¶
+Following the common frame format described in Section 12.4 of [QUIC-TRANSPORT], ACK_FREQUENCY frames have a type of 0xaf, and contain the +following fields:¶
+A variable-length integer representing the sequence number assigned to the +ACK_FREQUENCY frame by the sender to allow receivers to ignore obsolete +frames.¶
+A variable-length integer representing the maximum number of ack-eliciting +packets the recipient of this frame receives before sending an acknowledgment. +A receiving endpoint SHOULD send at least one ACK frame when more than this +number of ack-eliciting packets have been received. A value of 0 results in +a receiver immediately acknowledging every ack-eliciting packet. By default, an +endpoint sends an ACK frame for every other ack-eliciting packet, as specified in +Section 13.2.2 of [QUIC-TRANSPORT], which corresponds to a value of 1.¶
+A variable-length integer representing the value to which the endpoint
+requests the peer update its max_ack_delay
+(Section 18.2 of [QUIC-TRANSPORT]). The value of this field is in
+microseconds, unlike the 'max_ack_delay' transport parameter, which is in
+milliseconds. Sending a value smaller than the min_ack_delay advertised
+by the peer is invalid. Receipt of an invalid value MUST be treated as a
+connection error of type TRANSPORT_PARAMETER_ERROR. On receiving a valid value in
+this field, the endpoint MUST update its max_ack_delay to the value provided
+by the peer.¶
A variable-length integer that indicates the maximum packet +reordering before eliciting an immediate ACK, as specified in Section 6.2. +If no ACK_FREQUENCY frames have been received, the endpoint immediately +acknowledges any subsequent packets that are received out-of-order, as specified +in Section 13.2 of [QUIC-TRANSPORT], corresponding to a default value of 1. +A value of 0 indicates out-of-order packets do not elicit an immediate ACK.¶
+ACK_FREQUENCY frames are ack-eliciting and congestion controlled. When an +ACK_FREQUENCY frame is lost, the sender is encouraged to send another +ACK_FREQUENCY frame, unless an ACK_FREQUENCY frame with a larger Sequence Number +value has already been sent. However, it is not forbidden to retransmit the lost +frame (see Section 13.3 of [QUIC-TRANSPORT]), because the receiver will ignore +duplicate or out-of-order ACK_FREQUENCY frames based on the Sequence Number.¶
+An endpoint can send multiple ACK_FREQUENCY frames with different values within a +connection. A sending endpoint MUST send monotonically increasing values in the +Sequence Number field, since this field allows ACK_FREQUENCY frames to be processed +out of order. A receiving endpoint MUST ignore a received ACK_FREQUENCY frame +unless the Sequence Number value in the frame is greater than the largest currently +processed value.¶
+A sender can use an ACK_FREQUENCY frame to reduce the number of acknowledgments +sent by a receiver, but doing so increases the likelihood that time-sensitive +feedback is delayed as well. For example, as described in Section 8.3, delaying +acknowledgments can increase the time it takes for a sender to detect packet +loss. Sending an IMMEDIATE_ACK frame can help mitigate this problem.¶
+An IMMEDIATE_ACK frame can be useful in other situations as well. For example, +if a sender wants an immediate RTT measurement or if a sender wants to establish +receiver liveness as quickly as possible. PING frames +(Section 19.2 of [QUIC-TRANSPORT]) are ack-eliciting, but if a PING frame is +sent without an IMMEDIATE_ACK frame, the receiver might not immediately send +an ACK based on its local ACK strategy.¶
+By definition IMMEDIATE_ACK frames are ack-eliciting and they are also +congestion controlled. An endpoint SHOULD send a packet containing an ACK frame +immediately upon receiving an IMMEDIATE_ACK frame. An endpoint MAY delay sending +an ACK frame despite receiving an IMMEDIATE_ACK frame. For example, an endpoint +might do this if a large number of received packets contain an IMMEDIATE_ACK or +if the endpoint is under heavy load.¶
+
+IMMEDIATE_ACK Frame {
+ Type (i) = 0x1f,
+}
+¶
+Prior to receiving an ACK_FREQUENCY frame, endpoints send acknowledgments as +specified in Section 13.2.1 of [QUIC-TRANSPORT].¶
+On receiving an ACK_FREQUENCY frame and updating its max_ack_delay
+and Ack-Eliciting Threshold values (Section 4), the endpoint sends an
+acknowledgment when one of the following conditions are met:¶
Since the last acknowledgment was sent, the number of received ack-eliciting
+packets is greater than the Ack-Eliciting Threshold.¶
Since the last acknowledgment was sent, max_ack_delay amount of time has
+passed.¶
Section 6.2, Section 6.4, and Section 6.5 describe exceptions to this +strategy.¶
+It is important to receive timely feedback after long idle periods, +e.g. update stale RTT measurements. When no acknowledgment has been sent in +over one smoothed round trip time, receivers are encouraged to send an +acknowledgment soon after receiving an ack-eliciting packet. This is not an +issue specific to this document, but the mechanisms specified herein could +create excessive delays.¶
+As specified in Section 13.2.1 of [QUIC-TRANSPORT], endpoints are expected to +send an acknowledgment immediately on receiving a reordered ack-eliciting +packet. This extension modifies that behavior when an ACK_FREQUENCY frame with +a Reordering Threshold value other than 1 has been received.¶
+If the most recent ACK_FREQUENCY frame received from the peer has a Reordering
+Threshold value of 0, the endpoint SHOULD NOT send an immediate
+acknowledgment in response to packets received out of order, and instead
+rely on the peer's Ack-Eliciting Threshold and max_ack_delay thresholds
+for sending acknowledgments.¶
If the most recent ACK_FREQUENCY frame received from the peer has a Reordering
+Threshold value larger than 1, the endpoint tests the amount of reordering
+before deciding to send an acknowledgment. The specification uses the following
+definitions:¶
The largest packet number among all received ack-eliciting packets.¶
+The Largest Acknowledged value sent in an ACK frame.¶
+The largest packet number that could be declared lost with the specified +Reordering Threshold, which is Largest Acked - Reordering Threshold + 1.¶
+Packets with packet numbers between the Largest Unacked and Largest Reported +that have not yet been received.¶
+An endpoint that receives an ACK_FREQUENCY frame with a non-zero Reordering
+Threshold value SHOULD send an immediate ACK when the gap
+between the smallest Unreported Missing packet and the Largest Unacked is greater
+than or equal to the Reordering Threshold value. Sending this additional ACK will
+reset the max_ack_delay timer and Ack-Eliciting Threshold counter (as any ACK
+would do).¶
See Section 6.2.1 for examples explaining this behavior. See Section 6.3 +for guidance on how to choose the reordering threshold value when sending +ACK_FREQUENCY frames.¶
+When the reordering threshold is 1, any time a packet is received +and there is a missing packet, an immediate ACK is sent.¶
+If the reordering theshold is 3 and ACKs are only sent due to reordering:¶
++ Receive 1 + Receive 3 -> 2 Missing + Receive 4 -> 2 Missing + Receive 5 -> Send ACK because of 2 + Receive 8 -> 6,7 Missing + Receive 9 -> Send ACK because of 6, 7 Missing + Receive 10 -> Send ACK because of 7 +¶ +
If the reordering threshold is 5 and ACKs are only sent due to reordering:¶
++ Receive 1 + Receive 3 -> 2 Missing + Receive 5 -> 2 Missing, 4 Missing + Receive 6 -> 2 Missing, 4 Missing + Receive 7 -> Send ACK because of 2, 4 Missing + Receive 8 -> 4 Missing + Receive 9 -> Send ACK because of 4 +¶ +
To ensure timely loss detection, it is optimal to send a Reordering +Threshold value of 1 less than the packet threshold used by the data sender for +loss detection. If the threshold is smaller, an ACK_FRAME is sent before the +packet can be declared lost based on the packet threshold. If the value is +larger, it causes unnecessary delays. (Section 18.2 of [QUIC-RECOVERY]) +recommends a default packet threshold for loss detection of 3, equivalent to +a Reordering Threshold of 2.¶
+An endpoint SHOULD send an immediate acknowledgment when a packet marked +with the ECN Congestion Experienced (CE) [RFC3168] codepoint in the IP +header is received and the previously received packet was not marked CE.¶
+Doing this maintains the peer's response time to congestion events, while also +reducing the ACK rate compared to Section 13.2.1 of [QUIC-TRANSPORT] during +extreme congestion or when peers are using DCTCP [RFC8257] or other +congestion controllers (e.g. [I-D.ietf-tsvwg-aqm-dualq-coupled]) that mark +more frequently than classic ECN [RFC3168].¶
+To avoid sending multiple acknowledgments in rapid succession, an endpoint can +process all packets in a batch before determining whether to send an ACK frame +in response, as stated in Section 13.2.2 of [QUIC-TRANSPORT].¶
+On sending an update to the peer's max_ack_delay, an endpoint can use this new
+value in later computations of its Probe Timeout (PTO) period; see Section 5.2.1 of [QUIC-RECOVERY].¶
Until the packet carrying this frame is acknowledged, the endpoint MUST use the
+greater of the current max_ack_delay and the value that is in flight when
+computing the PTO period. Doing so avoids spurious PTOs that can be caused by an
+update that decreases the peer's max_ack_delay.¶
While it is expected that endpoints will have only one ACK_FREQUENCY frame in
+flight at any given time, this extension does not prohibit having more than one
+in flight. When using max_ack_delay for PTO computations, endpoints MUST use
+the maximum of the current value and all those in flight.¶
When the number of in-flight ack-eliciting packets is larger than the
+ACK-Eliciting Threshold, an endpoint can expect that the peer will not need to
+wait for its max_ack_delay period before sending an acknowledgment. In such
+cases, the endpoint MAY therefore exclude the peer's 'max_ack_delay' from its PTO
+calculation. When Reordering Threshold is set to 0 and loss causes the peer to
+not receive enough packets to trigger an immediate acknowledgment, the receiver
+will wait 'max_ack_delay', increasing the chances of a premature PTO.
+Therefore, if Reordering Threshold is set to 0, the PTO MUST be larger than the
+peer's 'max_ack_delay'.¶
When sending PTO packets, one can include an IMMEDIATE_ACK frame to elicit an +immediate acknowledgment. This avoids waiting the ack delay for +acknowledgments of PTO packets, reducing tail latency and allowing the sender +to exclude the peer's 'max_ack_delay' from subsequent PTO calculations.¶
+This section provides some guidance on a sender's choice of acknowledgment +frequency and discusses some additional considerations. Implementers can select +an appropriate strategy to meet the needs of their applications and congestion +controllers.¶
+A sender needs to be responsive to notifications of congestion, such as +a packet loss or an ECN CE marking. Decreasing the acknowledgment frequency +can delay a sender's response to network congestion or cause it to underutilize +the available bandwidth.¶
+To limit the consequences of reduced acknowledgment frequency, a sender +can cause a receiver to send an acknowledgment at least once per round trip +when there are ack-eliciting packets in flight.¶
+A sender can accomplish this by setting the Request Max Ack +Delay value to no more than the estimated round trip time. +The sender can also improve feedback and robustness to +variation in the path RTT by setting the Ack-Eliciting Threshold +to a value no larger than the current congestion window. Alternatively, +a sender can accomplish this by sending an IMMEDIATE_ACK frame once each +round trip time, although if the packet containing an IMMEDIATE_ACK is lost, +detection of that loss will be delayed by the reordering threshold or requested +max ack delay.¶
+When setting the Request Max Ack Delay as a function of RTT, it is usually +better to use the Smoothed RTT (Section 5.3 of [QUIC-RECOVERY]) or another +estimate of the typical round trip time, but not Min RTT. This avoids eliciting an +unnecessarily high number of acknowledgments when min_rtt is much smaller than +smoothed_rtt.¶
+Note that the congestion window and the RTT change over the lifetime of a +connection and therefore might require sending frequent ACK_FREQUENCY frames to +ensure optimal performance.¶
+It is possible that the RTT is smaller than the receiver's timer granularity, +as communicated via the 'min_ack_delay' transport parameter, preventing the +receiver from sending an acknowledgment every RTT in time unless packets are +acknowledged immediately. In these cases, Reordering Threshold values other than 1 +can delay loss detection more than an RTT.¶
+A congestion controller that is limited by the congestion window relies upon receiving +acknowledgments to send additional data into the network. An increase in +acknowledgment delay increases the delay in sending data, which can reduce the +achieved throughput. Congestion window growth can also depend upon receiving +acknowledgments. This can be particularly significant in slow start +(Section 7.3.1 of [QUIC-RECOVERY]), when delaying acknowledgments can delay +the increase in congestion window and can create larger packet bursts.¶
+If the sender is application-limited, acknowledgments can be delayed +unnecessarily when entering idle periods. Therefore, if no further data is +buffered to be sent, a sender can send an IMMEDIATE_ACK frame with the last data +packet before an idle period to avoid waiting for the ack delay.¶
+If there are no inflight packets, no acknowledgments will be received for at least +a round trip when sending resumes. The Max Ack Delay and Ack-Eliciting Threshold +values used by the receiver can further delay acknowledgments. In this case, the +sender can include an IMMEDIATE_ACK or ACK_FREQUENCY frame in the first +Ack-Eliciting packet to avoid waiting for substantially more than a round trip +for an acknowledgment.¶
+Receiving an acknowledgment can allow a sender to release new packets into the +network. If a sender is designed to rely on the timing of peer acknowledgments +("ACK clock"), delaying acknowledgments can cause undesirable bursts of data +into the network. In keeping with Section 7.7 of [QUIC-RECOVERY], a sender +can either employ pacing or limit bursts to the initial congestion window.¶
+Acknowledgments are fundamental to reliability in QUIC. Consequently, +delaying or reducing the frequency of acknowledgments can cause loss detection +at the sender to be delayed.¶
+A QUIC sender detects loss using packet thresholds on receiving an +acknowledgment (Section 6.1.1 of [QUIC-RECOVERY]); delaying the +acknowledgment therefore delays this method of detecting losses.¶
+Reducing acknowledgment frequency reduces the number of RTT samples that a +sender receives (Section 5 of [QUIC-RECOVERY]), making a sender's RTT estimate +less responsive to changes in the path's RTT. As a result, any mechanisms that +rely on an accurate RTT estimate, such as time-threshold loss detection (Section +6.1.2 of [QUIC-RECOVERY]) or Probe Timeout (Section 6.2 of [QUIC-RECOVERY]), +will be less responsive to changes in the path's RTT, resulting in either +delayed or unnecessary packet transmissions.¶
+A sender might use timers to detect loss of PMTU probe packets +(Section 14 of [QUIC-TRANSPORT]). A sender MAY +bundle an IMMEDIATE_ACK frame with any PMTU probes to avoid triggering such +timers.¶
+To avoid additional delays to connection migration confirmation when using this +extension, a client can bundle an IMMEDIATE_ACK frame with the first non-probing +frame (Section 9.2 of [QUIC-TRANSPORT]) it sends or it can send only an +IMMEDIATE_ACK frame, which is a non-probing frame.¶
+An endpoint's congestion controller and RTT estimator are reset upon +confirmation of migration (Section 9.4 of [QUIC-TRANSPORT]); +this changes the pattern of acknowledgments received after migration.¶
+Therefore, an endpoint that has sent an ACK_FREQUENCY frame earlier in the +connection ought to send a new ACK_FREQUENCY frame upon confirmation of +connection migration with updated information, e.g. to consider the new RTT estimate.¶
+An improperly configured or malicious data sender could cause a +data receiver to acknowledge more frequently than its available resources +permit. However, there are two limits that make such an attack largely +inconsequential. First, the acknowledgment rate is bounded by the rate at which +data is received. Second, ACK_FREQUENCY and IMMEDIATE_ACK frames can only request +an increase in the acknowledgment rate, but cannot force it.¶
+In general, with this extension, a sender cannot force a receiver to acknowledge +more frequently than the receiver considers safe based on its resource constraints.¶
+This document defines a new transport parameter to advertise support of the +extension described in this document and two new frame types to registered +by IANQ in the respective "QUIC Protocol" registries under +https://www.iana.org/assignments/quic/quic.xhtml.¶
+The following entry in Table 1 has been requested to be provisionally added to +the "QUIC Transport Parameters" registry under the "QUIC Protocol" heading.¶
+| Value | +Parameter Name. | +Specification | +
|---|---|---|
| 0xff04de1b | +min_ack_delay | ++ Section 3 + | +
When this document is approved, IANA is requested to assign a permanent allocation +of a codepoint in the 0-63 range to replace the provisional codepoint described above.¶
+The following frame types have requested to be provisionally added to the "QUIC Frame Types" +registry under the "QUIC Protocol" heading.¶
+| Value | +Frame Name | +Specification | +
|---|---|---|
| 0xaf | +ACK_FREQUENCY | ++ Section 4 + | +
| 0x1f | +IMMEDIATE_ACK | ++ Section 5 + | +
When this document is approved, IANA is requested to change the registration to +a permanent allocation of these frame types with the values described above.¶
+RFC Editor's Note: Please remove this section prior to publication of a +final version of this document.¶
+The following people directly contributed key ideas that shaped this draft: +Bob Briscoe, Kazuho Oku, Marten Seemann.¶
+| QUIC Acknowledgment Frequency | +plain text | +same as main | +
| QUIC Acknowledgement Frequency | -plain text | +QUIC Acknowledgment Frequency | +plain text | datatracker | diff with last submission |
| QUIC Acknowledgment Frequency | +plain text | +diff with main | +
| QUIC Acknowledgement Frequency | -plain text | +QUIC Acknowledgment Frequency | +plain text | diff with main |