Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gnrc/ipv6/nib: don't queue packets on 6lo neighbors and drop/flush if… #20834

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
11 changes: 11 additions & 0 deletions sys/include/net/gnrc/ipv6/nib/conf.h
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,17 @@ extern "C" {
#define CONFIG_GNRC_IPV6_NIB_QUEUE_PKT 1
#endif

#if CONFIG_GNRC_IPV6_NIB_QUEUE_PKT
/**
* @brief queue capacity for the packets waiting for address resolution,
* per neighbor. SHOULD always be smaller than @ref CONFIG_GNRC_IPV6_NIB_NUMOF
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHOULD always be smaller than @ref CONFIG_GNRC_IPV6_NIB_NUMOF

Why should that be?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because if it's >= CONFIG_GNRC_IPV6_NIB_NUMOF, then even for a single neighbor, you can't even get to drop old packets from it's queue because you can't allocate a new one in the first place. With multiple neighbors, this can happen anyway, so this is why I went for a SHOULD instead of MUST.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah

#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_QUEUE_PKT)
static gnrc_pktqueue_t _queue_pool[CONFIG_GNRC_IPV6_NIB_NUMOF];
#endif  /* CONFIG_GNRC_IPV6_NIB_QUEUE_PKT */

That is not just an array of queues, but rather every packet to be queued has to get a slot here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP is the queue capacity per neighbor.
I would define this as 1 by default and change the upper code snipped to:

#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_QUEUE_PKT)
static gnrc_pktqueue_t _queue_pool[CONFIG_GNRC_IPV6_NIB_NUMOF * CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP];
#endif  /* CONFIG_GNRC_IPV6_NIB_QUEUE_PKT */

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm but it would be bad if packets cannot be queued even if there are 15 available slots because only one slot per neighbor is allowed. The current definition nevertheless looks a bit strange to me.

What about a neighbor is not allowed to have more than halve of the slots?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure we still need space in pktbuf to actually send out the neighbor solicitation (and receive the response), but that isn't deepening on the number of queued packets but their size - or do I misunderstand something here?

I think this is a different problem. What I'm trying to solve is the scenario where a neighbor gets flooded with packets and we run out of free queue entries. In that case, the neighbor(s) queue(s) will be filled with stale packets because we can't even _alloc_queue_entry() in order to drop the oldest packets.

Then one neighbor could take all the slots and at some point _alloc_queue_entry() will always fail.

AFAIK we should always drop the oldest, but that can't be done if we can't allocate in the first place.

The current capping solution doesn't guarantee this scenario won't happen, just makes it less probable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also note that the default number of free queue entries is CONFIG_GNRC_IPV6_NIB_NUMOF == 16, which is rather small.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could as well drop the oldest packet, as soon as allocation fails and try to allocate again.
It could be that 16 slots are held by one neighbor and another neighbor for which allocation fails does not have an oldest packet to drop, so allocation fails again because one neighbor holds all the slots.

With your capacity limit per neighbor you want to make this case less likely, but also do not prevent this case.
Yes the idea is good, but at the same time you accept that a packet is not queued even though a slot could be allocated.

If I got that right I am not sure if this is really beneficial as long as we don´t note an issue that packets cannot be queued because one neighbor is taking most of the queue slots.
Did you observe this case?

Copy link
Contributor Author

@derMihai derMihai Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I got that right I am not sure if this is really beneficial as long as we don´t note an issue that packets cannot be queued because one neighbor is taking most of the queue slots.
Did you observe this case?

No, I only had issues with one host.

What bothers me most isn't failed allocations, but stale packets in a neighbor's queue. This goes against the first part of be strict when sending and tolerant when receiving.

It could be that 16 slots are held by one neighbor and another neighbor for which allocation fails does not have an oldest packet to drop, so allocation fails again because one neighbor holds all the slots.

This is partially true. We have the static _nib_onl_entry_t _nodes[CONFIG_GNRC_IPV6_NIB_NUMOF]; . We could iterate through that and drop either from the first neighbor (faster) or the one with the largest queue (may be slower for large CONFIG_GNRC_IPV6_NIB_NUMOF). Anyway, I don't see iterating through that list as a performance problem. We already do so when searching for free packets, adding one more run isn't changing much.

Copy link
Contributor Author

@derMihai derMihai Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the per-neighbor queue cap. Packet allocation now never fails, it just pops a packet from the neighbor with the most packets in its queue. I made the queue entry count one more than the neighbor count. That way, there must always be a neighbor with at least two packets in it's queue, so we never leave it packet-less.

*/
#ifndef CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP
#define CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP (CONFIG_GNRC_IPV6_NIB_NUMOF > 16 ? 16 : 1)
Copy link
Contributor

@benpicco benpicco Aug 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that drop is a bit strange. Up to 16 entries, each entry can queue only a single packet, but once there are 17 entries, we can queue 16 per entry?

How about

Suggested change
#define CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP (CONFIG_GNRC_IPV6_NIB_NUMOF > 16 ? 16 : 1)
#define CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP DIV_ROUND_UP(CONFIG_GNRC_IPV6_NIB_NUMOF, 2)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's fine, I don't know how probable is resolution of multiple neighbors at the same time.

#endif

#endif /* CONFIG_GNRC_IPV6_NIB_QUEUE_PKT */

/**
* @brief handle NDP messages according for stateless address
* auto-configuration (if activated on interface)
Expand Down
19 changes: 13 additions & 6 deletions sys/net/gnrc/network_layer/ipv6/nib/_nib-arsm.c
Original file line number Diff line number Diff line change
Expand Up @@ -228,9 +228,7 @@ void _handle_snd_ns(_nib_onl_entry_t *nbr)
case GNRC_IPV6_NIB_NC_INFO_NUD_STATE_PROBE:
if (nbr->ns_sent >= NDP_MAX_UC_SOL_NUMOF) {
gnrc_netif_t *netif = gnrc_netif_get_by_pid(_nib_onl_get_if(nbr));

_set_nud_state(netif, nbr,
GNRC_IPV6_NIB_NC_INFO_NUD_STATE_UNREACHABLE);
_set_unreachable(netif, nbr);
}
/* intentionally falls through */
case GNRC_IPV6_NIB_NC_INFO_NUD_STATE_UNREACHABLE:
Expand Down Expand Up @@ -387,11 +385,11 @@ void _handle_adv_l2(gnrc_netif_t *netif, _nib_onl_entry_t *nce,
nce->info &= ~GNRC_IPV6_NIB_NC_INFO_IS_ROUTER;
}
}
#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_QUEUE_PKT) && MODULE_GNRC_IPV6
#if MODULE_GNRC_IPV6
/* send queued packets */
gnrc_pktqueue_t *ptr;
DEBUG("nib: Sending queued packets\n");
while ((ptr = gnrc_pktqueue_remove_head(&nce->pktqueue)) != NULL) {
while ((ptr = _nbr_pop_pkt(nce)) != NULL) {
if (!gnrc_netapi_dispatch_send(GNRC_NETTYPE_IPV6,
GNRC_NETREG_DEMUX_CTX_ALL,
ptr->pkt)) {
Expand All @@ -400,7 +398,7 @@ void _handle_adv_l2(gnrc_netif_t *netif, _nib_onl_entry_t *nce,
}
ptr->pkt = NULL;
}
#endif /* CONFIG_GNRC_IPV6_NIB_QUEUE_PKT */
#endif /* MODULE_GNRC_IPV6 */
if ((icmpv6->type == ICMPV6_NBR_ADV) &&
!_sflag_set((ndp_nbr_adv_t *)icmpv6) &&
(_get_nud_state(nce) == GNRC_IPV6_NIB_NC_INFO_NUD_STATE_REACHABLE) &&
Expand Down Expand Up @@ -439,6 +437,15 @@ void _set_reachable(gnrc_netif_t *netif, _nib_onl_entry_t *nce)
netif->ipv6.reach_time);
}

void _set_unreachable(gnrc_netif_t *netif, _nib_onl_entry_t *nce)
{
DEBUG("nib: set %s to UNREACHABLE\n",
ipv6_addr_to_str(addr_str, &nce->ipv6, sizeof(addr_str)));

_nbr_flush_pktqueue(nce);
derMihai marked this conversation as resolved.
Show resolved Hide resolved
_set_nud_state(netif, nce, GNRC_IPV6_NIB_NC_INFO_NUD_STATE_UNREACHABLE);
}

void _set_nud_state(gnrc_netif_t *netif, _nib_onl_entry_t *nce,
uint16_t state)
{
Expand Down
12 changes: 10 additions & 2 deletions sys/net/gnrc/network_layer/ipv6/nib/_nib-arsm.h
Original file line number Diff line number Diff line change
Expand Up @@ -174,14 +174,22 @@ void _handle_adv_l2(gnrc_netif_t *netif, _nib_onl_entry_t *nce,
void _recalc_reach_time(gnrc_netif_ipv6_t *netif);

/**
* @brief Sets a neighbor cache entry reachable and starts the required
* @brief Sets a neighbor cache entry REACHABLE and starts the required
* event timers
*
* @param[in] netif Interface to the NCE
* @param[in] nce The neighbor cache entry to set reachable
* @param[in] nce The neighbor cache entry to set REACHABLE
*/
void _set_reachable(gnrc_netif_t *netif, _nib_onl_entry_t *nce);

/**
* @brief Sets a neighbor cache entry UNREACHABLE and flushes its packet queue
*
* @param[in] netif Interface to the NCE
* @param[in] nce The neighbor cache entry to set UNREACHABLE
*/
void _set_unreachable(gnrc_netif_t *netif, _nib_onl_entry_t *nce);

/**
* @brief Initializes interface for address registration state machine
*
Expand Down
13 changes: 1 addition & 12 deletions sys/net/gnrc/network_layer/ipv6/nib/_nib-internal.c
Original file line number Diff line number Diff line change
Expand Up @@ -276,18 +276,7 @@ void _nib_nc_remove(_nib_onl_entry_t *node)
#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_6LR)
evtimer_del((evtimer_t *)&_nib_evtimer, &node->addr_reg_timeout.event);
#endif /* CONFIG_GNRC_IPV6_NIB_6LR */
#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_QUEUE_PKT)
gnrc_pktqueue_t *tmp;
for (gnrc_pktqueue_t *ptr = node->pktqueue;
(ptr != NULL) && (tmp = (ptr->next), 1);
ptr = tmp) {
gnrc_pktqueue_t *entry = gnrc_pktqueue_remove(&node->pktqueue, ptr);
gnrc_icmpv6_error_dst_unr_send(ICMPV6_ERROR_DST_UNR_ADDR,
entry->pkt);
gnrc_pktbuf_release_error(entry->pkt, EHOSTUNREACH);
entry->pkt = NULL;
}
#endif /* CONFIG_GNRC_IPV6_NIB_QUEUE_PKT */
_nbr_flush_pktqueue(node);
/* remove from cache-out procedure */
clist_remove(&_next_removable, (clist_node_t *)node);
_nib_onl_clear(node);
Expand Down
36 changes: 36 additions & 0 deletions sys/net/gnrc/network_layer/ipv6/nib/_nib-internal.h
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ typedef struct _nib_onl_entry {
* @note Only available if @ref CONFIG_GNRC_IPV6_NIB_QUEUE_PKT != 0.
*/
gnrc_pktqueue_t *pktqueue;
size_t pktqueue_len; /**< Number of queued packets */
#endif
/**
* @brief Neighbors IPv6 address
Expand Down Expand Up @@ -872,6 +873,41 @@ void _nib_ft_get(const _nib_offl_entry_t *dst, gnrc_ipv6_nib_ft_t *fte);
int _nib_get_route(const ipv6_addr_t *dst, gnrc_pktsnip_t *ctx,
gnrc_ipv6_nib_ft_t *entry);

#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_QUEUE_PKT) || DOXYGEN
/**
* @brief Flush the packet queue of a on-link neighbor.
*
* @param node neighbor entry to be flushed
*/
void _nbr_flush_pktqueue(_nib_onl_entry_t *node);

/**
* @brief Remove oldest packet from a on-link neighbor's packet queue.
*
* @param node neighbor entry
*
* @retval pointer to the packet entry or NULL if the queue is empty
*/
gnrc_pktqueue_t *_nbr_pop_pkt(_nib_onl_entry_t *node);

/**
* @brief Push packet to a on-link neighbor's packet queue.
*
* @pre Neighbor is INCOMPLETE.
*
* @note If the queue size is @ref CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP,
* this will @ref _nbr_pop_pkt() the oldest packet and release it.
*
* @param node neighbor entry
* @param pkt packet to be pushed
*/
void _nbr_push_pkt(_nib_onl_entry_t *node, gnrc_pktqueue_t *pkt);
#else
#define _nbr_flush_pktqueue(node) ((void)node)
#define _nbr_pop_pkt(node) ((void)node, NULL)
#define _nbr_push_pkt(node, pkt) ((void)node, (void)pkt)
#endif
benpicco marked this conversation as resolved.
Show resolved Hide resolved

#ifdef __cplusplus
}
#endif
Expand Down
65 changes: 56 additions & 9 deletions sys/net/gnrc/network_layer/ipv6/nib/nib.c
Original file line number Diff line number Diff line change
Expand Up @@ -532,7 +532,7 @@
icmpv6_len -= (opt->len << 3), \
opt = (ndp_opt_t *)(((uint8_t *)opt) + (opt->len << 3)))

#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_ROUTER)

Check warning on line 535 in sys/net/gnrc/network_layer/ipv6/nib/nib.c

View workflow job for this annotation

GitHub Actions / static-tests

full block {} expected in the control structure
static void _handle_rtr_sol(gnrc_netif_t *netif, const ipv6_hdr_t *ipv6,
const ndp_rtr_sol_t *rtr_sol, size_t icmpv6_len)
{
Expand Down Expand Up @@ -1337,11 +1337,7 @@
queue_entry->pkt = gnrc_pkt_prepend(queue_entry->pkt, netif_hdr);
}

#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_QUEUE_PKT)
gnrc_pktqueue_add(&entry->pktqueue, queue_entry);
#else
(void)entry;
#endif
_nbr_push_pkt(entry, queue_entry);
return true;
}

Expand Down Expand Up @@ -1373,6 +1369,16 @@
return true;
}

/* don't do multicast address resolution on 6lo */
if (gnrc_netif_is_6ln(netif)) {
/* https://www.rfc-editor.org/rfc/rfc6775.html#section-5.6
* A LoWPAN node is not required to maintain a minimum of one buffer
* per neighbor as specified in [RFC4861], since packets are never
* queued while waiting for address resolution. */
gnrc_pktbuf_release(pkt);
return false;
}

bool reset = false;
DEBUG("nib: resolve address %s by probing neighbors\n",
ipv6_addr_to_str(addr_str, dst, sizeof(addr_str)));
Expand Down Expand Up @@ -1404,10 +1410,7 @@
return false;
}

/* don't do multicast address resolution on 6lo */
if (!gnrc_netif_is_6ln(netif)) {
_probe_nbr(entry, reset);
}
_probe_nbr(entry, reset);

return false;
}
Expand Down Expand Up @@ -1733,4 +1736,48 @@

return route_ltime;
}

#if IS_ACTIVE(CONFIG_GNRC_IPV6_NIB_QUEUE_PKT)
gnrc_pktqueue_t *_nbr_pop_pkt(_nib_onl_entry_t *node)
{
if (node->pktqueue_len == 0) {
assert(node->pktqueue == NULL);
return NULL;
}
assert(node->pktqueue != NULL);

node->pktqueue_len--;

return gnrc_pktqueue_remove_head(&node->pktqueue);
}

void _nbr_push_pkt(_nib_onl_entry_t *node, gnrc_pktqueue_t *pkt)
{
assert(_get_nud_state(node) == GNRC_IPV6_NIB_NC_INFO_NUD_STATE_INCOMPLETE);

assert(node->pktqueue_len <= CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP);
if (node->pktqueue_len == CONFIG_GNRC_IPV6_NIB_QUEUE_PKT_CAP) {
gnrc_pktqueue_t *oldest = _nbr_pop_pkt(node);

assert(oldest != NULL);

gnrc_icmpv6_error_dst_unr_send(ICMPV6_ERROR_DST_UNR_ADDR, oldest->pkt);
gnrc_pktbuf_release_error(oldest->pkt, ENOBUFS);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we should drop with ENOBUFS or silently.

oldest->pkt = NULL;
}

gnrc_pktqueue_add(&node->pktqueue, pkt);
node->pktqueue_len++;
}

void _nbr_flush_pktqueue(_nib_onl_entry_t *node)
{
gnrc_pktqueue_t *entry;
while ((entry = _nbr_pop_pkt(node))) {
gnrc_icmpv6_error_dst_unr_send(ICMPV6_ERROR_DST_UNR_ADDR, entry->pkt);
gnrc_pktbuf_release_error(entry->pkt, EHOSTUNREACH);
entry->pkt = NULL;
}
}
#endif /* CONFIG_GNRC_IPV6_NIB_QUEUE_PKT */
/** @} */
Loading