How to implement NUMA in CHI mesh network

[qqli579]

hi, all:
I saw in the commit https://gem5.atlassian.net/browse/GEM5-1187 that has implemented support for numa. But I have a question: is this submission targeted at 1 numa node? If I have 4 numa nodes, do I need to make corresponding modifications?
For example, I want to implement a 4X4 chi mesh network. Among them, node 0, 1, 2, 3, 12, 13, 14, 15 are HNF nodes. Then divide the 16 nodes into 4 numa nodes: nodes 0,1,4,5 are assigned to numa node 0, nodes 2,3,6,7 are assigned to numa node 1, nodes 8,9,12,13 are assigned to numa node 2, nodes 10,11,14,15 are assigned to numa node 3.
First I created 4 addr_range in riscv board.py

def _setup_memory_ranges(self):
    memory = self.get_memory()
    mem_size = memory.get_size()
    start_addr = 0x80000000
    size = mem_size // 4

    for i in range(4):
        addr_range = AddrRange(start=start_addr, size=size)
        self.mem_ranges.append(addr_range)

        start_addr += size

    memory.set_memory_range(self.mem_ranges)

nodes 0, 1 are assigned to addr_range[0], nodes 2, 3 are assigned to addr_range[1], nodes 12, 13 are assigned to addr_range[2], nodes14, 15 are assigned to addr_range[3].
Finally modify the following function in chi_config.py

def createNumaAddrRanges(cls, sys_mem_ranges, cache_line_size, hnfs):
    # Create the HNFs interleaved addr ranges
    block_size_bits = int(math.log(cache_line_size, 2))
    llc_bits = int(math.log(8, 2))
    numa_bit = block_size_bits + llc_bits - 1

    num_hnf_per_numa_node = (8 // 4)

    for i, hnf in enumerate(hnfs):
        ranges = []
        for r in sys_mem_ranges:
            addr_range = AddrRange(
                r.start,
                size=r.size(),
                intlvHighBit=numa_bit,
                intlvBits=llc_bits,
                intlvMatch=i,
            )
            ranges.append(addr_range)

        cls._addr_ranges[hnf] = ([ranges[i // num_hnf_per_numa_node]], numa_bit)

sys_mem_ranges is get from riscv board.
Is this the right way to implement it?

[powerjg]

Tagging @tiagormk

[tiagormk]

Hi @qqli579, is createNumaAddrRanges a modified version of the existing of createAddrRanges in CHI_config.py ? If I understand your issue correctly, you should be able to achieve the same effect by just calling createAddrRanges and passing a different list of ranges and HNFs for each NUMA domain, e.g, something like.:

createAddrRanges([addr_range[0]], cache_line_size, [0, 1])
createAddrRanges([addr_range[1]], cache_line_size, [2, 3])
createAddrRanges([addr_range[2]], cache_line_size, [4, 5])
createAddrRanges([addr_range[3]], cache_line_size, [6, 7])

[qqli579]

thanks.I implemented it by referring to https://gem5-review.googlesource.com/c/public/gem5/+/56610/1 patchset1.