What constrains the CPU->cache bandwidth with O3CPU and Ruby

[powerjg]

Original asker: @hegdeprajwal

I'm having trouble figuring out how to add more ports to the Instruction and data caches. Could you kindly direct me to some reference material or documentation on this topic?

[powerjg]

First, there are two parameters in the O3CPU:

• cacheStorePorts
• cacheLoadPorts

These are used in the LSQ.
On each cycle, the LSQ increments the number of usedStorePorts and usedLoadPorts and if the number of used ports exceeds the parameter value then the LSQ port is blocked and trySendPacket fails.

Note that by default, these values are large (200 in the codebase today).
So, this means that in a default O3CPU configuration, the LSQ should not stall except because the cache cannot accept the request.

You can find the number of times this parameter or generally the cache blocks the LSQ with the statistic blockedByCache.

The other place that could cause load or store instructions to stall is the cache itself.
Here, I'm going to talk about Ruby.
Someone could dig into the classic caches if we need to.

When the LSQ sends a packet into Ruby, it first goes to the RubyPort where it is then sent to the Sequencer.
The RubyPort simply calls makeRequest on the Sequencer and does not model any contention.

• makeRequest This function gets the correct "ruby request type" and then calls insertRequest and then issueRequest. The Ruby request type is a translation of the packet command into the Ruby command.
  • insertRequest This function gets the "request status" which is usually "ready" which means it can be issued or is "aliased" which means the request is accepted, but it waits for the aliasing request to finish in the cache before it is issued (or possibly satisfied by the aliasing request). This could also be "buffer full" if the current number of outstanding requests is greater than the maximum number of outstanding requests. By default, this maximum is 0, which means infinite. See buffer_size
  • issueRequest This function is where the request is pushed into Ruby via the mandatoryQueue. If you've gotten here, the request will be accepted by the cache. It is enqueued in the future, at the mandatoryQueueLatency cycles in the future. See mandatory_queue_latency. This is by default 1 cycle. Note that if there is a clock domain crossing from the CPU to the RubySystem this latency could be higher than 1 cycle. It will be 1 cycle in Ruby's clock domain after the current cycle. I.e., it's on the next full cycle's clock edge.

So, on the LSQ/cache side, there are only two constraints to the cache bandwidth:
(1) The number of ports in the LSQ and (2) the max number of outstanding requests in the cache.
You should be able to measure the times that both of these things happen with stats.
Specifically, you can look at the blockedByCache stat in the LSQ.
For the buffer full, there is not a stat in the sequencer right now, but adding one would be straightforward.
Or, you can make sure that the buffer_size is set to 0.

Finally, if there is something else constraining the cache bandwidth, it would be in O3CPU.
I haven't dug into that code in a while 😄.

[hegdeprajwal]

Thanks for the detailed answer @powerjg !

Does this imply that instruction fetch requests sends requests directly to RubyPort without any contention, given that RubyPort has an infinite number of ports by default?