For most of my early research, hardware behaved the way the documentation suggested it would. I was working on embedded platforms — ARM-based automotive SoCs, dedicated QoS hardware, memory controllers built for one job — where predictability was a design goal. The manuals described not only how to program the hardware but enough of its behavior to reason about performance. If a document said a mechanism enforced a bandwidth quota in a particular way, I could usually validate that understanding with a synthetic benchmark.
Then I started working on Intel server processors, expecting the same relationship between documentation and silicon. I was wrong, and figuring out exactly how wrong became an entire research paper.
What an explicit hardware contract looks like
On an embedded platform, I could usually design a synthetic benchmark that stressed exactly one component of the system because I already understood what that component was supposed to do. If I wanted to saturate a memory controller or evaluate a QoS mechanism, I could reason about the request path, arbitration policy, and enforcement hardware before writing a single line of code. The benchmark's job was to confirm a hypothesis I'd already formed from reading the documentation.
Intel's manual describes an interface, not necessarily a behavior
Intel's Resource Director Technology (RDT) appears similarly straightforward. It provides Cache Allocation Technology (CAT) for partitioning the shared cache, Memory Bandwidth Allocation (MBA) for regulating memory bandwidth, and the corresponding monitoring features Cache Monitoring Technology (CMT) and Memory Bandwidth Monitoring (MBM). The manuals clearly describe the programming interface: which model-specific registers to write, how Classes of Service are assigned, and how monitoring counters are read.
What the documentation doesn't fully describe is the implementation behavior that matters for building predictable systems. How effective are those controls under real contention? Do they behave consistently across processor generations? Are the monitoring counters accurate enough to drive resource-management decisions? Those questions motivated our RTNS paper because, as far as we could determine, no systematic evaluation of RDT's behavior across processor generations had been published.
The same knob, two different machines
We evaluated the same RDT mechanisms on two generations of Xeon processors: Cascade Lake and the newer Ice Lake. CAT alone demonstrated why experimentation was necessary.
On Ice Lake, assigning an application a private cache partition did not simply eliminate cache interference. Even with relatively few interfering cores, some workloads experienced a higher LLC miss rate than when CAT was disabled. The behavior was consistent and bounded, but it wasn't what someone might expect from reading only the programming interface.
Cascade Lake behaved differently. Initially CAT produced stable cache-miss rates as contention increased. Beyond a particular number of interfering cores, however, cache misses rose sharply, approaching the same degradation observed without cache partitioning. The mechanism still worked, but its effectiveness depended on the contention level in a way the programming interface alone could not predict.
We hypothesized that reduced cache associativity and undocumented address hashing contributed to the observed behavior, and evaluated that hypothesis using huge pages. Huge pages dramatically improved the Ice Lake results. On Cascade Lake they helped only partially, and only until contention reached a certain point. Even the mitigation depended on the processor generation.
Numbers without units
Memory Bandwidth Allocation exposed the same pattern. MBA accepts throttling levels—10%, 20%, and so on—but those percentages are not documented as corresponding to specific bandwidth values. To understand what each setting actually meant, we wrote a bandwidth-saturating benchmark and experimentally measured the bandwidth delivered at every available level.
The results differed substantially between generations. On Cascade Lake, settings above roughly 40% delivered nearly identical bandwidth, leaving much of the available control range effectively indistinguishable. On Ice Lake, read-intensive workloads scaled much more linearly with throttling level, while write-intensive workloads flattened early, resembling Cascade Lake's behavior. Reads and writes also could not be controlled independently despite having very different effects on memory contention.
The only reliable way to interpret an MBA setting was to measure it experimentally on the specific platform being used.
Monitoring that disagreed with reality
The monitoring mechanisms presented another challenge. We evaluated CMT using workloads with precisely controlled working-set sizes. Once huge pages were enabled, the reported cache occupancy became difficult to interpret, sometimes exceeding the application's actual working set. MBM also showed noticeable discrepancies compared to the theoretical amount of memory traffic generated by our synthetic benchmarks. The counters were useful, but they were not accurate enough for us to simply accept at face value.
Why this makes sense, even though it's frustrating
None of this means Intel did a poor job. It means Intel is solving a different problem than I was.
Embedded processors are frequently deployed into systems where predictability is part of the product itself. Automotive controllers, industrial systems, and avionics platforms often remain in service for years, and customers need to understand how timing-related mechanisms actually behave. In that environment, documenting implementation details increases the value of the hardware.
Server processors live under different constraints. Intel needs the same software interface to survive across multiple processor generations while allowing engineers to redesign cache hierarchies, memory controllers, replacement policies, and bandwidth regulators underneath. The programming interface is the contract. The implementation is intentionally free to evolve. That flexibility benefits cloud infrastructure and general-purpose software, but it leaves systems researchers with fewer guarantees about the behavior underneath the abstraction.
For someone interested in average throughput, that's usually enough. For someone trying to build predictable resource-management systems or derive worst-case performance bounds, it means experimentation becomes part of understanding the architecture itself.
What changed
The main conclusion of our paper was practical rather than philosophical: treat RDT as a hardware interface that must be characterized on each target platform. The differences between Cascade Lake and Ice Lake alone were enough to invalidate assumptions that appeared reasonable after studying only one processor generation.
That lesson fundamentally changed how I approached systems research.
In embedded systems, I wrote benchmarks to validate my understanding of the hardware. On modern server processors, I often write benchmarks to discover what the hardware actually does.