What is CPU Cache and How Does CPU Cache Work?
Definition of CPU Cache and the Types of CPU Cache. How CPU Cache Works? How Much CPU Cache is Good for Gaming and Video Editing?
Computer processors are becoming too complex- too fast. With the complex architecture of processors comes robust technology represented in terms of size, speed, and data. A lot of attention has always been on a processor's speed when buying new PCs or stand-alone processors. I must admit that the excitement of acquiring a new processor is based on the speed and generation of the CPU.
However, the CPU Cache is a processor's feature that should not be overlooked when analyzing the technical specifications of a processor. The CPU cache is critical in determining the performance of the processor and overall CPU performance.
What is CPU Cache?
In simple words, a CPU cache is simply a type of memory built into the processor or the CPU die - a block of functional circuits. But you might wonder about RAM ( Random Access Memory) and its role in CPU performance.
RAM is a volatile form of memory (primary storage) in a PC that stores data for active programs and processes. It provides the middle ground between a PC's secondary storage (HDD/SSD) and the CPU. However, as the CPU speeds increased, the data cycle between the RAM and the CPU became a limiting factor to the performance of the processor.
These performance challenges led to the development of the CPU Cache memory, simply known as CPU cache. Since cache memory is inbuilt into the CPU itself, it enables faster data transfer between the CPU and the Cache memory, up to 100 times faster than data transfer speeds between the CPU and the RAM.
How CPU Cache Works
The CPU Cache works according to a memory hierarchy. All programs are stored on a PC's secondary storage - either the hard disk drive (HDD) or the solid-state drive (SSD). Once a program is started, the set of instructions is first loaded onto the RAM (primary storage) and then sent to the CPU.
CPUs have the capability of executing a huge number of instructions every second, therefore, it requires access to fast memory.
Therefore, instead of transferring the data sets to the RAM, the memory controller sends the data to the cache memory, where the CPU can access it faster in the next data cycle. When the processes are completed, their data sets are deleted from the CPU cache memory to free space for other programs and processes to use the CPU cache.
The memory hierarchy plays a critical role in CPU processing. The memory controller first checks for data in the memory cache. When it is found, it is loaded onto the CPU for processing. When the data is not found on the CPU cache, the RAM is checked, and if found, it is retrieved for processing. If the data is not found in the RAM, it is retrieved from the device's secondary storage drive (HDD/SSD).
L1, L2, and L3 Memory Cache Levels
CPU cache memory is divided into 3 levels - L1, L2, and L3 - each describing its closeness to the CPU.
L1 Cache is the closest to the processor. It is extremely fast but relatively small compared to the L2 and L3 cache types. The fast speeds are attributed to the fact that it is embedded in the physical processor chip. There are no standard sizes for the L1 cache and most processors have between 4-32 KB of L1 cache storage size. Some high-end CPUs, such as the AMD Ryzen 9 7950X3D have up to 1 MB of L1 CPU cache.
L2 cache is also known as the secondary cache. It has a higher storage size compared to the L1 cache but less than the L3 cache. Modern CPUs average about 512 KB of storage space on the L2 cache. High-end processors have up to 8 MB of L2 cache and others such as the AMD Ryzen 9 7950X3D has 16 MB of L2 cache. L2 caches can be directly embedded on the processor chip or a separate chip connected via a high-speed bus to the CPU. This design makes it slower than the L1 cache in terms of data transfer speeds. Further, the architecture of the L2 cache makes it about 25 times faster than RAM.
The L3 cache is slower than the L1 cache and the L2 cache. However, it supports a larger storage size than both the L1 and L2 cache. For instance, the Intel Core i9-9980XE Extreme Edition Processor has up to 24.75 MB of L3 cache. The AMD Ryzen 9 7950X3Dprocessor has up to 128 MB of L3 cache. While each core in a processor can have independent and dedicated L1 and L2 cache, they can share a single L3 cache, thanks to its storage space.
With the introduction of the cache levels, we can expand the memory hierarchy to include the L1, L2, and L3 cache. Therefore, data flows from the RAM, to the L3 cache, then to the L2 cache, and finally to the L1 cache before being loaded onto the CPU for processing. When searching for data, the CPU will search for it in the L1 cache. When it finds it, the data is loaded onto the CPU. This is known as a cache hit. However, during a cache miss, the data is not found in the succeeding level, up the cache hierarchy and it is fetched from the RAM.
A duration is required to transfer the data back and forth between the CPU and the memory storage (cache, RAM, and HDD/SSD). This duration is known as latency. Therefore, based on the memory hierarchy, the L1 cache has the lowest latency, which increases with every cache miss. The secondary storage (HDD/SSD) has the highest latency and more time is needed to load the data from the storage to the CPU.
With the sophistication of technology, the latency is decreasing with each generation of processors. Modern PCs have low latency PC4 and PC5 RAM and high-speed SSD that significantly cut down latency, which has seen an overall increase in the performance of modern CPUs.
Limiting Factor of Larger CPU Cache
You might ask yourself, if the cache is faster, why not increase its storage capabilities?
As stated above, the CPU cache is inbuilt into the processor chips. Therefore, it becomes expensive, in terms of cost, to expand the storage size of the cache in the processors. For instance, at the time of writing this article, the Intel Core i9-9980 XE processor is retailing at $1,399. Although this cost is inclusive of other technical specifications, the cache size plays a critical role in the price.
Intel lists the major technical specifications of each processor as:
Number of Cores
Max Turbo Frequency
Processor Base Frequency
Cache
TDP - Thermal Design Power
The size of a CPU's cache determines how much storage space the CPU has access to for fast processing. Even with small CPU cache differences, there are huge CPU performance differences more so when running high-intensity applications such as games and video editing programs. In most instances, CPU designers have two similar processors where the core difference is the CPU cache - as is the case for Intel's i9-10900K and i7-10700K. The single difference introduces other dependent differences between the processors that they seem different on many fronts.
How much CPU Cache Memory Do I Need?
Newer generations of processors have more CPU cache than earlier ones.
However, one limiting factor is the price. The price difference between a processor with a slightly higher cache than another is unimaginable and can be in the hundreds or thousands of dollars. It is good to keep in mind that CPUs are designed to handle tasks that they are expected to. For instance, Intel's Core X-Series processors, that includes the i9-10900X X-series Processor and the i9-10920X X-series Processor, each with 19.25 MB cache, are marked as being great to meet the most demanding creative workflows in video editing.
Therefore, the processors can handle intensive video editing processes efficiently. Intel's 12th-generation processors are marked as "engineered to elevate gaming, productivity, and creation". Processors are designed to meet various needs from gaming to content creation, to machine learning and artificial intelligence research, to server environments, and more. When buying a processor, it is good to research, from the manufacturer, on the processors that fit your needs as the CPU cache fit into the processors will be enough for you.
