What is the purpose of speculative memory?

Question

Recently, I've found out that Darwin has speculative memory metric. But, I don't know what the purpose of speculative memory. The memory metric can be checked using vm_stat command on terminal.

Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free:                              334212.
Pages active:                           1450252.
Pages inactive:                         1367584.
Pages speculative:                        81968.
Pages throttled:                              0.
Pages wired down:                        794478.
Pages purgeable:                         377078.
"Translation faults":                1584392481.
Pages copy-on-write:                  316685071.
Pages zero filled:                    358257445.
Pages reactivated:                      9976919.
Pages purged:                           2557552.
File-backed pages:                       597869.
Anonymous pages:                        2301935.
Pages stored in compressor:             1170377.
Pages occupied by compressor:            165553.
Decompressions:                         7496783.
Compressions:                          14958548.
Pageins:                               29715699.
Pageouts:                               1058099.
Swapins:                                6422438.
Swapouts:                               7169648.

Refer to Darwin source, speculative is that used to hold pages that have been read from disk speculatively and the memory metric is the number of free pages and havent' actaually been used.

I can't find any information about speculative memory on the Internet. It's very strange that the memory is an official darwin memory metric, but no one says for it.

What's the purpose of this mertric unless used for any purpose?

Answer 1

Speculative memory (introduced with OS X 10.5) means that the kernel has marked a specific page of memory as being "speculative" - meaning that it is stored by the kernel because it predicts that it probably will be used for something later, but it hasn't happened yet. I.e. the kernel is speculating that reading something now will benefit it later.

In practice this occurs when the macOS kernel detects that it (because of application requests) are reading in a sequential manner from a disk drive. For example if an application reads block number 1, 2 and 3 - it is fair to assume that there's a high probability that the next block to be read is nummer 4. It might not be, but it is likely to happen.

The reason the kernel tries to detect this is that in many cases it is "cheaper" (in terms of performance) to read in larger blocks from the disk, and/or issue multiple requests to the drive for many blocks at the same time - rather than issuing them one at a time. Also it is possible for the kernel to request blocks from disk while the application itself is actually busy doing something else and hasn't asked for the block yet.

The idea behind speculative pages is thus to "prefetch" or "read ahead" on the disk and store these in memory, so that when the application later asks for the data, it can be provided very quickly. The current macOS kernel has different optimizations for ordinary hard disk drives and SSD drives in terms of when a prefetch is triggered, and how much is actually prefetched at a time.

The kernel marks these pages as "speculative", which is different from regular disk caching, because it knows that this is just a performance optimization on data that hasn't really been requested yet. So if the computer is lacking memory for other purposes, the speculative pages can be thrown out with little or no performance penalty at all.

In practice the kernel reads in the data into speculative pages and marks it as "protected" meaning that it won't be that easily thrown out under memory pressure. As the speculative pages age (i.e. time passes) they are moved from "bin to bin" so that older pages are stored together. When they reach a certain age, they are no longer protected - and can be readily discard if memory is needed for other purposes.

However, if an application actually uses the data that has been put in the speculative pages, they're immediately remarked as no longer speculative, and put in one of the other categories as relevant.

If you want you can completely disable speculative pages by setting speculative_reads_disabled to 1 with sysctl.

The reason statistics are made seperately for speculative pages so that it is not lumped together with free pages or the ordinary disk cache is that you can use it to measure if speculation is really being used on your machine, and how much, and you have a better idea on how much memory is actually being used by the applications - and not just by the kernel trying to optimize for performance.