【译】替 swap 辩护：常见的误解

内存的类型

There are many different types of memory in Linux, and each type has its own properties. Understanding the nuances of these is key to understanding why swap is important.

Linux 中内存分为好几种类型，每种都有各自的属性。想理解为什么交换区很重要的关键一点在于理解这些的细微区别。

For example, there are pages ("blocks" of memory, typically 4k) responsible for holding the code for each process being run on your computer. There are also pages responsible for caching data and metadata related to files accessed by those programs in order to speed up future access. These are part of the page cache , and I will refer to them as file memory.

比如说，有种 页面（「整块」的内存，通常 4K） 是用来存放电脑里每个程序运行时各自的代码的。 也有页面用来保存这些程序所需要读取的文件数据和元数据的缓存，以便加速随后的文件读写。 这些内存页面构成 页面缓存（page cache），后文中我称他们为文件内存。

There are also pages which are responsible for the memory allocations made inside that code, for example, when new memory that has been allocated with malloc is written to, or when using mmap 's MAP_ANONYMOUS flag. These are "anonymous" pages – so called because they are not backed by anything – and I will refer to them as anon memory.

还有一些页面是在代码执行过程中做的内存分配得到的，比如说，当代码调用 malloc 能分配到新内存区，或者使用 mmap 的 MAP_ANONYMOUS 标志分配的内存。 这些是「匿名(anonymous)」页面——之所以这么称呼它们是因为他们没有任何东西作后备—— 后文中我称他们为匿名内存。

There are other types of memory too – shared memory, slab memory, kernel stack memory, buffers, and the like – but anonymous memory and file memory are the most well known and easy to understand ones, so I will use these in my examples, although they apply equally to these types too.

还有其它类型的内存——共享内存、slab内存、内核栈内存、文件缓冲区（buffers），这种的—— 但是匿名内存和文件内存是最知名也最好理解的，所以后面的例子里我会用这两个说明， 虽然后面的说明也同样适用于别的这些内存类型。

可回收/不可回收内存

One of the most fundamental questions when thinking about a particular type of memory is whether it is able to be reclaimed or not. "Reclaim" here means that the system can, without losing data, purge pages of that type from physical memory.

考虑某种内存的类型时，一个非常基础的问题是这种内存是否能被回收。 「回收（Reclaim）」在这里是指系统可以，在不丢失数据的前提下，从物理内存中释放这种内存的页面。

For some page types, this is typically fairly trivial. For example, in the case of clean (unmodified) page cache memory, we're simply caching something that we have on disk for performance, so we can drop the page without having to do any special operations.

对一些内存类型而言，是否可回收通常可以直接判断。比如对于那些干净（未修改）的页面缓存内存， 我们只是为了性能在用它们缓存一些磁盘上现有的数据，所以我们可以直接扔掉这些页面， 不需要做什么特殊的操作。

For some page types, this is possible, but not trivial. For example, in the case of dirty (modified) page cache memory, we can't just drop the page, because the disk doesn't have our modifications yet. As such we either need to deny reclamation or first get our changes back to disk before we can drop this memory.

对有些内存类型而言，回收是可能的，但是不是那么直接。比如对脏（修改过）的页面缓存内存， 我们不能直接扔掉这些页面，因为磁盘上还没有写入我们所做的修改。这种情况下，我们可以选择拒绝回收， 或者选择先等待我们的变更写入磁盘之后才能扔掉这些内存。

For some page types, this is not possible. For example, in the case of the anonymous pages mentioned previously, they only exist in memory and in no other backing store, so they have to be kept there.

对还有些内存类型而言，是不能回收的。比如前面提到的匿名页面，它们只存在于内存中，没有任何后备存储， 所以它们必须留在内存里。

在无/低内存竞争的状态下

• With swap: We can choose to swap out rarely-used anonymous memory that may only be used during a small part of the process lifecycle, allowing us to use this memory to improve cache hit rate, or do other optimisations.
• Without swap We cannot swap out rarely-used anonymous memory, as it's locked in memory. While this may not immediately present as a problem, on some workloads this may represent a non-trivial drop in performance due to stale, anonymous pages taking space away from more important use.

• 有交换区: 我们可以选择换出那些只有在进程生存期内很小一部分时间会访问的匿名内存， 这允许我们空出更多内存空间用来提升缓存命中率，或者做别的优化。
• 无交换区: 我们不能换出那些很少使用的匿名内存，因为它们被锁在了内存中。虽然这通常不会直接表现出问题， 但是在一些工作条件下这可能造成卡顿导致不平凡的性能下降，因为匿名内存占着空间不能给 更重要的需求使用。

在中/高内存竞争的状态下

• With swap: All memory types have an equal possibility of being reclaimed. This means we have more chance of being able to reclaim pages successfully – that is, we can reclaim pages that are not quickly faulted back in again (thrashing).
• Without swap Anonymous pages are locked into memory as they have nowhere to go. The chance of successful long-term page reclamation is lower, as we have only some types of memory eligible to be reclaimed at all. The risk of page thrashing is higher. The casual reader might think that this would still be better as it might avoid having to do disk I/O, but this isn't true – we simply transfer the disk I/O of swapping to dropping hot page caches and dropping code segments we need soon.

• 有交换区: 所有内存类型都有平等的被回收的可能性。这意味着我们回收页面有更高的成功率—— 成功回收的意思是说被回收的那些页面不会在近期内被缺页异常换回内存中（颠簸）。
• 无交换区: 匿名内存因为无处可去所以被锁在内存中。长期内存回收的成功率变低了，因为我们成体上 能回收的页面总量少了。发生缺页颠簸的危险性更高了。缺乏经验的读者可能觉得这某时也是好事， 因为这能避免进行磁盘I/O，但是实际上不是如此——我们只是把交换页面造成的磁盘I/O变成了扔掉热缓存页 和扔掉代码段，这些页面很可能马上又要从文件中读回来。

在临时内存占用高峰时

• With swap: We're more resilient to temporary spikes, but in cases of severe memory starvation, the period from memory thrashing beginning to the OOM killer may be prolonged. We have more visibility into the instigators of memory pressure and can act on them more reasonably, and can perform a controlled intervention.
• Without swap The OOM killer is triggered more quickly as anonymous pages are locked into memory and cannot be reclaimed. We're more likely to thrash on memory, but the time between thrashing and OOMing is reduced. Depending on your application, this may be better or worse. For example, a queue-based application may desire this quick transfer from thrashing to killing. That said, this is still too late to be really useful – the OOM killer is only invoked at moments of severe starvation, and relying on this method for such behaviour would be better replaced with more opportunistic killing of processes as memory contention is reached in the first place.

• 有交换区: 我们对内存使用激增的情况更有抵抗力，但是在严重的内存不足的情况下， 从开始发生内存颠簸到 OOM 杀手开始工作的时间会被延长。内存压力造成的问题更容易观察到， 从而可能更有效地应对，或者更有机会可控地干预。
• 无交换区: 因为匿名内存被锁在内存中了不能被回收，所以 OOM 杀手会被更早触发。 发生内存颠簸的可能性更大，但是发生颠簸之后到 OOM 解决问题的时间间隔被缩短了。 基于你的程序，这可能更好或是更糟。比如说，基于队列的程序可能更希望这种从颠簸到杀进程的转换更快发生。 即便如此，发生 OOM 的时机通常还是太迟于是没什么帮助——只有在系统极度内存紧缺的情况下才会请出 OOM 杀手，如果想依赖这种行为模式，不如换成更早杀进程的方案，因为在这之前已经发生内存竞争了。

好吧，所以我需要系统交换区，但是我该怎么为每个程序微调它的行为？

You didn't think you'd get through this entire post without me plugging cgroup v2, did you? ;-)

你肯定想到了我写这篇文章一定会在哪儿插点 cgroup v2 的安利吧？ ;-)

Obviously, it's hard for a generic heuristic algorithm to be right all the time, so it's important for you to be able to give guidance to the kernel. Historically the only tuning you could do was at the system level, using vm.swappiness . This has two problems: vm.swappiness is incredibly hard to reason about because it only feeds in as a small part of a larger heuristic system, and it also is system-wide instead of being granular to a smaller set of processes.

显然，要设计一种对所有情况都有效的启发算法会非常难，所以给内核提一些指引就很重要。 历史上我们只能在整个系统层面做这方面微调，通过 vm.swappiness 。这有两方面问题： vm.swappiness 的行为很难准确控制，因为它只是传递给一个更大的启发式算法中的一个小参数； 并且它是一个系统级别的设置，没法针对一小部分进程微调。

You can also use mlock to lock pages into memory, but this requires either modifying program code, fun with LD_PRELOAD , or doing horrible things with a debugger at runtime. In VM-based languages this also doesn't work very well, since you generally have no control over allocation and end up having to mlockall , which has no precision towards the pages you actually care about.

你可以用 mlock 把页面锁在内存里，但是这要么必须改程序代码，或者折腾 LD_PRELOAD ，或者在运行期用调试器做一些魔法操作。对基于虚拟机的语言来说这种方案也不能 很好工作，因为通常你没法控制内存分配，最后得用上 mlockall ，而这个没有办法精确指定你实际上想锁住的页面。

cgroup v2 has a tunable per-cgroup in the form of memory.low , which allows us to tell the kernel to prefer other applications for reclaim below a certain threshold of memory used. This allows us to not prevent the kernel from swapping out parts of our application, but prefer to reclaim from other applications under memory contention. Under normal conditions, the kernel's swap logic is generally pretty good, and allowing it to swap out pages opportunistically generally increases system performance. Swap thrash under heavy memory contention is not ideal, but it's more a property of simply running out of memory entirely than a problem with the swapper. In these situations, you typically want to fail fast by self-killing non-critical processes when memory pressure starts to build up.

cgroup v2 提供了一套可以每个 cgroup 微调的 memory.low ，允许我们告诉内核说当使用的内存低于一定阈值之后优先回收别的程序的内存。这可以让我们不强硬禁止内核 换出程序的一部分内存，但是当发生内存竞争的时候让内核优先回收别的程序的内存。在正常条件下， 内核的交换逻辑通常还是不错的，允许它有条件地换出一部分页面通常可以改善系统性能。在内存竞争的时候 发生交换颠簸虽然不理想，但是这更多地是单纯因为整体内存不够了，而不是因为交换进程（swapper）导致的问题。 在这种场景下，你通常希望在内存压力开始积攒的时候通过自杀一些非关键的进程的方式来快速退出（fail fast）。

You can not simply rely on the OOM killer for this. The OOM killer is only invoked in situations of dire failure when we've already entered a state where the system is severely unhealthy and may well have been so for a while. You need to opportunistically handle the situation yourself before ever thinking about the OOM killer.

你不能依赖 OOM 杀手达成这个。 OOM 杀手只有在非常急迫的情况下才会出动，那时系统已经处于极度不健康的 状态了，而且很可能在这种状态下保持了一阵子了。需要在开始考虑 OOM 杀手之前，积极地自己处理这种情况。

Determination of memory pressure is somewhat difficult using traditional Linux memory counters, though. We have some things which seem somewhat related, but are merely tangential – memory usage, page scans, etc – and from these metrics alone it's very hard to tell an efficient memory configuration from one that's trending towards memory contention. There is a group of us at Facebook, spearheaded by Johannes , working on developing new metrics that expose memory pressure more easily that should help with this in future. If you're interested in hearing more about this, I go into detail about one metric being considered in my talk on cgroup v2.

不过，用传统的 Linux 内存统计数据还是挺难判断内存压力的。我们有一些看起来相关的系统指标，但是都 只是支离破碎的——内存用量、页面扫描，这些——单纯从这些指标很难判断系统是处于高效的内存利用率还是 在滑向内存竞争状态。我们在 Facebook 有个团队，由 Johannes 牵头，努力开发一些能评价内存压力的新指标，希望能在今后改善目前的现状。 如果你对这方面感兴趣， 在我的 cgroup v2 的演讲中介绍到一个被提议的指标 。

那么，我需要多少交换空间？

In general, the minimum amount of swap space required for optimal memory management depends on the number of anonymous pages pinned into memory that are rarely reaccessed by an application, and the value of reclaiming those anonymous pages. The latter is mostly a question of which pages are no longer purged to make way for these infrequently accessed anonymous pages.

通常而言，最优内存管理所需的最小交换空间取决于程序固定在内存中而又很少访问到的匿名页面的数量， 以及回收这些匿名页面换来的价值。后者大体上来说是问哪些页面不再会因为要保留这些很少访问的匿名页面而 被回收掉腾出空间。

If you have a bunch of disk space and a recent (4.0+) kernel, more swap is almost always better than less. In older kernels kswapd , one of the kernel processes responsible for managing swap, was historically very overeager to swap out memory aggressively the more swap you had. In recent times, swapping behaviour when a large amount of swap space is available has been significantly improved. If you're running kernel 4.0+, having a larger swap on a modern kernel should not result in overzealous swapping. As such, if you have the space, having a swap size of a few GB keeps your options open on modern kernels.

如果你有足够大的磁盘空间和比较新的内核版本（4.0+），越大的交换空间基本上总是越好的。 更老的内核上 kswapd ， 内核中负责管理交换区的内核线程，在历史上倾向于有越多交换空间就 急于交换越多内存出去。在最近一段时间，可用交换空间很大的时候的交换行为已经改善了很多。 如果在运行 4.0+ 以后的内核，即便有很大的交换区在现代内核上也不会很激进地做交换。因此， 如果你有足够的容量，现代内核上有个几个 GB 的交换空间大小能让你有更多选择。

If you're more constrained with disk space, then the answer really depends on the tradeoffs you have to make, and the nature of the environment. Ideally you should have enough swap to make your system operate optimally at normal and peak (memory) load. What I'd recommend is setting up a few testing systems with 2-3GB of swap or more, and monitoring what happens over the course of a week or so under varying (memory) load conditions. As long as you haven't encountered severe memory starvation during that week – in which case the test will not have been very useful – you will probably end up with some number of MB of swap occupied. As such, it's probably worth having at least that much swap available, in addition to a little buffer for changing workloads. atop in logging mode can also show you which applications are having their pages swapped out in the SWAPSZ column, so if you don't already use it on your servers to log historic server state you probably want to set it up on these test machines with logging mode as part of this experiment. This also tells you when your application started swapping out pages, which you can tie to log events or other key data.

如果你的磁盘空间有限，那么答案更多取决于你愿意做的取舍，以及运行的环境。理想上应该有足够的交换空间 能高效应对正常负载和高峰（内存）负载。我建议先用 2-3GB 或者更多的交换空间搭个测试环境， 然后监视在不同（内存）负载条件下持续一周左右的情况。只要在那一周里没有发生过严重的内存不足—— 发生了的话说明测试结果没什么用——在测试结束的时候大概会留有多少 MB 交换区占用。 作为结果说明你至少应该有那么多可用的交换空间，再多加一些以应对负载变化。用日志模式跑 atop 可以在 SWAPSZ 栏显示程序的页面被交换出去的情况，所以如果你还没用它记录服务器历史日志的话 ，这次测试中可以试试在测试机上用它记录日志。这也会告诉你什么时候你的程序开始换出页面，你可以用这个 对照事件日志或者别的关键数据。

Another thing worth considering is the nature of the swap medium. Swap reads tend to be highly random, since we can't reliably predict which pages will be refaulted and when. On an SSD this doesn't matter much, but on spinning disks, random I/O is extremely expensive since it requires physical movement to achieve. On the other hand, refaulting of file pages is likely less random, since files related to the operation of a single application at runtime tend to be less fragmented. This might mean that on a spinning disk you may want to bias more towards reclaiming file pages instead of swapping out anonymous pages, but again, you need to test and evaluate how this balances out for your workload.

另一点值得考虑的是交换空间所在存储设备的媒介。读取交换区倾向于很随机，因为我们不能可靠预测什么时候 什么页面会被再次访问。在 SSD 上这不是什么问题，但是在传统磁盘上，随机 I/O 操作会很昂贵， 因为需要物理动作寻道。另一方面，重新加载文件缓存可能不那么随机，因为单一程序在运行期的文件读操作 一般不会太碎片化。这可能意味着在传统磁盘上你想更多地回收文件页面而不是换出匿名页面，但仍旧， 你需要做测试评估在你的工作负载下如何取得平衡。

For laptop/desktop users who want to hibernate to swap, this also needs to be taken into account – in this case your swap file should be at least your physical RAM size.

对笔记本/桌面用户如果想要休眠到交换区，这也需要考虑——这种情况下你的交换文件应该至少是物理内存大小。

我的 swappiness 应该如何设置？

First, it's important to understand what vm.swappiness does. vm.swappiness is a sysctl that biases memory reclaim either towards reclamation of anonymous pages, or towards file pages. It does this using two different attributes: file_prio (our willingness to reclaim file pages) and anon_prio (our willingness to reclaim anonymous pages). vm.swappiness`plays into this, as it becomes the default value for :code:`anon_prio , and it also is subtracted from the default value of 200 for file_prio , which means for a value of vm.swappiness = 50 , the outcome is that anon_prio is 50, and file_prio is 150 (the exact numbers don't matter as much as their relative weight compared to the other).

首先很重要的一点是，要理解 vm.swappiness 是做什么的。 vm.swappiness 是一个 sysctl 用来控制在内存回收的时候，是优先回收匿名页面， 还是优先回收文件页面。内存回收的时候用两个属性： file_prio （回收文件页面的倾向） 和 anon_prio （回收匿名页面的倾向）。 vm.swappiness 控制这两个值， 因为它是 anon_prio 的默认值，然后也是默认 200 减去它之后 file_prio 的默认值。 意味着如果我们设置 vm.swappiness = 50 那么结果是 anon_prio 是 50， file_prio 是 150 （这里数值本身不是很重要，重要的是两者之间的权重比）。

This means that, in general, vm.swappiness is simply a ratio of how costly reclaiming and refaulting anonymous memory is compared to file memory for your hardware and workload. The lower the value, the more you tell the kernel that infrequently accessed anonymous pages are expensive to swap out and in on your hardware. The higher the value, the more you tell the kernel that the cost of swapping anonymous pages and file pages is similar on your hardware. The memory management subsystem will still try to mostly decide whether it swaps file or anonymous pages based on how hot the memory is, but swappiness tips the cost calculation either more towards swapping or more towards dropping filesystem caches when it could go either way. On SSDs these are basically as expensive as each other, so setting vm.swappiness = 100 (full equality) may work well. On spinning disks, swapping may be significantly more expensive since swapping in generally requires random reads, so you may want to bias more towards a lower value.

这意味着，通常来说 vm.swappiness 只是一个比例，用来衡量在你的硬件和工作负载下， 回收和换回匿名内存还是文件内存哪种更昂贵 。设定的值越低，你就是在告诉内核说换出那些不常访问的 匿名页面在你的硬件上开销越昂贵；设定的值越高，你就是在告诉内核说在你的硬件上交换匿名页和 文件缓存的开销越接近。内存管理子系统仍然还是会根据实际想要回收的内存的访问热度尝试自己决定具体是 交换出文件还是匿名页面，只不过 swappiness 会在两种回收方式皆可的时候，在计算开销权重的过程中左右 是该更多地做交换还是丢弃缓存。在 SSD 上这两种方式基本上是同等开销，所以设成 vm.swappiness = 100 （同等比重）可能工作得不错。在传统磁盘上，交换页面可能会更昂贵， 因为通常需要随机读取，所以你可能想要设低一些。

The reality is that most people don't really have a feeling about which their hardware demands, so it's non-trivial to tune this value based on instinct alone – this is something that you need to test using different values. You can also spend time evaluating the memory composition of your system and core applications and their behaviour under mild memory reclamation.

现实是大部分人对他们的硬件需求没有什么感受，所以根据直觉调整这个值可能挺困难的 —— 你需要用不同的值做测试。你也可以花时间评估一下你的系统的内存分配情况和核心应用在大量回收内存的时候的行为表现。

When talking about vm.swappiness , an extremely important change to consider from recent(ish) times is this change to vmscan by Satoru Moriya in 2012 , which changes the way that vm.swappiness = 0 is handled quite significantly.

讨论 vm.swappiness 的时候，一个极为重要需要考虑的修改是（相对）近期在 2012 年左右 Satoru Moriya 对 vmscan 行为的修改 ，它显著改变了代码对 vm.swappiness = 0 这个值的处理方式。

Essentially, the patch makes it so that we are extremely biased against scanning (and thus reclaiming) any anonymous pages at all with vm.swappiness = 0 , unless we are already encountering severe memory contention. As mentioned previously in this post, that's generally not what you want, since this prevents equality of reclamation prior to extreme memory pressure occurring, which may actually lead to this extreme memory pressure in the first place. vm.swappiness = 1 is the lowest you can go without invoking the special casing for anonymous page scanning implemented in that patch.

基本上来说这个补丁让我们在 vm.swappiness = 0 的时候会极度避免扫描（进而回收）匿名页面， 除非我们已经在经历严重的内存抢占。就如本文前面所属，这种行为基本上不会是你想要的， 因为这种行为会导致在发生内存抢占之前无法保证内存回收的公平性，这甚至可能是最初导致发生内存抢占的原因。 想要避免这个补丁中对扫描匿名页面的特殊行为的话， vm.swappiness = 1 是你能设置的最低值。

The kernel default here is vm.swappiness = 60 . This value is generally not too bad for most workloads, but it's hard to have a general default that suits all workloads. As such, a valuable extension to the tuning mentioned in the "how much swap do I need" section above would be to test these systems with differing values for vm.swappiness , and monitor your application and system metrics under heavy (memory) load. Some time in the near future, once we have a decent implementation of refault detection in the kernel, you'll also be able to determine this somewhat workload-agnostically by looking at cgroup v2's page refaulting metrics.

内核在这里设置的默认值是 vm.swappiness = 60 。这个值对大部分工作负载来说都不会太坏， 但是很难有一个默认值能符合所有种类的工作负载。因此，对上面「 那么，我需要多少交换空间？ 」那段讨论的一点重要扩展可以说，在测试系统中可以尝试使用不同的 vm.swappiness ，然后监视你的程序和系统在重（内存）负载下的性能指标。在未来某天，如果我们在内核中有了合理的 缺页检测 ，你也将能通过 cgroup v2 的页面缺页 指标来以负载无关的方式决定这个。

2019年07月更新：内核 4.20+ 中的内存压力指标

The refault metrics mentioned as in development earlier are now in the kernel from 4.20 onwards and can be enabled with CONFIG_PSI=y . See my talk at SREcon at around the 25:05 mark:

前文中提到的开发中的内存缺页检测指标已经进入 4.20+ 以上版本的内核，可以通过 CONFIG_PSI=y 开启。详情参见我在 SREcon 大约 25:05 左右的讨论。