Monthly Archives: February 2023

Improving performance with mitigations=off

I became aware that Spectre and Meltdown kernel mitigations could be turned off in Linux. I decided to give it a try.

DISCLAIMER & WARNING
You are making your system vulnerable to known types of attacks for marginal performance gains. I do not suggest or recommend it.

I am not explaining what the vulnerabilities are, and in what cases it would make sense to leave them open. My CPU is (selected lines from lscpu):

$ lscpu             GenuineIntel
  Model name:            Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz

Vulnerabilities:         
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushe
                         s, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Mitigation; Clear CPU buffers; SMT vulnerable
  Retbleed:              Mitigation; IBRS
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer
                          sanitization
  Spectre v2:            Mitigation; IBRS, IBPB conditional, RSB filling, PBRSB-
                         eIBRS Not affected
  Srbds:                 Mitigation; Microcode
  Tsx async abort:       Not affected

Deactivate Mitigations

This computer is running Fedora 37, booting using EFI and Grub2. I used to know LILO. Updating Grub was very easy when I knew how to do it. Obviously a restart is required.

# == To disable mitigations, making system vulnerable ==
# grubby --update-kernel=ALL --args="mitigations=off"
# grubby --info=ALL
# grub2-mkconfig -o /boot/grub2/grub.cfg

# == To enable mitigations, making system safe ==
# grubby --update-kernel=ALL --remove-args="mitigations=off"
# grubby --info=ALL
# grub2-mkconfig -o /boot/grub2/grub.cfg

After turning mitigations off, this is the pretty output from lscpu:

Vulnerability Itlb multihit:     KVM: Mitigation: VMX disabled
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX vulnerable
Vulnerability Mds:               Vulnerable; SMT vulnerable
Vulnerability Meltdown:          Vulnerable
Vulnerability Mmio stale data:   Vulnerable
Vulnerability Retbleed:          Vulnerable
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Vulnerability Spectre v2:        Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Not affected
Vulnerability Srbds:             Vulnerable
Vulnerability Tsx async abort:   Not affected

Benchmark

Rather than using some synthetic test I decided to use my most common heavy workload, a “precommit”-script that i run in my software project before committing code to git. All going well, it looks like this:

$ /usr/bin/time ./tools/precommit.sh
PRE COMMIT (5x: integrationtests, tests, htmllint, eslint, pkgjson)...
         pkgjson ---- Bad:0 Void:0 Skipped:0 Good:111 (111)
            html ---- Bad:0 Void:0 Skipped:0 Good:257 (257)
              es ---- Bad:0 Void:0 Skipped:0 Good:2 (2)
 integrationtest ---- Bad:0 Void:0 Skipped:7 Good:1487 (1494)
            test ---- Bad:0 Void:0 Skipped:0 Good:2622 (2622)
41.08user 4.37system 0:22.63elapsed 200%CPU (0avgtext+0avgdata 196640maxresident)k

This is a mix of different types of loads. Mostly it is Node.js running JavaScript code. In the beginning the five test categories run in parallel, but as the first checks complete less things are running. Also the tests contains some I/O, some waiting for I/O, some actual requests to services on the internet and things like that. So there is a lower limit to how fast it can run, regardless of CPU performance.

I ran several times to make sure everthing is cached and below are approximate avarages (this benchmark is not entirely stable from time to time):

Elapsed timeUser timeSystem time% CPU
i7-8809G25s41s4.3s190%
i7-8809G mitigations=off22s38s3.5s190%
i7-8809G hyperthreading off (BIOS)29s32s3.9s120%
i5-4250U34s89s9s290%
i5-4250U mitigations=off33s89s8s290%
Apple M1 Pro (10 Cores)23s18s3.6sn/a

Turning mitigations off gives about 10% performance on elapsed time for this real world problem. That is something (I have seen other people seeing more like 1% difference in gaming).

I found it interesting that my M1 Pro had the same performance, despite having more cores (10 vs 4/8) and lower total user time (kind of half time). I draw the conclusion that not so many cores are used in parallel and thought it was interesting to turn off hyperthreading (on a safe configuration) but that was quite bad for performance.

I also tested on an older NUC, finding basically no improvement at all with mitigations=off.

I will leave my computers safe.