Utilize ASIDs to reduce both the direct and indirect costs of context
authorAlan Cox <alc@FreeBSD.org>
Sun, 3 Nov 2019 17:45:30 +0000 (17:45 +0000)
committerAlan Cox <alc@FreeBSD.org>
Sun, 3 Nov 2019 17:45:30 +0000 (17:45 +0000)
commit50e3ab6bcf8c0e39e152ec356f26cf8c6df4487c
treecd33c9bed55b6a3c9757b70935a1790db91e70b7
parent05f39d1a2d65b2960584e92f1616d7e9f1645436
Utilize ASIDs to reduce both the direct and indirect costs of context
switching.  The indirect costs being unnecessary TLB misses that are
incurred when ASIDs are not used.  In fact, currently, when we perform a
context switch on one processor, we issue a broadcast TLB invalidation that
flushes the TLB contents on every processor.

Mark all user-space ("ttbr0") page table entries with the non-global flag so
that they are cached in the TLB under their ASID.

Correct an error in pmap_pinit0().  The pointer to the root of the page
table was being initialized to the root of the kernel-space page table
rather than a user-space page table.  However, the root of the page table
that was being cached in process 0's md_l0addr field correctly pointed to a
user-space page table.  As long as ASIDs weren't being used, this was
harmless, except that it led to some unnecessary page table switches in
pmap_switch().  Specifically, other kernel processes besides process 0 would
have their md_l0addr field set to the root of the kernel-space page table,
and so pmap_switch() would actually change page tables when switching
between process 0 and other kernel processes.

Implement a workaround for Cavium erratum 27456 affecting ThunderX machines.
(I would like to thank andrew@ for providing the code to detect the affected
machines.)

Address integer overflow in the definition of TCR_ASID_16.

Setup TCR according to the PARange and ASIDBits fields from
ID_AA64MMFR0_EL1.  Previously, TCR_ASID_16 was unconditionally set.

Modify build_l1_block_pagetable so that lower attributes, such as ATTR_nG,
can be specified as a parameter.

Eliminate some unused code.

Earlier versions were tested to varying degrees by: andrew, emaste, markj

MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D21922
14 files changed:
sys/arm64/arm64/cpu_errata.c
sys/arm64/arm64/cpufunc_asm.S
sys/arm64/arm64/efirt_machdep.c
sys/arm64/arm64/genassym.c
sys/arm64/arm64/locore.S
sys/arm64/arm64/machdep.c
sys/arm64/arm64/mp_machdep.c
sys/arm64/arm64/pmap.c
sys/arm64/arm64/vm_machdep.c
sys/arm64/include/armreg.h
sys/arm64/include/cpufunc.h
sys/arm64/include/pcpu.h
sys/arm64/include/pmap.h
sys/arm64/include/proc.h