Sepherosa Ziehau [Tue, 14 Feb 2017 13:54:36 +0000 (21:54 +0800)]
altq: Use microuptime() instead of microtime() - boottime.
Matthew Dillon [Tue, 14 Feb 2017 07:21:09 +0000 (23:21 -0800)]
kernel - Fix bug in pmap_unwire()
* Fix a bug introduced in the recent pmap work. For kernel pages,
pmap_unwire() was not testing the V bit and would return a bogus
vm_page_t for PTE values of 0, instead of returning NULL.
* Triggered by the mpipe code, used primarily by the crypto subsystem.
Also used for IP fragment reassembly and by tcp_maketemplate(). A
failed M_NOWAIT slab allocation tries to delete the partially filled
map_entry_t, encounters 0 PTEs, and gets confused.
Reported-by: Tim Darby, Antonio Huete Jimenez
Sepherosa Ziehau [Mon, 13 Feb 2017 13:51:20 +0000 (21:51 +0800)]
libkern: Make inet_ntoa MPSAFE by passing string buffer to it.
François Tigeot [Mon, 13 Feb 2017 12:40:21 +0000 (13:40 +0100)]
drm/i915: Set GPU freq to idle_freq initially
Set the initial frequency to idle_freq on Skylake+/Valleyview/Cherryview
GPUs in order to not waste power.
This was already done on all other platforms.
Obtained-from: Linux 4.7, commit
5fd9f52384935dacfdc4adb5cc3531a40346d475
Sepherosa Ziehau [Sun, 12 Feb 2017 14:19:52 +0000 (22:19 +0800)]
net: Make sure that local variable is really used.
John Marino [Mon, 13 Feb 2017 02:25:18 +0000 (20:25 -0600)]
sort(1): Fix some obvious issues
Taken-from: FreeBSD
François Tigeot [Sun, 12 Feb 2017 21:32:10 +0000 (22:32 +0100)]
drm: Partial revert of "drm: Sync drm_crtc.c with Linux 4.6"
This partially reverts commit
917ec290eeeeb73fe8bfa066cfc3303a55646424.
It broke Xorg on some machines.
Reported-by: many people
Sascha Wildner [Sun, 12 Feb 2017 18:06:25 +0000 (19:06 +0100)]
kernel: Add a simple kern.cp_times sysctl.
This is like kern.cp_time, just per CPU.
It will help devel/libgtop, and by extension mate-system-monitor, perhaps
other such utilities/plugins too. Therefore, bump __DragonFly_version as
well.
Sepherosa Ziehau [Sun, 12 Feb 2017 13:35:19 +0000 (21:35 +0800)]
pktgen: Ifnet lock is required by ifunit.
Sascha Wildner [Sun, 12 Feb 2017 09:50:36 +0000 (10:50 +0100)]
kernel: Use CPUSTATES constant.
Imre Vadász [Wed, 8 Feb 2017 21:05:35 +0000 (22:05 +0100)]
virtio: Fix and activate PCI MSI-X support.
* Change pci_msi_blacklisted() test to always assume working MSI and MSI-X
interrupts on virtual-machines. We should assume that a hypervisor
without MSI or MSI-X support shouldn't even indicate supported MSI(-X)
via the PCI config space.
* Move vtblk_alloc_disk() to the end of virtio_blk's attach method.
This avoids a race between irq enabling in virtio, and the disk probing.
* Fix allocation and teardown of MSI-X interrupts.
* Get rid of rather confusing VIRTIO_PCI_FLAG_NO_MSI and
VIRTIO_PCI_FLAG_NO_MSIX flags. The VIRTIO_PCI_FLAG_NO_MSI is unneeded
when using pci_alloc_1intr() for MSI allocation, and
instead of VIRTIO_PCI_FLAG_NO_MSIX we can just check sc->vtpci_msix_res.
* For now, MSI-X vectors are simply assigned to subsequent cores (plus
adding the device_get_unit() value to shuffle things a bit), i.e.:
(device_get_unit(dev) + cnt) % ncpus
* For the if_vtnet network driver this means that the rx and tx queues run
on different cores, but both queues still use the same serializer for
now.
* One major advantages in using MSI-X interrupts instead of the legacy
IRQ is, that this avoids interrupt sharing in many virtual-machine
configurations.
* Tested on qemu with TCG (unaccelerated emulation) and 1 core, and with
KVM acceleration and 2 cores, with both virtio network and disk devices.
zrj [Wed, 8 Feb 2017 16:48:50 +0000 (18:48 +0200)]
Add entries for initial clang import in mtree.
These will be used soon populated with initial base clang version.
zrj [Wed, 8 Feb 2017 16:40:21 +0000 (18:40 +0200)]
Prevent implicit -std=* passing from NXCFLAGS to NXCXXFLAGS.
C and c++ standards are not cross compatible, this is a major issue with clang
import into the base that explictly requires at least the -std=c++11 standard.
It gets complicated because even tablegen utilities used as buildtools depend
explictly on c++11. This implies that we no longer can avoid libstdc++ usage
in hostlib/hostprog infrastructure.
While there, explcitly filter out -flto options from host native tools FLAGS
and add a missing rule to allow generating host native c++ utilities for the
use in buildworld.
zrj [Fri, 10 Feb 2017 09:48:37 +0000 (11:48 +0200)]
devd(8): Remove build workaround.
After previous commit change it should be no longer needed.
Spotted-by: swildner
zrj [Wed, 8 Feb 2017 16:27:24 +0000 (18:27 +0200)]
Prevent implicit -Wsystem-headers passing from CFLAGS to CXXFLAGS.
Upcoming clang will be the first base system part that very heavily depends on
c++ headers, this prevents setting any level of WARNS in clang's makefiles
without performing certain unsafe manipulations of CXXFLAGS. To prevent that
it makes sense to allow only explict use directly in CXXFLAGS where it is
needed given c++11 headers are part of contrib and quite noisy with deprecated
attributes for previous features/interfaces to c++11 standard.
zrj [Fri, 10 Feb 2017 08:02:10 +0000 (10:02 +0200)]
gencat(1): Limit and mark bootstrap hack.
By the git history it was only needed for DragonFly 3.6 changes.
But be generous and mark it as 4.1 just in case there were other changes
piggybacking on it during crossworld stage. It no longer should be needed
and might cause unwanted issues for future developments.
While there remove the getline helper too.
zrj [Tue, 7 Feb 2017 15:28:54 +0000 (17:28 +0200)]
ssl: Unhook internal usage of include/openssl/.
Redirect all cases to include priv to take needed openssl headers and
remove mistakenly added copying of private base LibreSSL headers back to
public dir used previously by OpenSSL headers.
After this make upgrade REMOVE_OPENSSL_FILES=yes should work as initially
intended.
It is still not clear if LIBMD places also miss the libprivate_crypto.
For now just to prevent conflicts with base vs dports ssl libs and headers.
Matthew Dillon [Fri, 10 Feb 2017 00:51:44 +0000 (16:51 -0800)]
kernel - Fix vmspace termination race (2)
* Fix a race and fix dangling cached mount points. At the point where
dounmount() checks to see if there are any dangling cache refs left
it has already dropped its own ref on the mp. The test, however,
was (refs > 1). The test needs to be (refs > 0).
This race was probably causing the corruption, and in fact its the
smoking gun because the mp->mnt_refs field is at the same offset as
the pmap->copyin field, and the pmap->copyin field was getting
corrupted by being decremented by 1.
* Fix a race where the unmount code was issuing tue dounmount() call
without any ref or hold on the mp. This race was unlikely (requires
two unmounts of the same partition at the same time).
Sascha Wildner [Thu, 9 Feb 2017 19:12:00 +0000 (20:12 +0100)]
<sys/{_iovec,_timespec,_timeval}.h>: Namespace cleanup.
Don't include the full <sys/types.h>.
Include <machine/stdint.h> and derive the types which <sys/_timespec.h>
and <sys/_timeval.h> need from the __* versions.
<sys/_iovec.h> just needs __size_t, which is in <machine/stdint.h>.
This commit also fixes a circular #include where <sys/_timeval.h>
was including <sys/types.h> and vice versa.
Sascha Wildner [Thu, 9 Feb 2017 18:02:43 +0000 (19:02 +0100)]
<aio.h>: Prepare for the removal of <sys/types.h> from <sys/_timespec.h>.
Right now, <sys/aio.h> gets a number of required type definitions via
<sys/_timespec.h>, which includes <sys/types.h>. Instead of including
<sys/types.h> from <sys/aio.h> directly, go for a lighter namespace.
Sascha Wildner [Thu, 9 Feb 2017 17:58:57 +0000 (18:58 +0100)]
<sched.h>: Clean up the namespace a bit.
Move CPU_* macros and three non-posix functions into __BSD_VISIBLE.
While here, put the __BEGIN_DECLS/__END_DECLS around prototypes only.
Sascha Wildner [Thu, 9 Feb 2017 17:56:22 +0000 (18:56 +0100)]
<sched.h>: Clean up a bit (whitespace etc.). No functional change.
Matthew Dillon [Thu, 9 Feb 2017 18:12:05 +0000 (10:12 -0800)]
kernel - Fix pmap dynamic deletion code, leave disabled
* Fix the bug that caused pmap dynamic page table deletion to implode,
but leave the feature disabled by default for now.
* Cleanup the pmap code a bit. Use pv_put() more generically because
it turns out that the dynamic page table deletion code is just cleaner
that way.
* Assert the activated vmspaces are valid.
Matthew Dillon [Thu, 9 Feb 2017 18:09:43 +0000 (10:09 -0800)]
kernel - Fix vmspace termination race
* Fix a SMP race which can cause a vmspace structure to be double-freed
to its objcache, resulting in corruption.
* Typical panic was a kernel seg fault at 'vm_copyin+2'. What was in fact
happening was that corruption of the vmspace was racing against reuse and
a decrement of the pmap copyin function pointer. This decrement caused
function calls through the pointer to get very confused.
Reported-by: tuxillo
Matthew Dillon [Wed, 8 Feb 2017 20:11:26 +0000 (12:11 -0800)]
kernel - Remove some debug warnings for now
* Remove "Warning: bio_page_alloc: memory exhausted " for now
(conditionalize it on bootverbose).
* The underlying issue is still there, a potential deadlock between
the pageout system. But the warnings are confusing users.
François Tigeot [Tue, 7 Feb 2017 20:45:50 +0000 (21:45 +0100)]
drm: Sync drm_crtc.c with Linux 4.6
Start using ida functions.
Matthew Dillon [Tue, 7 Feb 2017 19:07:22 +0000 (11:07 -0800)]
build - Separate out CFLAGS for kernel & module build
* Make COPTFLAGS apply to module builds in addition to kernel builds.
* Implement a new make variable, KCFLAGS, for specifying overrides for
kernel and module builds.
* Fixup the module build infrastructure to use KCFLAGS, and get rid of
infiltration from the more generic CFLAGS onto the kernel & modules
build.
zrj [Tue, 7 Feb 2017 15:23:17 +0000 (17:23 +0200)]
ssl: Consistently use LibreSSL base headers.
Add missing include paths to prevent implicit search of <openssl/*> headers
where private crypto lib is linked in.
zrj [Tue, 7 Feb 2017 15:18:01 +0000 (17:18 +0200)]
Unbreak world after make upgrade REMOVE_OPENSSL_FILES=yes.
This is a partial solution to recover without requiring base system level
modifications. Patches to unhook /usr/include/openssl will follow next.
Matthew Dillon [Tue, 7 Feb 2017 03:07:43 +0000 (19:07 -0800)]
kernel - Default kernel build to -O2
* Default the kernel build to -O2, primarily so that code generated
from __predict_*() attributes is more optimal. There's slightly
more code bloat, but I think its time we move to this as a default.
Matthew Dillon [Tue, 7 Feb 2017 02:32:17 +0000 (18:32 -0800)]
kernel - Use __predict_* in spinlock2.h
* Use the __predict_* attributes in the spinlock hot-path.
Suggested-by: Mateusz Guzik
Matthew Dillon [Tue, 7 Feb 2017 02:06:14 +0000 (18:06 -0800)]
kernel - Reduce priority range from dynamic priority
* Reduce the priority range that the estcpu mechanism can move a process
over. The general effect is that a nice +20 process will now get much
less cpu than it used to verses a nice +0 process.
* In addition, due to the change, a nice +20 process will not get any cpu
at all verses a nice -12 or lower process. Before a nice +20 process
would always still get a little cpu verses a nice -20 process, but that
will no longer be the case.
Matthew Dillon [Mon, 6 Feb 2017 07:45:46 +0000 (23:45 -0800)]
kernel - Turn off pmap_dynamic_delete by default
* Turn this off until I can figure out how to fix it. It's panicing
systems.
Matthew Dillon [Mon, 6 Feb 2017 02:56:45 +0000 (18:56 -0800)]
kernel - Remove unneeded map token use from the vm_fault path.
* vm_fault() and vm_fault_page() does not need to hold the map_token.
They already hold the standard vm_map lock.
* This improves re-faulting in a multi-threaded environment, resulting
in a fully shared-lock and essentially uncontended path. It will not
improve initial faulting where a vm_page has to be allocated into the
backing vm_object (which requires an exclusive object token).
Matthew Dillon [Mon, 6 Feb 2017 02:51:24 +0000 (18:51 -0800)]
kernel - Refactor vm_page_alloc() to improve hot pages
* Since we do not pre-zero pages any more, nothing uses PG_ZERO and we no
longer have to use the head and tail of the free queue as an indication
of a possibly freed page being zerod or not.
For PQ_FREE, refactor the use case to always be LIFO, improving chances
that vm_page_alloc() will retrieve a page that is already hot in the
cache.
* Should improve the host, and in particular will improve the vkernel,
preventing it from unnecessarily cycling new pages (faulting them in
on the host) when normal operation is in steady-state, and also reducing
unnecessary initial/fresh faults.
Matthew Dillon [Mon, 6 Feb 2017 02:14:45 +0000 (18:14 -0800)]
kernel - Fix broken assertion for "pv->pv_m != NULL" panic
* A number of machines are getting this panic due to a broken assertion.
Fix the assertion.
Matthew Dillon [Mon, 6 Feb 2017 01:32:08 +0000 (17:32 -0800)]
vkernel - Change default pre-zeroing of main memory
* Do not pre-zero memory by default, because it interferes with the
ability to feed NUMA associations down into the vkernel.
* Add the -z option to force pre-zeroing of memory, which can be
useful when doing basic benchmarking (to avoid the overhead of the
vkernel having to do initial-pageins of its main memory, which can
extend over a long period of time).
Matthew Dillon [Mon, 6 Feb 2017 00:57:53 +0000 (16:57 -0800)]
kernel - Add missing p_token use in swapout scheduler callback
* allproc_scan() only PHOLDs(p). We also need p->p_token to safely
test the vmspace.
Matthew Dillon [Mon, 6 Feb 2017 00:46:59 +0000 (16:46 -0800)]
kernel - Disable vmm by default
* Leave hw.vmm.enabled at 0 even if VMM is detected for now. The vkernel
is not stable with it enabled.
John Marino [Sun, 5 Feb 2017 22:44:34 +0000 (16:44 -0600)]
libc: Add temporary (get|set)affinity DF404.0 symbols
The sched_getaffinity, sched_setaffinity and related work had the new
symbols erroneously given DF404.0 versioning. This was corrected by
commit 6262c9d, but that in turn broke any userland programs built
with the original symbol versions.
This patch redefines the old symbols while maintaining the correct
DF408.0 versions, but it is intended to be temporary. After the
branch (e.g. Release 4.8), this commit should be reverted so that
the next development trunk won't have the DF404.0 versions. It's
expected that all ports and third-party software is rebuilt when
upgrading to a new release. This serves as a reasonable bridge
to allow affected users to do that rebuild.
This patch is particularly intrusive regarding the syscall assembly
file creation, so it's not something we want to leave in place
permanently in my opinion.
Matthew Dillon [Sun, 5 Feb 2017 18:04:26 +0000 (10:04 -0800)]
kernel - Fix lockup due to recentn pmap change
* The recent change to move a test for skipping pages which do not
have PG_RW set was improperly restarting the loop from scratch,
resulting in an infinite loop.
Reported-by: David Shao, jh32
Matthew Dillon [Sun, 5 Feb 2017 05:09:51 +0000 (21:09 -0800)]
kernel - Increase temporary bootstack
* Increase the early boot temporary boot stack from 4KB to 16KB. Doesn't
fix anything, its just a good idea.
Matthew Dillon [Sun, 5 Feb 2017 05:06:40 +0000 (21:06 -0800)]
vkernel - Fix more pagein/pageout corruption
* There is a race when the real kernel walks a virtual page
table (VPAGETABLE) as created by a vkernel managing various
contexts. The real kernel may complete the lookup but get
interrupted by a pmap invalidation BEFORE it enters the results
into the pmap. The result is that the pmap invalidation is not
applied to the PTE entered into the pmap, leading to data corruption.
* Fix with a bit of a hack for now. Lock the VA in vm_fault and lock
the VA in MADV_INVAL operations (which is what the vkernel uses to
invalidate the pmap). This closes the hole.
* This race has to be fixed in the real kernel but normal programs outside
of a vkernel are not affected by it because they don't use VPAGETABLE
mappings.
* buildworld -j many in an intentionally hard-paging vkernel now completes
without error.
Sascha Wildner [Fri, 3 Feb 2017 18:14:40 +0000 (19:14 +0100)]
libc: Adjust Versions.def.
Sascha Wildner [Fri, 3 Feb 2017 18:06:39 +0000 (19:06 +0100)]
libc/sys: Clean up Symbol.map for the recent get/setaffinity work.
Sascha Wildner [Fri, 3 Feb 2017 17:21:00 +0000 (18:21 +0100)]
Remove the remaining traces of linux emulation.
List of other commits that removed linuxulator related things:
c98d47402ed663efb73e3bf836bb7d0963bae310
config(8): Comment out some linux emulation remains.
6737a4ffb92cef94699bb4eb7042e8125718fc02
Clean up some more after the sys/emulation/linux removal.
85aae976e20fc6b5be7ebb996fdf69c1c31e2b5b
kernel, world - Remove the remaining vestiges of linux emul
Sascha Wildner [Fri, 3 Feb 2017 17:20:31 +0000 (18:20 +0100)]
{rfork,setresuid}.2: Remove wrong/bogus sections.
Matthew Dillon [Wed, 1 Feb 2017 04:14:05 +0000 (20:14 -0800)]
kernel - Many fixes for vkernel support, plus a few main kernel fixes
REAL KERNEL
* The big enchillada is that the main kernel's thread switch code has
a small timing window where it clears the PM_ACTIVE bit for the cpu
while switching between two threads. However, it *ALSO* checks and
avoids loading the %cr3 if the two threads have the same pmap.
This results in a situation where an invalidation on the pmap in another
cpuc may not have visibility to the cpu doing the switch, and yet the
cpu doing the switch also decides not to reload %cr3 and so does not
invalidate the TLB either. The result is a stale TLB and bad things
happen.
For now just unconditionally load %cr3 until I can come up with code
to handle the case.
This bug is very difficult to reproduce on a normal system, it requires
a multi-threaded program doing nasty things (munmap, etc) on one cpu
while another thread is switching to a third thread on some other cpu.
* KNOTE after handling the vkernel trap in postsig() instead of before.
* Change the kernel's pmap_inval_smp() code to take a 64-bit npgs
argument instead of a 32-bit npgs argument. This fixes situations
that crop up when a process uses more than 16TB of address space.
* Add an lfence to the pmap invalidation code that I think might be
needed.
* Handle some wrap/overflow cases in pmap_scan() related to the use of
large address spaces.
* Fix an unnecessary invltlb in pmap_clearbit() for unmanaged PTEs.
* Test PG_RW after locking the pv_entry to handle potential races.
* Add bio_crc to struct bio. This field is only used for debugging for
now but may come in useful later.
* Add some global debug variables in the pmap_inval_smp() and related
paths. Refactor the npgs handling.
* Load the tsc_target field after waiting for completion of the previous
invalidation op instead of before. Also add a conservative mfence()
in the invalidation path before loading the info fields.
* Remove the global pmap_inval_bulk_count counter.
* Adjust swtch.s to always reload the user process %cr3, with an
explanation. FIXME LATER!
* Add some test code to vm/swap_pager.c which double-checks that the page
being paged out does not get corrupted during the operation. This code
is #if 0'd.
* We must hold an object lock around the swp_pager_meta_ctl() call in
swp_pager_async_iodone(). I think.
* Reorder when PG_SWAPINPROG is cleared. Finish the I/O before clearing
the bit.
* Change the vm_map_growstack() API to pass a vm_map in instead of
curproc.
* Use atomic ops for vm_object->generation counts, since objects can be
locked shared.
VKERNEL
* Unconditionally save the FP state after returning from VMSPACE_CTL_RUN.
This solves a severe FP corruption bug in the vkernel due to calls it
makes into libc (which uses %xmm registers all over the place).
This is not a complete fix. We need a formal userspace/kernelspace FP
abstraction. Right now the vkernel doesn't have a kernelspace FP
abstraction so if a kernel thread switches preemptively bad things
happen.
* The kernel tracks and locks pv_entry structures to interlock pte's.
The vkernel never caught up, and does not really have a pv_entry or
placemark mechanism. The vkernel's pmap really needs a complete
re-port from the real-kernel pmap code. Until then, we use poor hacks.
* Use the vm_page's spinlock to interlock pte changes.
* Make sure that PG_WRITEABLE is set or cleared with the vm_page
spinlock held.
* Have pmap_clearbit() acquire the pmobj token for the pmap in the
iteration. This appears to be necessary, currently, as most of the
rest of the vkernel pmap code also uses the pmobj token.
* Fix bugs in the vkernel's swapu32() and swapu64().
* Change pmap_page_lookup() and pmap_unwire_pgtable() to fully busy
the page. Note however that a page table page is currently never
soft-busied. Also other vkernel code that busies a page table page.
* Fix some sillycode in a pmap->pm_ptphint test.
* Don't inherit e.g. PG_M from the previous pte when overwriting it
with a pte of a different physical address.
* Change the vkernel's pmap_clear_modify() function to clear VTPE_RW
(which also clears VPTE_M), and not just VPTE_M. Formally we want
the vkernel to be notified when a page becomes modified and it won't
be unless we also clear VPTE_RW and force a fault. <--- I may change
this back after testing.
* Wrap pmap_replacevm() with a critical section.
* Scrap the old grow_stack() code. vm_fault() and vm_fault_page() handle
it (vm_fault_page() just now got the ability).
* Properly flag VM_FAULT_USERMODE.
Sascha Wildner [Thu, 2 Feb 2017 19:11:38 +0000 (20:11 +0100)]
Fix two typos in manual pages.
Matthew Dillon [Wed, 1 Feb 2017 02:33:33 +0000 (18:33 -0800)]
kernel - Avoid p->p_token when sending a signal to a particular lwp
* Avoid holding the per-process token excluysively when sending a signal
to a particular lwp. Hold it shared to locate the lwp, but otherwise do
all signal processing with the target lwp's token.
* Significantly improves vkernel timer operation. Timer distribution to
multiple cpus no longer trip over themselves.
Matthew Dillon [Wed, 1 Feb 2017 01:10:44 +0000 (17:10 -0800)]
kernel - Augment vm_fault_page() for vkernel operations
* Augment vm_fault_page(), adding required elements from vm_fault() that
were missing.
* In particular, this fixes a bug in any copyout to a growable stack segment
(including the copyouts the exec code does), where the stack was not being
properly grown and the copyout/suword64/etc calls were failing when they
shouldn't have.
* Note optimization in pmap_clean_pte(). When turning off VPTE_RW, we
do not have to MADV_INVAL to the real host if we determine that VPTE_M
is not set. Due to the way the vkernel works and host works, the host
will utilize a read-only real PTE for read faults on VPTE_RW entries,
in order to be able to detect writes by forcing a write fault when the
write occurs.
* Fix a race between pmap_enter() and pmap_page_protect(). The vkernel
doesn't have a convenient pv-lock like the read kernel pmap code, so
use the vm_page spinlock instead.
* Generally use atomic ops for other operations on ptes that should be
using atomic ops.
* Only clear PG_WRITEABLE while holding the vm_page spinlock.
Matthew Dillon [Wed, 1 Feb 2017 01:06:06 +0000 (17:06 -0800)]
vkernel - Adjust invalidation ABI a bit
* Make some adjustments to tighten up the atomic ops the vkernel uses to
modify VPTEs.
* Report unexpected VPTE_M races.
Matthew Dillon [Wed, 1 Feb 2017 01:04:42 +0000 (17:04 -0800)]
kernel - Validate that previous pte was cleaned.
* In pmap_enter(), validate that the previous pte was cleaned. Add
a debugging kprintf() if it wasn't.
Matthew Dillon [Wed, 1 Feb 2017 01:04:13 +0000 (17:04 -0800)]
kernel - Minor formatting cleanup
* Minor formatting cleanup
Matthew Dillon [Tue, 31 Jan 2017 06:23:47 +0000 (22:23 -0800)]
kernel - Fix race with vmspace_entry destroy race
* Fix a race in a host-side vkernel related structure that could
result in a double-free.
Matthew Dillon [Tue, 31 Jan 2017 02:25:17 +0000 (18:25 -0800)]
rc.d - Use kldstat -m to test existance before kldload
* Use kldstat -m module || kldload -n module || exit 1 (roughly) when
testing modules in rc scripts.
* Fixes issue where kernels boot up in secure mode (aka vkernels do this
by default) and rc scripts fail because kldload exits immediately with
an error, even if the module already exists.
Matthew Dillon [Mon, 30 Jan 2017 20:13:58 +0000 (12:13 -0800)]
vkernel - Change how VPTE_M works to fix seg-faults during paging
* Properly set and clear PG_WRITEABLE
* TAILQ_FOREACH() iterations on m->md.pv_list must be restarted
if we ever drop the spin-lock.
* Change VPAGETABLE semantics and operation, cleaning up some things
and fixing others.
Have the real-kernel only conditionally downgrade the real pte to
read-only for a VPTE_RW vpte. It only downgrades it if VPTE_M is
not set, improving performance.
Fix the virtual kernel to properly invalidate the real-kernel pte's
when clearing VPTE_M. This improves issues that crop up when the
vkernel is paging heavily.
* Replace the linear pv_plist with a RB tree. Also have pmap_remove_pages()
simply call pmap_remove().
Note that pmap_remove_pages()'s old code was broken because it only
scanned the pv_entry list and missed unmanaged pages. Fixing this
also fixes a vmspace reuse issue where the real-host pmap still
contained stale PTEs from prior use.
Matthew Dillon [Mon, 30 Jan 2017 17:45:58 +0000 (09:45 -0800)]
kernel - Fix races created by a comedy of circumstansces (3)
* Change pv semantics such that pv->pv_m must always exist while a pv is
installed in the pmap's RBTREE.
* Change pv_put() to assert that pv->pv_m exists. Use an unlock/drop
sequence for those cases where it might not exist.
* Fix an incorrect assertion.
* Move a pv_put() outside of the pmap spinlock that was incorrectly inside.
* Reorder how PG_MANGED_IDX / PG_UNMANAGED tests work.
Matthew Dillon [Mon, 30 Jan 2017 17:39:51 +0000 (09:39 -0800)]
kernel - Fix VTEXT/open race
* For now add O_RDWR to the list of flags that must use an exclusive
namecache lock when executing open(). This fixes a VTEXT/open race.
* This is not perfect w/regards to performance, so further enhancement may
be forthcoming.
Reported-by: mjg_ (Mateusz Guzik)
Matthew Dillon [Mon, 30 Jan 2017 17:37:58 +0000 (09:37 -0800)]
kernel - Fix shared/exclusive spinlock race
* Fix a long-standing bug in the shared spinlock code which could
unintentionally cause a contending exclusive waiter to acquire its
lock shared.
* Fixes a pmap issue exercised by the vkernel.
* The namecache also uses shared spinlocks but was far less likely to hit
the bug.
Matthew Dillon [Mon, 30 Jan 2017 04:25:42 +0000 (20:25 -0800)]
kernel - Add missing flag to vm_page_alloc() initializer.
* vm_page_alloc() (and the contig functions too) clear most vm_page->flags,
but not all. PG_ACTIONLIST was being improperly cleared.
* Fix the case (may fix occassional races in usched).
* Add a #define to ensure the flags we need to keep are defined in only
one place.
Sascha Wildner [Mon, 30 Jan 2017 08:56:27 +0000 (09:56 +0100)]
libc/libpthread: Some manual page cleanup after the *setaffinity* work.
Sascha Wildner [Mon, 30 Jan 2017 08:37:20 +0000 (09:37 +0100)]
libc: Remove the unused NO_P1003_1B variable checks.
Sascha Wildner [Sun, 29 Jan 2017 23:13:31 +0000 (00:13 +0100)]
Remove no longer needed include <sys/ccbque.h> (see
075c6d38244a).
Matthew Dillon [Sun, 29 Jan 2017 08:47:50 +0000 (00:47 -0800)]
kernel - Fix races created by a comedy of circumstances (2)
* Fix an issue where _pv_get() would not properly retry a pv lookup
after sleeping on a blocked placemark index.
Matthew Dillon [Sun, 29 Jan 2017 04:02:03 +0000 (20:02 -0800)]
kernel - Fix races created by a comedy of circumstances
* Fix several loop iteration races in the pmap code revealed by a chain
of events. Initially the commit to relax locking of vm_map for
MADV_INVAL operations (used by the vkernel) worked just fine. The
idea being that the pmap code was fine-grained locked and so a wider
encompassing lock should not have been needed.
However, a later commit to reduce the size of the zalloc pcpu caches
caused more cycling of pv_entry structures across cpus which revealed
a number of SMP races in the pmap code.
* A number of these races were fixed in recent commits and this commit
fixes a few more, primarily in the vm_page_t pv_entry loop iterators.
Matthew Dillon [Sat, 28 Jan 2017 22:08:39 +0000 (14:08 -0800)]
kernel - Fix races in the vm_map and vm_object page-scanning code
* Calling lwkt_yield() while depending on the vm_object token to stabilize
a vm_page_t is a bad idea. Move the calls to the end of the routine
where they don't hurt anyone (the RB tree scanning code can handle
ripouts, the callback functions cannot).
* Unconditionally busying a vm_page_t in a RB tree scanning callback
and then trying to re-check it to see if its still ok is a bad idea.
Instead, use vm_page_busy_try() so as not to break the object token
lock. If it fails, sleep on the page and set a retry condition but
do not act on it.
* Fix an issue in vm_object_pmap_remove() where the function operates
incorrect if passed a 0-length address range.
Matthew Dillon [Sat, 28 Jan 2017 22:07:02 +0000 (14:07 -0800)]
kernel - Use GDF_VIRTUSER to help measure user% in the vkernel host
* The vkernel host measures time in the vkernel VMM (when VMM is enabled)
as system time. Use the GDF_VIRTUSER flag to try to measure that time
as user time.
Matthew Dillon [Sat, 28 Jan 2017 18:30:53 +0000 (10:30 -0800)]
kernel - Fix vkernel race in host
* Obtaining a cache_ref before setting the DELETED flag deals with any
drop race on cache_refs from another thread.
This is actually quite elegant when done correctly. The VKE_REF_DELETED
on ve->refs determines when the ve has been destroyed, while
ve->cache_refs determines when to kfree() the ve. Acquiring an extra
cache_ref before setting DELETED interlocks the two fields for last-drop
purposes.
Matthew Dillon [Sat, 28 Jan 2017 08:19:59 +0000 (00:19 -0800)]
vkernel - Save/restore errno in signal code
* Save and restore errno in the signal handling code, as most of this code
can potentially make system calls.
Matthew Dillon [Sat, 28 Jan 2017 08:18:33 +0000 (00:18 -0800)]
vkernel - Fix numerous pmap invalidation bugs
* Fix various vkernel pmap invalidation bugs and normalize the code a bit.
Matthew Dillon [Sat, 28 Jan 2017 08:17:37 +0000 (00:17 -0800)]
vkernel - Zero out memory on startup
* Zero out memory on startup to force it to be paged in. This was already
done for non-VMM mode. Also do it for VMM mode. Work in progress.
Matthew Dillon [Sat, 28 Jan 2017 08:16:35 +0000 (00:16 -0800)]
kernel - Adjust vmm_guest_sync_addr()
* Adjust vmm_guest_sync_addr() to invalidate the current cpu's tlb
only for VMM when called with (NULL, NULL).
Matthew Dillon [Sat, 28 Jan 2017 08:08:05 +0000 (00:08 -0800)]
kernel - Change vm_fault_page[_quick]() semantics + vkernel fixes
* vm_fault_page[_quick]() needs to be left busied for PROT_WRITE so
modifications made by the caller do not race other operations in the
kernel. Modify the API to accomodate the behavior.
* Fix procfs write race with new vm_fault_page() API.
* Fix bugs in ept_swapu32() and ept_swapu64() (vkernel + VMM)
* pmap_fault_page_quick() doesn't understand EPT page tables, have it
fail for that case too. This fixes bugs in vkernel + VMM mode.
* Also do some minor normalization of variables names in pmap.c
* vkernel/pmap - Use atomic_swap_long() to modify PTEs instead of a
simple (non-atomic) assignment.
* vkernel/pmap - Fix numerous bugs in the VMM and non-VMM code for
pmap_kenter*(), pmap_qenter*(), etc.
* vkernel/pmap - Collapse certain pmap_qremove_*() routines into the
base pmap_qremove().
Matthew Dillon [Sat, 28 Jan 2017 08:00:54 +0000 (00:00 -0800)]
kernel - Remove ffs_rawread() (2)
* Remove conf/files entry
Matthew Dillon [Fri, 27 Jan 2017 23:05:11 +0000 (15:05 -0800)]
kernel - Remove numerous user VM page functions
* Remove vmapbuf() and vunmapbuf(), they are unsafe.
* Remove xio_init_ubuf() - It is not used and is unsafe.
* Remove vm_fault_quick_hold_pages() - It is not used as is unsafe.
Matthew Dillon [Fri, 27 Jan 2017 23:04:03 +0000 (15:04 -0800)]
kernel - Remove direct mapping of user buffers to CAM
* Remove direct mappings, always bounce. Nothing uses this code any
more and vmapbuf/vunmapbuf are about to go away, so just bounce.
Matthew Dillon [Fri, 27 Jan 2017 22:56:47 +0000 (14:56 -0800)]
kernel - Remove ffs_rawread()
* Remove ffs_rawread(), because we will be removing vmapbuf/vunmapbuf soon
as well (because it isn't safe).
Matthew Dillon [Fri, 27 Jan 2017 21:56:33 +0000 (13:56 -0800)]
vkernel - Partial fix to EPT swapu32 and swapu64
* EPT needed swapu32/swapu64 functions, write them.
* Fix bounds checking bug in std_swapu32()
* Misc cleanups.
Matthew Dillon [Fri, 27 Jan 2017 18:33:15 +0000 (10:33 -0800)]
kernel - Prune pcpu vmzone a bit
* Put a cap of 16MB worth of entries in the vm_zone pcpu cache, per cpu.
This cap is triggered on large-memory systems which would otherwise
set too high a limit on the vmzone pcpu caches.
This primarily affects pvzone, limiting the pcpu pmap pv_entry cache
to around 200000 elements (~800MB worth of page mappings).
* Reduces the amount of memory waste on a large-memory system. For example,
before roughly 2GB worth of unused pv_entry's could accumulate on a
32-way system with 128GB of ram during nominal use. Now only around 512MB
worth of unused pv_entry's can accumulate.
* Note that we do not want to set this cap too low because it is absolutely
necessary for concurrent fork/exec/wait/exit performance.
* Increase the zone_burst from the global cache to the pcpu cache from
32 to 128 elements to further reduce unnecessary lock collisions in
zalloc().
Sascha Wildner [Fri, 27 Jan 2017 18:29:23 +0000 (19:29 +0100)]
fetch.9/store.9: Adjust for recent changes.
Imre Vadász [Fri, 27 Jan 2017 12:28:39 +0000 (13:28 +0100)]
kernel/usb4bsd: Use pci_alloc_1intr() for MSI allocation in xhci_pci.c.
* The existing pci_alloc_msi() call was still expecting the FreeBSD API,
causing a panic on single-cpu machines when trying to allocate an MSI.
SMP machines didn't panic, but were accidentally always using CPU1 for
the xhci MSI interrupt.
Imre Vadász [Wed, 11 Jan 2017 11:47:14 +0000 (12:47 +0100)]
sdhci - Use pci_alloc_1intr in sdhci pci attachement.
Matthew Dillon [Fri, 27 Jan 2017 05:52:10 +0000 (21:52 -0800)]
msdosfs - Mount ricoh camera cards
* Ricoh cameras mis-format their sdcards, mis-programming the 64-bit
pm_nxtfree field with a 32-bit-truncated -1. Allow the case so we
can finish mounting the card.
Reported-by: luxh
Matthew Dillon [Thu, 26 Jan 2017 22:45:20 +0000 (14:45 -0800)]
kernel - Fix improper dereference in the vmspace subsystem
* Fix an improper deref in the vmspace subsystem (used by vkernels),
which could cause host memory to become corrupt.
Matthew Dillon [Thu, 26 Jan 2017 19:37:58 +0000 (11:37 -0800)]
vkernrel - Synchronize w/master, adjust for vmm_guest_sync_addr() changes
* synchronize suword, fuword, etc naming conventions with master.
* Use the new vmm_guest_sync_addr() ABI to more safely make adjustments
to virtual page tables.
Matthew Dillon [Thu, 26 Jan 2017 19:35:47 +0000 (11:35 -0800)]
kernel - Refactor suword, fuword, etc. change vmm_guest_sync_addr()
* Rename the entire family of functions to reduce confusion.
* Change how vmm_guest_sync_addr() works. Instead of loading one value
into a target location we exchange the two target locations, with the
first address using an atomic op. This will allow the vkernel to
drop privs and query pte state atomically.
Matthew Dillon [Thu, 26 Jan 2017 19:34:31 +0000 (11:34 -0800)]
kernel - Accomodate high-ncpu + low-mem configurations
* Make some adjustments to accomodate high cpu core count but low physical
memory configurations. This mainly accomodates test configurations.
Do set pageout minimums or targets to unreasonable values in these
configurations.
Matthew Dillon [Thu, 26 Jan 2017 02:31:48 +0000 (18:31 -0800)]
vkernel - Refactor pmap
* Refactor the pmap code. Use vm_page locking to protect PTEs.
* Change the accounting from using vm_page->hold_count to using
vm_page->wire_count.
* Replace unlocked pt/pd/pdp lookups with explicit page tests for non-kernel
pmaps.
Matthew Dillon [Thu, 26 Jan 2017 02:30:43 +0000 (18:30 -0800)]
kernel - Remove object->agg_pv_list_count
* Remove the object->agg_pv_list_count field. It represents an unnecessary
global cache bounce, was only being used to help report vkernel RSS,
and wasn't working very well anyway.
Matthew Dillon [Thu, 26 Jan 2017 01:39:01 +0000 (17:39 -0800)]
kernel - Optimize lwp-specific signaling. (2)
* Fix bug in last sig commit.
Matthew Dillon [Wed, 25 Jan 2017 19:18:20 +0000 (11:18 -0800)]
vkernel - Clock optimizations, pre-zero 'ram'.
* The vkernel now defaults to using an imprecise (host-tick-resolution)
timer.
* The vkernel now pre-faults all of its 'system' memory at startup in order
to avoid dynamically faulting it in after it has gone MP (which creates
a lot of contention on the host machine).
Matthew Dillon [Wed, 25 Jan 2017 17:25:40 +0000 (09:25 -0800)]
kernel - Optimize lwp-specific signaling.
* Optimize the signal code to remove most instances of needing proc->p_token
when lwp-specific signals are sent.
* Add a CURSIG_LCK_TRACE() macro which can now return with p_token held, and
pass the status to postsig() which then consumes it.
* lwpsignal() now tries very hard to avoid acquiring proc->p_token.
* Significantly improves vkernel operation under heavy (vkernel) IPI loads.
Matthew Dillon [Wed, 25 Jan 2017 16:11:57 +0000 (08:11 -0800)]
kernel - Allow VM_FAULT_DIRTY to use a shared lock
* Allows VM_FAULT_DIRTY (write) faults to use a shared object lock as
long as the evnetual page does not need to be unswapped.
* More than doubles vkernel performance when doing a parallel make
by removing a major contention spot.
Matthew Dillon [Wed, 25 Jan 2017 16:11:04 +0000 (08:11 -0800)]
procfs - don't try to count rss
* Don't try to count rss. Scanning individual pages for a 64-bit
mapping can take forever (literally!).
* Fixes problems accessing /proc/*/map for vkernel processes.
Sepherosa Ziehau [Wed, 25 Jan 2017 09:19:38 +0000 (17:19 +0800)]
cputimer: Update comment
Matthew Dillon [Mon, 23 Jan 2017 20:20:23 +0000 (12:20 -0800)]
kernel - pmap and vkernel work
* Remove the pmap.pm_token entirely. The pmap is currently protected
primarily by fine-grained locks and the vm_map lock. The intention
is to eventually be able to protect it without the vm_map lock at all.
* Enhance pv_entry acquisition (representing PTE locations) to include
a placemarker facility for non-existant PTEs, allowing the PTE location
to be locked whether a pv_entry exists for it or not.
* Fix dev_dmmap (struct dev_mmap) (for future use), it was returning a
page index for physical memory as a 32-bit integer instead of a 64-bit
integer.
* Use pmap_kextract() instead of pmap_extract() where appropriate.
* Put the token contention test back in kern_clock.c for real kernels
so token contention shows up as sys% instead of idle%.
* Modify the pmap_extract() API to also return a locked pv_entry,
and add pmap_extract_done() to release it. Adjust users of
pmap_extract().
* Change madvise/mcontrol MADV_INVAL (used primarily by the vkernel)
to use a shared vm_map lock instead of an exclusive lock. This
significantly improves the vkernel's performance and significantly
reduces stalls and glitches when typing in one under heavy loads.
* The new placemarkers also have the side effect of fixing several
difficult-to-reproduce bugs in the pmap code, by ensuring that
shared and unmanaged pages are properly locked whereas before only
managed pages (with pv_entry's) were properly locked.
* Adjust the vkernel's pmap code to use atomic ops in numerous places.
* Rename the pmap_change_wiring() call to pmap_unwire(). The routine
was only being used to unwire (and could only safely be called for
unwiring anyway). Remove the unused 'wired' and the 'entry'
arguments.
Also change how pmap_unwire() works to remove a small race condition.
* Fix race conditions in the vmspace_*() system calls which could lead
to pmap corruption. Note that the vkernel did not trigger any of
these conditions, I found them while looking for another bug.
* Add missing maptypes to procfs's /proc/*/map report.
Sepherosa Ziehau [Wed, 25 Jan 2017 02:24:36 +0000 (10:24 +0800)]
lwp: Restore lwp syscalls (except lwp_create*) declaration.
Sepherosa Ziehau [Wed, 25 Jan 2017 01:46:40 +0000 (09:46 +0800)]
lwp: Bump version for shuffle lwp syscalls.
Sepherosa Ziehau [Tue, 24 Jan 2017 13:42:34 +0000 (21:42 +0800)]
cputimer: Reorder fields a bit to avoid implicit padding.