freebsd.git
4 years agoMFV r356143:
cy [Tue, 31 Dec 2019 15:50:41 +0000 (15:50 +0000)]
MFV r356143:

Update unbound 1.9.2 --> 1.9.6.

MFC after: 3 days
Security: CVE-2019-18934 (fixed by 1.9.5)

4 years agoAdd flags for upcoming patches related to improved ECN handling.
tuexen [Tue, 31 Dec 2019 14:32:48 +0000 (14:32 +0000)]
Add flags for upcoming patches related to improved ECN handling.
No functional change.
Submitted by: Richard Scheffenegger
Reviewed by: rgrimes@, tuexen@
Differential Revision: https://reviews.freebsd.org/D22429

4 years agoClear the flag indicating that the last received packet was marked CE also
tuexen [Tue, 31 Dec 2019 14:23:52 +0000 (14:23 +0000)]
Clear the flag indicating that the last received packet was marked CE also
in the case where a packet not marked was received.

Submitted by: Richard Scheffenegger
Reviewed by: rgrimes@, tuexen@
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19143

4 years agopsm: properly check for atkbdc_open failure
kevans [Tue, 31 Dec 2019 13:56:48 +0000 (13:56 +0000)]
psm: properly check for atkbdc_open failure

atkbdc_open can return NULL if the unit's out of bounds or the softc isn't
setup. Check it to be safe.

4 years agoAdd curly braces missed in https://svnweb.freebsd.org/changeset/base/354773
tuexen [Tue, 31 Dec 2019 12:29:01 +0000 (12:29 +0000)]
Add curly braces missed in https://svnweb.freebsd.org/changeset/base/354773

Sponsored by: Netflix, Inc.
CID: 1407649

4 years agoriscv: Remove unused variable
kp [Tue, 31 Dec 2019 11:45:19 +0000 (11:45 +0000)]
riscv: Remove unused variable

Fix the build that was broken by r356221. Pointy hat to myself.

4 years agosifive: uart driver
kp [Tue, 31 Dec 2019 10:54:13 +0000 (10:54 +0000)]
sifive: uart driver

Implement support for the UART as found on the SiFive FU540. It should
also work on, but has not been tested with, the FU310.

Reviewed by: philip
Sponsored by: Axiado

4 years agoriscv: Remove pointless loop
kp [Tue, 31 Dec 2019 10:53:03 +0000 (10:53 +0000)]
riscv: Remove pointless loop

There's no point in checking for absent CPUs if we're not going to do anything
about either the present or absent case. This loop can just be removed.

Reviewed by: philip
Sponsored by: Axiado

4 years agortld(1): Do booleans like C99
cem [Tue, 31 Dec 2019 05:41:47 +0000 (05:41 +0000)]
rtld(1): Do booleans like C99

Reviewed by: kib, rlibby
Differential Revision: https://reviews.freebsd.org/D22964

4 years agoSwitch r356210 to use gone_in() instead of printf().
rmacklem [Tue, 31 Dec 2019 05:39:27 +0000 (05:39 +0000)]
Switch r356210 to use gone_in() instead of printf().

Suggested by: cem

4 years agoinetd: prefer strlcpy to strlen(3) check + strcpy(3), NFC
kevans [Tue, 31 Dec 2019 04:53:50 +0000 (04:53 +0000)]
inetd: prefer strlcpy to strlen(3) check + strcpy(3), NFC

This is again functionally equivalent but more concise.

4 years agoinetd: prefer strtonum(3) to strspn(3)+atoi(3), NFC
kevans [Tue, 31 Dec 2019 04:36:14 +0000 (04:36 +0000)]
inetd: prefer strtonum(3) to strspn(3)+atoi(3), NFC

strtonum(3) does effectively the same validation as we had, but it's more
concise.

4 years agoDon't spin on cleanup_lock if we are not interrupt.
mav [Tue, 31 Dec 2019 04:16:52 +0000 (04:16 +0000)]
Don't spin on cleanup_lock if we are not interrupt.

If somebody else holds that lock, it will likely do the work for us.
If it won't, then we return here later and retry.

Under heavy load it allows to avoid lock congestion between interrupt and
polling threads.

MFC after: 1 week
Sponsored by: iXsystems, Inc.

4 years agoinetd: knock out some clang analyze warnings
kevans [Tue, 31 Dec 2019 04:00:24 +0000 (04:00 +0000)]
inetd: knock out some clang analyze warnings

chargen_dg: clang-analyze is convinced that endring could be non-NULL at
entry, and thus wants to assume that rs == NULL. Just independently
initialize rs if it's NULL to appease the analyzer.

getconfigent: policy leaks on return

free_connlist: reorganize the loop to make it clear that we're not going to
access `conn` after it's been freed.

cpmip/hashval: left-shifts performed will result in UB as we take
signed 0xABC3D20F and left shift it by 5.

4 years agoAvoid few memory accesses in g_disk_done().
mav [Tue, 31 Dec 2019 03:43:13 +0000 (03:43 +0000)]
Avoid few memory accesses in g_disk_done().

4 years agoReplace .h files included from old NFS directories with the new NFS ones.
rmacklem [Tue, 31 Dec 2019 00:07:10 +0000 (00:07 +0000)]
Replace .h files included from old NFS directories with the new NFS ones.

Prior to this patch, nfsstat.c includes files from sys/nfs, sys/nfsclient
and sys/nfsserver. These .h files (particularily the ones in sys/nfsclient
and sys/nfsserver) are from the old NFS code and should eventually be
deprecated/removed.
This patch changes nfsstat.c to include files from the new/current NFS
code instead of the old ones in preparation for eventual removal.

4 years agoMFV r356163,r356197:
mm [Tue, 31 Dec 2019 00:05:06 +0000 (00:05 +0000)]
MFV r356163,r356197:
Update libarchive to 3.4.1

Relevant vendor changes since last update:
  Issue #351: Refactor and implement private state logic for write filters
  PR #1252: RAR5 reader - verify window size for solid files (OSS-Fuzz 15482)
  PR #1255: zip writer - don't append unused NUL for directories
  PR #1260: Fix sparse file offset overflow on 32-bit systems
  PR #1263: UNICODE filename support for reading lha/lzh format
  Issue #1276: Bugfix and optimize archive_wstring_append_from_mbs()
  PR #1288: Add the "xattrhdr" option to pax write options
  PR #1295: 7z reader - fix reading archives with digests in PackInfo
  PR #1296: RAR5 reader - verify window size for multivolume archives
  PR #1297: ZIP reader - support LZMA_STREAM_END marker in 'lzma alone' files
  Issue #1298: Fix a heap-buffer-overflow in archive_string_append_from_wcs()
  OSS-Fuzz 19360, 19362: LHA reader - plug two memory leaks on error
  Fix possible off-by-one when dealing with readlink(2)

MFC after: 2 weeks

4 years agoSet a "kernbase" symbol in 32-bit arm locore.S and use it with ldscript.arm.
ian [Mon, 30 Dec 2019 23:20:46 +0000 (23:20 +0000)]
Set a "kernbase" symbol in 32-bit arm locore.S and use it with ldscript.arm.

This brings arm into line with how every other arch does it.  For some
reason, only arm lacked a definition of a symbol named kernbase in its
locore.S file(s) for use in its ldscript.arm file.  Needlessly different
means harder to maintain.

Using a common symbol name also eases work in progress on a script to help
generate arm and arm64 kernels packaged in various ways (like with a header
blob needed for a bootloader prepended to the kernel file).

4 years agoAdd warning printf w.r.t. removal of sys/nfs/nfs_lock.c.
rmacklem [Mon, 30 Dec 2019 22:39:29 +0000 (22:39 +0000)]
Add warning printf w.r.t. removal of sys/nfs/nfs_lock.c.

The code in sys/nfs/nfs_lock.c has not been run by default since March 2008
when it was replaced by the in kernel sys/nlm code.
It uses Giant, so it needs to be removed before the FreeBSD 13 release.
This will happen in a couple of months, since few if any users run
the code anyhow and can easily switch to the default in kernel NFSLOCKD.

4 years agoFormatting fixes for tables, no content changes.
jhb [Mon, 30 Dec 2019 22:05:57 +0000 (22:05 +0000)]
Formatting fixes for tables, no content changes.

- Add missing .Pp after the end of some lists so that there is a blank
  line before the subsequent paragraph.
- Use a more typical '-tag' bullet list of the make variable descriptions
  at the end.  This adds separation between bullets and is the formatting
  typically used in manpages for this sort of list.

4 years agosh: Test that executing various binary files is rejected
jilles [Mon, 30 Dec 2019 21:32:55 +0000 (21:32 +0000)]
sh: Test that executing various binary files is rejected

If executing a file fails with an [ENOEXEC] error, the shell executes the
file as a shell script, except that this execution may instead result in an
error message if the file is binary.

Per a recent Austin Group interpretation, we will need to change this to
allow a concatenation of a shell script and a binary payload. See
Austin Group bugs #1226 and #1250.

MFC after: 1 week

4 years agoDetermine whether the MMU hardware is capable of updating a page table
alc [Mon, 30 Dec 2019 20:30:31 +0000 (20:30 +0000)]
Determine whether the MMU hardware is capable of updating a page table
entry's access flag and dirty state, and enable this feature when it's
available.

Ensure that we don't overlook a dirty state update that is concurrent
with a call to pmap_enter().  (Previously, all dirty state updates would
have occurred with the containing pmap's lock held, so a page table entry's
dirty state could not have changed while pmap_enter() held that same lock.)

Reviewed by: andrew, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22907

4 years agolinux(4): implement copy_file_range(2)
kaktus [Mon, 30 Dec 2019 18:11:06 +0000 (18:11 +0000)]
linux(4): implement copy_file_range(2)

copy_file_range(2) is implemented natively since r350315, make it available
for Linux binaries too.

Reviewed by: kib (mentor), trasz (previous version)
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D22959

4 years agoDon't hard-code field offsets of struct riscv_bootparams.
br [Mon, 30 Dec 2019 17:18:50 +0000 (17:18 +0000)]
Don't hard-code field offsets of struct riscv_bootparams.

Submitted by: James Clarke <jrtc27@jrtc27.com>
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22970

4 years agoinetd: don't leak `policy` on return
kevans [Mon, 30 Dec 2019 15:54:36 +0000 (15:54 +0000)]
inetd: don't leak `policy` on return

sep->se_policy gets a strdup'd version of policy, so we don't need it to
stick around afterwards.

While here, remove a couple of NULL checks prior to free(policy).

CID: 1006865
MFC after: 3 days

4 years agoDon't hard-code size of struct riscv_bootparams.
br [Mon, 30 Dec 2019 14:38:06 +0000 (14:38 +0000)]
Don't hard-code size of struct riscv_bootparams.

Submitted by: James Clarke <jrtc27@jrtc27.com>
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22968

4 years agoRound the kernel stack allocation up as required.
br [Mon, 30 Dec 2019 13:29:30 +0000 (13:29 +0000)]
Round the kernel stack allocation up as required.

Submitted by: James Clarke <jrtc27@jrtc27.com>
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22961

4 years agoFix spelling.
hselasky [Mon, 30 Dec 2019 09:22:52 +0000 (09:22 +0000)]
Fix spelling.

PR: 242891
MFC after: 1 week
Sponsored by: Mellanox Technologies

4 years agoUse atomic for start_count in devstat_start_transaction().
mav [Mon, 30 Dec 2019 03:13:38 +0000 (03:13 +0000)]
Use atomic for start_count in devstat_start_transaction().

Combined with earlier nstart/nend removal it allows to remove several locks
from request path of GEOM and few other places.  It would be cool if we had
more SMP-friendly statistics, but this helps too.

Sponsored by: iXsystems, Inc.

4 years ago[PowerPC] Fix panic when attempting to handle an HMI from an idle thread
bdragon [Mon, 30 Dec 2019 02:56:47 +0000 (02:56 +0000)]
[PowerPC] Fix panic when attempting to handle an HMI from an idle thread

In IRC, sfs_ finally managed to get a good trace of a kernel panic that was
happening when attempting to use webengine.

As it turns out, we were using vtophys() from interrupt context on an idle
thread in opal_hmi_handler2().

Since this involves locking the kernel pmap on PPC64 at the moment, this
ended up tripping a KASSERT in mtx_lock(), which then caused a parallel
panic stampede.

So, avoid this by preallocating the flags variable and storing it in PCPU.

Fixes "panic: mtx_lock() by idle thread 0x... on sleep mutex kernelpmap".

Differential Revision: https://reviews.freebsd.org/D22962

4 years agosc(4) md bits: stop setting sc->kbd entirely
kevans [Mon, 30 Dec 2019 02:07:55 +0000 (02:07 +0000)]
sc(4) md bits: stop setting sc->kbd entirely

The machdep parts no longer need to touch keyboard parts after r356043;
sc->kbd will be 0-initialized and this works as expected.

4 years agovirtio_random(4): Drop no longer needed unload kludge
cem [Mon, 30 Dec 2019 01:40:59 +0000 (01:40 +0000)]
virtio_random(4): Drop no longer needed unload kludge

After r356194, source deregistration no longer races with concurrent use.

Reviewed by: bryanv, markm
Differential Revision: https://reviews.freebsd.org/D22490

4 years agorandom(4): Make entropy source deregistration safe
cem [Mon, 30 Dec 2019 01:38:19 +0000 (01:38 +0000)]
random(4): Make entropy source deregistration safe

Allow loadable modules that provide random entropy source(s) to safely
unload.  Prior to this change, no driver could ensure that their
random_source structure was not being used by random_harvestq.c for any
period of time after invoking random_source_deregister().

This change converts the source_list LIST to a ConcurrencyKit CK_LIST and
uses an epoch(9) to protect typical read accesses of the list.  The existing
HARVEST_LOCK spin mutex is used to safely add and remove list entries.
random_source_deregister() uses epoch_wait() to ensure no concurrent
source_list readers are accessing a random_source before freeing the list
item and returning to the caller.

Callers can safely unload immediately after random_source_deregister()
returns.

Reviewed by: markj
Approved by: csprng(markm)
Discussed with: jhb
Differential Revision: https://reviews.freebsd.org/D22489

4 years agoRetire nstart/nend counters.
mav [Mon, 30 Dec 2019 00:46:10 +0000 (00:46 +0000)]
Retire nstart/nend counters.

Those counters were abused for decade to workaround broken orphanization
process in different classes by delaying the call while there are active
requests.  But from one side it did not close all the races, while from
another was quite expensive on SMP due to trashing twice per request cache
lines of consumer and provider and requiring locks.  It lost its sense
after I manually went through all the GEOM classes in base and made
orphanization wait for either provider close or request completion.

Consumer counters are still used under INVARIANTS to detect premature
consumer close and detach.  Provider counters are removed completely.

Sponsored by: iXsystems, Inc.

4 years agoAdd libdtrace support for arm64 USDT probes.
markj [Sun, 29 Dec 2019 21:46:50 +0000 (21:46 +0000)]
Add libdtrace support for arm64 USDT probes.

arm64 is still lacking a fasttrap implementation, which is required to
actually enable userland probes, but this at least allows USDT probes to
be linked into userland applications.

Submitted by: Klaus Küchemann <maciphone2@googlemail.com> (original)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22360

4 years agoAdd ARMv8 static relocation types used for control flow instructions.
markj [Sun, 29 Dec 2019 21:46:00 +0000 (21:46 +0000)]
Add ARMv8 static relocation types used for control flow instructions.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

4 years agoRemove GEOM_SCHED class and gsched tool.
mav [Sun, 29 Dec 2019 21:16:03 +0000 (21:16 +0000)]
Remove GEOM_SCHED class and gsched tool.

This code was not actively maintained since it was introduced 10 years ago.
It lacks support for many later GEOM features, such as direct dispatch,
unmapped I/O, stripesize/stripeoffset, resize, etc.  Plus it is the only
remaining use of GEOM nstart/nend request counters, used there to implement
live insertion/removal, questionable by itself.  Plus, as number of people
commented, GEOM is not the best place for I/O scheduler, since it has
limited information about layers both above and below it, required for
efficient scheduling.  Plus with the modern shift to SSDs there is just no
more significant need for this kind of scheduling.

Approved by: imp, phk, luigi
Relnotes: yes

4 years agopowerd(8): Fix a memory leak when we fail to read frequency levels.
markj [Sun, 29 Dec 2019 20:57:49 +0000 (20:57 +0000)]
powerd(8): Fix a memory leak when we fail to read frequency levels.

PR: 242967
Submitted by: Hans Christian Woithe <chwoithe@yahoo.com>
MFC after: 3 days

4 years agoRestore a vm_page_wired() check in vm_page_mvqueue() after r356156.
markj [Sun, 29 Dec 2019 20:01:03 +0000 (20:01 +0000)]
Restore a vm_page_wired() check in vm_page_mvqueue() after r356156.

We now set PGA_DEQUEUE on a managed page when it is wired after
allocation, and vm_page_mvqueue() ignores pages with this flag set,
ensuring that they do not end up in the page queues.  However, this is
not sufficient for managed fictitious pages or pages managed by the
TTM.  In particular, the TTM makes use of the plinks.q queue linkage
fields for its own purposes.

PR: 242961
Reported and tested by: Greg V <greg@unrelenting.technology>

4 years agoMissed part of r356162.
mav [Sun, 29 Dec 2019 19:33:41 +0000 (19:33 +0000)]
Missed part of r356162.

If we postpone consumer destruction till close, then the close calls should
not be ignored.  Delay geom withering till the last close too.

MFC after: 2 weeks
X-MFC-with: r356162
Sponsored by: iXsystems, Inc.

4 years agoFix GEOM_VIRSTOR orphanization.
mav [Sun, 29 Dec 2019 19:21:29 +0000 (19:21 +0000)]
Fix GEOM_VIRSTOR orphanization.

Previous code closed and destroyed consumer even with I/O in progress.
This patch postpones the destruction till the last close.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

4 years agoEliminate the generated ldscript for arm and arm64, and strip $a/$d marker
ian [Sun, 29 Dec 2019 18:17:12 +0000 (18:17 +0000)]
Eliminate the generated ldscript for arm and arm64, and strip $a/$d marker
symbols from the linked kernel.

The main thrust of this change is to generate a kernel that has the arm
"marker" symbols stripped. Marker symbols start with $a, $d, $t or $x, and
are emitted by the compiler to tell other toolchain components about the
locations of data embedded in the instruction stream (literal-pool
stuff). They are used for generating mixed-endian binaries (which we don't
support). The linked kernel has approximately 21,000 such symbols in it,
wasting space (500K in kernel.full, 190K in the final linked kernel), and
sometimes obscuring function names in stack tracebacks.

This change also simplifies the way the kernel is linked. Instead of using
sed to generate two different ldscript files to generate both an elf kernel
and a binary (elf headers stripped) kernel, we now use a single ldscript
that refers to a "text_start" symbol, and we provide the value for that
symbol using --defsym on the linker command line.

4 years agosysctlbyname(2): Remove temporary compatibility layer
kaktus [Sun, 29 Dec 2019 17:19:57 +0000 (17:19 +0000)]
sysctlbyname(2): Remove temporary compatibility layer

Remove temporary compatibility layer introduced in r351729. More that 3 months
should be enough for everybody who runs HEAD to upgrade to the new kernel
already.

Reviewed by: imp, mjg (mentor)
Approved by: mjg (mentor)
Differential Revision: https://reviews.freebsd.org/D22958

4 years agoFix GEOM_MOUNTVER orphanization.
mav [Sun, 29 Dec 2019 17:10:21 +0000 (17:10 +0000)]
Fix GEOM_MOUNTVER orphanization.

Previous code closed and detached consumer even with I/O still in progress.
This patch adds locking and request counting to postpone the close till
the last of running requests completes.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

4 years agoImplement Linux syslog(2) syscall; just enough to make Linux dmesg(8)
trasz [Sun, 29 Dec 2019 15:53:55 +0000 (15:53 +0000)]
Implement Linux syslog(2) syscall; just enough to make Linux dmesg(8)
utility work.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22465

4 years agognop: change the "count until fail" option
oshogbo [Sun, 29 Dec 2019 15:47:37 +0000 (15:47 +0000)]
gnop: change the "count until fail" option

Change the "count_until_fail" option of gnop, now it enables the failing
rating instead of setting them to 100%.

The original patch introduced the new flag, which sets the fail/rate to 100%
after N requests. In some cases, we don't want to have 100% of failure
probabilities. We want to start failing at some point.
For example, on the early stage, we may like to allow some read/writes requests
before having some requests delayed - when we try to mount the partition,
or when we are trying to import the pool.
Another case may be to check how scrub in ZFS will behave on different stages.

This allows us to cover more cases.
The previous behavior still may be configured.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D22632

4 years agoMake linprocfs(5) provide an empty /proc/modules. This should silence
trasz [Sun, 29 Dec 2019 15:46:59 +0000 (15:46 +0000)]
Make linprocfs(5) provide an empty /proc/modules.  This should silence
some warnings.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

4 years agognop: allow to change the name of created device
oshogbo [Sun, 29 Dec 2019 15:40:02 +0000 (15:40 +0000)]
gnop: allow to change the name of created device

Thanks to this option we can create more then one gnop provider from
single provider. This may be useful for temporary labeling some data
on the disk.

Reviewed by: markj, allanjude, bcr
Differential Revision: https://reviews.freebsd.org/D22304

4 years agoClear queue op flags in vm_page_mvqueue().
markj [Sun, 29 Dec 2019 15:39:43 +0000 (15:39 +0000)]
Clear queue op flags in vm_page_mvqueue().

This fixes a regression in r356155, introduced at the last minute.  In
particular, we must clear PGA_REQUEUE_HEAD before inserting into any
queue besides PQ_INACTIVE since that operation is implemented only for
PQ_INACTIVE.

Reported by: pho, Jenkins via lwhsu

4 years agoMake Linux stat(2) et al distinguish between block and character
trasz [Sun, 29 Dec 2019 13:54:02 +0000 (13:54 +0000)]
Make Linux stat(2) et al distinguish between block and character
devices.  It's required for LTP, among other things.  It's not
complete, but good enough for now.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22950

4 years agoMove type casts into a single place. No functional changes.
trasz [Sun, 29 Dec 2019 12:24:41 +0000 (12:24 +0000)]
Move type casts into a single place.  No functional changes.

MFC after: 2 weeks

4 years agoImplement Linux BLKGETSIZE64 ioctl.
trasz [Sun, 29 Dec 2019 12:22:11 +0000 (12:22 +0000)]
Implement Linux BLKGETSIZE64 ioctl.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

4 years agoMake this compile under external gcc toolchain if WITH_ICONV isn't defined.
adrian [Sun, 29 Dec 2019 06:59:09 +0000 (06:59 +0000)]
Make this compile under external gcc toolchain if WITH_ICONV isn't defined.

This quietens a bunch of unused variable warnings that are treated as errors.

4 years agoCorrectly implement PMAP_ENTER_NOREPLACE in pmap_enter_{l2,pde}() on kernel
alc [Sun, 29 Dec 2019 05:36:01 +0000 (05:36 +0000)]
Correctly implement PMAP_ENTER_NOREPLACE in pmap_enter_{l2,pde}() on kernel
mappings.

Reduce code duplication by defining a function, pmap_abort_ptp(), for
handling a common error case.

Simplify error handling in pmap_enter_quick_locked().

Reviewed by: kib
Tested by: pho
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22890

4 years agofuspi: Fix 'sleepable after non-sleepable' lock
philip [Sun, 29 Dec 2019 04:26:10 +0000 (04:26 +0000)]
fuspi: Fix 'sleepable after non-sleepable' lock

With WITNESS enabled we see the following warning:

    lock order reversal: (sleepable after non-sleepable)
     1st 0xffffffd0847c7210 fu540spi0 (fu540spi0) @
     /usr/home/kp/axiado/hornet-freebsd/src/sys/riscv/sifive/fu540_spi.c:297
      2nd 0xffffffc00372bb30 Clock topology lock (Clock topology lock) @
      /usr/home/kp/axiado/hornet-freebsd/src/sys/dev/extres/clk/clk.c:1137
      stack backtrace:
      #0 0xffffffc0002a579e at witness_checkorder+0xb72
      #1 0xffffffc0002a5556 at witness_checkorder+0x92a
      #2 0xffffffc000254c7a at _sx_slock_int+0x66
      #3 0xffffffc00025537a at _sx_slock+0x8
      #4 0xffffffc000123022 at clk_get_freq+0x38
      #5 0xffffffc0005463e4 at __clzdi2+0x2bb8
      #6 0xffffffc00014af58 at randomdev_getkey+0x76e
      #7 0xffffffc0001278b0 at simplebus_add_device+0x7ee
      #8 0xffffffc00027c9a8 at device_attach+0x2e6
      #9 0xffffffc00027c634 at device_probe_and_attach+0x7a
      #10 0xffffffc00027d76a at bus_generic_attach+0x10
      #11 0xffffffc00014aab0 at randomdev_getkey+0x2c6
      #12 0xffffffc00027c9a8 at device_attach+0x2e6
      #13 0xffffffc00027c634 at device_probe_and_attach+0x7a
      #14 0xffffffc00027d76a at bus_generic_attach+0x10
      #15 0xffffffc000278bd2 at config_intrhook_oneshot+0x52
      #16 0xffffffc000278b3e at config_intrhook_establish+0x146
      #17 0xffffffc000278cf2 at config_intrhook_disestablish+0xfe

The clock topology lock can sleep, which means we cannot attempt to
acquire it while holding the non-sleepable mutex.

Fix that by retrieving the clock speed once, during attach and not every
time during SPI transaction setup.

Submitted by:   kp
Sponsored by:   Axiado

4 years agoRegenerate src.conf(5) after r356164
kevans [Sun, 29 Dec 2019 02:12:18 +0000 (02:12 +0000)]
Regenerate src.conf(5) after r356164

4 years agoChange reference in HTTPD descriptions to 'simple_httpd'
kevans [Sun, 29 Dec 2019 02:11:58 +0000 (02:11 +0000)]
Change reference in HTTPD descriptions to 'simple_httpd'

This should help people examining src.conf(5) draw the connection between
the HTTPD knobs and the particular implementation we're installing,
simple_httpd.

Reported by: saken658 via GitHub

4 years agoUpdate vendor/libarchive/dist to git 1dae5a549fe4ab99fd3a49a9edcf897a7b2b1844
mm [Sat, 28 Dec 2019 23:40:32 +0000 (23:40 +0000)]
Update vendor/libarchive/dist to git 1dae5a549fe4ab99fd3a49a9edcf897a7b2b1844

Relevant vendor changes:
  Issue #351: Refactor and implement private state logic for write filters
  PR #1252: RAR5 reader - verify window size for solid files (OSS-Fuzz 15482)
  PR #1255: zip writer - don't append unused NUL for directories
  PR #1260: Fix sparse file offset overflow on 32-bit systems
  PR #1263: UNICODE filename support for reading lha/lzh format
  Issue #1276: Bugfix and optimize archive_wstring_append_from_mbs()
  PR #1288: Add the "xattrhdr" option to pax write options
  PR #1295: 7z reader - fix reading archives with digests in PackInfo
  PR #1296: RAR5 reader - verify window size for multivolume archives
  PR #1297: ZIP reader - support LZMA_STREAM_END marker in 'lzma alone' files
  Issue #1298: Fix a heap-buffer-overflow in archive_string_append_from_wcs()
  OSS-Fuzz 19360, 19362: LHA reader - plug two memory leaks on error
  Fix possible off-by-one when dealing with readlink(2)

4 years agoFix GEOM_SHSEC orphanization.
mav [Sat, 28 Dec 2019 23:21:53 +0000 (23:21 +0000)]
Fix GEOM_SHSEC orphanization.

Previous code closed and destroyed consumer even with I/O in progress.
This patch postpones the destruction till the last close, identical to
GEOM_STRIPE, since they seem to have common origin.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

4 years agoRemove some unused functions.
markj [Sat, 28 Dec 2019 19:04:29 +0000 (19:04 +0000)]
Remove some unused functions.

The previous series of patches orphaned some vm_page functions, so
remove them.

Reviewed by: dougm, kib
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22886

4 years agoUpdate the vm_page.h block comment to reflect recent changes.
markj [Sat, 28 Dec 2019 19:04:15 +0000 (19:04 +0000)]
Update the vm_page.h block comment to reflect recent changes.

Explain the new locking rules for per-page queue state updates.

Reviewed by: jeff, kib
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22884

4 years agoRemove page locking for queue operations.
markj [Sat, 28 Dec 2019 19:04:00 +0000 (19:04 +0000)]
Remove page locking for queue operations.

With the previous reviews, the page lock is no longer required in order
to perform queue operations on a page.  It is also no longer needed in
the page queue scans.  This change effectively eliminates remaining uses
of the page lock and also the false sharing caused by multiple pages
sharing a page lock.

Reviewed by: jeff
Tested by: pho
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22885

4 years agoGeneralize lazy dequeue logic for wired pages.
markj [Sat, 28 Dec 2019 19:03:46 +0000 (19:03 +0000)]
Generalize lazy dequeue logic for wired pages.

Some recent work aims to remove the use of the page lock for
synchronizing updates to page queue state.  This change adds a mechanism
to preserve the existing behaviour of lazily dequeuing wired pages,
which was previously synchronized using the page lock.

Handle this by setting PGA_DEQUEUE when a managed page's wire count
transitions from 0 to 1.  When the page daemon encounters a page with a
flag in PGA_QUEUE_OP_MASK set, it creates a batch queue entry for that
page, but in so doing it does not modify the page itself and thus racing
with a concurrent free of the page is harmless.  The flag is advisory;
the page daemon still checks for wirings after acquiring the object and
page xbusy locks.

vm_page_unwire_managed() now clears PGA_DEQUEUE on a 1->0 transition.
It must do this before dropping the reference to avoid a use-after-free
but also handles races with concurrent wirings to ensure that
PGA_DEQUEUE is not left unset on a wired page.

Reviewed by: jeff
Tested by: pho
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22882

4 years agoStart implementing queue state updates using fcmpset loops.
markj [Sat, 28 Dec 2019 19:03:32 +0000 (19:03 +0000)]
Start implementing queue state updates using fcmpset loops.

This is in preparation for eliminating the use of the vm_page lock for
protecting queue state operations.

Introduce the vm_page_pqstate_commit_*() functions.  These functions act
as helpers around vm_page_astate_fcmpset() and are specialized for
specific types of operations.  vm_page_pqstate_commit() wraps these
functions.

Convert a number of routines to use these new helpers.  Use
vm_page_release_toq() in vm_page_unwire() and vm_page_release() to
atomically release a wiring reference and release the page into a queue.
This has the side effect that vm_page_unwire() will leave the page in
the active queue if it is already present there.

Convert the page queue scans to use the new helpers.  Simplify
vm_pageout_reinsert_inactive(), which requeues pages that were found to
be busy during an inactive queue scan, to avoid duplicating the work of
vm_pqbatch_process_page().  In particular, if PGA_REQUEUE or
PGA_REQUEUE_HEAD is set, let that be handled during batch processing.

Reviewed by: jeff
Tested by: pho
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22770
Differential Revision: https://reviews.freebsd.org/D22771
Differential Revision: https://reviews.freebsd.org/D22772
Differential Revision: https://reviews.freebsd.org/D22773
Differential Revision: https://reviews.freebsd.org/D22776

4 years agoDon't update per-page activation counts in the swapout code.
markj [Sat, 28 Dec 2019 19:03:17 +0000 (19:03 +0000)]
Don't update per-page activation counts in the swapout code.

This avoids duplicating the work of the page daemon's active queue scan.
Moreover, this duplication was inconsistent:
- PGA_REFERENCED is not counted in act_count unless pmap_ts_referenced()
  returned 0, but the page daemon always counts PGA_REFERENCED towards
  the activation count.
- The swapout daemon always activates a referenced page, but the page
  daemon only does so when the containing object is mapped at least
  once.

The main purpose of swapout_deactivate_pages() is to shrink the number
of pages mapped into a given pmap.  To do this without unmapping active
pages, use the non-destructive pmap_is_referenced() instead of the
destructive pmap_ts_referenced() and deactivate pages accordingly.
This simplifies some future changes to the locking protocol for page
queue state.

Reviewed by: kib
Discussed with: jeff
Tested by: pho
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22674

4 years agoFix GEOM_GATE orphanization.
mav [Sat, 28 Dec 2019 17:52:53 +0000 (17:52 +0000)]
Fix GEOM_GATE orphanization.

Previous code closed and destroyed direct read consumer even with I/O still
in progress.  This patch adds locking and request counting to postpone the
close till the last of running requests completes.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

4 years agovm_object_shadow(): fix object reference leak.
kib [Sat, 28 Dec 2019 16:40:44 +0000 (16:40 +0000)]
vm_object_shadow(): fix object reference leak.

In r355270 by me, vm_object_shadow() was changed to handle the
reference counting for the shared case, but the extra reference that
was done in vmspace_fork() for the shared/need_copy case was not
removed.

Submitted by: jeff

4 years agoarm64: rockchip: Add driver for the io domain
manu [Sat, 28 Dec 2019 15:30:50 +0000 (15:30 +0000)]
arm64: rockchip: Add driver for the io domain

This driver configure the registers in the GRF according to the value
of the regulators for the platform.
Some IP can run with either 3.0V or 1.8V, if we don't configure them
correctly according to the external voltage used they will not work.
It's only done at boot time for now and might be needed at runtime for
IP like sdmmc.

Reviewed by: mmel
Tested On: RockPro64, Firefly-RK3399 (gonzo), AIO-3288 (mmel)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22854

4 years agoarm64: rockchip: rk808: Add remaining regulators
manu [Sat, 28 Dec 2019 15:28:39 +0000 (15:28 +0000)]
arm64: rockchip: rk808: Add remaining regulators

The RK808 driver was missing the LDO and switch regulators.
Add support for them.

Reviewed by: mmel
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22852

4 years agoMake linux mount(2) tolerate NULL 'from' argument, and fix flag
trasz [Sat, 28 Dec 2019 13:35:54 +0000 (13:35 +0000)]
Make linux mount(2) tolerate NULL 'from' argument, and fix flag
handling.

This should unbreak access04, acct01, chmod06, creat06,
and fchmod06 LTP tests.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

4 years agoAdd userland tests for route table/lltable rtsock operations.
melifaro [Sat, 28 Dec 2019 12:16:40 +0000 (12:16 +0000)]
Add userland tests for route table/lltable rtsock operations.

MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22860

4 years ago[ar71xx] generate a random mac address using eth_gen_addr()
adrian [Sat, 28 Dec 2019 06:56:21 +0000 (06:56 +0000)]
[ar71xx] generate a random mac address using eth_gen_addr()

This removes a hard-coded random mac address generator and
uses the (not so) new system routine.

Tested:

* TP-Link WDR-4300 (AR934x + AR9580)

4 years agoVendor import of Unbound 1.9.6.
cy [Sat, 28 Dec 2019 05:27:06 +0000 (05:27 +0000)]
Vendor import of Unbound 1.9.6.

4 years agoSPDX: update some tags with two licenses.
pfg [Sat, 28 Dec 2019 02:58:30 +0000 (02:58 +0000)]
SPDX: update some tags with two licenses.

4 years agoarm/ffs.S: remove stale comment.
pfg [Sat, 28 Dec 2019 02:11:41 +0000 (02:11 +0000)]
arm/ffs.S: remove stale comment.

We already use the CLZ instruction.

Discussed with: andrew

4 years agoepoch.9: Add missing functions, clean up documentation
cem [Sat, 28 Dec 2019 01:35:32 +0000 (01:35 +0000)]
epoch.9: Add missing functions, clean up documentation

Various rototilling.

4 years agoRemove some stale comments from the page allocator.
markj [Fri, 27 Dec 2019 23:19:21 +0000 (23:19 +0000)]
Remove some stale comments from the page allocator.

Since r352110 the page lock is not required to wire pages in any
context.

4 years agoFix GEOM_UZIP orphanization.
mav [Fri, 27 Dec 2019 21:44:13 +0000 (21:44 +0000)]
Fix GEOM_UZIP orphanization.

Previous code destroyed softc even with provider still open, that resulted
in panic under load.  This change postpones the free till the final close,
when we know for sure there will be no more I/O requests.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

4 years agoImplement dump_stats command for usbconfig(8).
hselasky [Fri, 27 Dec 2019 20:37:14 +0000 (20:37 +0000)]
Implement dump_stats command for usbconfig(8).

This command is useful when debugging USB device issues.

MFC after: 1 week
Sponsored by: Mellanox Technologies

4 years agoImplement new libusb v2.0 API function, libusb20_dev_get_stats().
hselasky [Fri, 27 Dec 2019 20:35:12 +0000 (20:35 +0000)]
Implement new libusb v2.0 API function, libusb20_dev_get_stats().

This function is useful when debugging USB device issues.

MFC after: 1 week
Sponsored by: Mellanox Technologies

4 years agoMake USB statistics per device instead of per bus.
hselasky [Fri, 27 Dec 2019 20:29:13 +0000 (20:29 +0000)]
Make USB statistics per device instead of per bus.

Bump the FreeBSD version due to structure change to
force recompilation of external USB modules.

MFC after: 1 week
Sponsored by: Mellanox Technologies

4 years ago[PowerPC] Ignore fortify-source warning in trap vector copy
bdragon [Fri, 27 Dec 2019 19:58:14 +0000 (19:58 +0000)]
[PowerPC] Ignore fortify-source warning in trap vector copy

Due to a bug in clang 9.0.0 source tracking, the trap vector copying will
always trigger a fortify-source warning.

The destination buffers are 0x2f00 bytes, and the bcopy region is 0x2e00
bytes, so there is not an overflow here.

(I have been running with this patch since September.)

4 years agoFix the powerpc copyout fixup from r356113
jhibbits [Fri, 27 Dec 2019 17:38:25 +0000 (17:38 +0000)]
Fix the powerpc copyout fixup from r356113

Summary:
r356113 used an older patch, which predated the
freebsd_copyout_auxargs() addition.  Fix this by using a private
powerpc_copyout_auxargs() instead, and keep it private to powerpc, not in MI
files.

Reviewed by: kib, bdragon
Differential Revision: https://reviews.freebsd.org/D22935

4 years agosrc.conf.5: regen after r356127, disable GPL_DTC by default
emaste [Fri, 27 Dec 2019 17:12:06 +0000 (17:12 +0000)]
src.conf.5: regen after r356127, disable GPL_DTC by default

4 years agoSwitch GPL_DTC to default off
emaste [Fri, 27 Dec 2019 16:59:26 +0000 (16:59 +0000)]
Switch GPL_DTC to default off

We use the BSDL devicetree compiler as long as we have a C++11 compiler.
dtc is not needed as a build tool on the platforms that are still using
GCC 4.2.1 (and it is being disabled very soon, anyhow).

Discussed with: imp, kevans

4 years agoufs: do not leave non-reclaimed vnodes with zero i_mode around.
kib [Fri, 27 Dec 2019 16:43:34 +0000 (16:43 +0000)]
ufs: do not leave non-reclaimed vnodes with zero i_mode around.

After a recent change, vput() relocks even the exclusively locked
vnode before inactivating it.  Before that, UFS could safely
instantiate a vnode for cleared inode, then the last vput() after
ffs_vgetf() noted that ip->i_mode == 0 and recycled.  Now, it is
possible for other threads to note the half-constructed vnode, e.g. to
insert it into hash, which makes other threads to use it despite mode
is zero, before inactivation and reclaim.

Handle the found cases in SU code, by explicitly doing reclaim.
Assert that other places get fully constructed inode from ffs_vgetf(),
which cannot be cleared before dependencies are resolved.

Reported and tested by: pho
Reviewed by: mckusick
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

4 years agosrc.opts.mk: switch LLVM_LIBUNWIND to opt-out
emaste [Fri, 27 Dec 2019 16:28:43 +0000 (16:28 +0000)]
src.opts.mk: switch LLVM_LIBUNWIND to opt-out

PowerPC switched to LLVM_LIBUNWIND along with the switch to Clang/LLVM
in r356111.  This leaves only 32-bit Arm and sparc64 not using LLVM's
unwinder, so switch the sense to opt-out.

I elected to list the individual arm MACHINE_ARCHs so future changes
are more clear if LLVM_LIBUNWIND is enabled for one or two but not all
32-bit Arm targets (see PR 233664).

4 years agosrc.opts.mk: switch LLVM toolchain to opt-out
emaste [Fri, 27 Dec 2019 15:47:32 +0000 (15:47 +0000)]
src.opts.mk: switch LLVM toolchain to opt-out

After PowerPC switched in r356111, the list of targets using LLVM as the
default toolchain is much longer than those not using it.  Switch the
sense of the test to exclude those not using LLVM.

Targets not using LLVM is currently mips, riscv5, and sparc64; work is
in progress to migrate the first two to LLVM.

4 years agosrc.conf.5: regenerate after recent changes
emaste [Fri, 27 Dec 2019 15:33:43 +0000 (15:33 +0000)]
src.conf.5: regenerate after recent changes

r355588 Fix WITHOUT_CLANG build
r355646 Revert r354348
r355943 add LDNS build knob dependency on OPENSSL
r356111 Use LLVM as default toolchain for all PowerPC targets

4 years agoPlug a warning in read-mostly spinlocks reported by gcc.
mjg [Fri, 27 Dec 2019 13:37:19 +0000 (13:37 +0000)]
Plug a warning in read-mostly spinlocks reported by gcc.

4 years agovfs: remove production kernel checks and mp == NULL support from vdrop
mjg [Fri, 27 Dec 2019 11:26:12 +0000 (11:26 +0000)]
vfs: remove production kernel checks and mp == NULL support from vdrop

1. The only place in the tree which calls getnewvnode with mp == NULL does it
for vp_crossmp which will never execute this codepath. Any vnode which legally
has ->v_mount == NULL is also doomed, which once more wont execute this code.
2. Remove an assertion for v_holdcnt from production kernels. It gets taken care
of by refcount macros in debug kernels.

Any code which would want to pass NULL mp can construct a fake one instead.

Reviewed by: kib (previous version)
Differential Revision: https://reviews.freebsd.org/D22722

4 years agomac: use a sleepable rmlock instead of an sx lock
mjg [Fri, 27 Dec 2019 11:23:32 +0000 (11:23 +0000)]
mac: use a sleepable rmlock instead of an sx lock

If any non-static modules are loaded (and mac_ntpd tends to be), the lock is
taken all the time al over the kernel. On platforms like arm64 this results in
an avoidable significant performance degradation. Since write-locking is almost
never needed, use a primitive optimized towards read-locking.

Sample result of building the kernel on tmpfs 11 times:
stock           11142.80s user 6704.44s system 4924% cpu 6:02.42 total
patched         11118.95s user 2374.94s system 4547% cpu 4:56.71 total

4 years agoAdd read-mostly sleepable locks
mjg [Fri, 27 Dec 2019 11:19:57 +0000 (11:19 +0000)]
Add read-mostly sleepable locks

To be used when like rmlocks, except when sleeping for readers needs to be
allowed. See the manpage for more information.

Reviewed by: kib (previous version)
Differential Revision: https://reviews.freebsd.org/D22823

4 years ago[PowerPC] Clang powerpcspe build fixes
bdragon [Fri, 27 Dec 2019 05:01:13 +0000 (05:01 +0000)]
[PowerPC] Clang powerpcspe build fixes

* Fix a couple of format errors.
* Add some extra compiler flags needed to force clang to build SPE code.
  (These are temporary until the target triple is fixed)

4 years agoFix the build from r356113.
jhibbits [Fri, 27 Dec 2019 04:52:17 +0000 (04:52 +0000)]
Fix the build from r356113.

Types had changed from when the patch was first created, and a final build
was not done pre-commit.

4 years agoAdd a note to UPDATING on the move to clang for powerpc*
jhibbits [Fri, 27 Dec 2019 04:42:03 +0000 (04:42 +0000)]
Add a note to UPDATING on the move to clang for powerpc*

4 years ago[gpioths] Fix GPIOTHS_DEBUG
brd [Fri, 27 Dec 2019 04:11:14 +0000 (04:11 +0000)]
[gpioths] Fix GPIOTHS_DEBUG

Chase the removal of dev from gpioths_dht_readbytes() in r355540.

Reviewed by: ian
Approved by: will (mentor)
Differential Revision: https://reviews.freebsd.org/D22926

4 years ago[PowerPC] Switch to PIC kernel modules on powerpc*
bdragon [Fri, 27 Dec 2019 04:07:51 +0000 (04:07 +0000)]
[PowerPC] Switch to PIC kernel modules on powerpc*

To improve reliability of kernel modules after the clang switch, switch to
-fPIC when building for now.

This bypasses some limitations to the way clang and LLD handle relocations,
and is a more robustly tested compilation regime than the
"static shared object" mode that we were previously attempting to convince
the compiler stack to use.

The kernel linker was recently augmented to be able to handle this mode.

Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D22798

4 years agoEliminate the last MI difference in AT_* definitions (for powerpc).
jhibbits [Fri, 27 Dec 2019 04:07:03 +0000 (04:07 +0000)]
Eliminate the last MI difference in AT_* definitions (for powerpc).

Summary:
As a transition aide, implement an alternative elfN_freebsd_fixup which
is called for old powerpc binaries.  Similarly, add a translation to rtld to
convert old values to new ones (as expected by a new rtld).

Translation of old<->new values  is incomplete, but sufficient to allow an
installworld of a new userspace from an old one when a new kernel is running.

Test Plan:
Someone needs to see how a new kernel/rtld/libc works with an old
binary.  If if works we can probalby ship this.  If not we probalby need
some more compat bits.

Submitted by: brooks
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D20799

4 years ago[PowerPC64] Starting from FreeBSD 13.0, default to ELFv2 ABI
jhibbits [Fri, 27 Dec 2019 04:00:04 +0000 (04:00 +0000)]
[PowerPC64] Starting from FreeBSD 13.0, default to ELFv2 ABI

This changes the LLVM default powerpc64 ABI to ELFv2, if target OS is
FreeBSD >= 13.0

This will also be sent upstream.

Submitted by: alfredo.junior_eldorado.org.br
Reviewed by: dim, luporl
Relnotes: YES
Differential Revision: https://reviews.freebsd.org/D20383