freebsd.git
7 years agoMerge llvm, clang, lld, lldb, compiler-rt and libc++ r304149, and update
dim [Mon, 29 May 2017 22:09:23 +0000 (22:09 +0000)]
Merge llvm, clang, lld, lldb, compiler-rt and libc++ r304149, and update
build glue.

7 years agoVendor import of libc++ trunk r304149:
dim [Mon, 29 May 2017 16:26:10 +0000 (16:26 +0000)]
Vendor import of libc++ trunk r304149:
https://llvm.org/svn/llvm-project/libcxx/trunk@304149

7 years agoVendor import of clang trunk r304149:
dim [Mon, 29 May 2017 16:25:46 +0000 (16:25 +0000)]
Vendor import of clang trunk r304149:
https://llvm.org/svn/llvm-project/cfe/trunk@304149

7 years agoVendor import of llvm trunk r304149:
dim [Mon, 29 May 2017 16:25:25 +0000 (16:25 +0000)]
Vendor import of llvm trunk r304149:
https://llvm.org/svn/llvm-project/llvm/trunk@304149

7 years agoMissed a few additional files in libllvm, for llvm-objdump and llvm-pdbdump.
dim [Sat, 27 May 2017 11:25:21 +0000 (11:25 +0000)]
Missed a few additional files in libllvm, for llvm-objdump and llvm-pdbdump.

7 years agoMerge ^/head r318658 through r318963.
dim [Fri, 26 May 2017 19:11:24 +0000 (19:11 +0000)]
Merge ^/head r318658 through r318963.

7 years agoAllow PROBE_SPINUP to fail in CAM ATA transport
avg [Fri, 26 May 2017 17:44:47 +0000 (17:44 +0000)]
Allow PROBE_SPINUP to fail in CAM ATA transport

The motivation for this is two-fold.

1. Some old WD SATA disks may appear as if they need to be spun up
when they are already spinning.  Those disks would respond with
an error to the spin-up request.

2. Even if we really fail to spin up the disk, we still can try to
proceed to the subsequent phases.  If we fail later on, then no
difference.  Otherwise we get a chance to communicate with the
disk which is better than completely ignoring it, because a user
can try to recover the disk.

Reviewed by: mav
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D10896

7 years agoAdd newsyslog capability to write RFC5424 compliant rotation message.
dab [Fri, 26 May 2017 16:36:30 +0000 (16:36 +0000)]
Add newsyslog capability to write RFC5424 compliant rotation message.

This modification adds the capability to newsyslog to write the
rotation message in a format that is compliant with RFC5424. This
capability is enabled on a per-log file basis through a new value
("T") in the flags field in newsyslog.conf. This is useful on systems
that use the RFC5424 format for log files so that the rotation message
format matches that of the other log messages. There has been recent
mention of adding an RFC5424 compliant mode to syslogd and at least
one alternative system log daemon (rsyslogd) that already has the
capability to use that format.

Reviewed by: vangyzen, ngie
Approved by: vangyzen (mentor)
MFC after: 2 months
Relnotes: yes
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10253

7 years agoDefine a new __INO64 macro in <sys/_types.h>, to indicate the system
dim [Fri, 26 May 2017 16:29:55 +0000 (16:29 +0000)]
Define a new __INO64 macro in <sys/_types.h>, to indicate the system
uses 64-bit inode numbers.  Programs can use this to avoid including
<sys/param.h>, with its associated namespace pollution.

Reviewed by: kib

7 years agoUse the SCTP_PCB_FLAGS_ACCEPTING flags to check for listeners.
tuexen [Fri, 26 May 2017 16:29:00 +0000 (16:29 +0000)]
Use the SCTP_PCB_FLAGS_ACCEPTING flags to check for listeners.

While there, use a macro for checking the listen state to allow for
easier changes if required.

This done to help glebius@ with his listen changes.

7 years agorm stale ptrace dependencies after r305012
emaste [Fri, 26 May 2017 16:03:28 +0000 (16:03 +0000)]
rm stale ptrace dependencies after r305012

This is similar to r318912, except that ptrace.[sS] was previously a
file in the source tree, not a generated assembly wrapper.

Check for the existence of ptrace.[sS] in the .depend file to determine
if we have to clean it up.  This is a bit hackish and will not be left
in place indefinitely, but provides a useful example case when
investigating a better solution in bmake.

Reviewed by: bdrewery
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10930

7 years agolibthr: increase WARNS to the default (6)
vangyzen [Fri, 26 May 2017 15:57:54 +0000 (15:57 +0000)]
libthr: increase WARNS to the default (6)

...and silence cast-align warnings from gcc.

Reviewed by: kib
MFC after: 3 days
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10935

7 years agolibthr: fix warnings at WARNS=6
vangyzen [Fri, 26 May 2017 15:56:28 +0000 (15:56 +0000)]
libthr: fix warnings at WARNS=6

Fix more warnings about redundant declarations.

Reviewed by: kib emaste
MFC after: 3 days
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10932

7 years agortld: fix warnings about redundant declarations
vangyzen [Fri, 26 May 2017 15:55:03 +0000 (15:55 +0000)]
rtld: fix warnings about redundant declarations

Fix warnings about redundant declarations in rtld
when libthr in increased to WARNS=6.

Reviewed by: kib
MFC after: 3 days
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10934

7 years agolibthr: fix style in previous commit
vangyzen [Fri, 26 May 2017 15:53:27 +0000 (15:53 +0000)]
libthr: fix style in previous commit

I intended to add this to the previous commit.

Reviewed by: kib
MFC after: 3 days
Sponsored by: Dell EMC

7 years agolibthr: prevent setcontext() from masking SIGTHR
vangyzen [Fri, 26 May 2017 15:51:51 +0000 (15:51 +0000)]
libthr: prevent setcontext() from masking SIGTHR

__thr_setcontext() mistakenly tested for the presence of SIGCANCEL
in its local ucontext_t instead of the parameter. Therefore,
if a thread calls setcontext() with a context whose signal mask
contains SIGTHR (a.k.a. SIGCANCEL), that signal will be blocked,
preventing the thread from being cancelled or suspended.

Reported by: gcc 6.1 via RISC-V tinderbox
Reviewed by: kib
MFC after: 3 days
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10933

7 years agomakefs: add -O (offset) option
emaste [Fri, 26 May 2017 15:49:20 +0000 (15:49 +0000)]
makefs: add -O (offset) option

NetBSD revs:
ffs.c 1.60
makefs.8 1.44
makefs.c 1.48
makefs.h 1.33
ffs/buf.c 1.20
ffs/mkfs.c 1.27

Obtained from: NetBSD
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10780

7 years agoMFV r318944: 8265 Reserve send stream flag for large dnode feature
avg [Fri, 26 May 2017 12:08:38 +0000 (12:08 +0000)]
MFV r318944: 8265 Reserve send stream flag for large dnode feature

illumos/illumos-gate@bc83969fdbd1cb0d97ba00218c0a3de5c89fba92
https://github.com/illumos/illumos-gate/commit/bc83969fdbd1cb0d97ba00218c0a3de5c89fba92

https://www.illumos.org/issues/8265
  Reserve bit 23 in the zfs send stream flags for the large
  dnode feature which has been implemented for Linux.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Brian Behlendorf <behlendorf1@llnl.gov>

MFC after: 1 week

7 years agoMFV r318942: 8166 zpool scrub thinks it repaired offline device
avg [Fri, 26 May 2017 12:04:21 +0000 (12:04 +0000)]
MFV r318942: 8166 zpool scrub thinks it repaired offline device

illumos/illumos-gate@2d2f193a21231a58c583466dc23ba71f1a25f424
https://github.com/illumos/illumos-gate/commit/2d2f193a21231a58c583466dc23ba71f1a25f424

https://www.illumos.org/issues/8166
  If we do a scrub while a leaf device is offline (via "zpool offline"),
  we will inadvertently clear the DTL (dirty time log) of the offline
  device, even though it is still damaged. When the device comes back
  online, we will incompletely resilver it, thinking that the scrub
  repaired blocks written before the scrub was started. The incomplete
  resilver can lead to data loss if there is a subsequent failure of a
  different leaf device.
  The fix is to never clear the DTL of offline devices. Note that if a
  device is onlined while a scrub is in progress, the scrub will be
  restarted.
  The problem can be worked around by running "zpool scrub" after
  "zpool online".
  See also https://github.com/zfsonlinux/zfs/issues/5806

Reviewed by: George Wilson george.wilson@delphix.com
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>

7 years agoMFV r318934: 8070 Add some ZFS comments
avg [Fri, 26 May 2017 11:49:42 +0000 (11:49 +0000)]
MFV r318934: 8070 Add some ZFS comments

illumos/illumos-gate@40713f2b249d289022c715107b3951055a63aef0
https://github.com/illumos/illumos-gate/commit/40713f2b249d289022c715107b3951055a63aef0

https://www.illumos.org/issues/8070
  Add some ZFS comments left by various developers at different times

Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Alan Somers <asomers@gmail.com>

MFC after: 1 week

7 years agoMFV r318931: 8063 verify that we do not attempt to access inactive txg
avg [Fri, 26 May 2017 11:37:11 +0000 (11:37 +0000)]
MFV r318931: 8063 verify that we do not attempt to access inactive txg

illumos/illumos-gate@b7b2590dd9f11b12a0b4878db3886068cce176af
https://github.com/illumos/illumos-gate/commit/b7b2590dd9f11b12a0b4878db3886068cce176af

https://www.illumos.org/issues/8063
  A standard practice in ZFS is to keep track of "per-txg" state. Any of
  the 3 active TXG's (open, quiescing, syncing) can have different values
  for this state. We should assert that we do not attempt to modify other
  (inactive) TXG's.

Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

MFC after: 2 weeks

7 years agoMFV r318929: 7786 zfs`vdev_online() needs better notification about state changes
avg [Fri, 26 May 2017 11:33:34 +0000 (11:33 +0000)]
MFV r318929: 7786 zfs`vdev_online() needs better notification about state changes

illumos/illumos-gate@5f368aef86387d6ef4eda84030ae9b402313ee4c
https://github.com/illumos/illumos-gate/commit/5f368aef86387d6ef4eda84030ae9b402313ee4c

https://www.illumos.org/issues/7786
  Currently, vdev_online() will only post sysevent if previous state was
  "offline". It should also post the event when the state changes from "removed"
  or "faulted" to "healthy" or "degraded".
  This will fix the following scenario:
  - pull disk from slot A
  - check that hotspare has taken its place (if available)
  - insert disk into slot B
  - check that hotspare moved back to "avail" state (if spare was used)
  The problem here is that we don't get any ESC_ZFS_VDEV_* notification and fail
  to update the vdev FRU.

Reviewed by: Matthew Ahrens mahrens@delphix.com
Reviewed by: George Wilson george.wilson@delphix.com
Approved by: Albert Lee <trisk@forkgnu.org>
Author: Yuri Pankov <yuri.pankov@nexenta.com>

MFC after: 1 week

7 years agoMFV r318927: 8025 dbuf_read() creates unnecessary zio_root() for bonus buf
avg [Fri, 26 May 2017 11:30:55 +0000 (11:30 +0000)]
MFV r318927: 8025 dbuf_read() creates unnecessary zio_root() for bonus buf

illumos/illumos-gate@def4fac5882b4ca67bd0f4a53509b6d1fa8ae14e
https://github.com/illumos/illumos-gate/commit/def4fac5882b4ca67bd0f4a53509b6d1fa8ae14e

https://www.illumos.org/issues/8025
  dbuf_read() creates a zio_root() to track and wait for all the zio's
  that may happen as part of this call. However, if the blkptr_t for
  this buffer is NULL or a hole, we will not create any more zio's, so
  this zio_root() is unnecessary. This is always the case when calling
  dbuf_read() on a bonus buffer, because it has no blkptr (it's part of
  the containing dnode). For workloads that read a lot of bonus buffers
  (e.g. file creation and removal), creating and destroying these
  unnecessary zio's can decrease performance by around 3%.

Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

7 years agoMFV r316929: 6914 kernel virtual memory fragmentation leads to hang
avg [Fri, 26 May 2017 11:23:16 +0000 (11:23 +0000)]
MFV r316929: 6914 kernel virtual memory fragmentation leads to hang

illumos/illumos-gate@af868f46a5b794687741d5424de9e3a2d684a84a
https://github.com/illumos/illumos-gate/commit/af868f46a5b794687741d5424de9e3a2d684a84a

https://www.illumos.org/issues/6914

FreeBSD note: only a ZFS part of the change is merged, changes to the VM
subsystem are not ported (obviously).  Also, now that FreeBSD has
vmem(9) we don't have to ifdef-out the code that uses it.

MFC after: 2 weeks

7 years agoarc_init: make code closer to upstream by introducing 'allmem' variable
avg [Fri, 26 May 2017 11:05:56 +0000 (11:05 +0000)]
arc_init: make code closer to upstream by introducing 'allmem' variable

All the differences in calculations are kept.
A comment about arc_max being 1/2 of all memory is fixed to reflect the
actual code that uses 5/8 as a factor.

MFC after: 1 week

7 years agozfs_putpages: assert that sa_bulk_update() must succeed
avg [Fri, 26 May 2017 10:37:55 +0000 (10:37 +0000)]
zfs_putpages: assert that sa_bulk_update() must succeed

Same as the upstream does in r316927.

MFC after: 1 week

7 years agoMFV r316928: 7256 low probability race in zfs_get_data
avg [Fri, 26 May 2017 10:31:05 +0000 (10:31 +0000)]
MFV r316928: 7256 low probability race in zfs_get_data

illumos/illumos-gate@0c94e1af6784c69a1dea25e0e35dd13b2b91e2e5
https://github.com/illumos/illumos-gate/commit/0c94e1af6784c69a1dea25e0e35dd13b2b91e2e5

https://www.illumos.org/issues/7256
                         error = dmu_sync(zio, lr->lr_common.lrc_txg,
                              zfs_get_done, zgd);
                         ASSERT(error || lr->lr_length <= zp->z_blksz);
  It's possible, although extremely rare, that the zfs_get_done() callback is
  executed before dmu_sync() returns.
  In that case the znode's range lock is dropped and the znode is unreferenced.
  Thus, the assertion can access some invalid or wrong data via the zp pointer.
  size variable caches the correct value of z_blksz and can be safely used here.

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <andriy.gapon@clusterhq.com>

MFC after: 1 week

7 years agoMFC r316924: 8061 sa_find_idx_tab can be declared more type-safely
avg [Fri, 26 May 2017 10:27:35 +0000 (10:27 +0000)]
MFC r316924: 8061 sa_find_idx_tab can be declared more type-safely

illumos/illumos-gate@7f0bdb4257bb4f1f76390b72665961e411da24c6
https://github.com/illumos/illumos-gate/commit/7f0bdb4257bb4f1f76390b72665961e411da24c6

https://www.illumos.org/issues/8061
  sa_find_idx_tab() is declared as taking and returning "void *" parameters.
  These can be declared to be the specific types.

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Chris Williamson <chris.williamson@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

MFC after: 1 week

7 years agoDisconnect heimdal version of qsort.c from build because we are already
delphij [Fri, 26 May 2017 06:09:11 +0000 (06:09 +0000)]
Disconnect heimdal version of qsort.c from build because we are already
using libc's version of qsort.

PR: bin/213922
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D10814

7 years agobsdgrep: use safer sizeof() construct
emaste [Fri, 26 May 2017 03:35:59 +0000 (03:35 +0000)]
bsdgrep: use safer sizeof() construct

Submitted by: Kyle Evans <kevans91@ksu.edu>

7 years agoSimplify parseval() by allocating a buffer the size of the input string,
araujo [Fri, 26 May 2017 03:27:06 +0000 (03:27 +0000)]
Simplify parseval() by allocating a buffer the size of the input string,
which will always be big enough to hold the output string.

Obtained from: OpenBSD (revision 1.36)

7 years agobsdgrep: correct assumptions to prepare for chunking
emaste [Fri, 26 May 2017 02:30:26 +0000 (02:30 +0000)]
bsdgrep: correct assumptions to prepare for chunking

Correct a couple of minor BSD grep assumptions that are valid for line
processing but not future chunk-based processing.

Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: bapt, cem
Differential Revision: https://reviews.freebsd.org/D10824

7 years agofts_open: move bogus initialization further below, before it is used.
pfg [Fri, 26 May 2017 01:14:58 +0000 (01:14 +0000)]
fts_open: move bogus initialization further below, before it is used.

Move an unneeded initialization, introduced in r54770 to quiet down GCC,
to a place nearer to its first use. This has no practical effect, it just
keeps the garbage better sorted.

Hinted by: OpenBSD (CVS rev. 1.56, without obfuscations)

7 years agolibc: rm stale generated files which are no longer syscalls
emaste [Fri, 26 May 2017 00:51:05 +0000 (00:51 +0000)]
libc: rm stale generated files which are no longer syscalls

This is an attempt to help -DNO_CLEAN builds after r302092 (which
removed the pipe libc syscall wrapper) and r318736 (which removed
getdents, lstat, mknod, and stat).

Dependencies cannot cope with certain source tree changes,
particularly with respect to removing source files and replacing
generated files.  Handle these cases from _worldtmp in an ad-hoc
fashion.

Reviewed by: bdrewery, cem
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10876

7 years ago[ath] fix short-GI wireshark flag.
adrian [Fri, 26 May 2017 00:48:21 +0000 (00:48 +0000)]
[ath] fix short-GI wireshark flag.

Yes, HAL_RX_GI means "short guard interval."

7 years agobsdgrep: add --mmap tests
emaste [Fri, 26 May 2017 00:19:50 +0000 (00:19 +0000)]
bsdgrep: add --mmap tests

Basic sanity tests as well as coverage for the bug fixed in r318565.

Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: bapt, ngie
Differential Revision: https://reviews.freebsd.org/D10827

7 years agoRemove some code, dead from the day one.
mav [Thu, 25 May 2017 23:19:09 +0000 (23:19 +0000)]
Remove some code, dead from the day one.

7 years agoPull in r303257 from upstream llvm trunk (by Krzysztof Parzyszek)
dim [Thu, 25 May 2017 23:14:51 +0000 (23:14 +0000)]
Pull in r303257 from upstream llvm trunk (by Krzysztof Parzyszek)

  [PPC] Properly update register save area offsets

  The variables MinGPR/MinG8R were not updated properly when resetting the
  offsets, which in the included testcase lead to saving the CR register
  in the same location as R30.

  This fixes another issue reported in PR26519.

  Differential Revision: https://reviews.llvm.org/D33017

Reported by: Mark Millard
PR: 206990
MFC after: 3 days

7 years agoTry to keep up with the ports system Makefiles.
phk [Thu, 25 May 2017 21:59:19 +0000 (21:59 +0000)]
Try to keep up with the ports system Makefiles.

7 years agomakefs: make buf generic
emaste [Thu, 25 May 2017 21:41:06 +0000 (21:41 +0000)]
makefs: make buf generic

it has nothing to do with ffs and will eventually be moved.
gc sectorsize.

This is a corrected version of r317744.

NetBSD versions:
ffs.c 1.58
ffs/buf.c 1.14 1.18
ffs/buf.h 1.8

Submitted by: Siva Mahadevan <smahadevan@freebsdfoundation.org>
Obtained from: NetBSD
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10803

7 years agoRemove the old man.template
bapt [Thu, 25 May 2017 21:16:39 +0000 (21:16 +0000)]
Remove the old man.template

In FreeBSD we only use mdoc(7) format. A template is available as mdoc.template
The usage of man(7) format is discouraged and this file was driving people into
the front direction as a template to use.

7 years agoupdate to 2017.05.25
bapt [Thu, 25 May 2017 21:11:25 +0000 (21:11 +0000)]
update to 2017.05.25

MFC after: 2 days

7 years agoMerge OpenSSL 1.0.2l.
jkim [Thu, 25 May 2017 20:52:16 +0000 (20:52 +0000)]
Merge OpenSSL 1.0.2l.

7 years ago Import OpenSSL 1.0.2l.
jkim [Thu, 25 May 2017 19:38:38 +0000 (19:38 +0000)]
 Import OpenSSL 1.0.2l.

7 years agoFix several problems with mapping code.
slm [Thu, 25 May 2017 19:20:06 +0000 (19:20 +0000)]
Fix several problems with mapping code.

Reviewed by:    ken, scottl, asomers, ambrisko, mav
Approved by: ken, mav
MFC after:      1 week
Differential Revision: https://reviews.freebsd.org/D10861

7 years agoFix several problems with mapping code.
slm [Thu, 25 May 2017 19:14:44 +0000 (19:14 +0000)]
Fix several problems with mapping code.

Reviewed by:    ken, scottl, asomers, ambrisko, mav
Approved by: ken, mav
MFC after:      1 week
Differential Revision: https://reviews.freebsd.org/D10878

7 years agoTurn on support for the Amazon "Elastic Network Adapter" in EC2 AMIs.
cperciva [Thu, 25 May 2017 19:02:54 +0000 (19:02 +0000)]
Turn on support for the Amazon "Elastic Network Adapter" in EC2 AMIs.

X-MFC-after: 318647 + fixes for some lock ordering warnings

7 years agoUpdate the diff3 manpage to reflect the fact the version in freebsd does
bapt [Thu, 25 May 2017 18:46:13 +0000 (18:46 +0000)]
Update the diff3 manpage to reflect the fact the version in freebsd does
not use temporary files nor uses a /usr/libexec/diff3prog

7 years agoFix long standing issue in bsdconfig's keymap selection
dteske [Thu, 25 May 2017 18:16:17 +0000 (18:16 +0000)]
Fix long standing issue in bsdconfig's keymap selection

Since the translation to vt as terminal emulator, the keymaps files
path has changed and this change does not get followed in bsdconfig.
This implicates boot time warnings about a wrong keymap file, what
is very confusing for the new users and for me too, so initialize
the default keymaps search path depending on terminal type.

Differential Revision: https://reviews.freebsd.org/D8734
Submitted by: Oliver Pinter <oliver.pinter@hardenedbsd.org>
Reviewed by: ed, jilles, dteske
MFC after: 3 days
X-MFC-to: stable/11
Sponsored by: HardenedBSD
Signed-off-by: Oliver Pinter <oliver.pinter@hardenedbsd.org>

7 years agoFor now comment tests for arguments which are not in par with GNU diff3 yet
bapt [Thu, 25 May 2017 17:58:01 +0000 (17:58 +0000)]
For now comment tests for arguments which are not in par with GNU diff3 yet

7 years agoRemove the MAX_CHECK macro, it was initially used to test if a file was a
bapt [Thu, 25 May 2017 17:55:40 +0000 (17:55 +0000)]
Remove the MAX_CHECK macro, it was initially used to test if a file was a
text file or not.

The check is not done by diff3 but by diff (the argument -a is directly passed
to diff(1))

7 years agoImport working progress BSD diff3
bapt [Thu, 25 May 2017 17:45:50 +0000 (17:45 +0000)]
Import working progress BSD diff3

import bsd diff3 from OpenBSD.
Differences with OpenBSD:
- lots of warning fixed
- no shell wrapper with diff3 actually living in libexec
- capsicumized

Keep it disconnected as it is not yet good enough to replace GNU diff

The motivation to import it now it to allow other people to jump in and also to
have an open development on it

Obtained from: OpenBSD

7 years agolldb: map TRAP_CAP to a trace trap
emaste [Thu, 25 May 2017 16:41:07 +0000 (16:41 +0000)]
lldb: map TRAP_CAP to a trace trap

In the absense of a more specific handler for TRAP_CAP (generated by
ENOTCAPABLE or ECAPMODE while in capability mode) treat it as a trace
trap.

Example usage (testing the bug in PR219173):

% proccontrol -m trapcap lldb usr.bin/hexdump/obj/hexdump -- -Cv -s 1 /bin/ls
...
(lldb) run
Process 12980 launching
Process 12980 launched: '.../usr.bin/hexdump/obj/hexdump' (x86_64)
Process 12980 stopped
* thread #1, stop reason = trace
    frame #0: 0x0000004b80c65f1a libc.so.7`__sys_lseek + 10
...

In the future we should have LLDB control the trapcap procctl itself
(as it does with ASLR), as well as report a specific stop reason.
This change eliminates an assertion failure from LLDB for now.

7 years agoNow that roff documentation has been disconnected from the build, it is no
bapt [Thu, 25 May 2017 16:31:53 +0000 (16:31 +0000)]
Now that roff documentation has been disconnected from the build, it is no
longer necessary to have groff(1) as a bootstrap tool

7 years agoIn preparation for the removal of the roff toolchain, disconnect the
bapt [Thu, 25 May 2017 14:54:22 +0000 (14:54 +0000)]
In preparation for the removal of the roff toolchain, disconnect the
roff documentation from the build.

Those documents will be added to the doc tree and distributed as PDF from
the documentation website. As they are valuable has history, but do not match
current FreeBSD

Further more, the ascii format we were using to distribute them is not really
accurate for such documents.

more details:
https://lists.freebsd.org/pipermail/freebsd-arch/2017-May/018211.html

7 years agoUnmask legacy interrupts on Marvell PCIE controller
zbb [Thu, 25 May 2017 14:34:21 +0000 (14:34 +0000)]
Unmask legacy interrupts on Marvell PCIE controller

This patch fixes a bug introduced with commit:
r294510  "Remove an extra '!' found by clang 3.8."

'!' was removed without inverting the logic, which
broke PCIe legacy interrupts operation for Marvell
controllers.

Submitted by: Michal Mazur <mkm@semihalf.com>
Obtained from: Semihalf
Sponsored by: Netgate

7 years agoImprove the decoding of the third argument of the socket() call.
tuexen [Thu, 25 May 2017 14:27:54 +0000 (14:27 +0000)]
Improve the decoding of the third argument of the socket() call.

Decoding of the third argument depends on the first one. For doing this,
add a corresponding function to libsysdecode.

Thanks to jhb@ for suggesting this.

7 years agoAdd workaround for CESA MBUS windows with 4GB DRAM
zbb [Thu, 25 May 2017 14:25:05 +0000 (14:25 +0000)]
Add workaround for CESA MBUS windows with 4GB DRAM

Armada 38x SoC's equipped with 4GB DRAM suffer freeze
during CESA operation, if MBUS window opened at given
DRAM CS reaches end of the address space. Apply a workaround
by setting the window size to the closest possible
value, i.e. divide it by 2 (it has to be power-of-2).

Submitted by: Marcin Wojtas <mw@semihalf.com>
Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D10724

7 years agoFix PM recognition on recent Marvell boards
zbb [Thu, 25 May 2017 14:23:49 +0000 (14:23 +0000)]
Fix PM recognition on recent Marvell boards

PM status is only supported on Kirkwood and Disvovery.
Cleanup the code to properly report its state on
other platforms.

Submitted by: Wojciech Macek <wma@semihalf.com>
Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D10718

7 years agoIntroduce separate watchdog driver for Armada to fix phony DELAY
zbb [Thu, 25 May 2017 14:22:00 +0000 (14:22 +0000)]
Introduce separate watchdog driver for Armada to fix phony DELAY

DELAY is a problematic routine called all over the kernel.
Armada38x using CA-9 CPUs are using mpcore timer to count events
and measure time but DELAY in the mpcore timer code is a weak
function reference and therefore will be replaced by the platform
implementation if the one is introduced. Since Armada38x uses
on-chip watchdog to which the driver is merged with the on-chip timer
driver there will be a platform DELAY implementation.
The latter however will not use any HW timers as it will not attempt
to configure any. Phony busy loop will be used instead.

To fix that we introduce a separate watchdog driver for Armada platforms,
(currently only A38X) and stop using Marvell timer driver. That
switches DELAY to the desired implementation.

Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D10710

7 years agoEnable SCU Speculative linefills to L2 on Armada 38x
zbb [Thu, 25 May 2017 14:19:20 +0000 (14:19 +0000)]
Enable SCU Speculative linefills to L2 on Armada 38x

Submitted by: Marcin Wojtas <mw@semihalf.com>
Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D10709

7 years agoFix memory corruption while configuring CPU windows on Marvell SoCs
zbb [Thu, 25 May 2017 14:16:43 +0000 (14:16 +0000)]
Fix memory corruption while configuring CPU windows on Marvell SoCs

Resolving CPU windows from localbus entry caused buffer overflow
and memory corruption. Fix wrong indexing and ensure the index
does not exceed table size.

Submitted by: Wojciech Macek <wma@semihalf.com>
Obtained from: Semihalf
Sponsored by: Stormshield
Differential revision: https://reviews.freebsd.org/D10720

7 years agoFix rendering with modern groff
bapt [Thu, 25 May 2017 12:57:15 +0000 (12:57 +0000)]
Fix rendering with modern groff

Reported by: tj

7 years agoEnable DHCP and IPv6 autoconfig on non-cloud VM images.
gjb [Thu, 25 May 2017 12:53:49 +0000 (12:53 +0000)]
Enable DHCP and IPv6 autoconfig on non-cloud VM images.

PR: 203653
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

7 years agofix vmxnet3 crash when LRO is enabled
avg [Thu, 25 May 2017 10:49:56 +0000 (10:49 +0000)]
fix vmxnet3 crash when LRO is enabled

The crash can occur when all of the following conditions are true:
- a packet consists of multiple segements (requires LRO enabled)
- there has been a failure to allocate an mbuf for the packet and
  the packet has to be dropped
- a host (vmware) still owned at least one segment of the packet,
  so the driver had to wait for another interrupt to proceed to
  discarding the remaning segment(s)

Reviewed by: rstone
MFC after: 2 weeks
Sponsored by: Panzura
Differential Revision: https://reviews.freebsd.org/D10874

7 years agoReplace stale handbook URL with the proper on.
jlh [Thu, 25 May 2017 09:21:21 +0000 (09:21 +0000)]
Replace stale handbook URL with the proper on.

MFC after: 0 days

7 years agoCreate /net by default, for autofs.
trasz [Thu, 25 May 2017 08:34:24 +0000 (08:34 +0000)]
Create /net by default, for autofs.

MFC after: 2 weeks

7 years agoDeclare the "snd_fxdiv_table" once. This shaves around 24Kbytes of
hselasky [Thu, 25 May 2017 05:23:47 +0000 (05:23 +0000)]
Declare the "snd_fxdiv_table" once. This shaves around 24Kbytes of
binary data from sound.ko and the kernel.

MFC after: 3 days

7 years ago[lib] disable libc++experimental on mips platforms for now.
adrian [Thu, 25 May 2017 05:02:43 +0000 (05:02 +0000)]
[lib] disable libc++experimental on mips platforms for now.

This breaks at least mips32 platform builds.

Reviewed by: dim

7 years agoBump UPDATING to cover the ath shuffle.
adrian [Thu, 25 May 2017 05:01:44 +0000 (05:01 +0000)]
Bump UPDATING to cover the ath shuffle.

7 years ago[ath] [ath_hal] retire AH_SUPPORT_AR5416 changing anything.
adrian [Thu, 25 May 2017 04:26:26 +0000 (04:26 +0000)]
[ath] [ath_hal] retire AH_SUPPORT_AR5416 changing anything.

Yes, the memory bloat is large, but it's 2017 and I'll fix it later
by making it runtime configurable / per-chip configurable if I ever need to.

7 years ago[ath] [ath_hal] (etc, etc) - begin the task of re-modularising the HAL.
adrian [Thu, 25 May 2017 04:18:46 +0000 (04:18 +0000)]
[ath] [ath_hal] (etc, etc) - begin the task of re-modularising the HAL.

In the deep past, when this code compiled as a binary module, ath_hal
built as a module.  This allowed custom, smaller HAL modules to be built.
This was especially beneficial for small embedded platforms where you
didn't require /everything/ just to run.

However, sometime around the HAL opening fanfare, the HAL landed here
as one big driver+HAL thing, and a lot of the (dirty) infrastructure
(ie, #ifdef AH_SUPPORT_XXX) to build specific subsets of the HAL went away.
This was retained in sys/conf/files as "ath_hal_XXX" but it wasn't
really floated up to the modules themselves.

I'm now in a position where for the reaaaaaly embedded boards (both the
really old and the last couple generation of QCA MIPS boards) having a
cut down HAL module and driver loaded at runtime is /actually/ beneficial.

This reduces the kernel size down by quite a bit.  The MIPS modules look
like this:

adrian@gertrude:~/work/freebsd/head-embedded/src % ls -l ../root/mips_ap/boot/kernel.CARAMBOLA2/ath*ko
-r-xr-xr-x  1 adrian  adrian    5076 May 23 23:45 ../root/mips_ap/boot/kernel.CARAMBOLA2/ath_dfs.ko
-r-xr-xr-x  1 adrian  adrian  100588 May 23 23:45 ../root/mips_ap/boot/kernel.CARAMBOLA2/ath_hal.ko
-r-xr-xr-x  1 adrian  adrian  627324 May 23 23:45 ../root/mips_ap/boot/kernel.CARAMBOLA2/ath_hal_ar9300.ko
-r-xr-xr-x  1 adrian  adrian  314588 May 23 23:45 ../root/mips_ap/boot/kernel.CARAMBOLA2/ath_main.ko
-r-xr-xr-x  1 adrian  adrian   23472 May 23 23:45 ../root/mips_ap/boot/kernel.CARAMBOLA2/ath_rate.ko

And the x86 versions, like this:

root@gertrude:/home/adrian # ls -l /boot/kernel/ath*ko
-r-xr-xr-x  1 root  wheel   36632 May 24 18:32 /boot/kernel/ath_dfs.ko
-r-xr-xr-x  1 root  wheel  134440 May 24 18:32 /boot/kernel/ath_hal.ko
-r-xr-xr-x  1 root  wheel   82320 May 24 18:32 /boot/kernel/ath_hal_ar5210.ko
-r-xr-xr-x  1 root  wheel  104976 May 24 18:32 /boot/kernel/ath_hal_ar5211.ko
-r-xr-xr-x  1 root  wheel  236144 May 24 18:32 /boot/kernel/ath_hal_ar5212.ko
-r-xr-xr-x  1 root  wheel  336104 May 24 18:32 /boot/kernel/ath_hal_ar5416.ko
-r-xr-xr-x  1 root  wheel  598336 May 24 18:32 /boot/kernel/ath_hal_ar9300.ko
-r-xr-xr-x  1 root  wheel  406144 May 24 18:32 /boot/kernel/ath_main.ko
-r-xr-xr-x  1 root  wheel   55352 May 24 18:32 /boot/kernel/ath_rate.ko

.. so you can see, not building the whole HAL can save quite a bit.
For example, if you don't need AR9300 support, you can actually avoid
wasting half a megabyte of RAM.  On embedded routers this is quite a
big deal.

The AR9300 HAL can be later further shrunk because, hilariously,
it indeed supports AH_SUPPORT_<xxx> for optionally adding chipset support.
(I'll chase that down later as it's quite a big savings if you're only
building for a single embedded target.)

So:

* Create a very hackish way to load/unload HAL modules
* Create module metadata for each HAL subtype - ah_osdep_arXXXX.c
* Create module metadata for ath_rate and ath_dfs (bluetooth is
  currently just built as part of it)
* .. yes, this means we could actually build multiple rate control
  modules and pick one at load time, but I'd rather just glue this
  into net80211's rate control code.  Oh well, baby steps.
* Main driver is now "ath_main"
* Create an "if_ath" module that does what the ye olde one did -
  load PCI glue, main driver, HAL and all child modules.
  In this way, if you have "if_ath_load=YES" in /boot/modules.conf
  it will load everything the old way and stuff should still work.
* For module autoloading purposes, I actually /did/ fix up
  the name of the modules in if_ath_pci and if_ath_ahb.

If you want to selectively load things (eg on ye cheape ARM/MIPS platforms
where RAM is at a premium) you should:

* load ath_hal
* load the chip modules in question
* load ath_rate, ath_dfs
* load ath_main
* load if_ath_pci and/or if_ath_ahb depending upon your particular
  bus bind type - this is where probe/attach is done.

TODO:

* AR5312 module and associated pieces - yes, we have the SoC side support
  now so the wifi support would be good to "round things out";
* Just nuke AH_SUPPORT_AR5416 for now and always bloat the packet
  structures; this'll simplify other things.
* Should add a simple refcnt thing to the HAL RF/chip modules so you
  can't unload them whilst you're using them.
* Manpage updates, UPDATING if appropriate, etc.

7 years agoMFV r316925: 6101 attempt to lzc_create() a filesystem under a volume results in...
avg [Wed, 24 May 2017 22:34:54 +0000 (22:34 +0000)]
MFV r316925: 6101 attempt to lzc_create() a filesystem under a volume results in a panic

illumos/illumos-gate@b127fe3c059af7adf772735498680b4f2e1405ef
https://github.com/illumos/illumos-gate/commit/b127fe3c059af7adf772735498680b4f2e1405ef

https://www.illumos.org/issues/6101
  lzc_create(), or more correctly, zfs_ioc_create() does not reject an attempt to
  create a filesystem as a child of a volume, instead it proceeds to a crash.
  A crash stack obtained on FreeBSD:
  page fault while in kernel mode

  zap_leaf_lookup()
  fzap_lookup()
  zap_lookup_norm()
  zap_lookup()
  zfs_get_zplprop()
  zfs_fill_zplprops_impl()
  zfs_ioc_create()
  zfsdev_ioctl()
  devfs_ioctl_f()
  kern_ioctl()
  sys_ioctl()
  This crash happened with a kernel without debugging assertions.
  The immediate cause of crash appears to an attempt to interpret a zvol object
  as a zap object.
  For filesystems:
  #define MASTER_NODE_OBJ 1
  For zvols:
  #define ZVOL_OBJ                1ULL
  #define ZVOL_ZAP_OBJ            2ULL
  So, I see two problems here:
     1. an attempt to create a filesystem under a zvol should be rejected as
        early as possible, maybe in zfs_fill_zplprops()
     2. maybe zap_lookup / zap_lockdir should reject objects that are not of one
        of the zap object types

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after: 2 weeks

7 years agoMFV r316923: 8026 retire zfs_throttle_delay and zfs_throttle_resolution
avg [Wed, 24 May 2017 22:32:56 +0000 (22:32 +0000)]
MFV r316923: 8026 retire zfs_throttle_delay and zfs_throttle_resolution

illumos/illumos-gate@6b036259815954b7ad86d651af18efba672cb7a9
https://github.com/illumos/illumos-gate/commit/6b036259815954b7ad86d651af18efba672cb7a9

https://www.illumos.org/issues/8026
  zfs_throttle_delay and zfs_throttle_resolution became disused since the new
  write throttling mechanism was introduced.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after: 1 week

7 years agoMFV r316922: 5380 receive of a send -p stream doesn't need to try renaming snapshots
avg [Wed, 24 May 2017 22:30:21 +0000 (22:30 +0000)]
MFV r316922: 5380 receive of a send -p stream doesn't need to try renaming snapshots

illumos/illumos-gate@471a88e499c660844f4590487ce7c4d5a7090294
https://github.com/illumos/illumos-gate/commit/471a88e499c660844f4590487ce7c4d5a7090294

https://www.illumos.org/issues/5380
  A stream created with zfs send -p -I contains properties of all snapshots of a
  given dataset as opposed to only properties of snapshots in a given range.
  Not only this is suboptimal but the receive code also does not filter
  properties by the range. So, properties of earlier snapshots would be updated
  even though the snapshots themselves are not in the stream (just their
  properties).
  Given that modifying the snapshot properties requires a TXG sync and that the
  snapshots are updated one by one the described behavior may lead to a sever
  performance penalty.

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after: 3 weeks

7 years agoMFC r316921: 8027 tighten up dsl_pool_dirty_delta
avg [Wed, 24 May 2017 22:27:48 +0000 (22:27 +0000)]
MFC r316921: 8027 tighten up dsl_pool_dirty_delta

illumos/illumos-gate@313ae1e182df6e6a04b56c4b73ded33e11b75666
https://github.com/illumos/illumos-gate/commit/313ae1e182df6e6a04b56c4b73ded33e11b75666

https://www.illumos.org/issues/8027
  dsl_pool_dirty_delta() should not wake up waiters when dp->dp_dirty_total ==
  zfs_dirty_data_max, because they wait for dp_dirty_total to fall strictly below
  the threshold.
  It's probably very rare for that condition to occur, but it's better to have
  more accurate code.

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after: 1 week

7 years agoMFV r316920: 8023 Panic destroying a metaslab deferred range tree
avg [Wed, 24 May 2017 22:25:26 +0000 (22:25 +0000)]
MFV r316920: 8023 Panic destroying a metaslab deferred range tree

illumos/illumos-gate@3991b535a8e990c0369be677746a87c259b13e9f
https://github.com/illumos/illumos-gate/commit/3991b535a8e990c0369be677746a87c259b13e9f

https://www.illumos.org/issues/8023
       $C
  ffffff0011bc0970 vpanic()
  ffffff0011bc0a00 strlog()
  ffffff0011bc0a30 range_tree_destroy+0x72(ffffff043769ad00)
  ffffff0011bc0a70 metaslab_fini+0xd5(ffffff0449acf380)
  ffffff0011bc0ab0 vdev_metaslab_fini+0x56(ffffff0462bae800)
  ffffff0011bc0af0 spa_unload+0x9b(ffffff03e3dac000)
  ffffff0011bc0b70 spa_export_common+0x115(ffffff047f4b4000, 2, 0, 0, 0)
  ffffff0011bc0b90 spa_destroy+0x1d(ffffff047f4b4000)
  ffffff0011bc0bd0 zfs_ioc_pool_destroy+0x20(ffffff047f4b4000)
  ffffff0011bc0c80 zfsdev_ioctl+0x4d7(11400000000, 5a01, 8040190, 100003,
  ffffff03e1956b10ffffff0011bc0e68)
  ffffff0011bc0cc0 cdev_ioctl+0x39(11400000000, 5a01, 8040190, 100003,
  ffffff03e1956b10ffffff0011bc0e68)
  ffffff0011bc0d10 spec_ioctl+0x60(ffffff03d9153b00, 5a01, 8040190, 100003,
  ffffff03e1956b10ffffff0011bc0e68, 0)
  ffffff0011bc0da0 fop_ioctl+0x55(ffffff03d9153b00, 5a01, 8040190, 100003,
  ffffff03e1956b10ffffff0011bc0e68, 0)
  ffffff0011bc0ec0 ioctl+0x9b(3, 5a01, 8040190)
  ffffff0011bc0f10 _sys_sysenter_post_swapgs+0x149()

Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: George Wilson <george.wilson@delphix.com>
MFC after: 2 weeks

7 years agoMFV r316917: 7968 multi-threaded spa_sync()
avg [Wed, 24 May 2017 22:21:24 +0000 (22:21 +0000)]
MFV r316917: 7968 multi-threaded spa_sync()

illumos/illumos-gate@94c2d0eb22e9624151ee84a7edbf7178e1bf4087
https://github.com/illumos/illumos-gate/commit/94c2d0eb22e9624151ee84a7edbf7178e1bf4087

https://www.illumos.org/issues/7968
  spa_sync() iterates over all the dirty dnodes and processes each of them by
  calling dnode_sync(). If there are many dirty dnodes (e.g. because we created
  or removed a lot of files), the single thread of spa_sync() calling
  dnode_sync() can become a bottleneck. Additionally, if many dnodes are dirtied
  concurrently in open context (e.g. due to concurrent file creation), the
  os_lock will experience lock contention via dnode_setdirty().
  The solution is to track dirty dnodes on a multilist_t, and for spa_sync() to
  use separate threads to process each of the sublists in the multilist.
  On the concurrent file creation microbenchmark, the performance improvement
  from dnode_setdirty() is up to 7%. Additionally, the wall clock time spent in
  spa_sync() is reduced to 15%-40% of the single-threaded case. In terms of cost/
  reward, once the other bottlenecks are addressed, fixing this bug will provide
  a medium-large performance gain and require a medium amount of effort to
  implement.

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
MFC after: 3 weeks

7 years agoMFV r316916: 7970 zfs_arc_num_sublists_per_state should be common to all multilists
avg [Wed, 24 May 2017 22:15:16 +0000 (22:15 +0000)]
MFV r316916: 7970 zfs_arc_num_sublists_per_state should be common to all multilists

illumos/illumos-gate@10fbdecb05f411234920f8d3c92c148d39106d7e
https://github.com/illumos/illumos-gate/commit/10fbdecb05f411234920f8d3c92c148d39106d7e

https://www.illumos.org/issues/7970
  The global tunable zfs_arc_num_sublists_per_state is used by the ARC and
  the dbuf cache, and other users are planned. We should change this
  tunable to be common to all multilists.

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
MFC after: 3 weeks

7 years agoMFC r316915: 7801 add more by-dnode routines (lint)
avg [Wed, 24 May 2017 21:52:20 +0000 (21:52 +0000)]
MFC r316915: 7801 add more by-dnode routines (lint)

illumos/illumos-gate@411be58a6e030a3b606f1afcc7f2e2459ffda844
https://github.com/illumos/illumos-gate/commit/411be58a6e030a3b606f1afcc7f2e2459ffda844
MFC after: 24 days
X-MFC with: r318823

7 years agoMFC r316914: 7801 add more by-dnode routines
avg [Wed, 24 May 2017 21:49:21 +0000 (21:49 +0000)]
MFC r316914: 7801 add more by-dnode routines

illumos/illumos-gate@b0c42cd4706ba01ce158bd2bb1004f7e59eca5fe
https://github.com/illumos/illumos-gate/commit/b0c42cd4706ba01ce158bd2bb1004f7e59eca5fe

https://www.illumos.org/issues/7801
  Add *_by_dnode() routines for accessing objects given their
  dnode_t *, this is more efficient than accessing the object by
  (objset_t *, uint64_t object). This change converts some but
  not all of the existing consumers. As performance-sensitive
  code paths are discovered they should be converted to use
  these routines.
  Ported from: https://github.com/zfsonlinux/zfs/commit/0eef1bde31d67091d3deed23fe2394f5a8bf2276

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: bzzz77 <bzzz.tomas@gmail.com>
MFC after: 24 days

7 years agoMFC r316913: 7869 panic in bpobj_space(): null pointer dereference
avg [Wed, 24 May 2017 21:45:52 +0000 (21:45 +0000)]
MFC r316913: 7869 panic in bpobj_space(): null pointer dereference

illumos/illumos-gate@a3905a45920de250d181b66ac0b6b71bd200d9ef
https://github.com/illumos/illumos-gate/commit/a3905a45920de250d181b66ac0b6b71bd200d9ef

https://www.illumos.org/issues/7869
  The issue fixed by this patch is a race condition in the deadlist code.
  A thread executing an administrative command that uses
  `dsl_deadlist_space_range()` holds the lock of the whole `deadlist_t` to
  protect the access of all its entries that the deadlist contains in an
  avl tree.
  Sync threads trying to insert a new entry in the deadlist
  (through `dsl_deadlist_insert()` -> `dle_enqueue()`) do not hold the
  deadlist lock at that moment. If the `dle_bpobj` is the empty bpobj (our
  sentinel value), we close and reopen it. Between these two operations,
  it is possible for the `dsl_deadlist_space_range()` thread to dereference
  that bpobj which is `NULL` during that window.
  Threads should hold the a deadlist's `dl_lock` when they manipulate its
  internal data so scenarios like the one above are avoided. In addition,
  threads should also hold the bpobj lock whenever they are allocating the
  subobj list of a bpobj, and not just when they actually insert the subobj
  to the list. This way we can avoid potential memory leaks.

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: George Melikov <mail@gmelikov.ru>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Serapheim Dimitropoulos <serapheim@delphix.com>
MFC after: 2 weeks

7 years agoMFC r316912: 7793 ztest fails assertion in dmu_tx_willuse_space
avg [Wed, 24 May 2017 21:43:34 +0000 (21:43 +0000)]
MFC r316912: 7793 ztest fails assertion in dmu_tx_willuse_space

illumos/illumos-gate@61e255ce7267b52208af9daf434b77d37fb75622
https://github.com/illumos/illumos-gate/commit/61e255ce7267b52208af9daf434b77d37fb75622

https://www.illumos.org/issues/7793
  Background information: This assertion about tx_space_* verifies that we
  are not dirtying more stuff than we thought we would. We “need” to know
  how much we will dirty so that we can check if we should fail this
  transaction with ENOSPC/EDQUOT, in dmu_tx_assign(). While the
  transaction is open (i.e. between dmu_tx_assign() and dmu_tx_commit() —
  typically less than a millisecond), we call dbuf_dirty() on the exact
  blocks that will be modified. Once this happens, the temporary
  accounting in tx_space_* is unnecessary, because we know exactly what
  blocks are newly dirtied; we call dnode_willuse_space() to track this
  more exact accounting.
  The fundamental problem causing this bug is that dmu_tx_hold_*() relies
  on the current state in the DMU (e.g. dn_nlevels) to predict how much
  will be dirtied by this transaction, but this state can change before we
  actually perform the transaction (i.e. call dbuf_dirty()).
  This bug will be fixed by removing the assertion that the tx_space_*
  accounting is perfectly accurate (i.e. we never dirty more than was
  predicted by dmu_tx_hold_*()). By removing the requirement that this
  accounting be perfectly accurate, we can also vastly simplify it, e.g.
  removing most of the logic in dmu_tx_count_*().
  The new tx space accounting will be very approximate, and may be more or
  less than what is actually dirtied. It will still be used to determine
  if this transaction will put us over quota. Transactions that are marked
  by dmu_tx_mark_netfree() will be excepted from this check. We won’t make
  an attempt to determine how much space will be freed by the transaction
  — this was rarely accurate enough to determine if a transaction should
  be permitted when we are over quota, which is why dmu_tx_mark_netfree()
  was introduced in 2014.
  We also won’t attempt to give “credit” when overwriting existing blocks,
  if those blocks may be freed. This allows us to remove the
  do_free_accounting logic in dbuf_dirty(), and associated routines. This

Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
MFC after: 3 weeks

7 years agoIncrease the allowed maximum number of audio channels from 31 to 127
hselasky [Wed, 24 May 2017 21:42:48 +0000 (21:42 +0000)]
Increase the allowed maximum number of audio channels from 31 to 127
in the PCM feeder mixer. Without this change a value of 32 channels is
treated like zero, due to using a mask of 0x1f, causing a kernel
assert when trying to playback bitperfect 32-channel audio. Also
update the AWK script which is generating the division tables to
handle more than 18 channels. This commit complements r282650.

MFC after: 3 days

7 years agoMFC r316908: 7541 zpool import/tryimport ioctl returns ENOMEM because provided buffer...
avg [Wed, 24 May 2017 21:32:35 +0000 (21:32 +0000)]
MFC r316908: 7541 zpool import/tryimport ioctl returns ENOMEM because provided buffer is too small for config

illumos/illumos-gate@8b65a70b763232c90a91f31eb2010314c02ed338
https://github.com/illumos/illumos-gate/commit/8b65a70b763232c90a91f31eb2010314c02ed338

https://www.illumos.org/issues/7541
  When calling zpool import, zpool does a few ioctls to ZFS.
  zpool allocates a buffer in userland and passes it to the kernel so that ZFS
  can copy info into it. ZFS will use it to put the nvlist that describes the
  pool configuration.
  If the allocated buffer is too small, ZFS will return ENOMEM and the call will
  have to be redone. This wastes CPU time and slows down the import process. This
  happens very often for the ZFS_IOC_POOL_TRYIMPORT call.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Pavel Zakharov <pavel.zakharov@delphix.com>
MFC after: 2 weeks

7 years agoMFC r316907: 1300 filename normalization doesn't work for removes
avg [Wed, 24 May 2017 21:29:31 +0000 (21:29 +0000)]
MFC r316907: 1300 filename normalization doesn't work for removes

illumos/illumos-gate@1c17160ac558f98048951327f4e9248d8f46acc0
https://github.com/illumos/illumos-gate/commit/1c17160ac558f98048951327f4e9248d8f46acc0

https://www.illumos.org/issues/1300

FreeBSD note: recent FreeBSD was not affected by the issue fixed as the
name cache is completely bypassed when normalization is enabled.
The change is imported for the sake of ZAP infrastructure modifications.

Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Kevin Crowe <kevin.crowe@nexenta.com>

MFC after: 3 weeks

7 years agoDon't end up manpage titles with a full stop.
trasz [Wed, 24 May 2017 21:02:53 +0000 (21:02 +0000)]
Don't end up manpage titles with a full stop.

MFC after: 2 weeks

7 years agoMFC r316904: 7729 libzfs_core`lzc_rollback() leaks result nvl
avg [Wed, 24 May 2017 20:53:01 +0000 (20:53 +0000)]
MFC r316904: 7729 libzfs_core`lzc_rollback() leaks result nvl

illumos/illumos-gate@ac428481f96be89add7a1edf43ae47dd71038553
https://github.com/illumos/illumos-gate/commit/ac428481f96be89add7a1edf43ae47dd71038553

https://www.illumos.org/issues/7729
  libzfs_core`lzc_rollback() doesn't free the result nvl after lzc_ioctl() call.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Yuri Pankov <yuri.pankov@nexenta.com>

MFC after: 2 weeks

7 years agoMFV r316860: 7545 zdb should disable reference tracking
avg [Wed, 24 May 2017 20:41:26 +0000 (20:41 +0000)]
MFV r316860: 7545 zdb should disable reference tracking

illumos/illumos-gate@4dd77f9e38ef05b39db128ff7608d926fd3218c6
https://github.com/illumos/illumos-gate/commit/4dd77f9e38ef05b39db128ff7608d926fd3218c6

https://www.illumos.org/issues/7545
  When evicting from the ARC, we manipulate some refcount_t's, e.g. arcs_size.
  When using zdb to examine a large amount of data (e.g. zdb -bb on a large pool
  with small blocks), the ARC may have a large number of entries. If reference
  tracking is enabled, there will be ~1 reference for each block in the ARC. When
  evicting, we decrement the refcount and have to search all the references to
  find the one that we are removing, which is very slow.
  Since zdb is typically used to find problems with the on-disk format, and not
  with the code it is running, we should disable reference tracking in zdb.

Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

MFC after: 2 weeks

7 years agoRemove constants and comments for unimplemented entries in the default LDT.
jhb [Wed, 24 May 2017 18:54:21 +0000 (18:54 +0000)]
Remove constants and comments for unimplemented entries in the default LDT.

These entries will never be added to the default LDT in the future.

7 years agoUpdate the "first appeared in" version in several manual pages.
gjb [Wed, 24 May 2017 17:50:34 +0000 (17:50 +0000)]
Update the "first appeared in" version in several manual pages.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

7 years agoUpdate the "first appeared in" version in several manual pages.
gjb [Wed, 24 May 2017 17:47:49 +0000 (17:47 +0000)]
Update the "first appeared in" version in several manual pages.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation

7 years agoo Rearrange struct inpcb fields to optimize the TCP output code path
glebius [Wed, 24 May 2017 17:47:16 +0000 (17:47 +0000)]
o Rearrange struct inpcb fields to optimize the TCP output code path
  considering cache line hits and misses.  Put the lock and hash list
  glue into the first cache line, put inp_refcount inp_flags inp_socket
  into the second cache line.
o On allocation zero out entire structure except the lock and list entries,
  including inp_route inp_lle inp_gencnt.  When inp_route and inp_lle were
  introduced, they were added below inp_zero_size, resulting on not being
  cleared after free/alloc.  This definitely was a source of bugs with route
  caching.  Could be that r315956 has just fixed one of them.
  The inp_gencnt is reinitialized on every alloc, so it is safe to clear it.

This has been proved to improve TCP performance at Netflix.

Obtained from: rrs
Differential Revision: D10686

7 years agoUPDATING: clarify ino64 upgrade instructions even further
emaste [Wed, 24 May 2017 17:11:10 +0000 (17:11 +0000)]
UPDATING: clarify ino64 upgrade instructions even further

- mention COMPAT_FREEBSD11 earlier so that the steps are in chronological
  order
- suggest removing /usr/obj before build to ensure there are no stale
  objects

Reviewed by: allanjude, kib
Sponsored by: The FreeBSD Foundation

7 years agoFix a buffer overflow in bootparamd(8)
asomers [Wed, 24 May 2017 16:30:38 +0000 (16:30 +0000)]
Fix a buffer overflow in bootparamd(8)

If /etc/bootparams contains a line with an excessively long pathname, and a
client asks for that path, then bootparamd will overflow a buffer and crash
while parsing that line.  This is not remotely exploitable since it requires
a malformed /etc/bootparams file.

Reported by: Coverity
CID: 1305954
MFC after: 1 week
Sponsored by: Spectra Logic Corp

7 years agoIfdef out a redundant if statement when LARGE_NAT is disabled.
cy [Wed, 24 May 2017 14:36:51 +0000 (14:36 +0000)]
Ifdef out a redundant if statement when LARGE_NAT is disabled.

MFC after: 1 week

7 years agobhyvegc_resize: make use of reallocarray(3) for bounds-checking.
pfg [Wed, 24 May 2017 14:24:47 +0000 (14:24 +0000)]
bhyvegc_resize: make use of reallocarray(3) for bounds-checking.

Also add __FBSDID.

Reviewed by: grehan

This file lacks a license(!) so for this change the following declaration
applies:

To the greatest extent permitted by, but not in contravention of,
applicable law, Affirmer hereby overtly, fully, permanently, irrevocably
and unconditionally waives, abandons, and surrenders all of Affirmer's
Copyright and Related Rights and associated claims and causes of action,
whether now known or unknown (including existing as well as future claims
and causes of action).

7 years agoAdd BIT_OR2(), BIT_AND2(), BIT_NAND2(), BIT_XOR() and BIT_XOR2().
kib [Wed, 24 May 2017 10:09:54 +0000 (10:09 +0000)]
Add BIT_OR2(), BIT_AND2(), BIT_NAND2(), BIT_XOR() and BIT_XOR2().

Submitted by: Sebastian Huber <sebastian.huber@embedded-brains.de>
MFC after: 2 weeks

7 years agoUse __BSD_VISIBLE test instead checking for absense of _POSIX_SOURCE.
kib [Wed, 24 May 2017 09:25:13 +0000 (09:25 +0000)]
Use __BSD_VISIBLE test instead checking for absense of _POSIX_SOURCE.

The Termios headers <termios.h> and <sys/_termios.h> used sometimes
_POSIX_SOURCE directly to determine if a thing should be exposed to
the user.  This circumvented the feature mechanisms of <sys/cdefs.h>.

Submitted by: Sebastian Huber <sebastian.huber@embedded-brains.de>
MFC after: 2 weeks

7 years agocxgbe/iw_cxgbe: sodisconnect failures are harmless and should not be
np [Wed, 24 May 2017 04:48:09 +0000 (04:48 +0000)]
cxgbe/iw_cxgbe: sodisconnect failures are harmless and should not be
treated as fatal errors.

MFC after: 3 days
Sponsored by: Chelsio Communications