John Marino [Sun, 6 Nov 2011 18:59:30 +0000 (19:59 +0100)]
Merge branch 'vendor/MPFR'
John Marino [Sun, 6 Nov 2011 16:41:46 +0000 (17:41 +0100)]
Upgrade MPFR from 2.4.2-p3 to 3.1.0 on the vendor branch
Antonio Huete Jimenez [Sun, 6 Nov 2011 10:52:24 +0000 (11:52 +0100)]
vmstat - Adapt memory limit field for bigger sizes.
For some types like filesystem inodes the limit is raised up to
KvaSize, which can be larger than the actual display field size,
so enlarge it a bit.
Sascha Wildner [Sat, 5 Nov 2011 22:39:37 +0000 (23:39 +0100)]
Sort some SEE ALSOs in manual pages.
John Marino [Sat, 5 Nov 2011 21:50:36 +0000 (22:50 +0100)]
gprof: silence buildworld errors
It seems that every binutils library and application redefines
PACKAGE_BUG REPORT, and the gprof version clashes with the libbfd
version, so it was disabled to avoid redefinition warnings during
buildworld.
John Marino [Sat, 5 Nov 2011 21:39:58 +0000 (22:39 +0100)]
libgmp: Fix README.DELETED
An experimental README.DELETED was committed. Currently the x86 and
x86_64 files are not vendor branch, so their directories should not have
been removed before.
John Marino [Sat, 5 Nov 2011 20:31:30 +0000 (21:31 +0100)]
libgmp: Upgrade to version 5.0.2
The GNU Multiple Precision Arithmetic Library is required for gcc44 and
all newer gcc compilers. It's alway been built with the "generic" C
files rather than the provided assembly. It should be possible to build
gmp with a base set of assembly for x86 and x86_64, and even use
more processor-specific assembly if CPU_TYPE is set, but implementing that
will require some more work and a lot of testing. So for now, gmp is
still built with the much slower (but more portable) C files.
BUGS FIXED
===================
1. Fat builds fixed.
2. Fixed crash for huge multiplies when old FFT_TABLE2 type of parameter
selection tables' sentinel was smaller than multiplied operands.
3. The solib numbers now reflect the removal of the documented but
preliminary mpn_bdivmod function; we correctly flag incompatibility
with GMP 4.3.
4. Many minor bugs related to portability fixed.
5. The support for HPPA 2.0N now works, after an assembly bug fix.
6. A test case type error has been fixed. The symptom of this bug
was spurious 'make check' failures.
SPEEDUPS
===================
1. Multiplication has been overhauled:
(1) Multiplication of larger same size operands has been improved with
the addition of two new Toom functions and a new internal function
mpn_mulmod_bnm1 (computing U * V mod (B^n-1), B being the word
base. This latter function is used for the largest products,
waiting for a better Schoenhage-Strassen U * V mod (B^n+1)
implementation.
(2) Likewise for squaring.
(3) Multiplication of different size operands has been improved with
the addition of many new Toom function, and by selecting
underlying functions better from the main multiply functions.
2. Division and mod have been overhauled:
(1) Plain "schoolbook" division is reimplemented using faster quotient
approximation.
(2) Division Q = N/D, R = N mod D where both the quotient and
remainder are needed now runs in time O(M(log(N))). This is an
improvement of a factor log(log(N))
(3) Division where just the quotient is needed is now O(M(log(Q))) on
average.
(4) Modulo operations using Montgomery REDC form now take time O(M(n))
(5) Exact division Q = N/D by means of mpz_divexact has been improved
for all sizes, and now runs in time O(M(log(N))).
3. The function mpz_powm is now faster for all sizes. Its complexity has
gone from O(M(n)log(n)m) to O(M(n)m) where n is the size of the modulo
argument and m is the size of the exponent. It is also radically
faster for even modulus, since it now partially factors such modulus
and performs two smaller modexp operations, then uses CRT.
4. The internal support for multiplication yielding just the lower n
limbs has been improved by using Mulders' algorithm.
5. Computation of inverses, both plain 1/N and 1/N mod B^n have been
improved by using well-tuned Newton iterations, and wrap-around
multiplication using mpn_mulmod_bnm1.
6. A new algorithm makes mpz_perfect_power_p asymptotically faster.
7. The function mpz_remove uses a much faster algorithm, is better tuned,
and also benefits from the division improvements.
8. Intel Atom and VIA Nano specific optimisations.
9. Multiplication of large numbers has indirectly been sped up through
better FFT tuning and processor recognition. Since many operations
depend on multiplication, there will be a general speedup.
10. Plus hundreds of smaller improvements and tweaks!
NEW FEATURES
===================
1. New mpz function: mpz_powm_sec for side-channel quiet modexp
computations.
2. New mpn functions: mpn_sqr, mpn_and_n, mpn_ior_n, mpn_xor_n,
mpn_nand_n, mpn_nior_n, mpn_xnor_n, mpn_andn_n, mpn_iorn_n,
mpn_com, mpn_neg, mpn_copyi, mpn_copyd, mpn_zero.
3. The function mpn_tdiv_qr now allows certain argument overlap.
4. Support for fat binaries for 64-bit x86 processors has been added.
5. A new type, mp_bitcnt_t for bignum bit counts, has been introduced.
6. More Core i3, i5 an Core i7 processor models are recognised.
John Marino [Sat, 5 Nov 2011 21:30:30 +0000 (22:30 +0100)]
Merge branch 'vendor/GMP'
John Marino [Wed, 2 Nov 2011 22:54:48 +0000 (23:54 +0100)]
Upgrade GMP from 4.3.2 to 5.0.2 on the vendor branch
Samuel J. Greear [Sat, 5 Nov 2011 21:01:26 +0000 (15:01 -0600)]
kernel - sysv - Bump semaphore limits
* Bump kern.ipc.semmns, the total number of system semaphores, to 341.
This count represents an array of 12-byte tracking structures, 341 of these
consumes a single hardware page.
* Bump kern.ipc.semmni to accomodate the new semmns limit per the PostgreSQL
calculations from
http://developer.postgresql.org/pgdocs/postgres/kernel-resources.html
Antonio Huete Jimenez [Sat, 5 Nov 2011 18:00:20 +0000 (19:00 +0100)]
hammer - Migration to libhammer (step 1/many)
- Start using libhammer
- Migrate info directive
Antonio Huete Jimenez [Wed, 17 Aug 2011 17:34:13 +0000 (19:34 +0200)]
libhammer - Hook it up into the build.
Antonio Huete Jimenez [Sat, 5 Nov 2011 17:49:24 +0000 (18:49 +0100)]
libhammer - Fix a misplaced #endif for the header guardian.
Antonio Huete Jimenez [Sat, 29 Oct 2011 18:15:45 +0000 (20:15 +0200)]
libhammer - inodes field was also overlooked.
Sascha Wildner [Sat, 5 Nov 2011 18:07:34 +0000 (19:07 +0100)]
ieee80211*(9) manual pages: Add some missing #include's.
Sascha Wildner [Sat, 5 Nov 2011 10:05:28 +0000 (11:05 +0100)]
Sync zoneinfo database with tzdata2011n from munnari.oz.au
australasia: 8.28 -> 8.29
backward: 8.10 -> 8.11
europe: 8.39 -> 8.40
northamerica: 8.50 -> 8.51
zone.tab: 8.50 -> 8.52
* australasia: Fiji has altered the end date for summer time this
summer, moving it from February to January. It is by no means sure
it won't shift again, but this does appear to be the current plan.
* backward, europe, zone.tab: Pridnestrovian Moldavian Republic
(Europe/Tiraspol) has not followed much of Russia, and will not
retain summer time - rather reverting to standard time along with
western Europe, and Ukraine, on Oct 30, as it was earlier scheduled
to do. This removes the Europe/Tiraspol zone (again) as the
variation never actually happened (and returns the entry in the
"backward" file).
* northamerica: Cuba (America/Havana) has extended summer time by two
weeks, now to end on Nov 13, rather than the (already past) Oct 30.
Matthew Dillon [Fri, 4 Nov 2011 17:52:33 +0000 (10:52 -0700)]
kernel - Attempt to workaround low memory deadlock
* Mark the hammer flusher threads as system threads and call
vm_wait_nominal() in the inode flush loop prior to acquiring
an inode lock.
* This attempts to work around an issue where the pageout daemon has
to do a BMAP indirectly via vnode_pager_put_pages(), which requires
a dive into hammer deep enough to need the inode lock.
The pageout daemon checks the vnode lock but has no visibility into
the inode lock. Only the hammer backend (theoretically) can acquire
the inode lock without holding the vnode lock. Hopefully this will
improve the issue.
Reported-by: Antonio Huete Jimenez <tuxillo@quantumachine.net>
Sascha Wildner [Fri, 4 Nov 2011 16:59:51 +0000 (17:59 +0100)]
twa(4): Remove some bogus NULL checks after kmalloc() with M_WAITOK.
Reported-by: alexh
Sascha Wildner [Fri, 4 Nov 2011 16:20:06 +0000 (17:20 +0100)]
kernel: Replace some bzero()s with M_ZERO in the preceding kmalloc().
Sepherosa Ziehau [Fri, 4 Nov 2011 14:01:32 +0000 (22:01 +0800)]
tcp: Bring back MSG_EOF flag support in sosendtcp()
Though it was originally designed for T/TCP, it is nice to have
While I'm here, clean up the 'async' setting
Sepherosa Ziehau [Fri, 4 Nov 2011 13:33:50 +0000 (21:33 +0800)]
send(2): Add MSG_SYNC to allow user to disable asynchronized pru_send per-socket
Sascha Wildner [Fri, 4 Nov 2011 12:32:11 +0000 (13:32 +0100)]
Remove some bogus CVS IDs.
Sepherosa Ziehau [Fri, 4 Nov 2011 11:35:02 +0000 (19:35 +0800)]
tcp: Enable asynchronized pru_send by default
Sepherosa Ziehau [Fri, 4 Nov 2011 11:29:21 +0000 (19:29 +0800)]
tcp: Partly revert f2a3782
We do not need to sync the target netisr before disconnect or shutdown,
the problem was fixed in 392cd26 and turned out to be ipi message
ordering problem.
Matthew Dillon [Fri, 4 Nov 2011 05:25:31 +0000 (22:25 -0700)]
kernel - Fix localhost packet misordering
* netisr thread ports are based on IPIs, but when we enable asynch socket
writes a user thread which gets moved between cpus sending async netmsgs
while doing so can result in the netisr receiving those messages out
of order, corrupting the data stream.
* Add TDF_FORCE_SPINPORT to allow the netisr threads to implement their
message ports as spinports instead of threadports, which guarantees
message ordering.
John Marino [Thu, 3 Nov 2011 21:38:50 +0000 (22:38 +0100)]
Binutils 2.20: Effectively remove from world
There is no longer a reason to maintain multiple versions of binutils
in the base system. While contrib/binutils-2.20 directory isn't being
removed quite yet, this commit effectively removed binutils 2.20 from
DragonFly.
Sometime in the future, binutils may be removed from the objformat
handler. The value of the BINUTILSVERS variable no longer has any
effect, and the only version of binutils on the system is 2.21.
Matthew Dillon [Thu, 3 Nov 2011 20:41:48 +0000 (13:41 -0700)]
kernel - Fix bug in last commit
* Ooops, lwkt_gettoken->lwkt_reltoken.
Reported-by: ftigeot
Markus Pfeiffer [Fri, 4 Nov 2011 17:51:47 +0000 (17:51 +0000)]
Added AMD Features2 bits 17 (TCE), 23 (PCX_CORE) and 24 (PCX_NB) to identcpu.c for pc32 and pc64
Matthew Dillon [Thu, 3 Nov 2011 17:51:40 +0000 (10:51 -0700)]
kernel - Fix /dev/mem access for memory >=4GB
* The (v) variable was a u_int, chopping off the top 32 bits of a 64 bit
physical address. Change to a long.
Matthew Dillon [Thu, 3 Nov 2011 17:49:16 +0000 (10:49 -0700)]
kernel - Hold required token when accessing p_flags, adjust kmem access
* Numerous adjustments to p->p_flag were not being done with p->p_token held.
In particular uiomove().
* Replace P_DEADLKTREAT with LWP_DEADLKTREAT in several places where it had
not been previously converted.
* Allow DMAP access in is_globaldata_space() for x86-64
Sepherosa Ziehau [Thu, 3 Nov 2011 16:32:35 +0000 (00:32 +0800)]
ioapic/x86_64: Per-cpu irqmap array
Sepherosa Ziehau [Thu, 3 Nov 2011 15:06:16 +0000 (23:06 +0800)]
MachIntrABI/x86_64: Remove unnecessary setidt in intr_setup/teardown
MachIntrABI.setdefault() has already done that
Sepherosa Ziehau [Thu, 3 Nov 2011 14:58:32 +0000 (22:58 +0800)]
ioapic/x86_64: Add missing imen_lock/unlock
Sepherosa Ziehau [Thu, 3 Nov 2011 13:38:49 +0000 (21:38 +0800)]
tcp: Allow pure asynchronized pru_send
- net.inet.tcp.sosnd_async is added to allow pure asynchronized pru_send.
It is default to off currently.
- To prevent soclose() and soshutdown() from interfering TCP processing on
the loopback interface, so_pru_sync() is added, which will make sure
that so_pru_disconnect() and so_pru_shutdown() run only after all of the
previous sent packets had been requeued to netisr (the semantics of the
original half asynchronized pru_send).
Sascha Wildner [Thu, 3 Nov 2011 10:19:20 +0000 (11:19 +0100)]
gcc41: Add a missing file to CLEANFILES.
Sascha Wildner [Wed, 2 Nov 2011 23:53:59 +0000 (00:53 +0100)]
loader.8: Fix typo.
Matthew Dillon [Wed, 2 Nov 2011 06:44:37 +0000 (23:44 -0700)]
kernel - reformulate the maxusers auto-sizing calculation
* Reformulate the maxusers auto-sizing calculation, which is used as a
basis for mbufs and mbuf cluster calculations. Base the values on
limsize (basically the lower of KVM vs physical memory).
* Remove artificial limits.
* This basically effects x86-64 systems with > 4G of ram, greatly
increasing the default maxusers value and related mbuf limits.
Matthew Dillon [Wed, 2 Nov 2011 06:43:11 +0000 (23:43 -0700)]
kernel - Fix spin-based msgports
* LWKT threads can use thread/IPI or spin-based message ports. The
default is thread-based. Spin-based ports had numerous problems which
would result in panics. This commit fixes those panics and makes the
spinlock version viable.
* However, currently there is no performance improvement so the default
is staying as it was.
Matthew Dillon [Wed, 2 Nov 2011 06:42:06 +0000 (23:42 -0700)]
kernel - Fix x86-64 pmap race
* Fix a x86-64 pmap race where a pte can get ripped out from under
the pmap_remove*() code. Recheck the pte after locking pt_pv.
Matthew Dillon [Wed, 2 Nov 2011 06:38:42 +0000 (23:38 -0700)]
kernel - Major MP work on kq and signal processing
* Remove the global kq_token and move to a per-kq and per-kqlist
pool token. This greatly improves postgresql and mysql performance
on MP systems.
* Adjust signal processing tokens to be per-LWP instead of per-PROC.
Signal delivery still utilizes a per-proc token but signal distribution
now uses a per-LWP token, which allows the trap code to only lock the
LWP when checking for pending signals.
This also significantly improves database performance.
* The socket code also now uses only its per-socket pool token instead
of kq_token for its kq operations. kq handles its own tokens.
Matthew Dillon [Wed, 2 Nov 2011 06:33:40 +0000 (23:33 -0700)]
kernel - add MAP_SIZEALIGN
* Add a mmap() MAP_SIZEALIGN flag which requests alignment the same
as the size argument (different from Solaris's MAP_ALIGN which uses
the address hint).
* Will be used in upcoming libc/stdlib/dmalloc.c work. The dmalloc code
will work without it just fine, too..
Matthew Dillon [Wed, 2 Nov 2011 06:31:50 +0000 (23:31 -0700)]
buildkernel - remove COMPAT_43 and COMPAT_DF12
* Remove old compats that we really should not be compiling into kernels
any more.
In particular, the old getpid() did some weird things which created
unnecessary slowdowns on MP systems.
Matthew Dillon [Wed, 2 Nov 2011 06:21:59 +0000 (23:21 -0700)]
kernel - Add bsflong() asm functions
* Add bsflong() inline asm functions which operate on the 'long' data type.
Sascha Wildner [Tue, 1 Nov 2011 21:29:17 +0000 (22:29 +0100)]
Bump __DragonFly_version for the removal of <crypt.h> (to be safe).
Jan Lentfer [Tue, 1 Nov 2011 21:02:29 +0000 (22:02 +0100)]
pf/pf_ioctl.c: Fix whitespace error
Sascha Wildner [Tue, 1 Nov 2011 10:34:17 +0000 (11:34 +0100)]
Unbreak LINT.
Jan Lentfer [Thu, 6 Jan 2011 10:03:55 +0000 (11:03 +0100)]
pf: convert to use kmalloc instead of zalloc
Matthew Dillon [Mon, 31 Oct 2011 21:18:52 +0000 (14:18 -0700)]
kernel - Fix mbuf cluster statistics, fix type change bug
* The mbuf cluster statistics were not properly handling a sharecount race case,
causing the cluster count to continuously increase under heavy loads.
* atomic_set_short() was being improperly used to set m->m_type, causing the type field
to collect a logical OR of changeouts. Just set it normally.
* We don't need to use atomic ops for per-cpu stats updates.
Reported-by: Peter Avalos <peter@theshell.com>, "Samuel J. Greear" <sjg@evilcode.net>
Sascha Wildner [Mon, 31 Oct 2011 20:00:42 +0000 (21:00 +0100)]
Remove various unneeded definitions of abs() in userland.
Matthew Dillon [Mon, 31 Oct 2011 19:55:46 +0000 (12:55 -0700)]
kernel - Expand panic message for invalid pte case
* Expand a panic assertion to provide more information.
Matthew Dillon [Mon, 31 Oct 2011 18:16:59 +0000 (11:16 -0700)]
kernel - Fix missing token release in msync() error path
* Fix a missing token release in the msync() error path that would lead
to a panic in the syscall return code.
Reported-by: swildner
Maurizio Lombardi [Mon, 31 Oct 2011 11:01:49 +0000 (12:01 +0100)]
Fix a macro argument expansion bug
Sascha Wildner [Mon, 31 Oct 2011 15:43:37 +0000 (16:43 +0100)]
adduser(8): Sync with FreeBSD.
Submitted-by: Juan Francisco Cantero Hurtado <iam@juanfra.info>
Dragonfly-bug: <http://bugs.dragonflybsd.org/issue2159>
<http://bugs.dragonflybsd.org/issue2160>
Sascha Wildner [Mon, 31 Oct 2011 12:00:24 +0000 (13:00 +0100)]
Fix x86_64 buildkernel with 'options DIAGNOSTIC'.
Sascha Wildner [Mon, 31 Oct 2011 00:02:04 +0000 (01:02 +0100)]
Fix buildkernel without 'options INVARIANTS'.
Submitted-by: Joel K. Pettersson <joelkpettersson@gmail.com>
Dragonfly-bug: <http://bugs.dragonflybsd.org/issue2172>
Sascha Wildner [Sun, 30 Oct 2011 20:20:32 +0000 (21:20 +0100)]
Remove /usr/include/crypt.h via 'make upgrade'.
Sascha Wildner [Sun, 30 Oct 2011 20:16:07 +0000 (21:16 +0100)]
Revert "libcrypt - install crypt.h header"
This reverts commit
b4ed82ece2b69f4a6711d35c5e42938dfc1d804c.
BSDs have libcrypt and the prototypes for its functions are in
<unistd.h>. The reason we had crypt.h installed for a while was
to make KDE link against libcrypt, due to a wrong check in KDE.
Unfortunately, at least one other package (chat/dircproxy)
assumed that if <crypt.h> exists, it would also find prototypes
for crypt() and friends there, which is not the case. So it
would crash on x86_64 due to defaulting to int as crypt()'s
return type (which is a pointer).
The check in KDE has been fixed since and it properly checks
for the presence of libcrypt:
https://bugs.kde.org/show_bug.cgi?id=247627
Hence this revert.
In-discussion-with: alexh
Sepherosa Ziehau [Sun, 30 Oct 2011 13:50:45 +0000 (21:50 +0800)]
ioapic_abi/x86_64: Optimize the GSI search a little bit
Use the recorded max line based IRQ instead of scanning the whole
IRQ map array
Sepherosa Ziehau [Sun, 30 Oct 2011 12:27:39 +0000 (20:27 +0800)]
ioapic_abi/x86_64: Record the max line based IRQ
Sepherosa Ziehau [Sun, 30 Oct 2011 11:18:57 +0000 (19:18 +0800)]
x86_64/ioapic_abi: Rework debug messages
John Marino [Sat, 29 Oct 2011 22:59:01 +0000 (00:59 +0200)]
grep: Upgrade to version 2.9
Release 2.9 (2011-06-21) [stable]
Release 2.8 (2011-05-13) [stable]
Bug Fixes
===================
1. echo c|grep '[c]' would fail for any c in 0x80..0xff,
and in many locales.
E.g., printf '\xff\n'|grep "$(printf '[\xff]')" || echo FAIL
would print FAIL rather than the required matching line.
[bug introduced in grep-2.6]
2. grep's interpretation of range expression is now more consistent with
that of other tools. [bug present since multi-byte character set
support was introduced in 2.5.2, though the steps needed to reproduce
it changed in grep-2.6]
3. grep erroneously returned with exit status 1 on some memory allocation
failure. [bug present since "the beginning"]
4. grep no longer clobbers heap for an ERE like '(^| )*( |$)'
[bug introduced in grep-2.6]
5. grep is faster on regular expressions that match multibyte characters
in brackets (such as '[áéíóú]').
6. echo c|grep '[c]' would fail for any c in 0x80..0xff, with a uni-byte
encoding for which the byte-to-wide-char mapping is nontrivial. For
example, the ISO-88591 locales are not affected, but ru_RU.KOI8-R is.
[bug introduced in grep-2.6]
7. grep -P no longer aborts when PCRE's backtracking limit is exceeded
Before, echo
aaaaaaaaaaaaaab |grep -P '((a+)*)+$' would abort. Now,
it diagnoses the problem and exits with status 2.
John Marino [Sat, 29 Oct 2011 23:57:46 +0000 (01:57 +0200)]
Merge branch 'vendor/GREP'
John Marino [Sat, 29 Oct 2011 22:00:25 +0000 (00:00 +0200)]
Upgrade grep version 2.7 to 2.9 on the vendor branch
John Marino [Sat, 29 Oct 2011 20:02:12 +0000 (22:02 +0200)]
diff: Remove location modification from 2004
diffutils has been carrying this modification since version 2.8.1. That
version was changed to support libgnuregex which has since been removed.
It appears that likely libgnuregex didn't support the
RE_NO_POSIX_BACKTRACKING option, and thus required the modification.
John Marino [Sat, 29 Oct 2011 17:47:23 +0000 (19:47 +0200)]
diffutils: Upgrade to version 3.2
The majority of the changes were inherited from gnulib. There were only
a few observable differences from version 3.0:
Release 3.2 (2011-09-02) [stable]
Release 3.1 (2011-08-10) [stable]
Bug fixes
===================
diff no longer reports spurious differences merely because two entries
in the same directory have names that compare equal in the current
locale, or compare equal because --ignore-file-name-case was given.
Changes in behavior
===================
--ignore-file-name-case now applies at the top level too.
For example, "diff dir inIt" might compare "dir/Init" to "inIt".
New features
===================
diff and sdiff have a new option --ignore-trailing-space (-Z).
John Marino [Sat, 29 Oct 2011 19:02:07 +0000 (21:02 +0200)]
Merge branch 'vendor/DIFFUTILS'
John Marino [Sat, 29 Oct 2011 15:39:48 +0000 (17:39 +0200)]
Upgrade diffutils from 3.0 to 3.2 on the vendor branch
Matthew Dillon [Sat, 29 Oct 2011 18:37:03 +0000 (11:37 -0700)]
kernel - Fix LINT compilation on 32-bit
* Fix conditional debug compilation that was breaking 32-bit LINT builds
Reported-by: swildner
Matthew Dillon [Sat, 29 Oct 2011 18:23:24 +0000 (11:23 -0700)]
kernel - Fix deadlock in vm_prefault
* vm_prefault*() was being called while the primary vm_fault page was
still being held busy, which could result in a deadlock.
* Reorder the case to unbusy the primary fault page before calling
vm_prefault().
Reported-by: tuxillo
Matthew Dillon [Sat, 29 Oct 2011 18:20:34 +0000 (11:20 -0700)]
ipcs - Make it compile w/WARNS=6
* Correct misc types, verify compilation on 32 and 64 bit
Jan Lentfer [Sat, 30 Apr 2011 16:51:23 +0000 (18:51 +0200)]
ipcs: Adjust ipcs display to take into account new shared memory sizes
Antonio Huete Jimenez [Sat, 29 Oct 2011 17:27:59 +0000 (19:27 +0200)]
libhammer - Include overlooked field freebigblocks.
Sepherosa Ziehau [Sat, 29 Oct 2011 14:35:23 +0000 (22:35 +0800)]
x86_64/nexus: Per-cpu IRQ rman
Now interrupt thread will be pin to the same CPU as where its GSI
will go.
Sepherosa Ziehau [Sat, 29 Oct 2011 13:13:43 +0000 (21:13 +0800)]
x86_64/ioapic: Allow GSI's target CPU to be configured
- Tuneable hw.ioapic.gsi.X.cpu is added, which could be used to specify
the GSI X's target CPU id
- If hw.ioapic.gsi.X is not set, then GSI X will be target to CPU Y,
Y = X % ncpus
Sascha Wildner [Sat, 29 Oct 2011 09:57:42 +0000 (11:57 +0200)]
kernel: Add missing MODULE_VERSION()s for file systems.
The loader will figure out by itself whether to load a module or not,
depending on whether it's already in the kernel config or not, iif
MODULE_VERSION() is present.
I.e., if MSDOSFS (that has MODULE_VERSION()) is in the config and
msdos_load="YES" is in /boot/loader.conf, msdos.ko will not be loaded
by the loader at all.
Without MODULE_VERSION() it will lead (in the best case) to whining in
dmesg like for ahci or (in the worst case) to weird behavior, such as
for nullfs:
# mount -a
null: vfsload(null): No such file or directory
Therefore, we definitely want MODULE_VERSION() for all new modules.
This commit is the first in a series to add the missing MODULE_VERSION()s.
I know that ufs is not a module, just included it for completeness' sake.
Reported-by: marino, tuxillo
Sascha Wildner [Sat, 29 Oct 2011 06:12:38 +0000 (08:12 +0200)]
Further shared memory adjustments to be in line with POSIX.
* shmat()'s and shmdt()'s addr argument shall be const.
* Make struct shmid_ds's shm_nattch unsigned and define the shmatt_t
type for it.
* More manual page adjustments.
Matthew Dillon [Sat, 29 Oct 2011 02:09:27 +0000 (19:09 -0700)]
kernel - Autosize maximum shm pages
* If not overridden with a tunable autosize sysv shm to 2/3 of available
ram.
Matthew Dillon [Sat, 29 Oct 2011 01:54:01 +0000 (18:54 -0700)]
kernel - Fix bug in shmget()
* Fix bug in shmget() which was truncating requests >= 4G.
Matthew Dillon [Sat, 29 Oct 2011 01:53:26 +0000 (18:53 -0700)]
kernel - Remove libc shm shims
* Remove the shims so the new system calls are used instead of shmsys().
Matthew Dillon [Sat, 29 Oct 2011 00:37:07 +0000 (17:37 -0700)]
kernel - shmget() adjustments
* Fix prototype and manual page
Matthew Dillon [Sat, 29 Oct 2011 00:19:58 +0000 (17:19 -0700)]
kernel - regenerate system calls
* Regenerate system calls (shm_ds).
Matthew Dillon [Sat, 29 Oct 2011 00:17:30 +0000 (17:17 -0700)]
kernel - shmid_ds structure needs to change on 64-bit :-(
* shmid_ds had very old parameters and used 'int' for the shm segment
size. It has to be adjusted to use size_t to accomodate shm segments
greater than 2GB.
This will break binary package compatibility on 64-bit systems until
the related packages are recompiled.
* shmget() system call now takes a size_t instead of an int.
Matthew Dillon [Fri, 28 Oct 2011 23:51:58 +0000 (16:51 -0700)]
killall - Add support for pts specifications
* killall -t <number> now uses /dev/pts/<number> instead of
/dev/tty<number>.
killall -t <alpha>* continues to use /dev/tty<alpha>*.
John Marino [Thu, 27 Oct 2011 22:33:06 +0000 (00:33 +0200)]
gcc44: Update version from 4.4.6-RELEASE to 4.4.7-
20111025
John Marino [Fri, 28 Oct 2011 20:08:30 +0000 (22:08 +0200)]
Merge branch 'vendor/GCC44'
John Marino [Thu, 27 Oct 2011 22:03:08 +0000 (00:03 +0200)]
Upgrade GCC from 4.4.6-RELEASE to 4.4.7 snapshot 2011-10-25
Matthew Dillon [Fri, 28 Oct 2011 17:20:26 +0000 (10:20 -0700)]
kernel - Fix vm_object->rb_memq race in pageout daemon
* We were not properly holding a VM object's token while scanning its
rb_memq. Hold the token properly and also assert that it is held.
Matthew Dillon [Fri, 28 Oct 2011 16:32:51 +0000 (09:32 -0700)]
kernel - Another huge HUGE VM performance improvement for many-cores
This requires a bit of explanation. The last single-point spinlocks in the
VM system were the spinlocks for the inactive and active queue. Even though
these two spinlocks are only held for a very short period of time they can
create a major point of contention when one has (e.g.) 48 cores all trying
to run a VM fault at the same time. This is an issue with multi-socket/
many-cores systems and not so much an issue with single-socket systems.
On many cores systems the global VM fault rate was limited to around
~200-250K zfod faults per second prior to this commit on our 48-core
opteron test box. Since any single compiler process can run ~35K zfod
faults per second the maximum concurrency topped out at around ~7 concurrent
processes.
With this commit the global VM fault rate was tested to almost 900K zfod
faults per second. That's 900,000 page faults per second (about 3.5 GBytes
per second). Typical operation was consistently above 750K zfod faults per
second. Maximum concurrency at a 35K fault rate per process is thus
increased from 7 processes to over 25 processes, and is probably approaching
the physical memory bus limit considering that one also has to take into
account generic page-fault overhead above and beyond the memory impact on the
page itself.
I can't stress enough how important it is to avoid contention entirely when
possible on a many-cores system. In this case even though the VM page queue
spinlocks are only held for a very short period of time, the convulsing of
the cache coherency management between physical cpu sockets when all the
cores need to use the spinlock still created an enormous bottleneck. Fixing
this one spinlock easily doubled concurrent compiler performance on our
48-core opteron.
* Fan-out the PQ_INACTIVE and PQ_ACTIVE page queues from 1 queue to
256 queues, each with its own spin lock.
* This removes the last major contention point in the VM system.
* -j48 buildkernel test on monster (48-core opteron) now runs in 55 seconds.
It was originally 167 seconds, and 101 seconds just prior to this commit.
Concurrent compiles are now three times faster (a +200% improvement) on
a many-cores box, with virtually no contention at all.
Matthew Dillon [Fri, 28 Oct 2011 16:29:28 +0000 (09:29 -0700)]
kernel - Clean up spinlock code, add more lwkt_yield()s
* Clean up some of the critical path in the spin_unlock() API
* Add a few more lwkt_yield()s in the buffer cache and vm_object cleaning
code.
Matthew Dillon [Fri, 28 Oct 2011 16:27:20 +0000 (09:27 -0700)]
kernel - add lwkt_set_interrupt_support_thread() API
* Add a new API that may be used by a device driver's support thread
to run the thread at a higher (near interrupt) priority and allow
it to preempt normal threads.
* Adjust the AHCI driver's helper threads to use the new API.
Sepherosa Ziehau [Fri, 28 Oct 2011 15:47:31 +0000 (23:47 +0800)]
swi: Pass cpuid to swi register and unregister
Pass -1 as cpuid then these functions will try pin the ithread to
different CPU based on the 'intr' to be registered/unregistered.
Device and taskqueue swi ithreads' cpuid is not explicitly specified,
i.e. -1 is used, swi_vm still runs on CPU0.
Sepherosa Ziehau [Fri, 28 Oct 2011 15:33:11 +0000 (23:33 +0800)]
intr: Pass cpuid to register_int and unregister_int
Sascha Wildner [Fri, 28 Oct 2011 12:30:41 +0000 (14:30 +0200)]
Fix i386 buildkernel.
Matthew Dillon [Fri, 28 Oct 2011 06:50:51 +0000 (23:50 -0700)]
kernel - More many-cores SMP work
* Add lwkt_yield() calls in a few critical places which can hog the cpu
on large many-cores boxes during periods of very heavy contention. This
allows other kernel threads on the same cpu to run and reduces symptoms
of e.g. high ping times under certain load conditions.
* Run the callout kernel threads at the same priority as other kernel
threads so cpu-hogging operations run from callouts can yield to
other kernel threads (e.g. yield to the netisr threads).
* Change the vm_page_alloc() API to catch situations where the allocation
races an insertion due to potentially blocking when dealing with
PQ_CACHE pages. VM_ALLOC_NULL_OK allows vm_page_alloc() to return NULL
in this case (otherwise it will panic).
* Change vm_page_insert() to return TRUE if the insertion succeeded and
FALSE if it didn't due to a race against another thread.
* Change the meaning of the cpuid argument to lwkt_alloc_thread() and
lwkt_create(). A cpuid of -1 will cause the kernel to choose a cpu
to run the thread on (instead of choosing the current cpu).
Eventually this specification will allow dynamic migration (but not at
the moment).
Adjust lwp_fork() to specify the current cpu, required for initial
LWKT calls when setting the forked thread up.
Numerous kernel threads will now be spread around available cpus for
now. devfs core threads, NFS socket threads, etc.
Interrupt threads are still fixed on cpu 0 awaiting additional work from
Sephe.
Put the emergency interrupt thread on the last cpu.
* Change the vm_page_grab() API. When VM_ALLOC_ZERO is specified the
vm_page_grab() code will automatically set an invalid page valid and
zero it (using the PG_ZERO optimization if possible). Pages which are
already valid are not zero'd.
This simplies several use cases.
* Change vm_fault_page() to enter the page into the pmap while the vm_map
is still locked, instead of after unlocking it. For now anyhow.
* Minor change to ensure that a deterministic value is stored in *freebuf
in vn_fullpath().
* Minor debugging features added to help track down a x86-64 sge-fault
issue.
Sascha Wildner [Fri, 28 Oct 2011 03:23:19 +0000 (05:23 +0200)]
zone.tab: Fix tzsetup(8) breakage.
If a country has >1 zones, each one needs a description.
tzdata2011m accidentally violated this rule (when Moldova was split),
which caused tzsetup(8) to exit early and whine about it:
tzsetup: /usr/share/zoneinfo/zone.tab:261: conflicting zone definition
To fix this, add the standard "most locations" for the Europe/Chisinau
zone until the next tzdata2011n arrives.
Matthew Dillon [Thu, 27 Oct 2011 03:14:26 +0000 (20:14 -0700)]
kernel - Fix deep recursion in vm_object_collapse() (2)
* Fix bug in previous deep recursion commit. A chainlock was being
released too late.
Matthew Dillon [Thu, 27 Oct 2011 02:03:42 +0000 (19:03 -0700)]
kernel - Fix memory leak when execv()ing certain paths.
* Fix a memory leak when execv()ing paths prefixed with a "./"
Matthew Dillon [Thu, 27 Oct 2011 01:56:39 +0000 (18:56 -0700)]
kernel - Fix deep recursion in vm_object_collapse()
* vm_object_collapse() will loop but its backing_object sometimes needs
to be deallocated as well and this can trigger another collapse against
a different parent object.
* Introduce vm_object_dealloc_list and friends to collect a list of objects
requiring deallocation so the caller can run the list in a way that avoids
a deep recursion.
Reported-by: juanfra
Matthew Dillon [Wed, 26 Oct 2011 22:48:10 +0000 (15:48 -0700)]
test - Add code to test recent bus error issue
Submitted-by: "Samuel J. Greear" <sjg@evilcode.net>
Matthew Dillon [Wed, 26 Oct 2011 22:44:13 +0000 (15:44 -0700)]
kernel - Fix recently introduced bus error w/postgres scoreboard
* The OBJ_ONEMAPPING flag has to be cleared when forking a shared
mapping.
* Fixed an issue with the postgres scoreboard.
Reported-by: Studbolt, thesjg