John Marino [Sat, 15 Aug 2015 13:10:32 +0000 (15:10 +0200)]
ctypedef: Replace entire "print" sections with one element
With a recent change to localedef, the "print" section became completely
redundant. Since it can consume several hundred thousand bytes, removing
it makes the ctype src files much smaller, usually by 33% - 50%.
However, non-breaking spaces have not been previously defined despite
being used in monetary definitions. To correct this mistake, the
non-breaking space has been introduced to ctype as a printable blank
character. Localedef considers "blanks" as "spaces" so there is no need
to explicitly define this as a space as well.
John Marino [Sat, 15 Aug 2015 13:04:22 +0000 (15:04 +0200)]
cldr2def: Slim down ctype src files
I originally modified the tool to use "ranges" to define the CTYPE,
e.g. "<a>;...;<z>" rather than "<a>;<b>;<c> ... <z>". This worked
great on UTF-8 but converting to other encodings is not supported
because part of the range may not exist, or the upper boundary may
come before the lower boundary in the target encoding. Thus I had
to remove that work, but I was able to retain the removal of the now
redundant "print" section.
I confirmed that the output without "print" section was identical to
before, and then I added back "print" section with a single element:
NO-BREAK_SPACE. This character is used in quite a few monetary
defintions, but it was never mapped to CTYPE, which I believe is a
mistake. NO-BREAK_SPACE is also defined as a blank, which is considered
a space as well by localedef tool (so there's no need to also define a
"space" section").
The net change is multibyte encodings now have non-breaking spaces
1) recognized and 2) defined as printable.
John Marino [Sat, 15 Aug 2015 12:03:22 +0000 (14:03 +0200)]
localedef(1): eliminate need for "print" definition
By having space automatically classified as "print" type, we can
eliminate the print section from ctype src files completely (they
are just "graph" plus "<space>".
Sepherosa Ziehau [Sat, 15 Aug 2015 12:34:39 +0000 (20:34 +0800)]
inet6: Set rtentry properly in nd6_output()
Sepherosa Ziehau [Sat, 15 Aug 2015 08:58:46 +0000 (16:58 +0800)]
inet6/nd6: Join and leave solicited node mcast group once on cpu0
Sepherosa Ziehau [Sat, 15 Aug 2015 08:58:18 +0000 (16:58 +0800)]
inet/arp: Announce proxied ARP once on cpu0
Sepherosa Ziehau [Sat, 15 Aug 2015 08:35:46 +0000 (16:35 +0800)]
inet6: Dispatch mcast routing expire upcalls to netisr0 to run
Sepherosa Ziehau [Sat, 15 Aug 2015 08:03:53 +0000 (16:03 +0800)]
inet6: Dispatch route timeouts to netisr to run
While I'm here, remove unnecessary crit sections.
Sepherosa Ziehau [Sat, 15 Aug 2015 03:47:01 +0000 (11:47 +0800)]
inet6: Avoid excessive calling of in6_addr2zoneid()
Obtained-from: FreeBSD
Sascha Wildner [Sat, 15 Aug 2015 11:13:41 +0000 (13:13 +0200)]
Update the pciconf(8) database.
July 31, 2015 snapshot from http://pciids.sourceforge.net/
Sascha Wildner [Sat, 15 Aug 2015 10:50:13 +0000 (12:50 +0200)]
kernel/lagg: Use __printflike().
Sascha Wildner [Sat, 15 Aug 2015 09:31:29 +0000 (11:31 +0200)]
kernel/ddb: Change some function definitions to static.
All these were already declared static.
Sascha Wildner [Sat, 15 Aug 2015 08:09:11 +0000 (10:09 +0200)]
i386 removal, part 35/x: Remove the i386 specific vesa(4) code.
Clean up vga(4) too.
Sascha Wildner [Thu, 10 Jul 2014 06:48:06 +0000 (08:48 +0200)]
kernel/vga: Remove unused code.
Matthew Dillon [Sat, 15 Aug 2015 03:29:32 +0000 (20:29 -0700)]
patch - Fix shell injection vulnerability
* Fix shell injection vulnerability in patch(1) via ed(1) by
tightening sanity check of the input. [1]
* While I'm there also replace ed(1) with red(1) because we do
not need the unrestricted functionality.
Obtained from: Bitrig [1], and discussions w/ FreeBSD
Security: CVE-2015-1418 [1]
Sascha Wildner [Thu, 13 Aug 2015 16:15:56 +0000 (18:15 +0200)]
Sync zoneinfo database with tzdata2015f from ftp://ftp.iana.org/tz/releases
* asia: North Korea switches to +0830 on 2015-08-15. (Thanks to
Steffen Thorsen.) The abbreviation remains "KST". (Thanks to
Robert Elz.)
* europe: Moldova starts and ends DST at 00:00 UTC, not at 01:00 UTC.
(Thanks to Roman Tudos.) Also mention that Herbert Samuel
introduced the term "Summer Time".
* northamerica: Comments for America/Halifax and America/Glace_Bay have
been improved. (Thanks to Brian Inglis.)
* southamerica: Uruguay no longer observes DST. (Thanks to
Steffen Thorsen and Pablo Camargo.)
* Theory: The Theory file mentions naming issues earlier, as these seem
to be poorly publicized (thanks to Gilmore Davidson for reporting
the problem).
* Various: Data entries have been simplified for Atlantic/Canary,
Europe/Simferopol, Europe/Sofia, and Europe/Tallinn. This yields
slightly smaller installed data files for Europe/Simferopol and
Europe/Tallinn. It does not affect timestamps. (Thanks to
Howard Hinnant.)
John Marino [Thu, 13 Aug 2015 12:05:51 +0000 (14:05 +0200)]
gnu man(1): Recognize locales with Scripts in name
Our GNU man considered locales out of the form of xx_YY (where x is
anything except "_" and Y isn't checked) to be invalid.
However, we have valid locales like sr_Cyrl_RS which were being
rejected. Adjust the validation code to handle these names.
It also appears that man requires a codeset to be specified, e.g.
fr_FR would be ignored but fr_FR.UTF-8 would be accepted. I didn't
do anything about this.
John Marino [Thu, 13 Aug 2015 10:57:40 +0000 (12:57 +0200)]
Regenerate locales (common.UTF-8 => xx_Comm_US.UTF-8)
A rename is required because some applications check the format of the
locale name, so we need to use a legal one (e.g. xx_Yyyy_ZZ.<encoding>)
John Marino [Thu, 13 Aug 2015 10:55:50 +0000 (12:55 +0200)]
cldr2def: Change locale from common.UTF-8 to xx_Comm_US.UTF-8
Some applications check the format of the locale so we need to stick
to the xx_Yyyy_ZZ format.
Sepherosa Ziehau [Thu, 13 Aug 2015 01:37:05 +0000 (09:37 +0800)]
inet6: Change scope to zone and use in6_clearscope() whenever possible
Obtained-from: KAME via FreeBSD
Sepherosa Ziehau [Thu, 13 Aug 2015 01:30:23 +0000 (09:30 +0800)]
socket: Fix soabort() race if crfree() blocks in soinherit().
In soinherit(), if the accepted socket was on the temp list
and crfree() blocked, the owner netisr's soabort() would cause
use-after-free.
After this commit, we move one accepted socket each time, so
that the potential blocking crfree() will not cause trouble
for us. And we no longer allow inheritting incomplete queue,
which could be dropped by definition.
Pointed-out-by: dillon@
John Marino [Thu, 13 Aug 2015 00:02:13 +0000 (02:02 +0200)]
locales: Add "common.UTF-8" locale
See previous commit message. This locale is en_US.UTF-8 with a new
LC_CTYPE that covers all common charactersets (not the full UTF-8, but
a very good chunk of it). This LC_CTYPE source file is manually
created, unfortunately, so it can slowly rot if not updated.
Other locales can use the versatile support of UTF-8 by setting
LC_CTYPE to "common.UTF-8" in the environment.
John Marino [Wed, 12 Aug 2015 23:54:31 +0000 (01:54 +0200)]
cldr2def: Modify tool to create a "common" UTF-8 locale
With the switch to CLDR definitions, each locale has specific encoding.
E.g. en_US, fr_FR has latin characters, el_GR has Greek characters,
etc. Seeing a filename written in Chinese characters while in en_US.UTF-8
will result in a bunch of "???" because the characters are illegal for
the locale's CTYPE definition.
Previously there was one shared CTYPE for all locales. This was handy
but incorrect. It was also not updated and became slightly bitrotted.
I am going to create a "fake" locale called "common.UTF-8". It is the
same as en_US.UTF-8, but with an LC_CTYPE that is practically universal
in that all common character sets (and symbols) are defined. To use it,
one can do either:
1) Set LANG env. variable to "common.UTF-8"
2) Set LC_CTYPE env. variable to "common.UTF-8"
e.g. "env LANG=fr_FR.UTF-8 LC_CTYPE=common.UTF-8 ls"
This commit only updates the tool. The locale comes later.
In addition to making changes that support the "common" base, the tool
was also tweaked to favor en_US.UTF-8 as the "LOCALES+=" value. This
should reduce a lot of the renaming churn we've been seeing on new
generations.
Sascha Wildner [Wed, 12 Aug 2015 21:48:11 +0000 (23:48 +0200)]
i386 removal, part 34/x: Remove some i386 traces from the kernel.
Sascha Wildner [Wed, 12 Aug 2015 17:20:06 +0000 (19:20 +0200)]
kernel/acpica: Adjust comments in the OS Services Layer.
Remove (outdated) chapter numbers and give precise chapter names, in
order to better find our way around the "ACPICA User Guide and
Programmer Reference".
Sascha Wildner [Wed, 12 Aug 2015 17:13:05 +0000 (19:13 +0200)]
kernel/orm: Use bus_alloc_resource_any() and improve description.
Taken-from: FreeBSD
Sascha Wildner [Wed, 12 Aug 2015 14:56:41 +0000 (16:56 +0200)]
Move lnc(4) manual page to the right place.
Sascha Wildner [Wed, 12 Aug 2015 14:56:22 +0000 (16:56 +0200)]
mdoc.local: Release engineering. Allow .Dx 4.3.
Sascha Wildner [Wed, 12 Aug 2015 11:16:51 +0000 (13:16 +0200)]
i386 removal, part 33/x: Remove LongRun and GLXSB support.
LongRun was the Transmeta Crusoe's power saving modes, similar to
Intel SpeedStep.
GLXSB was the AMD Geode LX's crypto accelerator.
Sepherosa Ziehau [Mon, 10 Aug 2015 13:34:20 +0000 (21:34 +0800)]
inet6: Accept non-neighbor NA if nd6_onlink_ns_rfc4861 is off.
Reported-by: Markus Pfeiffer
John Marino [Tue, 11 Aug 2015 20:30:57 +0000 (22:30 +0200)]
locales: stop creating @euro alias for 8 locales
These (dangling symlinks) were left to be created by mistake and then
removed by "make upgrade". Stop creating them.
Sascha Wildner [Tue, 11 Aug 2015 17:53:46 +0000 (19:53 +0200)]
i386 removal, part 32/x: Update getconf(1) manpage for platform removal.
Sascha Wildner [Tue, 11 Aug 2015 17:39:08 +0000 (19:39 +0200)]
i386 removal, part 31/x: Remove i386 definitions from getconf(1).
Remove some other platforms too and error if the platform is not
explicitly handled in progenv.gperf.
Sascha Wildner [Tue, 11 Aug 2015 17:33:29 +0000 (19:33 +0200)]
getconf(1): Sync with FreeBSD and bump WARNS to 6.
Sascha Wildner [Tue, 11 Aug 2015 17:06:37 +0000 (19:06 +0200)]
getconf(1): Remove hand-rolled intmax_t types.
Sascha Wildner [Tue, 11 Aug 2015 16:19:08 +0000 (18:19 +0200)]
kernel/syscons: Set kern.syscons_async default to 0.
Also document it.
Approved-by: dillon
Matthew Dillon [Tue, 11 Aug 2015 02:08:00 +0000 (19:08 -0700)]
kernel - Reduce slab allocator fragmentation
* Restores the intent of the original z_Next test removed by the last
commit and adjusts the related code comments. This allows fully free
zones at the head to be moved to the free list as long as other slabs
are present for the chunking.
* Switch the zone management from LIST to TAILQ so we can manipulate the
tail of the list.
* Define the head of a zone list as being more 'mature' zones, potentially
freeable by any code which tends to cycle allocations. The tail of the
zone list is less mature and subject to reuse more quickly.
- The allocator allocates from the tail (least mature).
- Fully free zones are moved to the head (most mature).
- First free of a fully allocated zone relists the zone at the head
(the zone is considered mature).
- Additional frees do not move the zone.
* TODO - We could also possibly shift the zone within the list based on
NFree vs the NFree of adjacent zones, in order to heuristically allocate
from the least-free zones and give the most-free zones a better chance
to become fully free.
Reported-by: Adrian Drzewiecki <z@drze.net>
Sascha Wildner [Mon, 10 Aug 2015 18:46:21 +0000 (20:46 +0200)]
kernel/sysctl: Rename SYSCTL_ASSERT_XLOCKED -> SYSCTL_ASSERT_LOCKED.
6f1a15dc79a822710cc37e99f6a8bd9910e5e3f1 added shared locking and the
macro is intended to cover both cases, so use a better name.
Sascha Wildner [Mon, 10 Aug 2015 18:42:41 +0000 (20:42 +0200)]
kdump(1): Adjust comment in mkioctls script.
Matthew Dillon [Mon, 10 Aug 2015 18:10:45 +0000 (11:10 -0700)]
kernel - Remove unused z_Next field, fix comments & debug helper
* The kernel slab allocator no longer uses the z_Next field, remove it.
* Remove a useless z_Next test in two places and adjust the comments to
describe the actual operation of the zone free case. It doesn't hurt for
us to leave one fully free zone structure on the main per-cpu ZoneAry[]
for each zone, there might even be a cache-locality-of-reference advantage.
* Fix the 'zoneinfo' code in test/debug so it properly reports the
kernel slab allocators topology.
Reported-by: Adrian Drzewiecki <z@drze.net>
Sascha Wildner [Mon, 10 Aug 2015 17:43:04 +0000 (19:43 +0200)]
kdump(1): Exclude more symlinks in the ioctl search.
Sascha Wildner [Mon, 10 Aug 2015 17:42:25 +0000 (19:42 +0200)]
ioctl.9: Add procfs subsystem ID.
Sascha Wildner [Mon, 10 Aug 2015 16:42:42 +0000 (18:42 +0200)]
ioctl.9: Expand the list of subsystem IDs a bit.
Sascha Wildner [Mon, 10 Aug 2015 16:42:22 +0000 (18:42 +0200)]
kdump(1): Catch mixer ioctls too.
Sascha Wildner [Mon, 10 Aug 2015 16:15:13 +0000 (18:15 +0200)]
kdump(1): Add test for duplicated ioctl cmd values to the Makefile.
'make test' tries to build kdump but runs the mkioctls script with -s
which generates a switch() that lets the compiler perform this test.
John Marino [Mon, 10 Aug 2015 15:48:40 +0000 (17:48 +0200)]
locales polish: Remove ISO-8859-1 encoding from 27 locales
The ISO-8859-1 encoding is considered obsolete compared to ISO-8859-19
(which in itself is obsolete compared to UTF-8). To help avoid some
potential mismatches, remove the ISO-8859-1 encoding in the following
cases:
1) European localities that already have ISO-8859-15 (25 of 27)
* ca_(AD|ES|FR|IT)
* da_dk
* de_(AT|CH|DE)
* en_GB
* (es|eu)_ES
* fi_FI
* fr_(BE|CH|FR)
* is_IS
* it_(CH|IT)
* (nb|nn)_NO
* nl_(BE|NL)
* pt_PT
* sv_(FI|SE)
2) Localities where ISO-8859-1 can't represent local currency symbol
* en_PH (PH Peso)
* es_CR (Colon)
John Marino [Mon, 10 Aug 2015 14:13:22 +0000 (16:13 +0200)]
locales polish: Remove ISO8859-15 from 8 non-European places
Let's limit usage of the "obsolete" ISO-8859-15 to countries in
Europe. Users of the eight locales affected should use UTF-8 instead.
John Marino [Mon, 10 Aug 2015 12:41:57 +0000 (14:41 +0200)]
locale polishing: pl_PL, sk_SK, sl_SI, tr_TR, uk_UA, sv_FI changes
pl_PL: Change shortname alias from ISO8859-2 to UTF-8. The former can not
show the Polish currency symbol.
sk_SK: Same as pl_PL, but for the Euro symbol
sl_SI: Same as pl_PL, but for the Euro Symbol
tr_TR: Switch shortname to UTF-8 to handle currency symbol
uk_UA: Same as tr_TR
sv_FI: Switch shortname to ISO-15 for consistency in Europe
John Marino [Mon, 10 Aug 2015 11:55:39 +0000 (13:55 +0200)]
locale polishing: lt_LT, et_EE, and en_IE changes
lt_LT: Remove ISO8859-4. If an ISO encoding is needed, -13 would be
used instead.
et_EE: Estonia could use ISO8859-4 (not present), the newer ISO8859-13
(not present) or ISO8859-15 (present). To be uniform with other
European countries that use -15, change the default from UTF-8
to ISO8859-15. Do not bring in -4 or -13.
en_IE: Ireland only had UTF-8. To be uniform with the UK and Western
Europe, bring in ISO8859-15 and set that as the alias for the
shortname. Do not bring in -1.
Sascha Wildner [Mon, 10 Aug 2015 13:17:51 +0000 (15:17 +0200)]
kernel/drm: Fix definition of the DRM_IOCTL_I915_GET_SPRITE_COLORKEY ioctl.
This is more or less cosmetic, but it helps eliminate duplicates in
kdump(1)'s ioctl list.
Linux did the same in
2c60fae1489c70206e66c28d72b69a3e496c313d.
Antonio Huete Jimenez [Mon, 10 Aug 2015 12:12:35 +0000 (05:12 -0700)]
vkernel - Fix build
John Marino [Mon, 10 Aug 2015 10:35:09 +0000 (12:35 +0200)]
locales: Switch several shortnames away from ISO-8859-1
For all western European countries that have their short names aliased
to ISO-8859-1, this is against POLA. Once the decision to not use UTF-8
is made, the short names should be linked to ISO-8859-15, the one with
the Euro symbol and other common characters that ISO-8859-1 lacks.
While here, change en_PH to UTF-8 since the Peso sign can't be represented
and do the same for Costa Rica and it's Colon currency.
John Marino [Mon, 10 Aug 2015 09:44:49 +0000 (11:44 +0200)]
locales: Remove new ISO-8859-15@euro symlinks
We're in the process of thinking hard about locale symlinks. These
new ISO-8859-15@euro symlinks (to their ISO8859-15 counterparts) were
a bad idea that I copied from Linuxland. Let's reverse this before it
gets released in DragonFly.
It doesn't even make sense. @euro is basically shorthand for ISO8859-15,
so what value does it have to "modify" that encoding? Stop the Linuxanity
now ...
Sascha Wildner [Sun, 9 Aug 2015 16:31:58 +0000 (18:31 +0200)]
kdump(1): Adjust to display DRM ioctl names in kdump output.
We have to pass ${.CURDIR} to the mkioctls script now too so it can get
at the DRM headers which are not installed.
Sascha Wildner [Sun, 9 Aug 2015 14:53:05 +0000 (16:53 +0200)]
tree.3: Document new tree.3 macros.
Fixes incomplete commit
61ec010eb93d253cf0f3d9b932cb520c9cbcc560.
Sepherosa Ziehau [Sun, 9 Aug 2015 14:31:20 +0000 (22:31 +0800)]
inet6: Revoke nd6_ifinfo.receivedra
Obtained-from: KAME via FreeBSD
Sepherosa Ziehau [Sun, 9 Aug 2015 14:23:02 +0000 (22:23 +0800)]
inet6: Fix linkmtu handling
While I'm here, minor white space cleanup and remove unused code.
Obtained-from: KAME via FreeBSD
Sepherosa Ziehau [Sun, 9 Aug 2015 12:15:45 +0000 (20:15 +0800)]
inet6: Remove CTL_IPV6PROTO_NAMES
Obtained-from: FreeBSD
John Marino [Sun, 9 Aug 2015 14:37:03 +0000 (16:37 +0200)]
<sys/tree.h>: Add RB_FOREACH_* macros from FreeBSD
At least one port is using RB_FOREACH* from sys/tree.h (x11-wm/spectrwm)
John Marino [Sun, 9 Aug 2015 14:17:36 +0000 (16:17 +0200)]
gcc50: Fix C99 regression for c++ cstdio, etc.
Sepherosa Ziehau [Sat, 8 Aug 2015 15:40:04 +0000 (23:40 +0800)]
inet6: Cosmetic clean up
No functional changes.
Obtained-from: KAME via FreeBSD
François Tigeot [Sun, 9 Aug 2015 10:32:43 +0000 (12:32 +0200)]
drm/linux: Add printk prefixes
François Tigeot [Sun, 9 Aug 2015 09:17:09 +0000 (11:17 +0200)]
drm/linux: Add get_jiffies_64()
John Marino [Sun, 9 Aug 2015 10:30:01 +0000 (12:30 +0200)]
<regex.h>: Pull in <xlocale.h> for xlocale_t definition
François Tigeot [Sun, 9 Aug 2015 08:18:33 +0000 (10:18 +0200)]
drm/linux: Add min3()
Sascha Wildner [Sun, 9 Aug 2015 08:17:45 +0000 (10:17 +0200)]
kernel/vmx: Fix sysctl types and descriptions.
zrj [Fri, 7 Aug 2015 05:31:18 +0000 (08:31 +0300)]
drm/radeon: Partial update to Linux 3.12
This brings new radeon features (mostly code refactoring):
DMA functions seperated from asics code;
UVD block separation;
Improvements to DPMS (by default disabled);
Better audio handling.
Notes:
drm/ttm layer is put into transitional state pending memory functions rewrite.
Currently VRAM handling capabilities are limited,
still enough for light gaming and watching videos.
X acceleration on radeon SI cards requires updated xorg+mesa(glamour support).
zrj [Fri, 7 Aug 2015 05:30:16 +0000 (08:30 +0300)]
drm: Implement order_base_2(n)
Matthew Dillon [Sun, 9 Aug 2015 01:44:25 +0000 (18:44 -0700)]
atmel_mxt - Issue RESET in Acer C720P touch-screen driver
* Sometimes the hardware appears to lose it, resulting in non-deterministic
reporting of phantom fixed coordinate presses and movement which messes
up X's pointer position. This problem can survive a reboot.
* Issue a RESET command to the chipset on attach. This will clear up
any issues that would otherwise survive reboot.
* Add a sysctl, setting debug.atmel_mxt_reset=1 will cause the chipset
to be reset. If the chipset goes nuts on a live system, issuing this
sysctl should clear it up.
Detection of the situation not yet in place, so for now this sysctl must
be issued manually if the problem occurs.
John Marino [Sun, 9 Aug 2015 00:12:35 +0000 (02:12 +0200)]
UTF-8: change upper limit from 0x1FFFFF to 0x10FFFF
In November 2003, RFC 3629 changed the upper bound from 0x7FFFFFFF (up to
6 bytes) to 0x10FFFF in order to match the constraints of UTF-16 encoding.
Last week, 5- and 6- bytes were considered illegal, now we also mark
a large portion of 4-byte sequences as illegal as well.
John Marino [Sat, 8 Aug 2015 23:48:56 +0000 (01:48 +0200)]
gcc50/c++locale: Fix locale regression in dports
When bits/c++locale.h is used, the build failes because locale_t
type is unknown. Include <xlocale.h> to define it and fix the
ports regressions I'm seeing now.
John Marino [Sat, 8 Aug 2015 23:28:47 +0000 (01:28 +0200)]
Make upgrade: Remove /usr/bin/colldef which is no longer built
John Marino [Sat, 8 Aug 2015 22:38:11 +0000 (00:38 +0200)]
remove mklocale(1) from base
This was replaced by localedef. I removed it from the bootstrap, but
it was still building in stage 4. It should have been removed entirely.
John Marino [Sat, 8 Aug 2015 22:21:39 +0000 (00:21 +0200)]
Change kk_KZ locale links to kk_Cyrl_KZ
This locale changed names but I forgot to also change the short
locale name symlinks (kk_KZ => kk_Cyrl_KZ)
John Marino [Sat, 8 Aug 2015 22:20:43 +0000 (00:20 +0200)]
Stop removing zh_CH.* locales during upgrade
These are used again as aliases to zh_Hans_CN.*
Tomohiro Kusumi [Sat, 8 Aug 2015 20:17:02 +0000 (05:17 +0900)]
sbin/hammer: Make hammer blockmap print crc on -v
Tomohiro Kusumi [Fri, 7 Aug 2015 20:01:35 +0000 (05:01 +0900)]
sys/vfs/hammer: Cleanups
No functional changes. Using layer1-size-aligned upper
limit makes better sense although it results the same.
Bootstrap code in userspace does this too.
Tomohiro Kusumi [Fri, 7 Aug 2015 18:40:16 +0000 (03:40 +0900)]
sys/vfs/hammer: kprintf invalid header
volume-del'eted volume will show 8 0x00s.
Tomohiro Kusumi [Fri, 7 Aug 2015 16:03:37 +0000 (01:03 +0900)]
sbin/hammer: Optimize freemap initialization
Bring in the way hammer_iterate_l1l2_entries() initializes
layer1/2 to initialize_freemap(). The layer1 crc needs to
be updated only once in every 2^19 layer2 entries. No need
to do it for every layer2 entry as they'll be overwritten
except for the one after the last layer2 (2^19th layer2 entry)
within layer1.
Also having nested loops here for layer1 and layer2 seems
to better explain the concept of hammer's blockmap.
Tomohiro Kusumi [Fri, 7 Aug 2015 15:10:44 +0000 (00:10 +0900)]
sys/vfs/hammer: Cleanup volume format callback
Apply the same code logic as volume free callback.
No need to wait for ready-to-update-layer1-case to increment
total_free_bigblocks via counter variable, as it has nothing
to do with generating layer1 undo.
Tomohiro Kusumi [Thu, 6 Aug 2015 19:20:36 +0000 (04:20 +0900)]
sbin/hammer: Fix inconsistent boot|mem area size
Bring in the code that initializes BootAreaSize and MemAreaSize
from newfs_hammer to hammer volume-add. hammer volume-add uses
static values for these and results having different offset for
freemap to start. This generates filesystem with less space for
data (and inodes, dirents, btree nodes, etc) compared to doing
newfs_hammer against multiple volumes initially.
newfs_hammer + volume-add should results the same as newfs_hammer
for multiple volumes initially, just like the previous commit
fixes wrong big-blocks count.
Example 1.
# newfs_hammer -L TEST /dev/da1 /dev/da2 /dev/da3 > /dev/null
# mount_hammer /dev/da1:/dev/da2:/dev/da3 /HAMMER
# hammer info /HAMMER > a
# umount /HAMMER
Example 2.
# newfs_hammer -L TEST /dev/da1 > /dev/null
# mount_hammer /dev/da1 /HAMMER
# hammer volume-add /dev/da2 /HAMMER
# hammer volume-add /dev/da3 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da2:/dev/da3
# hammer info /HAMMER > b
# umount /HAMMER
# diff -aNur a b
--- a 2015-08-07 04:29:49.
909855000 +0900
+++ b 2015-08-07 04:30:51.
989857000 +0900
<snip>
HAMMER Version 6
Big-block information
- Total 42740
+ Total 42548
Used 2 (0.00%)
Reserved 45 (0.11%)
- Free 42693 (99.89%)
+ Free 42501 (99.89%)
Space information
No. Inodes 1
- Total size 334G (
358529105920 bytes)
+ Total size 332G (
356918493184 bytes)
Used 16M (0.00%)
Reserved 360M (0.11%)
- Free 334G (99.89%)
+ Free 332G (99.89%)
PFS information
PFS ID Mode Snaps Mounted on
0 MASTER 0 /HAMMER
With this commit it results the same as the Example 1.
# newfs_hammer -L TEST /dev/da1 > /dev/null
# mount_hammer /dev/da1 /HAMMER
# hammer volume-add /dev/da2 /HAMMER
# hammer volume-add /dev/da3 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da2:/dev/da3
# hammer info /HAMMER > b
# umount /HAMMER
# diff -aNur a b /* only fsid diff */
--- a 2015-08-07 04:29:49.
909855000 +0900
+++ b 2015-08-07 04:32:39.
349860000 +0900
@@ -1,7 +1,7 @@
<snip>
HAMMER Version 6
Big-block information
Total 42740
Tomohiro Kusumi [Wed, 5 Aug 2015 17:46:58 +0000 (02:46 +0900)]
sys/vfs/hammer: Fix wrong big-block accounting
hammer volume-add|del should not count zone4 big-blocks as
a part of vol0_stat_bigblocks. newfs_hammer excludes zone4
big-blocks from vol0_stat_bigblocks on bootstrapping, then
volume-add|del should exclude either.
As the following example shows, it currently makes difference
on Used big-blocks/bytes between
1. newfs_hammer on multiple volumes
and
2. newfs_hammer on a single volume and then add volumes
which isn't something expected to happen.
Example 1.
# newfs_hammer -L TEST /dev/da1 /dev/da2 /dev/da3 > /dev/null
# mount_hammer /dev/da1:/dev/da2:/dev/da3 /HAMMER
# hammer info /HAMMER | grep Used
Used 2 (0.00%) /* btree + meta */
Used 16M (0.00%)
# umount /HAMMER
Example 2.
# newfs_hammer -L TEST /dev/da1 > /dev/null
# mount_hammer /dev/da1 /HAMMER
# hammer volume-add /dev/da2 /HAMMER
# hammer volume-add /dev/da3 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da2:/dev/da3
# hammer info /HAMMER | grep Used
Used 4 (0.01%) /* btree + meta + 2freemaps */
Used 32M (0.01%)
# umount /HAMMER
With this commit it results the same as the Example 1.
# newfs_hammer -L TEST /dev/da1 > /dev/null
# mount_hammer /dev/da1 /HAMMER
# hammer volume-add /dev/da2 /HAMMER
# hammer volume-add /dev/da3 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da2:/dev/da3
# hammer info /HAMMER | grep Used
Used 2 (0.00%)
Used 16M (0.00%)
# umount /HAMMER
Tomohiro Kusumi [Wed, 5 Aug 2015 14:51:37 +0000 (23:51 +0900)]
sys/vfs/hammer: Add error handling
Tomohiro Kusumi [Wed, 5 Aug 2015 14:48:38 +0000 (23:48 +0900)]
sys/vfs/hammer: Add missing assertion
_hammer_bread() and _hammer_bnew() should be the same
except for the isnew value.
Tomohiro Kusumi [Tue, 4 Aug 2015 20:55:12 +0000 (05:55 +0900)]
sys/vfs/hammer: Don't repeat layer1 blockmap check for no reason
When blockmap allocation/reservation finds no more space in
the current layer1, it jumps up to the next layer1 offset,
however this is meaningless when the next one isn't physically
available (beyond the end of the volume).
It should just jump up to the first layer1 of the next volume.
Otherwise it could repeat "goto again;" for max 1024 times or
somewhere near that unless the volume size is some PB scale.
Repetition of all the sanity checks between "again:" and here
are irrelevant when the layer1 isn't mapped to valid area.
This commit checks for the end of the volume when it finds no
more space in the current layer1, but not every time in the
main path of blockmap allocation/reservation code so the
additional check won't be overhead.
Tomohiro Kusumi [Tue, 4 Aug 2015 10:32:41 +0000 (19:32 +0900)]
sys/vfs/hammer: Get volume lock on volume-list
volume-add|del could be ongoing.
Tomohiro Kusumi [Tue, 4 Aug 2015 10:29:14 +0000 (19:29 +0900)]
sys/vfs/hammer: Make use of volume bitmap
Make use of macros for volume bitmaps that was added in the
previous commit to fix mount issue.
With HAMMER_VOLUME_NUMBER_FOREACH(), hammer_get_volume() is
used only against volumes currently exist, so the function
shouldn't fail but keep null-check anyway.
Tomohiro Kusumi [Mon, 3 Aug 2015 11:01:48 +0000 (20:01 +0900)]
sys/vfs/hammer: Fix mount issue after volume-del
When a volume other than the one with the max volume number
is removed from filesystem by volume-del, volume numbers are
no longer sequential. This makes unable to mount the filesystem
once it's unmounted because code assumes volumes numbers are
>=0 && <=(N-1) when N is the number of volumes. It needs to
be able to handle a hole in volume numbers.
For example, the following filesystem initially consists of
volumes A/B/C where volume A is the root volume(*). Each has
volume# 0/1/2 in its volume header written by newfs_hammer.
volume *A-B-C
volume# 0 1 2
Remove volume B by hammer volume-del. If the filesystem has
no data yet, it never fails.
volume *A-C
volume# 0 2
Unmount filesystem and try to mount again by specifying A:C,
but mount code assumes filesystem that consists of 2 volumes
means volume numbers are (A,C)=(0,1)|(1,0)
but not (A,C)=(0,2)|(2,0)
# newfs_hammer -L TEST /dev/da1 /dev/da2 /dev/da3 > /dev/null
# mount_hammer /dev/da1:/dev/da2:/dev/da3 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da2:/dev/da3
# hammer volume-del /dev/da2 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da3
# umount /HAMMER
# mount_hammer /dev/da1:/dev/da3 /HAMMER
mount_hammer: Unknown error: Invalid argument
# dmesg | tail -1
hammer_mount: Missing volumes, cannot mount!
This commit fixes that.
# newfs_hammer -L TEST /dev/da1 /dev/da2 /dev/da3 > /dev/null
# mount_hammer /dev/da1:/dev/da2:/dev/da3 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da2:/dev/da3
# hammer volume-del /dev/da2 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da3
# umount /HAMMER
# mount_hammer /dev/da1:/dev/da3 /HAMMER
# hammer volume-blkdevs /HAMMER
/dev/da1:/dev/da3
# umount /HAMMER
Note that hammer volume-add doesn't assume volume numbers are
sequential, so it can properly add a volume after removing a
volume with non-max volume number (see hammer_ioc_volume_add()).
Tomohiro Kusumi [Mon, 3 Aug 2015 20:46:52 +0000 (05:46 +0900)]
sys/vfs/hammer: Add hammer_test_free_freemap()
Add a wrapper for l1l2 iteration via test callback
just like other l1l2 iterations do.
Tomohiro Kusumi [Mon, 3 Aug 2015 20:39:14 +0000 (05:39 +0900)]
sys/vfs/hammer: Don't reblock when volume to delete is empty
hammer_ioc_volume_del() needs to reblock filesystem only
when the volume hammer volume-del is trying to remove is
not empty. If it's empty (means the volume's blockmap has
nothing other than layer2 big-block in non-root volume)
it can be removed right away.
The way it always reblocks on hammer volume-del is also
counter intuitive since not everyone expects this command
to do extra effort to delete it when it's not empty, and
force one to wait for reblock to finish.
Tomohiro Kusumi [Mon, 3 Aug 2015 11:00:59 +0000 (20:00 +0900)]
sbin/hammer: Make hammer blockmap print all blockmap offsets on -v
Tomohiro Kusumi [Tue, 4 Aug 2015 10:06:03 +0000 (19:06 +0900)]
sys/vfs/hammer: Add missing comments
Sascha Wildner [Sat, 8 Aug 2015 17:25:32 +0000 (19:25 +0200)]
kernel/netinet6: Change some function definitions to static.
All these were already declared static.
John Marino [Sat, 8 Aug 2015 17:17:44 +0000 (19:17 +0200)]
China locales: Return zh_CN.* as aliases
Right now vi is expecting Chinese locales in the form of zh_CN.
It may not be the only locale catalog like this, so bring back some
zh_CN.* locales as symlinks to their zh_Hans_CN counterparts.
John Marino [Sat, 8 Aug 2015 16:26:44 +0000 (18:26 +0200)]
Remove nvi/ files from contrib (replaced by nvi2/)
John Marino [Sat, 8 Aug 2015 16:20:13 +0000 (18:20 +0200)]
nvi: Bring in version 2.1.3 (update from 2.1.1)
All the headers that are not part of new nvi2 vendor branch have been
moved here. This resolves a major flaw where files written in one
locale and then edited in another would be corrupted.
See FreeBSD PR 196447
John Marino [Sat, 8 Aug 2015 15:23:59 +0000 (17:23 +0200)]
nvi2: Add locale modifications
The following required modification:
- zh_Hans_CN locales
- renaming all missing extern files (name pollution) [1]
- Added missing timespec header inclusion
- change location of missing version.h [2]
- SLIST_FOREACH_SAFE => SLIST_FOREACH_MUTABLE
[1] There were four separate extern.h files, all were missing from vendor
branch. They copied from FreeBSD and put at /usr.bin/vi
[2] The version.h header was also missing from vendor branch, so the
relative path was removed. It is now at /usr.bin/vi
John Marino [Sat, 8 Aug 2015 14:55:28 +0000 (16:55 +0200)]
NVI2: Add DragonFly README files
John Marino [Sat, 8 Aug 2015 16:58:59 +0000 (18:58 +0200)]
Merge branch 'vendor/NVI2'
John Marino [Sat, 8 Aug 2015 16:58:23 +0000 (18:58 +0200)]
Import nvi2 version 2.1.3 to vendor branch