Sascha Wildner [Wed, 9 Jul 2008 20:33:32 +0000 (20:33 +0000)]
Include HAMMER file systems when checking for setuid diffs.
Sascha Wildner [Wed, 9 Jul 2008 19:53:27 +0000 (19:53 +0000)]
Add HAMMER to the list of file systems to be searched by locate.updatedb.
Sascha Wildner [Wed, 9 Jul 2008 18:32:19 +0000 (18:32 +0000)]
Clarify that the argument to -t is in seconds.
Thomas Nikolajsen [Wed, 9 Jul 2008 18:27:30 +0000 (18:27 +0000)]
Fix error: `#' can be escaped by `\', not `\\'.
Obtained-from: FreeBSD
Thomas Nikolajsen [Wed, 9 Jul 2008 18:19:39 +0000 (18:19 +0000)]
Fix typo.
Thomas Nikolajsen [Wed, 9 Jul 2008 18:15:10 +0000 (18:15 +0000)]
Fix typos.
Thomas Nikolajsen [Wed, 9 Jul 2008 17:50:44 +0000 (17:50 +0000)]
Change .Fx to .Dx where appropriate.
Thomas Nikolajsen [Wed, 9 Jul 2008 17:38:31 +0000 (17:38 +0000)]
Fix paths;
paths changed in the past when source tree restructured.
Thomas Nikolajsen [Wed, 9 Jul 2008 16:23:56 +0000 (16:23 +0000)]
In manual pages and LINT,
- Fix paths;
paths changed in past when source tree restructured.
- Change .Fx/FreeBSD to .Dx/DragonFly where appropriate.
Thomas Nikolajsen [Wed, 9 Jul 2008 15:51:43 +0000 (15:51 +0000)]
Change name of kernel tunable from imtime to imtimer,
to match name of corresponding sysctl variable.
Update nfe.4 with support for MCP73/77/79, and
add description of sysctl variables / tunables.
Sepherosa Ziehau [Wed, 9 Jul 2008 15:45:23 +0000 (15:45 +0000)]
Update et(4) manpage after recent change
Sepherosa Ziehau [Wed, 9 Jul 2008 15:42:12 +0000 (15:42 +0000)]
- Recollect TX descs, if et(4) is going to enter OACTIVE state.
- Add additional parameter to et_txeof(), so that calling et_txeof() in
et_start() will not create recursion.
- Increase TX segments per interrupt to 256 after above changes.
This gives me additional +10Kpps 64bytes packet TX performance improvement
(now 918Kpps) and interrupt rate reduction.
Sepherosa Ziehau [Wed, 9 Jul 2008 14:00:44 +0000 (14:00 +0000)]
Update et(4) manpage according to recent changes
Sepherosa Ziehau [Wed, 9 Jul 2008 13:57:59 +0000 (13:57 +0000)]
- Increase RX coalesced packets from 32 to 129 to avoid being livelocked when
~1.48Mpps 64bytes packets are injected.
- Increase RX coalescing ticks from 20 to 25 and correct the comment about its
unit: it is 4us instead of 10us (thanks go to jsg@openbsd.org for giving me
the informational material :).
# An interrupt is generated if
# (RX pkt count > ET_RX_INTR_NPKTS) || (timer > ET_RX_INTR_DELAY)
#
# I thought the interrupt generation condition was
# (RX pkt count > ET_RX_INTR_NPKTS) && (timer > ET_RX_INTR_DELAY)
Matthew Dillon [Wed, 9 Jul 2008 10:32:30 +0000 (10:32 +0000)]
HAMMER Utilities: Sync with 60I
* Changes in the way PFS's are created and managed.
* Replace the pfs-create directive with pfs-master and pfs-slave.
* Add more checks against stupid user errors.
* A cycle file is no longer needed when using mirror-copy. Mirror-copy
now uses a two-way protocol and figures out what TID ranges need to
be copied.
Matthew Dillon [Wed, 9 Jul 2008 10:29:20 +0000 (10:29 +0000)]
HAMMER 60I/Many: Mirroring
* Flesh out the mirroring code a lot. It is now 90% done.
* Change the way PFS's are managed. The PFS management structure is no
longer associated with the PFS root inode. Instead all PFS management
structures, for all PFSs, are associated with the master root inode.
This allows a mirroring slave to also mirror the root inode itself.
* Remove the directory localization hacks used to link a PFS to its parent.
PFS's are no longer linked to their parent. Instead, a special
@@PFS softlink is created.
* The master/slave mode must now be selected when a PFS is created and
cannot be changed.
* PFSs are accessed via a special @@PFS softlink created in the parent
directory. HAMMER manipulates this special softlink in the readlink()
command and returns a longer form to the kernel which changes whenever
the snapshot TID changes.
This allows updates to slave PFSs by mirroring commands to not create
stale system caches. Every time you CD through the special softlink
you get the most recent completed snapshot of the PFS.
* Userland now manipulates PFS ids as a simple integer in the range 0-65535.
Thomas Nikolajsen [Wed, 9 Jul 2008 09:29:17 +0000 (09:29 +0000)]
Add PF_MPLSOPS, route flag `m', to netstat.8.
Add mpls as -f option: `netstat -f mpls'.
Sascha Wildner [Wed, 9 Jul 2008 07:54:34 +0000 (07:54 +0000)]
Add some OLD_PKGSRC_PACKAGES again (accidentally removed in the last commit).
Sascha Wildner [Wed, 9 Jul 2008 07:21:03 +0000 (07:21 +0000)]
Build the installer as part of normal build-/installworld if WANT_INSTALLER
is set.
Clean up nrelease/Makefile to define WANT_INSTALLER if the 'installer'
target is specified.
Thomas Nikolajsen [Tue, 8 Jul 2008 15:12:36 +0000 (15:12 +0000)]
Add description for CVS_UPDATE and update description for SUP* variables.
Sepherosa Ziehau [Tue, 8 Jul 2008 14:36:02 +0000 (14:36 +0000)]
Make et(4) aware ETHER_INPUT_CHAIN and ETHER_INPUT2
Thomas Nikolajsen [Tue, 8 Jul 2008 14:12:45 +0000 (14:12 +0000)]
ata(4) isn't used in GENERIC anymore.
Thomas Nikolajsen [Tue, 8 Jul 2008 14:04:11 +0000 (14:04 +0000)]
Update file list and description.
Sepherosa Ziehau [Tue, 8 Jul 2008 13:50:52 +0000 (13:50 +0000)]
Handle MPLS on ETHER_INPUT2 code path
Thomas Nikolajsen [Tue, 8 Jul 2008 12:08:29 +0000 (12:08 +0000)]
Update names of rc.d diskless scripts.
Thomas Nikolajsen [Tue, 8 Jul 2008 11:59:15 +0000 (11:59 +0000)]
Add description of DragonFly-src-supfile.
Expand TABs.
Matthew Dillon [Tue, 8 Jul 2008 04:34:41 +0000 (04:34 +0000)]
HAMMER 60H/Many: Stabilization pass
* Fix races in the in-memory/B-Tree scanning code introduced by the
mirror_tid update code. This fixes two assertion panics.
* Remove compile-time reminder #warning that no longer applies.
(Sascha will be happy).
Matthew Dillon [Tue, 8 Jul 2008 03:34:27 +0000 (03:34 +0000)]
Cleanup - move a warning so it doesn't spam the screen so much, cleanup
some syntax.
Matthew Dillon [Tue, 8 Jul 2008 01:39:27 +0000 (01:39 +0000)]
Add the HAMMER filesystem to GENERIC and VKERNEL.
Nuno Antunes [Mon, 7 Jul 2008 23:11:54 +0000 (23:11 +0000)]
Add 'options MPLS' to LINT.
Remove comment.
Sascha Wildner [Mon, 7 Jul 2008 23:03:25 +0000 (23:03 +0000)]
The behavior of -m introduced in r1.4 leads to an error message upon
booting if (for example) a laptop with a mouse pad is booted with an
additional USB mouse attached and both moused_enable and usbd_enable
are set in rc.conf.
Therefore, go back to the old behavior again, which is to allow
enabling the mouse cursor more than once.
Matthew Dillon [Mon, 7 Jul 2008 22:42:35 +0000 (22:42 +0000)]
HAMMER 60G/Many: Mirroring, bug fixes
* Fix a bug in the mirror_tid propagation that could cause a delete_range
iteration to get way way off. B-Tree insertions were not calling
hammer_cursor_inserted_element().
* Fix an assertion in the delete_range iteration, delete-visibility could
cause the assert to trigger when it shouldn't.
* Add a new assertion testing that the iteration is still within the
inode object to catch any remaining mirror_tid propagation bugs.
* Minor code documentation adjustments.
Reported-by: Gergo Szakal <bastyaelvtars@gmail.com>
Nuno Antunes [Mon, 7 Jul 2008 22:02:10 +0000 (22:02 +0000)]
Introduce experimental MPLS over ethernet support. Add 'options MPLS'
to the kernel config file to enable it. This modification increases
the footprint of each route in the FIB by 12 bytes, used to hold up
to 3 label operations per route.
Hints-from: Ayame, NiSTswitch implementations.
Reviewed-by: dillon@, sephe@, hsu@, hasso@.
Matthew Dillon [Mon, 7 Jul 2008 17:31:07 +0000 (17:31 +0000)]
UFS+softupdates can build up thousands of dirty 1K buffers and run out
of buffers before it even hits the lodirtybufspace point. The buf_daemon
is never triggered. This case occurs rarely but can be triggered e.g.
by a cvs update.
Add dirtybufcount back in and flush if it exceeds (nbuf / 2) to handle
this degenerate case.
Reported-by: "Sepherosa Ziehau" <sepherosa@gmail.com>
Matthew Dillon [Mon, 7 Jul 2008 16:02:36 +0000 (16:02 +0000)]
Rename fid_reserved to fid_ext.
Sascha Wildner [Mon, 7 Jul 2008 15:29:51 +0000 (15:29 +0000)]
Sync zoneinfo database with tzdata2008d from elsie.
africa: 8.11 -> 8.13
asia: 8.20 -> 8.21
australasia: 8.8 -> 8.9
europe: 8.12 -> 8.14
northamerica: 8.23 -> 8.24
southamerica: 8.19 -> 8.26
zone.tab: 8.16 -> 8.18
From Arthur David Olson's and Paul Eggert's comments:
Changes affecting current and future time stamps:
* africa: changes to reflect Mauritius' 2008-2009 DST experiment
* southamerica and zone.tab: changes to reflect Brazilian time zone
realignments
* America/Eirunepe also changed from ACT (UTC-5) to AMT (UTC-4) on June 24
Changes affecting past time stamps:
* For Mauritius, also include the 1982/1983 DST as reported by
Steffen Thorsen
* For America/Argentina/San_Luis, redo the history from 1990 through 2000
to match what's in Shanks & Pottenger
The following changes should not affect externally visible behavior:
* australasia, europe and southamerica: changes by Halasz Sandor Viktor to
fix URLs
* Redo comments about Brazil to make it clearer about what west Para contains
* Update Brazil comments to mention Decree 6,212 (2007-09-26)
* Make comments a bit more concise for Brazil and Mauritius
* Redo comments about Argentina to reflect the fact that San Luis is now
separate
* Redo the daylight-saving rules for Berlin in 1945 to make it clearer that
these are for the Soviet Zone, not for Berlin in general
* Redo Bahamas rules to make it clearer that the rules after 1975 are the
same as the US
* Add comments about time in China from 1949 to 1980
(Thanks to Thomas S. Mullaney for this.)
Aggelos Economopoulos [Mon, 7 Jul 2008 14:35:12 +0000 (14:35 +0000)]
Do copy{in,out} early in the {g,s}etsockopt system call paths
so the rest of the kernel can assume it's dealing with a
kernel address in ->sopt_val. This means that soopt{,_m}copy{in,out}
are now seriously misnamed and should be replaced by soopt_{from,to}_*
Suggested-by: dillon@
Sepherosa Ziehau [Mon, 7 Jul 2008 13:14:23 +0000 (13:14 +0000)]
IFF_BROADCAST will never be changed
Sepherosa Ziehau [Mon, 7 Jul 2008 11:31:28 +0000 (11:31 +0000)]
White space
Sepherosa Ziehau [Mon, 7 Jul 2008 11:29:28 +0000 (11:29 +0000)]
The MAC address of nfe(4), whose device id is above MCP55, is saved in
different order than older ones. Model numbers from forthdeth.
Reported-by: Michael Neumann <mneumann@ntecs.de>
Matthew Dillon [Mon, 7 Jul 2008 03:51:29 +0000 (03:51 +0000)]
HAMMER Utilities: Sync with 60F
* Use sync_end_tid as the snapshot TID rather then sync_beg_tid.
* Initialize sync_beg_tid to 0 instead of 1 in PFS 0's pfsd structure.
* Reported the random fsid assigned by newfs_hammer to a new HAMMER
filesystem.
Matthew Dillon [Mon, 7 Jul 2008 03:49:51 +0000 (03:49 +0000)]
HAMMER 60F/Many: Mirroring
* Properly propagate mirror_tid when splitting B-Tree nodes.
* Arrange for the fsid reported as st_dev to be based on the shared_uuid,
so the same value is reported on the slaves as on the master.
Munge the fsid based on the asof timestamp to try to make snapshots look
different from current filesystems, so programs like 'diff' don't get
confused.
This allows one to run (tar cf - directory@@<timestamp> | md5) on a
slave and get the same answer as he got on the master. Note, however,
that the tar cannot include the root directory of the PFS or master
because the root directory inode is not mirrored (only files and directories
underneath it), and will mess up the MD5.
* Properly update vol0_next_tid when writing to a mirroring slave.
* Attempt to regenerate the inode for the root PFS when doing a name lookup
based on the current latest snapshot. This may need more work.
Matthew Dillon [Mon, 7 Jul 2008 01:29:32 +0000 (01:29 +0000)]
When creating a new HAMMER filesystem also create a PFS record for it,
setting the unique and shared uuids to the fsid in the volume header.
Matthew Dillon [Mon, 7 Jul 2008 00:27:22 +0000 (00:27 +0000)]
HAMMER Utilities: Sync with 60E
* Change the cycle file to hold an entire B-Tree base key, plus an optional
TID (used by the mirroring code).
* Flesh out the mirroring code. Add timeout (-t) support. Add cycle file
support.
* When mirror-copy is used have the target sync the filesystem
and acknowledge completion and store the completed TID in the cycle
file.
* Incremental mirroring now works when using mirror-copy with a cycle file.
* Add mirror-dump, aka hammer mirror-read ... | hammer mirror-dump, for
debugging.
Matthew Dillon [Mon, 7 Jul 2008 00:24:31 +0000 (00:24 +0000)]
HAMMER 60E/Many: Mirroring, bug fixes
* Work on the mirror_tid propagation code. The code now retries on
EDEADLK so propagation is guaranteed to reach the root.
* Get most of the mirror_write code working.
* Add PFS support for NFS exports. Change fid_reserved to fid_ext and use
it to store the localization parameter that selects the PFS. This isn't
well tested yet.
* BUGFIX: Fix a bug in vol0_last_tid updates. Flush sequences might
not always update the field, creating issues with mirroring and snapshots.
* BUGFIX: Properly update the volume header CRC.
* CLEANUP: Fix some obj_id's that were u_int64_t's. They should be int64_t's.
* CLEANUP: #if 0 out unused code, remove other bits of unused code.
Thomas Nikolajsen [Mon, 7 Jul 2008 00:00:56 +0000 (00:00 +0000)]
Ensure AP is associated before we return.
Otherwise further rc.d processing, eg dhclient,
can fail to get working network connection.
Thomas Nikolajsen [Sun, 6 Jul 2008 23:56:32 +0000 (23:56 +0000)]
Fix typo.
Thomas Nikolajsen [Sun, 6 Jul 2008 23:55:51 +0000 (23:55 +0000)]
Describe which FW package is causing error;
we have multiple FW packages in base system.
Thomas Nikolajsen [Sun, 6 Jul 2008 23:50:19 +0000 (23:50 +0000)]
Update etc/inetd.conf:
- uucp removed from base system 5 years ago
- kerberosIV removed from base system 4 years ago
- pcnfsd removed from base system 14 years ago
- change path in examples from /usr/local/ to /usr/pkg/
as we changed from ports to pkgsrc several years ago
Nicolas Thery [Sun, 6 Jul 2008 19:01:57 +0000 (19:01 +0000)]
Change soopt_mcopyin() and soopt_mcopyout() to not modify soopt_val and
soopt_valsize. This fixes a "already free" panic when so_pru_ctloutput()
kfree()'s soopt_val.
Reported-by: Thomas Nikolajsen <thomas.nikolajsen@mail.dk>
Reviewed-by: Aggelos Economopoulos <aoiko@cc.ece.ntua.gr>
Michael Neumann [Sun, 6 Jul 2008 17:20:16 +0000 (17:20 +0000)]
Add Makefile that compiles serial port enabled boot loader.
Michael Neumann [Sun, 6 Jul 2008 17:17:01 +0000 (17:17 +0000)]
Extend boot loader so that it can be compiled to work over a serial port.
Obtained-From: FreeBSD
Sepherosa Ziehau [Sun, 6 Jul 2008 09:17:12 +0000 (09:17 +0000)]
Nuke unused macro and comment
Sepherosa Ziehau [Sun, 6 Jul 2008 08:46:33 +0000 (08:46 +0000)]
Print chip informantion in bge_attach() instead of bge_probe()
Sepherosa Ziehau [Sun, 6 Jul 2008 08:13:46 +0000 (08:13 +0000)]
Adjust default interrupt moderation variables
Sepherosa Ziehau [Sun, 6 Jul 2008 06:47:10 +0000 (06:47 +0000)]
For PCI Express devices:
1) Adjust max read request size to 4Kbytes
2) Turn on FIFO_LONG_BURST in RDMA during bge_blockinit()
Though 1) does not seem to have much to do with the poor TX performance (*)
observed on PCI Express bge(4), 2) does fix the problem.
Obtained-from: tg3
# (*)
# http://lists.freebsd.org/pipermail/freebsd-net/2007-June/014373.html
# FreeBSD kern/119361
Sepherosa Ziehau [Sun, 6 Jul 2008 05:54:09 +0000 (05:54 +0000)]
TX coalesce ticks can't be more than 1023
Obtained-from: tg3
Sepherosa Ziehau [Sun, 6 Jul 2008 05:47:14 +0000 (05:47 +0000)]
Add bits for PCI Express device control register
Matthew Dillon [Sat, 5 Jul 2008 23:59:38 +0000 (23:59 +0000)]
Error out if no volumes are specified instead of core-dumping.
Submitted-by: Antonio Huete Jimenez <tuxillo@quantumachine.net>
Matthew Dillon [Sat, 5 Jul 2008 18:59:28 +0000 (18:59 +0000)]
HAMMER 60D/Many: Mirroring, bug fixes
* Add more mirroring infrastructure.
* Fix support for unix domain sockets.
Reported-by: Gergo Szakal <bastyaelvtars@gmail.com>,
Rumko <rumcic@gmail.com>
Sepherosa Ziehau [Sat, 5 Jul 2008 13:37:17 +0000 (13:37 +0000)]
If boot verbose, print asicrev, chiprev and bus type.
Sepherosa Ziehau [Sat, 5 Jul 2008 12:22:37 +0000 (12:22 +0000)]
- Add delay in MII registers reading
- Minor style changes
Sepherosa Ziehau [Sat, 5 Jul 2008 07:29:44 +0000 (07:29 +0000)]
Use hardware timer to simulate interrupt moderation.
hw.nfeX.imtimer -> 0 no interrupt moderation at all.
hw.nfeX.imtimer -> -Y hardware timer simulated interrupt moderation,
the simultated interrupt moderation timer will
be set to Y.
hw.nfeX.imtimer -> Y hardware timer only interrupt moderation,
hardware timer will be set to Y.
Set hw.nfeX.imtimer to -125 (8000Hz) should work on most system.
With heavier interrupt moderation (e.g. hw.nfeX.imtimer to -250), it is
recommended to increase hw.nfe.rx_ring_count tunable to a higher value
(e.g. 512), since the default value (128) is obviously too small.
Sepherosa Ziehau [Sat, 5 Jul 2008 05:34:31 +0000 (05:34 +0000)]
Split hardware capabilities and softc flags
Sepherosa Ziehau [Sat, 5 Jul 2008 05:16:54 +0000 (05:16 +0000)]
First step to split hardware capabilities and softc flags
Sepherosa Ziehau [Sat, 5 Jul 2008 05:02:42 +0000 (05:02 +0000)]
Nuke unused field
Matthew Dillon [Fri, 4 Jul 2008 07:25:36 +0000 (07:25 +0000)]
HAMMER 60C/many: Mirroring
* Clean up the B-Tree mirror_tid update code, rewiring it a bit
to reduce code pollution.
* Properly detect the mirroring mode (master, slave, or no-mirror mode)
via the pfs configuration.
Matthew Dillon [Fri, 4 Jul 2008 07:20:43 +0000 (07:20 +0000)]
HAMMER Utilities: Stabilization
* Fix a bug in mirror-copy that was causing extra process forks to occur.
Sascha Wildner [Thu, 3 Jul 2008 10:42:37 +0000 (10:42 +0000)]
Regenerate the pciconf(8) database from the following files:
Hart: Jan 22, 2008 (version 671)
Boemler: Jul 1, 2008
Mares: Jul 1, 2008
Simon Schubert [Thu, 3 Jul 2008 08:22:36 +0000 (08:22 +0000)]
Make ip_forward public, so that it can be used from other places as well.
Ok'ed: sephe@
Matthew Dillon [Thu, 3 Jul 2008 06:36:27 +0000 (06:36 +0000)]
HAMMER Utilities: Sync with recent work.
* The masterid and slave options are now pseudo-fs options and not general
mount options.
Matthew Dillon [Thu, 3 Jul 2008 04:24:51 +0000 (04:24 +0000)]
HAMMER 60B/many: Stabilization
* Redo the estimated space usage calculation, it was horribly broken.
There may still be issues with read+write memory maps.
* Fix a bug in the reblocker that allowed it to exhaust all available
space on the media. While the reblocker eventually compacts the media
it can actually require additional disk space while running.
Note that file deletions in normal history retention mode do not recover
any space unless followed by pruning, and may even require additional space.
If worse comes to worse a full filesystem might have to be remounted in
nohistory mode to recover space.
Reported-by: Gergo Szakal <bastyaelvtars@gmail.com>
Matthew Dillon [Wed, 2 Jul 2008 22:05:59 +0000 (22:05 +0000)]
HAMMER Utilities: Mirroring and pseudo-fs directives
* Add mirror-read, mirror-write, and mirror-copy.
* Replace the pseudofs directive with pfs-status, pfs-create, and pfs-update.
Create a pseudofs configuration infrastructure. Configuration parameters
can be specified via pfs-create and pfs-update for any HAMMER filesystem or
HAMMER pseudo-filesystem. This is basically how you tell HAMMER that
a particular pfs is a master or slave, and allows both a unique and a shared
uuid to be associated with the pfs. The mirroring code will only mirror
to targets with the same shared uuid as the master sourcing the data.
Matthew Dillon [Wed, 2 Jul 2008 21:57:54 +0000 (21:57 +0000)]
HAMMER 60A/many: Mirroring work
* Add a pseudo-fs structure for tracking pseudo-fs configurations.
* Add pseudo-fs status and configuration ioctls
* Fix a b-tree node deletion race that could result in an
assertion (reported by Rumko <rumcic@gmail.com>)
Simon Schubert [Wed, 2 Jul 2008 05:40:43 +0000 (05:40 +0000)]
Don't try to run newaliases at all. This happens anyways on boot.
Matthew Dillon [Tue, 1 Jul 2008 17:30:42 +0000 (17:30 +0000)]
HAMMER 59J/Many: Features
* The pruning code now disposes of any disconnected inodes it encounters.
A disconnected inode can be created when a process holds an open
descriptor on an unlink()'d file, and the machine then crashes.
Matthew Dillon [Tue, 1 Jul 2008 16:48:51 +0000 (16:48 +0000)]
HAMMER 59I/Many: Stabilization pass
* Move calls to hammer_flusher_clean_loose_ios() to handle more cases.
This fixes a kmalloc memory blowout that can occur when removing
a 1.6TB file.
Hasso Tepper [Tue, 1 Jul 2008 08:21:25 +0000 (08:21 +0000)]
Fix [gs]etsockopt(IP_HDRINCL) which allows mere mortals like me to obtain
IP addresses via DHCP again.
Matthew Dillon [Tue, 1 Jul 2008 04:54:25 +0000 (04:54 +0000)]
HAMMER 59H/Many: Stabilization pass
* Fix a bug introduced in 59G that will corrupt the filesystem when it
becomes full. A pointer was not being NULL'd out and caused
hammer_blockmap_reserve() to return a bogus offset when the loop
could find no free space.
Matthew Dillon [Tue, 1 Jul 2008 02:08:58 +0000 (02:08 +0000)]
HAMMER 59G/Many: Stabilization pass (low memory issues)
Make adjustments in an attempt to remove deadlocks from the pageout and
buffer flushing paths when a system is low on memory.
* Restructure the low level storage allocator to use layer2->append_off
and to move the blockmap lock deeper into the routine, pass the buffer
ops.
* Do kmalloc()'s of critical structures accessed by the backend with
M_USE_RESERVE.
* Have hammer_vop_write() call bwillwrite() unconditionally, even if
called from the pageout daemon. Do not release or relock the vnode
at all.
Theoretically bwillwrite() in the pageout path should still allow
strategy calls from the buf_daemon threads to proceed, and not deadlock.
Reported-by: Gergo Szakal <bastyaelvtars@gmail.com>
Matthew Dillon [Tue, 1 Jul 2008 02:02:56 +0000 (02:02 +0000)]
Fix numerous pageout daemon -> buffer cache deadlocks in the main system.
These issues usually only occur on systems with small amounts of ram
but it is possible to trigger them on any system.
* Get rid of the IO_NOBWILL hack. Just have the VN device use IO_DIRECT,
which will clean out the buffer on completion of the write.
* Add a timeout argument to vm_wait().
* Add a thread->td_flags flag called TDF_SYSTHREAD. kmalloc()'s made
from designated threads are allowed to dip into the system reserve
when allocating pages. Only the pageout daemon and buf_daemon[_hw] use
the flag.
* Add a new static procedure, recoverbufpages(), which explicitly tries to
free buffers and their backing pages on the clean queue.
* Add a new static procedure, bio_page_alloc(), to do all the nasty work
of allocating a page on behalf of a buffer cache buffer.
This function will call vm_page_alloc() with VM_ALLOC_SYSTEM to allow
it to dip into the system reserve. If the allocation fails this
function will call recoverbufpages() to try to recycle from VM pages
from clean buffer cache buffers, and will then attempt to reallocate
using VM_ALLOC_SYSTEM | VM_ALLOC_INTERRUPT to allow it to dip into
the interrupt reserve as well.
Warnings will blare on the console. If the effort still fails we
sleep for 1/20 of a second and retry. The idea though is for all
the effort above to not result in a failure at the end.
Reported-by: Gergo Szakal <bastyaelvtars@gmail.com>
Hasso Tepper [Mon, 30 Jun 2008 19:04:49 +0000 (19:04 +0000)]
We are using 4.1.2 really.
Hasso Tepper [Mon, 30 Jun 2008 12:16:03 +0000 (12:16 +0000)]
Build objc support.
Hasso Tepper [Mon, 30 Jun 2008 11:22:39 +0000 (11:22 +0000)]
Merge from vendor branch GCC:
Add objc to the gcc-4.1.2.
Hasso Tepper [Mon, 30 Jun 2008 11:22:39 +0000 (11:22 +0000)]
Add objc to the gcc-4.1.2.
Matthew Dillon [Mon, 30 Jun 2008 04:18:30 +0000 (04:18 +0000)]
HAMMER 59F/Many: Stabilization pass
* Use kmalloc_limit() to calculate a better record limit. This fixes an
issue on machines with limited system memory where HAMMER could blow
out its kmalloc pool by allocating too many records.
Matthew Dillon [Mon, 30 Jun 2008 03:57:41 +0000 (03:57 +0000)]
Fix a low-memory deadlock in the VM system which can occur on systems
with small amounts of system memory. The pageout daemon uses LK_TIMELOCK
to avoid deadlocking on a vnode so it can continue operation. This only
works if lk_timo is non-zero, but all filesystems initialize their vnodes
with a timeout of zero so the feature is defeated. Adjust allocvnode()
to guarantee that lk_timo is never zero.
Matthew Dillon [Mon, 30 Jun 2008 03:00:36 +0000 (03:00 +0000)]
Add a new helper function, kmalloc_limit(). This function returns
the number of bytes that can be held in a M_XXX kernel memory pool.
Matthew Dillon [Mon, 30 Jun 2008 02:45:30 +0000 (02:45 +0000)]
HAMMER 59E/Many: Stabilization pass
* Sprinkle around calls to hammer_flusher_clean_loose_ios() to clean up
hammer buffers with no buffer cache buffer associations, so they can't
build up and blow out kernel memory.
* Fix a bug when deleting large files where not all records were being
properly moved to the new flush group.
* Fix a bug when truncating large files where a trunction followed by
write()s followed by another truncation would incorrectly reorder the
writes() in the middle to occur after the second truncation.
Matthew Dillon [Mon, 30 Jun 2008 02:11:53 +0000 (02:11 +0000)]
Fix a buf_daemon performance issue when running on machines with small
amounts of ram. The daemon was hitting a 1/2 sleep case that it should
not have been hitting.
Matthew Dillon [Mon, 30 Jun 2008 00:03:55 +0000 (00:03 +0000)]
HAMMER 59D/Many: Stabilization pass
* Fix a panic that occurs when the crash recovery undo space is larger
then conveniently fits in the buffer cache.
* Fix a bug in the truncation code (detected by fsx). The bug was related
to moving an inode from one flush group to another (which itself was done
to fix a bug related to blowing out the buffer cache).
Simon Schubert [Sun, 29 Jun 2008 22:22:17 +0000 (22:22 +0000)]
Clarify comment.
Matthew Dillon [Sun, 29 Jun 2008 21:38:21 +0000 (21:38 +0000)]
Fix a NULL pointer dereference when a DDB 'ps' attempts to
list a process which has no LWPs associated with it.
Sascha Wildner [Sun, 29 Jun 2008 20:45:44 +0000 (20:45 +0000)]
Fix typo in sln(4) PCI ID.
Sascha Wildner [Sun, 29 Jun 2008 20:43:49 +0000 (20:43 +0000)]
Regenerate
Sascha Wildner [Sun, 29 Jun 2008 20:33:57 +0000 (20:33 +0000)]
Fix typo in sln(4) PCI ID.
Matthew Dillon [Sun, 29 Jun 2008 19:15:34 +0000 (19:15 +0000)]
Fix an issue where CAM would attempt to illegally get a lockmgr() lock
from an interrupt via xpt_done(). Use a spin lock instead.
Fix an issue where camisr_runqueue() was not protecting sim->sim_doneq
which could then race against an interrupt via xpt_done() and corrupt
the list.
Matthew Dillon [Sun, 29 Jun 2008 19:04:02 +0000 (19:04 +0000)]
Adjust comments.
Reported-by: Jordan Gordeev <jgordeev@dir.bg>
Simon Schubert [Sun, 29 Jun 2008 10:34:04 +0000 (10:34 +0000)]
Clean up stale comment and unused umount.
Noticed-by: swildner@
Simon Schubert [Sun, 29 Jun 2008 10:23:23 +0000 (10:23 +0000)]
Don't run binaries in the new built root, but use host binaries instead.
This allows us to do cross-builds (older host kernel) again, which was
broken for some time now.
The mkfontdir command was not changed yet, but this is no regression.