Fix references and .Nm abuse.
[dragonfly.git] / share / man / man7 / tuning.7
CommitLineData
033a4603 1.\" Copyright (c) 2001 Matthew Dillon. Terms and conditions are those of
984263bc
MD
2.\" the BSD Copyright as specified in the file "/usr/src/COPYRIGHT" in
3.\" the source tree.
4.\"
5.\" $FreeBSD: src/share/man/man7/tuning.7,v 1.1.2.30 2002/12/17 19:32:08 dillon Exp $
4ad6607f 6.\" $DragonFly: src/share/man/man7/tuning.7,v 1.8 2006/05/11 08:23:19 swildner Exp $
984263bc 7.\"
4ad6607f 8.Dd May 11, 2006
984263bc
MD
9.Dt TUNING 7
10.Os
11.Sh NAME
12.Nm tuning
a3220ac5
SW
13.Nd performance tuning under
14.Dx
984263bc
MD
15.Sh SYSTEM SETUP - DISKLABEL, NEWFS, TUNEFS, SWAP
16When using
17.Xr disklabel 8
18or
19.Xr sysinstall 8
20to lay out your filesystems on a hard disk it is important to remember
21that hard drives can transfer data much more quickly from outer tracks
22than they can from inner tracks.
23To take advantage of this you should
24try to pack your smaller filesystems and swap closer to the outer tracks,
25follow with the larger filesystems, and end with the largest filesystems.
26It is also important to size system standard filesystems such that you
27will not be forced to resize them later as you scale the machine up.
28I usually create, in order, a 128M root, 1G swap, 128M
29.Pa /var ,
30128M
31.Pa /var/tmp ,
323G
33.Pa /usr ,
34and use any remaining space for
35.Pa /home .
36.Pp
37You should typically size your swap space to approximately 2x main memory.
38If you do not have a lot of RAM, though, you will generally want a lot
39more swap.
40It is not recommended that you configure any less than
41256M of swap on a system and you should keep in mind future memory
42expansion when sizing the swap partition.
43The kernel's VM paging algorithms are tuned to perform best when there is
44at least 2x swap versus main memory.
45Configuring too little swap can lead
46to inefficiencies in the VM page scanning code as well as create issues
47later on if you add more memory to your machine.
48Finally, on larger systems
49with multiple SCSI disks (or multiple IDE disks operating on different
50controllers), we strongly recommend that you configure swap on each drive
51(up to four drives).
52The swap partitions on the drives should be approximately the same size.
53The kernel can handle arbitrary sizes but
54internal data structures scale to 4 times the largest swap partition.
55Keeping
56the swap partitions near the same size will allow the kernel to optimally
57stripe swap space across the N disks.
58Do not worry about overdoing it a
59little, swap space is the saving grace of
60.Ux
61and even if you do not normally use much swap, it can give you more time to
62recover from a runaway program before being forced to reboot.
63.Pp
64How you size your
65.Pa /var
66partition depends heavily on what you intend to use the machine for.
67This
68partition is primarily used to hold mailboxes, the print spool, and log
69files.
70Some people even make
71.Pa /var/log
72its own partition (but except for extreme cases it is not worth the waste
73of a partition ID).
74If your machine is intended to act as a mail
75or print server,
76or you are running a heavily visited web server, you should consider
77creating a much larger partition \(en perhaps a gig or more.
78It is very easy
79to underestimate log file storage requirements.
80.Pp
81Sizing
82.Pa /var/tmp
83depends on the kind of temporary file usage you think you will need.
84128M is
85the minimum we recommend.
86Also note that sysinstall will create a
87.Pa /tmp
88directory.
89Dedicating a partition for temporary file storage is important for
90two reasons: first, it reduces the possibility of filesystem corruption
91in a crash, and second it reduces the chance of a runaway process that
92fills up
93.Oo Pa /var Oc Ns Pa /tmp
94from blowing up more critical subsystems (mail,
95logging, etc).
96Filling up
97.Oo Pa /var Oc Ns Pa /tmp
98is a very common problem to have.
99.Pp
100In the old days there were differences between
101.Pa /tmp
102and
103.Pa /var/tmp ,
104but the introduction of
105.Pa /var
106(and
107.Pa /var/tmp )
108led to massive confusion
109by program writers so today programs haphazardly use one or the
110other and thus no real distinction can be made between the two.
111So it makes sense to have just one temporary directory and
112softlink to it from the other tmp directory locations.
113However you handle
114.Pa /tmp ,
115the one thing you do not want to do is leave it sitting
116on the root partition where it might cause root to fill up or possibly
117corrupt root in a crash/reboot situation.
118.Pp
119The
120.Pa /usr
121partition holds the bulk of the files required to support the system and
122a subdirectory within it called
123.Pa /usr/local
124holds the bulk of the files installed from the
125.Xr ports 7
126hierarchy.
127If you do not use ports all that much and do not intend to keep
128system source
129.Pq Pa /usr/src
130on the machine, you can get away with
131a 1 gigabyte
132.Pa /usr
133partition.
134However, if you install a lot of ports
135(especially window managers and Linux-emulated binaries), we recommend
136at least a 2 gigabyte
137.Pa /usr
138and if you also intend to keep system source
139on the machine, we recommend a 3 gigabyte
140.Pa /usr .
141Do not underestimate the
142amount of space you will need in this partition, it can creep up and
143surprise you!
144.Pp
145The
146.Pa /home
147partition is typically used to hold user-specific data.
148I usually size it to the remainder of the disk.
149.Pp
150Why partition at all?
151Why not create one big
152.Pa /
153partition and be done with it?
154Then I do not have to worry about undersizing things!
155Well, there are several reasons this is not a good idea.
156First,
157each partition has different operational characteristics and separating them
158allows the filesystem to tune itself to those characteristics.
159For example,
160the root and
161.Pa /usr
162partitions are read-mostly, with very little writing, while
163a lot of reading and writing could occur in
164.Pa /var
165and
166.Pa /var/tmp .
167By properly
168partitioning your system fragmentation introduced in the smaller more
169heavily write-loaded partitions will not bleed over into the mostly-read
170partitions.
171Additionally, keeping the write-loaded partitions closer to
172the edge of the disk (i.e. before the really big partitions instead of after
173in the partition table) will increase I/O performance in the partitions
174where you need it the most.
175Now it is true that you might also need I/O
176performance in the larger partitions, but they are so large that shifting
177them more towards the edge of the disk will not lead to a significant
178performance improvement whereas moving
179.Pa /var
180to the edge can have a huge impact.
181Finally, there are safety concerns.
182Having a small neat root partition that
183is essentially read-only gives it a greater chance of surviving a bad crash
184intact.
185.Pp
186Properly partitioning your system also allows you to tune
187.Xr newfs 8 ,
188and
189.Xr tunefs 8
190parameters.
191Tuning
192.Xr newfs 8
193requires more experience but can lead to significant improvements in
194performance.
195There are three parameters that are relatively safe to tune:
196.Em blocksize , bytes/i-node ,
197and
198.Em cylinders/group .
199.Pp
9bb2a92d 200.Dx
984263bc
MD
201performs best when using 8K or 16K filesystem block sizes.
202The default filesystem block size is 16K,
203which provides best performance for most applications,
204with the exception of those that perform random access on large files
205(such as database server software).
206Such applications tend to perform better with a smaller block size,
207although modern disk characteristics are such that the performance
208gain from using a smaller block size may not be worth consideration.
209Using a block size larger than 16K
210can cause fragmentation of the buffer cache and
211lead to lower performance.
212.Pp
213The defaults may be unsuitable
214for a filesystem that requires a very large number of i-nodes
215or is intended to hold a large number of very small files.
216Such a filesystem should be created with an 8K or 4K block size.
217This also requires you to specify a smaller
218fragment size.
aa0d550a 219We recommend always using a fragment size that is \(18
984263bc
MD
220the block size (less testing has been done on other fragment size factors).
221The
222.Xr newfs 8
223options for this would be
224.Dq Li "newfs -f 1024 -b 8192 ..." .
225.Pp
226If a large partition is intended to be used to hold fewer, larger files, such
227as database files, you can increase the
228.Em bytes/i-node
229ratio which reduces the number of i-nodes (maximum number of files and
230directories that can be created) for that partition.
231Decreasing the number
232of i-nodes in a filesystem can greatly reduce
233.Xr fsck 8
234recovery times after a crash.
235Do not use this option
236unless you are actually storing large files on the partition, because if you
237overcompensate you can wind up with a filesystem that has lots of free
238space remaining but cannot accommodate any more files.
239Using 32768, 65536, or 262144 bytes/i-node is recommended.
240You can go higher but
241it will have only incremental effects on
242.Xr fsck 8
243recovery times.
244For example,
245.Dq Li "newfs -i 32768 ..." .
246.Pp
247.Xr tunefs 8
248may be used to further tune a filesystem.
249This command can be run in
250single-user mode without having to reformat the filesystem.
251However, this is possibly the most abused program in the system.
252Many people attempt to
253increase available filesystem space by setting the min-free percentage to 0.
254This can lead to severe filesystem fragmentation and we do not recommend
255that you do this.
256Really the only
257.Xr tunefs 8
258option worthwhile here is turning on
259.Em softupdates
260with
261.Dq Li "tunefs -n enable /filesystem" .
262(Note: in
263.Fx 4.5
264and later, softupdates can be turned on using the
265.Fl U
266option to
267.Xr newfs 8 ,
268and
269.Xr sysinstall 8
270will typically enable softupdates automatically for non-root filesystems).
271Softupdates drastically improves meta-data performance, mainly file
272creation and deletion.
273We recommend enabling softupdates on most filesystems; however, there
274are two limitations to softupdates that you should be aware of when
275determining whether to use it on a filesystem.
276First, softupdates guarantees filesystem consistency in the
277case of a crash but could very easily be several seconds (even a minute!)
278behind on pending writes to the physical disk.
279If you crash you may lose more work
280than otherwise.
281Secondly, softupdates delays the freeing of filesystem
282blocks.
283If you have a filesystem (such as the root filesystem) which is
284close to full, doing a major update of it, e.g.\&
285.Dq Li "make installworld" ,
286can run it out of space and cause the update to fail.
287For this reason, softupdates will not be enabled on the root filesystem
288during a typical install. There is no loss of performance since the root
289filesystem is rarely written to.
290.Pp
291A number of run-time
292.Xr mount 8
293options exist that can help you tune the system.
294The most obvious and most dangerous one is
295.Cm async .
296Do not ever use it; it is far too dangerous.
297A less dangerous and more
298useful
299.Xr mount 8
300option is called
301.Cm noatime .
302.Ux
303filesystems normally update the last-accessed time of a file or
304directory whenever it is accessed.
305This operation is handled in
9bb2a92d 306.Dx
984263bc
MD
307with a delayed write and normally does not create a burden on the system.
308However, if your system is accessing a huge number of files on a continuing
309basis the buffer cache can wind up getting polluted with atime updates,
310creating a burden on the system.
311For example, if you are running a heavily
312loaded web site, or a news server with lots of readers, you might want to
313consider turning off atime updates on your larger partitions with this
314.Xr mount 8
315option.
316However, you should not gratuitously turn off atime
317updates everywhere.
318For example, the
319.Pa /var
320filesystem customarily
321holds mailboxes, and atime (in combination with mtime) is used to
322determine whether a mailbox has new mail.
323You might as well leave
324atime turned on for mostly read-only partitions such as
325.Pa /
326and
327.Pa /usr
328as well.
329This is especially useful for
330.Pa /
331since some system utilities
332use the atime field for reporting.
333.Sh STRIPING DISKS
334In larger systems you can stripe partitions from several drives together
335to create a much larger overall partition.
336Striping can also improve
337the performance of a filesystem by splitting I/O operations across two
338or more disks.
339The
340.Xr vinum 8
341and
342.Xr ccdconfig 8
343utilities may be used to create simple striped filesystems.
344Generally
345speaking, striping smaller partitions such as the root and
346.Pa /var/tmp ,
347or essentially read-only partitions such as
348.Pa /usr
349is a complete waste of time.
350You should only stripe partitions that require serious I/O performance,
351typically
352.Pa /var , /home ,
353or custom partitions used to hold databases and web pages.
354Choosing the proper stripe size is also
355important.
356Filesystems tend to store meta-data on power-of-2 boundaries
357and you usually want to reduce seeking rather than increase seeking.
358This
359means you want to use a large off-center stripe size such as 1152 sectors
360so sequential I/O does not seek both disks and so meta-data is distributed
361across both disks rather than concentrated on a single disk.
362If
363you really need to get sophisticated, we recommend using a real hardware
364RAID controller from the list of
9bb2a92d 365.Dx
984263bc
MD
366supported controllers.
367.Sh SYSCTL TUNING
368.Xr sysctl 8
369variables permit system behavior to be monitored and controlled at
370run-time.
371Some sysctls simply report on the behavior of the system; others allow
372the system behavior to be modified;
373some may be set at boot time using
374.Xr rc.conf 5 ,
375but most will be set via
376.Xr sysctl.conf 5 .
377There are several hundred sysctls in the system, including many that appear
378to be candidates for tuning but actually are not.
379In this document we will only cover the ones that have the greatest effect
380on the system.
381.Pp
382The
383.Va kern.ipc.shm_use_phys
384sysctl defaults to 0 (off) and may be set to 0 (off) or 1 (on).
385Setting
386this parameter to 1 will cause all System V shared memory segments to be
387mapped to unpageable physical RAM.
388This feature only has an effect if you
389are either (A) mapping small amounts of shared memory across many (hundreds)
390of processes, or (B) mapping large amounts of shared memory across any
391number of processes.
392This feature allows the kernel to remove a great deal
393of internal memory management page-tracking overhead at the cost of wiring
394the shared memory into core, making it unswappable.
395.Pp
396The
397.Va vfs.vmiodirenable
398sysctl defaults to 1 (on).
399This parameter controls how directories are cached
400by the system.
401Most directories are small and use but a single fragment
402(typically 1K) in the filesystem and even less (typically 512 bytes) in
403the buffer cache.
404However, when operating in the default mode the buffer
405cache will only cache a fixed number of directories even if you have a huge
406amount of memory.
407Turning on this sysctl allows the buffer cache to use
408the VM Page Cache to cache the directories.
409The advantage is that all of
410memory is now available for caching directories.
411The disadvantage is that
412the minimum in-core memory used to cache a directory is the physical page
413size (typically 4K) rather than 512 bytes.
414We recommend turning this option off in memory-constrained environments;
415however, when on, it will substantially improve the performance of services
416that manipulate a large number of files.
417Such services can include web caches, large mail systems, and news systems.
418Turning on this option will generally not reduce performance even with the
419wasted memory but you should experiment to find out.
420.Pp
421The
422.Va vfs.write_behind
423sysctl defaults to 1 (on). This tells the filesystem to issue media
424writes as full clusters are collected, which typically occurs when writing
425large sequential files. The idea is to avoid saturating the buffer
426cache with dirty buffers when it would not benefit I/O performance. However,
427this may stall processes and under certain circumstances you may wish to turn
428it off.
429.Pp
430The
431.Va vfs.hirunningspace
432sysctl determines how much outstanding write I/O may be queued to
433disk controllers system wide at any given instance. The default is
434usually sufficient but on machines with lots of disks you may want to bump
435it up to four or five megabytes. Note that setting too high a value
436(exceeding the buffer cache's write threshold) can lead to extremely
437bad clustering performance. Do not set this value arbitrarily high! Also,
438higher write queueing values may add latency to reads occuring at the same
439time.
440.Pp
441There are various other buffer-cache and VM page cache related sysctls.
442We do not recommend modifying these values.
443As of
444.Fx 4.3 ,
445the VM system does an extremely good job tuning itself.
446.Pp
447The
448.Va net.inet.tcp.sendspace
449and
450.Va net.inet.tcp.recvspace
451sysctls are of particular interest if you are running network intensive
452applications.
453They control the amount of send and receive buffer space
454allowed for any given TCP connection.
455The default sending buffer is 32K; the default receiving buffer
456is 64K.
457You can often
458improve bandwidth utilization by increasing the default at the cost of
459eating up more kernel memory for each connection.
460We do not recommend
461increasing the defaults if you are serving hundreds or thousands of
462simultaneous connections because it is possible to quickly run the system
463out of memory due to stalled connections building up.
464But if you need
465high bandwidth over a fewer number of connections, especially if you have
466gigabit Ethernet, increasing these defaults can make a huge difference.
467You can adjust the buffer size for incoming and outgoing data separately.
468For example, if your machine is primarily doing web serving you may want
469to decrease the recvspace in order to be able to increase the
470sendspace without eating too much kernel memory.
471Note that the routing table (see
472.Xr route 8 )
473can be used to introduce route-specific send and receive buffer size
474defaults.
475.Pp
476As an additional management tool you can use pipes in your
477firewall rules (see
478.Xr ipfw 8 )
479to limit the bandwidth going to or from particular IP blocks or ports.
480For example, if you have a T1 you might want to limit your web traffic
481to 70% of the T1's bandwidth in order to leave the remainder available
482for mail and interactive use.
483Normally a heavily loaded web server
484will not introduce significant latencies into other services even if
485the network link is maxed out, but enforcing a limit can smooth things
486out and lead to longer term stability.
487Many people also enforce artificial
488bandwidth limitations in order to ensure that they are not charged for
489using too much bandwidth.
490.Pp
491Setting the send or receive TCP buffer to values larger then 65535 will result
492in a marginal performance improvement unless both hosts support the window
493scaling extension of the TCP protocol, which is controlled by the
494.Va net.inet.tcp.rfc1323
495sysctl.
496These extensions should be enabled and the TCP buffer size should be set
497to a value larger than 65536 in order to obtain good performance from
498certain types of network links; specifically, gigabit WAN links and
499high-latency satellite links.
500RFC1323 support is enabled by default.
501.Pp
502The
503.Va net.inet.tcp.always_keepalive
504sysctl determines whether or not the TCP implementation should attempt
505to detect dead TCP connections by intermittently delivering
506.Dq keepalives
507on the connection.
508By default, this is enabled for all applications; by setting this
509sysctl to 0, only applications that specifically request keepalives
510will use them.
511In most environments, TCP keepalives will improve the management of
512system state by expiring dead TCP connections, particularly for
513systems serving dialup users who may not always terminate individual
514TCP connections before disconnecting from the network.
515However, in some environments, temporary network outages may be
516incorrectly identified as dead sessions, resulting in unexpectedly
517terminated TCP connections.
518In such environments, setting the sysctl to 0 may reduce the occurrence of
519TCP session disconnections.
520.Pp
521The
522.Va net.inet.tcp.delayed_ack
523TCP feature is largly misunderstood. Historically speaking this feature
524was designed to allow the acknowledgement to transmitted data to be returned
525along with the response. For example, when you type over a remote shell
526the acknowledgement to the character you send can be returned along with the
527data representing the echo of the character. With delayed acks turned off
528the acknowledgement may be sent in its own packet before the remote service
529has a chance to echo the data it just received. This same concept also
530applies to any interactive protocol (e.g. SMTP, WWW, POP3) and can cut the
a3220ac5
SW
531number of tiny packets flowing across the network in half. The
532.Dx
984263bc
MD
533delayed-ack implementation also follows the TCP protocol rule that
534at least every other packet be acknowledged even if the standard 100ms
535timeout has not yet passed. Normally the worst a delayed ack can do is
536slightly delay the teardown of a connection, or slightly delay the ramp-up
537of a slow-start TCP connection. While we aren't sure we believe that
538the several FAQs related to packages such as SAMBA and SQUID which advise
a3220ac5
SW
539turning off delayed acks may be refering to the slow-start issue. In
540.Dx
984263bc
MD
541it would be more beneficial to increase the slow-start flightsize via
542the
543.Va net.inet.tcp.slowstart_flightsize
544sysctl rather then disable delayed acks.
545.Pp
546The
547.Va net.inet.tcp.inflight_enable
548sysctl turns on bandwidth delay product limiting for all TCP connections.
549The system will attempt to calculate the bandwidth delay product for each
550connection and limit the amount of data queued to the network to just the
551amount required to maintain optimum throughput. This feature is useful
552if you are serving data over modems, GigE, or high speed WAN links (or
553any other link with a high bandwidth*delay product), especially if you are
554also using window scaling or have configured a large send window. If
555you enable this option you should also be sure to set
556.Va net.inet.tcp.inflight_debug
557to 0 (disable debugging), and for production use setting
558.Va net.inet.tcp.inflight_min
559to at least 6144 may be beneficial. Note, however, that setting high
560minimums may effectively disable bandwidth limiting depending on the link.
561The limiting feature reduces the amount of data built up in intermediate
562router and switch packet queues as well as reduces the amount of data built
563up in the local host's interface queue. With fewer packets queued up,
564interactive connections, especially over slow modems, will also be able
565to operate with lower round trip times. However, note that this feature
566only effects data transmission (uploading / server-side). It does not
567effect data reception (downloading).
568.Pp
569Adjusting
570.Va net.inet.tcp.inflight_stab
571is not recommended.
1bf4b486 572This parameter defaults to 20, representing 2 maximal packets added
984263bc
MD
573to the bandwidth delay product window calculation. The additional
574window is required to stabilize the algorithm and improve responsiveness
575to changing conditions, but it can also result in higher ping times
1bf4b486 576over slow links (though still much lower then you would get without
984263bc
MD
577the inflight algorithm). In such cases you may
578wish to try reducing this parameter to 15, 10, or 5, and you may also
579have to reduce
580.Va net.inet.tcp.inflight_min
581(for example, to 3500) to get the desired effect. Reducing these parameters
582should be done as a last resort only.
583.Pp
584The
585.Va net.inet.ip.portrange.*
586sysctls control the port number ranges automatically bound to TCP and UDP
587sockets. There are three ranges: A low range, a default range, and a
1bf4b486 588high range, selectable via an IP_PORTRANGE setsockopt() call. Most
984263bc
MD
589network programs use the default range which is controlled by
590.Va net.inet.ip.portrange.first
591and
592.Va net.inet.ip.portrange.last ,
593which defaults to 1024 and 5000 respectively. Bound port ranges are
594used for outgoing connections and it is possible to run the system out
595of ports under certain circumstances. This most commonly occurs when you are
596running a heavily loaded web proxy. The port range is not an issue
597when running serves which handle mainly incoming connections such as a
598normal web server, or has a limited number of outgoing connections such
599as a mail relay. For situations where you may run yourself out of
600ports we recommend increasing
601.Va net.inet.ip.portrange.last
602modestly. A value of 10000 or 20000 or 30000 may be reasonable. You should
603also consider firewall effects when changing the port range. Some firewalls
604may block large ranges of ports (usually low-numbered ports) and expect systems
605to use higher ranges of ports for outgoing connections. For this reason
606we do not recommend that
607.Va net.inet.ip.portrange.first
608be lowered.
609.Pp
610The
611.Va kern.ipc.somaxconn
612sysctl limits the size of the listen queue for accepting new TCP connections.
613The default value of 128 is typically too low for robust handling of new
614connections in a heavily loaded web server environment.
615For such environments,
616we recommend increasing this value to 1024 or higher.
617The service daemon
618may itself limit the listen queue size (e.g.\&
619.Xr sendmail 8 ,
620apache) but will
621often have a directive in its configuration file to adjust the queue size up.
622Larger listen queues also do a better job of fending off denial of service
623attacks.
624.Pp
625The
626.Va kern.maxfiles
627sysctl determines how many open files the system supports.
628The default is
629typically a few thousand but you may need to bump this up to ten or twenty
630thousand if you are running databases or large descriptor-heavy daemons.
631The read-only
632.Va kern.openfiles
633sysctl may be interrogated to determine the current number of open files
634on the system.
635.Pp
636The
637.Va vm.swap_idle_enabled
638sysctl is useful in large multi-user systems where you have lots of users
639entering and leaving the system and lots of idle processes.
640Such systems
641tend to generate a great deal of continuous pressure on free memory reserves.
642Turning this feature on and adjusting the swapout hysteresis (in idle
643seconds) via
644.Va vm.swap_idle_threshold1
645and
646.Va vm.swap_idle_threshold2
647allows you to depress the priority of pages associated with idle processes
648more quickly then the normal pageout algorithm.
649This gives a helping hand
650to the pageout daemon.
651Do not turn this option on unless you need it,
652because the tradeoff you are making is to essentially pre-page memory sooner
653rather then later, eating more swap and disk bandwidth.
654In a small system
655this option will have a detrimental effect but in a large system that is
656already doing moderate paging this option allows the VM system to stage
657whole processes into and out of memory more easily.
658.Sh LOADER TUNABLES
659Some aspects of the system behavior may not be tunable at runtime because
660memory allocations they perform must occur early in the boot process.
661To change loader tunables, you must set their values in
662.Xr loader.conf 5
663and reboot the system.
664.Pp
665.Va kern.maxusers
666controls the scaling of a number of static system tables, including defaults
667for the maximum number of open files, sizing of network memory resources, etc.
668As of
669.Fx 4.5 ,
670.Va kern.maxusers
671is automatically sized at boot based on the amount of memory available in
672the system, and may be determined at run-time by inspecting the value of the
673read-only
674.Va kern.maxusers
675sysctl.
676Some sites will require larger or smaller values of
677.Va kern.maxusers
678and may set it as a loader tunable; values of 64, 128, and 256 are not
679uncommon.
680We do not recommend going above 256 unless you need a huge number
681of file descriptors; many of the tunable values set to their defaults by
682.Va kern.maxusers
683may be individually overridden at boot-time or run-time as described
684elsewhere in this document.
685Systems older than
686.Fx 4.4
687must set this value via the kernel
688.Xr config 8
689option
690.Cd maxusers
691instead.
692.Pp
693.Va kern.ipc.nmbclusters
694may be adjusted to increase the number of network mbufs the system is
695willing to allocate.
696Each cluster represents approximately 2K of memory,
697so a value of 1024 represents 2M of kernel memory reserved for network
698buffers.
699You can do a simple calculation to figure out how many you need.
700If you have a web server which maxes out at 1000 simultaneous connections,
701and each connection eats a 16K receive and 16K send buffer, you need
702approximately 32MB worth of network buffers to deal with it.
703A good rule of
704thumb is to multiply by 2, so 32MBx2 = 64MB/2K = 32768.
705So for this case
706you would want to set
707.Va kern.ipc.nmbclusters
708to 32768.
709We recommend values between
7101024 and 4096 for machines with moderates amount of memory, and between 4096
711and 32768 for machines with greater amounts of memory.
712Under no circumstances
713should you specify an arbitrarily high value for this parameter, it could
714lead to a boot-time crash.
715The
716.Fl m
717option to
718.Xr netstat 1
719may be used to observe network cluster use.
720Older versions of
721.Fx
722do not have this tunable and require that the
723kernel
724.Xr config 8
725option
726.Dv NMBCLUSTERS
727be set instead.
728.Pp
729More and more programs are using the
730.Xr sendfile 2
731system call to transmit files over the network.
732The
733.Va kern.ipc.nsfbufs
734sysctl controls the number of filesystem buffers
735.Xr sendfile 2
736is allowed to use to perform its work.
737This parameter nominally scales
738with
739.Va kern.maxusers
740so you should not need to modify this parameter except under extreme
741circumstances.
742.Sh KERNEL CONFIG TUNING
743There are a number of kernel options that you may have to fiddle with in
744a large-scale system.
745In order to change these options you need to be
746able to compile a new kernel from source.
747The
748.Xr config 8
749manual page and the handbook are good starting points for learning how to
750do this.
751Generally the first thing you do when creating your own custom
752kernel is to strip out all the drivers and services you do not use.
753Removing things like
754.Dv INET6
755and drivers you do not have will reduce the size of your kernel, sometimes
756by a megabyte or more, leaving more memory available for applications.
757.Pp
758.Dv SCSI_DELAY
984263bc 759may be used to reduce system boot times.
4ad6607f 760The default is fairly high and
984263bc
MD
761can be responsible for 15+ seconds of delay in the boot process.
762Reducing
763.Dv SCSI_DELAY
764to 5 seconds usually works (especially with modern drives).
984263bc
MD
765.Pp
766There are a number of
767.Dv *_CPU
768options that can be commented out.
769If you only want the kernel to run
770on a Pentium class CPU, you can easily remove
771.Dv I386_CPU
772and
773.Dv I486_CPU ,
774but only remove
775.Dv I586_CPU
776if you are sure your CPU is being recognized as a Pentium II or better.
777Some clones may be recognized as a Pentium or even a 486 and not be able
778to boot without those options.
779If it works, great!
780The operating system
781will be able to better-use higher-end CPU features for MMU, task switching,
782timebase, and even device operations.
783Additionally, higher-end CPUs support
7844MB MMU pages, which the kernel uses to map the kernel itself into memory,
785increasing its efficiency under heavy syscall loads.
786.Sh IDE WRITE CACHING
787.Fx 4.3
788flirted with turning off IDE write caching.
789This reduced write bandwidth
790to IDE disks but was considered necessary due to serious data consistency
791issues introduced by hard drive vendors.
792Basically the problem is that
793IDE drives lie about when a write completes.
794With IDE write caching turned
795on, IDE hard drives will not only write data to disk out of order, they
796will sometimes delay some of the blocks indefinitely under heavy disk
797load.
798A crash or power failure can result in serious filesystem
799corruption.
800So our default was changed to be safe.
801Unfortunately, the
802result was such a huge loss in performance that we caved in and changed the
803default back to on after the release.
804You should check the default on
805your system by observing the
806.Va hw.ata.wc
807sysctl variable.
808If IDE write caching is turned off, you can turn it back
809on by setting the
810.Va hw.ata.wc
811loader tunable to 1.
812More information on tuning the ATA driver system may be found in the
813.Xr ata 4
814man page.
815.Pp
816There is a new experimental feature for IDE hard drives called
817.Va hw.ata.tags
818(you also set this in the boot loader) which allows write caching to be safely
819turned on.
820This brings SCSI tagging features to IDE drives.
821As of this
822writing only IBM DPTA and DTLA drives support the feature.
823Warning!
824These
825drives apparently have quality control problems and I do not recommend
826purchasing them at this time.
827If you need performance, go with SCSI.
828.Sh CPU, MEMORY, DISK, NETWORK
829The type of tuning you do depends heavily on where your system begins to
830bottleneck as load increases.
831If your system runs out of CPU (idle times
832are perpetually 0%) then you need to consider upgrading the CPU or moving to
833an SMP motherboard (multiple CPU's), or perhaps you need to revisit the
834programs that are causing the load and try to optimize them.
835If your system
836is paging to swap a lot you need to consider adding more memory.
837If your
838system is saturating the disk you typically see high CPU idle times and
839total disk saturation.
840.Xr systat 1
841can be used to monitor this.
842There are many solutions to saturated disks:
843increasing memory for caching, mirroring disks, distributing operations across
844several machines, and so forth.
845If disk performance is an issue and you
846are using IDE drives, switching to SCSI can help a great deal.
847While modern
848IDE drives compare with SCSI in raw sequential bandwidth, the moment you
849start seeking around the disk SCSI drives usually win.
850.Pp
851Finally, you might run out of network suds.
852The first line of defense for
853improving network performance is to make sure you are using switches instead
854of hubs, especially these days where switches are almost as cheap.
855Hubs
856have severe problems under heavy loads due to collision backoff and one bad
857host can severely degrade the entire LAN.
858Second, optimize the network path
859as much as possible.
860For example, in
861.Xr firewall 7
862we describe a firewall protecting internal hosts with a topology where
863the externally visible hosts are not routed through it.
864Use 100BaseT rather
865than 10BaseT, or use 1000BaseT rather then 100BaseT, depending on your needs.
866Most bottlenecks occur at the WAN link (e.g.\&
867modem, T1, DSL, whatever).
868If expanding the link is not an option it may be possible to use the
869.Xr dummynet 4
870feature to implement peak shaving or other forms of traffic shaping to
871prevent the overloaded service (such as web services) from affecting other
872services (such as email), or vice versa.
873In home installations this could
874be used to give interactive traffic (your browser,
875.Xr ssh 1
876logins) priority
877over services you export from your box (web services, email).
878.Sh SEE ALSO
879.Xr netstat 1 ,
880.Xr systat 1 ,
881.Xr ata 4 ,
882.Xr dummynet 4 ,
883.Xr login.conf 5 ,
884.Xr rc.conf 5 ,
885.Xr sysctl.conf 5 ,
886.Xr firewall 7 ,
887.Xr hier 7 ,
888.Xr ports 7 ,
889.Xr boot 8 ,
890.Xr ccdconfig 8 ,
891.Xr config 8 ,
892.Xr disklabel 8 ,
893.Xr fsck 8 ,
894.Xr ifconfig 8 ,
895.Xr ipfw 8 ,
896.Xr loader 8 ,
897.Xr mount 8 ,
898.Xr newfs 8 ,
899.Xr route 8 ,
900.Xr sysctl 8 ,
901.Xr sysinstall 8 ,
902.Xr tunefs 8 ,
903.Xr vinum 8
904.Sh HISTORY
905The
906.Nm
907manual page was originally written by
908.An Matthew Dillon
909and first appeared
910in
911.Fx 4.3 ,
912May 2001.