Fixes for CAM_NEW_TRAN_CODE.
[dragonfly.git] / share / man / man7 / tuning.7
... / ...
CommitLineData
1.\" Copyright (c) 2001 Matthew Dillon. Terms and conditions are those of
2.\" the BSD Copyright as specified in the file "/usr/src/COPYRIGHT" in
3.\" the source tree.
4.\"
5.\" $FreeBSD: src/share/man/man7/tuning.7,v 1.1.2.30 2002/12/17 19:32:08 dillon Exp $
6.\" $DragonFly: src/share/man/man7/tuning.7,v 1.18 2007/12/13 20:51:36 swildner Exp $
7.\"
8.Dd March 4, 2007
9.Dt TUNING 7
10.Os
11.Sh NAME
12.Nm tuning
13.Nd performance tuning under DragonFly
14.Sh SYSTEM SETUP - DISKLABEL, NEWFS, TUNEFS, SWAP
15When using
16.Xr disklabel 8
17or the
18.Dx
19installer
20to lay out your filesystems on a hard disk it is important to remember
21that hard drives can transfer data much more quickly from outer tracks
22than they can from inner tracks.
23To take advantage of this you should
24try to pack your smaller filesystems and swap closer to the outer tracks,
25follow with the larger filesystems, and end with the largest filesystems.
26It is also important to size system standard filesystems such that you
27will not be forced to resize them later as you scale the machine up.
28I usually create, in order, a 128M root, 1G swap, 128M
29.Pa /var ,
30128M
31.Pa /var/tmp ,
323G
33.Pa /usr ,
34and use any remaining space for
35.Pa /home .
36.Pp
37You should typically size your swap space to approximately 2x main memory.
38If you do not have a lot of RAM, though, you will generally want a lot
39more swap.
40It is not recommended that you configure any less than
41256M of swap on a system and you should keep in mind future memory
42expansion when sizing the swap partition.
43The kernel's VM paging algorithms are tuned to perform best when there is
44at least 2x swap versus main memory.
45Configuring too little swap can lead
46to inefficiencies in the VM page scanning code as well as create issues
47later on if you add more memory to your machine.
48Finally, on larger systems
49with multiple SCSI disks (or multiple IDE disks operating on different
50controllers), we strongly recommend that you configure swap on each drive
51(up to four drives).
52The swap partitions on the drives should be approximately the same size.
53The kernel can handle arbitrary sizes but
54internal data structures scale to 4 times the largest swap partition.
55Keeping
56the swap partitions near the same size will allow the kernel to optimally
57stripe swap space across the N disks.
58Do not worry about overdoing it a
59little, swap space is the saving grace of
60.Ux
61and even if you do not normally use much swap, it can give you more time to
62recover from a runaway program before being forced to reboot.
63.Pp
64How you size your
65.Pa /var
66partition depends heavily on what you intend to use the machine for.
67This
68partition is primarily used to hold mailboxes, the print spool, and log
69files.
70Some people even make
71.Pa /var/log
72its own partition (but except for extreme cases it is not worth the waste
73of a partition ID).
74If your machine is intended to act as a mail
75or print server,
76or you are running a heavily visited web server, you should consider
77creating a much larger partition \(en perhaps a gig or more.
78It is very easy
79to underestimate log file storage requirements.
80.Pp
81Sizing
82.Pa /var/tmp
83depends on the kind of temporary file usage you think you will need.
84128M is
85the minimum we recommend.
86Also note that the
87.Dx
88installer will create a
89.Pa /tmp
90directory.
91Dedicating a partition for temporary file storage is important for
92two reasons: first, it reduces the possibility of filesystem corruption
93in a crash, and second it reduces the chance of a runaway process that
94fills up
95.Oo Pa /var Oc Ns Pa /tmp
96from blowing up more critical subsystems (mail,
97logging, etc).
98Filling up
99.Oo Pa /var Oc Ns Pa /tmp
100is a very common problem to have.
101.Pp
102In the old days there were differences between
103.Pa /tmp
104and
105.Pa /var/tmp ,
106but the introduction of
107.Pa /var
108(and
109.Pa /var/tmp )
110led to massive confusion
111by program writers so today programs haphazardly use one or the
112other and thus no real distinction can be made between the two.
113So it makes sense to have just one temporary directory and
114softlink to it from the other tmp directory locations.
115However you handle
116.Pa /tmp ,
117the one thing you do not want to do is leave it sitting
118on the root partition where it might cause root to fill up or possibly
119corrupt root in a crash/reboot situation.
120.Pp
121The
122.Pa /usr
123partition holds the bulk of the files required to support the system and
124a subdirectory within it called
125.Pa /usr/pkg
126holds the bulk of the files installed from the
127.Xr pkgsrc 7
128collection.
129If you do not use
130.Xr pkgsrc 7
131all that much and do not intend to keep system source
132.Pq Pa /usr/src
133on the machine, you can get away with
134a 1 gigabyte
135.Pa /usr
136partition.
137However, if you install a lot of packages
138(especially window managers and Linux-emulated binaries), we recommend
139at least a 2 gigabyte
140.Pa /usr
141and if you also intend to keep system source
142on the machine, we recommend a 3 gigabyte
143.Pa /usr .
144Do not underestimate the
145amount of space you will need in this partition, it can creep up and
146surprise you!
147.Pp
148The
149.Pa /home
150partition is typically used to hold user-specific data.
151I usually size it to the remainder of the disk.
152.Pp
153Why partition at all?
154Why not create one big
155.Pa /
156partition and be done with it?
157Then I do not have to worry about undersizing things!
158Well, there are several reasons this is not a good idea.
159First,
160each partition has different operational characteristics and separating them
161allows the filesystem to tune itself to those characteristics.
162For example,
163the root and
164.Pa /usr
165partitions are read-mostly, with very little writing, while
166a lot of reading and writing could occur in
167.Pa /var
168and
169.Pa /var/tmp .
170By properly
171partitioning your system fragmentation introduced in the smaller more
172heavily write-loaded partitions will not bleed over into the mostly-read
173partitions.
174Additionally, keeping the write-loaded partitions closer to
175the edge of the disk (i.e. before the really big partitions instead of after
176in the partition table) will increase I/O performance in the partitions
177where you need it the most.
178Now it is true that you might also need I/O
179performance in the larger partitions, but they are so large that shifting
180them more towards the edge of the disk will not lead to a significant
181performance improvement whereas moving
182.Pa /var
183to the edge can have a huge impact.
184Finally, there are safety concerns.
185Having a small neat root partition that
186is essentially read-only gives it a greater chance of surviving a bad crash
187intact.
188.Pp
189Properly partitioning your system also allows you to tune
190.Xr newfs 8 ,
191and
192.Xr tunefs 8
193parameters.
194Tuning
195.Xr newfs 8
196requires more experience but can lead to significant improvements in
197performance.
198There are three parameters that are relatively safe to tune:
199.Em blocksize , bytes/i-node ,
200and
201.Em cylinders/group .
202.Pp
203.Dx
204performs best when using 8K or 16K filesystem block sizes.
205The default filesystem block size is 16K,
206which provides best performance for most applications,
207with the exception of those that perform random access on large files
208(such as database server software).
209Such applications tend to perform better with a smaller block size,
210although modern disk characteristics are such that the performance
211gain from using a smaller block size may not be worth consideration.
212Using a block size larger than 16K
213can cause fragmentation of the buffer cache and
214lead to lower performance.
215.Pp
216The defaults may be unsuitable
217for a filesystem that requires a very large number of i-nodes
218or is intended to hold a large number of very small files.
219Such a filesystem should be created with an 8K or 4K block size.
220This also requires you to specify a smaller
221fragment size.
222We recommend always using a fragment size that is \(18
223the block size (less testing has been done on other fragment size factors).
224The
225.Xr newfs 8
226options for this would be
227.Dq Li "newfs -f 1024 -b 8192 ..." .
228.Pp
229If a large partition is intended to be used to hold fewer, larger files, such
230as database files, you can increase the
231.Em bytes/i-node
232ratio which reduces the number of i-nodes (maximum number of files and
233directories that can be created) for that partition.
234Decreasing the number
235of i-nodes in a filesystem can greatly reduce
236.Xr fsck 8
237recovery times after a crash.
238Do not use this option
239unless you are actually storing large files on the partition, because if you
240overcompensate you can wind up with a filesystem that has lots of free
241space remaining but cannot accommodate any more files.
242Using 32768, 65536, or 262144 bytes/i-node is recommended.
243You can go higher but
244it will have only incremental effects on
245.Xr fsck 8
246recovery times.
247For example,
248.Dq Li "newfs -i 32768 ..." .
249.Pp
250.Xr tunefs 8
251may be used to further tune a filesystem.
252This command can be run in
253single-user mode without having to reformat the filesystem.
254However, this is possibly the most abused program in the system.
255Many people attempt to
256increase available filesystem space by setting the min-free percentage to 0.
257This can lead to severe filesystem fragmentation and we do not recommend
258that you do this.
259Really the only
260.Xr tunefs 8
261option worthwhile here is turning on
262.Em softupdates
263with
264.Dq Li "tunefs -n enable /filesystem" .
265(Note: in
266.Dx ,
267softupdates can be turned on using the
268.Fl U
269option to
270.Xr newfs 8 ,
271and
272.Dx
273installer will typically enable softupdates automatically for
274non-root filesystems).
275Softupdates drastically improves meta-data performance, mainly file
276creation and deletion.
277We recommend enabling softupdates on most filesystems; however, there
278are two limitations to softupdates that you should be aware of when
279determining whether to use it on a filesystem.
280First, softupdates guarantees filesystem consistency in the
281case of a crash but could very easily be several seconds (even a minute!)
282behind on pending writes to the physical disk.
283If you crash you may lose more work
284than otherwise.
285Secondly, softupdates delays the freeing of filesystem
286blocks.
287If you have a filesystem (such as the root filesystem) which is
288close to full, doing a major update of it, e.g.\&
289.Dq Li "make installworld" ,
290can run it out of space and cause the update to fail.
291For this reason, softupdates will not be enabled on the root filesystem
292during a typical install. There is no loss of performance since the root
293filesystem is rarely written to.
294.Pp
295A number of run-time
296.Xr mount 8
297options exist that can help you tune the system.
298The most obvious and most dangerous one is
299.Cm async .
300Do not ever use it; it is far too dangerous.
301A less dangerous and more
302useful
303.Xr mount 8
304option is called
305.Cm noatime .
306.Ux
307filesystems normally update the last-accessed time of a file or
308directory whenever it is accessed.
309This operation is handled in
310.Dx
311with a delayed write and normally does not create a burden on the system.
312However, if your system is accessing a huge number of files on a continuing
313basis the buffer cache can wind up getting polluted with atime updates,
314creating a burden on the system.
315For example, if you are running a heavily
316loaded web site, or a news server with lots of readers, you might want to
317consider turning off atime updates on your larger partitions with this
318.Xr mount 8
319option.
320However, you should not gratuitously turn off atime
321updates everywhere.
322For example, the
323.Pa /var
324filesystem customarily
325holds mailboxes, and atime (in combination with mtime) is used to
326determine whether a mailbox has new mail.
327You might as well leave
328atime turned on for mostly read-only partitions such as
329.Pa /
330and
331.Pa /usr
332as well.
333This is especially useful for
334.Pa /
335since some system utilities
336use the atime field for reporting.
337.Sh STRIPING DISKS
338In larger systems you can stripe partitions from several drives together
339to create a much larger overall partition.
340Striping can also improve
341the performance of a filesystem by splitting I/O operations across two
342or more disks.
343The
344.Xr vinum 8
345and
346.Xr ccdconfig 8
347utilities may be used to create simple striped filesystems.
348Generally
349speaking, striping smaller partitions such as the root and
350.Pa /var/tmp ,
351or essentially read-only partitions such as
352.Pa /usr
353is a complete waste of time.
354You should only stripe partitions that require serious I/O performance,
355typically
356.Pa /var , /home ,
357or custom partitions used to hold databases and web pages.
358Choosing the proper stripe size is also
359important.
360Filesystems tend to store meta-data on power-of-2 boundaries
361and you usually want to reduce seeking rather than increase seeking.
362This
363means you want to use a large off-center stripe size such as 1152 sectors
364so sequential I/O does not seek both disks and so meta-data is distributed
365across both disks rather than concentrated on a single disk.
366If
367you really need to get sophisticated, we recommend using a real hardware
368RAID controller from the list of
369.Dx
370supported controllers.
371.Sh SYSCTL TUNING
372.Xr sysctl 8
373variables permit system behavior to be monitored and controlled at
374run-time.
375Some sysctls simply report on the behavior of the system; others allow
376the system behavior to be modified;
377some may be set at boot time using
378.Xr rc.conf 5 ,
379but most will be set via
380.Xr sysctl.conf 5 .
381There are several hundred sysctls in the system, including many that appear
382to be candidates for tuning but actually are not.
383In this document we will only cover the ones that have the greatest effect
384on the system.
385.Pp
386The
387.Va kern.ipc.shm_use_phys
388sysctl defaults to 0 (off) and may be set to 0 (off) or 1 (on).
389Setting
390this parameter to 1 will cause all System V shared memory segments to be
391mapped to unpageable physical RAM.
392This feature only has an effect if you
393are either (A) mapping small amounts of shared memory across many (hundreds)
394of processes, or (B) mapping large amounts of shared memory across any
395number of processes.
396This feature allows the kernel to remove a great deal
397of internal memory management page-tracking overhead at the cost of wiring
398the shared memory into core, making it unswappable.
399.Pp
400The
401.Va vfs.write_behind
402sysctl defaults to 1 (on). This tells the filesystem to issue media
403writes as full clusters are collected, which typically occurs when writing
404large sequential files. The idea is to avoid saturating the buffer
405cache with dirty buffers when it would not benefit I/O performance. However,
406this may stall processes and under certain circumstances you may wish to turn
407it off.
408.Pp
409The
410.Va vfs.hirunningspace
411sysctl determines how much outstanding write I/O may be queued to
412disk controllers system wide at any given instance. The default is
413usually sufficient but on machines with lots of disks you may want to bump
414it up to four or five megabytes. Note that setting too high a value
415(exceeding the buffer cache's write threshold) can lead to extremely
416bad clustering performance. Do not set this value arbitrarily high! Also,
417higher write queueing values may add latency to reads occurring at the same
418time.
419.Pp
420There are various other buffer-cache and VM page cache related sysctls.
421We do not recommend modifying these values.
422As of
423.Fx 4.3 ,
424the VM system does an extremely good job tuning itself.
425.Pp
426The
427.Va net.inet.tcp.sendspace
428and
429.Va net.inet.tcp.recvspace
430sysctls are of particular interest if you are running network intensive
431applications.
432They control the amount of send and receive buffer space
433allowed for any given TCP connection.
434The default sending buffer is 32K; the default receiving buffer
435is 64K.
436You can often
437improve bandwidth utilization by increasing the default at the cost of
438eating up more kernel memory for each connection.
439We do not recommend
440increasing the defaults if you are serving hundreds or thousands of
441simultaneous connections because it is possible to quickly run the system
442out of memory due to stalled connections building up.
443But if you need
444high bandwidth over a fewer number of connections, especially if you have
445gigabit Ethernet, increasing these defaults can make a huge difference.
446You can adjust the buffer size for incoming and outgoing data separately.
447For example, if your machine is primarily doing web serving you may want
448to decrease the recvspace in order to be able to increase the
449sendspace without eating too much kernel memory.
450Note that the routing table (see
451.Xr route 8 )
452can be used to introduce route-specific send and receive buffer size
453defaults.
454.Pp
455As an additional management tool you can use pipes in your
456firewall rules (see
457.Xr ipfw 8 )
458to limit the bandwidth going to or from particular IP blocks or ports.
459For example, if you have a T1 you might want to limit your web traffic
460to 70% of the T1's bandwidth in order to leave the remainder available
461for mail and interactive use.
462Normally a heavily loaded web server
463will not introduce significant latencies into other services even if
464the network link is maxed out, but enforcing a limit can smooth things
465out and lead to longer term stability.
466Many people also enforce artificial
467bandwidth limitations in order to ensure that they are not charged for
468using too much bandwidth.
469.Pp
470Setting the send or receive TCP buffer to values larger then 65535 will result
471in a marginal performance improvement unless both hosts support the window
472scaling extension of the TCP protocol, which is controlled by the
473.Va net.inet.tcp.rfc1323
474sysctl.
475These extensions should be enabled and the TCP buffer size should be set
476to a value larger than 65536 in order to obtain good performance from
477certain types of network links; specifically, gigabit WAN links and
478high-latency satellite links.
479RFC 1323 support is enabled by default.
480.Pp
481The
482.Va net.inet.tcp.always_keepalive
483sysctl determines whether or not the TCP implementation should attempt
484to detect dead TCP connections by intermittently delivering
485.Dq keepalives
486on the connection.
487By default, this is disabled for all applications, only applications
488that specifically request keepalives will use them.
489In most environments, TCP keepalives will improve the management of
490system state by expiring dead TCP connections, particularly for
491systems serving dialup users who may not always terminate individual
492TCP connections before disconnecting from the network.
493However, in some environments, temporary network outages may be
494incorrectly identified as dead sessions, resulting in unexpectedly
495terminated TCP connections.
496In such environments, setting the sysctl to 0 may reduce the occurrence of
497TCP session disconnections.
498.Pp
499The
500.Va net.inet.tcp.delayed_ack
501TCP feature is largely misunderstood. Historically speaking this feature
502was designed to allow the acknowledgement to transmitted data to be returned
503along with the response. For example, when you type over a remote shell
504the acknowledgement to the character you send can be returned along with the
505data representing the echo of the character. With delayed acks turned off
506the acknowledgement may be sent in its own packet before the remote service
507has a chance to echo the data it just received. This same concept also
508applies to any interactive protocol (e.g. SMTP, WWW, POP3) and can cut the
509number of tiny packets flowing across the network in half. The
510.Dx
511delayed-ack implementation also follows the TCP protocol rule that
512at least every other packet be acknowledged even if the standard 100ms
513timeout has not yet passed. Normally the worst a delayed ack can do is
514slightly delay the teardown of a connection, or slightly delay the ramp-up
515of a slow-start TCP connection. While we aren't sure we believe that
516the several FAQs related to packages such as SAMBA and SQUID which advise
517turning off delayed acks may be refering to the slow-start issue.
518.Pp
519The
520.Va net.inet.tcp.inflight_enable
521sysctl turns on bandwidth delay product limiting for all TCP connections.
522The system will attempt to calculate the bandwidth delay product for each
523connection and limit the amount of data queued to the network to just the
524amount required to maintain optimum throughput. This feature is useful
525if you are serving data over modems, GigE, or high speed WAN links (or
526any other link with a high bandwidth*delay product), especially if you are
527also using window scaling or have configured a large send window. If
528you enable this option you should also be sure to set
529.Va net.inet.tcp.inflight_debug
530to 0 (disable debugging), and for production use setting
531.Va net.inet.tcp.inflight_min
532to at least 6144 may be beneficial. Note, however, that setting high
533minimums may effectively disable bandwidth limiting depending on the link.
534The limiting feature reduces the amount of data built up in intermediate
535router and switch packet queues as well as reduces the amount of data built
536up in the local host's interface queue. With fewer packets queued up,
537interactive connections, especially over slow modems, will also be able
538to operate with lower round trip times. However, note that this feature
539only affects data transmission (uploading / server-side). It does not
540affect data reception (downloading).
541.Pp
542Adjusting
543.Va net.inet.tcp.inflight_stab
544is not recommended.
545This parameter defaults to 20, representing 2 maximal packets added
546to the bandwidth delay product window calculation. The additional
547window is required to stabilize the algorithm and improve responsiveness
548to changing conditions, but it can also result in higher ping times
549over slow links (though still much lower then you would get without
550the inflight algorithm). In such cases you may
551wish to try reducing this parameter to 15, 10, or 5, and you may also
552have to reduce
553.Va net.inet.tcp.inflight_min
554(for example, to 3500) to get the desired effect. Reducing these parameters
555should be done as a last resort only.
556.Pp
557The
558.Va net.inet.ip.portrange.*
559sysctls control the port number ranges automatically bound to TCP and UDP
560sockets. There are three ranges: A low range, a default range, and a
561high range, selectable via an IP_PORTRANGE setsockopt() call. Most
562network programs use the default range which is controlled by
563.Va net.inet.ip.portrange.first
564and
565.Va net.inet.ip.portrange.last ,
566which defaults to 1024 and 5000 respectively. Bound port ranges are
567used for outgoing connections and it is possible to run the system out
568of ports under certain circumstances. This most commonly occurs when you are
569running a heavily loaded web proxy. The port range is not an issue
570when running serves which handle mainly incoming connections such as a
571normal web server, or has a limited number of outgoing connections such
572as a mail relay. For situations where you may run yourself out of
573ports we recommend increasing
574.Va net.inet.ip.portrange.last
575modestly. A value of 10000 or 20000 or 30000 may be reasonable. You should
576also consider firewall effects when changing the port range. Some firewalls
577may block large ranges of ports (usually low-numbered ports) and expect systems
578to use higher ranges of ports for outgoing connections. For this reason
579we do not recommend that
580.Va net.inet.ip.portrange.first
581be lowered.
582.Pp
583The
584.Va kern.ipc.somaxconn
585sysctl limits the size of the listen queue for accepting new TCP connections.
586The default value of 128 is typically too low for robust handling of new
587connections in a heavily loaded web server environment.
588For such environments,
589we recommend increasing this value to 1024 or higher.
590The service daemon
591may itself limit the listen queue size (e.g.\&
592.Xr sendmail 8 ,
593apache) but will
594often have a directive in its configuration file to adjust the queue size up.
595Larger listen queues also do a better job of fending off denial of service
596attacks.
597.Pp
598The
599.Va kern.maxfiles
600sysctl determines how many open files the system supports.
601The default is
602typically a few thousand but you may need to bump this up to ten or twenty
603thousand if you are running databases or large descriptor-heavy daemons.
604The read-only
605.Va kern.openfiles
606sysctl may be interrogated to determine the current number of open files
607on the system.
608.Pp
609The
610.Va vm.swap_idle_enabled
611sysctl is useful in large multi-user systems where you have lots of users
612entering and leaving the system and lots of idle processes.
613Such systems
614tend to generate a great deal of continuous pressure on free memory reserves.
615Turning this feature on and adjusting the swapout hysteresis (in idle
616seconds) via
617.Va vm.swap_idle_threshold1
618and
619.Va vm.swap_idle_threshold2
620allows you to depress the priority of pages associated with idle processes
621more quickly then the normal pageout algorithm.
622This gives a helping hand
623to the pageout daemon.
624Do not turn this option on unless you need it,
625because the tradeoff you are making is to essentially pre-page memory sooner
626rather than later, eating more swap and disk bandwidth.
627In a small system
628this option will have a detrimental effect but in a large system that is
629already doing moderate paging this option allows the VM system to stage
630whole processes into and out of memory more easily.
631.Sh LOADER TUNABLES
632Some aspects of the system behavior may not be tunable at runtime because
633memory allocations they perform must occur early in the boot process.
634To change loader tunables, you must set their values in
635.Xr loader.conf 5
636and reboot the system.
637.Pp
638.Va kern.maxusers
639controls the scaling of a number of static system tables, including defaults
640for the maximum number of open files, sizing of network memory resources, etc.
641On
642.Dx ,
643.Va kern.maxusers
644is automatically sized at boot based on the amount of memory available in
645the system, and may be determined at run-time by inspecting the value of the
646read-only
647.Va kern.maxusers
648sysctl.
649Some sites will require larger or smaller values of
650.Va kern.maxusers
651and may set it as a loader tunable; values of 64, 128, and 256 are not
652uncommon.
653We do not recommend going above 256 unless you need a huge number
654of file descriptors; many of the tunable values set to their defaults by
655.Va kern.maxusers
656may be individually overridden at boot-time or run-time as described
657elsewhere in this document.
658.Pp
659The
660.Va kern.dfldsiz
661and
662.Va kern.dflssiz
663tunables set the default soft limits for process data and stack size
664respectively.
665Processes may increase these up to the hard limits by calling
666.Xr setrlimit 2 .
667The
668.Va kern.maxdsiz ,
669.Va kern.maxssiz ,
670and
671.Va kern.maxtsiz
672tunables set the hard limits for process data, stack, and text size
673respectively; processes may not exceed these limits.
674The
675.Va kern.sgrowsiz
676tunable controls how much the stack segment will grow when a process
677needs to allocate more stack.
678.Pp
679.Va kern.ipc.nmbclusters
680may be adjusted to increase the number of network mbufs the system is
681willing to allocate.
682Each cluster represents approximately 2K of memory,
683so a value of 1024 represents 2M of kernel memory reserved for network
684buffers.
685You can do a simple calculation to figure out how many you need.
686If you have a web server which maxes out at 1000 simultaneous connections,
687and each connection eats a 16K receive and 16K send buffer, you need
688approximately 32MB worth of network buffers to deal with it.
689A good rule of
690thumb is to multiply by 2, so 32MBx2 = 64MB/2K = 32768.
691So for this case
692you would want to set
693.Va kern.ipc.nmbclusters
694to 32768.
695We recommend values between
6961024 and 4096 for machines with moderates amount of memory, and between 4096
697and 32768 for machines with greater amounts of memory.
698Under no circumstances
699should you specify an arbitrarily high value for this parameter, it could
700lead to a boot-time crash.
701The
702.Fl m
703option to
704.Xr netstat 1
705may be used to observe network cluster use.
706.Pp
707More and more programs are using the
708.Xr sendfile 2
709system call to transmit files over the network.
710The
711.Va kern.ipc.nsfbufs
712sysctl controls the number of filesystem buffers
713.Xr sendfile 2
714is allowed to use to perform its work.
715This parameter nominally scales
716with
717.Va kern.maxusers
718so you should not need to modify this parameter except under extreme
719circumstances.
720.Sh KERNEL CONFIG TUNING
721There are a number of kernel options that you may have to fiddle with in
722a large-scale system.
723In order to change these options you need to be
724able to compile a new kernel from source.
725The
726.Xr config 8
727manual page and the handbook are good starting points for learning how to
728do this.
729Generally the first thing you do when creating your own custom
730kernel is to strip out all the drivers and services you do not use.
731Removing things like
732.Dv INET6
733and drivers you do not have will reduce the size of your kernel, sometimes
734by a megabyte or more, leaving more memory available for applications.
735.Pp
736.Dv SCSI_DELAY
737may be used to reduce system boot times.
738The default is fairly high and
739can be responsible for 15+ seconds of delay in the boot process.
740Reducing
741.Dv SCSI_DELAY
742to 5 seconds usually works (especially with modern drives).
743.Pp
744There are a number of
745.Dv *_CPU
746options that can be commented out.
747If you only want the kernel to run
748on a Pentium class CPU, you can easily remove
749.Dv I486_CPU ,
750but only remove
751.Dv I586_CPU
752if you are sure your CPU is being recognized as a Pentium II or better.
753Some clones may be recognized as a Pentium and not be able to boot
754without those options.
755If it works, great!
756The operating system
757will be able to better-use higher-end CPU features for MMU, task switching,
758timebase, and even device operations.
759Additionally, higher-end CPUs support
7604MB MMU pages, which the kernel uses to map the kernel itself into memory,
761increasing its efficiency under heavy syscall loads.
762.Sh IDE WRITE CACHING
763.Fx 4.3
764flirted with turning off IDE write caching.
765This reduced write bandwidth
766to IDE disks but was considered necessary due to serious data consistency
767issues introduced by hard drive vendors.
768Basically the problem is that
769IDE drives lie about when a write completes.
770With IDE write caching turned
771on, IDE hard drives will not only write data to disk out of order, they
772will sometimes delay some of the blocks indefinitely under heavy disk
773load.
774A crash or power failure can result in serious filesystem
775corruption.
776So our default was changed to be safe.
777Unfortunately, the
778result was such a huge loss in performance that we caved in and changed the
779default back to on after the release.
780You should check the default on
781your system by observing the
782.Va hw.ata.wc
783sysctl variable.
784If IDE write caching is turned off, you can turn it back
785on by setting the
786.Va hw.ata.wc
787loader tunable to 1.
788More information on tuning the ATA driver system may be found in the
789.Xr ata 4
790man page.
791.Pp
792There is a new experimental feature for IDE hard drives called
793.Va hw.ata.tags
794(you also set this in the boot loader) which allows write caching to be safely
795turned on.
796This brings SCSI tagging features to IDE drives.
797As of this
798writing only IBM DPTA and DTLA drives support the feature.
799Warning!
800These
801drives apparently have quality control problems and I do not recommend
802purchasing them at this time.
803If you need performance, go with SCSI.
804.Sh CPU, MEMORY, DISK, NETWORK
805The type of tuning you do depends heavily on where your system begins to
806bottleneck as load increases.
807If your system runs out of CPU (idle times
808are perpetually 0%) then you need to consider upgrading the CPU or moving to
809an SMP motherboard (multiple CPU's), or perhaps you need to revisit the
810programs that are causing the load and try to optimize them.
811If your system
812is paging to swap a lot you need to consider adding more memory.
813If your
814system is saturating the disk you typically see high CPU idle times and
815total disk saturation.
816.Xr systat 1
817can be used to monitor this.
818There are many solutions to saturated disks:
819increasing memory for caching, mirroring disks, distributing operations across
820several machines, and so forth.
821If disk performance is an issue and you
822are using IDE drives, switching to SCSI can help a great deal.
823While modern
824IDE drives compare with SCSI in raw sequential bandwidth, the moment you
825start seeking around the disk SCSI drives usually win.
826.Pp
827Finally, you might run out of network suds.
828The first line of defense for
829improving network performance is to make sure you are using switches instead
830of hubs, especially these days where switches are almost as cheap.
831Hubs
832have severe problems under heavy loads due to collision backoff and one bad
833host can severely degrade the entire LAN.
834Second, optimize the network path
835as much as possible.
836For example, in
837.Xr firewall 7
838we describe a firewall protecting internal hosts with a topology where
839the externally visible hosts are not routed through it.
840Use 100BaseT rather
841than 10BaseT, or use 1000BaseT rather than 100BaseT, depending on your needs.
842Most bottlenecks occur at the WAN link (e.g.\&
843modem, T1, DSL, whatever).
844If expanding the link is not an option it may be possible to use the
845.Xr dummynet 4
846feature to implement peak shaving or other forms of traffic shaping to
847prevent the overloaded service (such as web services) from affecting other
848services (such as email), or vice versa.
849In home installations this could
850be used to give interactive traffic (your browser,
851.Xr ssh 1
852logins) priority
853over services you export from your box (web services, email).
854.Sh SEE ALSO
855.Xr netstat 1 ,
856.Xr systat 1 ,
857.Xr ata 4 ,
858.Xr dummynet 4 ,
859.Xr login.conf 5 ,
860.Xr rc.conf 5 ,
861.Xr sysctl.conf 5 ,
862.Xr firewall 7 ,
863.Xr hier 7 ,
864.Xr boot 8 ,
865.Xr ccdconfig 8 ,
866.Xr config 8 ,
867.Xr disklabel 8 ,
868.Xr fsck 8 ,
869.Xr ifconfig 8 ,
870.Xr ipfw 8 ,
871.Xr loader 8 ,
872.Xr mount 8 ,
873.Xr newfs 8 ,
874.Xr route 8 ,
875.Xr sysctl 8 ,
876.Xr tunefs 8 ,
877.Xr vinum 8
878.Sh HISTORY
879The
880.Nm
881manual page was originally written by
882.An Matthew Dillon
883and first appeared
884in
885.Fx 4.3 ,
886May 2001.