From 33e8237d84dfbf75b9df24e671bd741184880532 Mon Sep 17 00:00:00 2001 From: jth Date: Tue, 10 Mar 2009 11:48:57 -0700 Subject: [PATCH] FIX: manpage links --- .../handbook-configtuning-kernel-limits.mdwn | 131 ++++++++++++------ 1 file changed, 87 insertions(+), 44 deletions(-) diff --git a/docs/handbook/handbook-configtuning-kernel-limits.mdwn b/docs/handbook/handbook-configtuning-kernel-limits.mdwn index 5853ccb4..133407f4 100644 --- a/docs/handbook/handbook-configtuning-kernel-limits.mdwn +++ b/docs/handbook/handbook-configtuning-kernel-limits.mdwn @@ -1,44 +1,87 @@ - - -## 6.13 Tuning Kernel Limits - -### 6.13.1 File/Process Limits - -#### 6.13.1.1 `kern.maxfiles` - -`kern.maxfiles` can be raised or lowered based upon your system requirements. This variable indicates the maximum number of file descriptors on your system. When the file descriptor table is full, ***`file: table is full`*** will show up repeatedly in the system message buffer, which can be viewed with the `dmesg` command. - -Each open file, socket, or fifo uses one file descriptor. A large-scale production server may easily require many thousands of file descriptors, depending on the kind and number of services running concurrently. - -`kern.maxfile`'s default value is dictated by the `MAXUSERS` option in your kernel configuration file. `kern.maxfiles` grows proportionally to the value of `MAXUSERS`. When compiling a custom kernel, it is a good idea to set this kernel configuration option according to the uses of your system. From this number, the kernel is given most of its pre-defined limits. Even though a production machine may not actually have 256 users connected at once, the resources needed may be similar to a high-scale web server. - - **Note:** Setting `MAXUSERS` to `0` in your kernel configuration file will choose a reasonable default value based on the amount of RAM present in your system. It is set to 0 in the default GENERIC kernel. - -#### 6.13.1.2 `kern.ipc.somaxconn` - -The `kern.ipc.somaxconn` sysctl variable limits the size of the listen queue for accepting new TCP connections. The default value of `128` is typically too low for robust handling of new connections in a heavily loaded web server environment. For such environments, it is recommended to increase this value to `1024` or higher. The service daemon may itself limit the listen queue size (e.g. [sendmail(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#sendmail§ion8), or **Apache** ) but will often have a directive in its configuration file to adjust the queue size. Large listen queues also do a better job of avoiding Denial of Service (DoS) attacks. - -### 6.13.2 Network Limits - -The `NMBCLUSTERS` kernel configuration option dictates the amount of network Mbufs available to the system. A heavily-trafficked server with a low number of Mbufs will hinder DragonFly's ability. Each cluster represents approximately 2 K of memory, so a value of 1024 represents 2 megabytes of kernel memory reserved for network buffers. A simple calculation can be done to figure out how many are needed. If you have a web server which maxes out at 1000 simultaneous connections, and each connection eats a 16 K receive and 16 K send buffer, you need approximately 32 MB worth of network buffers to cover the web server. A good rule of thumb is to multiply by 2, so 2x32 MB / 2 KB # 64 MB / 2 kB  32768. We recommend values between 4096 and 32768 for machines with greater amounts of memory. Under no circumstances should you specify an arbitrarily high value for this parameter as it could lead to a boot time crash. The `-m` option to [netstat(1)](http://leaf.dragonflybsd.org/cgi/web-man?command=netstat§ion=1) may be used to observe network cluster use. `kern.ipc.nmbclusters` loader tunable should be used to tune this at boot time. - -For busy servers that make extensive use of the [sendfile(2)](http://leaf.dragonflybsd.org/cgi/web-man?command#sendfile§ion2) system call, it may be necessary to increase the number of [sendfile(2)](http://leaf.dragonflybsd.org/cgi/web-man?command=sendfile§ion=2) buffers via the `NSFBUFS` kernel configuration option or by setting its value in `/boot/loader.conf` (see [loader(8)](http://leaf.dragonflybsd.org/cgi/web-man?command=loader§ion=8) for details). A common indicator that this parameter needs to be adjusted is when processes are seen in the `sfbufa` state. The sysctl variable `kern.ipc.nsfbufs` is a read-only glimpse at the kernel configured variable. This parameter nominally scales with `kern.maxusers`, however it may be necessary to tune accordingly. - - **Important:** Even though a socket has been marked as non-blocking, calling [sendfile(2)](http://leaf.dragonflybsd.org/cgi/web-man?command#sendfile§ion2) on the non-blocking socket may result in the [sendfile(2)](http://leaf.dragonflybsd.org/cgi/web-man?command=sendfile§ion=2) call blocking until enough `struct sf_buf`'s are made available. - -#### 6.13.2.1 `net.inet.ip.portrange.*` - -The `net.inet.ip.portrange.*` sysctl variables control the port number ranges automatically bound to TCP and UDP sockets. There are three ranges: a low range, a default range, and a high range. Most network programs use the default range which is controlled by the `net.inet.ip.portrange.first` and `net.inet.ip.portrange.last`, which default to 1024 and 5000, respectively. Bound port ranges are used for outgoing connections, and it is possible to run the system out of ports under certain circumstances. This most commonly occurs when you are running a heavily loaded web proxy. The port range is not an issue when running servers which handle mainly incoming connections, such as a normal web server, or has a limited number of outgoing connections, such as a mail relay. For situations where you may run yourself out of ports, it is recommended to increase `net.inet.ip.portrange.last` modestly. A value of `10000`, `20000` or `30000` may be reasonable. You should also consider firewall effects when changing the port range. Some firewalls may block large ranges of ports (usually low-numbered ports) and expect systems to use higher ranges of ports for outgoing connections -- for this reason it is recommended that `net.inet.ip.portrange.first` be lowered. - -#### 6.13.2.2 TCP Bandwidth Delay Product - -The TCP Bandwidth Delay Product Limiting is similar to TCP/Vegas in NetBSD. It can be enabled by setting `net.inet.tcp.inflight_enable` sysctl variable to `1`. The system will attempt to calculate the bandwidth delay product for each connection and limit the amount of data queued to the network to just the amount required to maintain optimum throughput. - -This feature is useful if you are serving data over modems, Gigabit Ethernet, or even high speed WAN links (or any other link with a high bandwidth delay product), especially if you are also using window scaling or have configured a large send window. If you enable this option, you should also be sure to set `net.inet.tcp.inflight_debug` to `0` (disable debugging), and for production use setting `net.inet.tcp.inflight_min` to at least `6144` may be beneficial. However, note that setting high minimums may effectively disable bandwidth limiting depending on the link. The limiting feature reduces the amount of data built up in intermediate route and switch packet queues as well as reduces the amount of data built up in the local host's interface queue. With fewer packets queued up, interactive connections, especially over slow modems, will also be able to operate with lower ***Round Trip Times***. However, note that this feature only effects data transmission (uploading / server side). It has no effect on data reception (downloading). - -Adjusting `net.inet.tcp.inflight_stab` is ***not*** recommended. This parameter defaults to 20, representing 2 maximal packets added to the bandwidth delay product window calculation. The additional window is required to stabilize the algorithm and improve responsiveness to changing conditions, but it can also result in higher ping times over slow links (though still much lower than you would get without the inflight algorithm). In such cases, you may wish to try reducing this parameter to 15, 10, or 5; and may also have to reduce `net.inet.tcp.inflight_min` (for example, to 3500) to get the desired effect. Reducing these parameters should be done as a last resort only. - - - -CategoryHandbook -CategoryHandbook-configuration + + + +## 6.13 Tuning Kernel Limits + + + +### 6.13.1 File/Process Limits + + + +#### 6.13.1.1 `kern.maxfiles` + + + +`kern.maxfiles` can be raised or lowered based upon your system requirements. This variable indicates the maximum number of file descriptors on your system. When the file descriptor table is full, ***`file: table is full`*** will show up repeatedly in the system message buffer, which can be viewed with the `dmesg` command. + + + +Each open file, socket, or fifo uses one file descriptor. A large-scale production server may easily require many thousands of file descriptors, depending on the kind and number of services running concurrently. + + + +`kern.maxfile`'s default value is dictated by the `MAXUSERS` option in your kernel configuration file. `kern.maxfiles` grows proportionally to the value of `MAXUSERS`. When compiling a custom kernel, it is a good idea to set this kernel configuration option according to the uses of your system. From this number, the kernel is given most of its pre-defined limits. Even though a production machine may not actually have 256 users connected at once, the resources needed may be similar to a high-scale web server. + + + + **Note:** Setting `MAXUSERS` to `0` in your kernel configuration file will choose a reasonable default value based on the amount of RAM present in your system. It is set to 0 in the default GENERIC kernel. + + + +#### 6.13.1.2 `kern.ipc.somaxconn` + + + +The `kern.ipc.somaxconn` sysctl variable limits the size of the listen queue for accepting new TCP connections. The default value of `128` is typically too low for robust handling of new connections in a heavily loaded web server environment. For such environments, it is recommended to increase this value to `1024` or higher. The service daemon may itself limit the listen queue size (e.g. [sendmail(8)](http://leaf.dragonflybsd.org/cgi/web-man?command=sendmail§ion=8), or **Apache** ) but will often have a directive in its configuration file to adjust the queue size. Large listen queues also do a better job of avoiding Denial of Service (DoS) attacks. + + + +### 6.13.2 Network Limits + + + +The `NMBCLUSTERS` kernel configuration option dictates the amount of network Mbufs available to the system. A heavily-trafficked server with a low number of Mbufs will hinder DragonFly's ability. Each cluster represents approximately 2 K of memory, so a value of 1024 represents 2 megabytes of kernel memory reserved for network buffers. A simple calculation can be done to figure out how many are needed. If you have a web server which maxes out at 1000 simultaneous connections, and each connection eats a 16 K receive and 16 K send buffer, you need approximately 32 MB worth of network buffers to cover the web server. A good rule of thumb is to multiply by 2, so 2x32 MB / 2 KB # 64 MB / 2 kB  32768. We recommend values between 4096 and 32768 for machines with greater amounts of memory. Under no circumstances should you specify an arbitrarily high value for this parameter as it could lead to a boot time crash. The `-m` option to [netstat(1)](http://leaf.dragonflybsd.org/cgi/web-man?command=netstat§ion=1) may be used to observe network cluster use. `kern.ipc.nmbclusters` loader tunable should be used to tune this at boot time. + + + +For busy servers that make extensive use of the [sendfile(2)](http://leaf.dragonflybsd.org/cgi/web-man?command=sendfile§ion=2) system call, it may be necessary to increase the number of [sendfile(2)](http://leaf.dragonflybsd.org/cgi/web-man?command=sendfile§ion=2) buffers via the `NSFBUFS` kernel configuration option or by setting its value in `/boot/loader.conf` (see [loader(8)](http://leaf.dragonflybsd.org/cgi/web-man?command=loader§ion=8) for details). A common indicator that this parameter needs to be adjusted is when processes are seen in the `sfbufa` state. The sysctl variable `kern.ipc.nsfbufs` is a read-only glimpse at the kernel configured variable. This parameter nominally scales with `kern.maxusers`, however it may be necessary to tune accordingly. + + + + **Important:** Even though a socket has been marked as non-blocking, calling [sendfile(2)](http://leaf.dragonflybsd.org/cgi/web-man?command=sendfile§ion=2) on the non-blocking socket may result in the [sendfile(2)](http://leaf.dragonflybsd.org/cgi/web-man?command=sendfile§ion=2) call blocking until enough `struct sf_buf`'s are made available. + + + +#### 6.13.2.1 `net.inet.ip.portrange.*` + + + +The `net.inet.ip.portrange.*` sysctl variables control the port number ranges automatically bound to TCP and UDP sockets. There are three ranges: a low range, a default range, and a high range. Most network programs use the default range which is controlled by the `net.inet.ip.portrange.first` and `net.inet.ip.portrange.last`, which default to 1024 and 5000, respectively. Bound port ranges are used for outgoing connections, and it is possible to run the system out of ports under certain circumstances. This most commonly occurs when you are running a heavily loaded web proxy. The port range is not an issue when running servers which handle mainly incoming connections, such as a normal web server, or has a limited number of outgoing connections, such as a mail relay. For situations where you may run yourself out of ports, it is recommended to increase `net.inet.ip.portrange.last` modestly. A value of `10000`, `20000` or `30000` may be reasonable. You should also consider firewall effects when changing the port range. Some firewalls may block large ranges of ports (usually low-numbered ports) and expect systems to use higher ranges of ports for outgoing connections -- for this reason it is recommended that `net.inet.ip.portrange.first` be lowered. + + + +#### 6.13.2.2 TCP Bandwidth Delay Product + + + +The TCP Bandwidth Delay Product Limiting is similar to TCP/Vegas in NetBSD. It can be enabled by setting `net.inet.tcp.inflight_enable` sysctl variable to `1`. The system will attempt to calculate the bandwidth delay product for each connection and limit the amount of data queued to the network to just the amount required to maintain optimum throughput. + + + +This feature is useful if you are serving data over modems, Gigabit Ethernet, or even high speed WAN links (or any other link with a high bandwidth delay product), especially if you are also using window scaling or have configured a large send window. If you enable this option, you should also be sure to set `net.inet.tcp.inflight_debug` to `0` (disable debugging), and for production use setting `net.inet.tcp.inflight_min` to at least `6144` may be beneficial. However, note that setting high minimums may effectively disable bandwidth limiting depending on the link. The limiting feature reduces the amount of data built up in intermediate route and switch packet queues as well as reduces the amount of data built up in the local host's interface queue. With fewer packets queued up, interactive connections, especially over slow modems, will also be able to operate with lower ***Round Trip Times***. However, note that this feature only effects data transmission (uploading / server side). It has no effect on data reception (downloading). + + + +Adjusting `net.inet.tcp.inflight_stab` is ***not*** recommended. This parameter defaults to 20, representing 2 maximal packets added to the bandwidth delay product window calculation. The additional window is required to stabilize the algorithm and improve responsiveness to changing conditions, but it can also result in higher ping times over slow links (though still much lower than you would get without the inflight algorithm). In such cases, you may wish to try reducing this parameter to 15, 10, or 5; and may also have to reduce `net.inet.tcp.inflight_min` (for example, to 3500) to get the desired effect. Reducing these parameters should be done as a last resort only. + + + + + + + +CategoryHandbook + +CategoryHandbook-configuration + -- 2.41.0