(no commit message)

[ikiwiki.git] / docs / ipfw3 / index.mdwn

[[!meta title="IPFW3 Documentation"]]
[[!meta robots="index, follow"]]


Bill Yuan

12 May, 2018

---

[[!toc  levels=3]]

# Introduction

[IPFW](https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls-ipfw.html) is a stateful firewall originally written for FreeBSD. It is comprised of several components, e.g. the kernel firewall filter rule processor and its integrated packet accounting facility, the logging facility, NAT, the [``dummynet(4)``](http://mdoc.su/d/4/dummynet) traffic shaper, a forward facility, a bridge facility, and an ipstealth facility. It is one of the most advanced opensource firewalls.

DragonFly BSD is a logical continuation of the FreeBSD 4.x series, but has diverged significantly from FreeBSD since the fork in 2004, e.g., DragonFly BSD has implemented a new Light Weight Kernel Threads implementation (LWKT) and a lightweight ports/messaging system.

In DragonFly, each CPU has its own thread scheduler. Upon creation, threads are assigned to processors and are never preemptively switched from one processor to another; they are only migrated by the passing of an inter-processor interrupt (IPI) message between the CPUs involved. Inter-processor thread scheduling is also accomplished by sending asynchronous IPI messages. One advantage to this clean compartmentalization of the threading subsystem is that the processors' on-board caches in Symmetric Multiprocessor Systems do not contain duplicated data, allowing for higher performance by giving each processor in the system the ability to use its own cache to store different things to work on.

The LWKT subsystem is being employed to partition work among multiple kernel threads (for example in the networking code there is one thread per protocol per processor), reducing competition by removing the need to share certain resources among various kernel tasks.

This IPFW3 is a rewritten from scratch of the FreeBSD's IPFW for DragonFly BSD. It is in modular design, and inherited the SMP-friendly feature from DragonFly BSD's LWKT, therefore IPFW3 is a lockless and stateful firewall.


## Brief notes on design

IPFW3 is in modular design, with different functionalities implemented in different loadable modules, which can be loaded on requirements. The IPFW3 **core** module only comes with the ``allow`` and ``deny`` actions, which may be loaded manually by:

    kldload ipfw3

More basic firewall filtering features, e.g., *filtering based on source IP*, are implemented in the **basic** module, which may be loaded by:

    kldload ipfw3_basic

Besides the **core** and **basic** modules, there are also the **layer2**, **layer4**, in-kernel **NAT**, and **dummynet** modules.

Note that the corresponding kernel module must be loaded, otherwise, [``ipfw3(8)``](http://mdoc.su/d/8/ipfw3) would complain errors of **unknown/bad command**.

Each module contains 2 parts. library in user-space which will be loaded and parse the command line into rules. and kernel space portion will be invoked when the traffic hit the firewall and it will trigger the correct action according to the firewall rules.

## Compare to FreeBSD's IPFW

IPFW3 not only inherited **most** features from FreeBSD's IPFW, but also introduced lots of new features from OpenBSD's PF and other rivals.

**Much more extensible**

Every filter/action needs to be identified using ID, but there are only 8 bits space to store the ID, so theoretically it can support 256 filters/actions in maximum in FreeBSD' IPFW. While in IPFW3, the space for ID are still the same, but one space introduced to keep the module's ID, so theoretically ipfw3 can have 256 modules and 256 filter/action in each module.

And in IPFW3, both user-space library and kernel space module are implemented with a simple interface, it is quite easy to build your own filter/module by following the interface.

**Much more concise**

The rules of IPFW3 are much more concise. For example, a simple rule of IPFW looks like:

    ipfw add allow ip from any to any
   
where the ``from any to any`` is actually just for more readable.  While IPFW3 supports the same syntax as FreeBSD's IPFW, we recommend to just use the simplified version, e.g.:

    #1. ipfw3 add allow all 
    #2. ipfw3 add allow icmp from 1.1.1.1
    #3. ipfw3 add allow tcp via em0

**Higher Performance**

All modern CPUs are having mutil-cores, and each core are running independently. So the LWKT of DragonFly BSD is the best way to fully utilize the CPU power. by duplicating the environment for each CPU, all the CPU can run as fast as it can without any interference. So it is a lockless and stateful firewall.


# Basic Configuration

## Core Framework

Below actions are directly supported from the core framework.

**accept**  -- accept the traffic

**deny**  -- deny the traffic

the default action of the firewall was compiled in the core framework. but it still can be interfered by below system tunable when the module was loaded into the kernel, e.g.,

    sysctl net.filters_default_to_accept=1


## Basic Module

Below filter/actions are supported in basic module.

**proto** -- matches traffic protocol, it is implicit after the action.

**from** -- matches the source.

Filter ``from`` supports multiple type of parameters:

    from 8.8.8.8 -- match traffic from IP 8.8.8.8
    from table 1    -- match traffic where source IP found in table 1
    from any  -- not filtering
    from me  -- match traffic from the host
    from 192.168.1.0/24 -- match traffic from the IP range
 
**to** -- matches the destination, which supports the same parameters as filter ``from``

**count** -- action count the traffic

**skipto** -- skipto another line in the rules

**forward** -- forward the current traffic to a destination

**in** -- matches the in direction traffic

**out** -- matches the out direction traffic

**via** -- matches the traffic go through an interface

**xmit** -- matches the out direction traffic through an interface

**recv** -- matches the in direction traffic through an interface

**src-port** --matches the src port of TCP/UDP traffic

**dst-port** --matches the dst port of TCP/UDP traffic

**prob** -- randomly match the traffic

**keep-state** -- setup a state in current CPU only

**check-state** -- check the traffic state against the state tables of current CPU

**tag** -- add a tag the traffic

**untag** -- remove the tag from the traffic

**tagged** -- matched the traffic with the tag

**//** -- append some comment at the end of the rule

## Layer2 Module

**layer2** -- matches layer2 traffic

**mac** -- matches layer2 traffic with src and dst MAC addresses

**mac-from** -- matches layer2 traffic with src MAC address (supports lookup table)

**mac-to** -- matches layer2 traffic with dst MAC address (supports lookup table)
 
## Layer4 Module

**tcpflag**  -- matches the TCP flag

**uid** -- matches the sockets owner ID

**gid** -- matches the sockets owner's group ID

**established** --matched the established TCP connection

**bpf**  -- filter traffic with bpf syntax
 
## NAT Module

**nat** -- NAT traffic with pre-defined NAT configuration
 
## Dummynet3 Module

**pipe** -- pipe traffic with pre-defined pipe configuration

**queue** -- queue traffic with pre-defined queue configuration


# Advanced Configuration

## Rule Set

Each rule belongs to one of 32 different sets , numbered 0 to 31. Set 31 is reserved for the default rule.

By default, rules are put in set 0, unless you use the set N attribute when entering a new rule. Sets can be individually and atomically enabled or disabled, so this mechanism permits an easy way to store multiple configurations of the firewall and quickly (and atomically) switch between them.  The command to enable/disable sets is

    ipfw3 set [disable number ...] [enable number ...]

where multiple enable or disable sections can be specified.  Command execution is atomic on all the sets specified in the command.  By default, all sets are enabled.

## Lookup Table

In the following example, We need to create several rules in order to block ICMP traffic from whole range of the network 192.168.0.0/24.

    ipfw3 add deny icmp from 192.168.0.1
    ipfw3 add deny icmp from 192.168.0.2
    ipfw3 add deny icmp from 192.168.0.3
    ...
    ipfw3 add deny icmp from 192.168.0.254

The firewall need to process the 254 lines of rule line by line. in this situation, the lookup table was introduced in order to increase the performance and enhance the usability.

    ipfw3 table 1 type ip  # create a table of id=1 and type=ip
    ipfw3 table 1 append range 192.168.0.0/24
    ipfw3 add deny icmp from table 1
 
and lookup table are supported by multiple filters.


## Forwarding

The forward action will change the next-hop on matching packets to ipaddr, which can be an IP address in dotted quad format or a host name.The search terminates if this rule matches.

If ipaddr it can be is a local addresses, then matching packets will be forwarded to port (or the port number in the packet if one is not specified in the rule) on the local machine. If ipaddr is not a local address, then the port number (if specified) is ignored, and the packet will be forwarded to the remote address, using the route as found in the local routing table for that IP.  Use commas to separate multiple ip addresses.

forward action supports multiple options, it can be `round-robin' or `sticky'.  `sticky' is calculated based on the src ip addresses, and if no forward-option, by default it will be 'random'.

    ipfw3 add forward 192.168.1.1:80,192.168.1.2:80 round-robin tcp from ....
 
Above example can forward the traffic to 2 destination in round-robin.

A forward rule will not match layer-2 packets (those received on ether_input() or ether_output()). The forward action does not change the contents of the packet at all.  In particular, the destination address remains unmodified, so packets forwarded to another system will usually be rejected by that system unless there is a matching rule on that system to capture them.  For packets forwarded locally, the local address of the socket will be set to the original destination address of the packet.  This makes the netstat(1) entry look rather weird but is intended for use with transparent proxy servers.


## BPF Filtering

Berkeley Packet Filter (BPF)'s filtering capabilities are implemented as an interpreter for a machine language for the BPF virtual machine; programs in that language can fetch data from the packet, perform arithmetic operations on data from the packet, and compare the results against constants or against data in the packet or test bits in the results, accepting or rejecting the packet based on the results of those tests. The original paper was written by Steven McCanne and Van Jacobson in 1992 while at Lawrence Berkeley Laboratory.

The ipfw3 firewall integrates with a BPF filter, a just-in-time compilation is used to convert virtual machine instructions into native code, and it will be invoked when the traffic hits a rule with BPF filter.
    
    ipfw3 add allow all bpf "icmp and src 8.8.8.8"
 
the bpf filter can be used to filter all parts of the packet, includes the payload.


## Stateful

Stateful operation is a way for the firewall to dynamically create states for specific flows when packets that match a given pattern are detected. Support for stateful operation comes through the check-state, keep-state of rules.

States are created when a packet matches a keep-state rule, causing the creation of a dynamic rule which will match all and only packets with a given protocol between a src-ip/src-port dst-ip/dst-port pair of addresses (src and dst are used here only to denote the initial match addresses, but they are completely equivalent afterwards).  Dynamic rules will be checked at the first check-state, keep-state or limit occurrence, and the action performed upon a match will be the same as in the parent rule.

Note that no additional attributes other than protocol and IP addresses and ports are checked on dynamic rules.

The typical use of dynamic rules is to keep a closed firewall configuration, but let the first TCP SYN packet from the inside network install a state for the flow so that packets belonging to that session will be allowed through the firewall:

    ipfw3 add check-state
    ipfw3 add allow tcp from my-subnet to any keep-state
    ipfw3 add deny tcp from any to any

A similar approach can be used for UDP, where an UDP packet coming from the inside will install a dynamic rule to let the response through the firewall:

    ipfw3 add check-state
    ipfw3 add allow udp from my-subnet to any keep-state
    ipfw3 add deny udp from any to any

States expire after some time, which depends on the status of the flow and the setting of some sysctl variables.  See Section SYSCTL VARIABLES for more details.  For TCP sessions, dynamic rules can be instructed to periodically send keepalive packets to refresh the state of the rule when it is about to expire.

States can be added/deleted using the ipfw3 utility. and the inserted states will be hosted on the current CPU only. once the live time expired, the state will be purged by the housekeeping process and will not be able to influence the traffic.
 
    ipfw3 state add rule 1000 udp 192.168.1.100:0 8.8.8.8:53 expiry 600

Above example can immediately  insert a state which will link to rule 1000 for the UDP traffic which is from 192.168.1.100 to 8.8.8.8:53, and the state will be expired in 600 seconds if no traffic update the states.
  
## In-Kernel NAT

Ipfw3 supports in-kernel NAT using the kernel version of libalias which is a moderately portable set of functions designed to assist in the process of IP masquerading and network address translation. Out-going packets from a local network with unregistered IP addresses can be aliased to appear as if they came from an accessible IP address. Incoming packets are then de-aliased so that they are sent to the correct machine on the local network. 

A certain amount of flexibility is built into the packet aliasing engine. In the simplest mode of operation, a many-to-one address mapping takes place between the local network and the packet aliasing host.  This is known as IP masquerading. In addition, one-to-one mappings between local and public addresses can also be implemented, which is known as static NAT.  In between these extremes, different groups of private addresses can be linked to different public addresses, comprising several distinct many-to-one mappings.  Also, a given public address and port can be statically redirected to a private address/port.

In ipfw3, each CPU has its own context, and by storing the alias link records into different context according to the CPU, the lock can be removed in order to achieve the best performance. and due to the nature of NAT, the outgoing and returning packets are possible to be handled by different CPUs, to ensure the return traffic and be translated back to the correct address, the newly created alias_link are required to be duplicated and inserted into contexts of both CPUs. 

So this in-kernel NAT in DragonFly BSD is the only true lockless in-kernel NAT amount all common opensource operating systems.


## IPFW3sync

IPFW3sync is the facility in IPFW3 which can synchronize firewall states between machines running ipfw3 firewall for high availability. It can be used together with CARP to make ensure a backup firewall has the same states as the main firewall. When the master machine in the firewall cluster dies, the slave machine will be able to takeover the services and accept current connections without loss.

In order to use this synchronization feature, the firewalls need to be configured into IPFW3sync **centre** and/or **edge** nodes, therefore, the **centre** firewall will continuously sync its states to the **edge**s using UDP protocol.

Use below commands to configure an IPFW3sync *edge* node, which will listen on the UDP port 5000:

    ipfw3 sync edge 5000
    ipfw3 sync start edge

Then configure an IPFW3sync *centre* node. Here the *centre* firewall will automatically sync its states to *edge* nodes 192.168.1.1:5000 and 192.168.1.2:5001 :

    ipfw3 sync centre 192.168.1.1:5000,192.168.1.2:5001
    ipfw3 sync start centre

This command to verify whether the IPFW3sync *centre* is able to send the test message to all the configured *edge* nodes:

    ipfw3 sync test centre 1


# Additional Topics

## Logging

IPFW3 supports up to 10 ``ipfw`` pseudo interfaces for logging, which can be created with:

    ifconfig ipfw1 create

Traffic matching a rule with the ``log`` action will be captured by the BPF and duplicated into the corresponding ``ipfw`` pseudo interface. e.g.

    ipfw3 add 100 allow log 1 icmp from 8.8.8.8

This rule attaches the ICMP packets from 8.8.8.8 to the ``ipfw1`` pseudo interface for logging. 


# Example Rules

Collection of the IPFW3 samples/articles.

* [IPFW3 and NAT](http://lists.dragonflybsd.org/pipermail/users/2015-June/207790.html)