Using Transparent Bridged Firewalls

Ted Knab

Your boss says you need to set up a firewall for the servers in the server room. So, you investigate and discover that there are a bunch of servers sharing the same switch. You trace the outside connection to a router. The router has a static route, but it is managed from an outside entity. After further investigation you discover that the router is owned, managed, and serviced from a consulting company that no longer exists. What do you do? If you are only familiar with routed firewalls, you may tell your boss that it cannot be done. However, in this article, I will show you how to approach the problem with another means: the transparent bridge.

Using this example situation, which is shown in Figure 1, we only control section "E" of the network. This leaves us with no control over the router. Firewalls typically are Layer 3 routing devices that drop packets matching particular port numbers or protocol types. These "routed firewalls" usually have two or more interfaces. And, usually, the internal router interface has one IP address and the external has a different IP address. To add another router or a routed firewall in-line with the network, sometimes painful topographical changes to the network must be made.

In contrast to routed firewalls, "transparent bridged firewalls" operate at Layer 2 and offer an alternative to making changes to the logical network topology. For example, transparent bridges do not have IP addresses associated with the interfaces connecting them to other networks. The transparent bridge just copies data from one port to the next, inspecting packets as they pass through.

Luckily for us, open source software does not rigidly conform to the Open Systems Interconnect (OSI) networking models. More specifically, the Linux kernel has software that allows us to manipulate Layer 3 on a bridged device. The upshot is that firewall rules work on a Linux bridge, resulting in a bridged firewall that can control the traffic flowing on a network without changing the logical network topology.

Remember your boss's request? Figure 2 is a diagram of a proposed solution. In the diagram, we add another switch, because we only control section "E". Between the two switches, we add a bridged firewall. I will show how to set up a transparent bridged firewall later in this article.

Hardware Considerations

Before a Linux firewall can be created, the hardware must be carefully considered. If time is not spent on hardware, stability problems may reveal themselves later. Most stability-related problems do not reveal themselves until the system is under a heavy I/O load. On a firewall, such heavy I/O loads are created when many people are either connected to our Web servers or when many people from inside our organization are connecting to outside sites. Although it is well known that software can create stability problems, hardware is sometimes overlooked. Bad hardware choices can ruin a firewall before it is ever plugged in. The wrong hardware can result in I/O problems and Linux compatibility issues.

There are many potential issues created by integrated hardware. For example, many integrated hardware components on modern motherboards share IRQ numbers. IRQ numbers, like channels on a radio, give the device a way to communicate interrupts to the CPU. If a device has to wait to send its interrupt, this can lower the overall performance of the system. Systems under heavy loads need clear channels to communicate to the microprocessor efficiently. Moreover, firewalls and bridges create heavy interrupt loads. Although some integrated boards may work fine for server or workstation applications, shared IRQ numbers can cause latency if two or more devices using the same IRQ channel are under heavy interrupt loads. If possible, system boards with shared components that use the same IRQ should be avoided.

Additionally, many integrated motherboards use cheaper parts that are designed to be used as "vanilla" workstations. If only integrated motherboards are available, they may work fine if all the integrated parts, like sound, and on-board Ethernet can be turned off in BIOS. However, this might not help. In short, the less integrated the motherboard, the greater potential performance.

Also note that some integrated system boards only have drivers for the most popular OS. Make sure your motherboard and its components are supported under Linux before attempting to create a firewall with it. On a final note, motherboards and system buses on them are not all the same. Some motherboards work better than others. You will want to choose a motherboard that has good I/O components. CPU speed and hard-drive speed may be important with servers and workstations. On a firewall, the motherboard is the most important part. This is followed by the quality of Ethernet cards, CPU speed, and hard-drive speed.

Setting up the Linux Bridge

Knoppix is a great tool for testing Linux hardware. It is a bootable Linux OS that allows you to make configuration checks to a system before any time is spent installing the OS. Additionally, the Knoppix CD comes with the 2.6.x Linux kernel, which is helpful in checking for hardware compatibility or driver issues.

Installing Network Cards

At the start of your system build, preferably only one network card will be in the system. This will help BIOS and the Linux OS identify the first Ethernet card as "eth0". With this basic configuration, we can begin the installation of network cards. Once the first network card is seen as "eth0", the system can be shut down, unplugged, and a new card can be added to take on the role of the first bridge interface "eth1".

When setting up your bridge, make sure each network card gets its own IRQ numbers. On 32-bit, x86i-based systems, BIOS numbers the hardware devices from 0 to 16. Linux, like other operating systems, refers to these devices through these numbers. The command lspci -v will help identify the devices and their IRQ numbers under Linux. If you have two or more network cards, you will want to attempt to separate the IRQ numbers. This will help reduce potential performance issues. The simplest way to change IRQ numbers is to assign them in BIOS. However, some BIOS codes no longer allow for changes. The next attempt may be to remove all the plug-in cards and check them on each boot.

If the BIOS does not assign the device numbers in the proper order, Linux allows for changes to IRQ numbers in the boot loader. Moreover, applications like "irqbalance" and "irqtune" help. Separated IRQ numbers on a network device will help reduce the potential for interrupt latency. When interrupt latency occurs on a system, it can slow down network communication. To help manage interrupts more efficiently, the 2.6.x Linux kernel created a daemon called the "ksoftirqd". According to the man page, this application can appear under heavy interrupt loads.

When data is copied from one interface to another, each network card must pause for a few milliseconds. On very busy networks, pauses can result in interrupt lag. Sometimes interrupt lag can be linked to network interfaces sharing IRQ numbers with other devices. For example, at my work place, we have a T3 connection. The T3 connection serves Internet for a college campus. When all the students get back from classes, they can create more than 500,000 sessions. These sessions must all be tracked, and the data being passed increases the system interrupt to about 10-20% of the total CPU use. Originally, I had inadvertently set up this firewall to use same IRQ number for its three network cards. When each card was given its own IRQ number, "ksoftirqd" was using less CPU on the system (see Listing 1).

In our Linux firewall, I used an old Netfinity 8665-6RY. I chose this machine because it had a motherboard that was designed to be put in a server, it was available, and it had a hardware warranty. Although the hardware is not too fancy, it works fine as a bridged firewall for our T3 and campus network of about 1500 users.

When our firewall was initially started, there were some bugs to work out. When running top, "ksoftirqd" appeared to be consuming 70-90% of the CPU. Normally, Linux reports these problems in the log file. However, nothing regarding interrupt problems was getting to the logs. A look at the devices with lspci -v showed that both CPU 2 and the network interfaces where using the same shared IRQ -- apparently the source of the problem. As a test, I turned off the extra CPU in the kernel. Immediately, the system latency disappeared, and the ksoftirqd process fell to 10%-20% CPU use.

In addition to IRQs, sometimes similar hardware can cause conflicts that cannot easily be resolved. For example, when I initially set up the Linux firewall, I added two Intel epro100 cards of similar serial numbers. On boot, the Linux OS seemed to get confused and would flip the interface identifiers. Although the Linux boot loader, LILO, allows you to set IRQ identifiers for cards with an append line, it seemed simpler to replace the card with another model type. Thus, I replaced one card with an Intel Gibabit Ethernet Controller Card. Although this was not the intent, using one gigabyte card and one 100-base TX card created an accidental media bridge. The gigabyte card was used on the switch link, and the 100-base TX Ethernet card was used to talk with the other switch.

In Linux, most of the network cards with the Becker drivers tend to be good. I like the 3Com 3c905 and the Intel EtherExpress Pro100 (Speedo 3) cards. Cards with similar serial numbers may cause the BIOS to get confused. Moreover, similar Ethernet cards may use the same IRQ numbers, which may create system lag. If BIOS assigns every network card the same IRQ number, you may be able to change the numbers with an append line added to the boot loader similar to:

append="netdev=irq=07,io=0x2840,name=eth0 \
netdev=irq=05,io=0x3000,name=eth1 netdev=irq11, io=0x2800,name=eth2"

Installing the Operating System

To create a Linux bridge, the first thing that needs to be installed is the OS. For our firewall, I used Debian Linux but any Linux distribution, including Knoppix, will work if it supports the 2.6.x Linux kernel. The 2.6.x Linux kernel is important because it has bridging included. In contrast, the 2.4.x Linux kernel requires a patch for bridging to function, which is a little more complicated.

Kernel Modules

Once the hardware has been selected and the IRQs on the system have all stabilized, you can compile the kernel. For a Linux bridge, you need the following kernel parameters:

CONFIG_BRIDGE=y

CONFIG_BRIDGE_NETFILTER=y

Adding Software

Once your Linux kernel is compiled and working, you can begin to add the software that allows Linux to control both the bridge and firewall. For this system, I used Debian, and the installation only required the following three main applications:

1. Bridge-utils: Applications to manage the bridge interfaces

2. Iptables: Applications to create the firewall rules

3. Ethtool: Application to modify Ethernet duplex and media types

Crossover Cables

Once the software has been downloaded, you can begin setting up the hardware. To connect the interfaces, you will need two crossover cables. The first crossover cable connects the incoming router or switch to the inside interface. The second crossover cable connects to the next device into the bridge. This second device can be either a switch or a router. In our boss example, this would be a switch on both ends.

Duplex and Auto-Negotiation

Duplex and auto-negotiation should be the same on both ends of the wires; otherwise, mismatches will occur. Mismatched duplex or auto-negotiation problems will reveal themselves in Ethernet Frame errors on Linux. Frame errors can be viewed using either the netstat -i or ifconfig -a commands. The output of the netstat command is more terse, but ifconfig will also show you which IRQ a given network interface is using, similar to the lspci -v command used previously. Additionally, the ethtool application can tell you the speed at which each device is functioning.

This script sets up duplex for 100 full for the admin and outside network:

#!/bin/sh
#filename: set_duplex.sh

ADMIN_INTERFACE="eth0"
INTERNAL_INTERFACE="eth1"
OUTSIDE_INTERFACE="eth2"

#admin interface
#ip address to access machine
ethtool -s $ADMIN_INTERFACE speed 100 duplex full \
autoneg off

#inside  (no ip address)
ethtool -s $INTERNAL_INTERFACE speed 100 duplex full \
autoneg off

#outside (no ip address)
ethtool -s $OUTSIDE_INTERFACE speed 100 duplex full \
full autoneg off

Once the duplex and is set up properly and the interfaces are connected with crossover cables, the bridge can be started. To start the bridge, I used the brctl command that was a part of the bridge-util's package:

#!/bin/sh #filename: turn_bridge_on.sh #description: sets up bridging
INTERNAL_INTERFACE="eth1" OUTSIDE_INTERFACE="eth2"
             echo "$OUTSIDE_INTERFACE NIC settings"; /sbin/ifconfig \ $OUTSIDE_INTERFACE 
            down; ifconfig $OUTSIDE_INTERFACE 0.0.0.0 up echo "$INSIDE_INTERFACE 
            NIC settings"; ifconfig $INSIDE_INTERFACE \ down ifconfig $INSIDE_INTERFACE 
            0.0.0.0 up echo "br0 (bridge) settings" echo "create logical unit 
            called br0" brctl addbr br0 echo "STP is only needed if there is more 
            than one bridge in-line" brctl stp br0 on #boss example 3 bridges 
            in-line echo "add $OUTSIDE_INTERFACE to virtual unit br0" brctrl addif 
            br0 $OUTSIDE_INTERFACE echo "add $INSIDE_INTERFACE to virtual unit 
            br0" brctrl addif br0 $INSIDE_INTERFACE echo "turning off and reseting 
            bridge one more time" ifconfig br0 down; ifconfig br0 0.0.0.0 up

This is a good point at which to test the bridge. Check whether you can ping both ends from a device on the other side of the transparent bridge. Also check whether the bridge interfaces are working properly. ifconfig -a will display the interfaces. The "br0" interface should exist and be linked to both "eth1" and "eth2".

Setting Firewall Rules

Once the bridge has been tested and is working, you can set up some simple firewall rules. The following script will create simple rules to only allow ping, httpd, and DNS traffic:

#!/bin/sh
#filename: simple_iptables.sh 

INTERNAL_INTERFACE="eth1"
OUTSIDE_INTERFACE="eth2"

echo "enable forwarding in proc"

echo "flush rules"

echo "flush input"
iptables -F INPUT

echo "flush OUTPUT"
iptables -F OUTPUT

echo "flush FORWARD"
iptables -F FORWARD

echo "accept all policy"
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT

echo "allow

echo "allow port 53 tcp" iptables -A FORWARD  -p tcp  -m state -d 0/0 \
--destination-port 53 --state NEW,ESTABLISHED,RELATED -j ACCEPT

echo "allow port 53 udp" iptables -A FORWARD  -p udp -m state -d 0/0 \
--destination-port 53 --state NEW,ESTABLISHED,RELATED -j ACCEPT

echo "allow port 80 tcp" iptables -A FORWARD  -p tcp -m state -d 0/0 \
--destination-port 80 --state NEW,ESTABLISHED,RELATED -j ACCEPT

echo "allow icmp packets from outside" iptables -A FORWARD  -p icmp -m \
physdev --physdev-in $OUTSIDE_INTERFACE -d 0/0 -j ACCEPT

echo "drop icmp packets going to admin interface" iptables -A FORWARD \
-p icmp -m physdev --physdev-in eth0 -d 0/0 -j DROP

echo "globally deny all connections to 1-1024" iptables -A FORWARD  \
-p tcp -m state  -m physdev --physdev-in $OUTSIDE_INTERFACE \
--destination-port 1:1024 --state NEW,ESTABLISHED,RELAT ED -j DROP

echo "DROP all udp connections from outside to port 1:1024 root udp \
ports" $IPTABLES  -A FORWARD  -p udp  -m state  -m physdev --physdev-in \
$OUTSIDE_INTERFACE --destination-port 1:1024 --state \
NEW,ESTABLISHED,RELATED -j DROP

Bridges, like switches, are viewed as magic boxes. Wires go in. Wires come out. But there is no routing. Unless you are looking at the management port, switches normally do not have IP addresses associated with the port. Various marketing terms only add to the confusion. Vendors say, "Our switch supports multiple VLANs" or "Our switch does routing". But here, the switch incorporates both routing and bridging. At its base, a switch is a multi-port "transparent" bridge. Transparency means these devices are communicating on the "data link layer" or one layer below the "network layer" on the OSI model.

Most routers work on the network layer. Commercial switches provide bells and whistles that allow them to function on both the "network layer" and the "data link layer" of the OSI model. Much of what commercial bridges provide can be duplicated in Linux or BSD. Using open source software with good hardware allows anyone to create inexpensive solutions that can serve a purpose. Additionally, open source software can be used as a learning tool to help demystify many of the complicated turnkey devices out there. The goal of this article was to demystify bridging and present a real-life example of how to use one as a transparent firewall. Although much of the document is Linux specific, a transparent bridged firewall can be created in OpenBSD, FreeBSD, or NetBSD.

Ted Knab runs the Annapolis Linux Users Group. He has been tinkering with computers since 1984. He works as a Systems Engineer at Washington College. He can be contacted at: sysop@washcoll.edu.