Archive for the 'Solution Writeups' Category

Scenario: you have a wireless AP that you allow someone to access. You don’t have unlimited Internet bandwidth, so you want to limit your ‘clients’ bandwidth to your network and the Internet. You have a very low budget. You have a free weekend. You have an old pc with two network cards.

We will be using FreeBSD 4.8 (should work with any 4.x) to make an Ethernet bridge between your wireless network and the Internet, and then using DUMMYNET so you can limit the bandwidth from your wireless access point by IP or MAC address.

DUMMYNET (aka ‘a traffic shaper’) is a kernel add-on to ipfw. ifpw is the FreeBSD default firewall package, pre-built into the kernel. DUMMYNET allows you to just add firewall rules into ipfw that will basically ‘slow the packets down’ in various ways (using WF2Q+: Worst-case Fair Weighted Fair Queuing) and the end result is you having control over the bandwidth and latency of data per IP or MAC of any host.

Overall Steps:
Install FreeBSD
Rebuild Kernel to support DUMMYNET
Enable bridge for the two NIC’s
Write your ‘pipe’ rules for DUMMYNET
Test and Deploy

(use http://www.freebsd.org/handbook for help on everything here)

1. Get a Pentium PC or above with two NIC’s, 32mb of RAM, and a 2GB hard drive from your local PC store for $50. (may work with less, but this is my setup)
2. Boot the box with a FreeBSD 4.8 CD ISO or Floppies from freebsd.org (you only need the first ISO CD). If you don’t have CD drive in the box (I didn’t) then you can make two floppies to boot from and pull the CD over FTP.

1. A cool thing to do is download the ISO, burn it onto CD, the copy the whole CD to a server that has FTP and enable anonymous access to the CD files so that I can just walk around with two floppies and make little FreeBSD babies. Note that you can actually pull everything down from public FTP servers with just the two floppies, but unless you’re on an OC-3 the local FTP server will be faster.

FreeBSD Install

1. I will mention the major decisions you need to make during install of FreeBSD here. http://www.freebsd.org/handbook does a much better job of describing this process.
2. Skip Kernel configuration and continue with installation (unless you have a very odd NIC or SCSI card)
3. Chose Standard Install
4. At FDISK, hit A to use entire disk and then Q to Finish
5. Standard MBR
6. At Disklabel Editor, hit A to auto default, and Q to Finish
7. Chose Kern-Developer Distribution
8. Yes to ports install
9. Exit Distributions menu
10. Chose method of install, #2 is used to install from your custom local FTP server if you followed my recommendations above. If you chose #2 then enter the URL manually on the next screen, which should just be the hostname.
11. Now you’ll see your network interfaces listed. Hopefully you see one of the two NIC’s you have installed. You may have to guess as to which one is connected to the network if you’re doing an FTP-based install. Use DHCP if you need it obviously.
12. Once install has finished: No to network gateway, NO to configure inetd, No to anonymous FTP, NO to NFS Server, NO to NFS Client, NO to select a default security profile, NO to customize system console, YES to setting time zone, NO to Linux binary compatibility, NO to non-USB mouse (you won’t need to use the mouse in console mode), NO to browsing ports collection.
13. Once install has finished, you need to add a new user and user group. The best way to do this is to first make a group with the same name as your first username, and then make the user and add it to the group you just created. In FreeBSD, you should always make a non-root user for SSH’ing into the box, and users should always be in groups with the same name as their username. Add the user to the ‘wheel’ member group so they can ‘su’ to root after they log in from SSH.
14. Exit install, it will reboot, log in as root at the console so we can start configuring.

Recompile Kernel

We first need to recompile the kernel with bridging and dummynet enabled so they will work when we configure them. Luckily, we had the install program dump the kernel source in /usr/src so we can recompile the kernel. The current kernel (aka GENERIC) is at /usr/src/sys/i386/conf/GENERIC. We need to copy it, edit it, and add the options we need, then compile it and install it.

(after logging in)

cp /usr/src/sys/i386/conf/GENERIC /usr/src/sys/i386/conf/BRIDGE
ee /usr/src/sys/i386/conf/BRIDGE

Now you should be in ee, an editor that’s a little more friendly then vi. Add the following lines around the other ‘options’ lines.
options BRIDGE
options IPFIREWALL_DEFAULT_TO_ACCEPT
options DUMMYNET
options HZ=1000

Now compile the kernel and install it. If it’s a P or PII machine, go read War & Peace while it compiles, if it’s sub 2GHz you can just read the first 1/2 of War & Peace.

cd /usr/src
make buildkernel KERNELCONF=BRIDGE
make installkernel KERNELCONF=BRIDGE

Now reboot and log in.

Configuring Dummynet

We need to edit three files to make it all work. You can use ee or vi for all of these. First one is /etc/rc.conf. You will only be giving your ‘internal’ NIC an IP address in rc.conf. This IP will be used for management access only, and this bridge could theoretically work without any IP’s assigned, because it’s acting like a 2-port switch, and NOT a router. Use the following options to make sure the 2nd NIC (connected to your AP) still turns on during boot (NIC’s don’t turn on if they don’t have a network address). Fill the XXX’s with your net info. /etc/rc.conf

defaultrouter=”XXX.XXX.XXX.XXX”
hostname=”XXXX”
#this is the line to turn on the 2nd NIC
#the xl0 needs to be whatever ifconfig shows you
#is the device name of your two NICs
ifconfig_xl0=”up”
ifconfig_xl1=”inet XXX.XXX.XXX.XXX netmask XXX.XXX.XXX.XXX”
firewall_enable=”YES”
firewall_type=”/etc/ipfw.rules”

You can have other lines in rc.conf if you like. Here are some others that my bridge box has:

kern_securelevel_enable=”NO”
nfs_reserved_port_only=”YES”
# this disables sendmail server but allows mail out
sendmail_enable=”NONE”
sshd_enable=”YES”
# disable usb if you don’t need it
usbd_enable=”NO”
# disable inetd which is for ftp and other services
inetd_enable=”NO”

Put these lines in your sysctl.conf to enable the bridge. Note that on the last line I entered xl0:1,xl1:1. The xl0 and xl1 are my two NIC’s, found by typing ipconfig and getting their interface names (the same ones in /etc/rc.conf), and the :1 after each says I want both NIC’s to be in ‘bridge group 1’. /etc/sysctl.conf

# enables bridge
net.link.ether.bridge=1
# tells bridge to go through the firewall
# by default the firewall won’t see bridged packets
net.link.ether.bridge_ipfw=1
# tells bridge what interfaces are bridged
net.link.ether.bridge_cfg=xl0:1,xl1:1,

The ipfw.rules file is where you do the rest of the work. You create this file to normally apply your firewall rules, but ‘dummynet’, the traffic shaper, is apart of ipfw. So, put your traffic limiting rules in here as well. /etc/ipfw.rules

#create a ‘traffic lane’ to put people in
pipe 1 config bw 2M
#add every connection to the ‘traffic lane’
add pipe 1 ip from any to any

This puts all traffic going across the bridge into the same que/pipe and limitless it to about 2MB/s.

Note that the above configuration puts the traffic in half-duplex mode, whereas if someone is pulling a file at 2MB one way and someone else is pulling a file the other way, then they will have to share that 2MB pipe. But this configuration makes a separate pipe for data in each direction so it becomes a full-duplex configuration.

ipfw pipe 1 config bw 2M
ipfw pipe 2 config bw 2M
ipfw add pipe 1 ip from any to any out
ipfw add pipe 2 ip from any to any in

Here is a configuration that will limit IP 192.168.1.50 to 1MB and the rest will flow freely (if packets don’t match a pipe then they won’t be regulated).

ipfw pipe 1 config bw 1mb/s
ipfw add pipe 1 ip from 192.168.1.50 to any
ipfw add pipe 1 ip from any to 192.168.1.50

This configuration adds the whole 192.168.2 subnet to the pipe and limits it to 300Kb/s.

ipfw add pipe 1 ip from 192.168.2.0/24 to any out
ipfw pipe 1 config bw 300Kbit/s

A REAL WORLD Example:
I allow my friend to access my DSL from his house across the street using some wireless gear. I tell him to use DHCP and then hard code his MAC in my dchp server so he get’s the same IP every time (or just have him statically assign it). I know he will be downloading stuff from my ftp server in my room, so I don’t want him maxing out my wireless link so that I can’t use it from downstairs on my laptop, so I want to limit him to about half or so of my wireless bandwidth on my 802.11b access point. The real world speed of 11b on a good day is almost 6Mb/s, so I will limit him to 3Mb/s. Since wireless is a half-duplex technology I will add his in and out traffic to the same queue. I stick this box between my AP and my network and use the following lines:

Put the pc between your wired network and the Access Point.
Smile at your accomplishment.

Note: To add Firewall functionally to this box you will need to make it a router. A limitation of ipfw seems to be that in bridge mode, it can handle firewall rules OR dummynet rules, but not both. Turning this box into a router and adding the line below to your /etc/sysctl.conf will allow packets to flow through the pipe rules and firewall rules.

net.inet.ip.fw.one_pass: 0

If you did do this then a lot of this document would be void, but you could take the dummynet lesions learned and apply them to a traditional firewall with dhcp/dns and all the typical services and slap this between your wireless access point and you. It would get complicated quickly, but hey, it’s a great idea. Maybe that’ll be my next project.

Other things you could do to make this sweeter:

• add your email address to /etc/mail/aliases as the ‘root’ so you get daily emails about the health of your new traffic shaper.
• install the webmin port for easier remote management… of something.
• install apache port and mrtg port to graph the packets used over your bridge.
• even better: install ntop port for a look at what’s really going on
• make really fancy dummynet rules, like a cron job that applies different rules at different times of day giving users more bandwidth at night or during your working hours when your not around…

References:
The creator of Dummynet for FreeBSD

http://info.iet.unipi.it/~luigi/ip_dummynet/

FreeBSD Handbook

http://www.freebsd.org/handbook