Lawrence Teo's Pseudorandom Thoughts

A Small MPLS Test Network Built With OpenBSD

A few months ago, I wrote a diff to simplify the calculation of ICMP extension header checksums in the OpenBSD kernel. It so happened that the code is only used by the OpenBSD MPLS subsystem. I didn’t have access to an OpenBSD-based MPLS network at the time, nor was I familiar with MPLS in general; so in the spirit of the OpenBSD hacker mantra “shut up and hack,” I set out to build a small MPLS test network to test my diff. :)

This blog post documents my experience setting up the MPLS network; hopefully someone out there will find it useful.

WARNING: I set up this network a few months ago and it’s very likely that I have forgotten a bunch of details. Also shortly after I committed my diff, one of the nodes in the network died from a power brownout/surge during a crazy thunderstorm and I haven’t had a chance to rebuild the network. So most of this blog post is based on my notes and fuzzy memory. You’ve been warned!

If you’re not familiar with OpenBSD’s implementation of MPLS, you’ll first need to read Claudio Jeker’s (claudio@) EuroBSDCon 2011 paper: Demystifying MPLS: The MPLS Framework in OpenBSD (PDF). I designed my small (tiny?) test network as a subset of Claudio’s example network (see page 6 of the PDF for a diagram of his network). To minimize variables, I used the exact same IP addresses and MPLS labels where possible. If you’re just getting your feet wet with MPLS on OpenBSD like I was, I suggest doing the same.

Here’s how my simple four-node test network looked like:

Diagram of a small MPLS test network

If you’re not familiar with MPLS terminology, P is Provider, and PE is Provider Edge. What they are is explained in the paper. :)

The goal is to get the Customer at 192.168.237.4 talking to the IP address behind PE2 (192.168.237.242).

By the way, if you feel like trying this out, I highly recommend using OpenBSD -current (or OpenBSD 5.4 when it is released in November 1, 2013); there were a bunch of ldpd(8) commits made at the t2k13 hackathon in June 2013 so it helps to be as up-to-date as possible.

Here’s how I set up each system. PE1, P1, and PE2 will all run ospfd(8) and ldpd(8). PE1 and PE2 will run bgpd(8).

Customer Setup

ifconfig rl0 192.168.237.4/28
route add default 192.168.237.2

PE1 Setup

PE1 interface config:

ifconfig em0 rdomain 1
ifconfig em0 192.168.237.2/28
route -T1 add default 192.168.237.1
ifconfig lo1 10.42.42.1/32
ifconfig em1 10.42.0.1/24 mpls
ifconfig mpe0 rdomain 1
ifconfig mpe0 mplslabel 666
ifconfig mpe0 192.168.237.2/32

PE1 /etc/ospfd.conf:

router-id 10.42.42.1
area 0.0.0.0 {
    interface em1
    interface lo1
}

PE1 /etc/ldpd.conf:

router-id 10.42.42.1
interface em1

PE1 /etc/bgpd.conf:

router-id 10.42.42.1
AS 3.10
rdomain 1 {
    descr "CUSTOMER1"
    rd 3.10:1
    import-target rt 3.10:1
    export-target rt 3.10:1
    depend on mpe0
    network inet connected
    network 0.0.0.0/0
}
group ibgp {
    announce IPv4 unicast
    announce IPv4 vpn
    remote-as 3.10
    local-address 10.42.42.1
    neighbor 10.42.42.2 {
        descr PE2
    }
}

PE2 Setup

PE2 interface config:

ifconfig rl0 rdomain 1
ifconfig rl0 192.168.237.242/28
ifconfig lo1 10.42.42.2/32
ifconfig rl1 10.42.6.2/24 mpls
ifconfig mpe0 rdomain 1
ifconfig mpe0 mplslabel 666
ifconfig mpe0 192.168.237.242/32

PE2 /etc/ospfd.conf:

router-id 10.42.42.2
area 0.0.0.0 {
    interface rl1
    interface lo1
}

PE2 /etc/ldpd.conf:

router-id 10.42.42.2
interface rl1

PE2 /etc/bgpd.conf:

router-id 10.42.42.2
AS 3.10
rdomain 1 {
    descr "CUSTOMER1"
    rd 3.10:1
    import-target rt 3.10:1
    export-target rt 3.10:1
    depend on mpe0
    network inet connected
}
group ibgp {
    announce IPv4 unicast
    announce IPv4 vpn
    remote-as 3.10
    route-reflector
    local-address 10.42.42.2
    neighbor 10.42.42.1 {
        descr PE1
    }
}

P1 Setup

P1 interface config:

ifconfig lo1 10.42.21.1/32
ifconfig fxp2 10.42.0.2/24 mpls
ifconfig fxp3 10.42.6.1/24 mpls
sysctl net.inet.ip.forwarding=1

P1 /etc/ospfd.conf:

router-id 10.42.21.1
area 0.0.0.0 {
    interface fxp2
    interface fxp3
    interface lo1
}

P1 /etc/ldpd.conf:

router-id 10.42.21.1
interface fxp2
interface fxp3

Putting it all together

Now that all the config files are in place, it’s time to start up the daemons and give the network a whirl. At this point, I really wish I still have the test network so that I can provide specific instructions on starting the daemons and confirming that they work as expected.

But I do recall an important bit of information. The daemons need to be started in this order: (1) ospfd, (2) ldpd, and (3) bgpd. (see the end of Claudio’s misc@ post).

So if I recall correctly, this is what I did:

  1. Set up the network as above.
  2. Start ospfd on PE1, P1, and PE2.
  3. Test that PE1 and ping P1 and P1 can ping PE2.
  4. Start ldpd on PE1, P1, and PE2.
  5. Start bgpd on PE1 and PE2.

If all goes well, the Customer system should now be able to access the IP address behind PE2 (192.168.237.242) over the MPLS network.

If I ever set up the test network again, I’ll update this blog post with more specific details.

Reinstalling All Your OpenBSD Packages With Pkg_add’s Fuzzy Matching Feature

Last week, in classic Lawrence fashion, I somehow hosed my OpenBSD ports tree while doing some crazy experiments. It got to the point where make package would fail on certain ports like archivers/xz; if I tried it on those ports, I would get this funky error:

Error: /usr/ports/pobj/xz-5.0.4/fake-amd64/usr/local/share/locale/cs/LC_MESSAGES/xz.mo does not exist

Since MO files are related to GNU gettext, I tried to delete my gettext package along with all its dependencies and reinstalling them but it still produced the same error.

At that point, I decided to use a bigger hammer. The plan was to uninstall all packages and then reinstall them. That sounds tedious, but then I remembered reading that OpenBSD’s pkg_add tool has a “fuzzy adding” feature. I have never tried it before, but OpenBSD’s tools tend to Just WorkTM so I decided to give it a shot.

The two relevant pkg_add flags are -l and -z, and they are described on the pkg_add man page as follows:

-l file  Installs packages from the raw output of pkg_info(1), as saved
     in file.  Generally, use with pkg_info -m >file, to reproduce an
     installation from machine to machine.  With -z and -l pkg_add
     will try its best to reproduce the installation, even if the
     version numbers don't quite match and even if some packages
     cannot be found.

-z   Fuzzy package addition: pkg_add should do its best to match
     package names passed on the command line, even if the versions
     don't match and it will proceed even if some packages can't be
     found.

That was pretty self-explanatory. The pkg_info -m command that is mentioned in the text is used to show only packages that have been manually installed. For example, if you install git with pkg_add git and it pulls down rsync as a dependency, git is considered the manually installed package while rsync is the automatically installed package.

So, armed with that knowledge, I proceeded with saving a list of manually installed packages:

pkg_info -m >packages.txt

And then came the scary step of uninstalling ALL packages!

pkg_delete -X

Next, I reinstalled all my packages using pkg_add’s fuzzy adding feature:

export PKG_PATH=${my_favorite_mirror}
pkg_add -z -l packages.txt

And off it went! It worked just like the usual pkg_add -ui.

At the end of the entire process, all my original packages have been reinstalled. And make package on those problematic ports works again!

As mentioned in the man page excerpt above, this feature can be used to quickly install the same set of packages on similar systems. For example, if you always install a selected set of packages on a laptop, you can quickly install the same set of packages on another laptop with this technique.

It’s always fun to find useful time-saving features in the excellent OpenBSD pkg_* tools. Hope this will help someone out there!

An Easy Way to Test Your Snort Rules

Have you ever wondered if your Snort rules are actually working after you have set up Snort?

You may have seen this line in your log file…

Oct 24 22:24:20 foo snort[4520]: 1745 Snort rules read

…but you may still be wondering if Snort can actually trigger alerts.

You could always let Snort run on a live network and hope that something shows up that would trigger an alert, but you might be waiting a long time, especially if you’ve accidentally misconfigured Snort so rules aren’t even being loaded in the first place!

Here’s a quick and easy way to test your Snort installation to confirm that it has loaded the Snort rules and can trigger alerts. Please note that the emphasis is on quick and easy; this is not meant to be a comprehensive guide to test each and every Snort rule that you have loaded!

The goal of this simple guide is to help you:

  • Confirm that you have actually loaded some rules
  • Ensure that your Snort setup can actually trigger alerts
  • Confirm that you are logging alerts to the right file

I assume that you are using the official Snort rules from Sourcefire for this test, and that you have loaded all the rules as a starting point.

A lot of the rules are complex and there is no easy way to test each and every one of them. So what I have done is to pick two simple rules that you can use to test Snort by making it trigger both an outbound and inbound alert. Triggering an outbound alert is much easier so let’s start there.

Triggering an outbound alert

To trigger an outbound alert, simply run the following command from the host running Snort:

ftp http://$EXTERNAL_HOST/cmd.exe

Where $EXTERNAL_HOST is a host that you own or have permission to access for this exercise. Also, if for some reason you aren’t using a BSD system, you can always use wget or curl instead of ftp. :)

It does not matter if cmd.exe actually exists on that external host; Snort will see that you are attempting to download cmd.exe and should trigger the following Snort alert:

[**] [1:1002:15] WEB-IIS cmd.exe access [**]
[Classification: Web Application Attack] [Priority: 1] 
10/24-22:25:13.937084 x.x.x.x:42200 -> y.y.y.y:80
TCP TTL:56 TOS:0x0 ID:15236 IpLen:20 DgmLen:288 DF
***AP*** Seq: 0x1B16EAE9  Ack: 0x905BABE1  Win: 0x43E0  TcpLen: 32

On an OpenBSD system, that alert would appear in /var/snort/log/alert if you used the default settings in the Snort package.

Triggering an inbound alert

If you would like to test an inbound alert, it is slightly more complicated but still doable. You will first need to install a simple webserver. I like to install a simple lightweight webserver like thttpd. On OpenBSD you can do this by setting PKG_PATH and then run:

pkg_add thttpd

And then make thttpd listen in a dummy directory like:

mkdir /tmp/xyz
cd /tmp/xyz
thttpd -D -l /dev/stdout

Then, from an external host, run the following command:

ftp http://x.x.x.x/NessusTest

x.x.x.x in the above command is the “public” IP address of the Snort host. This should trigger the following alert:

[**] [1:2585:5] WEB-MISC nessus 2.x 404 probe [**]
[Classification: Attempted Information Leak] [Priority: 2] 
10/24-22:26:06.355207 y.y.y.y:63514 -> x.x.x.x:80
TCP TTL:64 TOS:0x0 ID:2995 IpLen:20 DgmLen:362 DF
***AP*** Seq: 0x44C5AC41  Ack: 0x4D9B5997  Win: 0x43E0  TcpLen: 32
[Xref => http://cgi.nessus.org/plugins/dump.php3?id=10386]

When you are done with the test, make sure to stop thttpd by pressing Ctrl+C to break out of it.

That’s all there is to it! Before deploying Snort in an actual production environment, please remember to carefully review your Snort rules to pick those that are applicable to your environment, and tweak your Snort configuration file accordingly.

Have fun!