Lawrence Teo's Pseudorandom Thoughts

A developer's journal on OpenBSD, security, and beyond

reallocarray() in OpenBSD: Integer Overflow Detection for Free

The upcoming OpenBSD 5.6 release introduces a new libc function called reallocarray(3) that extends realloc(3) with built-in integer overflow detection. In this post, I’ll discuss why it’s useful and how it can be used to fix unsafe code.

If you’re not familiar with integer overflows or need a refresher, Ray Lai’s Undeadly article on integer overflows is a good place to start, especially if you would like to read about it in the context of OpenBSD.

In his article which was written in 2006 (way before reallocarray() existed), Ray wrote, “Unfortunately there is no safe replacement for realloc(3). To prevent multiplication overflow, a check must be added”, where the check is as follows:

1
2
3
4
5
6
7
8
9
10
11
size_t newsize, num, size;
char *newptr, *ptr;
...
if (num && size && SIZE_T_MAX / num < size) {
        errno = ENOMEM;
        return (NULL);
}
newsize = num * size;
if ((newptr = realloc(ptr, newsize)) == NULL)
        return (NULL);
ptr = newptr;

Adding that check before every realloc() call is clunky, not to mention error-prone. Ray recognized this and recommended the use of a function called xrealloc() — an early effort that attempts to solve what reallocarray() now solves. xrealloc() is implemented in parts of the OpenBSD source tree but is not part of OpenBSD’s libc.

reallocarray() solves this integer overflow problem more thoroughly by wrapping around realloc() like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
/*
 * This is sqrt(SIZE_MAX+1), as s1*s2 <= SIZE_MAX
 * if both s1 < MUL_NO_OVERFLOW and s2 < MUL_NO_OVERFLOW
 */
#define MUL_NO_OVERFLOW (1UL << (sizeof(size_t) * 4))

void *
reallocarray(void *optr, size_t nmemb, size_t size)
{
    if ((nmemb >= MUL_NO_OVERFLOW || size >= MUL_NO_OVERFLOW) &&
        nmemb > 0 && SIZE_MAX / nmemb < size) {
            errno = ENOMEM;
            return NULL;
    }
    return realloc(optr, size * nmemb);
}

As you can see, reallocarray() checks to see if the attempted multiplication will result in an overflow before calling realloc(). This means you can avoid writing checks every time you need to call realloc() – thus gaining integer overflow detection virtually for free.

Another advantage over xrealloc() is that reallocarray() is available in OpenBSD’s libc, which means all C programs in OpenBSD can easily use it.

“But wait! There’s more!”™

At this point, you may be thinking, “Okay, it’s a wrapper around realloc(). What’s the big deal?” Well, it is a wrapper around realloc() with built-in integer overflow detection, but it’s also so much more than that!

In my opinion, reallocarray() is useful in three cases:

  1. Replacing unsafe malloc() calls
  2. Avoiding unneeded calloc() calls
  3. Replacing unsafe realloc() calls

Let’s look at each case in turn.

1. Replacing unsafe malloc() calls

malloc() as we all know is used to allocate memory for an object with a given size.

Its prototype is simple:

1
2
void *
malloc(size_t size);

When a programmer wants to allocate memory for multiple objects, the obvious but unsafe way is to call malloc() like this:

1
2
3
p = malloc(num * size);     /* unsafe! potential overflow */
if (p == NULL)
        err(1, "malloc");

As noted in the comment, the multiplication of num and size could result in a potential overflow.

With reallocarray(), the above malloc(num * size) call can be replaced with:

1
2
3
p = reallocarray(NULL, num, size);
if (p == NULL)
        err(1, "reallocarray");

Since reallocarray() does the overflow check for you, you’re done!

2. Avoiding unneeded calloc() calls

Before reallocarray() came along, the recommended way in OpenBSD to fix malloc(num * size) calls is to replace them with calloc() instead:

1
2
3
p = calloc(num, size);
if (p == NULL)
        err(1, "calloc");

calloc() comes with built-in integer overflow detection, at least on OpenBSD. But calloc() also zeroes out the allocated memory. There are times when you don’t need or want to zero out the memory, for example if you’re going to intialize the entire allocated memory immediately after the calloc() call anyway.

This is where reallocarray() shines again. Just like the above case, the call can be replaced with:

1
2
3
p = reallocarray(NULL, num, size);
if (p == NULL)
        err(1, "reallocarray");

This allows the memory to be allocated without the cost of zeroing the memory, while at the same time detecting integer overflows. Neat, huh?

3. Replacing unsafe realloc() calls

As noted by Ray, there is no safe replacement for realloc(). realloc() is frequently called like this:

1
2
3
4
newp = realloc(p, num * size); /* unsafe! potential overflow */
if (newp == NULL)
        err(1, "realloc");
p = newp;

Like the malloc(num * size) case, the multiplication occurs without a check, which could result in an integer overflow.

reallocarray() can replace the unsafe realloc() call like this:

1
2
3
4
newp = reallocarray(p, num, size);
if (newp == NULL)
        err(1, "reallocarray");
p = newp;

You have to be careful here though, because if the code is part of a function which returns on error instead of exiting completely, the original p would not be freed and you will lose it. For example:

1
2
3
4
newp = reallocarray(p, num, size);
if (newp == NULL)
        return (NULL); /* original p is lost -> memory leak! */
p = newp;

A better way is to borrow the recommended idiom for realloc() from OpenBSD’s realloc(3) man page and apply it to reallocarray():

1
2
3
4
5
6
7
newp = reallocarray(p, num, size);
if (newp == NULL) {
        free(p);
        p = NULL;
        return (NULL);
}
p = newp;

What’s next?

reallocarray() is a non-portable OpenBSD extension. But thanks to its liberal ISC license, you are free to use it with your own code as long as you adhere to the terms of the license. So not only do you get integer overflow for free, you get the code for free as well!

Due to its usefulness, at the time of writing, more than 130 commits have been made to the OpenBSD tree that involve reallocarray(). Theo himself has been on a merciless reallocarray() rampage this past month. :–) Some of the fixes have been easy and obvious; others have been very tricky. The audit continues…

Happy hacking!

UPDATE (2014/10/29): reallocarray() has been added to libbsd 0.7.0. Note that I have never used libbsd so this is not an endorsement of libbsd. :–)

UPDATE (2014/10/30): Revised the “Replacing unsafe realloc() calls” section to discuss potential memory leak issues. Thank you to Andrew Dalgleish for pointing out this issue in the original version of the blog post.

A Sneak Peek at the Upcoming OpenBSD 5.6 Release

OpenBSD 5.6 CD cover In exactly one month from now, OpenBSD 5.6 will be released on November 1, 2014 — like clockwork as always. But, you don’t have to wait until November 1 if you want it earlier, because pre-orders are now up at the brand new OpenBSD store!

OpenBSD 5.6 is of course the first OpenBSD release with LibreSSL, the now-famous fork of the OpenSSL library. But while LibreSSL is an important milestone for OpenBSD, there are many other things in the OpenBSD 5.6 release that warrant attention as well.

The developers are still busy preparing the OpenBSD 5.6 release notes. Meanwhile, if you’re curious about what’s new in OpenBSD 5.6, you can get a sneak peek from various places on the Internet if you know where to look! I would like to highlight three places in particular.

plus56.html

The first place to look at is the big list of changes between OpenBSD 5.5 and 5.6 that has been painstakingly compiled by Brett Mahar (brett@) throughout the development cycle. I think Brett does an amazing job of collating all this information together for every release! I have no idea how he manages to go through every source-changes@ post to produce this list so consistently. :)

Hackathon reports

The second place that you can look at are the hackathon reports written by the developers. Two hackathons were held between OpenBSD 5.5 and 5.6: m2k14 in Marrakech, Morocco (April 17-22, 2014) and g2k14 in Ljubljana, Slovenia (July 8-14, 2014).

The hackathon reports are published on the OpenBSD Journal website. I have listed them here:

m2k14 hackathon reports

g2k14 hackathon reports

As for the g2k14 hackathon reports, thanks to Undeadly editor tbert, they are all listed in a single article on the OpenBSD Journal! Here it is:

g2k14: Reports Roundup

Slides and papers

The third place to look at are the slides and papers written by OpenBSD developers. Two notable events that took place between OpenBSD 5.5 and 5.6 are BSDCan 2014 and EuroBSDCon 2014. You can check out OpenBSD-related slides and papers for these events and more at:

As you can see there’s a wealth of information out there on what’s new in OpenBSD 5.6 that really shows the amazing amount of work that is done for every release.

I would like to end by highlighting this tweet/retweet from Ryan McBride (mcbride@), which I think summarizes the awesomeness of the OpenBSD release process like no other:

mcbride on OpenBSD releases

So remember to pre-order OpenBSD 5.6 or donate to OpenBSD to keep this awesome project alive and thriving!

A small MPLS test network built with OpenBSD

A few months ago, I wrote a diff to simplify the calculation of ICMP extension header checksums in the OpenBSD kernel. It so happened that the code is only used by the OpenBSD MPLS subsystem. I didn’t have access to an OpenBSD-based MPLS network at the time, nor was I familiar with MPLS in general; so in the spirit of the OpenBSD hacker mantra “shut up and hack,” I set out to build a small MPLS test network to test my diff. :)

This blog post documents my experience setting up the MPLS network; hopefully someone out there will find it useful.

WARNING: I set up this network a few months ago and it’s very likely that I have forgotten a bunch of details. Also shortly after I committed my diff, one of the nodes in the network died from a power brownout/surge during a crazy thunderstorm and I haven’t had a chance to rebuild the network. So most of this blog post is based on my notes and fuzzy memory. You’ve been warned!

If you’re not familiar with OpenBSD’s implementation of MPLS, you’ll first need to read Claudio Jeker’s (claudio@) EuroBSDCon 2011 paper: Demystifying MPLS: The MPLS Framework in OpenBSD (PDF). I designed my small (tiny?) test network as a subset of Claudio’s example network (see page 6 of the PDF for a diagram of his network). To minimize variables, I used the exact same IP addresses and MPLS labels where possible. If you’re just getting your feet wet with MPLS on OpenBSD like I was, I suggest doing the same.

Here’s how my simple four-node test network looked like:

Diagram of a small MPLS test network

If you’re not familiar with MPLS terminology, P is Provider, and PE is Provider Edge. What they are is explained in the paper. :)

The goal is to get the Customer at 192.168.237.4 talking to the IP address behind PE2 (192.168.237.242).

By the way, if you feel like trying this out, I highly recommend using OpenBSD -current (or OpenBSD 5.4 when it is released in November 1, 2013); there were a bunch of ldpd(8) commits made at the t2k13 hackathon in June 2013 so it helps to be as up-to-date as possible.

Here’s how I set up each system. PE1, P1, and PE2 will all run ospfd(8) and ldpd(8). PE1 and PE2 will run bgpd(8).

Customer Setup

ifconfig rl0 192.168.237.4/28
route add default 192.168.237.2

PE1 Setup

PE1 interface config:

ifconfig em0 rdomain 1
ifconfig em0 192.168.237.2/28
route -T1 add default 192.168.237.1
ifconfig lo1 10.42.42.1/32
ifconfig em1 10.42.0.1/24 mpls
ifconfig mpe0 rdomain 1
ifconfig mpe0 mplslabel 666
ifconfig mpe0 192.168.237.2/32

PE1 /etc/ospfd.conf:

router-id 10.42.42.1
area 0.0.0.0 {
    interface em1
    interface lo1
}

PE1 /etc/ldpd.conf:

router-id 10.42.42.1
interface em1

PE1 /etc/bgpd.conf:

router-id 10.42.42.1
AS 3.10
rdomain 1 {
    descr "CUSTOMER1"
    rd 3.10:1
    import-target rt 3.10:1
    export-target rt 3.10:1
    depend on mpe0
    network inet connected
    network 0.0.0.0/0
}
group ibgp {
    announce IPv4 unicast
    announce IPv4 vpn
    remote-as 3.10
    local-address 10.42.42.1
    neighbor 10.42.42.2 {
        descr PE2
    }
}

PE2 Setup

PE2 interface config:

ifconfig rl0 rdomain 1
ifconfig rl0 192.168.237.242/28
ifconfig lo1 10.42.42.2/32
ifconfig rl1 10.42.6.2/24 mpls
ifconfig mpe0 rdomain 1
ifconfig mpe0 mplslabel 666
ifconfig mpe0 192.168.237.242/32

PE2 /etc/ospfd.conf:

router-id 10.42.42.2
area 0.0.0.0 {
    interface rl1
    interface lo1
}

PE2 /etc/ldpd.conf:

router-id 10.42.42.2
interface rl1

PE2 /etc/bgpd.conf:

router-id 10.42.42.2
AS 3.10
rdomain 1 {
    descr "CUSTOMER1"
    rd 3.10:1
    import-target rt 3.10:1
    export-target rt 3.10:1
    depend on mpe0
    network inet connected
}
group ibgp {
    announce IPv4 unicast
    announce IPv4 vpn
    remote-as 3.10
    route-reflector
    local-address 10.42.42.2
    neighbor 10.42.42.1 {
        descr PE1
    }
}

P1 Setup

P1 interface config:

ifconfig lo1 10.42.21.1/32
ifconfig fxp2 10.42.0.2/24 mpls
ifconfig fxp3 10.42.6.1/24 mpls
sysctl net.inet.ip.forwarding=1

P1 /etc/ospfd.conf:

router-id 10.42.21.1
area 0.0.0.0 {
    interface fxp2
    interface fxp3
    interface lo1
}

P1 /etc/ldpd.conf:

router-id 10.42.21.1
interface fxp2
interface fxp3

Putting it all together

Now that all the config files are in place, it’s time to start up the daemons and give the network a whirl. At this point, I really wish I still have the test network so that I can provide specific instructions on starting the daemons and confirming that they work as expected.

But I do recall an important bit of information. The daemons need to be started in this order: (1) ospfd, (2) ldpd, and (3) bgpd. (see the end of Claudio’s misc@ post).

So if I recall correctly, this is what I did:

  1. Set up the network as above.
  2. Start ospfd on PE1, P1, and PE2.
  3. Test that PE1 and ping P1 and P1 can ping PE2.
  4. Start ldpd on PE1, P1, and PE2.
  5. Start bgpd on PE1 and PE2.

If all goes well, the Customer system should now be able to access the IP address behind PE2 (192.168.237.242) over the MPLS network.

If I ever set up the test network again, I’ll update this blog post with more specific details.

Reinstalling all your OpenBSD packages with pkg_add’s fuzzy matching feature

Last week, in classic Lawrence fashion, I somehow hosed my OpenBSD ports tree while doing some crazy experiments. It got to the point where make package would fail on certain ports like archivers/xz; if I tried it on those ports, I would get this funky error:

Error: /usr/ports/pobj/xz-5.0.4/fake-amd64/usr/local/share/locale/cs/LC_MESSAGES/xz.mo does not exist

Since MO files are related to GNU gettext, I tried to delete my gettext package along with all its dependencies and reinstalling them but it still produced the same error.

At that point, I decided to use a bigger hammer. The plan was to uninstall all packages and then reinstall them. That sounds tedious, but then I remembered reading that OpenBSD’s pkg_add tool has a “fuzzy adding” feature. I have never tried it before, but OpenBSD’s tools tend to Just WorkTM so I decided to give it a shot.

The two relevant pkg_add flags are -l and -z, and they are described on the pkg_add man page as follows:

-l file  Installs packages from the raw output of pkg_info(1), as saved
     in file.  Generally, use with pkg_info -m >file, to reproduce an
     installation from machine to machine.  With -z and -l pkg_add
     will try its best to reproduce the installation, even if the
     version numbers don't quite match and even if some packages
     cannot be found.

-z   Fuzzy package addition: pkg_add should do its best to match
     package names passed on the command line, even if the versions
     don't match and it will proceed even if some packages can't be
     found.

That was pretty self-explanatory. The pkg_info -m command that is mentioned in the text is used to show only packages that have been manually installed. For example, if you install git with pkg_add git and it pulls down rsync as a dependency, git is considered the manually installed package while rsync is the automatically installed package.

So, armed with that knowledge, I proceeded with saving a list of manually installed packages:

pkg_info -m >packages.txt

And then came the scary step of uninstalling ALL packages!

pkg_delete -X

Next, I reinstalled all my packages using pkg_add’s fuzzy adding feature:

export PKG_PATH=${my_favorite_mirror}
pkg_add -z -l packages.txt

And off it went! It worked just like the usual pkg_add -ui.

At the end of the entire process, all my original packages have been reinstalled. And make package on those problematic ports works again!

As mentioned in the man page excerpt above, this feature can be used to quickly install the same set of packages on similar systems. For example, if you always install a selected set of packages on a laptop, you can quickly install the same set of packages on another laptop with this technique.

It’s always fun to find useful time-saving features in the excellent OpenBSD pkg_* tools. Hope this will help someone out there!

An easy way to test your Snort rules

Have you ever wondered if your Snort rules are actually working after you have set up Snort?

You may have seen this line in your log file…

Oct 24 22:24:20 foo snort[4520]: 1745 Snort rules read

…but you may still be wondering if Snort can actually trigger alerts.

You could always let Snort run on a live network and hope that something shows up that would trigger an alert, but you might be waiting a long time, especially if you’ve accidentally misconfigured Snort so rules aren’t even being loaded in the first place!

Here’s a quick and easy way to test your Snort installation to confirm that it has loaded the Snort rules and can trigger alerts. Please note that the emphasis is on quick and easy; this is not meant to be a comprehensive guide to test each and every Snort rule that you have loaded!

The goal of this simple guide is to help you:

  • Confirm that you have actually loaded some rules
  • Ensure that your Snort setup can actually trigger alerts
  • Confirm that you are logging alerts to the right file

I assume that you are using the official Snort rules from Sourcefire for this test, and that you have loaded all the rules as a starting point.

A lot of the rules are complex and there is no easy way to test each and every one of them. So what I have done is to pick two simple rules that you can use to test Snort by making it trigger both an outbound and inbound alert. Triggering an outbound alert is much easier so let’s start there.

Triggering an outbound alert

To trigger an outbound alert, simply run the following command from the host running Snort:

ftp http://$EXTERNAL_HOST/cmd.exe

Where $EXTERNAL_HOST is a host that you own or have permission to access for this exercise. Also, if for some reason you aren’t using a BSD system, you can always use wget or curl instead of ftp. :)

It does not matter if cmd.exe actually exists on that external host; Snort will see that you are attempting to download cmd.exe and should trigger the following Snort alert:

[**] [1:1002:15] WEB-IIS cmd.exe access [**]
[Classification: Web Application Attack] [Priority: 1] 
10/24-22:25:13.937084 x.x.x.x:42200 -> y.y.y.y:80
TCP TTL:56 TOS:0x0 ID:15236 IpLen:20 DgmLen:288 DF
***AP*** Seq: 0x1B16EAE9  Ack: 0x905BABE1  Win: 0x43E0  TcpLen: 32

On an OpenBSD system, that alert would appear in /var/snort/log/alert if you used the default settings in the Snort package.

Triggering an inbound alert

If you would like to test an inbound alert, it is slightly more complicated but still doable. You will first need to install a simple webserver. I like to install a simple lightweight webserver like thttpd. On OpenBSD you can do this by setting PKG_PATH and then run:

pkg_add thttpd

And then make thttpd listen in a dummy directory like:

mkdir /tmp/xyz
cd /tmp/xyz
thttpd -D -l /dev/stdout

Then, from an external host, run the following command:

ftp http://x.x.x.x/NessusTest

x.x.x.x in the above command is the “public” IP address of the Snort host. This should trigger the following alert:

[**] [1:2585:5] WEB-MISC nessus 2.x 404 probe [**]
[Classification: Attempted Information Leak] [Priority: 2] 
10/24-22:26:06.355207 y.y.y.y:63514 -> x.x.x.x:80
TCP TTL:64 TOS:0x0 ID:2995 IpLen:20 DgmLen:362 DF
***AP*** Seq: 0x44C5AC41  Ack: 0x4D9B5997  Win: 0x43E0  TcpLen: 32
[Xref => http://cgi.nessus.org/plugins/dump.php3?id=10386]

When you are done with the test, make sure to stop thttpd by pressing Ctrl+C to break out of it.

That’s all there is to it! Before deploying Snort in an actual production environment, please remember to carefully review your Snort rules to pick those that are applicable to your environment, and tweak your Snort configuration file accordingly.

Have fun!