reallocarray() in OpenBSD: Integer Overflow Detection for Free
The upcoming OpenBSD 5.6 release introduces a new libc function called reallocarray(3) that extends realloc(3) with built-in integer overflow detection. In this post, I’ll discuss why it’s useful and how it can be used to fix unsafe code.
If you’re not familiar with integer overflows or need a refresher, Ray Lai’s Undeadly article on integer overflows is a good place to start, especially if you would like to read about it in the context of OpenBSD.
In his article which was written in 2006 (way before reallocarray() existed), Ray wrote, “Unfortunately there is no safe replacement for realloc(3). To prevent multiplication overflow, a check must be added”, where the check is as follows:
size_t newsize, num, size;
char *newptr, *ptr;
...
if (num && size && SIZE_T_MAX / num < size) {
errno = ENOMEM;
return (NULL);
}
newsize = num * size;
if ((newptr = realloc(ptr, newsize)) == NULL)
return (NULL);
ptr = newptr;
Adding that check before every realloc() call is clunky, not to mention error-prone. Ray recognized this and recommended the use of a function called xrealloc() – an early effort that attempts to solve what reallocarray() now solves. xrealloc() is implemented in parts of the OpenBSD source tree but is not part of OpenBSD’s libc.
reallocarray() solves this integer overflow problem more thoroughly by wrapping around realloc() like this:
/*
* This is sqrt(SIZE_MAX+1), as s1*s2 <= SIZE_MAX
* if both s1 < MUL_NO_OVERFLOW and s2 < MUL_NO_OVERFLOW
*/
#define MUL_NO_OVERFLOW (1UL << (sizeof(size_t) * 4))
void *
reallocarray(void *optr, size_t nmemb, size_t size)
{
if ((nmemb >= MUL_NO_OVERFLOW || size >= MUL_NO_OVERFLOW) &&
nmemb > 0 && SIZE_MAX / nmemb < size) {
errno = ENOMEM;
return NULL;
}
return realloc(optr, size * nmemb);
}
As you can see, reallocarray() checks to see if the attempted multiplication will result in an overflow before calling realloc(). This means you can avoid writing checks every time you need to call realloc() - thus gaining integer overflow detection virtually for free.
Another advantage over xrealloc() is that reallocarray() is available in OpenBSD’s libc, which means all C programs in OpenBSD can easily use it.
“But wait! There’s more!”™
At this point, you may be thinking, “Okay, it’s a wrapper around realloc(). What’s the big deal?” Well, it is a wrapper around realloc() with built-in integer overflow detection, but it’s also so much more than that!
In my opinion, reallocarray() is useful in three cases:
- Replacing unsafe malloc() calls
- Avoiding unneeded calloc() calls
- Replacing unsafe realloc() calls
Let’s look at each case in turn.
1. Replacing unsafe malloc() calls
malloc() as we all know is used to allocate memory for an object with a given size.
Its prototype is simple:
void *
malloc(size_t size);
When a programmer wants to allocate memory for multiple objects, the obvious but unsafe way is to call malloc() like this:
p = malloc(num * size); /* unsafe! potential overflow */
if (p == NULL)
err(1, "malloc");
As noted in the comment, the multiplication of num and size could result
in a potential overflow.
With reallocarray(), the above malloc(num * size) call can be replaced with:
p = reallocarray(NULL, num, size);
if (p == NULL)
err(1, "reallocarray");
Since reallocarray() does the overflow check for you, you’re done!
2. Avoiding unneeded calloc() calls
Before reallocarray() came along, the recommended way in OpenBSD to fix malloc(num * size) calls is to replace them with calloc() instead:
p = calloc(num, size);
if (p == NULL)
err(1, "calloc");
calloc() comes with built-in integer overflow detection, at least on OpenBSD. But calloc() also zeroes out the allocated memory. There are times when you don’t need or want to zero out the memory, for example if you’re going to intialize the entire allocated memory immediately after the calloc() call anyway.
This is where reallocarray() shines again. Just like the above case, the call can be replaced with:
p = reallocarray(NULL, num, size);
if (p == NULL)
err(1, "reallocarray");
This allows the memory to be allocated without the cost of zeroing the memory, while at the same time detecting integer overflows. Neat, huh?
3. Replacing unsafe realloc() calls
As noted by Ray, there is no safe replacement for realloc(). realloc() is frequently called like this:
p = realloc(p, num * size); /* unsafe! potential overflow */
if (p == NULL)
err(1, "realloc");
Like the malloc(num * size) case, the multiplication occurs without a check, which could result in an integer overflow.
Replacing the unsafe realloc() call with reallocarray() is very straightforward:
p = reallocarray(p, num, size);
if (p == NULL)
err(1, "reallocarray");
Bam! Integer overflow detection for free. :-)
What’s next?
reallocarray() is a non-portable OpenBSD extension. But thanks to its liberal ISC license, you are free to use it with your own code as long as you adhere to the terms of the license. So not only do you get integer overflow for free, you get the code for free as well!
Due to its usefulness, at the time of writing, more than 130 commits have been made to the OpenBSD tree that involve reallocarray(). Theo himself has been on a merciless reallocarray() rampage this past month. :-) Some of the fixes have been easy and obvious; others have been very tricky. The audit continues…
Happy hacking!