Preventing dynamic memory problems in C

When writing C programs that use structures kept in dynamic memory, there are a few techniques that are useful to prevent problems. The ones I mention here are not new, and well written software (e.g, much of Plan 9 from Bell Labs) use them a lot. This post is mostly for new programmers or not-so-old programmers in the hope that it might help them.

For example, consider a server that allocates/releases some structures to serve clients:
typedef struct MBuf MBuf;
struct MBuf {
uint64_t pc;
MBuf *next; // next in free or used list
MBuf *anext; // next in global allocation list
... the actual data goes here ...
};

It does not matter what this structure is used for. But the code will do many (de)allocations for these. Instead of using malloc/free directly, we keep all allocated structures in a single list linked through the anext field, and all unused structures in a free list, linked through the next field.

For allocation, if there are structures in the free list, we are done. Otherwise we allocate a new one and link it in the global list.

Now, in the allocation, we take the program counter of the caller and store it in the pc field. In deallocation, we set the pc to zero and set most of the structure (but for the pc and the link fields) to a silly value. That is, we poison the data when it is free.

Doing so, makes it quite easy to detect references to free structures. Also, it is easy to write a leak function that, at the end of the program or at any other point, scans the entire global allocation list and prints those program counters that are not zero, so we know the places in the program where we are allocating the structures without actually releasing them.

It's an easy thing to do, it makes the program faster because it avoids extra allocations, and it makes the program safer because it aids in testing and debugging. Hope you enjoy it.

The same thing can be done to structures pre-allocated in arrays, thus, the leaks detected are those that are actually leaks (because the leak tool provided by the OS or the language does not know if something that we allocated is actually a leak or not; it might be a correct pre-allocation that was never used).

For example, this might be the code:

MBuf*

bufalloc(void)

{

MBuf *buf;

buf = freebuf;

if (buf != NULL) {

freebuf = buf->next;

} else {

buf = xmalloc(sizeof(MBuf));

buf->anext = allbufs;

allbufs = buf;

}

CALLERPC(buf->pc); // set buf->pc with the caller pc

return buf;

}

void

buffree(MBuf *buf)

{

MBuf *n;

if (buf != NULL) {

n = buf->anext;

if (buf->pc == 0) {

xfatal("buffree: double free");

}

memset(buf, 3, sizeof(*buf));// poison

buf->anext = n;

buf->pc = 0;

buf->next = freebuf;

freebuf = buf;

}

}

void

bufleaks(void)

{

MBuf *buf;

for (buf = allbufs; buf != NULL; buf = buf->anext) {

if (buf->pc != 0) {

xwarn("bufalloc: leak pc %llx", buf->pc);

xwarn("\tlldb %s", argv0);

xwarn("\tdi -m -s %llx", buf->pc);

}

}

}

Nested control requests in Clive ZX file trees

In a previous post I wrote about ZX file system stacks in Clive. Now I look into their control interfaces, which exhibit a nice feature shown here.

When a ZX file tree operates by relying on another ZX tree deeper in the stack, it adapts its control interface to behave as a facade for it.

For example, the /Ctl file of a ZX file tree can be user to see the status of the debug flags, to see who is using the file tree (in the cases of trees served to the network) and to see usage statistics for the system.

This is the result of reading /Ctl on a stack made out of a CFS that uses a MFS as a cache, and a RFS as the remote tree. The RFS was connected to a remote server exporting a CFS that uses a MFS cache to serve a LFS. What CFS, MFS, RFS, and LFS means is not really relevant. They are just ZX file servers that rely on underlying file servers to do its job (but, CFS is a caching file sever, MFS is a RAM FS, RFS is a remote file server, and LFS is a local file server).

I have removed some of the lines when they do not help to further illustrate the case.

> cat /zx/Ctl

cfs:

fdebug off

vdebug off

noperm off

stat 8220 calls 252 errs 16440 msgs 0 bytes

bgn: min    4.911µs avg  625.628µs max 1.333559776s

end: min    5.045µs avg  625.722µs max 1.333559891s

get 3147 calls 0 errs 16501 msgs 38660497 bytes

bgn: min   54.389µs avg 2.657265ms max 856.808309ms

end: min   63.349µs avg 5.141675ms max 856.841591ms

...

cmfs:

debug off

noperm off

stat 78449 calls 2721 errs 156898 msgs 0 bytes

...

lsub:

user rfs:193.147.15.27:65348 nemo as nemo on 2015-03-26

user rfs:193.147.15.27:65349 nemo as nemo on 2015-03-26

ldebug off

rdebug off

stat 2656 calls 0 errs 5312 msgs 0 bytes

...

mfs:lsub:

debug off

noperm off

stat 47588 calls 2487 errs 95176 msgs 0 bytes

...

lsub:

debug off

rdonly off

noperm off

stat 2854 calls 0 errs 5708 msgs 0 bytes

...

Now, note the lines starting with cfs:, cmfs:, and mfs:lsub:. Only the first section comes from the top-level CFS. Remaining lines come from such server reading the /Ctl file of the underlying ZX file tree and appending it to the data served to the user; and so on as we go down on the stack.

Thus, the user may inspect the ZX stack as deep as it wants.

In the same way, we can set debug on for CFS by executing

> echo debug on > /zx/Ctl

The ZX file server understands writes to the control file as textual commands

to perform control requests.

But we can also set debug on the third tree in the stack:

> echo pass pass debug on > /zx/Ctl

When a control request starts with pass, this word is removed from the request

and the server writes what remains of the control requests to the next tree in the stack.

This makes the control interface for all the trees in the stack available to the final user.

Tiny go debug tools

A recent post from Rob Pike in his blog reminded me that I didn't post about some tiny tools I use to debug programs by enabling traces and conditional prints. I think these were borrowed or inspired by those written by him or by others using Go time ago.

These are packaged in a tiny git.lsub.org/go.git/dbg.go package.

First, this is a well known idiom to trace function calls, only that I modified it to report also the file and line number in a convenient way.

It is used as in this excerpt

func ReadMsgsFrom(r io.Reader, c chan<- []byte) (int64, error) {
defer dbg.Trace(dbg.Call("ReadMsgsFrom"))

...

And produces messages like these ones:

bgn ReadMsgsFrom nchan.go:116

end ReadMsgsFrom nchan.go:116

The functions are like follows:

// For use as in defer dbg.Trace(Call("funcname"))

func Trace(s string) {

fmt.Printf("end %s\n", s)

}

// For use as in defer dbg.Trace(Call("funcname"))

func Call(s string) string {

if _, file, lno, ok := runtime.Caller(1); ok {

rel, _ := filepath.Rel(cwd, file)

s = fmt.Sprintf("%s %s:%d", s, rel, lno)

}

fmt.Printf("bgn %s\n", s)

return s

}

The second tool is a couple of functions that enable prints only if a flag is set or if another function says so. They are used like in

var Printf = dbg.FuncPrintf(os.Stdout, testing.Verbose)

func TestChanSend(t *testing.T) {

...

Printf("receiver %v\n", msg)

}

The variable name should reflect that it is a conditional print, but in this case I took the code from a testing file, which prints only if verbose is set during the test. The idea is that you can declare as many print functions (variables) you want to print conditionally when certain flags are enabled.

This is the code from the debug package

type PrintFunc func(fmts string, arg ...interface{}) (int, error)

/*

Return a function that calls fmt.Printf only if fn returns true.

To be used like in

var Printf = verb.PrintfFunc(testing.Verbose)

...

Printf(...)

 */

func FuncPrintf(w io.Writer, fn func()bool) PrintFunc {

return func(fmts string, arg ...interface{}) (int, error) {

if fn() {

return fmt.Fprintf(w, fmts, arg...)

}

return 0, nil

}

}

The function returns a function that prints, only that it prints only if the function given as an argument says so. Thus, when you declare your print function variable you can set the condition to trigger the print and the writer the print should use. The rest of the code is relieved from the burden of testing the condition or typing more to use a particular output stream.

There is a similar function (returning a conditional printing function) that takes a pointer to a boolean flag instead of a function, for those cases when checking a flag suffice.