Visualizing the Heap on Embedded Systems – Part II

In the last article, I described a method to acquire heap allocation data from an embedded system. Next, I’ll describe how to visualize the data. First, though, to make things easier, I’ll acquire some real data from a regular Linux application.

For simplicity, I profiled a Linux application built with gcc, but the same principle applies to an embedded application. Using gcc, there is a useful shortcut to wrap the allocation function calls, which doesn’t require editing the code or modifying the objects directly. The linker ld provides a built-in option –wrap which will replace a symbol with __wrap_symbol which in turn can call __real_symbol to call the original function. You can pass this option through gcc to the linker as appropriate:

CFLAGS = -g -O0 -Wall -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc -Wl,--wrap,realloc

Next, just define your wrappers in a fashion similar to the previous article:

#include <stdio.h>

void *__wrap_malloc(size_t size)
{
	void *ret;

	ret = __real_malloc(size);
	fprintf(stderr, "m,%d,0x%8.8x\n", size, (unsigned int)ret);
	return ret;
}

void __wrap_free(void *ptr) {
	__real_free(ptr);
	fprintf(stderr, "f,,0x%8.8x\n", (unsigned int)ptr);
	return;
}

void *__wrap_realloc(void *ptr, size_t size)
{
	void *ret = __real_realloc(ptr, size);
	fprintf(stderr, "f,,0x%8.8x\n", (unsigned int)ptr);
	fprintf(stderr, "m,%d,0x%8.8x\n", size, (unsigned int)ret);
	return ret;
}

void *__wrap_calloc(size_t nmemb, size_t size)
{
	void *ret = __real_calloc(nmemb, size);

	fprintf(stderr, "c,%d,0x%8.8x\n", size*nmemb, (unsigned int)ret);
	return ret;
}

Now, I just ran the newly compiled program and directed stderr to a file, and then ran the malloc_analyze.py program as before.

Finally, we can produce something useful from this data to analyze our heap usage. First, it’s good to take a look at the overall heap usage over time (or in this case, over number of allocation calls). You can do this easily enough in a pylab session:

x = csv2rec('fetchorigin_analyzed.txt', names=('type','size','addr','total','count'))
plot(x.total)
xlabel('Number of allocations')
ylabel('Heap Allocated (bytes)')

Total Heap Usage

With this graph, you can now choose some points at which you’d like to visualize the heap usage and fragmentation. For this example, I chose three points: 400, 800 and 900. I wrote a short script to take the analyzed data and plot it as a bar, color-coded to based on how much space is in use at a given point in the memory space. The general idea is:

  1. Find the first (lowest) address where heap memory is allocated.
  2. Map each addressable byte in the memory region to a 1 or 0 depending on whether it is in use
  3. Bin the data into reasonable sized chunk (e.g. 512 bytes) and calculate how much of the space is in use
  4. Produce a graph with a colored box for each bin, coded by how full it is.

I wrote a simple script to do this. It’s certainly not very efficient, but it works just fine for small data sets that I was dealing with. The script needs to be adjusted for the start address and points in time at which to produce a graph.

import csv
import sys
import cPickle
import numpy as N
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import matplotlib.colors as colors
import time

binsize = int(sys.argv[2])

fig = plt.figure()
p = 1

for end in [400, 800, 900]:
    mem = csv.reader(open(sys.argv[1]), delimiter=',')

    ax = fig.add_subplot(9,1,p)
    p = p + 1

    mmap = []
    mmap.extend([0 for x in range(0,1024*160)])

    mdata = []
    for row in mem:
        mdata.append(row)

    for row in mdata[0:end]:

        size = long(row[1])
        addr = long(row[2], 16) - 0x12c0010

        if row[0] == 'm' or row[0] == 'c':
            for loc in range(addr, addr + size):
                mmap[loc] = 1

        elif row[0] == 'f':
            for loc in range(addr, addr + abs(size)):
                mmap[loc] = 0

    print str(len(mdata[0:end]))

    mmapbin = []
    mmapbin.extend([0 for x in range(0,len(mmap) / binsize)])

    for x in range(0, len(mmap)):
        mmapbin[x / binsize] = mmapbin[x / binsize] + mmap[x]

    patches = ax.bar(N.arange(len(mmapbin)), [1 for x in mmapbin], linewidth=0, width=1)

    fracs = [float(x) / binsize for x in mmapbin]
    norm = colors.normalize(0, max(fracs))

    for thisfrac, thispatch in zip(fracs, patches):
        color = cm.jet(norm(thisfrac))
        thispatch.set_facecolor(color)

    ax.set_yticks([0, 1])
    ax.set_ylabel(str(end))

    fig.show()

And the result you get is a nice picture that can give you an intuitive view of how your heap is fragmented.

Visualization of the heap at three points in time

The graph could be cleaned up, but you can get a very good idea of what the heap looks like just from this graph. The x-axis units are in bin sizes, and could be converted to memory addresses to be more useful.

This entry was posted in Uncategorized. Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>