For complex embedded applications, it often makes sense to incorporate a real-time operating system (RTOS). Introducing an RTOS into your system helps manage that complexity, but it also comes at the cost of its own challenges.

Whether or not an RTOS is appropriate for a given application is beyond the scope of this blog post. We assume that you have already made that choice, and now need to manage the development and testing cycle.

Challenges

Without an RTOS, most embedded applications use one of two major architectures.

The first contains an infinite loop at the topmost level, which calls several "work functions." These functions generally drive internal state machines, in the process waiting for IO events and performing calculations. I'll refer to this as a "round-robin" architecture.

The second is slightly more complex. It uses a scheduler and some extra abstraction to provide "events" (at regular times, when IO becomes available, etc.) to which various modules can react. I'll refer to this as an "event-driven" architecture.

Though round-robin and event-driven architectures require their own testing techniques, they share one critical commonality -- both are "run-to-completion" models. That is, whenever a method is invoked (a work function or an event handler), it does its work and then completes. This is extremely convenient for testing, and fits the classic model where the test sets up a certain state, calls the function under test, and then validates the changed state.

In contrast, most (or all) RTOSes provide threading abstraction. An application developed for this kind of architecture will generally dispatch several virtual threads, each of which waits in its own infinite loop. To facilitate this multitasking, the OS provides blocking functions to wait on various synchronization primitives and IO. This can be extremely convenient from a functional standpoint -- but since most threads don't run to completion, it presents some issues to testability. A test which naively calls the function for such a thread's function will lock up the entire test suite, which is obviously unacceptable behavior.

Testing Strategies

There are a number of possible ways to deal with this particular issue. Which you choose depends on the needs of your particular application.

Please keep in mind that the code examples below are exemplary only -- they're not ready for a direct copy-paste into your project. You should read and understand them, and adapt them to your particular circumstances.

Loop Redefinition

In this case, we will simply modify the thread function so that it doesn't loop under testing. This is the most obvious (and perhaps simple) way to deal with this issue, but it comes with its share of disadvantages.

Imagine we have the following simplistic thread:

/* thing.c */

void thing_thread(void * arg)
{
    thing_init(arg);

    while (true) {
        thing_work(arg);
    }
}

Now, consider the following modification:

/* loop.h */

#ifdef TEST
#   define LOOP
#else
#   define LOOP while (true)
#endif /* ifdef TEST */

/* thing.c */

#include "loop.h"

void thing_thread(void * arg)
{
    thing_init(arg);

    LOOP {
        thing_work(arg);
    }
}

The redefinition of the LOOP macro allows us to call thing_thread() with impunity from our tests.

Here's a slightly more complex version, which allows control over the number of loops from the test:

/* loop.h */

#ifdef TEST
extern uint32_t __loop;
extern uint32_t __n_loops;
#   define LOOP for (__loop = 0; __loop < __n_loops; __loop++)
#   define N_LOOPS(N) __n_loops = (N)
#else
#   define LOOP while (true)
#endif /* ifdef TEST */

/* loop.c */

#include "loop.h"

#ifdef TEST
uint32_t __loop;
uint32_t __n_loop;
#endif /* ifdef TEST */

/* thing.c */

#include "loop.h"

void thing_thread(void * arg)
{
    thing_init(arg);

    LOOP {
        thing_work(arg);
    }
}

This overcomes one of the major disadvantages of the above -- namely, if thing_init() changes the test state, it can be hard to make sure thing_work() gets called with the right state if the loop only executes once. This is still not a perfect solution, since it doesn't allow quite the same degree of flexibility as some of the other solutions.

Pros:

• Extremely simple, just a couple lines of code

Cons:

• Production code is different from test code!
• Difficult to control state in the loop body (we always pass through thing_init())

Loop Breaking

The point of this strategy is to avoid the most egregious problem from the previous one: namely, that production code gets polluted with test artifacts, and behaves differently when under test!

setjmp() and longjmp() are two C functions which provide non-local control flow. A detailed tutorial is out of scope here, but what follows is a brief introduction.

setjmp() sets up a context to which longjmp() can later return execution. In effect, a call of longjmp(buf, val) will not return locally, but will cause the previous setjmp(buf) to return again with val as its return code. The first time setjmp(buf) is called, it will return 0, allowing this case to be distinguished. More detail can be found here, or you can take a gander at the man pages.

This is primarily useful when your test already "mocks" some of the underlying functions that it'll be using. Mocking itself is a topic for another blog post.

Consider the first example again, this time with thing_work() elaborated a bit:

/* io.h */

char io_get(void);

/* io.c */

char io_get(void)
{
    /* Do blocking, hardware, and OS-dependent stuff that won't compile on
     * our development machine */

    return char_from_io;
}

/* thing.c */

#include "io.h"

void thing_work(void * arg)
{
    /* io_get() blocks */
    char c = io_get();

    thing_process(c);
}

void thing_thread(void * arg)
{
    thing_init(arg);

    while (true) {
        thing_work(arg);
    }
}

Now, we'll make some modifications to allow it to be tested:

/* io.h */

char io_get(void);

/* io.c */

char io_get(void)
{
    /* Do blocking, hardware, and OS-dependent stuff that won't compile on
     * our development machine */

    return char_from_io;
}

/* fake_io.h */

#include "io.h"

extern jmp_buf io_get_buf;

void io_get_bail_after(uint32_t n);

/* fake_io.c */

#include <setjmp.h>

jmp_buf io_get_buf;

static uint32_t io_get_total_calls;
static uint32_t io_get_prev_calls;

int io_get_bail_after(uint32_t n)
{
    io_get_prev_calls = 0;
    io_get_total_calls = n;

    return setjmp(io_get_buf);
}

char io_get(void)
{
    if (io_get_total_calls >= io_get_prev_calls) {
        longjmp(io_get_buf, 1);
    }

    io_get_prev_calls += 1;

    /* Somewhere else, we'd allow the definition of these injected values */
    return test_value;
}

/* thing.c */

#include "io.h"

void thing_work(void * arg)
{
    /* io_get() blocks */
    char c = io_get();

    thing_process(c);
}

void thing_thread(void * arg)
{
    thing_init(arg);

    while (true) {
        thing_work(arg);
    }
}

/* test_thing.c */

#include "thing.h"
#include "fake_io.h"

void test_thing_does_stuff(void)
{
    /* Set up test state */

    if (io_get_bail_after(5) == 0) {
        thing_thread(arg);
    } else {
        /* Run done, check results */
    }
}

The above is considerably more complex than what we previously built up. But notice one key point: the code in thing.c, io.h, and io.c is completely unchanged. When the test for thing is built, of course, io.c will not be included -- instead, fake_io.c will. But since the test is not about how io.c works and instead about how thing.c interacts with the api defined in io.h, this is often acceptable. In fact, it's often critical to building tests that can compile and run outside the target hardware.

As an aside, the above sort of thing is quite a bit easier when using CMock and CException, two tools from the Throw The Switch collection. Thusly:

/* io.h */

char io_get(void);

/* io.c */

char io_get(void)
{
    /* Do blocking, hardware, and OS-dependent stuff that won't compile on
     * our development machine */

    return char_from_io;
}

/* thing.c */

#include "io.h"

void thing_work(void * arg)
{
    /* io_get() blocks */
    char c = io_get();

    thing_process(c);
}

void thing_thread(void * arg)
{
    thing_init(arg);

    while (true) {
        thing_work(arg);
    }
}

/* test_thing.c */

#include "thing.h"

#include "CException.h"
#include "mock_io.h" /* CMock will generate this and mock_io.c automatically */

void test_thing_thread(void)
{
    io_get_ExpectAndReturn('f');
    io_get_ExpectAndReturn('o');
    io_get_ExpectAndReturn('o');
    io_get_ExpectAndThrow(0xDEADBEEF);

    CEXCEPTION_T e = CEXCEPTION_NONE;
    Try {
        thing_thread(arg);
    } Catch (e) {
    }

    if (e != 0xDEADBEEF) {
        /* Test failed! Something got wonky and another (or no) exception
         * thrown
         *
         * This test may be overly pedantic, since CMock will generally
         * detect these cases as a failure to match the expected function
         * call sequence.
         */
    }

    /* Check the rest of test state */
}

Between CMock's automatic code generation for the mocked io module, and CException's hiding of the ugly setjmp()/longjmp() logic, we get the same result with a great deal less cruft -- and a more readable test to boot!

Pros:

• Leaves production code and test code the same (within a given module)

Cons:

• More complex, especially when not using a framework with appropriate tools
• Difficult to control state in the loop body (we always pass through thing_init())
• Non-local non-linear control flow can be terribly abused, and many developers will shy away from it even more strongly than goto

Loop Raising

This is a strategy which sidesteps the looping problem rather than dealing with it directly. With that said, it can isolate loop-aware tests to one section of your project, and allow a significant amount of code re-use.

Again, we return to the original example:

/* thing.c */

void thing_thread(void * arg)
{
    thing_init(arg);

    while (true) {
        thing_work(arg);
    }
}

In the example below, we elevate the abstraction of a thread to that of a 'task', which has an initialization and a body:

/* task.h */

typedef void (*task_func_t)(void * context);

typedef struct {
    task_func_t init;
    task_func_t work;
    void * context;
} task_t;

void task_start(task_t const * task);

/* task.c */

static void task_thread(task_t const * task)
{
    task->init(task->context);

    while (true) {
        task->work(task->context);
    }
}

void task_start(task_t const * task)
{
    /* Do some os-dependent stuff to start task_thread as a thread, with
    task as its argument */
}

/* thing.h */

#include "task.h"

extern task_t const * thing_task;

/* thing.c */

static void thing_init(void * context)
{
    /* Do initialization */
}

static void thing_work(void * context)
{
    /* Do periodic work */
}

static const task_t _thing_task = {
    .init = thing_init,
    .work = thing_work,
    .context = something
};

task_t const * thing_task = &_thing_task;

/* test_thing.c */

#include "thing.h"

#include "task.h"

/* There would be several of these */
void test_thing(void)
{
    /* Set up test state, probably setting variables in thing_task->context */

    thing_task->work(thing_task->context);

    /* Validate changed state */
}

/* main.c */

#include "task.h"
#include "thing.h"

void main(void)
{
    task_start(thing_task);

    while (true) {
        /* Go to sleep, blink an LED, or something else */
    }
}

By adding the task module, we can actually simplify other modules which use the standard init/work infinite loop architecture. We separate logical portions of the project.

The test for the task module itself will have to deal with the same looping problem, but it's both isolated and much simpler than a module doing meaningful work is. The test for the thing module can focus on testing its functionality, calling thing_work() as many or as few times as necessary without being forced to call thing_init(). And the test for main can simply ensure that it starts whatever tasks are necessary to form the final application.

Pros:

• Reasonably simple, depending on the task module. What I've laid out above is a functional, if bare-bones implementation, assuming an appropriate OS-dependent definition for task_start()
• Leaves production code and test code the same (within a given module)
• No non-local control flow
• More straightforward control of test state

Cons:

• Still requires another loop-management strategy for testing task and possibly main

• Requires function pointers, which can cause issues for stack-usage profiling and (possibly) safety. It's worth noting, though, that most or all schedulers in C will operate based on function pointers

  Keep in mind that as defined here, task deals with const pointers, which can mitigate the danger of passing function pointers. Depending on your architecture and linker settings, it's entirely possible that the entire thing_task struct could end up in program memory, where accidental modification will be much harder

Conclusion

As you can see, this is a complex issue. Any strategy will have its share of tradeoffs, and if you plan to implement any of them you'll have to think critically about what fits for your own situation. Additionally, this is by no means an exhaustive list of possible strategies. Your mileage may vary.

Interested in learning more about Boulder Engineering Studio? Let's chat!

Previous Blog Posts

Functional Test Fixtures: An Integral Part of PCBA Design	Building Raspberry Pi Disk Images with Docker: a case study in software automation