Perhaps the most critical element for unit testing is a test framework. In this blog post, I'll discuss various options for the C language, and provide a brief analysis of pros and cons.

I've experimented with all of the below frameworks, but the one I use on a day-to-day basis is Unity.

Considerations

Depending on your situation, there are some important factors to consider when choosing a test framework. Below I've listed a couple which I've had occasion to think a lot about.

Test Runners

Most embedded programming uses C, and all the frameworks I discuss below are for C or C++. This in and of itself can present some difficulties. Many unit test frameworks take advantage from language features known as "introspection." Effectively, this is the ability to determine information about the structure of the executing program at runtime. It can be useful for tests that self-register with the framework, and is used extensively for frameworks targeted to languages like Ruby, Python, and Java.

Practically, this means that test frameworks in C require more setup work on the part of the programmer. Most frameworks will need a manually written "runner" which invokes the test suite, leading to possible errors such as forgetting to add a particular test to the runner.

There are tools to help automate this process, but they fall outside the purview of a strict "test framework"--they're more akin to full build systems in their own right. I'll address these options in a future blog post.

C++ has some features allowing introspection, so test suites for that language can implement automatic test registration. In most cases, it's possible to compile and test a project written entirely in C as C++. This can often be a good solution, but can be jarring for programmers unused to the language.

Running on the Hardware

The ThrowTheSwitch team has a great writeup discussing the best location to run tests. I won't repeat their reasoning here, but I will note that it lays out three different options: the development system through a simulator, the workstation natively, or the system hardware. Generally, it makes sense to use one of the first two options, but there are situations (or different tests) where it might make sense to do otherwise.

With that said, the choice of your test framework will factor into whether it's possible to do any of these. Depending on the type of simulator available for your platform, it may be difficult to use a tool which depends on being run under an operating system (such as check). This will certainly not work on the bare-metal hardware itself. On the other hand, having features like the ability to run tests in separate address spaces can be really nice when a failing test might cause a segmentation fault.

Framework Options

There are many more test frameworks available than the ones I discuss here. My main criteria for inclusion in this list was a project still under active development. With that said, some of the ones not mentioned (CUnit, embUnit, AceUnit) may fulfill your requirements as-is, without the need for a developer team to respond to bug reports, feature requests, etc.

Unity

As I said before, I use this test framework every day. It's quite lightweight, and highly portable. The developers claim that it can be ported to just about any architecture, from a high-powered x86 machine to an 8-bit microcontroller, including ones with esoteric features such as weird integers widths. It's specifically targeted to embedded development.

The price it pays for its portability is a dearth of high-level features like address space isolation, or automatic runner generation and test integration.

It integrates well with the ThrowTheSwitch suite of tools, which includes CMock, CException, and Ceedling. When used together, many of the disadvantages of Unity's simplicity are mitigated.

Language: C

Pros:

• Lightweight
• Highly portable
• Actively developed
• Integrates with mock framework
• Highly configurable (i.e. to support custom data types)

Cons:

• Relatively simplistic
• No address-space isolation
• No automatic test registration (but see above about Ceedling integration)

check

Check is another relatively lightweight C unit testing framework. Unlike Unity, it's not designed with bare-metal embedded development in mind. Instead, it focuses on being safe and simple to use. It's main claim to fame is that it runs each test in an isolated address space, which means that invalid memory accesses can be caught. In contrast, a test using Unity will die outright on a segmentation fault.

It has a few more convenience functions than Unity as well, including parameterized tests which can be convenient for writing routines that should validate output over a range of different values.

check pays some prices for running its tests in separate processes. The first is, clearly, a dependence on the operating system. This means that it would be very difficult to directly port to an embedded system, and may have trouble when cross-compiled to run under a simulator.

Secondly, spawning a child process is far more expensive than a simple function call, so check tends to bog down when it has to run many tests.

Like any other pure-C framework, test runners have to be written manually.

Language: C

Pros:

• Runs tests in an isolated address space

Cons:

• No automatic test registration
• Relies on OS support
• Can be slow
• Less configurable for esoteric platforms
• No direct mock framework integration

Google Test

GoogleTest is Google's unit test framework. It's written in C++, but can be used to test C applications with a little tweaking. It supports the use of Google Mock, which is necessary for testing against hardware-dependent modules.

GoogleTest does support running tests in separate address spaces, but doesn't do this by default. These tests are called "Death Tests," and they're mostly used for catching expected failures, rather than a protection against unexpected segfaults and the like.

This test framework has far more features than I have the space for here, but its documentation is very good, and worth glancing over.

Language: C++

Pros:

• Automatic test registration
• Works similarly to most familiar XUnit (i.e. JUnit) test frameworks
• Integrates with mock framework
• Possibility of isolated tests
• Very full-featured

Cons:

• Relies on OS support
• Less configurable for esoteric platforms

Interested in learning more about Boulder Engineering Studio? Let's chat!