Synthesis 2.0.1: chasing bugs out of the woodwork with coverage analysis
Sterling Camden
I just released version 2.0.1 of my Synthesis library for Synergy/DE, which you can download below.
As you can see from the change log, this version grew mostly from the application of my tools for analyzing code coverage to my own creations. By fleshing out my own unit tests to nearly 100% coverage, I shook out all nine corrections listed for this version. So you can see from my own example that achieving 100% test coverage is not merely an academic exercise. Code always looks more solid when it’s never executed.
These tests also revealed some areas in which my code coverage analysis needed improvement. The six enhancements listed for the CodeCoverage class all came from those observations.
In fact, this version includes only one change that did not spring from my automated navel-gazing: the addition of the ‘x’ option to Regex. This, of course, allows white space, line breaks, and comments within a regular expression. My other enhancement to Regex (optimizing out unnecessary code) was a result of code coverage analysis — because otherwise I might never have determined that the code removed was not necessary.
My code coverage only suffers from one major flaw, and that is not its own: the debugger (on which it relies for sampling) sometimes exits prematurely when stepping through code (Synergex tr#30947). You can tell when this happens by examining the coverage report for your mainline to see where coverage stops. Unfortunately, one of the places where it regularly happens is within U_START, so analyzing an application that uses the Synergy UI Toolkit shall be an exercise in futility until such time as that tracker shall have been resolved.
However, I would highly recommend using this utility to analyze test coverage for code that doesn’t rely on the Toolkit – as well as for Toolkit code once that debugger bug is fixed. It’s certainly pointed out some oversights in my code.
I have a few ideas on how to enhance this further:
- Currently, the command line tools only include sampling from one executable. I’d like to add the ability to sample multiple runs, combining their statistics. That way, you could sample a whole test suite and see how well your code is covered by it.
- Unreachable code is currently included in the analysis. For instance, code that’s .ifdef’d out, or an mreturn at the end of a method that always throws an exception (the mreturn being required by the compiler). I’m thinking of creating a way to tag code as “not to be included in coverage analysis”.
- Once analyzing Toolkit code becomes possible, I can foresee sampling taking a lot longer than it needs to, because most users wouldn’t want to include the internals of Toolkit routines in the analysis. I’d like to create a way to specify the source files that you want to include in sampling, so that the sampler could simply skip over the rest.
Anything else that you’d like to see here?
Posted in Regexen, SynergyDE, UI Toolkit |
1 Comment » RSS 2.0 | Sphere it!




