Chip's Tips for Developers

Contains coding, but not narcotic.

Synthesis 2.0.1: chasing bugs out of the woodwork with coverage analysis

December 31st, 2009 2:50:45 pm pst by Sterling Camden

I just released version 2.0.1 of my Synthesis library for Synergy/DE, which you can download below.

As you can see from the change log, this version grew mostly from the application of my tools for analyzing code coverage to my own creations.  By fleshing out my own unit tests to nearly 100% coverage, I shook out all nine corrections listed for this version.  So you can see from my own example that achieving 100% test coverage is not merely an academic exercise.  Code always looks more solid when it’s never executed.

These tests also revealed some areas in which my code coverage analysis needed improvement.  The six enhancements listed for the CodeCoverage class all came from those observations.

In fact, this version includes only one change that did not spring from my automated navel-gazing:  the addition of the ‘x’ option to Regex.  This, of course, allows white space, line breaks, and comments within a regular expression.  My other enhancement to Regex (optimizing out unnecessary code) was a result of code coverage analysis — because otherwise I might never have determined that the code removed was not necessary.

My code coverage only suffers from one major flaw, and that is not its own:  the debugger (on which it relies for sampling) sometimes exits prematurely when stepping through code (Synergex tr#30947).  You can tell when this happens by examining the coverage report for your mainline to see where coverage stops.  Unfortunately, one of the places where it regularly happens is within U_START, so analyzing an application that uses the Synergy UI Toolkit shall be an exercise in futility until such time as that tracker shall have been resolved.

However, I would highly recommend using this utility to analyze test coverage for code that doesn’t rely on the Toolkit – as well as for Toolkit code once that debugger bug is fixed.  It’s certainly pointed out some oversights in my code.

I have a few ideas on how to enhance this further:

  • Currently, the command line tools only include sampling from one executable.  I’d like to add the ability to sample multiple runs, combining their statistics.  That way, you could sample a whole test suite and see how well your code is covered by it.
  • Unreachable code is currently included in the analysis.  For instance, code that’s .ifdef’d out, or an mreturn at the end of a method that always throws an exception (the mreturn being required by the compiler).  I’m thinking of creating a way to tag code as “not to be included in coverage analysis”.
  • Once analyzing Toolkit code becomes possible, I can foresee sampling taking a lot longer than it needs to, because most users wouldn’t want to include the internals of Toolkit routines in the analysis.  I’d like to create a way to specify the source files that you want to include in sampling, so that the sampler could simply skip over the rest.

Anything else that you’d like to see here?

Posted in Regexen, SynergyDE, UI Toolkit | 1 Comment » RSS 2.0 | Sphere it!

WiPeD debugger for WordPress now has traceback

October 10th, 2009 4:13:41 pm pst by Sterling Camden

Sam Mela liked my WiPeD debugger for WordPress, but he wanted an easy way to include call traceback information.  So he added it, and he kindly shared his addition with me.

I’ve adapted what he did to my own tastes and included it in the download below.  Whenever you want to add the current call stack to the debug log, just call:

WPD_backtrace();

In order to test this, I added a new option in the Options page:

Trace the following actions (comma-separated)

Just enter the name of any action hooks on which you want to get a traceback added to the debug log.  This is not only useful for testing the plugin, it can also be used to find out where in the twisty little passages of WordPress each of the actions gets invoked.

Largely because of this feature, I now initialize this plugin whenever it is activated.  Previously I delayed setting it up until WPD_print was invoked.  So this plugin now imposes a small overhead if it is activated and not used.  Be advised.

I also adopted Sam’s idea of providing default styling (which I modified significantly).  If you don’t like it, you can remove or modify WiPeD.css.

Posted in CSS, PHP, Web, WordPress | 2 Comments » RSS 2.0 | Sphere it!

Debugging the Brainbugger

May 25th, 2009 2:32:24 pm pst by Sterling Camden

I just couldn’t leave it alone.  The charm of learning two languages at once by writing an interpreter for one using the other consumed my attention.  Furthermore, I had left the door open for adding debugging facilities, and then I found myself needing them.

Daniel Cristonfani’s Fibonacci generator had me hooked.  It’s so small, yet it produces a seemingly endless series of Fibonacci numbers in ASCII representation.  These soon exceed the storage capacity for 64-bit, yet his program just keeps on spitting them out.  I let it run until each one was nearly 1000 digits long.

You may call me a weak geek, but I wasn’t able to decipher what the code was doing in my head, nor with the aid of pencil and paper.  So I decided to create a debugger for Brainfuck, so I could step through the code and watch its progress.

You can download the updated Brainfuck interpreter in Lisp with the debugger (ibf.lisp) below.   Run that, and you get an “ibf>” prompt.  There, you may enter the following commands:

add <code>          - Add BF code to the end of the code stream
back                - Back up one instruction (does not alter data)
clear               - Clear all data cells
code                - Display the code, with () around the current instruction
data                - Display the data cells, with () around the current cell
del                 - Delete the current instruction
exit                - Exit (same as quit)
go                  - Execute to the end
help                - Display this helpful information
ins <code>          - Insert code before the current instruction
load <file>*        - Load instructions from file(s)
new                 - Start over with no instructions or data
quit                - Quit (same as exit)
repeat <count> <cmd>- Repeat debugger command the specified number of times
reset               - Clear all data cells and rewind to first instruction
rew                 - Move back to the first instruction (does not alter data)
s                   - Combination of step;code;data
save <file>         - Save instructions to a file
skip                - Skip the next instruction (does not alter data)
step                - Execute the next instruction 

Commands are case-insensitive, and may be combined on the same line separated by semi-colons (;).  Thus, to step through Daniel’s Fibonacci generator, the session looks like this:

ibf> load fib.b

ibf> s

>(+)+++++++++>+>+[[+++++[>++++++++<-]>.<++++++[>--------<-]+<<<]>.>>[[-]<[>+<-]>

>[<<+>+>-]<[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>[-]>+>+<<<-[>+<-]]]]]]

]]]]]+>>>]<<<]…

0 (0)

ibf> s

>+(+)++++++++>+>+[[+++++[>++++++++<-]>.<++++++[>--------<-]+<<<]>.>>[[-]<[>+<-]>

>[<<+>+>-]<[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>[-]>+>+<<<-[>+<-]]]]]]

]]]]]+>>>]<<<]…

0 (1)

ibf> s

>++(+)+++++++>+>+[[+++++[>++++++++<-]>.<++++++[>--------<-]+<<<]>.>>[[-]<[>+<-]>

>[<<+>+>-]<[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>+<-[>[-]>+>+<<<-[>+<-]]]]]]

]]]]]+>>>]<<<]…

0 (2)

ibf>

You can see the current instruction and the current data cell in context but highlighted by parentheses.  Using this tool, I found that Daniel keeps the numbers in decimal (one digit per cell) and simply adds 48 to output each one as ASCII.  Thus, he’s only limited by the number of cells that a specific Brainfuck engine supports.  Typically, that’s 3000 – but my version is only limited by available memory.  We could keep on fibbing all day.

Since the debugger supports inserting and deleting code as well as loading from and saving to a file, you could use it as a limited IDE for Brainfuck.  The reset command will clear the data and rewind the program counter, so you can rerun your modified program until it works.

Posted in Brainfuck, Lisp | 1 Comment » RSS 2.0 | Sphere it!

Creating a log of the debugger’s output

November 11th, 2005 11:17:22 am pst by Sterling Camden

The Synergy/DE debugger is a command-oriented tool that does not provide any mechanism for logging its output to a file. That can be especially frustrating on Windows, because the debugger only displays the last 25 lines of output, so you can easily lose track of what went on before. But there is good news. I just saved a bunch of money on my — no, Sorry. There is a way to get this output to a file as you go.

When the output from dbr.exe is redirected to a file or a pipe, the debugger’s output goes there, too. It also works for redirected input, but that doesn’t help us here (although it could possibly help with automated debugging…hmmm). Anyway,

dbr -d myprog > myprog.log

isn’t very useful, because then you can’t see the debug output as you are running the debug session. However, there is a utility that first appeared on UNIX called ‘tee’. The tee utility sends its standard input to standard output, and also to any filename specified as a command argument. There are many versions of tee available, but if you need a free one for Windows, click thou hither. Using tee, the following

dbr -d myprog | tee myprog.log

sends the output of the debugger to the console as well as to myprog.log. On Windows, this has an interesting effect. You must focus the debugger window to enter commands, but the output goes to the console window. So, I usually place them one above the other so I can see what is going on. Not so on UNIX or OpenVMS, where all of the I/O goes to/from the terminal. On the other hand, the log file on Windows only contains the debugger commands and output (and any low-level terminal I/O the application might perform), because UI Toolkit output is not redirected. That makes the log file much more comprehensible than on UNIX/OpenVMS, where all of the application’s screen I/O is included in the file as well. UNIX and OpenVMS would also include any session with the window debugger (WINDBG at the debugger prompt), while on Windows any attempt to bring up the window debugger is ignored when output is redirected.

Another perhaps better solution to this problem is to use the remote debugging facility in conjunction with a Telnet client that allows you to record its session. Here is a batch file that will start a Synergy/DE program in remote debug mode on port 2500, then attach to it via Microsoft telnet, logging the output to debug.log:

@echo off
start dbr -rd 2500 %*
telnet localhost 2500 -f debug.log

This approach works nicely because both debug input and output will go to the console window. You can also use a similar approach with a program running on UNIX or OpenVMS, and avoid getting all of the application screen I/O in the log file. Only the content of the telnet session (the debugger’s I/O) will be included.

Posted in OpenVMS, SynergyDE, Unix, Windows | No Comments » RSS 2.0 | Sphere it!

Better Tag Cloud