Jerome has released a 2.0 beta version of his famous Jerome’s Keywords plugin, on which my tag cloud widget is based. Some of the functions I used in my widget are no longer available, so the previous version of my widget will not work with it. Thus, I have updated the widget, which you can download below.

Jerome seems to have fixed the problems in the plugin itself that I mentioned here and here. When he releases this version from beta, I’ll remove my corrected versions. I’ll go ahead now and add updates to let people know about the beta.

Jerome also included all of my widget’s options (other than Title) in a new “Tag Cloud Display” section of the plugin’s options. So, I have removed those options from the widget option panel. Please go into “Options/Jerome’s Keywords” and update your tag cloud options there.

Note also that this version of the widget is not compatible with earlier versions of the plugin. If you need a widget for earlier versions of Jerome’s Keywords, get version 1.1 of the widget here.

UPDATE: Get the latest version with improved cache control here.

Mike Koepke reported a problem he was having with the Jerome’s Keywords plugin when he searched on a tag that contained a single quote:

WordPress database error: [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's Creek' OR jkeywords_meta.meta_value LIKE 'Wilson's Creek,%' OR jkeywords_meta.' at line 1] SELECT DISTINCT * FROM wp_posts LEFT JOIN wp_postmeta AS jkeywords_meta ON (wp_posts.ID = jkeywords_meta.post_id) WHERE 1=1 AND post_date_gmt


I was unable to reproduce the behavior on my version, but it appears from the error message that the quote is not getting “quoted“. So, I added the following line to the “keywords_postsWhere” function in jeromes-keywords.php:

$keyword = str_replace("'", "''", $keyword);

Using the simple quote-doubling escape route. This solved the problem for Mike, but it didn’t work with my version, and from the resulting error message I could see that in my version the quote was already being escaped via a backslash (\). Modifying my fix to use backslash quoting didn’t work for me, either, because then the backslash gets backslashed.

Comparing versions, I was using MySQL 3.23.54 with PHP 4.4.1. Mike is on MySQL 4.1.14 and PHP 5.0.4. Then Mike tried this out on a local server with MySQL 5.0.22 and PHP 5.14, and got the same results as I did. So we both concluded that something got broken in MySQL or PHP between my early version and Mike’s live version, but has since been fixed again. Or was there some option that needed enabling on Mike’s live server? My Google searches for an explanation of this phenomenon were fruitless.

Clusty to the rescue. Clusty lead me to this post. It turns out that PHP has a runtime option, magic_quotes_gpc, that is enabled by default. This option automagically escapes quotes, backslashes, and nulls in a GET, POST, or cookie with a backslash. Bingo.

As with most (all?) PHP options, you can query the setting of magic_quotes_gpc at runtime via (20 guesses?) get_magic_quotes_gpc(), natch. So I changed my fix as follows:

if (!get_magic_quotes_gpc()) {
$keyword = addslashes($keyword);
}

You can download the corrected version below. If Jerome would like to adopt this fix in his master version, I’ll take down my copy and point to him instead. Johannes, you might want to implement a similar test in your Jerome’s Keywords Manager plugin. Sorry to give you one more place to go after your “final destination”.

UPDATE: Jerome has released a 2.0 beta version of the plugin that appears to address this issue. Mike Koepke, can you verify? If you’re using my tag cloud widget, you’ll need version 2.0 of that as well.

In case you didn’t notice, I’ve converted Chip’s Tips into a WordPress blog. How do you like the new look?

I started Chip’s Tips back in 2001, before most people had ever heard of blogs. Yet, that’s more or less what it was from the start. I created my own XML grammar for outlining the topics, and another grammar for the posts themselves. Then I wrote Active Server Pages (ASP) to transform those XML documents into HTML pages using XSLT. It was my first project in both ASP and XSLT, and it worked quite well.

Lately, though, it’s been showing its age. ASP (not ASP.NET) is now an old technology. Blogs have evolved rapidly, and I wanted to be able to easily implement several key features:

1. An RSS feed for the posts
2. Comments and trackback/pingback
3. Tagging and multiple categories per post

Besides, my process for posting was less than smooth. I would copy an existing XML document for a post, modify its content, add an entry for it in the outline XML file, then upload both of those plus any download file. Often I would create unmatched tags in the body of the topic file so the XML parser would fail. Count on at least fifteen minutes of fiddling in addition to however long it took me to write a post. With a blog, all of that is automagic.

So, how did I migrate?

First, I needed a content migration strategy. I wrote a PHP script to read the existing outline and topic XML files and generate an RSS feed. That way, I could use WordPress’ RSS importer to pull the content.

Second, I set up the WordPress site and tweaked the theme I was using for Chip’s Quips to work here. Different color scheme, some new pages, etc. Add the Google ads and search, and the scripts for Technorati and ReefeRSS.

Third, I applied the same hack I used on Chip’s Quips to redirect the feeds, and burned the feeds at FeedBurner. Then I setup landing pages at FeedPass. Added the subscribe links along with the Chicklet generator and R-mail widget to the sidebar.

The hardest part was setting up all of the new categories (I wanted to reshuffle those) and tag all of the existing posts. I also included one unique tag per post that matched the original mnemonic from the old system.

Next, I modified showtopic.asp to permanently redirect to a tag search for the mnemonic requested, so any existing external topic links will be unbroken. Thanks to Steven Hargrove for providing the most excellent way to do this in ASP, as well as just about any other web language). Then I redirected default.asp to the main page, chip.asp to the “About” page, and resources.asp to the “Resources” page. I’m leaving topics.asp alone, because attempting to map from the old categories to the new proved too daunting. So, any external links to a general topic will still show the old outline page, but clicking on a link to a topic there will take you to the corresponding page on the new site.

Last but not least, I converted licensing to the CCD CopyWrite license, which I have also adopted for Chip’s Quips. No big change there from Creative Commons Attribution – you’re free to copy and reuse this content to your heart’s content, provided you give appropriate credit to yours truly.

Enjoy, and please feel free to comment!

This little Ruby on Rails application demonstrates some of the capabilities of AJAX with Rails.

The Game

Ten rats, chosen at random from a database, race against each other along a laned track to reach the cheese. Sometimes they move backwards, but ultimately one of them wins, another one places, and a third one shows. Their movements are controlled by a random number generator, biased according to their race history. That is, rats that have done well in the past are favored, which is reflected in their displayed odds. The race is updated once a second (or thereabouts), via XmlHTTPRequest (aka AJAX) in the standard Rails fashion using ActionView::Helpers::JavaScriptHelper#periodically_call_remote.

Quick Start

First, install Ruby and Rails. Unzip the download into a directory. Load the “ratrace” database from ratrace.sql. The database.yml file in the rats\config directory is setup to use MySQL, username “root”, no password. Make any changes you need to that file. cd into the “rats” directory, and startup the WEBrick server which is included by Rails. On Windows, this would be accomplished with the command:

ruby script\server

Now bring up your favorite browser and navigate to http://localhost:3000/rats/run. Make sure that if you have a firewall enabled, you are not blocking local access to port 3000.

Performance

Even though updating the race only once per second gives the game’s animation a distinctly “freeze-frame” flavor, it still seems to be a bit too quick for a browser-based applicaton. To see this, start the race. Let the rats get going a bit, then try clicking on the “Rats” link near the top of the page. On my system (1.8 Ghz P4 running XP SP2) using a local WEBrick server, I get the following very unscientific results with different browsers:

I haven’t tried the IE 7 beta yet. By viewing the WEBrick server’s output, I can see that Maxthon and IE6 have queued up many of the “update_race” calls, and these continue to be invoked even after successfully navigating to the Rats:list page. If you let the race run long enough, you can get so many of these queued up that you can’t ever get to the Rats:list page, and even after you close the browser the calls continue to come into the server. Several extra “update_race” calls occur with Opera, and only a few in Firefox. Thus it would seem that Firefox either manages to more quickly terminate the timer on navigation away from the page, or else it allows fewer calls to become queued. I don’t think it is processing the calls any faster, because the race appears to run at about the same pace in all of the browsers — although that is difficult to benchmark, given the random nature of the application.

This game has a long and fun history for me beginning in the 1970′s. See my blog on that here.

If you get good traffic to your WordPress blog, the sidebar may be displayed many times per minute. You want that to occur efficiently. So, if the content of a widget doesn’t change often, but requires quite a bit of processing or a database access in order to construct it, then caching the output makes a lot of sense.

After implementing caching in two of my widgets (opml-blogroll and keywords-tagcloud) it occurred to me that a template for a cached widget could prove useful to widget developers. So here it is.

This widget includes controls for the cache expiration (defaulting to 900 seconds = 15 minutes), as well as whether to display caching debug info. If the latter option is checked, the widget will output “(cached)” or “(not cached)” at the end of its content, depending on whether the content was found in the cache or not, respectively.

Note that saving changes to the widget in the dashboard releases the cache immediately.

This example will function as a widget, though it isn’t very useful. To create your own cached widget, copy the file, replace all “cached-example” and “cached_example” with a string that names your widget, add your options to the _control function, and add your content to the _content function.

If you’ve ever worked on a web server that doesn’t report Perl errors back to the browser, then this script is for you. Not only will it send the errors back to the browser, but also the output of the target script up to the point of failure. All you do is pass the target script as a “script” parameter to cgidebug.pl:

http://www.yoururl.com/cgidebug.pl?script=yourscript.pl

If you need to pass parameters to your script, just add them as parameters to cgidebug.pl:

http://www.yoururl.com/cgidebug.pl?script=yourscript.pl&param1=param1&param2=param2...

If you’ve used xfServer much, you already know that you can improve its performance by compressing the data both from the server and the client. You do this by setting SCSCOMPR=1 in the client’s environment, and by turning compression on in the xfServer service via the Synergy Configuration Program on the server. Unfortunately, xfServerPlus does not provide a similar compression mechanism, and when transferring large amounts of data (using the new “handle” argument type, for instance), you could really use it. In one test, transferring a data item in excess of 5 MB over a 100 Mbit link took about 2 and a half minutes, which is far longer than even a web client is likely to want to wait for it.

The answer: compress the data yourself. I invented a compression algorithm that is 8-bit character compatible, and I created implementations of it in C++ and C#. On the Synergy (server) side, create a DLL in C++ and include the C++ routines as entry points. You can then call them from your Synergy code using the SynergyDE DLL interface. If your client uses xfNetLink.NET, add the C# class to your project. You can now pass compressed data in the handle argument. In my test case, this reduced the 2:30 to just over 16 seconds, including compression/decompression time. If your client uses xfNetLink COM Edition, include the C++ version in a COM component. I haven’t gotten around to translating this into Java yet, so if you use xfNetLink Java Edition, that will be an exercise left for the reader.

UPDATE: xfServerPlus now has a compression option  (XFPL_COMPRESS=ON).  I found, though, that using both my own compression and XFPL_COMPRESS yielded the best performance.  Weird, huh?

Mostly for performance reasons, Synergy/DE’s xfServerPlus product uses a special version of the Synergy runtime for processing remote execution requests. This runtime is dbs.exe, which has all of the user-interface features stripped out, except for limited console I/O. In most cases, dbs.exe provides lightweight support for all you need to do on an application server.

However, some features of the Synergy runtime that were developed for user interface support can be used in a non-interactive way to provide useful server-side features. One example is an approach to creating printed output that uses Synergy UI Toolkit windows to format the fields (via I_DISPLAY) and then extracts the formatted text to send it to the printer (see Using UI Toolkit windows for print formatting). Unfortunately, Synergy Windows (and therefore Toolkit windows) are not supported by dbs.exe.

Not to worry. You can create a second xfServerPlus service that uses dbr.exe (the full Synergy runtime) instead of dbs.exe. Simply create a new service with a unique port number (the Synergy Configuration Program in version 7.3 and up is the easiest way to do this), and in the environment settings for the service set XFPL_DBR=1. This instructs xfServerPlus to use dbr.exe instead of dbs.exe. Connect to this service using its unique port number, and you’re in. The terminal number (%tnmbr) for the process is still -1 (indicating a non-UI mode). If you need to distinguish this process from a dbs.exe session, set your own custom environment variable(s) in the environment settings for each xfServerPlus service.

Of course, dbr.exe has considerably more overhead, both in memory/resource usage and in startup time, so you don’t want to use it if you don’t need it. For applications that require Synergy Windows support, however, it sure beats rewriting them.

Different browsers and versions of browsers have different capabilities that can affect what kind of JavaScript or HTML code will execute successfully. The downloadable code below contains two JavaScripts and an Active Server Page. I used an Active Server Page merely in order to include the two JavaScripts for demo purposes, since my web server uses IIS. You can use the JavaScripts with whatever server page include mechanism you use, or simply paste them into your HTML page.

The first JavaScript, nnBrowChk.js, is a publicly distributed script from Netscape that provides a wealth of information about the browser, derived from the navigator.UserAgent property.

The second JavaScript, dombrowser.js, was generously provided by Patrick Crever (formerly) of Synergex International Corporation in Gold River, CA. This script uses DOM (Document Object Model) to determine browser capabilities.

To determine browser capabilities in VBScript on the client side or the server (IIS) side, see Determining the client’s browser type and version in VBScript.

Different browsers and versions of browsers have different capabilities that can affect the type of HTML or client-side script code that your Active Server Page should generate. The downloadable code below contains an Active Server Page that demonstrates three methods for determining the client’s browser type and version. The first two methods are executed on the IIS server using the HTTP header from the client. The third is executed in the browser itself in VBScript. The example also shows how to determine whether or not the browser can execute VBScript.

To determine browser capabilities in JavaScript on the client side, see Determining the client’s browser type and version in JavaScript topic category.

XSLT provides an abstract and flexible means of formatting XML data for output. Going well beyond the concept of “style sheets”, XSLT contains powerful elements that almost qualify it as a full-blown programming language. The xsl:call-template in conjunction with xsl:with-param and xsl:param tags even allows for encapsulation and reuse of common routines with parameters. Sometimes, however, the parameters you need lie outside the XML documents in the external operating environment. For instance, when generating web pages it may be useful to have access to the parameters passed on the HTTP request.

While XML and XSLT do not provide a direct means of accessing external information, you can pass parameters to a transformation by adding nodes to one of the documents prior to performing the transformation. The downloadable example below shows how this can be done in an ASP, using the Microsoft DOM parser to load the XML documents, add the parameters as elements in the data document, and then perform the transformation to generate an HTML page.

To test the example, you will need to copy the files to an Internet Information Server (IIS) or Personal Web Server, and you must have installed the Microsoft DOM parser on the server. Click here for the link to Microsoft’s page for downloading the parser.

Once you have everything set up on the server, request the testparams.asp page in a browser. If the parameters contain “show=desc”, then the full description of each node of the testparams.xml document will be displayed. Otherwise, only the name of each node is displayed. All parameters passed to the page are displayed with their values at the top of the page. Try it with different parameters to see the results.