The script: xfolk-colorize

The script, xfolk-colorize, performs a ridiculously simple function. It identifies portions of web pages formatted using the xFolk microformat and changes their background to a vibrant and inspiring red. The script is almost entirely a remix of scripts found in the tutorial, “Dive into Greasemonkey”, written by Mark Pilgrim. The key processing strategy in xfolk-colorize is to use XPath to identify the microformat and then use dhtml scripting methods to modify the DOM. These technologies are well-suported in both Firefox and Internet Explorer, leading me to believe the scripting strategy, if not the code itself is highly portable.

To play with xfolk-colorize, you need to follow these steps:

• Install Firefox if you do not already have it.
• Install greasemonkey into firefox as described at this link. The latest version of greasemonkey eliminates the recently publicized security issues, and xfolk-colorize never used the affected functionality anyway.
• Install the xfolk-colorize script by clicking on this link and then selecting Tools > Install User Script from the Firefox menu.

You can see xfolk-colorize in action at two very different sites. Here is how the script alters the front page at my linkblog de.licio.us webcites:

A key feature of this site is that it tends to be valid, well-formed xhtml. This type of page presents few challenges for almost any reasonable processing strategy in a modern browser. More challenging is when the page is not valid or even well-formed xhtml. This situation arises quite often in sites with user-generated content. Such is the case with de.lirio.us, and xfolk-colorizer performs like a charm there as well as can be seen in this screenshot:

xfolk-colorize works on this site because it takes advantage of Firefox's ability to construct reasonable DOM trees for sites that are not valid xhtml or even well-formed. Internet Explorer possesses a similar capability.

How xfolk-colorize works

Here are the ten lines that make xfolk-colorize go:

//This function identifies all instances of xFolk entries
allDivs = document.evaluate(
    “//*[contains(@class, 'xfolkentry')]”,
    document,
    null,
    XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE,
    null);
//This loop changes the background of the individual xFolk entries
for (var i = 0; i < allDivs.snapshotLength; i++) {
    thisDiv = allDivs.snapshotItem(i);
    thisDiv.style.backgroundColor = 'red';
}

The function I use to identify xFolk microformatted content is document.evaluate(). As explained by Mark Pilgrim, this function can be used to select an array of nodes based on an XPath expression. Here, that expression is: “//*[contains(@class, 'xfolkentry')]” which has the effect of selecting any element in the document where the value of the class attribute contains the string 'xfolkentry'. Examining the xFolk spec, this class attribute value indicates that the element by standard convention does indeed contain an xFolk entry. I have yet to find a microformat that does not have an attribute value to indicate a containing element, so this strategy should be pretty generally applicable.

Altering the XPath expression to identify microformats other than xFolk frequently involves only changing the text string 'xfolkentry' to the attribute value that identifies the element containing the target microformat. Infrequently, one must also alter @class to indicate that the expression applies to something other than the contents of the class attribute. Here are two examples:

• “//*[contains(@class, 'vcard')]”: Identifies all instances of hCard formatted content in the html page. These are identified by elements with a class attribute containing the text string 'vcard'.
• “//*[contains(@rel, 'tag')]”: Identifies all instances of reltag formatted content in the html page. These are identified by elements with a rel attribute containing the text string 'tag'. However, since reltag also specifies that these elements must be anchor <a> tags, we can be even more precise with “//a[contains(@rel, 'tag')]”.

This last example brings up a complaint I often hear about XPath. It's too arcane and complex. How did you know to do all of that? Mark Pilgrim recommends this online tutorial for learning about XPath which I personally found too abstract when I first started a few years ago. I myself recommend that beginners read one of two relatively inexpensive books: XSLT for Dummies or XPath and XPointer by O'Reilly.

More advanced extensions

Just altering the background color of a bit of microformatted content to a rich, vibrant red will seem like a big ho-hum to many readers. The value of this script is more in the strategy it introduces for identifying microformatted content. Two, more ambitious, extensions come to mind:

Creating rich interfaces to web applications

Microformatted content is intended to more explicitly convey meaning through its structure. Some microformat specs describe content so well structured that it could serve as the input or output of an application. xFolk, hCard, and hCalendar are three specs which immediately come to mind in this category. hReview and any potential location microformats may also be suitable.

An issue with many server-side microformat processing strategies is that they require processing the whole web page to “discover” microformatted content. This process has the inconvenience of taking control out of the hands of the user. If more than one microformatted element is present in the document, which was intended for processing?

Using the strategy I have outlined and illustrated here, it should be possible to identify microformatted content in the browser and then add elements to the web page that allow that content to be manipulated in various ways. For instance, one might send a bit of xFolk microformatted content to be republished in his or her link blog. The typical O'Reilly network article which has a specific recommended citation format would be perfect for this treatment.

Alternatively, one may wish to send one bit of hCard formatted content among many such bits on a page to his or her online “met” hcard repository. The identification strategy I have illustrated here could be used to add a form to each bit of microformatted content on a web page that would permit just that bit to be sent for processing.

I note in passing that were George Hotelling to use XPath to identify hCalendar microformatted content in his very useful hCalendar converter greasemonkey script as I have done in my simple example, he could eliminate a substantial amount of his code. The resulting code might be more robust to boot.

Creating microformat validation test suites

Microformats such as xFolk and reltag have given sets of components, some of which must occur fixed numbers of times or in specific forms in order for the format instance to be considered valid. Currently, page authors must interpret spec documents to determine whether the formatted content they are creating is valid. Using the microformat identification strategy I have shown here, one can apply tests written in javascript to each bit of microformatted content to determine which aspects of an author's effort do and do not live up to spec.

The advantages of this test-driven approach for checking microformat compliance over interminable email back-and-forth should be obvious. The page author can do most of it themselves and eliminate the non-obvious issues before engaging the spec author.

Obviously, test-driven approaches favor more tightly specified microformats and may not be appropriate until the format has reached a given level of maturity. Alternatively, if one believes in test-driven design, then a test-driven approach is appropriate from the earliest design phase.

Extending this approach outside of greasemonkey

XPath and dhtml scripting are not greasemonkey or Firefox specific. Greasemonkey just happens to provide a remarkably easy-to-use platform to experiment with user scripting. It also eliminates the complexity of having to think cross-platform when laying out core processing strategy.

It would seem to me that a natural progression would be to take some of the ideas I have layed out here and move them into cross-platform bookmarklets that would re-render a page's content based on the available microformats. These efforts might use del.icio.us direc.tor as an archetype.

Caveats

The strategies I am proposing here assume that the person repurposing the microformatted content has the right to do so. When people publish their business cards or calendar entries in hCard or hCalendar, it seems only reasonable that they are putting them on the web for the taking. Publicly published linkblog entries seem also to frequently fall under this heading. More extensive writings such as full blog posts clearly require the author's permission for republishing.

Acknowledgements

Ryan King originally got me thinking in the direction I have outlined here after he pointed out the behavior.js library which uses CSS selectors as an answer to the complexity of the XPath selectors in Brian Suda's X2V. Since CSS selectors only use the class and id attributes along with element type, I don't think they offer a good enough solution for microformats. But I do agree with the basic direction that that initial conversation got me going, and I thank Ryan for the reference.

Carl Beeth and Dimitri Glazkov outlined javascript and CSS selector strategies for manipulating microformatted content to me in private emails. Their solutions are potentially more encompassing than the small, didactic example I have provided here. Hopefully, they will be able to use this work to inform their own.

Mike Migurski, Mike Frumin, and I have been discussing how microformats might work with reblog, a popular, open source RSS republishing tool. Trying to think through how that would work motivated me to try to find a solution to the (at least) perceived microformat identification and selection problems. I expect this post to be the first in a series as we work toward solutions there.

Marc Canter and Lucas Gonze have been working on a number of projects like “Redirect This!” to make republishing of micro-content easier. Their approach extends beyond microformats and does not directly address the issue of identifying micro-content in a web page. Personally, I think focusing on microformatted content is likely to yield more progress. Since microformats derive from xhtml, the building blocks of today's web, they can take advantage of all of the infrastructure built for xhtml. XPath, greasemonkey, and dhtml scripting are all parts of that infrastructure.