I worked for about a year on an UnCommon Web-based application.  This was an interesting experience; UCW provides a great way to express complex behaviour in a Web application, as it is continuation-based, which enabled me to code app’s logic as regular control flow, complete with looping, conditions, etc., from time to time presenting user a form and receiving user-supplied values (form presentation was as simple as calling out to a function, which returned values supplied by user).  Cool, huh?  Currently, UCW seems to be mostly a dead project, and there is an alternative, which may be more interesting—namely, Weblocks—but I didn’t check that out yet.  What’s more, UCW, with all its complexity, had very little documentation, and what was avalilable, was mostly outdated, as the framework was constantly evolving at that time.  Not so cool, but it forced me to learn to RTFS when needed (with Slime’s M-. it was not as hard as it seems).  But I digress; UCW made use of some other projects of Bese (UCW developers seemed to have quite a bad case of NIH), including HTML output and templating library Yaclml—Yet Another Common Lisp Markup Language—which I really loved, and which, in my opinion, surpasses CL-WHO + HTML-Template in many ways.  Unfortunately it is almost undocumented, which is what I’ll try to fix in this article, along with comparing Yaclml to its Ediware counterparts.

Formatting HTML output

Edi, when designing CL-WHO, decided to use keyword symbols to represent HTML tags; Bese guys decided to create separate package for tags—it.bese.yaclml.tags, conveniently nicknamed <.  Let’s compare a simple example code:

;; CL-WHO
(with-html-output-to-string (s)
  (:ul (:li "foo")
       (:li "bar")))

;; Yaclml
(with-yaclml-output-to-string
  (<:ul (<:li "foo")
        (<:li "bar")))

Looks like a trivial difference, only on visual level (but this level is also important: we see the < sign, which is distinct visually and actually looks like an HTML tag, not a keyword that looks just like any other keyword scattered throughout the code), but it actually has many subtle consequences.  First, keywords can’t have a function or macro definition, at least not in a practical way without conflicting with anything.  Thus, with CL-WHO syntax, one has to explicitly pass the sexprs through sexp-to-html translation function or macro; escaping to Lisp code from HTML-describing sexprs, and back to HTML is not trivial and requires a separate macro layer to switch back and forth.  Using separate package to describe HTML tags makes it possible to implement tags as macros, which was what Bese guys did.  As an important side effect, this enables basic HTML validation support, and guards users from their own typos, which CL-WHO is simply unable to do.  Yaclml also avoided the stream variable binding (they just use a dynamically bound special variable to hold output stream), so we don’t see this binding contributing to visual noise.  Now, let’s see what happens with the code when we wish to use loop to generate list items:

;; CL-WHO
(with-html-output-to-string (s)
  (:ul (dolist (i '("foo" "bar" "baz" 23))
         (htm (:li (esc (princ-to-string i)))))))

;; Yaclml
(with-yaclml-output-to-string
  (<:ul (dolist (i '("foo" "bar" "baz" 23))
          (<:li (<:as-html i)))))

We see now that Yaclml actually includes a whole lot less visual noise.  There is an <:AS-HTML macro, which is necessary because tags are implemented as macros, not as functions, which allows for some optimizations, but requires some more thought from user.  Yaclml authors decided to ignore value returned by forms included in tag macros, and require them to write directly to YACLML:*YACLML-STREAM*.  There are helper macros <:AS-HTML (with shorthand <:AH), which quotes its arguments’ evaluation results to Yaclml output stream, and <:AS-IS (<:AI), which outputs evaluation results directly to the stream, allowing called function to generate its own HTML, but also requiring it to escape anything that needs to be escaped.

Using special variable YACLML:*YACLML-STREAM* instead of a lexical binding (which is what CL-WHO does) has one more benefit, when we want to delegate some parts of HTML generation to separate functions.  Let’s compare once again:

;; CL-WHO
(defun item-html/who (val)
  (with-html-output-to-string (s)
    (:li :class "item"
         (esc (princ-to-string val)))))
(with-html-output-to-string (s)
  (:ul (dolist (i '("foo" "bar" "baz" 23))
          (str (item-html/who i)))))

;; Yaclml
(defun <item (val)
  (<:li :class "item"
    (<:ah val)))
(with-yaclml-output-to-string
  (<:ul (dolist (i '("foo" "bar" "baz" 23))
          (<item i)))

The CL-WHO functions include very much noise, which Yaclml manages to avoid.  The visual distinction of tags with the < package alias helps with code readability.  We also use a convention to put formatting-only functions and macros in a separate package, with name starting with <, or to start their names with <.  This convention clearly indicates formatting/visual layer; for simplicity, I used function name starting with <, but in a larger project, I’d use separate package for formatting, nicknamed <project, so I would use <project:item here.

Could Yaclml HTML generation be better?  Sure thing.  It could step away from macros (at a performance penalty, though), and use functions, which would return and consume HTML fragments (or consume strings, or actually any Lisp values—including closures for lazy evaluation).  This would introduce first-class HTML fragment object, giving us sort of DOM and possibility of passing around and manipulating already generated HTML fragments, and wouldn’t requre the <:AH and <:AI helper macros (or actually would require only one of those).  That seems to be what core-server guys are doing with their DOM programming support.  I don’t use it yet, partly because I don’t need this kind of functionality and I’m more acquainted with Yaclml; partly because I don’t want to buy into core-server as a whole—core-server is a monolith and ripping DOM support out of it would require substantial amount of work—and partly because I use the second part of Yaclml, which I will describe in the following article: HTML and XML templating.