I worked for about a year on an UnCommon Web-based application.  This was an interesting experience; UCW provides a great way to express complex behaviour in a Web application, as it is continuation-based, which enabled me to code app’s logic as regular control flow, complete with looping, conditions, etc., from time to time presenting user a form and receiving user-supplied values (form presentation was as simple as calling out to a function, which returned values supplied by user).  Cool, huh?  Currently, UCW seems to be mostly a dead project, and there is an alternative, which may be more interesting—namely, Weblocks—but I didn’t check that out yet.  What’s more, UCW, with all its complexity, had very little documentation, and what was avalilable, was mostly outdated, as the framework was constantly evolving at that time.  Not so cool, but it forced me to learn to RTFS when needed (with Slime’s M-. it was not as hard as it seems).  But I digress; UCW made use of some other projects of Bese (UCW developers seemed to have quite a bad case of NIH), including HTML output and templating library Yaclml—Yet Another Common Lisp Markup Language—which I really loved, and which, in my opinion, surpasses CL-WHO + HTML-Template in many ways.  Unfortunately it is almost undocumented, which is what I’ll try to fix in this article, along with comparing Yaclml to its Ediware counterparts.

Formatting HTML output

Edi, when designing CL-WHO, decided to use keyword symbols to represent HTML tags; Bese guys decided to create separate package for tags—it.bese.yaclml.tags, conveniently nicknamed <.  Let’s compare a simple example code:

;; CL-WHO
(with-html-output-to-string (s)
  (:ul (:li "foo")
       (:li "bar")))

;; Yaclml
(with-yaclml-output-to-string
  (<:ul (<:li "foo")
        (<:li "bar")))

Looks like a trivial difference, only on visual level (but this level is also important: we see the < sign, which is distinct visually and actually looks like an HTML tag, not a keyword that looks just like any other keyword scattered throughout the code), but it actually has many subtle consequences.  First, keywords can’t have a function or macro definition, at least not in a practical way without conflicting with anything.  Thus, with CL-WHO syntax, one has to explicitly pass the sexprs through sexp-to-html translation function or macro; escaping to Lisp code from HTML-describing sexprs, and back to HTML is not trivial and requires a separate macro layer to switch back and forth.  Using separate package to describe HTML tags makes it possible to implement tags as macros, which was what Bese guys did.  As an important side effect, this enables basic HTML validation support, and guards users from their own typos, which CL-WHO is simply unable to do.  Yaclml also avoided the stream variable binding (they just use a dynamically bound special variable to hold output stream), so we don’t see this binding contributing to visual noise.  Now, let’s see what happens with the code when we wish to use loop to generate list items:

;; CL-WHO
(with-html-output-to-string (s)
  (:ul (dolist (i '("foo" "bar" "baz" 23))
         (htm (:li (esc (princ-to-string i)))))))

;; Yaclml
(with-yaclml-output-to-string
  (<:ul (dolist (i '("foo" "bar" "baz" 23))
          (<:li (<:as-html i)))))

We see now that Yaclml actually includes a whole lot less visual noise.  There is an <:AS-HTML macro, which is necessary because tags are implemented as macros, not as functions, which allows for some optimizations, but requires some more thought from user.  Yaclml authors decided to ignore value returned by forms included in tag macros, and require them to write directly to YACLML:*YACLML-STREAM*.  There are helper macros <:AS-HTML (with shorthand <:AH), which quotes its arguments’ evaluation results to Yaclml output stream, and <:AS-IS (<:AI), which outputs evaluation results directly to the stream, allowing called function to generate its own HTML, but also requiring it to escape anything that needs to be escaped.

Using special variable YACLML:*YACLML-STREAM* instead of a lexical binding (which is what CL-WHO does) has one more benefit, when we want to delegate some parts of HTML generation to separate functions.  Let’s compare once again:

;; CL-WHO
(defun item-html/who (val)
  (with-html-output-to-string (s)
    (:li :class "item"
         (esc (princ-to-string val)))))
(with-html-output-to-string (s)
  (:ul (dolist (i '("foo" "bar" "baz" 23))
          (str (item-html/who i)))))

;; Yaclml
(defun <item (val)
  (<:li :class "item"
    (<:ah val)))
(with-yaclml-output-to-string
  (<:ul (dolist (i '("foo" "bar" "baz" 23))
          (<item i)))

The CL-WHO functions include very much noise, which Yaclml manages to avoid.  The visual distinction of tags with the < package alias helps with code readability.  We also use a convention to put formatting-only functions and macros in a separate package, with name starting with <, or to start their names with <.  This convention clearly indicates formatting/visual layer; for simplicity, I used function name starting with <, but in a larger project, I’d use separate package for formatting, nicknamed <project, so I would use <project:item here.

Could Yaclml HTML generation be better?  Sure thing.  It could step away from macros (at a performance penalty, though), and use functions, which would return and consume HTML fragments (or consume strings, or actually any Lisp values—including closures for lazy evaluation).  This would introduce first-class HTML fragment object, giving us sort of DOM and possibility of passing around and manipulating already generated HTML fragments, and wouldn’t requre the <:AH and <:AI helper macros (or actually would require only one of those).  That seems to be what core-server guys are doing with their DOM programming support.  I don’t use it yet, partly because I don’t need this kind of functionality and I’m more acquainted with Yaclml; partly because I don’t want to buy into core-server as a whole—core-server is a monolith and ripping DOM support out of it would require substantial amount of work—and partly because I use the second part of Yaclml, which I will describe in the following article: HTML and XML templating.

A language, to be considered serious, needs to be self-sufficient, a serious language can’t be a mere parasite on some host language or environment, and its bus factor can’t be finite.  That translates to just a couple of features:

• A serious language has to have a defining standard, it can’t be implementation-defined.  A good standard bumps language’s bus factor to aleph null: after a nuclear catastrophe, archæologists of future generations should be able to re-implement the language on any hardware, including the Calculor;
• A serious language has to be self-hosting, or at least have some self-hosting implementations.  Only then language stops being dependent on other languages.

Leaving aside other important traits (such as having multiple implementations, having a machine code compiler, extensibility, and so on), these two alone are necessary and sufficient for language to be serious; all programing languages not having these traits are just toys that can’t be guaranteed to last.

Toy languages may stay this way (as did Tcl and Perl, I think, and TeX is one-of-a-kind toy language which managed to grow enormously, staying implementation-defined and standardless).  On the other hand, starting a toy language may be just a way to bootstrap a serious language without starting from gathering a commitee, and I think that’s what is currently happening to Python.  I hope that will also happen to Clojure; I looked at it, and I liked what I saw (good concurrency support, decoupling polymorphism from data structures even more than in Common Lisp, a tiny bit more modern syntax, and it feels really well designed), however at the moment it is a toy language, feeding off Java, at very early stage of development.

Toy languages have short-term advantages (they are fast to create, usually simple, they leverage the host environment (libraries, operating system interface), they solve problem at hand quickly), and long-term disadvantages (lack of stability).  Serious languages have short-term disadvantages (multitude of slightly differing implementations to choose from, every single one of which has its own differences and extensions to the standard; implementation/library compatibility matrix; standard features are set in stone and it may be tricky to work around them), and long-term advantages (stability, one can rely on a serious language to be there in one form or another in ten years).

I have nothing against toy languages: they do their job, and do it well.  As a glue language, as a scripting language, as a solution to short-term problems, they’re good enough, and short-term advantages win over long-term disadvantages.  Brian was trying to solve a short-term problem and, not surprisingly, toy language worked better.  But: will the solution still work in ten, twenty years?  Heck, will it work with next release of Clojure, or next release of JVM, or if Rich Hickey (Clojure author) gets hit by a bus?  Brian didn’t need to answer these questions; do you?

When you get to choose a programming language, or any other tool, you may have some personal preferences, but don’t forget to ask yourself the single most important question: What is the problem you’re trying to solve?  The answer to this question determines all further questions and answers.

darcs get http://common-lisp.net/project/bese/repos/parse-html/