Nick Mudge Ignition Software Consulting & Development

I thought I’d shoot off an idea about programming webpages.

To me writing a script to generate HTML has always been a problem. It’s a problem because you have to mix a bunch of code with HTML. No matter how you mix more than a little bit of code and HTML the result is inelegance, ugly code, hard to read code. It’s not very good because you’re mixing two different syntaxes together in a jumbled fashion. Code that is just code without HTML in it so much nicer and easy to follow and read. And gosh, I’m always perturbed about how to indent code and HTML together so that structure and things can be made clearer. The elegance, clarity and structure provided by indentation is reduced a lot by the combination of programming logic and HTML intermingled, at least from my experience.

There’s been various ways to help make this better. For instance in PHP you can move in and out of HTML and PHP modes which makes the transitions nicer sometimes. The Smarty templating system is largely a system to make this easier. The Smarty language syntax and tools integrate nicer with HTML and help keep programming away from HTML. This also has a nice benefit of being nice to designers that need to work with HTML and CSS but don’t want to hassle with programming.

Smarty is a nice improvement and the best solution I’ve used so far but the problem is still there. Sometimes you want to add quite a bit of logic to your Smarty templates, adding loops, and inner loops, adding lots of if conditions, making function calls etc. You might want to do this because the display of the webpage you are making is complex and requires lots of if conditions and loops or because you want to reuse the same templates for different webpages (this cuts down the amount of work you need to do and manage – reuse is good) and so need to add additional logic that runs depending on which page the template is being used to generate. Too much logic in Smarty templates quickly gets inelegant and hard.

Things get inelegant and harder to follow when you start adding a bunch of logic to HTML pages. It’s like HTML wasn’t meant to have logic in it.

A related problem that is also a big problem with HTML is that it does not scale. The bigger an HTML file gets the harder it is to follow and work with. With a zillion div tags and table tags it gets harder and harder to know what exactly is happening anywhere in a document. It just doesn’t scale. This combined with logic makes for general ickyness.

I wonder if there is another way.

I’ve looked into Lisp a little in the past and what I’ve seen is that in Common Lisp the combination of programming and HTML are handled in a completely different way. I wonder if something here could be a real solution to this inelegance pest, and actually really bring about an elegance and smoothness in the combination of programming logic and HTML.

From what I've seen in Common Lisp when you program webpages you don’t really mix HTML and programming. Instead you make it all programming. I’m thinking it might be something along these lines: You have a library of functions and/or macors. Each function or macro or arguement corresponds to an HTML tag. For instance you have the div function that is used to generate the HTML for a div tag etc. You just use all these functions that generate all the HTML for you. And it’s easy and natural to add logic because it’s all consistent programming code. You keep the consistency, elegance and scalability of a programming language by only programming in the programming language.

The main problem I see with this approach of using functions to generate HTML is that HTML is highly flexible -- some tags can have certain attributes and can be nestled within some tags but not others and there’s a ton of variance and things you can do in HTML. In lots of programming languages I think functions are probably not flexible enough to cope with all the possibilities. I think these functions would limit what you could do with HTML and cut your flexibility of how you want the HTML to be. It really might not be very workable. I certainly don’t like to code with constraints on me.

But maybe it’s possible with something like Lisp. You’d need finely grained functions/macros that had the same flexibility and limitations as the HTML tags. And I think that’s something Lisp might be able to provide. In Lisp you have finely grained control on what your functions are and how they work – more than I’ve seen anywhere. In addition Lisp has powerful macro abilities that enable you to mold the syntax and semantics of the language to your will. This fine grainess of control on the syntax and semantics of the language might enable you to write a library of HTML functions that you could use to write pure elegant code that generates all the HTML you need and with as much flexibility as you need. Perhaps this already exists.

I think something like this would make it much easier and faster to work with webpages because pure well written code is much easier to write and read than a mess of HTML and code together. With code you have one consistent syntax and it scales better than HTML does. In addition if your functions are perfect (have no bugs in their implementation), then any HTML you generate with them should automatically be perfect too – that’s real scalability power when talking about HTML correctness.

Of course this pure programming way of handling HTML might not be so good for designers. So I was thinking that there could be some kind of separate interface for designers – like CSS is a separate interface into the display of HTML and a webpage. One idea is that a designer could design stuff in pure HTML and CSS and then feed it into a program that analyses it and if it is correct HTML and CSS generates or modifies the programming code to generate that HTML and CSS and if it is wrong spits the HTML and CSS back to the designer to fix.


From Chris Minnick:

Good thoughts, Nick.

I see what you’re getting at, but I’m skeptical of solutions to make code or HTML more 'pure'. It seems like no matter what you do, someone is going to start slipping some HTML in where it doesn’t belong, or is going to put something that should be in the CSS into the HTML, or is going to use tables for layout, or what-have-you. Usually, people resist doing things the 'right' way because the right way is more difficult or doesn't work right.

CSS positioning is a perfect example: purists insist that all Web page layout should be done with divs and CSS, and that tables should only be used for tabular data. This makes perfect sense, of course, but the reality is that you just can’t do some of the things you can do with tables using divs, or it’s very difficult—especially considering browser differences.

So, what usually ends up happening as a result of this is that the people who were insisting on purity start backing off … so, now RSS 2.0 is popular, while the strict RDF version (RSS 1.1) was a big flop, and HTML 5 is going to be the successor to XHTML because people continued to use the much more flexible HTML 4.01 years and years after the stricter XHTML 1.0 came out.

My question is: would the benefits of writing HTML using functions outweigh the added complexity and inconvenience to people who design webpages?


From Nick Mudge:

Hey Chris,

I totally agree and excellent point. Your email made me realize that ideally the functions should be flexible enough to allow people to be as non-standard and/or hacky as they want. I think people should have the freedom to do things how they want. I think the functions should prevent obvious errors like a <div> without a </div> and things that will obviously break webpages in web browsers. It would be nice if it could also help people be as standard as they want by letting them set some global setting or something so that functions enforce more strictness to a standard or whatever they want.

I think the goal of a library like this would be to enable programmers to combine programming and HTML in way that is syntactically consistent, easier to read, write and maintain without loosing the flexibility that they want. Or to achieve a scalable elegance with the combination of logic and HTML. Perhaps a secondary goal would be to help people create HTML as correct or as standard/unstandard as they want.

Any system around the functions/macro library such as a program that translates HTML into code that generates the same HTML should be as flexible to the user as the functions.

I think a designer who does HTML and CSS work would use some program that does an HTML to code translation and the designer should have a way to specify how loose the translator is in accepting HTML and CSS. And I think programmers would just work in the code.

Comments

David Overton
15 April 2008

Hi Nick, you might be interested in Haskell's Text.Html library module. Documentation is at http://www.haskell.org/ghc/docs/latest/html/libraries/html/Text-Html.html

An example of how to use it is at http://search.cpan.org/src/AUTRIJUS/Language-Haskell-0.01/
hugs98-Nov2003/fptools/hslibs/text/html/doc/doc.htm

I came to your blog from Planet Haskell, so maybe you've seen this already.

David
wren ng thornton
16 April 2008

At the risk of heresy, you should take a look at the CGI.pm module for Perl because it does exactly this. Some of the XML modules also have a similar interface without all the HTTP crud.

The big downside to this approach is that the functions really do have to be arbitrarily customizable which quickly heads in the direction of type unsafety. And frankly there's no way around that because the SGML specification of HTML really is unbounded (and the XML specification of XHTML is only somewhat better). Though embedding the html rather than using a bunch of print statements is very nice.

What I'd like to see is is not a reimplementation of CGI.pm in Haskell, but rather some new approach to embedding xml/html/http... one based on combinators, folds, and all the other things that make functional programming fun.
sclv
fmapfixreturn.wordpress.com/
16 April 2008

The other approach is to limit the power of a templating language to prevent too much logic from spilling in. That's how StringTemplate (http://www.stringtemplate.org/) works. (disclosure: I'm the author of the Haskell StringTemplate port.)

Along with this, the XHTML and CSS should be seen as two separate languages, and finally, on the "HTML scaling well" problem, a careful module system of template includes should help.

A system to translate between programmatic and "real" HTML would be nice, but I doubt it could handle all the cruft that seeps in, and ideally HTML/CSS should be done in a visual editor anyway, or at least in a system that allows on-the-fly editing with direct display of results and without recompilation.
Jan Van lent
16 April 2008

Two interesting Common Lisp packages by Edi Weitz.

http://weitz.de/cl-who/
http://weitz.de/html-template/

A system to go between XML and Scheme S-expressions by Oleg Kiselyov.

http://okmij.org/ftp/Scheme/xml.html

See also

http://celtic.benderweb.net/webit/
http://ssax.sourceforge.net/
George Frick
iconnect.aurora.org
16 April 2008

Honestly I've had fantastic success using J2EE with struts-tags (and other tag libraries). I know I'll get flamed for saying it; but when you do the tags correctly you get a very powerful way to put minimal logic into a jsp. CSS + JSP + Tags is it for me.
Vezquex
vortices.appjet.net/
16 April 2008

AppJet has a JavaScript function for every tag.

print(P({id: 'firstparagraph'},'some text'), P('some more text'));
James Justin Harrell
jamesjustinharrell.com/
16 April 2008

A blog post about mixing code with HTML, and nobody mentions E4X? Absurd! E4X is the ultimate mix of XML (such as XHTML) and code. I don't know whether it's wonderful or hideous, but it's sure fantastical.

http://en.wikipedia.org/wiki/E4X
Kum Sackey
17 April 2008

Seaside, a smalltalk framework, uses an HTML canvas class to generate HTML from code.

It's capable of completely expressing any HTML, including embedding tags within tags.

http://seaside.st/documentation/generating-html
Josef Svenningsson
17 April 2008

I suggest you look at HSP, Haskell Server Pages. It allows for an extremely flexible approach to mixing code and HTML (or XML). First of all it includes a library for writing HTML programmatically just as with the Lisp approach that you were after. But it also supports HTML syntax to be mixed with code in the style of PHP. So it basically gives you the whole spectrum. If you're a designer and don't do much coding you use the HTML syntax but if you're a coder and have a lot of logic you may want to refrain from the HTML syntax except maybe in some key places. I think it's pretty neat.
http://www.cs.chalmers.se/~d00nibro/hsp/
This page is unfortunately not updated, there was a major release just last month.
mrd
17 April 2008

As someone who's written a lot of CL w/HTML macro packages, I must say that it is definitely a lot more convenient than HTML itself. However, it is not suitable for web applications because it mixes code and HTML. Now this may be fine when you are the one writing the HTML (as I was) but it is not fine when dealing with the a real web designer.

And by web designer, I mean someone who was too stupid to be a programmer. Don't expect them to be able to handle anything different or new, much less S-expressions (even though they are simpler than XML expressions). I know this sounds mean, but after dealing with random schmucks to try and get extra help, I know it's sadly all too true.

To separate code from HTML, and to use standards for which web designers presumably train: use XSLT.
nick kallen
18 April 2008

For Ruby, Markaby and Erector are builder batterns for this:

ul do
photos.each do |p|
li { img :src => p.src }
end
end
Name: (required)
Email: (required)
Website:
What has four legs, rhymes with bat and says, "Meow?" (One word answer.)
Spam Filter: