* Getting Started with UncommonWeb. ** What UCW is. UCW is a multi-paradigm framework for building web-based applications in Common Lisp. It has an extensible core, the RERL, that enables many methods and styles of web development, and can operate using almost any web server as a front end, including a few that come built in. UCW includes a component oriented system allows both the graphical elements and the presentation logic to be easily reused and adapted, and has features that allow developers to write complex page flow logic as if it was a "regular" sequence of function calls. UCW has been used to develop all kinds of web sites and applications just using the standard library (ucw-standard) but also serves as an excellent platform for developing higher-level Common Lisp web frameworks, such as lisp-on-lines, ucw_ajax and core-server. ** What UCW isn't. UCW is not a full-stack, all or nothing approach. UCW does not include form builders, Object/Relational mapping or any database support. UCW deals only with the web side of things, like dealing with HTTP requests, sessions, mashalling/unmarshalling of GET/POST variables and the like. Users are encouraged to explore the higher-level UCW-based frameworks once a solid grounding in UCW concepts has been acheived. ** Supported Platforms SBCL/linux/x86. UCW attempts to be portable, and reports of it working elsewhere are welcome. ** Dependencies :arnesi :swank :iterate :yaclml :local-time :usocket :rfc2109 :net-telent-date :cl-fad :trivial-garbage :bordeaux-threads :rfc2388-binary :puri :cl-ppcre ** Installation Currently, the only way to install UCW is from the darcs archive. Most of its dependencies should be installed in the same way. ** Starting the demo load demo/demo.lisp and run (startup-demo) ** Understanding UCW. UncommonWeb provides a framework that abstracts the details of HTTP. Understanding UCW is the simple matter of understanding these abstractions, and how they come together to form a UCW application. Web developers starting with UCW often have trouble understanding how it works. Its more advanced features, such as callbacks and actions using continuations, seem to work like magic. This guide will attempt to explain UCW from the bottom up. Hopefully this will serve to demistify both the operation and the purpose of these features. We'll start with an explaination of the "Standard" web application features of UCW, such as session management and HTML generation, and leads towards the advanced features, like Component Oriented UI and Linear Page Flow Logic using delimited continuations. The usage examples included below are presented in a "bottom up" style moving progressivley from early un-abstracted illustrations before presenting the more idiomatic abstractions which make UCW so powerful. If some early example code seems long-winded or convoluted don't let that scare you off, it cleans up nicely at the end. ** The RERL UCW is based around the Request Eval Response Loop (RERL). The RERL is an extensible CLOS protocol that takes care of the details, while still allowing the programmer to hook in at any point. Because of this extensibility, UCW is suitable for all types of web applications. The more advanced features of UncommonWeb are especially suited for highly stateful applications with a complex control flow. The developer is free to use an much or as little of the RERL as desired for any given request. One can freely intermix a REST-style with a heavily stateful CPS approach where appropriate. This manual will demonstrate both approaches, but will focus on the unique features of UCW and assume the reader already knows how to make 'standard' web applications. The RERL protocol is fully documented in src/rerl/protocol.lisp. *** Handling Incoming Requests BACKEND->SERVER->APPLICATION->DISPATCHER->HANDLER When a user requests an HTTP URL, that request is handled by an HTTP server. UCW calls this a BACKEND. UCW can be made to work with any backend, including mod_lisp, Lisp HTTP servers, or its own internal HTTPD. We'll be using the internal HTTPD, but this is by no means required. *** Backends and Servers The MAKE-BACKEND method creates and returns an instance of a BACKEND class. This backend will be included as part of a SERVER object. The BACKEND, for each request, is responsible for creating REQUEST and RESPONSE objects, and the three objects are passed to HANDLE-REQUEST. This method sets up the UCW enviroment and passed control to the SERVER object. The SERVER object is the logical 'server', through which all requests pass. In the default implementation, a SERVER matches the request against those expected by applications registered with the SERVER. (in-package :ucw-user) (defvar *example-server* (make-instance 'standard-server :backend (make-backend :httpd :port 8080))) (defun start-example-server () (startup-server *example-server*)) *** Applications and Dispatchers An APPLICATION is just that, a cohesive collection of features. It's main responsibilty in UCW is as a containter for DISPATCHERS and ENTRY-POINTS, and as a place to store the SESSION objects. An APPLICTION will attempt to match the REQUEST against its DISPATCHERS. By default, there are two types of dispatchers that will delegate their requests to the RERL, those created as ENTRY-POINTS and ACTION dispatchers. (defclass example-application (standard-application cookie-session-application-mixin) () (:default-initargs :url-prefix "/example/")) (defvar *example-application* (make-instance 'example-application)) (register-application *example-server* *example-application*) So, when a backend receives a request and passes it to UCW, the server object searches for a matching application (in our case matched via a url-prefix). The application matches the request object against its dispatchers, and if a match is found that dispatcher calls its handler function. ** The EVAL and RESPONSE stages. *** Entry Points (and a "Standard" hello world). Once a request has been properly dispatched to the correct handlers, We enter UCW's EVAL stage. 'External' URLs in UCW are known as ENTRY-POINTs. These are bookmarkable, REST-style linkable resources exposed directly to the web. Lets look at a simple example. (defentry-point "hello.ucw" (:application *example-application* :with-call/cc nil) () (format (html-stream (context.response *context*)) "Hello World")) Here, we define an entry point into our application. The URL used to access this entry point would be: http://localhost:8080/example/hello.ucw If you do a (start-example-server), and point your browser to the right spot, you should be able to see a "Hello World" in your browser. The :WITH-CALL/CC option is set to NIL here. This is not strictly neccesary, but were not using UCW's continuation based features yet, so we can gain a minor performance increase by avoiding the overhead of the CPS interpreter. Also, notice the *CONTEXT* special variable. Within the RERL, *CONTEXT* is always bound to the current REQUEST-CONTEXT. For an overview of what's available via *CONTEXT* see src/rerl/protocol.lisp. The HTML-STREAM of the RESPONSE object contains a stream that will write to the users browser, and here we simply use CL:FORMAT to write our message. *** Entry Points with Arguements (GET/POST parameters part 1) An entry point is very much like the concept of a 'page' or 'file' in other frameworks/languages.. with all the issues that entails. While a developer should avoid putting too much logic and presenation code in an entry point, there are a number of cases where it is quite useful. In particular, if one were doing a REST-like application mixed with stateful components, entry points (or something built in the style of an entry point handler) with arguments are a perfect solution. (defentry-point "hello2.ucw" (:application *example-application* :with-call/cc nil) ((message "World")) (format (html-stream (context.response *context*)) "Hello ~A" message)) In this example, the follwoing URL produdes our standard greeting: "http://localhost:8080/example/hello2.ucw" We can change the greeting with some like: "http://localhost:8080/example/hello2.ucw?message=Universe". As expected the second URL will display "Hello Universe". See the documentation of defentry-point for more details about the REQUEST-LAMBDA-LIST and other arguments. *** YACLML: Yet Another Common Lisp Markup Language. Obviously, nobody wants to generate HTML via CL:FORMAT. It would get ugly fast. UCW does not enforce any particular way of generating HTML, developers are free to use an existing HTML macro or template library. Internally, UCW uses the YACLML macro language to generate HTML, so that is the path of least resistance. We only look at YACLML here, but it is trivial to add support for other methods. Developers who prefer to generate thier HTML from templates will be glad to know that UCW includes support XML templating via TAL (Template Attribute Language), see http://en.wikipedia.org/wiki/Template_Attribute_Language. YACLML tag macros are defined on symbols in the conveniently nicknamed package: "<". A nice consequence of the terse "<" package quilifier is that the symbol names of qualified YACLML tag macros appear in Lisp source closely resembling the HTML they abstract upon. Attributes are supplied as keyword arguments, and must appear directly after the tag name. For example, these YACLML symbols are as their HTML counterpart: <:a :href "hello-session.ucw" <:span :class "message" <:div :class "node"
Some additional benefits of this scheme are: - tags with HTML context are easily distinguised from Lisp code; - guards against trivial typos when context switching from HTML/Lisp; - promotes basic "visual validation" of the abstracted HTML; - when coupled with TAL use of XML/XHTML editing modes is possible; The following should serve to better demonstrate YACML by illustraing a simple use of the FORM tag. (defun render-page-wrapper (title thunk) (<:html (<:head (<:title (<:as-html title))) (<:body :id "body" (funcall thunk)))) (defun render-message (message) (<:span :class "message" (<:as-html "Hello " message "!"))) (defun render-message-form (name) (<:form (<:as-html "Enter a new message:") (<:input :type "text" :name (string-downcase name)) (<:submit))) (defentry-point "hello-yaclml.ucw" (:application *example-application* :with-call/cc nil) ((message nil)) (render-page-wrapper "UCW Example" (lambda () (<:style ".message {font-size:2em;font-weight:bold") (if message (render-message message) (render-message-form 'message))))) By composing functions and entry points in the above manner one could build an entire application. However, we've yet only begun to scratch the surface of what UCW can do for you. *** UCW Components, a UI toolkit for the web. Up to this point we've been well within the realm of standard web development. While building application in this style is possible, it gets messy quickly. The control flow of our application is obscured by the manner in which we must keep state, by passing variables via get or post, and our display code, though nicely factored out into functions, is still all mixed up with our logic. Components solve the first part of the problem, the display or 'view' in MVC terms. Components are pieces of your interface, similar to 'widgets' or 'gadgets' in desktop GUI toolkits. One composes any number of components inside a top-level WINDOW-COMPONENT, and this component tree is then rendered to HTML and presented to the user. UCW components are instances of CLOS classes with the STANDARD-COMPONENT-CLASS metaclass. (defclass example-message () ((message :accessor message :initarg :message :initform "World!")) (:metaclass standard-component-class)) (defmethod render :before ((self example-message)) (<:style ".message {font-size:2em;font-weight:bold")) (defmethod render ((self example-message)) (render-message (message self))) (defclass example-form () () (:metaclass standard-component-class)) (defmethod render ((self example-form)) (render-message-form 'message)) We've created a couple of simple components here. In this very basic example, using components over functions buys us very little.. it's more verbose and there's more overhead. However, components play an important role when using UCW's advanced control flow features, and it is useful to abstract the RENDERing to a single method. WINDOW-COMPONENTS are the top-level wrappers. Generally, they print the HTML header and then delegate to another component for the rest. When nesting components, the slot that contains the component should be marked ":component t". For the simple example that follows it doesn't matter, but we will be re-using the window component in later more involved examples where it will. (defclass example-window (window-component) ((title :accessor window.title :initform "UCW Example" :initarg :title) (body :accessor window.body :component t :initarg :body)) (:metaclass standard-component-class)) (defmethod render :around ((window example-window)) (render-page-wrapper (window.title window) (lambda () (call-next-method)))) (defmethod render ((window example-window)) (render (window.body window))) This should be pretty self-explanitory, but do take note of the body slot with the :COMPONENT initarg. This next utility function will help us clean up our entry point, and also serves to show how one might 'manually' render a component. In general, UCW calls the RENDER method as part of the RERL. However, in this case we never pass control to a component (we havn't made it to control flow yet), so something like this is neccessary. (defun render-example-window (body-component-name &rest initargs) (render (make-instance 'example-window :body (apply #'make-instance body-component-name initargs)))) There are situations, usually when mixing REST with UCW control flow, where this approach may be warranted, and it does serve as a useful example. Finally, an entry point to bring it all together. (defentry-point "hello-components.ucw" (:application *example-application*) ((message nil)) (if message (render-example-window 'example-message :message message) (render-example-window 'example-form))) It must be said that components can do a lot more than encapsulate the display code. Components can call other components, which can answer with real Lisp values. Components are also convenient places to store dataa related to the application that one might traditionally keep in a 'session' variable. ** Keeping and Frobbing State : from sessions to actions. Up until this very point, we've basically been doing CGI in Lisp. Form variables were used to keep state, in proper REST fashion, and as such the actual control flow is hidden in an adhoc state machine. Wouldn't it be nice to simply be able to say something like: (unless message (setf message (get-message-from-user))) (display-message message) ... and have the computer figure out the details? Would it not be a relief to have links and buttons call functions, and to have form inputs become simple accessors? UCW does all this and more. UCW builds on simple sessions to provide a stateful HTTP. *** Sessions. When a user hits an entry point, UCW automatically associates that user with a session id, and that session id with a session object. Just like in other frameworks, you can either store the session id in a cookie, (we've included in our EXAMPLE-APPLICATION above the COOKIE-SESSION-APPLICATION-MIXIN for this reason), or you can append it to any URLs you render within that session. If the session is not stored in a cookie, UCW expects to see it passed via GET or POST as a parameter named, appropriately enough, +SESSION-PARAMETER-NAME+. This behavior can be changed at the dispatcher level. Although the session object does a lot more than simply store values, that is a valid and common use of sessions. An accessor, GET-SESSION-VALUE, allows you to use sessions as a hash-table. I like to use a symbol macro for 'global' values, such as those stored in sessions, or toplevel components etc. (define-symbol-macro $message (get-session-value :message)) Since we are now storing state on the server as part of the session object, a way to 'reset' the value is a good idea. We'll include the logic as part of our entry point. (defun render-reset-link () (<:div (<:a :href "hello-session.ucw?reset" (<:as-html "Reset Message")))) Now we'll need to modify our message component to pull the message from the session. and display the reset link. We can reuse the same form as well. (defclass example-session-message (example-message)() (:metaclass standard-component-class)) (defmethod message ((self example-session-message)) $message) (defmethod render :after ((self example-session-message)) (render-reset-link)) (defclass example-session-form ()() (:metaclass standard-component-class)) (defmethod render :after ((self example-session-form)) (render-reset-link)) And again, an entry point to bring it all together. We still have way too much logic in our entry points.. we're basically still making web pages here. The next section is the beginning of the end of this, I promise. (defentry-point "hello-session.ucw" (:application *example-application* :with-call/cc nil) ((message nil) (reset nil reset-requested-p)) (when message (setf $message message)) (when reset-requested-p (setf $message nil)) (if $message (render-example-window 'example-session-message) (render-example-window 'example-session-form))) *** Frames, Actions, Callbacks. A session is a really coarse way to keep track of application state. It doesn't deal well with multiple windows, the back button, and a host of other things that web users do to innocent session-based applications. UCW adds another object to the equation.. a FRAME. A frame represents an indivual hit, a page rendered.. an interaction with the application. When a session is created, an initial frame is created. Other hits to the same session might create other frames.. so a frame is like a point in time. Usually, frames are created as required to maintain state. FRAMES are used to store two important types of data, ACTIONS and CALLBACKS. Actions are essentially functions.. they run to side-effect the components that will be displayed. CALLBACKS, are setters. Usually called before an ACTION is run, callbacks take values from GET and POST variables and call a function on them. Most often this function uses the value to set a slot or a variable. A good example of an action is our reset functionality. Having to pass a variable to an entry point is not at the level of abstraction we'd like to be working in.. I want a reset function to run when a user clicks a reset link, and damn the details. Since we are still not using UCW's control flow features, we have to do a little setup first. Without control flow, we are still stuck in the age of GOTO, and we have to specify which entry-point to hit in order to run our action and render our components. There is a method, COMPUTE-URL, that returns a URL with all the details needed to perform an action. We'll simply create a mixin that adds the name of our entry point to the URL. (defclass entry-point-action-mixin () ((entry-point :accessor entry-point :initarg :entry-point :initform "hello-action.ucw"))) (defmethod compute-url :around (action (component entry-point-action-mixin)) (let ((url (call-next-method))) (setf (uri.path url) (entry-point component)) url)) One might use this technique to create 'bookmarkable' or permanent URLs within an application that uses the control flow features. Also, a FRAME can contain a top level WINDOW-COMPONENT. If it exists, the RENDER method will be called on it. We can use this to create the primitive control flow operator mentioned earler. (defun go-to (body-component-name &rest initargs) (setf (frame.window-component (context.current-frame *context*)) (make-instance 'example-window :body (apply #'make-instance body-component-name initargs)))) **** Actions Next, we'll use the compute-url method, along with REGISTER-ACTION, to create a function that is attached to a URL. When that URL is reqested, the function will be called, within the context of the requests and components we've created. (defclass action-reset-link-mixin () ()) (defun reset-message () (setf $message nil) (go-to 'example-form)) (defmethod render-reset-link ((self action-reset-link-mixin)) (let* ((action (register-action (:with-call/cc nil) (reset-message))) (url (compute-url action self))) (<:a :href (print-uri-to-string url) "Reset Message"))) In RENDER-RESET-LINK, we register an action (a closure on a message) in the current frame. The action is given a unique id that is used by COMPUTE-URL, and with the changes we made to it produces the correct magic URL. Again, note that we specify WITH-CALL/CC is not needed here. If you think that's a little long-winded and belongs in a macro, you're right. The YACLML tag