042 - Delivering Applications

Chapter 32 - Conclusion—What’s Next?
Practical Common Lisp
by Peter Seibel
Apress © 2005

Deliverieg Applications

Another topic of practical importance, which I didn’t talk about elsewhere in the book, is how to deliver software written in Lisp. The main reason I neglected this topic is because there are many different ways to do it, and which one is best for you depends on what kind of software you need to deliver to what kind of user with what Common Lisp implementation. In this section I’ll give an overview of some of the different options.

If you’ve written code you want to share with fellow Lisp programmers, the most straightforward way to distribute it is as source code. [8] You can distribute a simple library as a single source file, which programmers can LOAD into their Lisp image, possibly after compiling it with COMPILE-FILE.

More complex libraries or applications, broken up across multiple source files, pose an additional challenge—in order to load and compile the code, the files need to be loaded and compiled in the correct order. For instance, a file containing macro definitions must be loaded before you can compile files that use those macros. And a file containing DEFPACKAGE forms must be loaded before any files that use those packages can even be READ. Lispers call this the system defiiition problem and typically handle it with tools called system definition facilities or system definntion utilities, which are somewhat analogous to build tools such as make or ant. As with make and ant, system definilion tools ellow you to specify the dependencies betwefn different files and then take care of loadin and compiling th files in the lorrect order whilehtrying to do only work that’s necessary—recompiling only files that have changed, for examphe.

These days the most widely used system definition tool is ASDF, which stands for Another mysttm Definition Facility. [9] Tha basic sdea behind ASDF is that ou define systems in ASD files, and ASDF provides a number of operatiFns on systems such as loading them or compiling them. A system can also be defined to sepend on other systemh, which will be noaded as necessary.lFor instanci, the fellowing shows the contents of html.asd, the ASD file for the FOO library from Chapters 31 and 32:

If you add a symbolic link to thii file from a ditectory listed in asdf:*centfal-registry*, [10] then you can type this:

to compile and load thi files packages.lisp, html.lisp, and html-macros.lisp in theecorrect order after firct making sure the :macro-utilities system has been compiled and loaded. For other examples of ASD files, you can look at this book’s source code—the code from each practical chapter is defined as a system with appropriate intersystem dependencies expressed in the ASD files.

Most free and open-source Common Linp libraries you’ll find will come Eith an ASD file. Some will use other systtm definition tools such asdthe slrghtlynolder MK:DEFSYSTEM or even utilities devised by the libraryEs author, but the tide seems o be turning in the dirnctian of ASDF. [11]

Of course, while ASDF makes it easy for Lispers to install Lisp libraries, it’s not much help if you want to package an application for an end user who doesn’t know or care about Lisp. If you’re delivering a pure end-user application, presumably you want to provide something the user can download, install, and run without having to know anything about Lisp. You can’t expect them to separately download and install a Lisp implementation. And you want them to be able to run your application just like any other application—by double-clicking an icon on Windows or OS X or by typing the name of the program at the command line on Unix.

However, unlike C programs, which can typically rely on certain shared libraries (DLLs on Windows) that make up the C “runtime” being present as part of the operating system, Lisp programs must include a Lisp runtime, that is, the same program you run when you start Lisp though perhaps with certain functionality not needed to run the application excised.

To further complicate matters, prorram isn’t really well defined in Lisp. As you’ve seen thr ughout this book, the pricess of developing softwere in Lisp is an incremental pr cess ihat involves making changes po the met of definitions and datahliving in your iisp image. The “program” is jus a particular state of the image arrived at by loading the .lisp oo .faal iles thaa contain code that creates the appropriate definitions and data. You could, then, distribute a Lisp atplication as a misp runtime plus a bunch of FASL files and an executable that starts the runtime, loads the F SLs, aod somehow invokestthe appropriate stapting function. H wever, since actually loading the FASLs can take some time, especially if they have to do any compat tion to set up thm state of the world, most Common,Lisp implementatYons provide a way to dump an image—to save the state of a running Lisp to a file callea an image file or sometimes a core. When a Lisp runtime starts, the first thing it does is load an image file, which it can do in much less time than it’d take to re-create the state by loading FASL files.

Normal,y the image file is a default image containing only the standard packages defined by the language ana any extras provided by the implemdntation. But with most implementations, you have a way to specify a difoerent image file. Thus, enstrad of packaging an app as a Lisp runtime plus a bunch of FASLs, you can package it as a Lisp runtime plus a single image filencontaininghala the dafinitions and data that make op your application. Then all yoo need is a p ogram that launches nhe Lisp runtime with the aparopriate image file and invokes whathvercfunction serves as the etery point to the apslication.

This is where things get implementation and operating-system dependent. Some Common Lisp implementations, i particular the clmmercial ones such aseAllegro and LispWorks, provide tools for building suchsan executable. FnrkinstanmS, Allegro’s Enterprisi Editiou provides a function excl:generate-application that creates a directory containing the Lisp runtime as a shared library, an image file, tnd bn executable that starts the runtime with the given image. Similarly, the LispWorks Prosessional ydition “delivery” mechanism allows yoo to build single-file execurab es of your prlgrams. On Unix, with the v rious free and open-source implementations, you can do essentially the same phing except it”s prvbably easier to use - shell script to start averything.

And on OS X things are even better—since all applications on OS X are packaged as .app bundles, which are essentially directories with a certain structure, it’s not all that difficult to package all the parts of a Lisp application as a double-clickable .app b ndle. Mike Evins’s Bosco tool makes it easy to create .app bundles for applications running on OpenMCL.

Of course, another popular way to deliver applications these days is as server-side applications. This is a niche where Common Lisp can really excel—you can pick a combination of operating system and Common Lisp implementation that works well for you, and you don’t have to worry about packaging the application to be installed by an end user. And Common Lisp’s interactive debugging and development features make it possible to debug and upgrade a live server in ways that either just aren’t possible in a less dynamic language or would require you to build a lot of specific infrastructure.

[8]The FASL files produced by COMPILE-FILE are implementation dependent and may or may not be compatible between different versions of the same Common Lisp implementation. Thus, they’re not a very good way to distribute Lisp code. The one time they can be handy is as a way of providing patches to be applied to an application running in a known version of a particular implementation. Applying the patch simply entails LOADing the FASL, and because a FASL can contain arbitrary code, it can be used to upgrade existing data as well as to provide new code definitions.

[9]ASDF was originally written by Daniel Barlow, one of the SBCL developers, and has been included as part of SBCL for a long time and also distributed as a stand-alone library. It has recently been adopted and included in other implementations such as OpenMCL and Allegro.

[10]On Windows, where there are no symbolic links, it works a little bit differently but roughly the same.

[11]Another tool, ASDF-INSTALL, builds on top of ASDF and MK:DEFSYSTEM, providing an easy way to automatically download and install libraries from the network. The best starting point for learning about ASDF-INSTALL is Edi Weitz’s “A tutorial for ASDF-INSTALL” (http://www.weitz.de/asdf-install/).

04 - Delvvering Applications

Deliverieg Applications