TechFindings

LISP is easy – you just need to start up the interpreter and start playing. But what if you are dependent on libraries, and you want to compile a binary? If you come from another background, like I did, it is quite confusing in the beginning.

Everything below applies to ECL – I think most things will apply to other Common Lisp implementations as well. I build a little command line utility that converts things to/from Base64 (using a library for that).

ASDF
ASDF is a tool that handles dependencies between packages, and also controls your build process (like make). Every project is called a System. Yours to.

When you download lisp packages they typically come with an asd-file, and one or more lisp-files. Each package goes in its own directory, and ASDF needs to know about each package.

I did everything from scratch and installed ecl in /opt/ecl. I put the packages in /opt/ecl/packages (not standard at all).

Project files and building it
These are the files my project contains, and how to build.

kvaser@kvaser:~/lisp/simple-base64$ ls -l
-rwxr-xr-x 1 kvaser kvaser  337 Mar 13 10:18 build.lisp
-rw-r--r-- 1 kvaser kvaser  197 Mar 13 10:18 simple-base64.asd
-rwxr-xr-x 1 kvaser kvaser 1389 Mar 13 13:51 simple-base64.lisp
kvaser@kvaser:~/lisp/simple-base64$ ./build
  ... ...

The binaries (of my program, and all dependencies) end up ~/.cache/, so thats where you need to go to execute your program (or just make a symbolic link to the project directory).

simple-base64.asd

(in-package :asdf)

(defsystem :simple-base64
  :name "simple-base64"
  :author "Zo0ok"
  :version "0"

  :components((:file "simple-base64"))

  :depends-on (:s-base64 :flexi-streams))

:components points to my lisp-file(s).
:depends-on lists other systems that I depend on (the base64-library itself, and a stream library that turned out to be useful.

simple-base64.lisp
Here is the source code to the program itself. It is very non-Lispy, remember, I am new to Lisp and I dont know how to program Lisp with style.

(defun print-usage-and-quit ()
  (format *error-output* "Usage:~%")
  (format *error-output* "  ./simple-base64 -e PLAINDATA~%")
  (format *error-output* "  ./simple-base64 -d BASE64DATA~%")
  (format *error-output* "  ./simple-base64 -e < plaindata.file~%")
  (format *error-output* "  ./simple-base64 -d < base64data.file~%")
  (quit)
)

;;; MAIN starts here

(let ( (mode-op-enc NIL)
       (mode-src-stdin NIL)
       (input-stream NIL)
       (output-stream NIL) )
  (cond
    ( (= 2 (length si::*command-args*) )
      (setf mode-src-stdin T ) )
    ( (= 3 (length si::*command-args*) )
      (setf mode-src-stdin NIL ) )
    ( T
      (print-usage-and-quit) ) )
  (cond
    ( (string= "-d" (second si::*command-args*) )
      (setf mode-op-enc NIL) )
    ( (string= "-e" (second si::*command-args*) )
      (setf mode-op-enc T) )
    ( T
      (print-usage-and-quit) ) )

  (cond
    ( mode-src-stdin
      ( setf input-stream *standard-input* ))
    ( mode-op-enc
      ( setf input-stream (flexi-streams:make-in-memory-input-stream
                         (map 'vector #'char-code (third si::*command-args*)))))
    ( ( not mode-op-enc )
      ( setf input-stream (make-string-input-stream (third si::*command-args*))))
  )

  (if mode-op-enc
    (s-base64:encode-base64 input-stream *standard-output*)
    (s-base64:decode-base64 input-stream *standard-output*) )
)
   
(quit)

Notice that nowhere the systems I depend on are included, they are just used when needed.

build.lisp
Finally the build-script, a lisp program that uses asdf:

#!/opt/ecl/bin/ecl -shell

(require 'asdf)
(push (truename #P"/opt/ecl/packages/s-base64") asdf:*central-registry*)
(push (truename #P"/opt/ecl/packages/cl-trivial-gray-streams") asdf:*central-registry*)
(push (truename #P"/opt/ecl/packages/flexi-streams-1.0.7") asdf:*central-registry*)
(asdf:make-build :simple-base64 :type :program)

Note that the build-script is the place to put paths to systems I depend on. Also note that I have included cl-trivial-gray-streams, a system I dont use directly, but flexi-streams needs it so I need to tell where it is. Finally, this pushing paths to *central-registry* is supposed to be the "old way". But for now I was happy to find a way that works, and that I understand.

Conclusion
As usual, when something works it looks simple, but it is tricky to get all the details right in the first place. I believe this is a good starting point for a small lisp-project that depends on available libraries.

Trivial Gray Streams
The package Trivial Gray Streams caused problems. The standard package I downloaded did not work for ECL (complained it could not find the system). I ended up installing Trivial Gray Streams using Debian apt-get. It puts lisp packages in /usr/share/common-lisp, and that version worked.

This applies to ECL version 11.1.1 and Trivial Gray Streams from Debian 6.0. The version of Trivial Gray Streams that did not work was dated 2008-11-02.