\section{Extending CLISP}

Since CLISP is provided with source, you can add features you want yourself.

Additions in Lisp are accomplished by writing a Lisp module and adding
a {\tt load} form for it at the end of {\tt init.lsp}. Please conditionalize
with \verb-#+- and \verb+#-+ if you  use features that are not present
in all platforms supported by CLISP.

Additions in C or assembly language are also possible, as well as interfacing
code written in other languages.

As general rules for extensions to CLISP:

\begin{description}
\item[\rm 1] Add what you want in Lisp. This is both more portable and easier to
   maintain.

\item[\rm 2] If that is not possible, you may choose to write it in C.

  If it makes up a separate module, then write it as a stand-alone program,
   with the usual {\tt argv}/{\tt argc} interface, and call it from Lisp using
   \verb+(SHELL "your-program argument ...")+ or\\
   \verb+(EXECUTE "your-program" "argument" ...)+

\item[\rm Otherwise] you'll indeed have to extend CLISP.
\end{description}

For  now, CLISP has no  foreign  function call facility with automatic
type conversion   between Lisp and C  types  and dynamic loading  of C
object modules.  Thus you will create your own version of CLISP, which
will have {\tt .mem} files  incompatible with what I distribute. ({\tt
.fas} files remain valid, you needn't recompile everything.)

Split your project into three parts:
\begin{description}
\item[\rm A.] The high-level things that may be implemented in Lisp. Do them in Lisp.
\item[\rm B.] C code that is already present or doesn't need access to Lisp data
   structures.
\item[\rm C.] C code that makes up the interface between CLISP and part B. You may use
   CLISP's dynamic memory management here for the purposes of part~B.
\end{description}

\subsection{Part A code}

Make the names of the {\tt .lsp} files known to your makefile. This is
done by modifying  the  lines  \verb+CODE=...+ in  {\tt  makemake.in}.
Make them also contained in {\tt lispinit.mem}. This is done by adding
{\tt load} forms at the end of {\tt init.lsp}.

\subsection{Part B and C code}

Code in   modules that begin  with a  \verb+#include "lispbibl.c"+ are
considered internal to CLISP, code  in  other modules or libraries  is
considered  external.  {\it    Important\/}: Calls  from internal   to
external code must be protected by  a pair of \verb+begin_call();+ and
\verb+end_call();+  statements. Calls from  external to  internal code
must either be avoided altogether, or  the called code must begin with
\verb+begin_callback();+ and end with \verb+end_callback();+.

Part C code is usually (although not absolutely necessary) written in a
modified or enhanced C, which has been created to overcome the limitations
of some C compilers. Such code is contained in {\tt .d} files, which are
preprocessed to {\tt .c} files. The modifications against C are the following:
\begin{itemize}
\item \verb+#+ and space are introduce comments up to the end of the line.
\item Preprocessor directives may be preceded by spaces, so that they nest
  properly with the rest of the program.
\item Function declarations and definitions have the {\it storage class} {\tt global}
  or {\tt local}. {\tt global} implies global visibility, {\tt local} corresponds to
  {\tt static}.
\item Function declarations are written with full parameter types, as in ANSI C.
\item  Function definitions are written as in traditional C, with only the
  parameter names listed within the parentheses, and with full parameter
  declarations afterwards, one declaration for each parameter.
\item \verb+#if defined(...)+, \verb+#elif+, \verb+#error+ may be used as in ANSI C.
\item Adjacent strings are merged after preprocessing, as in ANSI C.
\item Variable declarations within functions are usually introduced by {\tt var}
  (for readability), They may receive the storage class {\tt reg1}\ldots {\tt reg10}
  which stand for {\tt register} or nothing, depending on the CPU's number of
  registers. {\tt reg1} is to be used for variables of highest priority, {\tt reg10}
  for rarely used variables which may as well be {\tt auto}.
\end{itemize}

This allows you to use the same source for both traditional (= deficient) and
ANSI C compilers.

Part C code may use the macros and functions listed in {\tt lispbibl.d}.

Make the names  of the {\tt .d} files  known to your makefile. This is
done by modifying  the lines \verb+MODULES=...+ in {\tt  makemake.in}.
The other  modules ({\tt  .o}  object files  and {\tt  .a}  libraries)
should be added to the lines \verb+LIBS=...+ in {\tt makemake.in}.

When building an interface to CLISP, you must of course work with Lisp objects,
at least with the arguments and the value to be returned.
There are some rules that govern working with Lisp objects. The C type of a
Lisp object is always {\tt object}.
\begin{enumerate}
 \item  C variables may hold  Lisp objects only as  long as no garbage
   collection may be invoked. Those routines that may invoke a garbage
   collection  are marked   in  {\tt lispbibl.d}   as  {\tt   kann  GC
   ausl\"osen}; they include \verb+allocate_cons()+, {\tt funcall()} and
   {\tt eval()} and  even {\tt equalp()}. Only  low level routines and
   macros such  as type  tests, {\tt eq()} and  accessing  the slots of Lisp
   data types can't invoke a garbage collection. 

 \item The regular place for Lisp objects is on a special stack called
   the {\tt STACK}.  The top element is \verb+STACK_0+, the next-to-top
   element is \verb+STACK_1+ etc. There are macros {\tt pushSTACK()} to
   push an  object  onto the  stack  and popSTACK() to remove  the top
   element.

 \item  Arguments to   functions  are passed on    the stack.  In  the
   simplest case, a function with $n$  required arguments receives the
   first argument in \\\verb+STACK_(+$n-1${\tt)} and the last argument in 
   \verb+STACK_0+. When the function has  done its work,  it must remove its
   arguments  from the {\tt  STACK} ---$n$ times \verb+popSTACK()+ or,
   equivalently, {\tt skipSTACK($n$)}--- and  put its return value  in
   the variable {\tt value1},  as well as  the number of return values
   (1 in most  cases) in the  variable \verb+mv_count+. (Note that {\tt
   value1}   is not GC-safe in   the sense of   paragraph  1!) You may
   enforce  a check  that the {\tt  STACK}  has been set correctly  by
   defining {\tt STACKCHECKS} and  {\tt STACKCHECKC} as {\tt  TRUE} in
   {\tt lispbibl.d}.

 \item Intermediate values of type {\tt object} should be saved on the
   {\tt STACK}  when needed.  Of  course, optimizations  are possible:
   assuming   that {\tt foo()}, {\tt  foo1()}  and {\tt foo2()} return
   objects and  may invoke GC, you  may  well write:
\begin{verbatim}
var object temp = foo1(foo(x)); # don't use the contents 
                                # of x afterwards!
\end{verbatim}
  but not
\begin{verbatim}
var object temp = foo2(foo(x),foo(y));
\end{verbatim}
 Be careful!  If you are not
   sure   what is     allowed   and what    not,   you   can   let  me\\
   \verb+haible@ma2s2.mathematik.uni-karlsruhe.de+ proof-read your code.
   

\item If  you need a global variable  of type {\tt object},  add it at the
   end of {\tt constobj.d} like this: 
\begin{verbatim}
LISPOBJ(foo,"initial value, to be read at init. time")
\end{verbatim}
 You can then refer  to the variable as
   {\tt O(foo)}.  Of  course, you will have  a reentrancy problem when using
   global variables.
\end{enumerate}

Lisp functions with $n$ required arguments are declared like this:
\begin{verbatim}
LISPFUNN(foo_bar,n) { /* ... function body ... */ }
\end{verbatim}
\noindent Also add a line:
\begin{verbatim}
LISPFUNN(foo_bar,n)
\end{verbatim}
at the end of {\tt subr.d}.

With every Lisp function is associated its name, a symbol. To declare a
symbol known at compile-time, add a line:
\begin{verbatim}
LISPSYM(foo_bar,"FOO-BAR",pack)
\end{verbatim}
at the end of {\tt constsym.d}. The string is the symbol's print name,
and pack is  either  {\tt lisp}, {\tt system},  {\tt  keyword}  or any
package name defined in {\tt  constpack.d}.  Be careful when  choosing
the value {\tt lisp} for pack: you  may produce package conflicts with
applications.  You can then refer  to the symbol as \verb+S(foo_bar)+.
{\tt Nil} is nothing more than {\tt S(nil)}.


\subsection{An example}

As an example, consider the 8-queens problem. The brute-force search may
gain speed when written in assembly language or in C.

We may generate {\tt queens.o} either from:

\smallskip\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{\tt queens.c}\\\hline
\begin{verbatim}
/* Compute the number of solutions to the n-queens problem on a nxn checkboard. */

/* dynamic data structures not needed for such a simple problem */
#define nmax 100

int queens (n) /* function definition in traditional C style */
  int n;
{ /* Compute the solutions of the n-queens problem. Assume n>0, n<=nmax. 
     We look for a function D:{1,...,n} -> {1,...,n} such that D, D+id, D-id 
     are injective. We use backtracking on D(1),...,D(n). We use three arrays
     which contain information about which values are still available for 
     D(i) resp. D(i)+i resp. D(i)-i. */
  int dtab[nmax]; /* values D(1),...D(n) */
  int freetab1[nmax+1]; /* contains 0 if available for D(i) in {1,...,n} */
  int freetab2[2*nmax+1]; /* contains 0 if available for D(i)+i in {2,...,2n} */
  int freetab3a[2*nmax-1]; /* contains 0 if available for D(i)-i  in 
                              {-(n-1),...,n-1} */
\end{verbatim}\\\hline\end{tabular}

\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{{\tt queens.c} (cont.)}\\\hline
\begin{verbatim}
#define freetab3 (&freetab3a[nmax-1])
  /* clear tables */
  { int i; for (i=1; i<=n; i++) { freetab1[i] = 0; } }
  { int i; for (i=2; i<=2*n; i++) { freetab2[i] = 0; } }
  { int i; for (i=-(n-1); i<n; i++) { freetab3[i] = 0; } }
 {int counter = 0;
  int i = 0; /* recursion depth */
  int* Dptr = &dtab[0]; /* points to next free D(i) */
  entry: /* enter recursion */
    i++;
    if (i > n)
      { counter++; }
      else
      { int try;
        for (try = 1; try <= n; try++)
          { if (freetab1[try]==0 && freetab2[try+i]==0 && freetab3[try-i]==0)
              { freetab1[try]=1; freetab2[try+i]=1; freetab3[try-i]=1;
                *Dptr++ = try;
                goto entry;
                comeback:
                try = *--Dptr;
                freetab1[try]=0; freetab2[try+i]=0; 
                freetab3[try-i]=0;
      }   }   }
    i--;
    if (i>0) goto comeback;
  return counter;
}}
\end{verbatim}\\\hline\end{tabular}

\smallskip\noindent or

\smallskip\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{{\tt queens.d} }\\\hline
\begin{verbatim}
# queens.d
# Compute the number of solutions to the n-queens problem on a nxn checkboard.

#include "lispbibl.c"
# dynamic data structures not needed for such a simple problem define nmax 100

global uintL queens (n) # function definition in traditional  C style
  var reg4 uintC n; # use `var' and `register', n needs only 16 bit
{ # Compute the solutions of the n-queens problem. Assume n>0, n<=nmax. 
  # We look for a function D:{1,...,n} -> {1,...,n} such that D, D+id, D-id are 
  # injective. We use backtracking on D(1),...,D(n). We use three arrays which
  # contain information about which values are still available for D(i) resp. D(i)+i 
  # resp. D(i)-i.
\end{verbatim}\\\hline\end{tabular}


\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{{\tt queens.d} (cont.)}\\\hline
\begin{verbatim}
  var uintC dtab[nmax]; # values D(1),...D(n)
  var uintB freetab1[nmax+1]; # contains 0 if available for D(i) in {1,...,n}
  var uintB freetab2[2*nmax+1]; # contains 0 if available for D(i)+i in {2,...,2n}
  var uintB freetab3a[2*nmax-1]; # contains 0 if available for D(i)-i in 
                                 # {-(n-1),...,n-1}
  #define freetab3 (&freetab3a[nmax-1])  # indent this correctly
  # clear tables
  { var reg2 uintC count;
    var reg1 uintB* ptr = &freetab1[1];
    dotimespC(count,n, { *ptr++ = 0; } ); # clear n>0 array  elements
  { var reg2 uintC count;
    var reg1 uintB* ptr = &freetab2[2];
    dotimespC(count,2*n-1, { *ptr++ = 0; } ); # clear 2n-1>0 array elements
  { var reg2 uintC count;
    var reg1 uintB* ptr = &freetab3[-(n-1)];
    dotimespC(count,2*n-1, { *ptr++ = 0; } ); # clear 2n-1>0 array elements
 {var reg5 uintL counter = 0; # may need 32 bits for the counter
  var reg2 uintC i = 0; # recursion depth
  var reg3 uintC* Dptr = &dtab[0]; # points to next free D(i)
  entry: # enter recursion
    i++;
    if (i > n)
      { counter++; }
      else
      { var reg1 uintC try;
        for (try = 1; try <= n; try++)
          { if (freetab1[try]==0 && freetab2[try+i]==0 && 
                freetab3[try-i]==0)
              { freetab1[try]=1; freetab2[try+i]=1; 
                freetab3[try-i]=1;
                *Dptr++ = try;
                goto entry;
                comeback:
                try = *--Dptr;
                freetab1[try]=0; freetab2[try+i]=0; 
                freetab3[try-i]=0;
      }   }   }
    i--;
    if (i>0) goto comeback;
  return counter;
}}
\end{verbatim}\\\hline\end{tabular}

\clearpage
\smallskip\noindent or on 68000 based Unix machines:

\smallskip\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{{\tt queens.S}}\\\hline
\begin{verbatim}
! Processor: 680x0
! Parameter passing: on the stack sp@(4), sp@(8), ...
! Return value: in d0
! Registers a0-a1,d0-d1 may be used freely.
! Registers a2-a4,d2-d7 must be saved when used.

#define nmax 100

           .text
           .globl _queens
_queens:   moveml a2-a3/d2-d6,sp@-   ! save registers
           movel sp@(28+4),d6        ! n
           subw #7*nmax+2,sp         ! make space in the stack: 7*nmax+1 bytes
           leal sp@(2*nmax),a1       ! freetab1
           leal a2@(3*nmax),a3       ! freetab3
           ! clear tables:
           leal a1(1),a0             ! &freetab1[1]
           movew d6,d0
           bras q12
q11:         clrb a0@+
q12:         dbra d0,q11
           leal a2(1),a0             ! &freetab2[1]
           movew d6,d0
           bras q22
q21:         clrb a0@+
             clrb a0@+
q22:         dbra d0,q21
           movel a3,a0
           movew d6,d0
           subqw #1,d0
           subw d0,a0                ! &freetab3[-(n-1)]
           addw d6,d0
           bras q32
q31:         clrb a0@+
q32:         dbra d0,q31
           clrl d7                   ! counter := 0
           clrw d5                   ! i := 0
           leal sp@(0),a0            ! dptr := dtab
entry:     ! enter recursion
           addqw #1,d5               ! i++
           cmpw d6,d5
           bhis count                ! i > n ?
           moveq #1,d1               ! try := 1
tryup:       movew d1,d2
             movew d1,d3
\end{verbatim}\\\hline\end{tabular}
\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{{\tt queens.S} (cont.)}\\\hline
\begin{verbatim}
             addw d5,d2              ! try + i
             subw d5,d3              ! try - i
             moveb a1@(d1.w),d0
             orb a2@(d2.w),d0
             orb a3@(d3.w),d0
             bnes skip
             st a1@(d1.w)
             st a2@(d2.w)
             st a3@(d3.w)
             movew d1,a0@+
             bras entry
comeback:    movew a0@-,d1
             movew d1,d2
             movew d1,d3
             addw d5,d2              ! try + i
             subw d5,d3              ! try - i
             clrb a1@(d1.w)
             clrb a2@(d2.w)
             clrb a3@(d3.w)
skip:        addqw #1,d1             ! try++
             cmpw d6,d1
             blss tryup              ! try <= n ?
return:    subqw #1,d5
           bnes comeback
           movel d4,d0               ! return counter;
           addw #7*nmax+2,sp
           moveml sp@+,a2-a3/d2-d6   ! restore registers
           rts
count:     addql #1,d4               ! counter++
           bras return
\end{verbatim}\\\hline\end{tabular}

\bigskip\noindent We furthermore need an interface:

\smallskip\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{{\tt callqueens.d}}\\\hline
\begin{verbatim}
# Interface to the queens() function

#include "lispbibl.c"

# Conditionalize, such that we don't lose the ability to 
# build CLISP without the QUEENS function. The flag -DQUEENS 
# must be given to the C compiler.

#ifdef QUEENS

#define nmax 100
\end{verbatim}\\\hline\end{tabular}

\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{{\tt callqueens.d} (cont.)}\\\hline
\begin{verbatim}
# extern declaration
extern int queens (int n); # use this one if using 
                           # queens.c above
extern uintL queens (uintC n); # use this one if using 
                               # queens.d above
extern uintL queens (uintL n); # use this one if using 
                               # queens.S above

# (LISP:QUEENS n) returns the number of solutions to the 
# n-queens problem. n ought to be an integer > 0, <= nmax. 
# Otherwise it returns NIL.
LISPFUNN(queens,1)
{ # No garbage collection is a problem. So we get the 
  # argument from the STACK immediately.

  var reg1 object arg = popSTACK(); # clean up STACK at the 
  # same time. If arg is an integer > 0, <= 100, ist must be 
  # a nonnegative fixnum. We do the argument check in two 
  # steps: 1. check whether arg is a nonnegative fixnum. 
  # 2. Extract its value. 3. Check its value.
  if (!posfixnump(arg)) goto bad_arg;
 {var reg2 uintL n = posfixnum_to_L(arg);
  if (!(n>0 && n<=nmax)) goto bad_arg;

  # Arguments are checked. Do our job:
  { var reg3 uintL result;
    begin_call(); # not needed when using queens.d above
    result = queens(n); # call external function
    end_call(); # not needed when using queens.d above
    # Assume result is >=0 and <2^32 (which is guaranteed by 
    # the type of problem we have and the amount of time 
    # queens() may have run). So an uintL is enough, and the 
    # following call is appropriate.
    value1 = UL_to_I(result); # convert result to nonnegative 
                              #integer
    mv_count=1; # no "multiple" values
  }
  return;
 }
  bad_arg:
    # We could issue an error. We prefer to return NIL here.
    value1 = NIL; mv_count=1; return;
}

#endif # QUEENS
\end{verbatim}\\\hline\end{tabular}
\clearpage
\bigskip\noindent To {\tt subr.d} we add the lines

\smallskip\noindent\begin{tabular}{|p{6.335in}|}\hline
\multicolumn{1}{|c|}{{\tt subr.d}}\\\hline
\begin{verbatim}
# ---------- CALLQUEENS ----------
#ifdef QUEENS
LISPFUNN(queens,1)
#endif

To constsym.d we add the lines

# ---------- CALLQUEENS ----------
#ifdef QUEENS
LISPSYM(queens,"QUEENS",lisp) # put it into the LISP package 
                              # (quick and dirty) such that it 
                              # will be visible in the USER 
                              # package
#endif
\end{verbatim}\\\hline\end{tabular}


\bigskip
That's all. After  having built a  new CLISP you will  be able to call
{\tt (queens 8)}.

