Chapter 8  Alien Objects

• Introduction to Aliens
• Alien Types
  • Defining Alien Types
  • Alien Types and Lisp Types
  • Alien Type Specifiers
  • The C-Call Package
• Alien Operations
  • Alien Access Operations
  • Alien Coercion Operations
  • Alien Dynamic Allocation
• Alien Variables
  • Local Alien Variables
  • External Alien Variables
• Alien Data Structure Example
• Loading Unix Object Files
• Alien Function Calls
  • The alien-funcall Primitive
  • The def-alien-routine Macro
  • def-alien-routine Example
  • Calling Lisp from C
  • Callback Example
  • Accessing Lisp Arrays
• Step-by-Step Alien Example

8.1  Introduction to Aliens

Because of Lisp’s emphasis on dynamic memory allocation and garbage collection, Lisp implementations use unconventional memory representations for objects. This representation mismatch creates problems when a Lisp program must share objects with programs written in another language. There are three different approaches to establishing communication:

• The burden can be placed on the foreign program (and programmer) by requiring the use of Lisp object representations. The main difficulty with this approach is that either the foreign program must be written with Lisp interaction in mind, or a substantial amount of foreign “glue” code must be written to perform the translation.
• The Lisp system can automatically convert objects back and forth between the Lisp and foreign representations. This is convenient, but translation becomes prohibitively slow when large or complex data structures must be shared.
• The Lisp program can directly manipulate foreign objects through the use of extensions to the Lisp language. Most Lisp systems make use of this approach, but the language for describing types and expressing accesses is often not powerful enough for complex objects to be easily manipulated.

CMUCL relies primarily on the automatic conversion and direct manipulation approaches: Aliens of simple scalar types are automatically converted, while complex types are directly manipulated in their foreign representation. Any foreign objects that can’t automatically be converted into Lisp values are represented by objects of type

. Since Lisp is a dynamically typed language, even foreign objects must have a run-time type; this type information is provided by encapsulating the raw pointer to the foreign data within an

object.

The Alien type language and operations are most similar to those of the C language, but Aliens can also be used when communicating with most other languages that can be linked with C.

8.2  Alien Types

Alien types have a description language based on nested list structure. For example:

struct foo {
    int a;
    struct foo *b[100];
};

has the corresponding Alien type:

(struct foo
  (a int)
  (b (array (* (struct foo)) 100)))

8.2.1  Defining Alien Types

Types may be either named or anonymous. With structure and union types, the name is part of the type specifier, allowing recursively defined types such as:

(struct foo (a (* (struct foo))))

An anonymous structure or union type is specified by using the name

. The

macro defines a local scope which “captures” any named type definitions. Other types are not inherently named, but can be given named abbreviations using

.

This macro globally defines name as a shorthand for the Alien type type. When introducing global structure and union type definitions, name may be nil, in which case the name to define is taken from the type’s name.

8.2.2  Alien Types and Lisp Types

The Alien types form a subsystem of the CMUCL type system. An

type specifier provides a way to use any Alien type as a Lisp type specifier. For example

(typep foo ’(alien (* int)))

can be used to determine whether

is a pointer to an

.

type specifiers can be used in the same ways as ordinary type specifiers (like

.) Alien type declarations are subject to the same precise type checking as any other declaration (see section 4.5.2.)

Note that the Alien type system overlaps with normal Lisp type specifiers in some cases. For example, the type specifier

is identical to

, since Alien floats are automatically converted to Lisp floats. When

is called on an Alien value that is not automatically converted to a Lisp value, then it will return an

type specifier.

8.2.3  Alien Type Specifiers

Some Alien type names are Common Lisp symbols, but the names are still exported from the

package, so it is legal to say

. These are the basic Alien type specifiers:

A pointer to an object of the specified

type

. If

type

is

t

, then it means a pointer to anything, similar to “

void *

” in ANSI C. Currently, the only way to detect a null pointer is:

  (zerop (sap-int (alien-sap ptr)))

See section 6.5

An array of the specified

dimensions

, holding elements of type

type

. Note that

(* int)

and

(array int)

are considered to be different types when type checking is done; pointer and array types must be explicitly coerced using

cast

.

Arrays are accessed using

deref

, passing the indices as additional arguments. Elements are stored in column-major order (as in C), so the first dimension determines only the size of the memory block, and not the layout of the higher dimensions. An array whose first dimension is variable may be specified by using

nil

as the first dimension. Fixed-size arrays can be allocated as array elements, structure slots or

with-alien

variables. Dynamic arrays can only be allocated using

make-alien

.

A structure type with the specified

name

and

fields

. Fields are allocated at the same positions used by the implementation’s C compiler.

bits

is intended for C-like bit field support, but is currently unused. If

name

is

nil

, then the type is anonymous.

If a named Alien

struct

specifier is passed to

def-alien-type

or

with-alien

, then this defines, respectively, a new global or local Alien structure type. If no

fields

are specified, then the fields are taken from the current (local or global) Alien structure type definition of

name

.

Similar to

struct

, but defines a union type. All fields are allocated at the same offset, and the size of the union is the size of the largest field. The programmer must determine which field is active from context.

An enumeration type that maps between integer values and keywords. If

name

is

nil

, then the type is anonymous. Each

spec

is either a keyword, or a list

(keyword value)

. If

integer

is not supplied, then it defaults to one greater than the value for the preceding spec (or to zero if it is the first spec.)

A signed integer with the specified number of bits precision. The upper limit on integer precision is determined by the machine’s word size. If no size is specified, the maximum size will be used.

Identical to signed—the distinction between signed and integer is purely stylistic.

Like signed, but specifies an unsigned integer.

Similar to an enumeration type that maps 0 to nil and all other values to t. bits determines the amount of storage allocated to hold the truth value.

A floating-point number in IEEE single format.

A floating-point number in IEEE double format.

A Alien function that takes arguments of the specified arg-types and returns a result of type result-type. Note that the only context where a function type is directly specified is in the argument to alien-funcall (see section alien-funcall.) In all other contexts, functions are represented by function pointer types: (* (function ...)).

A pointer which is represented in Lisp as a system-area-pointer object (see section 6.5.)

8.2.4  The C-Call Package

The

package exports these type-equivalents to the C type of the same name:

,

,

,

,

,

,

,

,

,

.

also exports these types:

This type is used in function types to declare that no useful value is returned. Evaluation of an alien-funcall form will return zero values.

This type is similar to (* char), but is interpreted as a null-terminated string, and is automatically converted into a Lisp string when accessed. If the pointer is C NULL (or 0), then accessing gives Lisp nil.

With Unicode, a Lisp string is not the same as a C string since a Lisp string uses two bytes for each character. In this case, a C string is converted to a Lisp string by taking each byte of the C-string and applying

code-char

to create each character of the Lisp string.

Similarly, a Lisp string is converted to a C string by taking the low 8 bits of the

char-code

of each character and assigning that to each byte of the C string.

In either case,

string-encode

and

string-decode

may be useful to convert Unicode Lisp strings to or from C strings.

Assigning a Lisp string to a

c-string

structure field or variable stores the contents of the string to the memory already pointed to by that variable. When an Alien of type

(* char)

is assigned to a

c-string

, then the

c-string

pointer is assigned to. This allows

c-string

pointers to be initialized. For example:

  (def-alien-type nil (struct foo (str c-string)))
  
  (defun make-foo (str)
    (let ((my-foo (make-alien (struct foo))))
      (setf (slot my-foo ’str) (make-alien char (length str)))
      (setf (slot my-foo ’str) str)
      my-foo))

Storing Lisp

nil

writes C

NULL

to the

c-string

pointer.

8.3  Alien Operations

This section describes the basic operations on Alien values.

8.3.1  Alien Access Operations

This function returns the value pointed to by an Alien pointer or the value of an Alien array element. If a pointer, an optional single index can be specified to give the equivalent of C pointer arithmetic; this index is scaled by the size of the type pointed to. If an array, the number of indices must be the same as the number of dimensions in the array type.

deref

can be set with

setf

to assign a new value.

This function extracts the value of slot

slot-name

from the an Alien

struct

or

union

. If

struct-or-union

is a pointer to a structure or union, then it is automatically dereferenced. This can be set with

setf

to assign a new value. Note that

slot-name

is evaluated, and need not be a compile-time constant (but only constant slot accesses are efficiently compiled.)

8.3.2  Alien Coercion Operations

This macro returns a pointer to the location specified by

alien-expr

, which must be either an Alien variable, a use of

deref

, a use of

slot

, or a use of

extern-alien

.

This macro converts

alien

to a new Alien with the specified

new-type

. Both types must be an Alien pointer, array or function type. Note that the result is not

eq

to the argument, but does refer to the same data bits.

[Function]
alien:alien-sap alien-value    

sap-alien

converts

sap

(a system area pointer see section 6.5) to an Alien value with the specified

type

.

type

is not evaluated.

alien-sap

returns the SAP which points to

alien-value

’s data.

The

type

to

sap-alien

and the type of the

alien-value

to

alien-sap

must some Alien pointer, array or record type.

8.3.3  Alien Dynamic Allocation

Dynamic Aliens are allocated using the

library, so foreign code can call

on the result of

, and Lisp code can call

on objects allocated by foreign code.

This macro returns a dynamically allocated Alien of the specified

type

(which is not evaluated.) The allocated memory is not initialized, and may contain arbitrary junk. If supplied,

size

is an expression to evaluate to compute the size of the allocated object. There are two major cases:

• When type is an array type, an array of that type is allocated and a pointer to it is returned. Note that you must use deref to change the result to an array before you can use deref to read or write elements:

  (defvar *foo* (make-alien (array char 10)))
  
  (type-of *foo*) ==> (alien (* (array (signed 8) 10)))
  
  (setf (deref (deref foo) 0) 10) ==> 10
  

  If supplied,

  size

  is used as the first dimension for the array.

• When type is any other type, then then an object for that type is allocated, and a pointer to it is returned. So (make-alien int) returns a (* int). If size is specified, then a block of that many objects is allocated, with the result pointing to the first one.

This function frees the storage for

alien

(which must have been allocated with

make-alien

or

malloc

.)

See also

, which stack-allocates Aliens.

8.4  Alien Variables

Both local (stack allocated) and external (C global) Alien variables are supported.

8.4.1  Local Alien Variables

This macro establishes local alien variables with the specified Alien types and names for dynamic extent of the body. The variable

names

are established as symbol-macros; the bindings have lexical scope, and may be assigned with

setq

or

setf

. This form is analogous to defining a local variable in C: additional storage is allocated, and the initial value is copied.

with-alien

also establishes a new scope for named structures and unions. Any

type

specified for a variable may contain name structure or union types with the slots specified. Within the lexical scope of the binding specifiers and body, a locally defined structure type

foo

can be referenced by its name using:

  (struct foo)

8.4.2  External Alien Variables

External Alien names are strings, and Lisp names are symbols. When an external Alien is represented using a Lisp variable, there must be a way to convert from one name syntax into the other. The macros

,

and

use this conversion heuristic:

• Alien names are converted to Lisp names by uppercasing and replacing underscores with hyphens.
• Conversely, Lisp names are converted to Alien names by lowercasing and replacing hyphens with underscores.
• Both the Lisp symbol and Alien string names may be separately specified by using a list of the form:

    (alien-string lisp-symbol)
  

This macro defines

name

as an external Alien variable of the specified Alien

type

.

name

and

type

are not evaluated. The Lisp name of the variable (see above) becomes a global Alien variable in the Lisp namespace. Global Alien variables are effectively “global symbol macros”; a reference to the variable fetches the contents of the external variable. Similarly, setting the variable stores new contents—the new contents must be of the declared

type

.

For example, it is often necessary to read the global C variable

errno

to determine why a particular function call failed. It is possible to define errno and make it accessible from Lisp by the following:

(def-alien-variable "errno" int)

;; Now it is possible to get the value of the C variable errno simply by
;; referencing that Lisp variable:
;;
(print errno)

This macro returns an Alien with the specified

type

which points to an externally defined value.

name

is not evaluated, and may be specified either as a string or a symbol.

type

is an unevaluated Alien type specifier.

8.5  Alien Data Structure Example

Now that we have Alien types, operations and variables, we can manipulate foreign data structures. This C declaration can be translated into the following Alien type:

struct foo {
    int a;
    struct foo *b[100];
};

 <==>

(def-alien-type nil
  (struct foo
    (a int)
    (b (array (* (struct foo)) 100))))

With this definition, the following C expression can be translated in this way:

struct foo f;
f.b[7].a

 <==>

(with-alien ((f (struct foo)))
  (slot (deref (slot f ’b) 7) ’a)
  ;;
  ;; Do something with f...
  )

Or consider this example of an external C variable and some accesses:

struct c_struct {
        short x, y;
        char a, b;
        int z;
        c_struct *n;
};

extern struct c_struct *my_struct;

my_struct->x++;
my_struct->a = 5;
my_struct = my_struct->n;

which can be made be manipulated in Lisp like this:

(def-alien-type nil
  (struct c-struct
          (x short)
          (y short)
          (a char)
          (b char)
          (z int)
          (n (* c-struct))))

(def-alien-variable "my_struct" (* c-struct))

(incf (slot my-struct ’x))
(setf (slot my-struct ’a) 5)
(setq my-struct (slot my-struct ’n))

8.6  Loading Unix Object Files

CMUCL is able to load foreign object files at runtime, using the function

. This function is able to load shared libraries (that are typically named .so) via the dlopen mechanism. It can also load .a or .o object files by calling the linker on the files and libraries to create a loadable object file. Once loaded, the external symbols that define routines and variables are made available for future external references (e.g. by

.)

must be run before any of the defined symbols are referenced.

Note that if a Lisp core image is saved (using

), all loaded foreign code is lost when the image is restarted.

files

is a

simple-string

or list of

simple-string

s specifying the names of the object files. If

files

is a simple-string, the file that it designates is loaded using the platform’s dlopen mechanism. If it is a list of strings, the platform linker

ld

is invoked to transform the object files into a loadable object file.

libraries

is a list of

simple-string

s specifying libraries in a format that the platform linker expects. The default value for

libraries

is

("-lc")

(i.e., the standard C library).

base-file

is the file to use for the initial symbol table information. The default is the Lisp start up code:

path:lisp

.

env

should be a list of simple strings in the format of Unix environment variables (i.e.,

A=B

, where

A

is an environment variable and

B

is its value). The default value for

env

is the environment information available at the time Lisp was invoked. Unless you are certain that you want to change this, you should just use the default.

8.7  Alien Function Calls

The foreign function call interface allows a Lisp program to call functions written in other languages. The current implementation of the foreign function call interface assumes a C calling convention and thus routines written in any language that adheres to this convention may be called from Lisp.

Lisp sets up various interrupt handling routines and other environment information when it first starts up, and expects these to be in place at all times. The C functions called by Lisp should either not change the environment, especially the interrupt entry points, or should make sure that these entry points are restored when the C function returns to Lisp. If a C function makes changes without restoring things to the way they were when the C function was entered, there is no telling what will happen.

8.7.1  The alien-funcall Primitive

This function is the foreign function call primitive:

alien-function

is called with the supplied

arguments

and its value is returned. The

alien-function

is an arbitrary run-time expression; to call a constant function, use

extern-alien

or

def-alien-routine

.

The type of

alien-function

must be

(alien (function ...))

or

(alien (* (function ...)))

, See section 8.2.3. The function type is used to determine how to call the function (as though it was declared with a prototype.) The type need not be known at compile time, but only known-type calls are efficiently compiled. Limitations:

• Structure type return values are not implemented.
• Passing of structures by value is not implemented.

Here is an example which allocates a

, calls a foreign function to initialize it, then returns a Lisp vector of all the

objects filled in by the foreign call:

;; Allocate a foo on the stack.
(with-alien ((f (struct foo)))
  ;;
  ;; Call some C function to fill in foo fields.
  (alien-funcall (extern-alien "mangle_foo" (function void (* foo)))
                 (addr f))
  ;;
  ;; Find how many foos to use by getting the A field.
  (let* ((num (slot f ’a))
         (result (make-array num)))
    ;;
    ;; Get a pointer to the array so that we don’t have to keep
    ;; extracting it:
    (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f ’b))))
      ;;
      ;; Loop over the first N elements and stash them in the
      ;; result vector.
      (dotimes (i num)
        (setf (svref result i) (deref (deref a) i)))
      result)))

8.7.2  The def-alien-routine Macro

This macro is a convenience for automatically generating Lisp interfaces to simple foreign functions. The primary feature is the parameter style specification, which translates the C pass-by-reference idiom into additional return values.

name

is usually a string external symbol, but may also be a symbol Lisp name or a list of the foreign name and the Lisp name. If only one name is specified, the other is automatically derived, (see section 8.4.2.)

result-type

is the Alien type of the return value. Each remaining subform specifies an argument to the foreign function.

aname

is the symbol name of the argument to the constructed function (for documentation) and

atype

is the Alien type of corresponding foreign argument. The semantics of the actual call are the same as for

alien-funcall

.

style

should be one of the following:

:in

specifies that the argument is passed by value. This is the default. :in arguments have no corresponding return value from the Lisp function.

:out

specifies a pass-by-reference output value. The type of the argument must be a pointer to a fixed sized object (such as an integer or pointer). :out and :in-out cannot be used with pointers to arrays, records or functions. An object of the correct size is allocated, and its address is passed to the foreign function. When the function returns, the contents of this location are returned as one of the values of the Lisp function.

:copy

is similar to :in, but the argument is copied to a pre-allocated object and a pointer to this object is passed to the foreign routine.

:in-out

is a combination of :copy and :out. The argument is copied to a pre-allocated object and a pointer to this object is passed to the foreign routine. On return, the contents of this location is returned as an additional value.

Any efficiency-critical foreign interface function should be inline expanded by preceding

def-alien-routine

with:

(declaim (inline lisp-name))

In addition to avoiding the Lisp call overhead, this allows pointers, word-integers and floats to be passed using non-descriptor representations, avoiding consing (see section 5.11.2.)

8.7.3  def-alien-routine Example

Consider the C function

with the following calling convention:

/* a for update
 * i out
 */
void cfoo (char *str, char *a, int *i);

which can be described by the following call to

:

(def-alien-routine "cfoo" void
  (str c-string)
  (a char :in-out)
  (i int :out))

The Lisp function

will have two arguments (

and

) and two return values (

and

).

8.7.4  Calling Lisp from C

CMUCL supports calling Lisp from C via the

macro:

This macro defines a Lisp function that can be called from C and a Lisp variable. The arguments to the function must be alien types, and the return type must also be an alien type. This Lisp function can be accessed via the callback macro.name

is the name of the Lisp function. It is also the name of a variable to be used by the

callback

macro.

return-type

is the return type of the function. This must be a recognized alien type.

arg-name

specifies the name of the argument to the function, and the argument has type

arg-type

, which must be an alien type.

This macro extracts the appropriate information for the function named callback-symbol so that it can be called by a C function. callback-symbol must be a symbol created by the def-callback macro.

This macro does the necessary stuff to call the callback named callback-name with the given arguments.

8.7.5  Callback Example

Here is a simple example of using callbacks.

(use-package :alien)
(use-package :c-call)

(def-callback foo (int (arg1 int) (arg2 int))
  (format t "~&foo: ~S, ~S~%" arg1 arg2)
  (+ arg1 arg2))

(defun test-foo ()
  (callback-funcall foo 555 444444))

In this example, the callback function

is defined which takes two C

parameters and returns a

. As this shows, we can use arbitrary Lisp inside the function.

The function

shows how we can call this callback function from Lisp. The macro

extracts the necessary information for the callback function

which can be converted into a pointer which we can call via

.

The following code is a more complete example where a foreign routine calls our Lisp routine.

(use-package :alien)
(use-package :c-call)

(def-alien-routine qsort void
  (base (* t))
  (nmemb int)
  (size int)
  (compar (* (function int (* t) (* t)))))

(def-callback my< (int (arg1 (* double))
                       (arg2 (* double)))
  (let ((a1 (deref arg1))
        (a2 (deref arg2)))
    (cond ((= a1 a2)  0)
          ((< a1 a2) -1)
          (t         +1))))

(defun test-qsort ()
  (let ((a (make-array 10 :element-type ’double-float
                       :initial-contents ’(0.1d0 0.5d0 0.2d0 1.2d0 1.5d0
                                           2.5d0 0.0d0 0.1d0 0.2d0 0.3d0))))
    (print a)
    (qsort (sys:vector-sap a)
           (length a)
           (alien-size double :bytes)
           (alien:callback my<))
    (print a)))

We define the alien routine,

, and a callback,

, to determine whether two

’s are less than, greater than or equal to each other.

The test function

shows how we can call the alien sort routine with our Lisp comparison routine to produce a sorted array.

8.7.6  Accessing Lisp Arrays

Due to the way CMUCL manages memory, the amount of memory that can be dynamically allocated by

or

is limited¹.

To overcome this limitation, it is possible to access the content of Lisp arrays which are limited only by the amount of physical memory and swap space available. However, this technique is only useful if the foreign function takes pointers to memory instead of allocating memory for itself. In latter case, you will have to modify the foreign functions.

This technique takes advantage of the fact that CMUCL has specialized array types (see section 5.11.8) that match a typical C array. For example, a

is stored in memory in essentially the same way as the C array

would be. The following function allows us to get the physical address of such a Lisp array:

(defun array-data-address (array)
  "Return the physical address of where the actual data of an array is
stored.

ARRAY must be a specialized array type in CMUCL.  This means ARRAY
must be an array of one of the following types:

                  double-float
                  single-float
                  (unsigned-byte 32)
                  (unsigned-byte 16)
                  (unsigned-byte  8)
                  (signed-byte 32)
                  (signed-byte 16)
                  (signed-byte  8)
"
  (declare (type (or (simple-array (signed-byte 8))
                     (simple-array (signed-byte 16))
                     (simple-array (signed-byte 32))
                     (simple-array (unsigned-byte 8))
                     (simple-array (unsigned-byte 16))
                     (simple-array (unsigned-byte 32))
                     (simple-array single-float)
                     (simple-array double-float)
                     (simple-array (complex single-float))
                     (simple-array (complex double-float)))
                 array)
           (optimize (speed 3) (safety 0))
           (ext:optimize-interface (safety 3)))
  ;; with-array-data will get us to the actual data.  However, because
  ;; the array could have been displaced, we need to know where the
  ;; data starts.
  (lisp::with-array-data ((data array)
                          (start)
                          (end))
    (declare (ignore end))
    ;; DATA is a specialized simple-array.  Memory is laid out like this:
    ;;
    ;;   byte offset    Value
    ;;        0         type code (should be 70 for double-float vector)
    ;;        4         4 * number of elements in vector
    ;;        8         1st element of vector
    ;;      ...         ...
    ;;
    (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data))))
          (type-size
           (let ((type (array-element-type data)))
             (cond ((or (equal type ’(signed-byte 8))
                        (equal type ’(unsigned-byte 8)))
                    1)
                   ((or (equal type ’(signed-byte 16))
                        (equal type ’(unsigned-byte 16)))
                    2)
                   ((or (equal type ’(signed-byte 32))
                        (equal type ’(unsigned-byte 32)))
                    4)
                   ((equal type ’single-float)
                    4)
                   ((equal type ’double-float)
                    8)
                   (t
                    (error "Unknown specialized array element type"))))))
      (declare (type (unsigned-byte 32) addr)
               (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3)))
      (system:int-sap (the (unsigned-byte 32)
                        (+ addr (* type-size start)))))))

We note, however, that the system function

will do the same thing as above does.

Assume we have the C function below that we wish to use:

  double dotprod(double* x, double* y, int n)
  {
    int k;
    double sum = 0;

    for (k = 0; k < n; ++k) {
      sum += x[k] * y[k];
    }
    return sum;
  }

The following example generates two large arrays in Lisp, and calls the C function to do the desired computation. This would not have been possible using

or

since we need about 16 MB of memory to hold the two arrays.

  (alien:def-alien-routine "dotprod" c-call:double
    (x (* double-float) :in)
    (y (* double-float) :in)
    (n c-call:int :in))
    
  (defun test-dotprod ()
    (let ((x (make-array 10000 :element-type ’double-float 
                         :initial-element 2d0))
          (y (make-array 10000 :element-type ’double-float
                         :initial-element 10d0)))
        (sys:without-gcing
          (let ((x-addr (sys:vector-sap x))
                (y-addr (sys:vector-sap y)))
            (dotprod x-addr y-addr 10000)))))

In this example, we have used

instead of

, but we could have used

as well.

Also, we have wrapped the inner

expression in a

that disables garbage collection for the duration of the body. This will prevent garbage collection from moving

and

arrays after we have obtained the (now erroneous) addresses but before the call to

is made.

8.8  Step-by-Step Alien Example

This section presents a complete example of an interface to a somewhat complicated C function. This example should give a fairly good idea of how to get the effect you want for almost any kind of C function. Suppose you have the following C function which you want to be able to call from Lisp in the file

:

                
struct c_struct
{
  int x;
  char *s;
};
 
struct c_struct *c_function (i, s, r, a)
    int i;
    char *s;
    struct c_struct *r;
    int a[10];
{
  int j;
  struct c_struct *r2;
 
  printf("i = %d\n", i);
  printf("s = %s\n", s);
  printf("r->x = %d\n", r->x);
  printf("r->s = %s\n", r->s);
  for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]);
  r2 = (struct c_struct *) malloc (sizeof(struct c_struct));
  r2->x = i + 5;
  r2->s = "A C string";
  return(r2);
};

It is possible to call this function from Lisp using the file

whose contents is:

;;; -*- Package: test-c-call -*-
(in-package "TEST-C-CALL")
(use-package "ALIEN")
(use-package "C-CALL")

;;; Define the record c-struct in Lisp.
(def-alien-type nil
    (struct c-struct
            (x int)
            (s c-string)))

;;; Define the Lisp function interface to the C routine.  It returns a
;;; pointer to a record of type c-struct.  It accepts four parameters:
;;; i, an int; s, a pointer to a string; r, a pointer to a c-struct
;;; record; and a, a pointer to the array of 10 ints.
;;;
;;; The INLINE declaration eliminates some efficiency notes about heap
;;; allocation of Alien values.
(declaim (inline c-function))
(def-alien-routine c-function
    (* (struct c-struct))
  (i int)
  (s c-string)
  (r (* (struct c-struct)))
  (a (array int 10)))

;;; A function which sets up the parameters to the C function and
;;; actually calls it.
(defun call-cfun ()
  (with-alien ((ar (array int 10))
               (c-struct (struct c-struct)))
    (dotimes (i 10)                     ; Fill array.
      (setf (deref ar i) i))
    (setf (slot c-struct ’x) 20)
    (setf (slot c-struct ’s) "A Lisp String")

    (with-alien ((res (* (struct c-struct))
                 (c-function 5 "Another Lisp String" (addr c-struct) ar)))
      (format t "Returned from C function.~%")
      (multiple-value-prog1
          (values (slot res ’x)
                  (slot res ’s))
        ;;              
        ;; Deallocate result  after we are done using it.
        (free-alien res)))))

To execute the above example, it is necessary to compile the C routine as follows:

cc -c test.c

In order to enable incremental loading with some linkers, you may need to say:

cc -G 0 -c test.c

Once the C code has been compiled, you can start up Lisp and load it in:

% lisp
;;; Lisp should start up with its normal prompt.

;;; Compile the Lisp file.  This step can be done separately.  You don’t have
;;; to recompile every time.
* (compile-file "test.lisp")

;;; Load the foreign object file to define the necessary symbols.  This must
;;; be done before loading any code that refers to these symbols.  next block
;;; of comments are actually the output of LOAD-FOREIGN.  Different linkers
;;; will give different warnings, but some warning about redefining the code
;;; size is typical.
* (load-foreign "test.o")

;;; Running library:load-foreign.csh...
;;; Loading object file...
;;; Parsing symbol table...
Warning:  "_gp" moved from #x00C082C0 to #x00C08460.
Warning:  "end" moved from #x00C00340 to #x00C004E0.

;;; o.k. now load the compiled Lisp object file.
* (load "test")

;;; Now we can call the routine that sets up the parameters and calls the C
;;; function.
* (test-c-call::call-cfun)

;;; The C routine prints the following information to standard output.
i = 5
s = Another Lisp string
r->x = 20
r->s = A Lisp string
a[0] = 0.
a[1] = 1.
a[2] = 2.
a[3] = 3.
a[4] = 4.
a[5] = 5.
a[6] = 6.
a[7] = 7.
a[8] = 8.
a[9] = 9.
;;; Lisp prints out the following information.
Returned from C function.
;;; Return values from the call to test-c-call::call-cfun.
10
"A C string"
*

If any of the foreign functions do output, they should not be called from within Hemlock. Depending on the situation, various strange behavior occurs. Under X, the output goes to the window in which Lisp was started; on a terminal, the output will overwrite the Hemlock screen image; in a Hemlock slave, standard output is

by default, so any output is discarded.

1

CMUCL mmaps a large piece of memory for its own use and this memory is typically about 256 MB above the start of the C heap. Thus, only about 256 MB of memory can be dynamically allocated. In earlier versions, this limit was closer to 8 MB.