source: trunk/source/doc/src/ffi.xml @ 8903

Last change on this file since 8903 was 8903, checked in by mikel, 11 years ago

added discussion of support for foreign types as classes

File size: 159.5 KB
Line 
1<?xml version="1.0" encoding="utf-8"?>
2<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
3          <!ENTITY rest "<varname>&amp;rest</varname>">
4          <!ENTITY key "<varname>&amp;key</varname>">
5          <!ENTITY optional "<varname>&amp;optional</varname>">
6          <!ENTITY body "<varname>&amp;body</varname>">
7          <!ENTITY aux "<varname>&amp;aux</varname>">
8          <!ENTITY allow-other-keys "<varname>&amp;allow-other-keys</varname>">
9          <!ENTITY CCL "<literal>CCL</literal>">
10          ]>
11
12<chapter id="The-Foreign-Function-Interface">
13  <title>The Foreign-Function Interface</title>
14 
15  <!-- ******************************************  -->
16  <sect1 id="Specifying-And-Using-Foreign-Types">
17    <title>Specifying And Using Foreign Types</title>
18   
19    <sect2 id="Overview-foreign-types">
20      <title>Overview</title>
21      <para>&CCL; provides a fairly rich language for defining and
22        specifying foreign data types (this language is derived from
23        CMUCL's "alien type" system.)</para>
24      <para>In practice, most foreign type definitions are
25        introduced into &CCL; via its interface database (see ),
26        though it's also possible to define foreign types
27        interactively and/or programmatically.</para>
28      <para>&CCL;'s foreign type system is "evolving" (a polite word
29        for not-quite-complete): there are some inconsistencies
30        involving package usage, for instance. Symbols used in foreign
31        type specifiers <emphasis>should</emphasis> be keywords, but
32        this convention isn't always enforced.</para> <para>Foreign
33        type, record, and field names are case-sensitive; &CCL; uses
34        some escaping conventions (see ) to allow keywords to be used to
35        denote these names.</para>
36
37      <sect3 id="type annotations">
38        <title>Type Annotations</title>
39        <para>As of version 1.2, &CCL; supports annotating the types of
40          foreign pointers on Mac OS X. Forms that create pointers to
41          foreign memory&mdash;that is, <code>MACPTR</code>s&mdash;store
42          with the <code>MACPTR</code> object a type annotation that
43          identifies the foreign type of the object pointed
44          to. Calling <code>PRINT-OBJECT</code> on a <code>MACPTR</code>
45          attempts to print information about the identified foreign
46          type, including whether it was allocated on the heap or the
47          stack, and whether it's scheduled for automatic reclamation by
48          the garbage collector.</para>
49
50        <para>Support for type annotation is not yet complete. In
51          particular, some uses of <code>PREF</code>
52          and <code>SLOT-VALUE</code> do ot yet take type annotations into
53          account, and neither do <code>DESCRIBE</code>
54          and <code>INSPECT</code>.</para>
55      </sect3>
56
57      <sect3 id="foreign-type-classes">
58        <title>Foreign Types as Classes</title>
59        <para>Some types of foreign pointers take advantage of the
60          support for type annotations, and pointers of these types
61          can be treated as instances of known classes. Specifically,
62          a pointer to an <code>:&lt;NSR&gt;ect</code> is recognized
63          as an instance of the built-in
64          class <code>NS:NS-RECT</code>, a pointer to
65          an <code>&lt;NSS&gt;ize</code> is treated as an instance
66          of <code>NS:NS-SIZE</code>, a pointer to
67          an <code>&lt;NSP&gt;oint</code> is recognized as an
68          instsance of <code>NS:NS-POINT</code>, and a pointer to
69          an <code>&lt;NSR&gt;ange</code> is recognized as an
70          instance of <code>NS:NS-RANGE</code>.</para>
71
72        <para>A few more obscure structure types also support this
73        mechanism, and it's possible that a future version will
74        support user definition of similar type mappings.</para>
75
76        <para>This support for foreign types as classes provides the
77        following conveniences for each supported type:</para>
78
79      <itemizedlist>
80        <listitem>
81          <para>a <code>PRINT-OBJECT</code> method is defined</para>
82        </listitem>
83        <listitem>
84          <para>a foreign type name is created and treated as an alias
85          for the corresponding type. As an example, the
86          name <code>:NS-RECT</code> is a name for the type that
87          corresponds to <code>NS:NS-RECT</code>, and you can
88          use <code>:NS-RECT</code> as a type designator
89          in <link linkend="anchor_rlet"><code>RLET</code></link> forms to
90          specify a structure of type <code>NS-RECT</code>.</para>
91        </listitem>
92        <listitem>
93          <para>the class is integrated into the type system so that
94            <code>(TYPEP R 'NS:NS-RECT)</code> is implemented with
95            fair efficiency.</para>
96        </listitem>
97        <listitem>
98          <para>inlined accessor and <code>SETF</code> inverses are
99            defined for the structure type's fields.  In the case of
100            an <code>&lt;NSR*gt;ect</code>, for example, the fields in
101            question are the fields of the embedded point and size, so
102            that <code>NS:NS-RECT-X</code>, <code>NS:NS-RECT-Y</code>, <code>NS:NS-RECT-WIDTH</code>,
103            <code>NS-RECT-HEIGHT</code> and <code>SETF</code> inverses
104            are defined.  The accessors and setter functions typecheck
105            their arguments and the setters handle coercion to the
106            approprate type of <code>CGFLOAT</code> where
107            applicable.</para>
108        </listitem>
109        <listitem>
110          <para>an initialization function is defined; for
111            example,</para> 
112
113          <programlisting>
114(NS:INIT-NS-SIZE s w h)
115          </programlisting>
116
117          <para>is roughly equivalent to</para>
118
119          <programlisting>
120(SETF (NS:NS-SIZE-WIDTH s) w
121      (NS:NS-SIZE-HEIGHT s) h)
122          </programlisting>
123
124          <para>but might be a little more efficient.</para>
125        </listitem>
126        <listitem>
127          <para>a creation function is defined; for
128            example</para>
129
130          <programlisting>
131(NS:NS-MAKE-POINT x y)
132          </programlisting> 
133
134          <para>is functionally equivalent to</para>
135
136          <programlisting>
137(LET ((P (MAKE-GCABLE-RECORD :NS-POINT)))
138  (NS:INIT-NS-POINT P X Y)
139  p)
140          </programlisting>
141
142        </listitem>
143        <listitem>
144          <para>a macro is defined which, like <code>RLET</code>,
145            stack-allocates an instance of the foreign record type,
146            optionally iniitializes that instance, and executes a body
147            of code with a variable bound to that instance.</para>
148
149          <para>For example,</para>
150          <programlisting>
151(ns:with-ns-range (r loc len)
152  (format t "~&amp; range has location ~s, length ~s"
153     (ns:ns-range-location r) (ns:ns-range-length r)))
154          </programlisting>
155        </listitem>
156        <listitem>
157          <para></para>
158        </listitem>
159      </itemizedlist>
160      </sect3>
161
162    </sect2>
163
164    <sect2 id="Syntax-of-Foreign-Type-Specifiers">
165      <title>Syntax of Foreign Type Specifiers</title>
166      <itemizedlist>
167        <listitem>
168          <para>Some foreign types are builtin: keywords denote
169            primitive,builtin types such as the IEEE-double-float type
170            (denoted:DOUBLE-FLOAT), in much the same way as certain
171            symbols(CONS, FIXNUM,etc.) define primitive CL
172            types.</para>
173        </listitem>
174        <listitem>
175          <para>Constructors such as :SIGNED and :UNSIGNED can be
176            used to denotesigned and unsigned integer subtypes
177            (analogous to the CL typespecifiers SIGNED-BYTE and
178            UNSIGNED-BYTE.) :SIGNED is shorthand for(:SIGNED 32) and
179            :UNSIGNED is shorthand for (:UNSIGNED 32).</para>
180        </listitem>
181        <listitem>
182          <para>Aliases for other (perhaps more complicated) types
183            can bedefined via CCL:DEF-FOREIGN-TYPE (sort of like
184            CL:DEFTYPE or the Ctypedef facility). The type :CHAR is
185            defined as an alias for (:SIGNED8) on some platforms, as
186            (:UNSIGNED 8) on others.</para>
187            </listitem>
188        <listitem>
189              <para>The construct (:STRUCT <emphasis>name</emphasis>)
190                can be used torefer to a named structure type; (:UNION
191                <emphasis>name</emphasis>)can be used to refer to a named
192                union type. It isn't necessary toenumerate a structure or
193                union type's fields in order to refer tothe type.</para>
194            </listitem>
195        <listitem>
196              <para>If <emphasis>X</emphasis> is a valid foreign type
197                reference,then (:* <emphasis>X</emphasis>) denotes the
198                foreign type "pointerto<emphasis> X</emphasis>". By
199                convention, (:* T) denotes ananonymous pointer type,
200                vaguely equivalent to "void*" in C.</para>
201            </listitem>
202        <listitem>
203              <para>If a fieldlist is a list of lists, each of whose CAR
204                is a foreign field name (keyword) and whose CADR is a
205                foreign type specifier, then (:STRUCT
206                <emphasis>name</emphasis> ,@fieldlist) is adefinition of
207                the structure type <emphasis>name</emphasis>,
208                and (:UNION<emphasis> name</emphasis> ,@fieldlist) is a
209                definition of theunion type
210                <emphasis>name</emphasis>. Note that it's necessary
211                todefine a structure or union type in order to include
212                that type in a structure, union, or array, but only
213                necessary to "refer to" a strucure or union type in order
214                to define a type alias or a pointer type.</para>
215            </listitem>
216            <listitem>
217              <para>If <emphasis>X</emphasis> is a defined foreign type
218                , then (:array <emphasis>X</emphasis> &amp;rest dims)
219                denotes the foreigntype "array of
220                <emphasis>X</emphasis>". Although multiplearray dimensions
221                are allowed by the :array constructor,
222                only single-dimensioned arrays are (at all) well-supported
223                in &CCL;.</para>
224            </listitem>
225      </itemizedlist>
226    </sect2>
227  </sect1>
228
229  <!-- ******************************************  -->
230  <sect1 id="Foreign-Function-Calls">
231    <title>Foreign Function Calls</title>
232
233    <sect2 id="Overview-foreign-calls">
234          <title>Overview</title>
235      <para>&CCL; provides a number of constructs for calling
236        foreign functions from Lisp code (all of them based on the
237        function CCL:%FF-CALL).  In many cases, &CCL;'s interface
238        translator (see ) provides information about the foreign
239        function's entrypoint name and argument and return types; this
240        enables the use of the #_ reader macro (described below),
241        which may be more concise and/or more readable than other
242        constructs.</para>
243      <para>&CCL; also provides a mechanism for defining
244        <emphasis>callbacks</emphasis>: lisp functions which can be
245        called from foreign code.</para>
246      <para>There's no supported way to directly pass lisp data to
247        foreign functions: scalar lisp data must be coerced to an
248        equivalent foreign representatation, and lisp arrays (notably
249        strings) must be copied to non-GCed memory.</para>
250
251      <sect3 id="Type-Designators-for-Arguments-and-Return-Values">
252            <title>Type Designators for Arguments and Return Values</title>
253        <para>The types of foreign argument and return values in foreign
254              function calls and callbacks can be specified by any of the following
255          keywords:</para>
256            <variablelist>
257              <varlistentry>
258                <term>:UNSIGNED-BYTE</term>
259
260                <listitem>
261                      <para>The argument/return value is of type (UNSIGNED-BYTE 8)</para>
262                </listitem>
263              </varlistentry>
264
265              <varlistentry>
266                <term>:SIGNED-BYTE</term>
267
268                <listitem>
269                      <para>The argument/return value is of type (SIGNED-BYTE 8)</para>
270                </listitem>
271              </varlistentry>
272
273              <varlistentry>
274                <term>:UNSIGNED-HALFWORD</term>
275
276                <listitem>
277                      <para>The argument/return value is of type (UNSIGNED-BYTE 16)</para>
278                </listitem>
279              </varlistentry>
280
281              <varlistentry>
282                <term>:SIGNED-HALFWORD</term>
283
284                <listitem>
285                      <para>The argument/return value is of type (SIGNED-BYTE 16)</para>
286                </listitem>
287              </varlistentry>
288
289              <varlistentry>
290                <term>:UNSIGNED-FULLWORD</term>
291
292                <listitem>
293                      <para>The argument/return value is of type (UNSIGNED-BYTE 32)</para>
294                </listitem>
295              </varlistentry>
296
297              <varlistentry>
298                <term>:SIGNED-FULLWORD</term>
299
300                <listitem>
301                      <para>The argument/return value is of type (SIGNED-BYTE 32)</para>
302                </listitem>
303              </varlistentry>
304
305              <varlistentry>
306                <term>:UNSIGNED-DOUBLEWORD</term>
307
308                <listitem>
309                      <para>The argument/return value is of type (UNSIGNED-BYTE 64)</para>
310                </listitem>
311              </varlistentry>
312
313              <varlistentry>
314                <term>:SIGNED-DOUBLEWORD</term>
315
316                <listitem>
317                      <para>The argument/return value is of type (SIGNED-BYTE 64)</para>
318                </listitem>
319              </varlistentry>
320
321              <varlistentry>
322                <term>:SINGLE-FLOAT</term>
323
324                <listitem>
325                      <para>The argument/return value is of type SINGLE-FLOAT</para>
326                </listitem>
327              </varlistentry>
328
329              <varlistentry>
330                <term>:DOUBLE-FLOAT</term>
331
332                <listitem>
333                      <para>The argument/return value is of type DOUBLE-FLOAT</para>
334                </listitem>
335              </varlistentry>
336
337              <varlistentry>
338                <term>:ADDRESS</term>
339
340                <listitem>
341                      <para>The argument/return values
342                        is <link linkend="Referencing-and-Using-Foreign-Memory-Addresses">a MACPTR</link>.</para>
343                </listitem>
344              </varlistentry>
345
346              <varlistentry>
347                <term>:VOID</term>
348
349                <listitem>
350                      <para>or NIL Not valid as an argument type specifier; specifies
351                        that there is no meaningful return value</para>
352                </listitem>
353              </varlistentry>
354            </variablelist>
355
356        <para>On some platforms, a small positive integer
357          <emphasis>N</emphasis> can also be used as an argument
358          specifier; it indicates that the corresponding argument is a
359          pointer to an <emphasis>N</emphasis>-word structure or union
360          which should be passed by value to the foreign
361          function.  Exactly which foreign structures are passed
362              by value and how is very dependent on the Application
363              Binary Interface (ABI) of the platform; unless you're
364              very familar with ABI detatils (some of which are quite
365              baroque), it's often easier to let higher-level constructs
366              deal with these details.</para>
367      </sect3>
368
369      <sect3 id="External-Entrypoints-and-Named-External-Entrypoints">
370            <title>External Entrypoints and Named External Entrypoints</title>
371        <para>PowerPC machine instructions are always aligned on
372          32-bit boundaries, so the two least significant bits of the
373          first instruction ("entrypoint") of a foreign function are
374          always 0. &CCL; often represents an entrypoint address as
375          a fixnum that's binary-equivalent to the entrypoint address:
376          if<emphasis> E</emphasis> is an entrypoint address expressed
377          as a signed 32-bit integer, then (ash <emphasis>E</emphasis>
378          -2) is an equivalent fixnum representation of that
379          address. An entrypoint address can also be encapsulated in a
380          MACPTR (see FIXTHIS), but that's somewhat less efficient.</para>
381        <para>Although it's possible to use fixnums or macptrs to
382          represent entrypoint addresses, it's somewhat cumbersome to
383          do so. &CCL; can cache the addresses of named external
384          functions in structure-like objects of type
385          CCL:EXTERNAL-ENTRY-POINT (sometimes abbreviated as EEP).
386          Through the use of LOAD-TIME-VALUE, compiled lisp functions
387          are able to reference EEPs as constants; the use of an
388          indirection allows &CCL; runtime system to ensure that the
389          EEP's address is current and correct.</para>
390      </sect3>
391    </sect2>
392
393    <sect2 id="Return-Conventions-for-C-Structures">
394          <title>Return Conventions for C Structures</title>
395      <para> On some platforms, C functions that are defined to
396        return structures do so by reference: they actually
397        accept a first parameter of type "pointer to returned
398        struct/union" - which must be allocated by the caller - and
399        don't return a meaningful value.</para>
400          <para><emphasis>Exactly</emphasis> how a C function that's
401            defined to return a foreign structure does so is dependent on
402            the ABI (and on the size ad composition of the structure/union
403            in many cases.)</para>
404    </sect2>
405  </sect1>
406
407  <!-- ******************************************  -->
408  <sect1 id="Referencing-and-Using-Foreign-Memory-Addresses">
409    <title>Referencing and Using Foreign Memory Addresses</title>
410
411    <sect2 id="Overview-memory-addresses">
412      <title>Overview</title>
413
414      <sect3 id="Basics">
415            <title>Basics</title>
416        <para>For a variety of technical reasons, it isn't generally
417          possible to directly reference arbitrary absolute addresses
418          (such as those returned by the C library function malloc(),
419          for instance) in &CCL;. In &CCL; (and in MCL), such
420          addresses need to be <emphasis>encapsulated</emphasis> in
421          objects of type CCL:MACPTR; one can think of a MACPTR as
422          being a specialized type of structure whose sole purpose is
423          to provide a way of referring to an underlying "raw"
424          address.</para>
425        <para>It's sometimes convenient to blur the distinction
426          between a MACPTR and the address it represents; it's
427          sometimes necessary to maintain that distiction. It's
428          important to remember that a MACPTR is (generally) a
429          first-class Lisp object in the same sense that a CONS cell
430          is: it'll get GCed when it's no longer possible to reference
431          it. The "lifetime" of a MACPTR doesn't generally have
432          anything to do with the lifetime of the block of memory its
433          address points to.</para>
434        <para>It might be tempting to ask "How does one obtain the
435          address encapsulated by a MACPTR ?". The answer to that
436          question is that one doesn't do that (and there's no way to
437          do that): addresses aren't first-class objects, and there's
438          no way to refer to one.</para>
439        <para>Two MACPTRs that encapsulate the same address are EQL
440          to each other.</para>
441        <para>There are a small number of ways to directly create a
442          MACPTR (and there's a fair amount of syntactic sugar built
443          on top of of those primitives.) These primitives will be
444          discussed in greater detail below, but they include:</para>
445
446            <itemizedlist>
447          <listitem>
448                <para>Creating a MACPTR with a specified address, usually
449                  via thefunction CCL:%INT-TO-PTR.</para>
450              </listitem>
451          <listitem>
452                <para>Referencing the return valueof a foreign function
453                  call (see )that's specified to return an address.</para>
454              </listitem>
455          <listitem>
456                <para>Referencing a memory location that's specified to
457                  contain an address.</para>
458              </listitem>
459            </itemizedlist>
460
461        <para>All of these primitive MACPTR-creating operations are
462          usually open-coded by the compiler; it has a fairly good
463          notion of what low-level operations "produce" MACPTRs and
464          which operations "consume" the addresses that the
465          encapsulate, and will usually optimize out the introduction
466          of intermediate MACPTRs in a simple expression.</para>
467        <para>One consequence of the use of MACPTR objects to
468          encapsulate foreign addresses is that (naively)
469          <emphasis>every reference to a foreign address causes a
470            MACPTR to be allocated.</emphasis></para>
471        <para>Consider a code fragment like the following:</para>
472        <programlisting>
473(defun get-next-event ()
474  "get the next event from a hypothetical window system"
475  (loop
476     (let* ((event (#_get_next_window_system_event))) ; via an FF-CALL
477       (unless (null-event-p event)
478         (handle-event event)))))
479        </programlisting>
480        <para>As this is written, each call to the (hypothetical)
481          foreign function #_get_next_window_system_event will return
482          a new MACPTR object.  Ignoring for the sake of argument the
483          question of whether this code fragment exhibits a good way
484          to poll for external events (it doesn't), it's not hard to
485          imagine that this loop could execute several millon times
486          per second (producing several million MACPTRs per second.)
487          Clearly, the "naive" approach is impractical in many
488          cases.</para>
489      </sect3>
490
491      <sect3 id="Stack-allocation-of---and-destructive-operations-on---MACPTRs-">
492        <title>Stack allocation of&mdash;and destructive operations on&mdash;MACPTRs.</title>
493            <para>If certain conditions held in the environment in which
494              GET-NEXT-EVENT ran&mdash;namely, if it was guaranteed that
495              neither NULL-EVENT-P nor HANDLE-EVENT cached or otherwise
496              retained their arguments (the "event" pointer)&mdash;there'd be
497              a few alternatives to the naive approach. One of those
498              approaches would be to use the primitive function
499              %SETF-MACPTR (described in greater detail below) to
500              destructively modify a MACPTR (to change the value of the
501              address it encapsulates.) The GET-NEXT-EVENT example could
502              be re-written as:</para>
503        <programlisting>
504(defun get-next-event ()
505  (let* ((event (%int-to-ptr 0)))     ; create a MACPTR with address 0
506    (loop
507       (%setf-macptr event (#_get_next_window_system_event)) ; re-use it
508       (unless (null-event-p event)
509         (handle-event event)))))
510        </programlisting>
511        <para>That version's a bit more realistic: it allocates a
512          single MACPTR outside if the loop, then changes its address
513          to point to the current address of the hypothetical event
514          structure on each loop iteration. If there are a million
515          loop iterations per call to GET-NEXT-EVENT, we're allocating
516          a million times fewer MACPTRs per call; that sounds like a
517          Good Thing.</para>
518        <para>An Even Better Thing would be to advise the compiler
519          that the initial value (the null MACPTR) bound to the
520          variable event has dynamic extent (that value won't be
521          referenced once control leaves the extent of the binding of
522          that variable.) Common Lisp allows us to make such an
523          assertion via a DYNAMIC-EXTENT declaration; &CCL;'s
524          compiler can recognize the "primitive MACPTR-creating
525          operation" involved and can replace it with an equivalent
526          operation that stack-allocates the MACPTR object. If we're
527          not worried about the cost of allocating that MACPTR on
528          every iteration (the cost is small and there's no hidden GC
529          cost), we could move the binding back inside the
530          loop:</para>
531        <programlisting>
532(defun get-next-event ()
533  (loop
534     (let* ((event (%null-ptr))) ; (%NULL-PTR) is shorthand for (%INT-TO-PTR 0)
535       (declare (dynamic-extent event))
536       (%setf-macptr event (#_get_next_window_system_event))
537       (unless (null-event-p event)
538         (handle-event event)))))
539        </programlisting>
540        <para>The idiom of binding one or more variables to
541          stack-allocated MACPTRs, then destructively modifying those
542          MACPTRs before executing a body of code is common enough
543          that &CCL; provides a macro (WITH-MACPTRS) that handles
544          all of the gory details. The following version of
545          GET-NEXT-EVENT is semantically equivalent to the previous
546          version, but hopefully a bit more concise:</para>
547        <programlisting>
548(defun get-next-event ()
549  (loop
550     (with-macptrs ((event (#_get_next_window_system_event)))
551       (unless (null-event-p event)
552         (handle-event event)))))
553        </programlisting>
554      </sect3>
555
556      <sect3 id="Stack-allocated-memory--and-stack-allocated-pointers-to-it--">
557        <title>Stack-allocated memory (and stack-allocated pointers to it.)</title>
558            <para>Fairly often, the blocks of foreign memory (obtained
559              by malloc or something similar) have well-defined lifetimes
560              (they can safely be freed at some point when it's known that
561              they're no longer needed and it's known that they're no
562              longer referenced.) A common idiom might be:</para>
563        <programlisting>
564(with-macptrs (p (#_allocate_foreign_memory size))
565  (unwind-protect
566       (use-foreign-memory p)
567    (#_deallocate_foreign_memory p)))
568        </programlisting>
569        <para>That's not unreasonable code, but it's fairly
570          expensive for a number of reasons: foreign functions calls
571          are themselves fairly expensive (as is UNWIND-PROTECT), and
572          most library routines for allocating and deallocating
573          foreign memory (things like malloc and free) can be fairly
574          expensive in their own right.</para>
575        <para>In the idiomatic code above, both the MACPTR P and the
576          block of memory that's being allocated and freed have
577          dynamic extent and are therefore good candidates for stack
578          allocation. &CCL; provides the %STACK-BLOCK macro, which
579          executes a body of code with one or more variables bound to
580          stack-allocated MACPTRs which encapsulate the addresses of
581          stack-allocated blocks of foreign memory. Using
582          %STACK-BLOCK, the idiomatic code is:</para>
583        <programlisting>
584(%stack-block ((p size))
585              (use-foreign-memory p))
586        </programlisting>
587        <para>which is a bit more efficient and a bit more concise
588          than the version presented earlier.</para>
589        <para>%STACK-BLOCK is used as the basis for slightly
590          higher-level things like RLET. (See FIXTHIS for more information
591          about RLET.)</para>
592      </sect3>
593
594      <sect3 id="Caveats-">
595            <title>Caveats</title>
596        <para>Reading from, writing to, allocating, and freeing
597          foreign memory are all potentially dangerous operations;
598          this is no less true when these operations are performed in
599          &CCL; than when they're done in C or some other
600          lower-level language. In addition, destructive operations on
601          Lisp objects be dangerous, as can stack allocation if it's
602          abused (if DYNAMIC-EXTENT declarations are violated.)
603          Correct use of the constructs and primitives described here
604          is reliable and safe; slightly incorrect use of these
605          constructs and primitives can crash &CCL;.</para>
606      </sect3>
607    </sect2>
608
609    <sect2 id="Foreign-Memory-Addresses-Dictionary">
610          <title>Foreign-Memory-Addresses Dictionary</title>
611      <para>Unless otherwise noted, all of the symbols mentioned
612        below are exported from the CCL package.</para>
613
614      <sect3 id="Scalar-memory-reference">
615        <title>Scalar memory reference</title>
616            <variablelist>
617              <varlistentry>
618                <term>Syntax</term>
619
620                <listitem>
621                      <para>%get-signed-byte ptr &#38;optional (offset 0)</para>
622
623                      <para>%get-unsigned-byte ptr &#38;optional (offset 0)</para>
624
625                      <para>%get-signed-word ptr &#38;optional (offset 0)</para>
626
627                      <para>%get-unsigned-word ptr &#38;optional (offset 0)</para>
628
629                      <para>%get-signed-long ptr &#38;optional (offset 0)</para>
630
631                      <para>%get-unsigned-long ptr &#38;optional (offset 0)</para>
632
633                      <para>%%get-signed-longlong ptr &#38;optional (offset 0)</para>
634
635                      <para>%%get-unsigned-longlong ptr &#38;optional (offset 0)</para>
636
637                      <para>%get-ptr ptr &#38;optional (offset 0)</para>
638
639                      <para>%get-single-float ptr &#38;optional (offset 0)</para>
640
641                      <para>%get-double-float ptr &#38;optional (offset 0)</para>
642                </listitem>
643              </varlistentry>
644              <varlistentry>
645                <term>Description</term>
646
647                <listitem>
648                      <para>References and returns the signed or unsigned 8-bit byte,
649                        signed or unsigned 16-bit word, signed or unsigned 32-bit long
650                        word, signed or unsigned 64-bit long long word, 32-bit address,
651                        32-bit single-float, or 64-bit double-float at the effective byte
652                        address formed by adding offset to the address encapsulated by
653                        ptr.</para>
654                </listitem>
655              </varlistentry>
656
657              <varlistentry>
658                <term>Arguments</term>
659
660                <listitem>
661
662                      <variablelist>
663                        <varlistentry>
664                          <term>ptr</term>
665
666                          <listitem>
667                            <para>A MACPTR</para>
668                          </listitem>
669                        </varlistentry>
670
671                        <varlistentry>
672                          <term>offset</term>
673
674                          <listitem>
675                            <para>A fixnum</para>
676                          </listitem>
677                        </varlistentry>
678                      </variablelist>
679                </listitem>
680              </varlistentry>
681            </variablelist>
682
683        <para>All of the memory reference primitives described above can be</para>
684        <para>used with SETF.</para>
685      </sect3>
686
687      <sect3 id="iget-bit--Function-">
688            <title>%get-bit [Function]</title>
689
690            <variablelist>
691              <varlistentry>
692                <term>Syntax</term>
693
694                <listitem>
695                      <para>%get-bit ptr bit-offset</para>
696                </listitem>
697              </varlistentry>
698
699              <varlistentry>
700                <term>Description</term>
701
702                <listitem>
703                      <para>References and returns the bit-offsetth bit at the address
704                        encapsulated by ptr. (Bit 0 at a given address is the most
705                        significant bit of the byte at that address.) Can be used with
706                        SETF.</para>
707                </listitem>
708              </varlistentry>
709
710              <varlistentry>
711                <term>Arguments</term>
712
713                <listitem>
714                      <para>&#x00A0;</para>
715
716                      <variablelist>
717                        <varlistentry>
718                          <term>ptr</term>
719
720                          <listitem>
721                            <para>A MACPTR</para>
722                          </listitem>
723                        </varlistentry>
724
725                        <varlistentry>
726                          <term>bit-offset</term>
727
728                          <listitem>
729                            <para>A fixnum</para>
730                          </listitem>
731                        </varlistentry>
732                      </variablelist>
733                </listitem>
734              </varlistentry>
735            </variablelist>
736      </sect3>
737
738      <sect3 id="iget-bitfield--Function-">
739        <title>%get-bitfield [Function]</title>
740            <variablelist>
741              <varlistentry>
742                <term>Syntax</term>
743
744                <listitem>
745                      <para>%get-bitfield ptr bit-offset width</para>
746                </listitem>
747              </varlistentry>
748
749              <varlistentry>
750                <term>Description</term>
751
752                <listitem>
753                      <para>References and returns an unsigned integer composed from the
754                        width bits found bit-offsetbits from the address encapsulated by
755                        ptr. (The least significant bit of the result is the value of
756                        (%get-bit ptr (1- (+ bit-offset width))). Can be used with SETF.</para>
757                </listitem>
758              </varlistentry>
759
760              <varlistentry>
761                <term>Arguments</term>
762
763                <listitem>
764                      <para>&#x00A0;</para>
765                </listitem>
766              </varlistentry>
767
768              <varlistentry>
769                <term>ptr</term>
770
771                <listitem>
772                      <para>A MACPTR</para>
773
774                      <variablelist>
775                        <varlistentry>
776                          <term>bit-offset</term>
777
778                          <listitem>
779                            <para>A fixnum</para>
780                          </listitem>
781                        </varlistentry>
782
783                        <varlistentry>
784                          <term>width</term>
785
786                          <listitem>
787                            <para>A positive fixnum</para>
788                          </listitem>
789                        </varlistentry>
790                      </variablelist>
791                </listitem>
792              </varlistentry>
793            </variablelist>
794      </sect3>
795
796          <sect3>
797            <title>%int-to-ptr [Function]</title>
798
799            <variablelist>
800              <varlistentry>
801                <term>Syntax</term>
802
803                <listitem>
804                      <para>%int-to-ptr int</para>
805                </listitem>
806              </varlistentry>
807
808              <varlistentry>
809                <term>Description</term>
810
811                <listitem>
812                      <para>Creates and returns a MACPTR whose address matches int.</para>
813                </listitem>
814              </varlistentry>
815
816              <varlistentry>
817                <term>Arguments</term>
818
819                <listitem>
820                      <para>&#x00A0;</para>
821
822                      <variablelist>
823                        <varlistentry>
824                          <term>int</term>
825
826                          <listitem>
827                            <para>An (unsigned-byte 32)</para>
828                          </listitem>
829                        </varlistentry>
830                      </variablelist>
831                </listitem>
832              </varlistentry>
833            </variablelist>
834          </sect3>
835
836          <sect3>
837            <title>%inc-ptr [Function]</title>
838
839            <variablelist>
840              <varlistentry>
841                <term>Syntax</term>
842
843                <listitem>
844                      <para>%inc-ptr ptr &#38;optional (delta 1)</para>
845                </listitem>
846              </varlistentry>
847
848              <varlistentry>
849                <term>Description</term>
850
851                <listitem>
852                      <para>Creates and returns a MACPTR whose address is the address of
853                        ptr plus delta. The idiom (%inc-ptr ptr 0) is sometimes used to
854                        copy a MACPTR, e.g., to create a new MACPTR encapsulating the same
855                        address as ptr.</para>
856                </listitem>
857              </varlistentry>
858
859              <varlistentry>
860                <term>Arguments</term>
861
862                <listitem>
863                      <para>&#x00A0;</para>
864
865                      <variablelist>
866                        <varlistentry>
867                          <term>ptr</term>
868
869                          <listitem>
870                            <para>A MACPTR</para>
871                          </listitem>
872                        </varlistentry>
873
874                        <varlistentry>
875                          <term>delta</term>
876
877                          <listitem>
878                            <para>A fixnum</para>
879                          </listitem>
880                        </varlistentry>
881                      </variablelist>
882                </listitem>
883              </varlistentry>
884            </variablelist>
885          </sect3>
886
887          <sect3>
888            <title>%ptr-to-int [Function]</title>
889
890            <variablelist>
891              <varlistentry>
892                <term>Syntax</term>
893
894                <listitem>
895                      <para>%ptr-to-int ptr</para>
896                </listitem>
897              </varlistentry>
898
899              <varlistentry>
900                <term>Description</term>
901
902                <listitem>
903                      <para>Returns the address encapsulated by ptr, as an
904                        (unsigned-byte 32).</para>
905                </listitem>
906              </varlistentry>
907
908              <varlistentry>
909                <term>Arguments</term>
910
911                <listitem>
912                      <para>&#x00A0;</para>
913
914                      <variablelist>
915                        <varlistentry>
916                          <term>ptr</term>
917
918                          <listitem>
919                            <para>A MACPTR</para>
920                          </listitem>
921                        </varlistentry>
922                      </variablelist>
923                </listitem>
924              </varlistentry>
925            </variablelist>
926          </sect3>
927
928          <sect3>
929            <title>%null-ptr [Macro]</title>
930
931            <variablelist>
932              <varlistentry>
933                <term>Syntax</term>
934
935                <listitem>
936                      <para>%null-ptr</para>
937                </listitem>
938              </varlistentry>
939
940              <varlistentry>
941                <term>Description</term>
942
943                <listitem>
944                      <para>Equivalent to (%ptr-to-int 0).</para>
945                </listitem>
946              </varlistentry>
947            </variablelist>
948          </sect3>
949
950          <sect3>
951            <title>%null-ptr-p [Function]</title>
952
953            <variablelist>
954              <varlistentry>
955                <term>Syntax</term>
956
957                <listitem>
958                      <para>%null-ptr-p ptr</para>
959                </listitem>
960              </varlistentry>
961
962              <varlistentry>
963                <term>Description</term>
964
965                <listitem>
966                      <para>Returns T If ptr is a MACPTR encapsulating the address 0,
967                        NIL if ptr encapsulates some other address.</para>
968                </listitem>
969              </varlistentry>
970
971              <varlistentry>
972                <term>Arguments</term>
973
974                <listitem>
975                      <para>&#x00A0;</para>
976
977                      <variablelist>
978                        <varlistentry>
979                          <term>ptr</term>
980
981                          <listitem>
982                            <para>A MACPTR</para>
983                          </listitem>
984                        </varlistentry>
985                      </variablelist>
986                </listitem>
987              </varlistentry>
988            </variablelist>
989          </sect3>
990
991          <sect3>
992            <title>%setf-macptr [Function]</title>
993
994            <variablelist>
995              <varlistentry>
996                <term>Syntax</term>
997
998                <listitem>
999                      <para>%setf-macptr dest-ptr src-ptr</para>
1000                </listitem>
1001              </varlistentry>
1002
1003              <varlistentry>
1004                <term>Description</term>
1005
1006                <listitem>
1007                      <para>Causes dest-ptr to encapsulate the same address that src-ptr
1008                        does, then returns dest-ptr.</para>
1009                </listitem>
1010              </varlistentry>
1011
1012              <varlistentry>
1013                <term>Arguments</term>
1014
1015                <listitem>
1016                      <para>&#x00A0;</para>
1017
1018                      <variablelist>
1019                        <varlistentry>
1020                          <term>dest-ptr</term>
1021
1022                          <listitem>
1023                            <para>A MACPTR</para>
1024                          </listitem>
1025                        </varlistentry>
1026
1027                        <varlistentry>
1028                          <term>src-ptr</term>
1029
1030                          <listitem>
1031                            <para>A MACPTR</para>
1032                          </listitem>
1033                        </varlistentry>
1034                      </variablelist>
1035                </listitem>
1036              </varlistentry>
1037            </variablelist>
1038          </sect3>
1039
1040          <sect3>
1041            <title>%incf-ptr [Macro]</title>
1042
1043            <variablelist>
1044              <varlistentry>
1045                <term>Syntax</term>
1046
1047                <listitem>
1048                      <para>%incf-ptr ptr &#38;optional (delta 1)</para>
1049                </listitem>
1050              </varlistentry>
1051
1052              <varlistentry>
1053                <term>Description</term>
1054
1055                <listitem>
1056                      <para>Destructively modifies ptr, by adding delta to the address
1057                        it encapsulates. Returns ptr.</para>
1058                </listitem>
1059              </varlistentry>
1060
1061              <varlistentry>
1062                <term>Arguments</term>
1063
1064                <listitem>
1065                      <para>&#x00A0;</para>
1066
1067                      <variablelist>
1068                        <varlistentry>
1069                          <term>ptr</term>
1070
1071                          <listitem>
1072                            <para>A MACPTR</para>
1073                          </listitem>
1074                        </varlistentry>
1075
1076                        <varlistentry>
1077                          <term>delta</term>
1078
1079                          <listitem>
1080                            <para>A fixnum</para>
1081                          </listitem>
1082                        </varlistentry>
1083                      </variablelist>
1084                </listitem>
1085              </varlistentry>
1086            </variablelist>
1087          </sect3>
1088
1089          <sect3>
1090            <title>with-macptrs [Macro]</title>
1091
1092            <variablelist>
1093              <varlistentry>
1094                <term>Syntax</term>
1095
1096                <listitem>
1097                      <para>with-macptrs (var expr)* &#38;body body</para>
1098                </listitem>
1099              </varlistentry>
1100
1101              <varlistentry>
1102                <term>Description</term>
1103
1104                <listitem>
1105                      <para>Executes body in an environment in which each var is bound
1106                        to a stack-allocated macptr which encapsulates the foreign address
1107                        yielded by the corresponding expr. Returns whatever value(s) body
1108                        returns.</para>
1109                </listitem>
1110              </varlistentry>
1111
1112              <varlistentry>
1113                <term>Arguments</term>
1114
1115                <listitem>
1116                      <para>&#x00A0;</para>
1117
1118                      <variablelist>
1119                        <varlistentry>
1120                          <term>var</term>
1121
1122                          <listitem>
1123                            <para>A symbol (variable name)</para>
1124                          </listitem>
1125                        </varlistentry>
1126
1127                        <varlistentry>
1128                          <term>expr</term>
1129
1130                          <listitem>
1131                            <para>A MACPTR-valued expression</para>
1132                          </listitem>
1133                        </varlistentry>
1134                      </variablelist>
1135                </listitem>
1136              </varlistentry>
1137            </variablelist>
1138          </sect3>
1139
1140          <sect3>
1141            <title>%stack-block [Macro]</title>
1142
1143            <variablelist>
1144              <varlistentry>
1145                <term>Syntax</term>
1146
1147                <listitem>
1148                      <para>%stack-block (var expr)* &#38;body body</para>
1149                </listitem>
1150              </varlistentry>
1151
1152              <varlistentry>
1153                <term>Description</term>
1154
1155                <listitem>
1156                      <para>Executes body in an environment in which each var is bound
1157                        to a stack-allocated macptr which encapsulates the address of a
1158                        stack-allocated region of size expr bytes. Returns whatever
1159                        value(s) body returns.</para>
1160                </listitem>
1161              </varlistentry>
1162
1163              <varlistentry>
1164                <term>Arguments</term>
1165
1166                <listitem>
1167                      <para>&#x00A0;</para>
1168
1169                      <variablelist>
1170                        <varlistentry>
1171                          <term>var</term>
1172
1173                          <listitem>
1174                            <para>A symbol (variable name)</para>
1175                          </listitem>
1176                        </varlistentry>
1177
1178                        <varlistentry>
1179                          <term>expr</term>
1180
1181                          <listitem>
1182                            <para>An expression which should evaluate to a non-negative
1183                              fixnum</para>
1184                          </listitem>
1185                        </varlistentry>
1186                      </variablelist>
1187                </listitem>
1188              </varlistentry>
1189            </variablelist>
1190          </sect3>
1191
1192          <sect3>
1193            <title>make-cstring [Function]</title>
1194
1195            <variablelist>
1196              <varlistentry>
1197                    <term>Syntax</term>
1198
1199                    <listitem>
1200                      <para>make-cstring string</para>
1201                    </listitem>
1202              </varlistentry>
1203
1204              <varlistentry>
1205                    <term>Description</term>
1206
1207                    <listitem>
1208                      <para>Allocates a block of memory (via malloc) of length (1+
1209                        (length string)). Copies the string to this block and appends a
1210                        trailing NUL byte; returns a MACPTR to the block.</para>
1211                    </listitem>
1212              </varlistentry>
1213
1214              <varlistentry>
1215                    <term>Arguments</term>
1216
1217                    <listitem>
1218                      <para>&#x00A0;</para>
1219
1220                      <variablelist>
1221                        <varlistentry>
1222                          <term>string</term>
1223
1224                          <listitem>
1225                                <para>A lisp string</para>
1226                          </listitem>
1227                        </varlistentry>
1228                      </variablelist>
1229                    </listitem>
1230              </varlistentry>
1231            </variablelist>
1232          </sect3>
1233
1234          <sect3>
1235            <title>with-cstrs [Macro]</title>
1236
1237            <variablelist>
1238              <varlistentry>
1239                <term>Syntax</term>
1240
1241                <listitem>
1242                      <para>with-cstrs (var string)* &#38;body body</para>
1243                </listitem>
1244              </varlistentry>
1245
1246              <varlistentry>
1247                <term>Description</term>
1248
1249                <listitem>
1250                      <para>Executes body in an environment in which each var is bound
1251                        to a stack-allocated macptr which encapsulates the %address of a
1252                        stack-allocated region of into which each string (and a trailing
1253                        NUL byte) has been copied. Returns whatever value(s) body returns.</para>
1254                </listitem>
1255              </varlistentry>
1256
1257              <varlistentry>
1258                <term>Arguments</term>
1259
1260                <listitem>
1261                      <para>&#x00A0;</para>
1262
1263                      <variablelist>
1264                        <varlistentry>
1265                          <term>var</term>
1266
1267                          <listitem>
1268                            <para>A symbol (variable name)</para>
1269                          </listitem>
1270                        </varlistentry>
1271
1272                        <varlistentry>
1273                          <term>string</term>
1274
1275                          <listitem>
1276                            <para>An expression which should evaluate to a lisp string</para>
1277                          </listitem>
1278                        </varlistentry>
1279                      </variablelist>
1280                </listitem>
1281              </varlistentry>
1282            </variablelist>
1283          </sect3>
1284
1285      <sect3>
1286        <title>with-encoded-cstrs [Macro]</title>
1287
1288        <variablelist>
1289              <varlistentry>
1290                <term>Syntax</term>
1291
1292                <listitem>
1293                      <para>with-encoded-cstrs ENCODING-NAME (varI stringI)* &#38;body body</para>
1294                </listitem>
1295              </varlistentry>
1296
1297              <varlistentry>
1298                <term>Description</term>
1299
1300                <listitem>
1301                      <para>Executes body in an environment in which each varI is
1302                        bound to a macptr which encapsulates the %address of a
1303                        stack-allocated region of into which each stringI (and a
1304                        trailing NUL character) has been copied. Returns whatever
1305                        value(s) body returns.</para>
1306
1307              <para>ENCODING-NAME is a keyword constant that names a
1308                character encoding. Each foreign string is encoded in the
1309                named encoding. Each foreign string has dynamic
1310                extent.</para>
1311
1312              <para>WITH-ENCODED-CSTRS does not automatically prepend
1313                bte-order marks to its output; the size of the terminating
1314                #\NUL character depends on the number of octets per code unit
1315                in the encoding.</para>
1316
1317              <para>The expression</para>
1318
1319              <programlisting>(ccl:with-cstrs ((x "x")) (#_puts x))</programlisting>
1320
1321              <para>is functionally equivalent to</para>
1322
1323              <programlisting>(ccl:with-encoded-cstrs :iso-8859-1 ((x "x")) (#_puts x))</programlisting>
1324                </listitem>
1325              </varlistentry>
1326
1327              <varlistentry>
1328                <term>Arguments</term>
1329
1330                <listitem>
1331                      <para>&#x00A0;</para>
1332
1333                      <variablelist>
1334                        <varlistentry>
1335                          <term>varI</term>
1336
1337                          <listitem>
1338                            <para>A symbol (variable name)</para>
1339                          </listitem>
1340                        </varlistentry>
1341
1342                        <varlistentry>
1343                          <term>stringI</term>
1344
1345                          <listitem>
1346                            <para>An expression which should evaluate to a lisp string</para>
1347                          </listitem>
1348                        </varlistentry>
1349                      </variablelist>
1350                </listitem>
1351              </varlistentry>
1352        </variablelist>
1353      </sect3>
1354
1355          <sect3>
1356            <title>%get-cstring [Function]</title>
1357
1358            <variablelist>
1359              <varlistentry>
1360                <term>Syntax</term>
1361
1362                <listitem>
1363                      <para>%get-cstring ptr</para>
1364                </listitem>
1365              </varlistentry>
1366
1367              <varlistentry>
1368                <term>Description</term>
1369
1370                <listitem>
1371                      <para>Interprets ptr as a pointer to a (NUL -terminated) C string;
1372                        returns an equivalent lisp string.</para>
1373                </listitem>
1374              </varlistentry>
1375
1376              <varlistentry>
1377                <term>Arguments</term>
1378
1379                <listitem>
1380                      <para></para>
1381
1382                      <variablelist>
1383                        <varlistentry>
1384                          <term>ptr</term>
1385
1386                          <listitem>
1387                            <para>A MACPTR</para>
1388                          </listitem>
1389                        </varlistentry>
1390                      </variablelist>
1391                </listitem>
1392              </varlistentry>
1393            </variablelist>
1394          </sect3>
1395
1396          <sect3>
1397            <title>%str-from-ptr [Function]</title>
1398
1399            <variablelist>
1400              <varlistentry>
1401                <term>Syntax</term>
1402
1403                <listitem>
1404                      <para>%str-from-ptr ptr length</para>
1405                </listitem>
1406              </varlistentry>
1407
1408              <varlistentry>
1409                <term>Description</term>
1410
1411                <listitem>
1412                      <para>Returns a lisp string of length <varname>length</varname>,
1413                        whose contents are initialized from the bytes at<varname> ptr.</varname>
1414                      </para>
1415                </listitem>
1416              </varlistentry>
1417
1418              <varlistentry>
1419                <term>Arguments</term>
1420
1421                <listitem>
1422                      <para><variablelist><varlistentry><term>ptr</term><listitem><para>A
1423                                MACPTR</para></listitem></varlistentry><varlistentry><term>length</term><listitem><para>a
1424                                non-negative fixnum</para></listitem></varlistentry></variablelist></para>
1425                </listitem>
1426              </varlistentry>
1427            </variablelist>
1428          </sect3>
1429    </sect2>
1430  </sect1>
1431
1432  <!-- ******************************************  -->
1433  <sect1 id="The-Interface-Database">
1434    <title>The Interface Database</title>
1435
1436    <sect2 id="interface-database-Overview">
1437          <title>Overview</title>
1438      <para>&CCL; uses a set of database files which contain
1439        foreign type, record, constant, and function definitions
1440        derived from the operating system's header files, be that
1441        Linux or Darwin.  An archive containing these database files
1442        (and the shell scripts which were used in their creation) is
1443        available; see the Distributions page for information about
1444        obtaining current interface database files.</para>
1445      <para>Not surprisingly, different platforms use different database files.</para>
1446      <para>&CCL; defines reader macros that consult these databases:</para>
1447          <itemizedlist>
1448        <listitem>
1449              <para>#$foo looks up the value of the constant definition of foo</para>
1450            </listitem>
1451            <listitem>
1452              <para>#_foo looks up the foreign function definition for foo</para>
1453            </listitem>
1454       
1455      </itemizedlist>
1456      <para>In both cases, the symbol foo is interned in the "OS"
1457        package. The #$ reader macro has the side-effect of defining
1458        foo as a constant (as if via DEFCONSTANT); the #_ reader macro
1459        has the side effect of defining foo as a macro which will
1460        expand into an (EXTERNAL-CALL form.)</para>
1461      <para>It's important to remember that the side-effect happens
1462        when the form containing the reader macro is
1463        read. Macroexpansion functions that expand into forms which
1464        contain instances of those reader macros don't do what one
1465        might think that they do, unless the macros are expanded in
1466        the same lisp session as the reader macro was read in.</para>
1467      <para>In addition, references to foreign type,
1468        structure/union, and field names (when used in the RREF/PREF
1469        and RLET macros) will cause these database files to be
1470        consulted.</para>
1471      <para>Since the &CCL; sources contain instances of these
1472        reader macros (and references to foreign record types and
1473        fields), compiling &CCL; from those sources depends on the
1474        ability to find and use (see <xref
1475                                        linkend="Building-the-heap-image"/>).</para>
1476    </sect2>
1477
1478    <sect2 id="Other-issues">
1479      <title>Other issues:</title>
1480      <itemizedlist>
1481        <listitem>
1482              <para>&CCL; now preserves the case of external symbols
1483                in itsdatabase files. See for information about case in
1484                foreign symbol names.</para>
1485            </listitem>
1486            <listitem>
1487              <para>The Linux databases are derived from a somewhat
1488                arbitrary set of Linux header files. Linux is enough of a
1489                moving target that it may be difficult to define a standard,
1490                reference set of interfaces from which to derive a standard,
1491                reference set of database files.This seems to be less of
1492                an issue with Darwin and FreeBSD.</para>
1493            </listitem>
1494          </itemizedlist>
1495      <para>For information about building the database files,
1496            see <xref linkend="The-Interface-Translator"/>.</para>
1497    </sect2>
1498  </sect1>
1499
1500  <!-- ******************************************  -->
1501  <sect1 id="Using-Interface-Directories">
1502    <title>Using Interface Directories</title>
1503
1504    <sect2 id="Interface-Directory-Overview">
1505          <title>Overview</title>
1506      <para>As distributed, the "ccl:headers;" (for LinuxPPC)
1507        directory is organized like:</para>
1508      <programlisting>
1509        headers/
1510        headers/gl/
1511        headers/gl/C/
1512        headers/gl/C/populate.sh
1513        headers/gl/constants.cdb
1514        headers/gl/functions.cdb
1515        headers/gl/records.cdb
1516        headers/gl/objc-classes.cdb
1517        headers/gl/objc-methods.cdb
1518        headers/gl/types.cdb
1519        headers/gnome/
1520        headers/gnome/C/
1521        headers/gnome/C/populate.sh
1522        headers/gnome/constants.cdb
1523        headers/gnome/functions.cb
1524        headers/gnome/records.cdb
1525        headers/gnome/objc-classes.cdb
1526        headers/gnome/objc-methods.cdb
1527        headers/gnome/types.cdb
1528        headers/gtk/
1529        headers/gtk/C/
1530        headers/gtk/C/populate.sh
1531        headers/gtk/constants.cdb
1532        headers/gtk/functions.cdb
1533        headers/gtk/records.cdb
1534        headers/gtk/objc-classes.cdb
1535        headers/gtk/objc-methods.cdb
1536        headers/gtk/types.cdb
1537        headers/libc/
1538        headers/libc/C/
1539        headers/libc/C/populate.sh
1540        headers/libc/constants.cdb
1541        headers/libc/functions.cdb
1542        headers/libc/records.cdb
1543        headers/libc/objc-classes.cdb
1544        headers/libc/objc-methods.cdb
1545        headers/libc/types.cdb
1546      </programlisting>
1547      <para>e.g, as a set of parallel subdirectories, each with a
1548        lowercase name and each of which contains a set of 6 database
1549        files and a "C" subdirectory which contains a shell script
1550        used in the database creation process.</para>
1551      <para>As one might assume, the database files in each of these
1552        subdirectories contain foreign type, constant, and function
1553        definitions - as well as Objective-C class and method info -that
1554        correspond (roughly) to the information contained in the
1555        header files associated with a "-dev" package in a Linux
1556        distribution.  "libc" corresponds pretty closely to the
1557        interfaces associated with "glibc/libc6" header files, "gl"
1558        corresponds to an "openGL+GLUT" developmnent package, "gtk"
1559        and "gnome" contain interface information from the GTK+1.2 and
1560        GNOME libraries, respectively.</para>
1561      <para>For Darwin, the "ccl:darwin-headers" directory contains
1562        a "libc" subdirectory, whose contents roughly correspond to
1563        those of "/usr/include" under Darwin, as well as
1564        subdirectories corresponding to the MacOSX Carbon and Cocoa
1565        frameworks.</para>
1566      <para>To see the precise set of .h files used to generate the
1567        database files in a given interface directory, consult the
1568        corresponding "populate.sh" shell script (in the interface
1569        directory's "C" subdirectory.)</para>
1570      <para>The intent is that this initial set can be augmented to
1571        meet local needs, and that this can be done in a fairly
1572        incremental fashion: one needn't have unrelated header files
1573        installed in order to generate interface databases for a
1574        package of interest.</para>
1575      <para>Hopefully, this scheme will also make it easier to
1576        distribute patches and bug fixes.</para>
1577      <para>&CCL; maintains a list of directories; when looking
1578        for a foreign type, constant, function, or record definition,
1579        it'll consult the database files in each directory on that
1580        list. Initially, the list contains an entry for the "libc"
1581        interface directory. &CCL; needs to be explicitly told to
1582        look in other interface directories should it need to do
1583        so.</para>
1584    </sect2>
1585
1586    <sect2 id="Creating-new-interface-directories">
1587          <title>Creating new interface directories</title>
1588      <para>This example refers to "ccl:headers;", which is
1589        appropriate for LinuxPPC. The procedure's analogous under
1590        Darwin, where the "ccl:darwin-headers;" directory would be
1591        used instead.</para>
1592      <para>To create a new interface directory, "foo", and a set of
1593        database files in that directory:</para>
1594          <orderedlist continuation="restarts" inheritnum="ignore">
1595            <listitem>
1596              <para>Create a subdirectory of &#34;ccl:headers;&#34; named
1597                &#34;foo&#34;.</para>
1598            </listitem>
1599
1600            <listitem>
1601              <para>Create a subdirectory of &#34;ccl:headers;foo;&#34; named
1602                &#34;C&#34;.</para>
1603            </listitem>
1604
1605            <listitem>
1606              <para>Create a file in &#34;ccl:headers;foo;C;&#34; named
1607                &#34;populate.sh&#34;.</para>
1608
1609              <para>One way of accomplishing the above steps is:</para>
1610
1611              <programlisting format="linespecific">
1612            ? (close (open &#34;ccl:headers;foo;C;populate.sh&#34; :direction :output :
1613                           if-does-not-exist :create :if-exists :overwrite))
1614          </programlisting>
1615            </listitem>
1616
1617            <listitem>
1618              <para>Edit the file created above, using the &#34;populate.sh&#34;
1619                files in the distribution as guidelines.</para>
1620
1621              <para>The file might wind up looking something like:</para>
1622
1623              <programlisting format="linespecific">#/bin/sh
1624            h-to-ffi.sh `foo-config -cflags` /usr/include/foo/foo.h</programlisting>
1625            </listitem>
1626          </orderedlist>
1627
1628      <para>Refer to <xref linkend="The-Interface-Translator"/> for
1629        information about running the interface translator and .ffi
1630        parser.</para>
1631      <para>Assuming that all went well, there should now be .cdb
1632        files in "ccl:headers;foo;". You can then do
1633        <programlisting>
1634          ? (use-interface-dir :foo)
1635            </programlisting> 
1636            whenever you need to
1637        access the foreign type information in those database
1638        files.</para>
1639    </sect2>
1640  </sect1>
1641
1642  <!-- ******************************************  -->
1643  <sect1 id="Using-Shared-Libraries">
1644    <title>Using Shared Libraries</title>
1645
1646    <sect2 id="Shared-Library-Overview">
1647          <title>Overview</title>
1648
1649      <para>&CCL; provides facilities to open and close shared
1650        libraries.</para>
1651      <para>"Opening" a shared library, which is done with <xref
1652                                                              linkend="f_open-shared-library"/>, maps the library's code and
1653        data into &CCL;'s address space and makes its exported
1654        symbols accessible to &CCL;.</para>
1655      <para>"Closing" a shared library, which is done with <xref
1656                                                              linkend="f_close-shared-library"/>, unmaps the library's code
1657        and and removes the library's symbols from the global
1658        namespace.</para>
1659      <para>A small number of shared libraries (including libc,
1660        libm, libdl under Linux, and the "system" library under
1661        Darwin) are opened by the lisp kernel and can't be
1662        closed.</para>
1663      <para>&CCL; uses data structures of type
1664        EXTERNAL-ENTRY-POINT to map a foreign function name (string)
1665        to that foreign function's <emphasis>current</emphasis>
1666        address. (A function's address may vary from session to
1667        session as different versions of shared libraries may load at
1668        different addresses; it may vary within a session for similar
1669        reasons.)</para>
1670      <para>An EXTERNAL-ENTRY-POINT whose address is known is said
1671        to be <emphasis>resolved</emphasis>. When an external entry
1672        point is resolved, the shared library which defines that entry
1673        point is noted; when a shared library is closed, the entry
1674        points that it defines are made unresolved.  An
1675        EXTERNAL-ENTRY-POINT must be in the resolved state in order to
1676        be FF-CALLed; calling an unresolved entry point causes a "last
1677        chance" attempt to resolve it. Attempting to resolve an
1678        entrypoint that was defined in a closed library will cause an
1679        attempt to reopen that library.</para>
1680      <para>&CCL; keeps track of all libraries that have been
1681        opened in a lisp session. When a saved application is first
1682        started, an attempt is made to reopen all libraries that were
1683        open when the image was saved, and an attempt is made to
1684        resolve all entrypoints that had been referenced when the
1685        image was saved. Either of these attempts can fail "quietly",
1686        leaving some entry points in an unresolved state.</para>
1687      <para>Linux shared libraries can be referred to either by a
1688        string which describes their full pathname or by their
1689        <emphasis>soname</emphasis>, a shorter string that can be
1690        defined when the library is created. The dynamic linker
1691        mechanisms used in Linux make it possible (through a series of
1692        filesystem links and other means) to refer to a library via
1693        several names; the library's soname is often the most
1694        appropriate identifier.</para>
1695      <para>sonames are often less version-specific than other names
1696        for libraries; a program that refers to a library by the name
1697        "libc.so.6" is more portable than one which refers to
1698        "libc-2.1.3.so" or to "libc-2.2.3.so", even though the latter
1699        two names might each be platform-specific aliases of the
1700        first.</para>
1701      <para>All of the global symbols described below are exported
1702        from the CCL package.</para>
1703    </sect2>
1704
1705    <sect2 id="Limitations-and-known-bugs--1-">
1706      <title>Limitations and known bugs</title>
1707          <itemizedlist>
1708        <listitem>
1709              <para>Don't get me started.</para>
1710            </listitem>
1711        <listitem>
1712              <para>The underlying functionality has a poor notion of
1713                dependency;it's not always possible to open libraries that
1714                depend on unopened libraries, but it's possible to close
1715                libraries on which other libraries depend. It
1716                <emphasis>may</emphasis> be possible to generate
1717                more explicit dependency information by parsing the output
1718                of the Linux ldd and ldconfig programs.</para>
1719            </listitem>
1720       
1721          </itemizedlist>
1722    </sect2>
1723
1724    <sect2 id="Darwin-Notes">
1725      <title>>Darwin Notes</title>
1726          <para>Darwin shared libraries come in two (basic) flavors:</para>
1727          <itemizedlist>
1728        <listitem>
1729              <para>"dylibs" (which often have the extension".dylib")
1730                are primarily intended to be linked against atcompile/link
1731                time. They can be loaded dynamically,<emphasis>but can't
1732                  be unloaded</emphasis>. Accordingly,OPEN-SHARED-LIBRARY
1733                can be used to open a .dylib-style library;calling
1734                CLOSE-SHARED-LIBRARY on the result of such a callproduces
1735                a warning, and has no other effect.It appears that (due to
1736                an OS bug) attempts to open .dylibshared-libraries that
1737                are already open can cause memory corruptionunless the
1738                full pathname of the .dylib file is specified on thefirst
1739                and all subsequent calls.</para>
1740            </listitem>
1741        <listitem>
1742              <para>"bundles" are intended to serve as application
1743                extensions; they can be opened multiple times (creating
1744                multiple instances of the library!) and closed
1745                properly.</para>
1746        </listitem>
1747          </itemizedlist>
1748      <para>Thanks to Michael Klingbeil for getting both kinds of
1749        Darwin shared libraries working in &CCL;.</para>
1750    </sect2>
1751  </sect1>
1752
1753  <!-- ******************************************  -->
1754  <sect1 id="The-Interface-Translator">
1755    <title>The Interface Translator</title>
1756
1757    <sect2 id="Inteface-translator-overview">
1758          <title>Overview</title>
1759          <para>&CCL; uses an interface translation system based on the FFIGEN
1760            system, which is described at
1761            http://www.ccs.neu.edu/home/lth/ffigen/.
1762            The interface translator makes
1763            the constant, type, structure, and function definitions in a set of
1764            C-language header files available to lisp code.</para>
1765      <para>The basic idea of the FFIGEN scheme is to use the C
1766        compiler's frontend and parser to translate .h files into
1767        semantically equivalent .ffi files, which represent the
1768        definitions from the headers using a syntax based on
1769        S-expressions.  Lisp code can then concentrate on the .ffi
1770        representation, without having to concern itself with the
1771        semantics of header file inclusion or the arcana of C
1772        parsing.</para>
1773      <para>The original FFIGEN system used a modified version of
1774        the LCC C compiler to produce .ffi files. Since many LinuxPPC
1775        header files contain GCC-specific constructs, &CCL;'s
1776        translation system uses a modified version of GCC (called,
1777        somewhat confusingly, ffigen.)</para>
1778      <para>A version of ffigen based on GCC-4.0 was developed
1779        during the spring and summer of 2005.  Sources (diffs relative
1780        to the GCC-4.0 release) are available here, and binaries are
1781        available for DarwinPPC and for LinuxPPC.  These versions
1782        should be insensitive to to the version of GCC (and its
1783        preprocessor) installed on the system.</para>
1784      <para>An older version was developed in 2001-2002; it depended
1785        on the installed version of GCC being 2.95.  It may still be
1786        of interest for people unable to run the GCC-4.0-based version
1787        for whatever reason.</para>
1788      <para>A LinuxPPC binary of this older version is available at
1789        ftp://clozure.com/pub/ffigen-0.1.tar.gz, and LinuxPPC source
1790        differences are at
1791        ftp://clozure.com/pub/ffigen-src.tar.gz.</para>
1792      <para>For Darwin, the binary of the older FFIGEN is available
1793        at ftp://clozure.com/pub/ffigen-darwin.tar.gz, and the source
1794        differences are at
1795        ftp://clozure.com/pub/ffigen-darwin-src.tar.gz.</para>
1796      <para>A shell script (distributed with the source and binary
1797        packages) called h-to-ffi.sh reads a specified .h file (and
1798        optional preprocessor arguments) and writes a (hopefully)
1799        equivalent .ffi file to standard output, calling the installed
1800        C preprocessor and the ffigen program with appropriate
1801        arguments.</para>
1802      <para>For each interface directory (see FIXTHIS)
1803        <emphasis>subdir</emphasis> distributed with &CCL;, a shell
1804        script (distributed with &CCL; as
1805        "ccl:headers;<emphasis>subdir</emphasis>;C;populate.sh"
1806        ("ccl:darwin-headers;<emphasis>subdir</emphasis>;C;populate.sh"
1807        for Darwin)) calls h-to-ffi.sh on a large number of the header
1808        files in /usr/include (or some other <emphasis>system header
1809          path</emphasis>) and creates a parallel directory tree in
1810        "ccl:headers;<emphasis>subdir</emphasis>;C;<emphasis>system</emphasis>;<emphasis>header</emphasis>;<emphasis>path</emphasis>;"
1811        (or
1812        "ccl:darwin-headers;<emphasis>subdir</emphasis>;C;<emphasis>system</emphasis>;<emphasis>header</emphasis>;<emphasis>path</emphasis>;"),
1813        populating that directory with .ffi files.</para>
1814      <para>A lisp function defined in "ccl:library;parse-ffi.lisp"
1815        reads the .ffi files in a specified interface directory
1816        <emphasis>subdir</emphasis> and generates new versions of the
1817        databases (files with the extension .cdb).</para>
1818      <para>The CDB databases are used by the #$ and #_ reader
1819        macros and are used in the expansion of RREF, RLET, and
1820        related macros.</para>
1821    </sect2>
1822
1823    <sect2 id="Details--rebuilding-the-CDB-databases--step-by-step">
1824      <title>Details: rebuilding the CDB databases, step by step</title>
1825          <orderedlist>
1826            <listitem>
1827              <para>Ensure that the FFIGEN program is installed. See
1828                the"README" file in the source or binary archive for
1829                specificinstallation instructions.This example assumes
1830                LinuxPPC; for 32-bit DarwinPPC,
1831                substitute"ccl:darwin-headers;" for "ccl:headers;".  For
1832                64-bit DarwinPPC,substitute
1833                "ccl:darwin-headers64;".</para>
1834            </listitem>
1835        <listitem>
1836              <para>Edit the
1837                "ccl:headers;<emphasis>subdir</emphasis>;C;populate.sh"shell
1838                script. When you're confident that the files
1839                andpreprocessor options match your environment, cd to
1840                the"ccl:headers;<emphasis>subdir</emphasis>;C;" directory
1841                andinvoke ./populate.sh. Repeat this step until you're
1842                able tocleanly translate all files refrenced in the shell
1843                script.</para>
1844            </listitem>
1845            <listitem>
1846              <para>Run &CCL;:
1847            <programlisting>
1848              ? (require "PARSE-FFI")
1849              PARSE-FFI
1850
1851              ? (parse-standard-ffi-files :SUBDIR)
1852              ;;; lots of output ... after a while, shiny new .cdb files should
1853              ;;; appear in "ccl:headers;subdir;"
1854              ;;; (or "ccl:darwin-headers;subdir;" under Darwin)
1855          </programlisting></para>
1856            </listitem>
1857          </orderedlist>
1858    </sect2>
1859  </sect1>
1860
1861  <!-- ******************************************  -->
1862  <sect1 id="Case-sensitivity-of-foreign-names-in-CCL">
1863    <title>Case-sensitivity of foreign names in &CCL;</title>
1864
1865    <sect2 id="Case-sensitivity-overview">
1866          <title>Overview</title>
1867          <para>As of release 0.11, &CCL; addresses the fact that
1868            foreign type, constant, record, field, and function nams are
1869            case-sensitive and provides mechanisms to refer to these names
1870            via lisp symbols.</para>
1871      <para>Previous versions of &CCL; have tried to ignore that
1872        fact, under the belief that case conflicts were rare and that
1873        many users (and implementors) would prefer not to deal with
1874        case-related issues. The fact that some information in the
1875        interface databases was incomplete or inaccessable because of
1876        this policy made it clearer that the policy was untenable. I
1877        can't claim that the approach described here is aesthetically
1878        pleasing, but I can honestly say that it's less unpleasant
1879        than other approaches that I'd thought of. I'd be interested
1880        to hear alternate proposals.</para>
1881      <para>The issues described here have to do with how lisp
1882        symbols are used to denote foreign functions, constants,
1883        types, records, and fields. It doesn't affect how other lisp
1884        objects are sometimes used to denote foreign objects. For
1885        instance, the first argument to the EXTERNAL-CALL macros is
1886        now and has always been a case-sensitive string.</para>
1887    </sect2>
1888
1889    <sect2 id="Foreign-constant-and-function-names">
1890          <title>Foreign constant and function names</title>
1891      <para>The primary way of referring to foreign constant and
1892        function names in &CCL; is via the #$ and #_ reader
1893        macros. These reader macro functions each read a symbol into
1894        the "OS" package, look up its constant or function definition
1895        in the interface database, and assign the value of the
1896        constant to the symbol or install a macroexpansion function on
1897        the symbol.</para>
1898      <para>In order to observe case-sensitivity, the reader-macros
1899        now read the symbol with (READTABLE-CASE :PRESERVE) in
1900        effect.</para>
1901      <para>This means that it's necessary to type the foreign
1902        constant or function name in correct case, but it isn't
1903        necessary to use any special escaping constructs when writing
1904        the variable name. For instance:</para>
1905      <programlisting>
1906        (#_read fd buf n) ; refers to foreign symbol "read"
1907        (#_READ fd buf n) ; refers to foreign symbol "READ", which may
1908        ; not exist ...
1909        #$o_rdonly ; Probably doesn't exist
1910        #$O_RDONLY ; Exists on most platforms
1911      </programlisting>
1912    </sect2>
1913
1914    <sect2 id="Foreign-type--record--and-field-names">
1915          <title>Foreign type, record, and field names</title>
1916          <para>Constructs like RLET expect a foreign type or record
1917            name to be denoted by a symbol (typically a keyword); RREF
1918            (and PREF) expect an "accessor" form, typically a keyword
1919            formed by concatenating a foreign type or record name with a
1920            sequence of one or more foreign field names, separated by
1921            dots. These names are interned by the reader as other lisp
1922            symbols are, with an arbitrary value of READTABLE-CASE in
1923            effect (typically :UPCASE.) It seems like it would be very
1924            tedious to force users to manually escape (via vertical bar or
1925            backslash syntax) all lowercase characters in symbols used to
1926            specify foreign type, record, and field names (especially
1927            given that many traditional POSIX structure, type, and field
1928            names are entirely lowercase.)</para>
1929      <para>The approach taken by &CCL; is to allow the symbols
1930        (keywords) used to denote foreign type, record, and field
1931        names to contain angle brackets (<literal>&lt;</literal> and
1932        <literal>&gt;</literal>). Such symbols are translated to
1933            foreign names via the following set of conventions:</para>
1934          <itemizedlist>
1935        <listitem>
1936              <para>All instances of &lt; and &gt; in the symbol's pname
1937                are balanced and don't nest.</para>
1938            </listitem>
1939        <listitem>
1940              <para>Any alphabetic characters in the symbol's pname
1941                thataren't enclosed in angle brackets are treated as
1942                lower-case,regardless of the value of READTABLE-CASE and
1943                regardless of the case in which they were written.</para>
1944            </listitem>
1945        <listitem>
1946              <para>Alphabetic characters that appear within angle
1947                brackets aremapped to upper-case, again regardless of how
1948                they were written orinterned.</para>
1949            </listitem>
1950      </itemizedlist>
1951          <para>There may be many ways of "escaping" (with angle
1952            brackets) sequences of upper-case and non-lower-case
1953            characters in a symbol used to denote a foreign name. When
1954            translating in the other direction, &CCL; always escapes the
1955            longest sequence that starts with an upper-case character and
1956            doesn't contain a lower-case character.</para>
1957      <para>It's often preferable to use this canonical form of a
1958        foreign type name.</para>
1959      <para>The accessor forms used by PREF/RREF should be viewed as
1960        a series of foreign type/record and field names; upper-case
1961        sequences in the component names should be escaped with angle
1962        brackets, but those sequences shouldn't span components. (More
1963        simply, the separating dots shouldn't be enclosed, even if
1964        both surrounding characters need to be.)</para>
1965      <para>Older POSIX code tends to use lower-case exclusively for
1966        type, record, and field names; there are only a few cases in
1967        the &CCL; sources where mixed-case names need to be
1968        escaped.</para>
1969         
1970    </sect2>
1971
1972    <sect2 id="Examples--1-">
1973      <title>Examples</title>
1974      <programlisting>
1975        ;;; Allocate a record of type "window".
1976        (rlet ((w :window)) ...)
1977        ;;; Allocate a record of type "Window", which is probably a
1978        ;;;  different type
1979        (rlet ((w :&lt;w&gt;indow)) ...)
1980        ;;; This is equivalent to the last example
1981        (rlet ((w :&lt;w&gt;INDOW)))
1982      </programlisting>
1983    </sect2>
1984  </sect1>
1985
1986  <!-- ******************************************  -->
1987  <sect1 id="Reading-Foreign-Names">
1988    <title>Reading Foreign Names</title> <para>Clozure Common Lisp
1989      provides several reader macros to make it more convenient to
1990      handle foreign type, function, variable, and constant
1991      names. Each of these reader macros reads symbols preserving the
1992      case of the source text, and selects an appropriate package in
1993      which to intern the resulting symbol. These reader macros are
1994      especially useful when your Lisp code interacts extensively with
1995      a foreign library&mdash;for example, when using Mac OS X's Cocoa
1996      frameworks.</para>
1997
1998    <para>These reader macros include "#_" to read foreign function
1999      names, "#&amp;" to read foreign variable names (note that in
2000      earlier versions of OpenMCL the reader macro "#?" was used for
2001      this same purpose), "#$" to read foreign constant names, "#/" to
2002      read the names of foreign Objective-C methods, and "#>" to read
2003      keywords that can be used as the names of types, records, and
2004      accessors.</para>
2005
2006    <para>All of these reader macros preserve the case of the text
2007      that they read; beyond that similarity, each performs some
2008      additional work, unique to each reader macro, to create symbols
2009      suitable for a particular use. For example, the function,
2010      variable, and constant reader macros intern the resulting symbol
2011      in the "OS" package of the running platform, but the reader
2012      macro for Objective-C method names interns symbols in the
2013      "NEXTSTEP-FUNCTIONS" package.</para>
2014
2015    <para>You are likely to see these reader macros used extensively
2016      in Lisp code that works with foreign libraries; for example,
2017      &CCL; IDE code, which defines numerous Objective-C classes
2018      and methods, uses these reader macros extensively.</para>
2019
2020    <para>For more detailed descriptions of each of these reader
2021      macros, see the Foreign-Function-Interface Dictionary
2022      section.</para>
2023  </sect1>
2024
2025  <!-- ******************************************  -->
2026  <sect1 id="Tutorial--Using-Basic-Calls-and-Types">
2027    <title>Tutorial: Using Basic Calls and Types</title>
2028    <para>This tutorial is meant to cover the basics of &CCL; for
2029      calling external C functions and passing data back and forth.
2030      These basics will provide the foundation for more advanced
2031      techniques which will allow access to the various external
2032      libraries and toolkits.</para>
2033    <para>The first step is to start with a simple C dynamic library
2034      in order to actually observe what is actually passing between
2035      &CCL; and C.  So, some C code is in order:</para>
2036    <para>Create the file typetest.c, and put the following code
2037      into it:</para>
2038    <programlisting>
2039#include &lt;stdio.&gt;
2040
2041void
2042void_void_test(void)
2043{
2044    printf("Entered %s:\n", __FUNCTION__);
2045    printf("Exited  %s:\n", __FUNCTION__);
2046    fflush(stdout);
2047}
2048
2049signed char
2050sc_sc_test(signed char data)
2051{
2052    printf("Entered %s:\n", __FUNCTION__);
2053    printf("Data In: %d\n", (signed int)data);
2054    printf("Exited  %s:\n", __FUNCTION__);
2055    fflush(stdout);
2056    return data;
2057}
2058
2059unsigned char
2060uc_uc_test(unsigned char data)
2061{
2062    printf("Entered %s:\n", __FUNCTION__);
2063    printf("Data In: %d\n", (signed int)data);
2064    printf("Exited  %s:\n", __FUNCTION__);
2065    fflush(stdout);
2066    return data;
2067}
2068    </programlisting>
2069    <para>This defines three functions.  If you're familiar with C,
2070      notice that there's no <literal>main()</literal>, because we're
2071      just building a library, not an executable.</para>
2072    <para>The function <literal>void_void_test()</literal> doesn't
2073      take any parameters, and doesn't return anything, but it prints
2074      two lines to let us know it was called.
2075      <literal>sc_sc_test()</literal> takes a signed char as a
2076      parameter, prints it, and returns it.
2077      <literal>uc_uc_test()</literal> does the same thing, but with an
2078      unsigned char.  Their purpose is just to prove to us that we
2079      really can call C functions, pass them values, and get values
2080      back from them.</para>
2081    <para>This code is compiled into a dynamic library on OS X
2082      10.3.4 with the command:</para>
2083    <programlisting>
2084
2085      gcc -dynamiclib -Wall -o libtypetest.dylib typetest.c \
2086      -install_name ./libtypetest.dylib
2087    </programlisting>
2088    <tip><para>Users of 64-bit platforms may need to pass options such
2089        as "-m64" to gcc, may need to give the output library a different
2090        extension (such as ".so"), and may need to user slightly different
2091        values for other options in order to create an equivalent test
2092        library.</para></tip>
2093
2094    <para>The -dynamiclib tells gcc that we will be compiling this
2095      into a dynamic library and not an executable binary program.
2096      The output filename is "libtypetest.dylib".  Notice that we
2097      chose a name which follows the normal OS X convention, being in
2098      the form "libXXXXX.dylib", so that other programs can link to
2099      the library.  &CCL; doesn't need it to be this way, but it is
2100      a good idea to adhere to existing conventions.</para>
2101    <para>The -install_name flag is primarily used when building OS
2102      X "bundles".  In this case, we are not using it, so we put a
2103      placeholder into it, "./libtypetest.dylib".  If we wanted to use
2104      typetest in a bundle, the -install_name argument would be a
2105      relative path from some "current" directory.</para>
2106    <para>After creating this library, the first step is to tell
2107      &CCL; to open the dynamic library.  This is done by calling
2108      .</para>
2109    <programlisting>
2110
2111      Welcome to &CCL; Version (Beta: Darwin) 0.14.2-040506!
2112
2113      ? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
2114      #&lt;SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E&gt;
2115    </programlisting>
2116    <para>You should use an absolute path here; using a relative
2117      one, such as just "libtypetest.dylib", would appear to work, but
2118      there are subtle problems which occur after reloading it.  See
2119      the Darwin notes on for details.  It would be a bad idea anyway,
2120      because software should never rely on its starting directory
2121      being anything in particular.</para>
2122    <para>This command returns a reference to the opened shared library, and
2123      &CCL; also adds one to the global variable
2124      <literal>ccl::*shared-libraries*</literal>:</para>
2125    <programlisting>
2126
2127      ? ccl::*shared-libraries*
2128      (#&lt;SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
2129       #&lt;SHLIB /usr/lib/libSystem.B.dylib #x606179E>)
2130    </programlisting>
2131    <para>Before we call anything, let's check that the individual
2132      functions can actually be found by the system.  We don't have to
2133      do this, but it helps to know how to find out whether this is
2134      the problem, when something goes wrong.  We use <xref
2135                                                         linkend="m_external-call"/>:</para>
2136    <programlisting>
2137
2138      ? (external "_void_void_test")
2139      #&lt;EXTERNAL-ENTRY-POINT "_void_void_test" (#x000CFDF8) /Users/andewl/openmcl/libtypetest.dylib #x638EDF6>
2140
2141      ? (external "_sc_sc_test")
2142      #&lt;EXTERNAL-ENTRY-POINT "_sc_sc_test" (#x000CFE50) /Users/andewl/openmcl/libtypetest.dylib #x638EB3E>
2143
2144      ? (external "_uc_uc_test")
2145      #&lt;EXTERNAL-ENTRY-POINT "_uc_uc_test" (#x000CFED4) /Users/andewl/openmcl/libtypetest.dylib #x638E626>
2146    </programlisting>
2147    <para>Notice that the actual function names have been "mangled"
2148      by the C linker.  The first function was named "void_void_test"
2149      in typetest.c, but in libtypetest.dylib, it has an underscore (a
2150      "_" symbol) before it: "_void_void_test".  So, this is the name
2151      which you have to use.  The mangling - the way the name is
2152      changed - may be different for other operating systems or other
2153      versions, so you need to "just know" how it's done...</para>
2154    <para>Also, pay particular attention to the fact that a
2155      hexadecimal value appears in the EXTERNAL-ENTRY-POINT.
2156      (#x000CFDF8, for example - but what it is doesn't matter.)
2157      These hex numbers mean that the function can be dereferenced.
2158      Functions which aren't found will not have a hex number.  For
2159      example:</para>
2160    <programlisting>
2161
2162      ? (external "functiondoesnotexist")
2163      #&lt;EXTERNAL-ENTRY-POINT "functiondoesnotexist" {unresolved}  #x638E3F6>
2164    </programlisting>
2165    <para>The "unresolved" tells us that &CCL; wasn't able to find this
2166      function, which means you would get an error, "Can't resolve foreign
2167      symbol," if you tried to call it.</para>
2168    <para>These external function references also are stored in a
2169      hash table which is accessible through a global variable,
2170      <literal>ccl::*eeps*</literal>.</para>
2171    <para>At this point, we are ready to try our first external
2172      function call:</para>
2173    <programlisting>
2174
2175      ? (external-call "_void_void_test" :void)
2176      Entered void_void_test:
2177      Exited  void_void_test:
2178      NIL
2179    </programlisting>
2180    <para>We used , which is is the normal mechanism for accessing
2181      externally linked code.  The "_void_void_test" is the mangled
2182      name of the external function.  The :void refers to the return
2183      type of the function.</para>
2184    <para>The next step is to try passing a value to C, and getting one
2185      back:</para>
2186    <programlisting>
2187
2188      ? (external-call "_sc_sc_test" :signed-byte -128 :signed-byte)
2189      Entered sc_sc_test:
2190      Data In: -128
2191      Exited  sc_sc_test:
2192      -128
2193    </programlisting>
2194    <para>The first :signed-byte gives the type of the first
2195      argument, and then -128 gives the value to pass for it.  The
2196      second :signed-byte gives the return type.  The return type is
2197      always given by the last argument to .</para>
2198    <para>Everything looks good.  Now, let's try a number outside
2199      the range which fits in one byte:</para>
2200    <programlisting>
2201
2202      ? (external-call "_sc_sc_test" :signed-byte -567 :signed-byte)
2203      Entered sc_sc_test:
2204      Data In: -55
2205      Exited  sc_sc_test:
2206      -55
2207    </programlisting>
2208    <para>Hmmmm.  A little odd.  Let's look at the unsigned stuff to
2209      see how it reacts:</para>
2210    <programlisting>
2211
2212      ? (external-call "_uc_uc_test" :unsigned-byte 255 :unsigned-byte)
2213      Entered uc_uc_test:
2214      Data In: 255
2215      Exited  uc_uc_test:
2216      255
2217    </programlisting>
2218    <para>That looks okay.  Now, let's go outside the valid range again:</para>
2219    <programlisting>
2220
2221      ? (external-call "_uc_uc_test" :unsigned-byte 567 :unsigned-byte)
2222      Entered uc_uc_test:
2223      Data In: 55
2224      Exited  uc_uc_test:
2225      55
2226
2227      ? (external-call "_uc_uc_test" :unsigned-byte -567 :unsigned-byte)
2228      Entered uc_uc_test:
2229      Data In: 201
2230      Exited  uc_uc_test:
2231      201
2232    </programlisting>
2233    <para>Since a signed byte can only hold values from -128 through 127, and
2234      an unsigned one can only hold values from 0 through 255, any number
2235      outside that range gets "clipped": only the low eight bits of it
2236      are used.</para>
2237    <para>What is important to remember is that <emphasis>external
2238        function calls have
2239        very few safety checks.</emphasis>
2240      Data outside the valid range for its type will silently do
2241      very strange things; pointers outside the valid range can very well
2242      crash the system.</para>
2243    <para>That's it for our first example library.  If you're still
2244      following along, let's add some more C code to look at the rest
2245      of the primitive types.  Then we'll need to recompile the
2246      dynamic library, load it again, and then we can see what
2247      happens.</para>
2248    <para>Add the following code to typetest.c:</para>
2249    <programlisting>
2250int
2251si_si_test(int data)
2252{
2253    printf("Entered %s:\n", __FUNCTION__);
2254    printf("Data In: %d\n", data);
2255    printf("Exited  %s:\n", __FUNCTION__);
2256    fflush(stdout);
2257    return data;
2258}
2259
2260long
2261sl_sl_test(long data)
2262{
2263    printf("Entered %s:\n", __FUNCTION__);
2264    printf("Data In: %ld\n", data);
2265    printf("Exited  %s:\n", __FUNCTION__);
2266    fflush(stdout);
2267    return data;
2268}
2269
2270long long
2271sll_sll_test(long long data)
2272{
2273    printf("Entered %s:\n", __FUNCTION__);
2274    printf("Data In: %lld\n", data);
2275    printf("Exited  %s:\n", __FUNCTION__);
2276    fflush(stdout);
2277    return data;
2278}
2279
2280float
2281f_f_test(float data)
2282{
2283    printf("Entered %s:\n", __FUNCTION__);
2284    printf("Data In: %e\n", data);
2285    printf("Exited  %s:\n", __FUNCTION__);
2286    fflush(stdout);
2287    return data;
2288}
2289
2290double
2291d_d_test(double data)
2292{
2293    printf("Entered %s:\n", __FUNCTION__);
2294    printf("Data In: %e\n", data);
2295    printf("Exited  %s:\n", __FUNCTION__);
2296    fflush(stdout);
2297    return data;
2298}
2299    </programlisting>
2300    <para>The command line to compile the dynamic library is the same as before:</para>
2301    <programlisting>
2302
2303      gcc -dynamiclib -Wall -o libtypetest.dylib typetest.c \
2304      -install_name ./libtypetest.dylib
2305    </programlisting>
2306    <para>Now, restart &CCL;.  This step is required because
2307      &CCL; cannot close and reload a dynamic library on OS
2308      X.</para>
2309    <para>Have you restarted?  Okay, try out the new code:</para>
2310    <programlisting>
2311
2312      Welcome to &CCL; Version (Beta: Darwin) 0.14.2-040506!
2313
2314      ? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
2315      #&lt;SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
2316
2317      ? (external-call "_si_si_test" :signed-fullword -178965 :signed-fullword)
2318      Entered si_si_test:
2319      Data In: -178965
2320      Exited  si_si_test:
2321      -178965
2322
2323      ? ;; long is the same size as int on 32-bit machines.
2324      (external-call "_sl_sl_test" :signed-fullword -178965 :signed-fullword)
2325      Entered sl_sl_test:
2326      Data In: -178965
2327      Exited  sl_sl_test:
2328      -178965
2329
2330      ? (external-call "_sll_sll_test"
2331      :signed-doubleword -973891578912 :signed-doubleword)
2332      Entered sll_sll_test:
2333      Data In: -973891578912
2334      Exited  sll_sll_test:
2335      -973891578912
2336    </programlisting>
2337    <para>Okay, everything seems to be acting as expected.  However,
2338      just to remind you that most of this stuff has no safety net,
2339      here's what happens if somebody mistakes
2340      <literal>sl_sl_test()</literal> for
2341      <literal>sll_sll_test()</literal>, thinking that a long is
2342      actually a doubleword:</para>
2343    <programlisting>
2344
2345      ? (external-call "_sl_sl_test"
2346      :signed-doubleword -973891578912 :signed-doubleword)
2347      Entered sl_sl_test:
2348      Data In: -227
2349      Exited  sl_sl_test:
2350      -974957576192
2351    </programlisting>
2352    <para>Ouch.  The C function changes the value with no warning
2353      that something is wrong.  Even worse, it manages to pass the
2354      original value back to &CCL;, which hides the fact that
2355      something is wrong.</para>
2356    <para>Finally, let's take a look at doing this with
2357      floating-point numbers.</para>
2358    <programlisting>
2359
2360      Welcome to &CCL; Version (Beta: Darwin) 0.14.2-040506!
2361
2362      ? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
2363      #&lt;SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
2364
2365      ? (external-call "_f_f_test" :single-float -1.256791e+11 :single-float)
2366      Entered f_f_test:
2367      Data In: -1.256791e+11
2368      Exited  f_f_test:
2369      -1.256791E+11
2370
2371      ? (external-call "_d_d_test" :double-float -1.256791d+290 :double-float)
2372      Entered d_d_test:
2373      Data In: -1.256791e+290
2374      Exited  d_d_test:
2375      -1.256791D+290
2376    </programlisting>
2377    <para>Notice that the number ends with "...e+11" for the single-float,
2378      and "...d+290" for the
2379      double-float.  Lisp has both of these float types itself, and the
2380      d instead of the e is how you specify which to create.  If
2381      you tried to pass :double-float 1.0e2 to external-call, Lisp would
2382      be nice enough to notice and give you a type error.  Don't get the
2383      :double-float wrong, though, because then there's no protection.</para>
2384    <para>Congratulations!  You now know how to call external C functions from
2385      within &CCL;, and pass numbers back and forth.  Now that the basic
2386      mechanics of calling and passing work, the next step is to examine how
2387      to pass more complex data structures around.</para>
2388
2389    <sect2 id="Acknowledgement">
2390      <title>Acknowledgement</title>
2391          <para>This chapter was generously contributed by Andrew
2392            P. Lentvorski Jr.</para>
2393    </sect2>
2394  </sect1>
2395
2396  <!-- ******************************************  -->
2397  <sect1 id="Tutorial--Allocating-Foreign-Data-on-the-Lisp-Heap">
2398    <title>Tutorial: Allocating Foreign Data on the Lisp Heap </title>
2399    <para>Not every foreign function is so marvelously easy to use
2400      as the ones we saw in the last section.  Some of them require
2401      you to allocate a C struct, fill it in with your own
2402      information, and pass it a pointer to the struct.  Some of them
2403      require you to allocate an empty struct so they can fill it in,
2404      and then you can read the information out of it.</para>
2405    <para>Also, some of them have their own structs and return a
2406      pointer to that same struct every time you call them, but those
2407      are easier to deal with, so they won't be covered in this
2408      section.</para>
2409    <para>You might know that Lisp (and, indeed, most programming
2410      languages) has two separate regions of memory.  There's the
2411      stack, which is where variable bindings are kept.  Memory on the
2412      stack is allocated every time any function is called, and
2413      deallocated when it returns, so it's useful for anything that
2414      doesn't need to last longer than one function call, when there's
2415      only one thread.  If that's all you need, you can do it with
2416      .</para>
2417    <para>Then, there's the heap, which holds everything else, and
2418      is our subject here.  There are two advantages and one big
2419      disadvantage to putting things on the heap rather than the
2420      stack.  First, data allocated on the heap can be passed outside
2421      of the scope in which it was created.  This is useful for data
2422      which may need to be passed between multiple C calls or multiple
2423      threads. Also, some data may be too large to copy multiple times
2424      or may be too large to allocate on the stack.</para>
2425    <para>The second advantage is security.  If incoming data is
2426      being placed directly onto the stack, the input data can cause
2427      stack overflows and underflows.  This is not something which
2428      Lisp users generally worry about since garbage collection
2429      generally handles memory management.  However, "stack smashing"
2430      is one of the classic exploits in C which malicious hackers can
2431      use to gain control of a machine.  Not checking external data is
2432      always a bad idea; however, allocating it into the heap at least
2433      offers more protection than direct stack allocation.</para>
2434    <para>The big disadvantage to allocating data on the heap is
2435      that it must be explicitly deallocated&mdash;you need to "free" it
2436      when you're done with it.  Ordinarily, in Lisp, you wouldn't
2437      allocate memory yourself, and the garbage collector would know
2438      about it, so you wouldn't have to think about it again.  When
2439      you're doing it manually, it's very different.  Memory
2440      management becomes a manual process, just like in C and
2441      C++.</para>
2442    <para>What that means is that, if you allocate something and
2443      then lose track of the pointer to it, there's no way to ever
2444      free that memory.  That's what's called a memory leak, and if
2445      your program leaks enough memory it will eventually use up all
2446      of it!  So, you need to be careful to not lose your
2447      pointers.</para>
2448    <para>That disadvantage, though, is also an advantage for using
2449      foreign functions.  Since the garbage collector doesn't know
2450      about this memory, it will never move it around.  External C
2451      code needs this, because it doesn't know how to follow it to
2452      where it moved, the way that Lisp code does.  If you allocate
2453      data manually, you can pass it to foreign code and know that no
2454      matter what that code needs to do with it, it will be able to,
2455      until you deallocate it.  Of course, you'd better be sure it's
2456      done before you do.  Otherwise, your program will be unstable
2457      and might crash sometime in the future, and you'll have trouble
2458      figuring out what caused the trouble, because there won't be
2459      anything pointing back and saying "you deallocated this too
2460      soon."</para>
2461    <para>And, so, on to the code...</para>
2462    <para>As in the last tutorial, our first step
2463      is to create a local dynamic library in order to help show
2464      what is actually going on between &CCL; and C.  So, create the file
2465      ptrtest.c, with the following code:</para>
2466    <programlisting>
2467#include &lt;stdio.h&gt;
2468
2469void reverse_int_array(int * data, unsigned int dataobjs)
2470{
2471    int i, t;
2472   
2473    for(i=0; i&lt;dataobjs/2; i++)
2474        {
2475            t = *(data+i);
2476            *(data+i) = *(data+dataobjs-1-i);
2477            *(data+dataobjs-1-i) = t;
2478        }
2479}
2480
2481void reverse_int_ptr_array(int **ptrs, unsigned int ptrobjs)
2482{
2483    int *t;
2484    int i;
2485   
2486    for(i=0; i&lt;ptrobjs/2; i++)
2487        {
2488            t = *(ptrs+i);
2489            *(ptrs+i) = *(ptrs+ptrobjs-1-i);
2490            *(ptrs+ptrobjs-1-i) = t;
2491        }
2492}
2493
2494void
2495reverse_int_ptr_ptrtest(int **ptrs)
2496{
2497    reverse_int_ptr_array(ptrs, 2);
2498   
2499    reverse_int_array(*(ptrs+0), 4);
2500    reverse_int_array(*(ptrs+1), 4);
2501}
2502    </programlisting>
2503    <para>This defines three functions.
2504      <literal>reverse_int_array</literal> takes a pointer to an array
2505      of <literal>int</literal>s, and a count telling how many items
2506      are in the array, and loops through it putting the elements in
2507      reverse.  <literal>reverse_int_ptr_array</literal> does the same
2508      thing, but with an array of pointers to <literal>int</literal>s.
2509      It only reverses the order the pointers are in; each pointer
2510      still points to the same thing.
2511      <literal>reverse_int_ptr_ptrtest</literal> takes an array of
2512      pointers to arrays of <literal>int</literal>s.  (With me?)  It
2513      doesn't need to be told their sizes; it just assumes that the
2514      array of pointers has two items, and that both of those are
2515      arrays which have four items.  It reverses the array of
2516      pointers, then it reverses each of the two arrays of
2517      <literal>int</literal>s.</para>
2518    <para>Now, compile ptrtest.c into a dynamic library using the
2519      command:</para>
2520    <programlisting>
2521      gcc -dynamiclib -Wall -o libptrtest.dylib ptrtest.c -install_name ./libptrtest.dylib
2522    </programlisting>
2523    <para>If that command doesn't make sense to you, feel free to go back
2524      and read about it at .</para>
2525    <para>Now, start &CCL; and enter:</para>
2526    <programlisting>
2527      ? ;; make-heap-ivector courtesy of Gary Byers
2528      (defun make-heap-ivector (element-count element-type)
2529       (let* ((subtag (ccl::element-type-subtype element-type)))
2530        (unless (= (logand subtag target::fulltagmask)
2531                 target::fulltag-immheader)
2532         (error "~s is not an ivector subtype." element-type))
2533        (let* ((size-in-bytes (ccl::subtag-bytes subtag element-count)))
2534         (ccl::%make-heap-ivector subtag size-in-bytes element-count))))     
2535      MAKE-HEAP-IVECTOR
2536
2537      ? ;; dispose-heap-ivector created for symmetry
2538      (defmacro dispose-heap-ivector (a mp)
2539       `(progn
2540         (ccl::%dispose-heap-ivector ,a)
2541         ;; Demolish the arguments for safety
2542         (setf ,a nil)
2543         (setf ,mp nil)))
2544      DISPOSE-HEAP-IVECTOR
2545    </programlisting>
2546    <para>If you don't understand how those functions do what they do.
2547      That's okay; it gets into very fine detail which really doesn't
2548      matter, because you don't need to change them.</para>
2549    <para>The function <literal>make-heap-ivector</literal> is the
2550      primary tool for allocating objects in heap memory.  It
2551      allocates a fixed-size &CCL; object in heap memory.  It
2552      returns both an array reference, which can be used directly from
2553      &CCL;, and a <literal>macptr</literal>, which can be used to
2554      access the underlying memory directly.  For example:</para>
2555    <programlisting>
2556      ? ;; Create an array of 3 4-byte-long integers
2557      (multiple-value-bind (la lap)
2558          (make-heap-ivector 3 '(unsigned-byte 32))
2559        (setq a la)
2560        (setq ap lap))
2561      ;Compiler warnings :
2562      ;   Undeclared free variable A, in an anonymous lambda form.
2563      ;   Undeclared free variable AP, in an anonymous lambda form.
2564      #&lt;A Mac Pointer #x10217C>
2565
2566      ? a
2567      #(1396 2578 97862649)
2568
2569      ? ap
2570      #&lt;A Mac Pointer #x10217C>
2571    </programlisting>
2572    <para>It's important to realize that the contents of the
2573      <literal>ivector</literal> we've just created haven't been
2574      initialized, so their values are unpredictable, and you should
2575      be sure not to read from them before you set them, to avoid
2576      confusing results.</para>
2577    <para>At this point, <literal>a</literal> references an object
2578      which works just like a normal array.  You can refer to any item
2579      of it with the standard <literal>aref</literal> function, and
2580      set them by combining that with <literal>setf</literal>.  As
2581      noted above, the <literal>ivector</literal>'s contents haven't
2582      been initialized, so that's the next order of business:</para>
2583    <programlisting>
2584      ? a
2585      #(1396 2578 97862649)
2586
2587      ? (aref a 2)
2588      97862649
2589
2590      ? (setf (aref a 0) 3)
2591      3
2592
2593      ? (setf (aref a 1) 4)
2594      4
2595
2596      ? (setf (aref a 2) 5)
2597      5
2598
2599      ? a
2600      #(3 4 5)
2601    </programlisting>
2602    <para>In addition, the <literal>macptr</literal> allows direct
2603      access to the same memory:</para>
2604    <programlisting>
2605      ? (setq *byte-length-of-long* 4)
2606      4
2607
2608      ? (%get-signed-long ap (* 2 *byte-length-of-long*))
2609      5
2610
2611      ? (%get-signed-long ap (* 0 *byte-length-of-long*))
2612      3
2613
2614      ? (setf (%get-signed-long ap (* 0 *byte-length-of-long*)) 6)
2615      6
2616
2617      ? (setf (%get-signed-long ap (* 2 *byte-length-of-long*)) 7)
2618      7
2619
2620      ? ;; Show that a actually got changed through ap
2621      a
2622      #(6 4 7)
2623    </programlisting>
2624    <para>So far, there is nothing about this object that could not
2625      be done much better with standard Lisp.  However, the
2626      <literal>macptr</literal> can be used to pass this chunk of
2627      memory off to a C function.  Let's use the C code to reverse the
2628      elements in the array:</para>
2629    <programlisting>
2630      ? ;; Insert the full path to your copy of libptrtest.dylib
2631      (open-shared-library "/Users/andrewl/openmcl/openmcl/gtk/libptrtest.dylib")
2632      #&lt;SHLIB /Users/andrewl/openmcl/openmcl/gtk/libptrtest.dylib #x639D1E6>
2633
2634      ? a
2635      #(6 4 7)
2636
2637      ? ap
2638      #&lt;A Mac Pointer #x10217C>
2639
2640      ? (external-call "_reverse_int_array" :address ap :unsigned-int (length a) :address)
2641      #&lt;A Mac Pointer #x10217C>
2642
2643      ? a
2644      #(7 4 6)
2645
2646      ? ap
2647      #&lt;A Mac Pointer #x10217C>
2648    </programlisting>
2649    <para>The array gets passed correctly to the C function,
2650      <literal>reverse_int_array</literal>.  The C function reverses
2651      the contents of the array in-place; that is, it doesn't make a
2652      new array, just keeps the same one and reverses what's in it.
2653      Finally, the C function passes control back to &CCL;.  Since
2654      the allocated array memory has been directly modifed, &CCL;
2655      reflects those changes directly in the array as well.</para>
2656    <para>There is one final bit of housekeeping to deal with.
2657      Before moving on, the memory needs to be deallocated:</para>
2658    <programlisting>
2659      ? ;; dispose-heap-ivector created for symmetry
2660      ;; Macro repeated here for pedagogy
2661      (defmacro dispose-heap-ivector (a mp)
2662      `(progn
2663      (ccl::%dispose-heap-ivector ,a)
2664      ;; Demolish the arguments for safety
2665      (setf ,a nil)
2666      (setf ,mp nil)))
2667      DISPOSE-HEAP-IVECTOR
2668
2669      ? (dispose-heap-ivector a ap)
2670      NIL
2671
2672      ? a
2673      NIL
2674
2675      ? ap
2676      NIL
2677    </programlisting>
2678    <para>The <literal>dispose-heap-ivector</literal> macro actually
2679      deallocates the ivector, releasing its memory into the heap for
2680      something else to use.  In addition, it makes sure that the
2681      variables which it was called with are set to nil, because
2682      otherwise they would still be referencing the memory of the
2683      ivector - which is no longer allocated, so that would be a bug.
2684      Making sure there are no other variables set to it is up to
2685      you.</para>
2686    <para>When do you call <literal>dispose-heap-ivector</literal>?
2687      Anytime after you know the ivector will never be used again, but
2688      no sooner.  If you have a lot of ivectors, say, in a hash table,
2689      you need to make sure that when whatever you were doing with the
2690      hash table is done, those ivectors all get freed.  Unless
2691      there's still something somewhere else which refers to them, of
2692      course!  Exactly what strategy to take depends on the situation,
2693      so just try to keep things simple unless you know better.</para>
2694    <para>The simplest situation is when you have things set up so
2695      that a Lisp object "encapsulates" a pointer to foreign data,
2696      taking care of all the details of using it.  In this case, you
2697      don't want those two things to have different lifetimes: You
2698      want to make sure your Lisp object exists as long as the foreign
2699      data does, and no longer; and you want to make sure the foreign
2700      data doesn't get deallocated while your Lisp object still refers
2701      to it.</para>
2702    <para>If you're willing to accept a few limitations, you can
2703      make this easy.  First, you can't let foreign code keep a
2704      permanent pointer to the memory; it has to always finish what
2705      it's doing, then return, and not refer to that memory again.
2706      Second, you can't let any Lisp code that isn't part of your
2707      encapsulating "wrapper" refer to the pointer directly.  Third,
2708      nothing, either foreign code or Lisp code, should explicitly
2709      deallocate the memory.</para>
2710    <para>If you can make sure all of these are true, you can at
2711      least ensure that the foreign pointer is deallocated when the
2712      encapsulating object is about to become garbage, by using
2713      &CCL;'s nonstandard "termination" mechanism, which is
2714      essentially the same as what Java and other languages call
2715      "finialization".</para>
2716    <para>Termination is a way of asking the garbage collector to
2717      let you know when it's about to destroy an object which isn't
2718      used anymore.  Before destroying the object, it calls a function
2719      which you write, called a terminator.</para>
2720    <para>So, you can use termination to find out when a particular
2721      <literal>macptr</literal> is about to become garbage.  That's
2722      not quite as helpful as it might seem: It's not exactly the same
2723      thing as knowing that the block of memory it points to is
2724      unreferenced.  For example, there could be another
2725      <literal>macptr</literal> somewhere to the same block; or, if
2726      it's a struct, there could be a <literal>macptr</literal> to one
2727      of its fields.  Most problematically, if the address of that
2728      memory has been passed to foreign code, it's sometimes hard to
2729      know whether that code has kept the pointer.  Most foreign
2730      functions don't, but it's not hard to think of
2731      exceptions.</para>
2732    <para>You can use code such as this to make all this happen:</para>
2733    <programlisting>
2734      (defclass wrapper (whatever)
2735        ((element-type :initarg :element-type)
2736         (element-count :initarg :element-count)
2737         (ivector)
2738         (macptr)))
2739
2740      (defmethod initialize-instance ((wrapper wrapper) &rest; initargs)
2741        (declare (ignore initargs))
2742        (call-next-method)
2743        (ccl:terminate-when-unreachable wrapper)
2744        (with-slots (ivector macptr element-type element-count) wrapper
2745          (multiple-value-bind (new-ivector new-macptr)
2746              (make-heap-ivector element-count element-type)
2747            (setq ivector new-ivector
2748                  macptr new-macptr))))
2749
2750      (defmethod ccl:terminate ((wrapper wrapper))
2751        (with-slots (ivector macptr) wrapper
2752          (when ivector
2753            (dispose-heap-ivector ivector macptr)
2754            (setq ivector nil
2755                  macptr nil))))
2756    </programlisting>
2757    <para>The <literal>ccl:terminate</literal> method will be called
2758      on some arbitrary thread sometime (hopefully soon) after the GC
2759      has decided that there are no strong references to an object
2760      which has been the argument of a
2761      <literal>ccl:terminate-when-unreachable</literal> call.</para>
2762    <para>If it makes sense to say that the foreign object should
2763      live as long as there's Lisp code that references it (through
2764      the encapsulating obect) and no longer, this is one way of doing
2765      that.</para>
2766    <para>Now we've covered passing basic types back and forth with
2767      C, and we've done the same with pointers.  You may think this is
2768      all...  but we've only done pointers to basic types.  Join us
2769      next time for pointers... to pointers.</para>
2770
2771    <sect2 id="Acknowledgement--1-">
2772          <title>Acknowledgement</title>
2773          <para>Much of this chapter was generously contributed by
2774            Andrew P. Lentvorski Jr.</para>
2775    </sect2>
2776  </sect1>
2777
2778  <!-- ******************************************  -->
2779  <sect1>
2780    <title>The Foreign-Function-Interface Dictionary</title>
2781    <anchor id="anchor_Foreign-Function-Interface-Dictionary"/>
2782    <!-- ====================================  -->
2783    <refentry id="rm_sharpsign-underscore">
2784      <indexterm zone="rm_sharpsign-underscore">
2785            <primary>#_</primary>
2786      </indexterm>
2787
2788      <refnamediv>
2789            <refname>#_</refname>
2790            <refpurpose></refpurpose>
2791            <refclass>Reader Macro</refclass>
2792      </refnamediv>
2793
2794      <refsect1>
2795            <title>Description</title>
2796
2797            <para>Reads a symbol from the current input stream, with *PACKAGE*
2798              bound to the &#34;OS&#34; package and with readtable-case preserved.</para>
2799           
2800            <para>Does a lookup on that symbol in <link
2801                                                     linkend="The-Interface-Database">the &CCL; interface
2802                database</link>, signalling an error if no foreign function
2803              information can be found for the symbol in any active <link
2804                                                                       linkend="Using-Interface-Directories">interface
2805                directory</link>.</para>
2806
2807            <para>Notes the foreign function information, including the foreign
2808              function&#39;s return type, the number and type of the foreign
2809              function&#39;s required arguments, and an indication of whether or
2810              not the function accepts additional arguments (via e.g., the
2811              &#34;varargs&#34; mechanism in C).</para>
2812
2813            <para>Defines a macroexpansion function on the symbol, which expand
2814              macro calls involving the symbol into EXTERNAL-CALL forms where
2815              foreign argument type specifiers for required arguments and the
2816              return value specifer are provided from the information in the
2817              database.</para>
2818
2819            <para>Returns the symbol.</para>
2820
2821            <para>The effect of these steps is that it&#39;s possible to call
2822              foreign functions that take fixed numbers of arguments by simply
2823              providing argument values, as in:</para>
2824
2825            <programlisting format="linespecific">(#_isatty fd)
2826          (#_read fd buf n)</programlisting>
2827
2828            <para>and to call foreign functions that take variable numbers of
2829              arguments by specifying the types of non-required args, as in:</para>
2830
2831            <programlisting format="linespecific">(with-cstrs ((format-string &#34;the answer is: %d&#34;))
2832          (#_printf format-string :int answer))</programlisting>
2833
2834        <para>You can query whether a given name is defined in the
2835          interface databases by appending the '?' character to the reader
2836          macro; for example:</para>
2837
2838        <programlisting>
2839          CL-USER&gt; #_?printf
2840          T
2841          CL-USER&gt; #_?foo
2842          NIL
2843        </programlisting>
2844
2845      </refsect1>
2846    </refentry>
2847
2848    <!-- ====================================  -->
2849    <refentry id="rm_sharpsign-ampersand">
2850      <indexterm zone="rm_sharpsign-ampersand">
2851            <primary>#&amp;</primary>
2852      </indexterm>
2853
2854      <refnamediv>
2855            <refname>#&amp;</refname>
2856            <refpurpose></refpurpose>
2857            <refclass>Reader Macro</refclass>
2858      </refnamediv>
2859
2860      <refsect1>
2861            <title>Description</title>
2862
2863            <para>In &CCL; 1.2 and later, the #&amp; reader macro can be used to
2864              access foreign variables; this functionality depends on the presence of
2865              &#34;vars.cdb&#34; files in the interface database. The current behavior
2866              of the #&amp; reader macro is to:</para>
2867
2868            <para>Read a symbol from the current input stream, with *PACKAGE*
2869              bound to the &#34;OS&#34; package and with readtable-case preserved.</para>
2870           
2871            <para>Use that symbol&#39;s pname to access the &CCL; interface
2872              database, signalling an error if no appropriate foreign variable
2873              information can be found with that name in any active interface
2874              directory.</para>
2875
2876            <para>Use type information recorded in the database to construct a
2877              form which can be used to access the foreign variable, and return
2878              that form.</para>
2879
2880            <para>Please note that the set of foreign variables declared in header files
2881              may or may not match the set of foreign variables exported from
2882              libraries (we&#39;re generally talking about C and Unix here ...). When
2883              they do match, the form constructed by the #&amp; reader macro manages the
2884              details of resolving and tracking changes to the foreign variable&#39;s
2885              address.</para>
2886
2887            <para>Future extensions (via prefix arguments to the reader macro) may
2888              offer additional behavior; it might be convenient (for instance) to be
2889              able to access the address of a foreign variable without dereferencing
2890              that address.</para>
2891
2892            <para>Foreign variables in C code tend to be platform- and
2893              packge-specific (the canonical example - &#34;errno&#34; - is typically
2894              not a variable when threads are involved. )</para>
2895
2896            <para>In LinuxPPC, </para>
2897
2898            <programlisting>? #&amp;stderr</programlisting>
2899
2900            <para>returns a pointer to the stdio error stream (&#34;stderr&#34; is a
2901              macro under OSX/Darwin).</para>
2902
2903            <para>On both LinuxPPC and DarwinPPC, </para>
2904
2905            <programlisting>? #&amp;sys_errlist</programlisting>
2906
2907            <para>returns a pointer to a C array of C error message strings.</para>
2908
2909        <para>You can query whether a given name is defined in the
2910          interface databases by appending the '?' character to the reader
2911          macro; for example:</para>
2912
2913        <programlisting>
2914          CL-USER&gt; #&amp;?sys_errlist
2915          T
2916          CL-USER&gt; #&amp;?foo
2917          NIL
2918        </programlisting>
2919
2920      </refsect1>
2921    </refentry>
2922
2923    <!-- ====================================  -->
2924    <refentry id="rm_sharpsign-dollarsign">
2925      <indexterm zone="rm_sharpsign-dollarsign">
2926        <primary>#$</primary>
2927      </indexterm>
2928     
2929      <refnamediv>
2930        <refname>#$</refname>
2931        <refpurpose></refpurpose>
2932        <refclass>Reader Macro</refclass>
2933      </refnamediv>
2934     
2935      <refsect1>
2936        <title>Description</title>
2937       
2938            <para>In &CCL; 0.14.2 and later, the #? reader macro can be used
2939              to access foreign constants; this functionality depends on the
2940              presence of &#34;constants.cdb&#34; files in the interface
2941              database. The current behavior of the #$ reader macro is
2942              to:</para>
2943
2944            <para>Read a symbol from the current input stream, with
2945              *PACKAGE* bound to the &#34;OS&#34; package and with
2946              readtable-case preserved.</para>
2947           
2948            <para>Use that symbol&#39;s pname to access the &CCL; interface
2949              database, signalling an error if no appropriate foreign constant
2950              information can be found with that name in any active interface
2951              directory.</para>
2952
2953            <para>Use type information recorded in the database to construct a
2954              form which can be used to access the foreign constant, and return
2955              that form.</para>
2956
2957            <para>Please note that the set of foreign constants declared in
2958              header files may or may not match the set of foreign constants
2959              exported from libraries. When they do match, the form
2960              constructed by the #$ reader macro manages the details of
2961              resolving and tracking changes to the foreign constant's
2962              address.</para>
2963
2964        <para>You can query whether a given name is defined in the
2965          interface databases by appending the '?' character to the reader
2966          macro; for example:</para>
2967
2968        <programlisting>
2969          CL-USER&gt; #$?SO_KEEPALIVE
2970          T
2971          CL-USER&gt; #$?foo
2972          NIL
2973        </programlisting>
2974
2975      </refsect1>
2976    </refentry>
2977
2978    <!-- ====================================  -->
2979    <refentry id="rm_sharpsign-slash">
2980      <indexterm zone="rm_sharpsign-slash">
2981        <primary>#/</primary>
2982      </indexterm>
2983     
2984      <refnamediv>
2985        <refname>#/</refname>
2986        <refpurpose></refpurpose>
2987        <refclass>Reader Macro</refclass>
2988      </refnamediv>
2989     
2990      <refsect1>
2991        <title>Description</title>
2992       
2993            <para>In &CCL; 1.2 and later, the #/ reader macro can be used to
2994              access foreign functions on the Darwin platform. The current
2995              behavior of the #/ reader macro is to:</para>
2996
2997            <para>Read a symbol from the current input stream, with
2998              *PACKAGE* bound to the "NEXTSTEP-FUNCTIONS" package, with
2999              readtable-case preserved, and with any colons included.</para>
3000
3001        <para>Do limited sanity-checking on the resulting symbol; for
3002          example, any name that contains at least one colon is required
3003          also to end with a colon, to conform to Objective-C
3004          method-naming conventions.</para>
3005
3006        <para>Export the resulting symbol from the "NEXTSTEP-FUNCTIONS"
3007          package and return it.</para>
3008
3009        <para>For example, reading "#/alloc" interns and returns
3010          NEXTSTEP-FUNCTIONS:|alloc|. Reading "#/initWithFrame:" interns
3011          and returns NEXTSTEP-FUNCTIONS:|initWithFrame:|.</para>
3012
3013        <para>A symbol read using this macro can be used as an operand
3014          in most places where an Objective-C message name can be used, such as
3015          in the (OBJ:@SELECTOR ...) construct.</para>
3016
3017        <para>Please note: the reader macro is not rigorous about
3018          enforcing Objective-C method-naming conventions. Despite the
3019          simple checking done by the reader macro, it may still be
3020          possible to use it to construct invalid names.</para>
3021
3022        <para>The act of interning a new symbol in the
3023          NEXTSTEP-FUNCTIONS package triggers an interface database lookup
3024          of Objective-C methods with the corresponding message name.  If any
3025          such information is found, a special type of dispatching
3026          function is created and initialized and the new symbol is given
3027          the newly-created dispatching function as its function
3028          definition.</para>
3029
3030        <para>The dispatching knows how to call declared Objective-C methods
3031          defined on the message. In many cases, all methods have the same
3032          foreign type signature, and the dispatching function merely
3033          passes any arguments that it receives to a function that does an
3034          Objective-C message send with the indicated foreign argument and return
3035          types. In other cases, where different Objective-C messages have
3036          different type signatures, the dispatching function tries to
3037          choose a function that handles the right type signature based on
3038          the class of the dispatching function's first argument.</para>
3039           
3040        <para>If new information about Objective-C methods is introduced
3041          (e.g., by using additional interface files or as Objective-C
3042          methods are defined from lisp), the dispatch function is
3043          reinitialized to recognize newly-introduced foreign type
3044          signatures.</para>
3045
3046        <para>The argument and result coercion that the bridge has
3047          tradionally supported is supported by the new mechanism (e.g.,
3048          :&lt;BOOL&gt; arguments can be specified as lisp booleans and :&lt;BOOL&gt;
3049          results are returned as lisp boolean values, and an argument
3050          value of NIL is coerced to a null pointer if the corresponding
3051          argument type is :ID.</para>
3052
3053        <para>Some Objective-C methods accept variable numbers of
3054          arguments; the foreign types of non-required arguments are
3055          determined by the lisp types of those arguments (e.g., integers
3056          are passed as integers, floats as floats, pointers as pointers,
3057          record types by reference.)</para>
3058
3059        <para>Examples:</para>
3060
3061        <programlisting>
3062          ;;; #/alloc is a known message.
3063          ? #'#/alloc
3064          #&lt;OBJC-DISPATCH-FUNCTION NEXTSTEP-FUNCTIONS:|alloc| #x300040E94EBF&gt;
3065          ;;; Sadly, #/foo is not ...
3066          ? #'#/foo
3067          &gt; Error: Undefined function: NEXTSTEP-FUNCTIONS:|foo|
3068
3069          ;;; We can send an "init" message to a newly-allocated instance of
3070          ;;; "NSObject" by:
3071
3072          (send (send ns:ns-object 'alloc) 'init)
3073
3074          ;;; or by
3075
3076          (#/init (#/alloc ns:ns-object))
3077        </programlisting>
3078
3079        <para>Objective-C methods that "return" structures return them
3080          as garbage-collectable pointers when called via dispatch
3081          functions.  For example, if "my-window" is an NS:NS-WINDOW
3082          instance, then</para>
3083
3084        <programlisting>
3085          (#/frame my-window)
3086        </programlisting>
3087
3088        <para>returns a garbage-collectable pointer to a structure that
3089          describes that window's frame rectangle. This convention means
3090          that there's no need to use SLET or special structure-returning
3091          message send syntax; keep in mind, though, that #_malloc,
3092          #_free, and the GC are all involved in the creation and eventual
3093          destruction of structure-typed return values. In some programs
3094          these operations may have an impact on performance.</para>
3095
3096      </refsect1>
3097    </refentry>
3098
3099    <!-- ====================================  -->
3100    <refentry id="rm_sharpsign-greaterthan">
3101      <indexterm zone="rm_sharpsign-greaterthan">
3102        <primary>#&gt;</primary>
3103      </indexterm>
3104     
3105      <refnamediv>
3106        <refname>#&gt;</refname>
3107        <refpurpose></refpurpose>
3108        <refclass>Reader Macro</refclass>
3109      </refnamediv>
3110     
3111      <refsect1>
3112        <title>Description</title>
3113       
3114        <para>In Clozure CL 1.2 and later, the #&gt; reader macro reads
3115          the following text as a keyword, preserving the case of the
3116          text. For example:</para>
3117
3118        <programlisting>
3119          CL-USER&gt; #&gt;FooBar
3120          :&lt;F&gt;OO&lt;B&gt;AR
3121        </programlisting>
3122
3123        <para>The resulting keyword can be used as the name of foreign
3124          types, records, and accessors.</para>
3125       
3126      </refsect1>
3127    </refentry>
3128
3129    <!-- ====================================  -->
3130    <refentry id="f_close-shared-library">
3131          <indexterm zone="f_close-shared-library">
3132            <primary>close-shared-library</primary>
3133          </indexterm>
3134
3135          <refnamediv>
3136            <refname>CLOSE-SHARED-LIBRARY</refname>
3137            <refpurpose>Stops using a shared library, informing the operating
3138              system that it can be unloaded if appropriate.</refpurpose>
3139            <refclass>Function</refclass>
3140          </refnamediv>
3141
3142          <refsynopsisdiv>
3143            <synopsis>
3144              <function>close-shared-library</function> library &key;
3145              completely</synopsis>
3146          </refsynopsisdiv>
3147
3148          <refsect1>
3149            <title>Values</title>
3150
3151            <variablelist>
3152              <varlistentry>
3153                <term>library</term>
3154
3155                <listitem>
3156                      <para>either an object of type SHLIB, or a string which
3157                        designates one by its so-name.</para>
3158                </listitem>
3159              </varlistentry>
3160
3161              <varlistentry>
3162                <term>completely</term>
3163
3164                <listitem>
3165                      <para>a boolean.  The default is T.</para>
3166                </listitem>
3167              </varlistentry>
3168            </variablelist>
3169          </refsect1>
3170
3171          <refsect1>
3172            <title>Description</title>
3173
3174            <para>If <varname>completely</varname> is T, sets the
3175              reference count of <varname>library</varname> to 0.  Otherwise,
3176              decrements it by 1.  In either case, if the reference count
3177              becomes 0, <function>close-shared-library</function>
3178              frees all memory resources consumed <varname>library</varname>
3179              and
3180              causes any EXTERNAL-ENTRY-POINTs known to be defined by it to
3181              become unresolved.</para>
3182          </refsect1>
3183    </refentry>
3184
3185    <!-- ====================================  -->
3186    <refentry id="m_defcallback">
3187          <indexterm zone="m_defcallback">
3188            <primary>defcallback</primary>
3189          </indexterm>
3190
3191          <refnamediv>
3192            <refname>DEFCALLBACK</refname>
3193            <refpurpose></refpurpose>
3194            <refclass>Macro</refclass>
3195          </refnamediv>
3196
3197          <refsynopsisdiv>
3198            <synopsis>
3199              <function>defcallback</function> name
3200              ({arg-type-specifier var}* &optional; result-type-specifier)
3201              &body; body
3202            </synopsis>
3203          </refsynopsisdiv>
3204
3205          <refsect1>
3206            <title>Values</title>
3207
3208            <variablelist>
3209              <varlistentry>
3210                <term>name</term>
3211
3212                <listitem>
3213                      <para>A symbol which can be made into a special variable</para>
3214                </listitem>
3215              </varlistentry>
3216
3217              <varlistentry>
3218                <term>arg-type-specifer</term>
3219
3220                <listitem>
3221                      <para>One of the foreign argument-type keywords,
3222                        described above, or an equivalent <link
3223                                                             linkend="Specifying-And-Using-Foreign-Types">foreign
3224                          type specifier</link>.  In addition, if the keyword
3225                        :WITHOUT-INTERRUPTS is specified, the callback will be
3226                        executed with lisp interrupts disabled if the
3227                        corresponding var is non-NIL. If :WITHOUT-INTERRUPTS
3228                        is specified more than once, the rightmost instance
3229                        wins.</para>
3230                </listitem>
3231              </varlistentry>
3232
3233              <varlistentry>
3234                <term>var</term>
3235
3236                <listitem>
3237                      <para>A symbol (lisp variable), which will be bound to a
3238                        value of the specified type.</para>
3239                </listitem>
3240              </varlistentry>
3241
3242              <varlistentry>
3243                <term>body</term>
3244
3245                <listitem>
3246                      <para>A sequence of lisp forms, which should return a value
3247                        which can be coerced to the specified result-type.</para>
3248                </listitem>
3249              </varlistentry>
3250            </variablelist>
3251          </refsect1>
3252
3253          <refsect1>
3254            <title>Description</title>
3255
3256            <para>Proclaims <varname>name</varname>
3257              to be a special variable; sets its value to a
3258              MACPTR which, when called by foreign code, calls a lisp function
3259              which expects foreign arguments of the specified types and which
3260              returns a foreign value of the specified result type. Any argument
3261              variables which correspond to foreign arguments of type :ADDRESS
3262              are bound to stack-allocated MACPTRs.</para>
3263           
3264            <para>If <varname>name</varname>
3265              is already a callback function pointer, its value is
3266              not changed; instead, it&#39;s arranged
3267              that an
3268              updated version of the lisp callback function will be called.
3269              This feature allows for callback functions to be redefined
3270              incrementally, just like Lisp functions are.</para>
3271
3272            <para><function>defcallback</function>
3273              returns the callback pointer, e.g., the
3274              value of <varname>name</varname>.</para>
3275          </refsect1>
3276    </refentry>
3277
3278    <!-- ====================================  -->
3279    <refentry id="m_def-foreign-type">
3280          <indexterm zone="m_def-foreign-type">
3281            <primary>def-foreign-type</primary>
3282          </indexterm>
3283
3284          <refnamediv>
3285            <refname>DEF-FOREIGN-TYPE</refname>
3286            <refpurpose></refpurpose>
3287            <refclass>Macro</refclass>
3288          </refnamediv>
3289
3290          <refsynopsisdiv>
3291            <synopsis>
3292              <function>def-foreign-type</function> name foreign-type-spec
3293            </synopsis>
3294          </refsynopsisdiv>
3295
3296          <refsect1>
3297            <title>Values</title>
3298
3299            <variablelist>
3300              <varlistentry>
3301                <term>name</term>
3302               
3303                <listitem>
3304                      <para>NIL or a keyword; the keyword may contain
3305                        <link linkend="Case-sensitivity-of-foreign-names-in-CCL" >escaping constructs</link>.</para>
3306                </listitem>
3307              </varlistentry>
3308             
3309              <varlistentry>
3310                <term>foreign-type-spec</term>
3311               
3312                <listitem>
3313                      <para>A foreign type specifier, whose syntax is (loosely)
3314                        defined above.</para>
3315                </listitem>
3316              </varlistentry>
3317            </variablelist>
3318          </refsect1>
3319
3320          <refsect1>
3321            <title>Description</title>
3322
3323            <para>If name is non-NIL, defines name to be an alias for the
3324              foreign type specified by foreign-type-spec. If foreign-type-spec
3325              is a named structure or union type, additionally defines that
3326              structure or union type.</para>
3327           
3328            <para>If name is NIL, foreign-type-spec must be a named foreign
3329              struct or union definition, in which case the foreign structure
3330              or
3331              union definition is put in effect.</para>
3332           
3333            <para>Note that there are two separate namespaces for foreign
3334              type names, one for the names of ordinary types and one for
3335              the names of structs and unions.  Which one
3336              <varname>name</varname> refers to depends on
3337              <varname>foreign-type-spec</varname> in the obvious manner.
3338            </para>
3339          </refsect1>
3340    </refentry>
3341
3342    <!-- ====================================  -->
3343    <refentry id="m_external">
3344          <indexterm zone="m_external">
3345            <primary>external</primary>
3346          </indexterm>
3347         
3348          <refnamediv>
3349            <refname>EXTERNAL</refname>
3350            <refpurpose>Resolves a reference to an external symbol which
3351              is defined in a shared library.</refpurpose>
3352            <refclass>Macro</refclass>
3353          </refnamediv>
3354
3355          <refsynopsisdiv>
3356            <synopsis>
3357              <function>external</function> name => entry
3358            </synopsis>
3359          </refsynopsisdiv>
3360
3361          <refsect1>
3362            <title>Values</title>
3363
3364            <variablelist>
3365              <varlistentry>
3366                <term>name</term>
3367                <listitem>
3368                      <para>
3369                        a simple-string which names an external symbol.
3370                        Case-sensitive.
3371                      </para>
3372                </listitem>
3373              </varlistentry>
3374              <varlistentry>
3375                <term>entry</term>
3376                <listitem>
3377                      <para>
3378                        an object of type EXTERNAL-ENTRY-POINT which maintains
3379                        the address of the foreign symbol named by
3380                        <varname>name</varname>.
3381                      </para>
3382                </listitem>
3383              </varlistentry>
3384            </variablelist>
3385          </refsect1>
3386
3387          <refsect1>
3388            <title>Description</title>
3389
3390            <para>If there is already an EXTERNAL-ENTRY-POINT for
3391              the symbol named by <varname>name</varname>, finds it and
3392              returns it.  If not, creates one and returns it.</para>
3393
3394            <para>Tries to resolve the entry point to a memory address,
3395              and identify the containing library.</para>
3396
3397            <para>Be aware that under Darwin, external functions which
3398              are callable from C have underscores prepended to their names,
3399              as in "_fopen".</para>
3400          </refsect1>
3401    </refentry>
3402
3403    <!-- ====================================  -->
3404    <refentry id="m_external-call">
3405          <indexterm zone="m_external-call">
3406            <primary>external-call</primary>
3407          </indexterm>
3408
3409          <refnamediv>
3410            <refname>EXTERNAL-CALL</refname>
3411            <refpurpose></refpurpose>
3412            <refclass>Macro</refclass>
3413          </refnamediv>
3414
3415          <refsynopsisdiv>
3416            <synopsis>
3417              <function>external-call</function> name
3418              {arg-type-specifier arg}* &optional; result-type-specifier
3419            </synopsis>
3420          </refsynopsisdiv>
3421
3422          <refsect1>
3423            <title>Values</title>
3424
3425            <variablelist>
3426              <varlistentry>
3427                <term>name</term>
3428
3429                <listitem>
3430                      <para>A lisp string. See external, above.</para>
3431                </listitem>
3432              </varlistentry>
3433
3434              <varlistentry>
3435                <term>arg-type-specifer</term>
3436
3437                <listitem>
3438                      <para>One of the foreign argument-type keywords, described
3439                        above, or an equivalent <link linkend="Specifying-And-Using-Foreign-Types">foreign
3440                          type specifier</link>.</para>
3441                </listitem>
3442              </varlistentry>
3443
3444              <varlistentry>
3445                <term>arg</term>
3446
3447                <listitem>
3448                      <para>A lisp value of type indicated by the corresponding
3449                        arg-type-specifier</para>
3450                </listitem>
3451              </varlistentry>
3452
3453              <varlistentry>
3454                <term>result-type-specifier</term>
3455
3456                <listitem>
3457                      <para>One of the foreign argument-type keywords, described
3458                        above, or an equivalent <link linkend="Specifying-And-Using-Foreign-Types">foreign
3459                          type specifier</link>.</para>
3460                </listitem>
3461              </varlistentry>
3462            </variablelist>
3463          </refsect1>
3464
3465          <refsect1>
3466            <title>Description</title>
3467
3468            <para>Calls the foreign function at the address obtained by
3469              resolving the external-entry-point associated with name, passing
3470              the values of each arg as a foreign argument of type indicated by
3471              the corresponding arg-type-specifier. Returns the foreign function
3472              result (coerced to a Lisp object of type indicated by
3473              result-type-specifier), or NIL if result-type-specifer is :VOID or
3474              NIL</para>
3475          </refsect1>
3476    </refentry>
3477
3478    <!-- ====================================  -->
3479    <refentry id="f_Pff-call">
3480          <indexterm zone="f_Pff-call">
3481            <primary>%ff-call</primary>
3482          </indexterm>
3483
3484          <refnamediv>
3485            <refname>%FF-CALL</refname>
3486            <refpurpose></refpurpose>
3487            <refclass>Function</refclass>
3488          </refnamediv>
3489
3490          <refsynopsisdiv>
3491            <synopsis>
3492              <function>%ff-call</function> entrypoint
3493              {arg-type-keyword arg}* &optional; result-type-keyword
3494            </synopsis>
3495          </refsynopsisdiv>
3496
3497          <refsect1>
3498            <title>Values</title>
3499
3500            <variablelist>
3501              <varlistentry>
3502                <term>entrypoint</term>
3503               
3504                <listitem>
3505                      <para>A fixnum or MACPTR</para>
3506                </listitem>
3507              </varlistentry>
3508
3509              <varlistentry>
3510                <term>arg-type-keyword</term>
3511
3512                <listitem>
3513                      <para>One of the foreign argument-type keywords, described
3514                        above</para>
3515                </listitem>
3516              </varlistentry>
3517
3518              <varlistentry>
3519                <term>arg</term>
3520
3521                <listitem>
3522                      <para>A lisp value of type indicated by the corresponding
3523                        arg-type-keyword</para>
3524                </listitem>
3525              </varlistentry>
3526
3527              <varlistentry>
3528                <term>result-type-keyword</term>
3529
3530                <listitem>
3531                      <para>One of the foreign argument-type keywords, described
3532                        above</para>
3533                </listitem>
3534              </varlistentry>
3535            </variablelist>
3536          </refsect1>
3537
3538          <refsect1>
3539            <title>Description</title>
3540
3541            <para>Calls the foreign function at address entrypoint passing the
3542              values of each arg as a foreign argument of type indicated by the
3543              corresponding arg-type-keyword. Returns the foreign function
3544              result (coerced to a Lisp object of type indicated by
3545              result-type-keyword), or NIL if result-type-keyword is :VOID or
3546              NIL</para>
3547          </refsect1>
3548    </refentry>
3549
3550    <!-- ====================================  -->
3551    <refentry id="m_ff-call">
3552          <indexterm zone="m_ff-call">
3553            <primary>ff-call</primary>
3554          </indexterm>
3555
3556          <refnamediv>
3557            <refname>FF-CALL</refname>
3558            <refpurpose></refpurpose>
3559            <refclass>Macro</refclass>
3560          </refnamediv>
3561
3562          <refsynopsisdiv>
3563            <synopsis>
3564              <function>ff-call</function> entrypoint
3565              {arg-type-specifier arg}* &optional; result-type-specifier
3566            </synopsis>
3567          </refsynopsisdiv>
3568
3569          <refsect1>
3570            <title>Values</title>
3571
3572            <variablelist>
3573              <varlistentry>
3574                <term>entrypoint</term>
3575
3576                <listitem>
3577                      <para>A fixnum or MACPTR</para>
3578                </listitem>
3579              </varlistentry>
3580
3581              <varlistentry>
3582                <term>arg-type-specifer</term>
3583
3584                <listitem>
3585                      <para>One of the foreign argument-type keywords, described
3586                        above, or an equivalent <link linkend="Specifying-And-Using-Foreign-Types">foreign
3587                          type specifier</link>.</para>
3588                </listitem>
3589              </varlistentry>
3590
3591              <varlistentry>
3592                <term>arg</term>
3593
3594                <listitem>
3595                      <para>A lisp value of type indicated by the corresponding
3596                        arg-type-specifier</para>
3597                </listitem>
3598              </varlistentry>
3599
3600              <varlistentry>
3601                <term>result-type-specifier</term>
3602
3603                <listitem>
3604                      <para>One of the foreign argument-type keywords, described
3605                        above, or an equivalent <link linkend="Specifying-And-Using-Foreign-Types">foreign
3606                          type specifier</link>.</para>
3607                </listitem>
3608              </varlistentry>
3609            </variablelist>
3610          </refsect1>
3611
3612          <refsect1>
3613            <title>Description</title>
3614
3615            <para>Calls the foreign function at address entrypoint passing the
3616              values of each arg as a foreign argument of type indicated by the
3617              corresponding arg-type-specifier. Returns the foreign function
3618              result (coerced to a Lisp object of type indicated by
3619              result-type-specifier), or NIL if result-type-specifer is :VOID or
3620              NIL</para>
3621          </refsect1>
3622    </refentry>
3623
3624    <!-- ====================================  -->
3625    <refentry id="f_foreign-symbol-address">
3626          <indexterm zone="f_foreign-symbol-address">
3627            <primary>foreign-symbol-address</primary>
3628          </indexterm>
3629
3630          <refnamediv>
3631            <refname>FOREIGN-SYMBOL-ADDRESS</refname>
3632            <refpurpose></refpurpose>
3633            <refclass>Function</refclass>
3634          </refnamediv>
3635
3636          <refsynopsisdiv>
3637            <synopsis>
3638              <function>foreign-symbol-address</function> name
3639            </synopsis>
3640          </refsynopsisdiv>
3641
3642          <refsect1>
3643            <title>Values</title>
3644
3645            <variablelist>
3646              <varlistentry>
3647                <term>name</term>
3648
3649                <listitem>
3650                      <para>A lisp string.</para>
3651                </listitem>
3652              </varlistentry>
3653            </variablelist>
3654          </refsect1>
3655
3656          <refsect1>
3657            <title>Description</title>
3658
3659            <para>Tries to resolve the address of the foreign symbol
3660              name. If successful, returns that address encapsulated in
3661              <link
3662                 linkend="Referencing-and-Using-Foreign-Memory-Addresses">a
3663                MACPTR</link>, else returns NIL.</para>
3664          </refsect1>
3665    </refentry>
3666
3667    <!-- ====================================  -->
3668    <refentry id="f_foreign-symbol-entry">
3669          <indexterm zone="f_foreign-symbol-entry">
3670            <primary>foreign-symbol-entry</primary>
3671          </indexterm>
3672
3673          <refnamediv>
3674            <refname>FOREIGN-SYMBOL-ENTRY</refname>
3675            <refpurpose></refpurpose>
3676            <refclass>Function</refclass>
3677          </refnamediv>
3678
3679          <refsynopsisdiv>
3680            <synopsis>
3681              <function>foreign-symbol-entry</function> name
3682            </synopsis>
3683          </refsynopsisdiv>
3684
3685          <refsect1>
3686            <title>Values</title>
3687
3688            <variablelist>
3689              <varlistentry>
3690                <term>name</term>
3691
3692                <listitem>
3693                      <para>A lisp string.</para>
3694                </listitem>
3695              </varlistentry>
3696            </variablelist>
3697          </refsect1>
3698
3699          <refsect1>
3700            <title>Description</title>
3701
3702            <para>Tries to resolve the address of the foreign symbol name. If
3703              successful, returns a fixnum representation of that address, else
3704              returns NIL.</para>
3705          </refsect1>
3706    </refentry>
3707
3708    <!-- ====================================  -->
3709    <refentry id="f_free">
3710      <indexterm zone="f_free">
3711        <primary>free</primary>
3712      </indexterm>
3713     
3714      <refnamediv>
3715        <refname>FREE</refname>
3716        <refpurpose></refpurpose>
3717        <refclass>Function</refclass>
3718      </refnamediv>
3719     
3720          <refsynopsisdiv>
3721            <synopsis>
3722              <function>free</function> ptr
3723            </synopsis>
3724          </refsynopsisdiv>
3725
3726          <refsect1>
3727            <title>Values</title>
3728
3729            <variablelist>
3730              <varlistentry>
3731                <term>ptr</term>
3732
3733                <listitem>
3734                      <para>A <code>MACPTR</code> that points to a block of
3735                      foreign, heap-allocated memory.</para>
3736                </listitem>
3737              </varlistentry>
3738            </variablelist>
3739          </refsect1>
3740      <refsect1>
3741        <title>Description</title>
3742       
3743        <para>In Clozure CL 1.2 and later, the <code>CCL:FREE</code>
3744          function invokes the foreign <code>free</code> function from
3745          the platform's standard C library to deallocate a block of
3746          foreign memory.</para>
3747
3748        <para>Previous versions of &CCL; implemented this function,
3749          but it was not exported.</para>
3750
3751        <para>If the argument to <code>CCL:FREE</code> is a gcable
3752        pointer (for example, an object reurned
3753        by <code>MAKE-GCABLE-RECORD</code>)
3754        then <code>CCL:FREE</code> informs the garbage collector that
3755        the foreign memory has been deallocated before calling the
3756        foreign <code>free</code> function.</para>
3757       
3758      </refsect1>
3759    </refentry>
3760
3761    <!-- ====================================  -->
3762    <refentry id="m_makegcable--record">
3763      <indexterm zone="m_make-record">
3764            <primary>make-gcable-record</primary>
3765      </indexterm>
3766
3767      <refnamediv>
3768            <refname>MAKE-GCABLE-RECORD</refname>
3769            <refpurpose></refpurpose>
3770            <refclass>Macro</refclass>
3771      </refnamediv>
3772
3773      <refsynopsisdiv>
3774            <synopsis>
3775              <function>make-gcable-record</function> typespec
3776              &rest; initforms => result
3777            </synopsis>
3778      </refsynopsisdiv>
3779
3780      <refsect1>
3781            <title>Values</title>
3782
3783            <variablelist>
3784              <varlistentry>
3785                <term>typespec</term>
3786
3787                <listitem>
3788                      <para>A foreign type specifier, or a keyword which is used
3789                        as the name of a foreign struct or union.</para>
3790                </listitem>
3791              </varlistentry>
3792
3793              <varlistentry>
3794                <term>initforms</term>
3795
3796                <listitem>
3797                      <para>If the type denoted by <varname>typespec</varname>
3798                        is scalar, a single value appropriate for that type;
3799                        otherwise, a list of alternating field names and
3800                        values appropriate for the types of those fields.</para>
3801                </listitem>
3802              </varlistentry>
3803
3804              <varlistentry>
3805                <term>result</term>
3806
3807                <listitem>
3808                      <para>
3809                        A <type>macptr</type> which encapsulates the address of a
3810                        newly-allocated record on the foreign heap. The foreign
3811                        object returned by <function>make-gcable-record</function>
3812                        is freed when the garbage collector determines that
3813                        the <code>MACPTR</code> object that describes it is
3814                        unreachable.</para>
3815                </listitem>
3816              </varlistentry>
3817            </variablelist>
3818      </refsect1>
3819
3820      <refsect1>
3821            <title>Description</title>
3822
3823            <para>
3824              Allocates a block of foreign memory suitable to hold the foreign
3825              type described by <code>typespec</code>, in the same manner
3826              as <link linkend="anchor_make-record">MAKE-RECORD</link>. In
3827              addition, <code>MAKE-GCABLE-RECORD</code> marks the
3828              returned object gcable; in other words, it informs the garbage
3829              collector that it may reclaim the object when it becomes
3830              unreachable.
3831            </para>
3832
3833        <para>In all other respects, <code>MAKE-GCABLE-RECORD</code> works
3834          the same way
3835          as <link linkend="anchor_make-record">MAKE-RECORD</link></para>
3836
3837        <para> When using gcable pointers, it's important to remember the
3838          distinction between a <code>MACPTR</code> object (which is a
3839          lisp object, more or less like any other) and the block of
3840          foreign memory that the <code>MACPTR</code> object points to.
3841          If a gcable <code>MACPTR</code> object is the only thing in the
3842          world (lisp world or foreign world) that references the
3843          underlying block of foreign memory, then freeing the foreign
3844          memory when it becomes impossible to reference it is convenient
3845          and sane.  If other lisp <code>MACPTR</code>s reference the
3846          underlying block of foreign memory or if the address of that
3847          foreign memory is passed to and retained by foreign code, having
3848          the GC free the memory may have unpleasant consequences if those
3849          other references are used.</para>
3850
3851        <para>Take care, therefore, not to create a gcable record unless
3852          you are sure that the returned <code>MACPTR</code> will be the
3853          only reference to the allocated memory that will ever be
3854          used.</para>
3855      </refsect1>
3856
3857    </refentry>
3858
3859
3860    <!-- ====================================  -->
3861    <refentry id="m_make-record">
3862          <indexterm zone="m_make-record">
3863            <primary>make-record</primary>
3864          </indexterm>
3865
3866          <refnamediv>
3867        <anchor id="anchor_make-record"/>
3868            <refname>MAKE-RECORD</refname>
3869            <refpurpose></refpurpose>
3870            <refclass>Macro</refclass>
3871          </refnamediv>
3872
3873          <refsynopsisdiv>
3874            <synopsis>
3875              <function>make-record</function> typespec
3876              &rest; initforms => result
3877            </synopsis>
3878          </refsynopsisdiv>
3879
3880          <refsect1>
3881            <title>Values</title>
3882
3883            <variablelist>
3884              <varlistentry>
3885                <term>typespec</term>
3886
3887                <listitem>
3888                      <para>A foreign type specifier, or a keyword which is used
3889                        as the name of a foreign struct or union.</para>
3890                </listitem>
3891              </varlistentry>
3892
3893              <varlistentry>
3894                <term>initforms</term>
3895
3896                <listitem>
3897                      <para>If the type denoted by <varname>typespec</varname>
3898                        is scalar, a single value appropriate for that type;
3899                        otherwise, a list of alternating field names and
3900                        values appropriate for the types of those fields.</para>
3901                </listitem>
3902              </varlistentry>
3903
3904              <varlistentry>
3905                <term>result</term>
3906
3907                <listitem>
3908                      <para>
3909                        A <type>macptr</type> which encapsulates the address of a
3910                        newly-allocated record on the foreign heap.
3911                      </para>
3912                </listitem>
3913              </varlistentry>
3914            </variablelist>
3915          </refsect1>
3916
3917          <refsect1>
3918            <title>Description</title>
3919
3920            <para>
3921              Expands into code which allocates and initalizes
3922              an instance of the type
3923              denoted by <varname>typespec</varname>, on the foreign
3924              heap.  The record is allocated using the C function
3925              <function>malloc</function>, and the user of
3926              <function>make-record</function> must explicitly call
3927              the function <function>CCL:FREE</function> to deallocate the
3928              record, when it is no longer needed.
3929            </para>
3930
3931            <para>
3932              If <varname>initforms</varname> is provided, its value
3933              or values are used in the initialization.  When the type
3934              is a scalar, <varname>initforms</varname> is either a single
3935              value which can be coerced to that type, or no value, in which
3936              case binary 0 is used.  When the type is a <type>struct</type>,
3937              <varname>initforms</varname> is a list, giving field names
3938              and the values for each.  Each field is treated in the same way
3939              as a scalar is: If a value for it is given, it must be
3940              coerceable to the field's type; if not, binary 0 is used.
3941            </para>
3942
3943            <para>
3944              When the type is an array, <varname>initforms</varname> may
3945              not be provided, because <function>make-record</function>
3946              cannot initialize its values.  <function>make-record</function>
3947              is also unable to initialize fields of a <type>struct</type>
3948              which are themselves
3949              <type>struct</type>s.  The user of
3950              <function>make-record</function> should set these values
3951              by another means.
3952            </para>
3953
3954            <para>
3955              A possibly-significant limitation is that it must be possible to
3956              find the foreign type at the time the macro is expanded;
3957              <function>make-record</function> signals an error if this is
3958              not the case.
3959            </para>
3960          </refsect1>
3961
3962          <refsect1>
3963            <title>Notes</title>
3964
3965            <para>
3966              It is inconvenient that <function>make-record</function> is a
3967              macro, because this means that <varname>typespec</varname>
3968              cannot be a variable; it must be an immediate value.
3969            </para>
3970           
3971            <para>
3972              If it weren't for this requirement,
3973              <function>make-record</function> could be a function.  However,
3974              that would mean that any stand-alone application using it would
3975              have to include a copy of the interface database
3976              (see <xref linkend="The-Interface-Database"/>), which is undesireable
3977              because it's large.
3978            </para>
3979          </refsect1>
3980    </refentry>
3981
3982    <!-- ====================================  -->
3983    <refentry id="f_open-shared-library">
3984          <indexterm zone="f_open-shared-library">
3985            <primary>open-shared-library</primary>
3986          </indexterm>
3987         
3988          <refnamediv>
3989            <refname>OPEN-SHARED-LIBRARY</refname>
3990            <refpurpose>Asks the operating system to load a shared library
3991              for &CCL; to use.</refpurpose>
3992            <refclass>Function</refclass>
3993          </refnamediv>
3994         
3995          <refsynopsisdiv>
3996            <synopsis>
3997              <function>open-shared-library</function> name => library
3998            </synopsis>
3999          </refsynopsisdiv>
4000         
4001          <refsect1>
4002            <title>Values</title>
4003           
4004            <variablelist>
4005              <varlistentry>
4006                <term>name</term>       
4007                <listitem>
4008                      <para>A SIMPLE-STRING which is presumed to be the so-name of
4009                        or a filesystem path to the library.</para>
4010                </listitem>
4011              </varlistentry>
4012             
4013              <varlistentry>
4014                <term>library</term>
4015                <listitem>
4016                      <para>An object of type SHLIB which describes the
4017                        library denoted by <varname>name</varname>.</para>
4018                </listitem>
4019              </varlistentry>
4020            </variablelist>
4021          </refsect1>
4022         
4023          <refsect1>
4024            <title>Description</title>
4025
4026            <para>If the library denoted by <varname>name</varname> can
4027              be loaded by the
4028              operating system, returns an object of type SHLIB that describes
4029              the library; if the library is already open, increments a
4030              reference count. If the library can&#39;t be loaded, signals a
4031              SIMPLE-ERROR which contains an often-cryptic message from the
4032              operating system.</para>
4033          </refsect1>
4034
4035          <refsect1>
4036            <title>Examples</title>
4037
4038            <programlisting format="linespecific">;;; Try to do something simple.
4039          ? (open-shared-library &#34;libgtk.so&#34;)
4040          &#62; Error: Error opening shared library &#34;libgtk.so&#34;: /usr/lib/libgtk.so: undefined symbol: gdk_threads_mutex
4041          &#62; While executing: OPEN-SHARED-LIBRARY
4042
4043          ;;; Grovel around, curse, and try to find out where &#34;gdk_threads_mutex&#34;
4044          ;;; might be defined. Then try again:
4045
4046          ? (open-shared-library &#34;libgdk.so&#34;)
4047          #&#60;SHLIB libgdk.so #x3046DBB6&#62;
4048
4049          ? (open-shared-library &#34;libgtk.so&#34;)
4050          #&#60;SHLIB libgtk.so #x3046DC86&#62;
4051
4052          ;;; Reference an external symbol defined in one of those libraries.
4053
4054          ? (external &#34;gtk_main&#34;)
4055          #&#60;EXTERNAL-ENTRY-POINT &#34;gtk_main&#34; (#x012C3004) libgtk.so #x3046FE46&#62;
4056
4057          ;;; Close those libraries.
4058
4059          ? (close-shared-library &#34;libgtk.so&#34;)
4060          T
4061
4062          ? (close-shared-library &#34;libgdk.so&#34;)
4063          T
4064
4065          ;;; Reference the external symbol again.
4066
4067          ? (external &#34;gtk_main&#34;)
4068          #&#60;EXTERNAL-ENTRY-POINT &#34;gtk_main&#34; {unresolved} libgtk.so #x3046FE46&#62;</programlisting>
4069          </refsect1>
4070
4071          <refsect1>
4072            <title>Notes</title>
4073
4074            <para>It would be helpful to describe what an soname is and give
4075              examples of one.</para>
4076
4077            <para>Does the SHLIB still get returned if the library is
4078              already open?</para>
4079          </refsect1>
4080    </refentry>
4081   
4082    <!-- ====================================  -->
4083    <refentry id="m_pref">
4084          <indexterm zone="m_pref">
4085            <primary>pref</primary>
4086          </indexterm>
4087
4088          <refnamediv>
4089            <refname>PREF</refname>
4090            <refpurpose></refpurpose>
4091            <refclass>Macro</refclass>
4092          </refnamediv>
4093
4094          <refsynopsisdiv>
4095            <synopsis>
4096              <function>pref</function> ptr accessor-form
4097            </synopsis>
4098
4099          </refsynopsisdiv>
4100
4101          <refsect1>
4102            <title>Values</title>
4103
4104            <variablelist>
4105              <varlistentry>
4106                <term>ptr</term>
4107
4108                <listitem>
4109                      <para><link linkend="Referencing-and-Using-Foreign-Memory-Addresses">a MACPTR</link>.</para>
4110                </listitem>
4111              </varlistentry>
4112
4113              <varlistentry>
4114                <term>accessor-form</term>
4115
4116                <listitem>
4117                      <para>a keyword which names a foreign type or record, as
4118                        described in <xref linkend="Foreign-type--record--and-field-names"/>.
4119                      </para>
4120                </listitem>
4121              </varlistentry>
4122            </variablelist>
4123          </refsect1>
4124
4125          <refsect1>
4126            <title>Description</title>
4127
4128            <para>References an instance of a foreign type (or a component of
4129              a foreign type) accessible via ptr.</para>
4130           
4131            <para>Expands into code which references the indicated scalar type
4132              or component, or returns a pointer to a composite type.</para>
4133           
4134            <para>PREF can be used with SETF.</para>
4135           
4136            <para>RREF is a deprecated alternative to PREF. It accepts a
4137              :STORAGE keyword and rather loudly ignores it.</para>
4138          </refsect1>
4139    </refentry>
4140
4141    <!-- ====================================  -->
4142    <refentry id="f_Preference-external-entry-point">
4143          <indexterm zone="f_Preference-external-entry-point">
4144            <primary>%reference-external-entry-point</primary>
4145          </indexterm>
4146
4147          <refnamediv>
4148            <refname>%REFERENCE-EXTERNAL-ENTRY-POINT</refname>
4149            <refpurpose></refpurpose>
4150            <refclass>Function</refclass>
4151          </refnamediv>
4152
4153          <refsynopsisdiv>
4154            <synopsis>
4155              <function>%reference-external-entry-point</function> eep
4156            </synopsis>
4157          </refsynopsisdiv>
4158
4159          <refsect1>
4160            <title>Values</title>
4161
4162            <variablelist>
4163              <varlistentry>
4164                <term>eep</term>
4165
4166                <listitem>
4167                      <para>An EXTERNAL-ENTRY-POINT, as obtained by the EXTERNAL
4168                        macro.</para>
4169                </listitem>
4170              </varlistentry>
4171            </variablelist>
4172          </refsect1>
4173
4174          <refsect1>
4175            <title>Description</title>
4176
4177            <para>Tries to resolve the address of the EXTERNAL-ENTRY-POINT
4178              eep; returns a fixnum representation of that address if
4179              successful, else signals an error.</para>
4180          </refsect1>
4181    </refentry>
4182
4183    <!-- ====================================  -->
4184    <anchor id="anchor_rlet"/>
4185    <refentry id="m_rlet">
4186          <indexterm zone="m_rlet">
4187            <primary>rlet</primary>
4188          </indexterm>
4189
4190          <refnamediv>
4191            <refname>RLET</refname>
4192            <refpurpose></refpurpose>
4193            <refclass>Macro</refclass>
4194          </refnamediv>
4195
4196          <refsynopsisdiv>
4197            <synopsis>
4198              <function>rlet</function> (var typespec &rest; initforms)*
4199              &body; body
4200            </synopsis>
4201          </refsynopsisdiv>
4202
4203          <refsect1>
4204            <title>Values</title>
4205
4206            <variablelist>
4207              <varlistentry>
4208                <term>var</term>
4209
4210                <listitem>
4211                      <para>A symbol (a lisp variable)</para>
4212                </listitem>
4213              </varlistentry>
4214
4215              <varlistentry>
4216                <term>typespec</term>
4217
4218                <listitem>
4219                      <para>A foreign type specifier or foreign record name.</para>
4220                </listitem>
4221              </varlistentry>
4222
4223          <varlistentry>
4224                <term>initforms</term>
4225
4226                <listitem>
4227                      <para>As described above, for
4228                        <xref linkend="m_make-record"/></para>
4229                </listitem>
4230              </varlistentry>
4231            </variablelist>
4232          </refsect1>
4233
4234          <refsect1>
4235            <title>Description</title>
4236
4237            <para>Executes <varname>body</varname>
4238              in an environment in which each var is bound
4239              to <link linkend="Referencing-and-Using-Foreign-Memory-Addresses">a MACPTR</link> encapsulating the
4240              address of a stack-allocated foreign memory block, allocated and
4241              initialized from typespec and initforms as per
4242              <xref linkend="m_make-record"/>.
4243              Returns whatever value(s) <varname>body</varname>
4244              returns.</para>
4245           
4246            <para>Record fields that aren&#39;t explicitly initialized have
4247              unspecified contents.</para>
4248          </refsect1>
4249    </refentry>
4250
4251    <!-- ====================================  -->
4252    <refentry id="m_rletz">
4253          <indexterm zone="m_rletz">
4254            <primary>rletz</primary>
4255          </indexterm>
4256
4257          <refnamediv>
4258            <refname>RLETZ</refname>
4259            <refpurpose></refpurpose>
4260            <refclass>Macro</refclass>
4261          </refnamediv>
4262
4263          <refsynopsisdiv>
4264            <synopsis>
4265              <function>rletz</function> (var typespec &rest; initforms)*
4266              &body; body
4267            </synopsis>
4268          </refsynopsisdiv>
4269
4270          <refsect1>
4271            <title>Values</title>
4272
4273            <variablelist>
4274              <varlistentry>
4275                <term>var</term>
4276
4277                <listitem>
4278                      <para>A symbol (a lisp variable)</para>
4279                </listitem>
4280              </varlistentry>
4281
4282              <varlistentry>
4283                <term>typespec</term>
4284
4285                <listitem>
4286                      <para>A foreign type specifier or foreign record name.</para>
4287                </listitem>
4288              </varlistentry>
4289
4290              <varlistentry>
4291                <term>initforms</term>
4292
4293                <listitem>
4294                      <para>As described above, for ccl:make-record</para>
4295                </listitem>
4296              </varlistentry>
4297            </variablelist>
4298          </refsect1>
4299
4300          <refsect1>
4301            <title>Description</title>
4302
4303            <para>Executes body in an environment in which each var is
4304              bound to <link
4305                          linkend="Referencing-and-Using-Foreign-Memory-Addresses">a
4306                MACPTR</link> encapuslating the address of a stack-allocated
4307              foreign memory block, allocated and initialized from
4308              typespec and initforms as ccl:make-record.</para>
4309           
4310            <para>Returns whatever value(s) body returns.</para>
4311
4312            <para>Unlike rlet, record fields that aren&#39;t explicitly
4313              initialized are set to binary 0.</para>
4314          </refsect1>
4315    </refentry>
4316
4317    <!-- ====================================  -->
4318    <refentry id="f_terminate-when-unreachable">
4319          <indexterm zone="f_terminate-when-unreachable">
4320            <primary>terminate-when-unreachable</primary>
4321          </indexterm>
4322
4323          <refnamediv>
4324            <refname>TERMINATE-WHEN-UNREACHABLE</refname>
4325            <refpurpose></refpurpose>
4326            <refclass>Function</refclass>
4327          </refnamediv>
4328
4329          <refsynopsisdiv>
4330            <synopsis>
4331              <function>terminate-when-unreachable</function> object
4332            </synopsis>
4333          </refsynopsisdiv>
4334
4335          <refsect1>
4336            <title>Values</title>
4337
4338            <variablelist>
4339              <varlistentry>
4340                <term>object</term>
4341
4342                <listitem>
4343                      <para>A CLOS object of a class for which there exists
4344                        a method of the generic function
4345                        <function>ccl:terminate</function>.
4346                      </para>
4347                </listitem>
4348              </varlistentry>
4349            </variablelist>
4350          </refsect1>
4351
4352          <refsect1>
4353            <title>Description</title>
4354
4355            <para>
4356              The "termination" mechanism is a way to have the garbage
4357              collector run a function right before an object is about to
4358              become garbage.  It is very similar to the "finalization"
4359              mechanism which Java has.  It is not standard Common Lisp,
4360              although other Lisp implementations have similar features.
4361              It is useful when there is some sort of special cleanup,
4362              deallocation, or releasing of resources which needs to happen
4363              when a certain object is no longer being used.
4364            </para>
4365
4366            <para>
4367              When the garbage collector discovers that an object is no
4368              longer referred to anywhere in the program, it deallocates
4369              that object, freeing its memory.  However, if
4370              <function>ccl:terminate-when-unreachable</function> has been
4371              called on the object at any time, the garbage collector first
4372              invokes the generic function <function>ccl:terminate</function>,
4373              passing it the object as a parameter.
4374            </para>
4375
4376            <para>
4377              Therefore, to make termination do something useful, you need to
4378              define a method on <function>ccl:terminate</function>.
4379            </para>
4380
4381            <para>
4382              Because calling
4383              <function>ccl:terminate-when-unreachable</function> only
4384              affects a single object, rather than all objects of its
4385              class, you
4386              may wish to put a call to it in the
4387              <function>initialize-instance</function> method of a
4388              class.  Of course, this is only appropriate if you do in fact
4389              want to use termination for all objects of a given class.
4390            </para>
4391          </refsect1>
4392
4393          <refsect1>
4394            <title>Example</title>
4395
4396        <programlisting format="linespecific">
4397          (defclass resource-wrapper ()
4398            ((resource :accessor resource)))
4399
4400          (defmethod initialize-instance :after ((x resource-wrapper) &amp;rest initargs)
4401             (ccl:terminate-when-unreachable x))
4402
4403          (defmethod ccl:terminate ((x resource-wrapper))
4404             (when (resource x)
4405                (deallocate (resource x))))</programlisting>
4406          </refsect1>
4407
4408          <refsect1>
4409            <title>See Also</title>
4410
4411            <simplelist type="inline">
4412              <member><xref linkend="Tutorial--Allocating-Foreign-Data-on-the-Lisp-Heap"/></member>
4413            </simplelist>
4414          </refsect1>
4415
4416    </refentry>
4417
4418     <!-- ====================================  -->
4419    <refentry id="f_unuse-interface-dir">
4420          <indexterm zone="f_unuse-interface-dir">
4421            <primary>unuse-interface-dir</primary>
4422          </indexterm>
4423
4424          <refnamediv>
4425            <refname>UNUSE-INTERFACE-DIR</refname>
4426            <refpurpose></refpurpose>
4427            <refclass>Function</refclass>
4428          </refnamediv>
4429
4430          <refsynopsisdiv>
4431            <synopsis>
4432              <function>unuse-interface-dir</function> dir-id
4433            </synopsis>
4434          </refsynopsisdiv>
4435
4436          <refsect1>
4437            <title>Values</title>
4438
4439            <variablelist>
4440              <varlistentry>
4441                <term>dir-id</term>
4442
4443                <listitem>
4444                      <para>A keyword whose pname, mapped to lower case, names a
4445                        subdirectory of &#34;ccl:headers;&#34; (or
4446                        "ccl:darwin-headers;")</para>
4447                </listitem>
4448              </varlistentry>
4449            </variablelist>
4450          </refsect1>
4451
4452          <refsect1>
4453            <title>Description</title>
4454
4455            <para>Tells &CCL; to remove the interface directory denoted by
4456              dir-id from the list of interface directories which are
4457              consulted for
4458              foreign type and function information. Returns T if the directory
4459              was on the search list, NIL otherwise.</para>
4460          </refsect1>
4461    </refentry>
4462
4463   <!-- ====================================  -->
4464    <refentry id="f_use-interface-dir">
4465          <indexterm zone="f_use-interface-dir">
4466            <primary>use-interface-dir</primary>
4467          </indexterm>
4468
4469          <refnamediv>
4470            <refname>USE-INTERFACE-DIR</refname>
4471            <refpurpose></refpurpose>
4472            <refclass>Function</refclass>
4473          </refnamediv>
4474
4475          <refsynopsisdiv>
4476            <synopsis>
4477              <function>use-interface-dir</function> dir-id
4478            </synopsis>
4479          </refsynopsisdiv>
4480
4481          <refsect1>
4482            <title>Values</title>
4483
4484            <variablelist>
4485              <varlistentry>
4486                <term>dir-id</term>
4487
4488                <listitem>
4489                      <para>A keyword whose pname, mapped to lower case, names a
4490                        subdirectory of &#34;ccl:headers;&#34; (or
4491                        "ccl:darwin-headers;")</para>
4492                </listitem>
4493              </varlistentry>
4494            </variablelist>
4495          </refsect1>
4496
4497          <refsect1>
4498            <title>Description</title>
4499
4500            <para>Tells &CCL; to add the interface directory denoted by
4501              dir-id to the list of interface directories which it consults for
4502              foreign type and function information. Arranges that that
4503              directory is searched before any others.</para>
4504
4505            <para>Note that <function>use-interface-dir</function>
4506              merely adds an entry
4507              to a search list.
4508              If the named directory doesn&#39;t exist in the file system
4509              or doesn&#39;t
4510              contain a set of database files, a runtime error may occur
4511              when &CCL;
4512              tries to open some database file in that directory, and it
4513              will try to
4514              open such a database file whenever it needs to find any
4515              foreign type or
4516              function information. <xref linkend="f_unuse-interface-dir"/>
4517              may come in
4518              handy in that case.</para>
4519          </refsect1>
4520
4521          <refsect1>
4522            <title>Examples</title>
4523
4524            <para>One typically wants interface information to be
4525              available at compile-time (or, in many cases, at read-time).
4526              A typical idiom would be:</para>
4527
4528            <programlisting format="linespecific">(eval-when (:compile-toplevel :execute)
4529          (use-interface-dir :GTK))</programlisting>
4530
4531            <para>Using the :GTK interface directory makes available
4532              information on
4533              foreign types, functions, and constants.  It's generally
4534              necessary to
4535              load foreign libraries before actually calling the
4536              foreign code, which for GTK can be done like this:</para>
4537
4538            <programlisting>(load-gtk-libraries)</programlisting>
4539
4540            <para>It should now be possible to do things like:</para>
4541
4542            <programlisting>(#_gtk_widget_destroy w)</programlisting>
4543          </refsect1>
4544    </refentry>
4545
4546  </sect1>
4547</chapter>
Note: See TracBrowser for help on using the repository browser.