source: trunk/source/doc/src/ffi.xml @ 8981

Last change on this file since 8981 was 8981, checked in by mikel, 11 years ago

additions to ObjC and ffi docs; many mechanical edits; some standardization of XML elements and formatting

File size: 159.7 KB
Line 
1<?xml version="1.0" encoding="utf-8"?>
2<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
3          <!ENTITY rest "<varname>&amp;rest</varname>">
4          <!ENTITY key "<varname>&amp;key</varname>">
5          <!ENTITY optional "<varname>&amp;optional</varname>">
6          <!ENTITY body "<varname>&amp;body</varname>">
7          <!ENTITY aux "<varname>&amp;aux</varname>">
8          <!ENTITY allow-other-keys "<varname>&amp;allow-other-keys</varname>">
9          <!ENTITY CCL "<literal>CCL</literal>">
10          ]>
11
12<chapter id="The-Foreign-Function-Interface">
13  <title>The Foreign-Function Interface</title>
14 
15  <!-- ******************************************  -->
16  <sect1 id="Specifying-And-Using-Foreign-Types">
17    <title>Specifying And Using Foreign Types</title>
18   
19    <sect2 id="Overview-foreign-types">
20      <title>Overview</title>
21      <para>&CCL; provides a fairly rich language for defining and
22        specifying foreign data types (this language is derived from
23        CMUCL's "alien type" system.)</para>
24      <para>In practice, most foreign type definitions are
25        introduced into &CCL; via its interface database (see ),
26        though it's also possible to define foreign types
27        interactively and/or programmatically.</para>
28      <para>&CCL;'s foreign type system is "evolving" (a polite word
29        for not-quite-complete): there are some inconsistencies
30        involving package usage, for instance. Symbols used in foreign
31        type specifiers <emphasis>should</emphasis> be keywords, but
32        this convention isn't always enforced.</para> <para>Foreign
33        type, record, and field names are case-sensitive; &CCL; uses
34        some escaping conventions (see ) to allow keywords to be used to
35        denote these names.</para>
36
37      <sect3 id="type annotations">
38        <title>Type Annotations</title>
39        <para>As of version 1.2, &CCL; supports annotating the types of
40          foreign pointers on Mac OS X. Forms that create pointers to
41          foreign memory&mdash;that is, <code>MACPTR</code>s&mdash;store
42          with the <code>MACPTR</code> object a type annotation that
43          identifies the foreign type of the object pointed
44          to. Calling <code>PRINT-OBJECT</code> on a <code>MACPTR</code>
45          attempts to print information about the identified foreign
46          type, including whether it was allocated on the heap or the
47          stack, and whether it's scheduled for automatic reclamation by
48          the garbage collector.</para>
49
50        <para>Support for type annotation is not yet complete. In
51          particular, some uses of <code>PREF</code>
52          and <code>SLOT-VALUE</code> do ot yet take type annotations into
53          account, and neither do <code>DESCRIBE</code>
54          and <code>INSPECT</code>.</para>
55      </sect3>
56
57      <sect3 id="foreign-type-classes">
58        <title>Foreign Types as Classes</title>
59        <para>Some types of foreign pointers take advantage of the
60          support for type annotations, and pointers of these types
61          can be treated as instances of known classes. Specifically,
62          a pointer to an <code>:&lt;NSR&gt;ect</code> is recognized
63          as an instance of the built-in
64          class <code>NS:NS-RECT</code>, a pointer to
65          an <code>&lt;NSS&gt;ize</code> is treated as an instance
66          of <code>NS:NS-SIZE</code>, a pointer to
67          an <code>&lt;NSP&gt;oint</code> is recognized as an
68          instance of <code>NS:NS-POINT</code>, and a pointer to
69          an <code>&lt;NSR&gt;ange</code> is recognized as an
70          instance of <code>NS:NS-RANGE</code>.</para>
71
72        <para>A few more obscure structure types also support this
73        mechanism, and it's possible that a future version will
74        support user definition of similar type mappings.</para>
75
76        <para>This support for foreign types as classes provides the
77        following conveniences for each supported type:</para>
78
79      <itemizedlist>
80        <listitem>
81          <para>a <code>PRINT-OBJECT</code> method is defined</para>
82        </listitem>
83        <listitem>
84          <para>a foreign type name is created and treated as an alias
85          for the corresponding type. As an example, the
86          name <code>:NS-RECT</code> is a name for the type that
87          corresponds to <code>NS:NS-RECT</code>, and you can
88          use <code>:NS-RECT</code> as a type designator
89          in <link linkend="anchor_rlet"><code>RLET</code></link> forms to
90          specify a structure of type <code>NS-RECT</code>.</para>
91        </listitem>
92        <listitem>
93          <para>the class is integrated into the type system so that
94            <code>(TYPEP R 'NS:NS-RECT)</code> is implemented with
95            fair efficiency.</para>
96        </listitem>
97        <listitem>
98          <para>inlined accessor and <code>SETF</code> inverses are
99            defined for the structure type's fields.  In the case of
100            an <code>&lt;NSR*gt;ect</code>, for example, the fields in
101            question are the fields of the embedded point and size, so
102            that <code>NS:NS-RECT-X</code>, <code>NS:NS-RECT-Y</code>, <code>NS:NS-RECT-WIDTH</code>,
103            <code>NS-RECT-HEIGHT</code> and <code>SETF</code> inverses
104            are defined.  The accessors and setter functions typecheck
105            their arguments and the setters handle coercion to the
106            appropriate type of <code>CGFLOAT</code> where
107            applicable.</para>
108        </listitem>
109        <listitem>
110          <para>an initialization function is defined; for
111            example,</para> 
112
113          <programlisting>
114(NS:INIT-NS-SIZE s w h)
115          </programlisting>
116
117          <para>is roughly equivalent to</para>
118
119          <programlisting>
120(SETF (NS:NS-SIZE-WIDTH s) w
121      (NS:NS-SIZE-HEIGHT s) h)
122          </programlisting>
123
124          <para>but might be a little more efficient.</para>
125        </listitem>
126        <listitem>
127          <para>a creation function is defined; for
128            example</para>
129
130          <programlisting>
131(NS:NS-MAKE-POINT x y)
132          </programlisting> 
133
134          <para>is functionally equivalent to</para>
135
136          <programlisting>
137(LET ((P (MAKE-GCABLE-RECORD :NS-POINT)))
138  (NS:INIT-NS-POINT P X Y)
139  p)
140          </programlisting>
141
142        </listitem>
143        <listitem>
144          <para>a macro is defined which, like <code>RLET</code>,
145            stack-allocates an instance of the foreign record type,
146            optionally initializes that instance, and executes a body
147            of code with a variable bound to that instance.</para>
148
149          <para>For example,</para>
150          <programlisting>
151(ns:with-ns-range (r loc len)
152  (format t "~&amp; range has location ~s, length ~s"
153     (ns:ns-range-location r) (ns:ns-range-length r)))
154          </programlisting>
155        </listitem>
156        <listitem>
157          <para></para>
158        </listitem>
159      </itemizedlist>
160      </sect3>
161
162    </sect2>
163
164    <sect2 id="Syntax-of-Foreign-Type-Specifiers">
165      <title>Syntax of Foreign Type Specifiers</title>
166      <itemizedlist>
167        <listitem>
168          <para>Some foreign types are builtin: keywords denote
169            primitive,builtin types such as the IEEE-double-float type
170            (denoted:DOUBLE-FLOAT), in much the same way as certain
171            symbols(CONS, FIXNUM,etc.) define primitive CL
172            types.</para>
173        </listitem>
174        <listitem>
175          <para>Constructors such as :SIGNED and :UNSIGNED can be
176            used to denote signed and unsigned integer subtypes
177            (analogous to the CL type specifiers SIGNED-BYTE and
178            UNSIGNED-BYTE.) :SIGNED is shorthand for(:SIGNED 32) and
179            :UNSIGNED is shorthand for (:UNSIGNED 32).</para>
180        </listitem>
181        <listitem>
182          <para>Aliases for other (perhaps more complicated) types
183            can be defined via CCL:DEF-FOREIGN-TYPE (sort of like
184            CL:DEFTYPE or the C typedef facility). The type :CHAR is
185            defined as an alias for (:SIGNED8) on some platforms, as
186            (:UNSIGNED 8) on others.</para>
187            </listitem>
188        <listitem>
189              <para>The construct (:STRUCT <emphasis>name</emphasis>)
190                can be used to refer to a named structure type; (:UNION
191                <emphasis>name</emphasis>)can be used to refer to a named
192                union type. It isn't necessary to enumerate a structure or
193                union type's fields in order to refer to the type.</para>
194            </listitem>
195        <listitem>
196              <para>If <emphasis>X</emphasis> is a valid foreign type
197                reference,then (:* <emphasis>X</emphasis>) denotes the
198                foreign type "pointer to<emphasis> X</emphasis>". By
199                convention, (:* T) denotes an anonymous pointer type,
200                vaguely equivalent to "void*" in C.</para>
201            </listitem>
202        <listitem>
203              <para>If a fieldlist is a list of lists, each of whose CAR
204                is a foreign field name (keyword) and whose CADR is a
205                foreign type specifier, then (:STRUCT
206                <emphasis>name</emphasis> ,@fieldlist) is a definition of
207                the structure type <emphasis>name</emphasis>,
208                and (:UNION<emphasis> name</emphasis> ,@fieldlist) is a
209                definition of the union type
210                <emphasis>name</emphasis>. Note that it's necessary
211                to define a structure or union type in order to include
212                that type in a structure, union, or array, but only
213                necessary to "refer to" a structure or union type in order
214                to define a type alias or a pointer type.</para>
215            </listitem>
216            <listitem>
217              <para>If <emphasis>X</emphasis> is a defined foreign type
218                , then (:array <emphasis>X</emphasis> &amp;rest dims)
219                denotes the foreign type "array of
220                <emphasis>X</emphasis>". Although multiple array dimensions
221                are allowed by the :array constructor,
222                only single-dimensioned arrays are (at all) well-supported
223                in &CCL;.</para>
224            </listitem>
225      </itemizedlist>
226    </sect2>
227  </sect1>
228
229  <!-- ******************************************  -->
230  <sect1 id="Foreign-Function-Calls">
231    <title>Foreign Function Calls</title>
232
233    <sect2 id="Overview-foreign-calls">
234          <title>Overview</title>
235      <para>&CCL; provides a number of constructs for calling
236        foreign functions from Lisp code (all of them based on the
237        function CCL:%FF-CALL).  In many cases, &CCL;'s interface
238        translator (see ) provides information about the foreign
239        function's entrypoint name and argument and return types; this
240        enables the use of the #_ reader macro (described below),
241        which may be more concise and/or more readable than other
242        constructs.</para>
243      <para>&CCL; also provides a mechanism for defining
244        <emphasis>callbacks</emphasis>: lisp functions which can be
245        called from foreign code.</para>
246      <para>There's no supported way to directly pass lisp data to
247        foreign functions: scalar lisp data must be coerced to an
248        equivalent foreign representation, and lisp arrays (notably
249        strings) must be copied to non-GCed memory.</para>
250
251      <sect3 id="Type-Designators-for-Arguments-and-Return-Values">
252            <title>Type Designators for Arguments and Return Values</title>
253        <para>The types of foreign argument and return values in foreign
254              function calls and callbacks can be specified by any of the following
255          keywords:</para>
256            <variablelist>
257              <varlistentry>
258                <term>:UNSIGNED-BYTE</term>
259
260                <listitem>
261                      <para>The argument/return value is of type (UNSIGNED-BYTE 8)</para>
262                </listitem>
263              </varlistentry>
264
265              <varlistentry>
266                <term>:SIGNED-BYTE</term>
267
268                <listitem>
269                      <para>The argument/return value is of type (SIGNED-BYTE 8)</para>
270                </listitem>
271              </varlistentry>
272
273              <varlistentry>
274                <term>:UNSIGNED-HALFWORD</term>
275
276                <listitem>
277                      <para>The argument/return value is of type (UNSIGNED-BYTE 16)</para>
278                </listitem>
279              </varlistentry>
280
281              <varlistentry>
282                <term>:SIGNED-HALFWORD</term>
283
284                <listitem>
285                      <para>The argument/return value is of type (SIGNED-BYTE 16)</para>
286                </listitem>
287              </varlistentry>
288
289              <varlistentry>
290                <term>:UNSIGNED-FULLWORD</term>
291
292                <listitem>
293                      <para>The argument/return value is of type (UNSIGNED-BYTE 32)</para>
294                </listitem>
295              </varlistentry>
296
297              <varlistentry>
298                <term>:SIGNED-FULLWORD</term>
299
300                <listitem>
301                      <para>The argument/return value is of type (SIGNED-BYTE 32)</para>
302                </listitem>
303              </varlistentry>
304
305              <varlistentry>
306                <term>:UNSIGNED-DOUBLEWORD</term>
307
308                <listitem>
309                      <para>The argument/return value is of type (UNSIGNED-BYTE 64)</para>
310                </listitem>
311              </varlistentry>
312
313              <varlistentry>
314                <term>:SIGNED-DOUBLEWORD</term>
315
316                <listitem>
317                      <para>The argument/return value is of type (SIGNED-BYTE 64)</para>
318                </listitem>
319              </varlistentry>
320
321              <varlistentry>
322                <term>:SINGLE-FLOAT</term>
323
324                <listitem>
325                      <para>The argument/return value is of type SINGLE-FLOAT</para>
326                </listitem>
327              </varlistentry>
328
329              <varlistentry>
330                <term>:DOUBLE-FLOAT</term>
331
332                <listitem>
333                      <para>The argument/return value is of type DOUBLE-FLOAT</para>
334                </listitem>
335              </varlistentry>
336
337              <varlistentry>
338                <term>:ADDRESS</term>
339
340                <listitem>
341                      <para>The argument/return values
342                        is <link linkend="Referencing-and-Using-Foreign-Memory-Addresses">a MACPTR</link>.</para>
343                </listitem>
344              </varlistentry>
345
346              <varlistentry>
347                <term>:VOID</term>
348
349                <listitem>
350                      <para>or NIL Not valid as an argument type specifier; specifies
351                        that there is no meaningful return value</para>
352                </listitem>
353              </varlistentry>
354            </variablelist>
355
356        <para>On some platforms, a small positive integer
357          <emphasis>N</emphasis> can also be used as an argument
358          specifier; it indicates that the corresponding argument is a
359          pointer to an <emphasis>N</emphasis>-word structure or union
360          which should be passed by value to the foreign
361          function.  Exactly which foreign structures are passed
362              by value and how is very dependent on the Application
363              Binary Interface (ABI) of the platform; unless you're
364              very familiar with ABI details (some of which are quite
365              baroque), it's often easier to let higher-level constructs
366              deal with these details.</para>
367      </sect3>
368
369      <sect3 id="External-Entrypoints-and-Named-External-Entrypoints">
370            <title>External Entrypoints and Named External Entrypoints</title>
371        <para>PowerPC machine instructions are always aligned on
372          32-bit boundaries, so the two least significant bits of the
373          first instruction ("entrypoint") of a foreign function are
374          always 0. &CCL; often represents an entrypoint address as
375          a fixnum that's binary-equivalent to the entrypoint address:
376          if<emphasis> E</emphasis> is an entrypoint address expressed
377          as a signed 32-bit integer, then (ash <emphasis>E</emphasis>
378          -2) is an equivalent fixnum representation of that
379          address. An entrypoint address can also be encapsulated in a
380          MACPTR (see FIXTHIS), but that's somewhat less efficient.</para>
381        <para>Although it's possible to use fixnums or macptrs to
382          represent entrypoint addresses, it's somewhat cumbersome to
383          do so. &CCL; can cache the addresses of named external
384          functions in structure-like objects of type
385          CCL:EXTERNAL-ENTRY-POINT (sometimes abbreviated as EEP).
386          Through the use of LOAD-TIME-VALUE, compiled lisp functions
387          are able to reference EEPs as constants; the use of an
388          indirection allows &CCL; runtime system to ensure that the
389          EEP's address is current and correct.</para>
390      </sect3>
391    </sect2>
392
393    <sect2 id="Return-Conventions-for-C-Structures">
394          <title>Return Conventions for C Structures</title>
395      <para> On some platforms, C functions that are defined to
396        return structures do so by reference: they actually
397        accept a first parameter of type "pointer to returned
398        struct/union" - which must be allocated by the caller - and
399        don't return a meaningful value.</para>
400          <para><emphasis>Exactly</emphasis> how a C function that's
401            defined to return a foreign structure does so is dependent on
402            the ABI (and on the size and composition of the structure/union
403            in many cases.)</para>
404    </sect2>
405  </sect1>
406
407  <!-- ******************************************  -->
408  <sect1 id="Referencing-and-Using-Foreign-Memory-Addresses">
409    <title>Referencing and Using Foreign Memory Addresses</title>
410
411    <sect2 id="Overview-memory-addresses">
412      <title>Overview</title>
413
414      <sect3 id="Basics">
415            <title>Basics</title>
416        <para>For a variety of technical reasons, it isn't generally
417          possible to directly reference arbitrary absolute addresses
418          (such as those returned by the C library function malloc(),
419          for instance) in &CCL;. In &CCL; (and in MCL), such
420          addresses need to be <emphasis>encapsulated</emphasis> in
421          objects of type CCL:MACPTR; one can think of a MACPTR as
422          being a specialized type of structure whose sole purpose is
423          to provide a way of referring to an underlying "raw"
424          address.</para>
425        <para>It's sometimes convenient to blur the distinction
426          between a MACPTR and the address it represents; it's
427          sometimes necessary to maintain that distinction. It's
428          important to remember that a MACPTR is (generally) a
429          first-class Lisp object in the same sense that a CONS cell
430          is: it'll get GCed when it's no longer possible to reference
431          it. The "lifetime" of a MACPTR doesn't generally have
432          anything to do with the lifetime of the block of memory its
433          address points to.</para>
434        <para>It might be tempting to ask "How does one obtain the
435          address encapsulated by a MACPTR ?". The answer to that
436          question is that one doesn't do that (and there's no way to
437          do that): addresses aren't first-class objects, and there's
438          no way to refer to one.</para>
439        <para>Two MACPTRs that encapsulate the same address are EQL
440          to each other.</para>
441        <para>There are a small number of ways to directly create a
442          MACPTR (and there's a fair amount of syntactic sugar built
443          on top of of those primitives.) These primitives will be
444          discussed in greater detail below, but they include:</para>
445
446            <itemizedlist>
447          <listitem>
448                <para>Creating a MACPTR with a specified address, usually
449                  via the function CCL:%INT-TO-PTR.</para>
450              </listitem>
451          <listitem>
452                <para>Referencing the return value of a foreign function
453                  call (see )that's specified to return an address.</para>
454              </listitem>
455          <listitem>
456                <para>Referencing a memory location that's specified to
457                  contain an address.</para>
458              </listitem>
459            </itemizedlist>
460
461        <para>All of these primitive MACPTR-creating operations are
462          usually open-coded by the compiler; it has a fairly good
463          notion of what low-level operations "produce" MACPTRs and
464          which operations "consume" the addresses that the
465          encapsulate, and will usually optimize out the introduction
466          of intermediate MACPTRs in a simple expression.</para>
467        <para>One consequence of the use of MACPTR objects to
468          encapsulate foreign addresses is that (naively)
469          <emphasis>every reference to a foreign address causes a
470            MACPTR to be allocated.</emphasis></para>
471        <para>Consider a code fragment like the following:</para>
472        <programlisting>
473(defun get-next-event ()
474  "get the next event from a hypothetical window system"
475  (loop
476     (let* ((event (#_get_next_window_system_event))) ; via an FF-CALL
477       (unless (null-event-p event)
478         (handle-event event)))))
479        </programlisting>
480        <para>As this is written, each call to the (hypothetical)
481          foreign function #_get_next_window_system_event will return
482          a new MACPTR object.  Ignoring for the sake of argument the
483          question of whether this code fragment exhibits a good way
484          to poll for external events (it doesn't), it's not hard to
485          imagine that this loop could execute several million times
486          per second (producing several million MACPTRs per second.)
487          Clearly, the "naive" approach is impractical in many
488          cases.</para>
489      </sect3>
490
491      <sect3 id="Stack-allocation-of---and-destructive-operations-on---MACPTRs-">
492        <title>Stack allocation of&mdash;and destructive operations on&mdash;MACPTRs.</title>
493            <para>If certain conditions held in the environment in which
494              GET-NEXT-EVENT ran&mdash;namely, if it was guaranteed that
495              neither NULL-EVENT-P nor HANDLE-EVENT cached or otherwise
496              retained their arguments (the "event" pointer)&mdash;there'd be
497              a few alternatives to the naive approach. One of those
498              approaches would be to use the primitive function
499              %SETF-MACPTR (described in greater detail below) to
500              destructively modify a MACPTR (to change the value of the
501              address it encapsulates.) The GET-NEXT-EVENT example could
502              be re-written as:</para>
503        <programlisting>
504(defun get-next-event ()
505  (let* ((event (%int-to-ptr 0)))     ; create a MACPTR with address 0
506    (loop
507       (%setf-macptr event (#_get_next_window_system_event)) ; re-use it
508       (unless (null-event-p event)
509         (handle-event event)))))
510        </programlisting>
511        <para>That version's a bit more realistic: it allocates a
512          single MACPTR outside if the loop, then changes its address
513          to point to the current address of the hypothetical event
514          structure on each loop iteration. If there are a million
515          loop iterations per call to GET-NEXT-EVENT, we're allocating
516          a million times fewer MACPTRs per call; that sounds like a
517          Good Thing.</para>
518        <para>An Even Better Thing would be to advise the compiler
519          that the initial value (the null MACPTR) bound to the
520          variable event has dynamic extent (that value won't be
521          referenced once control leaves the extent of the binding of
522          that variable.) Common Lisp allows us to make such an
523          assertion via a DYNAMIC-EXTENT declaration; &CCL;'s
524          compiler can recognize the "primitive MACPTR-creating
525          operation" involved and can replace it with an equivalent
526          operation that stack-allocates the MACPTR object. If we're
527          not worried about the cost of allocating that MACPTR on
528          every iteration (the cost is small and there's no hidden GC
529          cost), we could move the binding back inside the
530          loop:</para>
531        <programlisting>
532(defun get-next-event ()
533  (loop
534     (let* ((event (%null-ptr))) ; (%NULL-PTR) is shorthand for (%INT-TO-PTR 0)
535       (declare (dynamic-extent event))
536       (%setf-macptr event (#_get_next_window_system_event))
537       (unless (null-event-p event)
538         (handle-event event)))))
539        </programlisting>
540        <para>The idiom of binding one or more variables to
541          stack-allocated MACPTRs, then destructively modifying those
542          MACPTRs before executing a body of code is common enough
543          that &CCL; provides a macro (WITH-MACPTRS) that handles
544          all of the gory details. The following version of
545          GET-NEXT-EVENT is semantically equivalent to the previous
546          version, but hopefully a bit more concise:</para>
547        <programlisting>
548(defun get-next-event ()
549  (loop
550     (with-macptrs ((event (#_get_next_window_system_event)))
551       (unless (null-event-p event)
552         (handle-event event)))))
553        </programlisting>
554      </sect3>
555
556      <sect3 id="Stack-allocated-memory--and-stack-allocated-pointers-to-it--">
557        <title>Stack-allocated memory (and stack-allocated pointers to it.)</title>
558            <para>Fairly often, the blocks of foreign memory (obtained
559              by malloc or something similar) have well-defined lifetimes
560              (they can safely be freed at some point when it's known that
561              they're no longer needed and it's known that they're no
562              longer referenced.) A common idiom might be:</para>
563        <programlisting>
564(with-macptrs (p (#_allocate_foreign_memory size))
565  (unwind-protect
566       (use-foreign-memory p)
567    (#_deallocate_foreign_memory p)))
568        </programlisting>
569        <para>That's not unreasonable code, but it's fairly
570          expensive for a number of reasons: foreign functions calls
571          are themselves fairly expensive (as is UNWIND-PROTECT), and
572          most library routines for allocating and deallocating
573          foreign memory (things like malloc and free) can be fairly
574          expensive in their own right.</para>
575        <para>In the idiomatic code above, both the MACPTR P and the
576          block of memory that's being allocated and freed have
577          dynamic extent and are therefore good candidates for stack
578          allocation. &CCL; provides the %STACK-BLOCK macro, which
579          executes a body of code with one or more variables bound to
580          stack-allocated MACPTRs which encapsulate the addresses of
581          stack-allocated blocks of foreign memory. Using
582          %STACK-BLOCK, the idiomatic code is:</para>
583        <programlisting>
584(%stack-block ((p size))
585              (use-foreign-memory p))
586        </programlisting>
587        <para>which is a bit more efficient and a bit more concise
588          than the version presented earlier.</para>
589        <para>%STACK-BLOCK is used as the basis for slightly
590          higher-level things like RLET. (See FIXTHIS for more information
591          about RLET.)</para>
592      </sect3>
593
594      <sect3 id="Caveats-">
595            <title>Caveats</title>
596        <para>Reading from, writing to, allocating, and freeing
597          foreign memory are all potentially dangerous operations;
598          this is no less true when these operations are performed in
599          &CCL; than when they're done in C or some other
600          lower-level language. In addition, destructive operations on
601          Lisp objects be dangerous, as can stack allocation if it's
602          abused (if DYNAMIC-EXTENT declarations are violated.)
603          Correct use of the constructs and primitives described here
604          is reliable and safe; slightly incorrect use of these
605          constructs and primitives can crash &CCL;.</para>
606      </sect3>
607    </sect2>
608
609    <sect2 id="Foreign-Memory-Addresses-Dictionary">
610          <title>Foreign-Memory-Addresses Dictionary</title>
611      <para>Unless otherwise noted, all of the symbols mentioned
612        below are exported from the CCL package.</para>
613
614      <sect3 id="Scalar-memory-reference">
615        <title>Scalar memory reference</title>
616            <variablelist>
617              <varlistentry>
618                <term>Syntax</term>
619
620                <listitem>
621                      <para>%get-signed-byte ptr &#38;optional (offset 0)</para>
622
623                      <para>%get-unsigned-byte ptr &#38;optional (offset 0)</para>
624
625                      <para>%get-signed-word ptr &#38;optional (offset 0)</para>
626
627                      <para>%get-unsigned-word ptr &#38;optional (offset 0)</para>
628
629                      <para>%get-signed-long ptr &#38;optional (offset 0)</para>
630
631                      <para>%get-unsigned-long ptr &#38;optional (offset 0)</para>
632
633                      <para>%%get-signed-longlong ptr &#38;optional (offset 0)</para>
634
635                      <para>%%get-unsigned-longlong ptr &#38;optional (offset 0)</para>
636
637                      <para>%get-ptr ptr &#38;optional (offset 0)</para>
638
639                      <para>%get-single-float ptr &#38;optional (offset 0)</para>
640
641                      <para>%get-double-float ptr &#38;optional (offset 0)</para>
642                </listitem>
643              </varlistentry>
644              <varlistentry>
645                <term>Description</term>
646
647                <listitem>
648                      <para>References and returns the signed or unsigned 8-bit byte,
649                        signed or unsigned 16-bit word, signed or unsigned 32-bit long
650                        word, signed or unsigned 64-bit long long word, 32-bit address,
651                        32-bit single-float, or 64-bit double-float at the effective byte
652                        address formed by adding offset to the address encapsulated by
653                        ptr.</para>
654                </listitem>
655              </varlistentry>
656
657              <varlistentry>
658                <term>Arguments</term>
659
660                <listitem>
661
662                      <variablelist>
663                        <varlistentry>
664                          <term>ptr</term>
665
666                          <listitem>
667                            <para>A MACPTR</para>
668                          </listitem>
669                        </varlistentry>
670
671                        <varlistentry>
672                          <term>offset</term>
673
674                          <listitem>
675                            <para>A fixnum</para>
676                          </listitem>
677                        </varlistentry>
678                      </variablelist>
679                </listitem>
680              </varlistentry>
681            </variablelist>
682
683        <para>All of the memory reference primitives described above can be</para>
684        <para>used with SETF.</para>
685      </sect3>
686
687      <sect3 id="iget-bit--Function-">
688            <title>%get-bit [Function]</title>
689
690            <variablelist>
691              <varlistentry>
692                <term>Syntax</term>
693
694                <listitem>
695                      <para>%get-bit ptr bit-offset</para>
696                </listitem>
697              </varlistentry>
698
699              <varlistentry>
700                <term>Description</term>
701
702                <listitem>
703                      <para>References and returns the bit-offsetth bit at the address
704                        encapsulated by ptr. (Bit 0 at a given address is the most
705                        significant bit of the byte at that address.) Can be used with
706                        SETF.</para>
707                </listitem>
708              </varlistentry>
709
710              <varlistentry>
711                <term>Arguments</term>
712
713                <listitem>
714                      <para>&#x00A0;</para>
715
716                      <variablelist>
717                        <varlistentry>
718                          <term>ptr</term>
719
720                          <listitem>
721                            <para>A MACPTR</para>
722                          </listitem>
723                        </varlistentry>
724
725                        <varlistentry>
726                          <term>bit-offset</term>
727
728                          <listitem>
729                            <para>A fixnum</para>
730                          </listitem>
731                        </varlistentry>
732                      </variablelist>
733                </listitem>
734              </varlistentry>
735            </variablelist>
736      </sect3>
737
738      <sect3 id="iget-bitfield--Function-">
739        <title>%get-bitfield [Function]</title>
740            <variablelist>
741              <varlistentry>
742                <term>Syntax</term>
743
744                <listitem>
745                      <para>%get-bitfield ptr bit-offset width</para>
746                </listitem>
747              </varlistentry>
748
749              <varlistentry>
750                <term>Description</term>
751
752                <listitem>
753                      <para>References and returns an unsigned integer composed from the
754                        width bits found bit-offset bits from the address encapsulated by
755                        ptr. (The least significant bit of the result is the value of
756                        (%get-bit ptr (1- (+ bit-offset width))). Can be used with SETF.</para>
757                </listitem>
758              </varlistentry>
759
760              <varlistentry>
761                <term>Arguments</term>
762
763                <listitem>
764                      <para>&#x00A0;</para>
765                </listitem>
766              </varlistentry>
767
768              <varlistentry>
769                <term>ptr</term>
770
771                <listitem>
772                      <para>A MACPTR</para>
773
774                      <variablelist>
775                        <varlistentry>
776                          <term>bit-offset</term>
777
778                          <listitem>
779                            <para>A fixnum</para>
780                          </listitem>
781                        </varlistentry>
782
783                        <varlistentry>
784                          <term>width</term>
785
786                          <listitem>
787                            <para>A positive fixnum</para>
788                          </listitem>
789                        </varlistentry>
790                      </variablelist>
791                </listitem>
792              </varlistentry>
793            </variablelist>
794      </sect3>
795
796          <sect3>
797            <title>%int-to-ptr [Function]</title>
798
799            <variablelist>
800              <varlistentry>
801                <term>Syntax</term>
802
803                <listitem>
804                      <para>%int-to-ptr int</para>
805                </listitem>
806              </varlistentry>
807
808              <varlistentry>
809                <term>Description</term>
810
811                <listitem>
812                      <para>Creates and returns a MACPTR whose address matches int.</para>
813                </listitem>
814              </varlistentry>
815
816              <varlistentry>
817                <term>Arguments</term>
818
819                <listitem>
820                      <para>&#x00A0;</para>
821
822                      <variablelist>
823                        <varlistentry>
824                          <term>int</term>
825
826                          <listitem>
827                            <para>An (unsigned-byte 32)</para>
828                          </listitem>
829                        </varlistentry>
830                      </variablelist>
831                </listitem>
832              </varlistentry>
833            </variablelist>
834          </sect3>
835
836          <sect3>
837            <title>%inc-ptr [Function]</title>
838
839            <variablelist>
840              <varlistentry>
841                <term>Syntax</term>
842
843                <listitem>
844                      <para>%inc-ptr ptr &#38;optional (delta 1)</para>
845                </listitem>
846              </varlistentry>
847
848              <varlistentry>
849                <term>Description</term>
850
851                <listitem>
852                      <para>Creates and returns a MACPTR whose address is the address of
853                        ptr plus delta. The idiom (%inc-ptr ptr 0) is sometimes used to
854                        copy a MACPTR, e.g., to create a new MACPTR encapsulating the same
855                        address as ptr.</para>
856                </listitem>
857              </varlistentry>
858
859              <varlistentry>
860                <term>Arguments</term>
861
862                <listitem>
863                      <para>&#x00A0;</para>
864
865                      <variablelist>
866                        <varlistentry>
867                          <term>ptr</term>
868
869                          <listitem>
870                            <para>A MACPTR</para>
871                          </listitem>
872                        </varlistentry>
873
874                        <varlistentry>
875                          <term>delta</term>
876
877                          <listitem>
878                            <para>A fixnum</para>
879                          </listitem>
880                        </varlistentry>
881                      </variablelist>
882                </listitem>
883              </varlistentry>
884            </variablelist>
885          </sect3>
886
887          <sect3>
888            <title>%ptr-to-int [Function]</title>
889
890            <variablelist>
891              <varlistentry>
892                <term>Syntax</term>
893
894                <listitem>
895                      <para>%ptr-to-int ptr</para>
896                </listitem>
897              </varlistentry>
898
899              <varlistentry>
900                <term>Description</term>
901
902                <listitem>
903                      <para>Returns the address encapsulated by ptr, as an
904                        (unsigned-byte 32).</para>
905                </listitem>
906              </varlistentry>
907
908              <varlistentry>
909                <term>Arguments</term>
910
911                <listitem>
912                      <para>&#x00A0;</para>
913
914                      <variablelist>
915                        <varlistentry>
916                          <term>ptr</term>
917
918                          <listitem>
919                            <para>A MACPTR</para>
920                          </listitem>
921                        </varlistentry>
922                      </variablelist>
923                </listitem>
924              </varlistentry>
925            </variablelist>
926          </sect3>
927
928          <sect3>
929            <title>%null-ptr [Macro]</title>
930
931            <variablelist>
932              <varlistentry>
933                <term>Syntax</term>
934
935                <listitem>
936                      <para>%null-ptr</para>
937                </listitem>
938              </varlistentry>
939
940              <varlistentry>
941                <term>Description</term>
942
943                <listitem>
944                      <para>Equivalent to (%ptr-to-int 0).</para>
945                </listitem>
946              </varlistentry>
947            </variablelist>
948          </sect3>
949
950          <sect3>
951            <title>%null-ptr-p [Function]</title>
952
953            <variablelist>
954              <varlistentry>
955                <term>Syntax</term>
956
957                <listitem>
958                      <para>%null-ptr-p ptr</para>
959                </listitem>
960              </varlistentry>
961
962              <varlistentry>
963                <term>Description</term>
964
965                <listitem>
966                      <para>Returns T If ptr is a MACPTR encapsulating the address 0,
967                        NIL if ptr encapsulates some other address.</para>
968                </listitem>
969              </varlistentry>
970
971              <varlistentry>
972                <term>Arguments</term>
973
974                <listitem>
975                      <para>&#x00A0;</para>
976
977                      <variablelist>
978                        <varlistentry>
979                          <term>ptr</term>
980
981                          <listitem>
982                            <para>A MACPTR</para>
983                          </listitem>
984                        </varlistentry>
985                      </variablelist>
986                </listitem>
987              </varlistentry>
988            </variablelist>
989          </sect3>
990
991          <sect3>
992            <title>%setf-macptr [Function]</title>
993
994            <variablelist>
995              <varlistentry>
996                <term>Syntax</term>
997
998                <listitem>
999                      <para>%setf-macptr dest-ptr src-ptr</para>
1000                </listitem>
1001              </varlistentry>
1002
1003              <varlistentry>
1004                <term>Description</term>
1005
1006                <listitem>
1007                      <para>Causes dest-ptr to encapsulate the same address that src-ptr
1008                        does, then returns dest-ptr.</para>
1009                </listitem>
1010              </varlistentry>
1011
1012              <varlistentry>
1013                <term>Arguments</term>
1014
1015                <listitem>
1016                      <para>&#x00A0;</para>
1017
1018                      <variablelist>
1019                        <varlistentry>
1020                          <term>dest-ptr</term>
1021
1022                          <listitem>
1023                            <para>A MACPTR</para>
1024                          </listitem>
1025                        </varlistentry>
1026
1027                        <varlistentry>
1028                          <term>src-ptr</term>
1029
1030                          <listitem>
1031                            <para>A MACPTR</para>
1032                          </listitem>
1033                        </varlistentry>
1034                      </variablelist>
1035                </listitem>
1036              </varlistentry>
1037            </variablelist>
1038          </sect3>
1039
1040          <sect3>
1041            <title>%incf-ptr [Macro]</title>
1042
1043            <variablelist>
1044              <varlistentry>
1045                <term>Syntax</term>
1046
1047                <listitem>
1048                      <para>%incf-ptr ptr &#38;optional (delta 1)</para>
1049                </listitem>
1050              </varlistentry>
1051
1052              <varlistentry>
1053                <term>Description</term>
1054
1055                <listitem>
1056                      <para>Destructively modifies ptr, by adding delta to the address
1057                        it encapsulates. Returns ptr.</para>
1058                </listitem>
1059              </varlistentry>
1060
1061              <varlistentry>
1062                <term>Arguments</term>
1063
1064                <listitem>
1065                      <para>&#x00A0;</para>
1066
1067                      <variablelist>
1068                        <varlistentry>
1069                          <term>ptr</term>
1070
1071                          <listitem>
1072                            <para>A MACPTR</para>
1073                          </listitem>
1074                        </varlistentry>
1075
1076                        <varlistentry>
1077                          <term>delta</term>
1078
1079                          <listitem>
1080                            <para>A fixnum</para>
1081                          </listitem>
1082                        </varlistentry>
1083                      </variablelist>
1084                </listitem>
1085              </varlistentry>
1086            </variablelist>
1087          </sect3>
1088
1089          <sect3>
1090            <title>with-macptrs [Macro]</title>
1091
1092            <variablelist>
1093              <varlistentry>
1094                <term>Syntax</term>
1095
1096                <listitem>
1097                      <para>with-macptrs (var expr)* &#38;body body</para>
1098                </listitem>
1099              </varlistentry>
1100
1101              <varlistentry>
1102                <term>Description</term>
1103
1104                <listitem>
1105                      <para>Executes body in an environment in which each var is bound
1106                        to a stack-allocated macptr which encapsulates the foreign address
1107                        yielded by the corresponding expr. Returns whatever value(s) body
1108                        returns.</para>
1109                </listitem>
1110              </varlistentry>
1111
1112              <varlistentry>
1113                <term>Arguments</term>
1114
1115                <listitem>
1116                      <para>&#x00A0;</para>
1117
1118                      <variablelist>
1119                        <varlistentry>
1120                          <term>var</term>
1121
1122                          <listitem>
1123                            <para>A symbol (variable name)</para>
1124                          </listitem>
1125                        </varlistentry>
1126
1127                        <varlistentry>
1128                          <term>expr</term>
1129
1130                          <listitem>
1131                            <para>A MACPTR-valued expression</para>
1132                          </listitem>
1133                        </varlistentry>
1134                      </variablelist>
1135                </listitem>
1136              </varlistentry>
1137            </variablelist>
1138          </sect3>
1139
1140          <sect3>
1141            <title>%stack-block [Macro]</title>
1142
1143            <variablelist>
1144              <varlistentry>
1145                <term>Syntax</term>
1146
1147                <listitem>
1148                      <para>%stack-block (var expr)* &#38;body body</para>
1149                </listitem>
1150              </varlistentry>
1151
1152              <varlistentry>
1153                <term>Description</term>
1154
1155                <listitem>
1156                      <para>Executes body in an environment in which each var is bound
1157                        to a stack-allocated macptr which encapsulates the address of a
1158                        stack-allocated region of size expr bytes. Returns whatever
1159                        value(s) body returns.</para>
1160                </listitem>
1161              </varlistentry>
1162
1163              <varlistentry>
1164                <term>Arguments</term>
1165
1166                <listitem>
1167                      <para>&#x00A0;</para>
1168
1169                      <variablelist>
1170                        <varlistentry>
1171                          <term>var</term>
1172
1173                          <listitem>
1174                            <para>A symbol (variable name)</para>
1175                          </listitem>
1176                        </varlistentry>
1177
1178                        <varlistentry>
1179                          <term>expr</term>
1180
1181                          <listitem>
1182                            <para>An expression which should evaluate to a non-negative
1183                              fixnum</para>
1184                          </listitem>
1185                        </varlistentry>
1186                      </variablelist>
1187                </listitem>
1188              </varlistentry>
1189            </variablelist>
1190          </sect3>
1191
1192          <sect3>
1193            <title>make-cstring [Function]</title>
1194
1195            <variablelist>
1196              <varlistentry>
1197                    <term>Syntax</term>
1198
1199                    <listitem>
1200                      <para>make-cstring string</para>
1201                    </listitem>
1202              </varlistentry>
1203
1204              <varlistentry>
1205                    <term>Description</term>
1206
1207                    <listitem>
1208                      <para>Allocates a block of memory (via malloc) of length (1+
1209                        (length string)). Copies the string to this block and appends a
1210                        trailing NUL byte; returns a MACPTR to the block.</para>
1211                    </listitem>
1212              </varlistentry>
1213
1214              <varlistentry>
1215                    <term>Arguments</term>
1216
1217                    <listitem>
1218                      <para>&#x00A0;</para>
1219
1220                      <variablelist>
1221                        <varlistentry>
1222                          <term>string</term>
1223
1224                          <listitem>
1225                                <para>A lisp string</para>
1226                          </listitem>
1227                        </varlistentry>
1228                      </variablelist>
1229                    </listitem>
1230              </varlistentry>
1231            </variablelist>
1232          </sect3>
1233
1234          <sect3>
1235            <title>with-cstrs [Macro]</title>
1236
1237            <variablelist>
1238              <varlistentry>
1239                <term>Syntax</term>
1240
1241                <listitem>
1242                      <para>with-cstrs (var string)* &#38;body body</para>
1243                </listitem>
1244              </varlistentry>
1245
1246              <varlistentry>
1247                <term>Description</term>
1248
1249                <listitem>
1250                      <para>Executes body in an environment in which each var is bound
1251                        to a stack-allocated macptr which encapsulates the %address of a
1252                        stack-allocated region of into which each string (and a trailing
1253                        NUL byte) has been copied. Returns whatever value(s) body returns.</para>
1254                </listitem>
1255              </varlistentry>
1256
1257              <varlistentry>
1258                <term>Arguments</term>
1259
1260                <listitem>
1261                      <para>&#x00A0;</para>
1262
1263                      <variablelist>
1264                        <varlistentry>
1265                          <term>var</term>
1266
1267                          <listitem>
1268                            <para>A symbol (variable name)</para>
1269                          </listitem>
1270                        </varlistentry>
1271
1272                        <varlistentry>
1273                          <term>string</term>
1274
1275                          <listitem>
1276                            <para>An expression which should evaluate to a lisp string</para>
1277                          </listitem>
1278                        </varlistentry>
1279                      </variablelist>
1280                </listitem>
1281              </varlistentry>
1282            </variablelist>
1283          </sect3>
1284
1285      <sect3>
1286        <title>with-encoded-cstrs [Macro]</title>
1287
1288        <variablelist>
1289              <varlistentry>
1290                <term>Syntax</term>
1291
1292                <listitem>
1293                      <para>with-encoded-cstrs ENCODING-NAME (varI stringI)* &#38;body body</para>
1294                </listitem>
1295              </varlistentry>
1296
1297              <varlistentry>
1298                <term>Description</term>
1299
1300                <listitem>
1301                      <para>Executes body in an environment in which each varI is
1302                        bound to a macptr which encapsulates the %address of a
1303                        stack-allocated region of into which each stringI (and a
1304                        trailing NUL character) has been copied. Returns whatever
1305                        value(s) body returns.</para>
1306
1307              <para>ENCODING-NAME is a keyword constant that names a
1308                character encoding. Each foreign string is encoded in the
1309                named encoding. Each foreign string has dynamic
1310                extent.</para>
1311
1312              <para>WITH-ENCODED-CSTRS does not automatically prepend
1313                byte-order marks to its output; the size of the terminating
1314                #\NUL character depends on the number of octets per code unit
1315                in the encoding.</para>
1316
1317              <para>The expression</para>
1318
1319              <programlisting>(ccl:with-cstrs ((x "x")) (#_puts x))</programlisting>
1320
1321              <para>is functionally equivalent to</para>
1322
1323              <programlisting>(ccl:with-encoded-cstrs :iso-8859-1 ((x "x")) (#_puts x))</programlisting>
1324                </listitem>
1325              </varlistentry>
1326
1327              <varlistentry>
1328                <term>Arguments</term>
1329
1330                <listitem>
1331                      <para>&#x00A0;</para>
1332
1333                      <variablelist>
1334                        <varlistentry>
1335                          <term>varI</term>
1336
1337                          <listitem>
1338                            <para>A symbol (variable name)</para>
1339                          </listitem>
1340                        </varlistentry>
1341
1342                        <varlistentry>
1343                          <term>stringI</term>
1344
1345                          <listitem>
1346                            <para>An expression which should evaluate to a lisp string</para>
1347                          </listitem>
1348                        </varlistentry>
1349                      </variablelist>
1350                </listitem>
1351              </varlistentry>
1352        </variablelist>
1353      </sect3>
1354
1355          <sect3>
1356            <title>%get-cstring [Function]</title>
1357
1358            <variablelist>
1359              <varlistentry>
1360                <term>Syntax</term>
1361
1362                <listitem>
1363                      <para>%get-cstring ptr</para>
1364                </listitem>
1365              </varlistentry>
1366
1367              <varlistentry>
1368                <term>Description</term>
1369
1370                <listitem>
1371                      <para>Interprets ptr as a pointer to a (NUL -terminated) C string;
1372                        returns an equivalent lisp string.</para>
1373                </listitem>
1374              </varlistentry>
1375
1376              <varlistentry>
1377                <term>Arguments</term>
1378
1379                <listitem>
1380                      <para></para>
1381
1382                      <variablelist>
1383                        <varlistentry>
1384                          <term>ptr</term>
1385
1386                          <listitem>
1387                            <para>A MACPTR</para>
1388                          </listitem>
1389                        </varlistentry>
1390                      </variablelist>
1391                </listitem>
1392              </varlistentry>
1393            </variablelist>
1394          </sect3>
1395
1396          <sect3>
1397            <title>%str-from-ptr [Function]</title>
1398
1399            <variablelist>
1400              <varlistentry>
1401                <term>Syntax</term>
1402
1403                <listitem>
1404                      <para>%str-from-ptr ptr length</para>
1405                </listitem>
1406              </varlistentry>
1407
1408              <varlistentry>
1409                <term>Description</term>
1410
1411                <listitem>
1412                      <para>Returns a lisp string of length <varname>length</varname>,
1413                        whose contents are initialized from the bytes at<varname> ptr.</varname>
1414                      </para>
1415                </listitem>
1416              </varlistentry>
1417
1418              <varlistentry>
1419                <term>Arguments</term>
1420
1421                <listitem>
1422                      <para><variablelist><varlistentry><term>ptr</term><listitem><para>A
1423                                MACPTR</para></listitem></varlistentry><varlistentry><term>length</term><listitem><para>a
1424                                non-negative fixnum</para></listitem></varlistentry></variablelist></para>
1425                </listitem>
1426              </varlistentry>
1427            </variablelist>
1428          </sect3>
1429    </sect2>
1430  </sect1>
1431
1432  <!-- ******************************************  -->
1433  <sect1 id="The-Interface-Database">
1434    <title>The Interface Database</title>
1435
1436    <sect2 id="interface-database-Overview">
1437          <title>Overview</title>
1438      <para>&CCL; uses a set of database files which contain
1439        foreign type, record, constant, and function definitions
1440        derived from the operating system's header files, be that
1441        Linux or Darwin.  An archive containing these database files
1442        (and the shell scripts which were used in their creation) is
1443        available; see the Distributions page for information about
1444        obtaining current interface database files.</para>
1445      <para>Not surprisingly, different platforms use different database files.</para>
1446      <para>&CCL; defines reader macros that consult these databases:</para>
1447          <itemizedlist>
1448        <listitem>
1449              <para>#$foo looks up the value of the constant definition of foo</para>
1450            </listitem>
1451            <listitem>
1452              <para>#_foo looks up the foreign function definition for foo</para>
1453            </listitem>
1454       
1455      </itemizedlist>
1456      <para>In both cases, the symbol foo is interned in the "OS"
1457        package. The #$ reader macro has the side-effect of defining
1458        foo as a constant (as if via DEFCONSTANT); the #_ reader macro
1459        has the side effect of defining foo as a macro which will
1460        expand into an (EXTERNAL-CALL form.)</para>
1461      <para>It's important to remember that the side-effect happens
1462        when the form containing the reader macro is
1463        read. Macroexpansion functions that expand into forms which
1464        contain instances of those reader macros don't do what one
1465        might think that they do, unless the macros are expanded in
1466        the same lisp session as the reader macro was read in.</para>
1467      <para>In addition, references to foreign type,
1468        structure/union, and field names (when used in the RREF/PREF
1469        and RLET macros) will cause these database files to be
1470        consulted.</para>
1471      <para>Since the &CCL; sources contain instances of these
1472        reader macros (and references to foreign record types and
1473        fields), compiling &CCL; from those sources depends on the
1474        ability to find and use (see <xref
1475                                        linkend="Building-the-heap-image"/>).</para>
1476    </sect2>
1477
1478    <sect2 id="Other-issues">
1479      <title>Other issues:</title>
1480      <itemizedlist>
1481        <listitem>
1482              <para>&CCL; now preserves the case of external symbols in
1483                its database
1484                files. See <link linkend="Case-sensitivity-of-foreign-names-in-CCL">Case-sensitivity
1485                of foreign names in &CCL;</link> for information about
1486                case in foreign symbol names.</para>
1487            </listitem>
1488            <listitem>
1489              <para>The Linux databases are derived from a somewhat
1490                arbitrary set of Linux header files. Linux is enough of a
1491                moving target that it may be difficult to define a standard,
1492                reference set of interfaces from which to derive a standard,
1493                reference set of database files.This seems to be less of
1494                an issue with Darwin and FreeBSD.</para>
1495            </listitem>
1496          </itemizedlist>
1497      <para>For information about building the database files,
1498            see <xref linkend="The-Interface-Translator"/>.</para>
1499    </sect2>
1500  </sect1>
1501
1502  <!-- ******************************************  -->
1503  <sect1 id="Using-Interface-Directories">
1504    <title>Using Interface Directories</title>
1505
1506    <sect2 id="Interface-Directory-Overview">
1507          <title>Overview</title>
1508      <para>As distributed, the "ccl:headers;" (for LinuxPPC)
1509        directory is organized like:</para>
1510      <programlisting>
1511        headers/
1512        headers/gl/
1513        headers/gl/C/
1514        headers/gl/C/populate.sh
1515        headers/gl/constants.cdb
1516        headers/gl/functions.cdb
1517        headers/gl/records.cdb
1518        headers/gl/objc-classes.cdb
1519        headers/gl/objc-methods.cdb
1520        headers/gl/types.cdb
1521        headers/gnome/
1522        headers/gnome/C/
1523        headers/gnome/C/populate.sh
1524        headers/gnome/constants.cdb
1525        headers/gnome/functions.cdb
1526        headers/gnome/records.cdb
1527        headers/gnome/objc-classes.cdb
1528        headers/gnome/objc-methods.cdb
1529        headers/gnome/types.cdb
1530        headers/gtk/
1531        headers/gtk/C/
1532        headers/gtk/C/populate.sh
1533        headers/gtk/constants.cdb
1534        headers/gtk/functions.cdb
1535        headers/gtk/records.cdb
1536        headers/gtk/objc-classes.cdb
1537        headers/gtk/objc-methods.cdb
1538        headers/gtk/types.cdb
1539        headers/libc/
1540        headers/libc/C/
1541        headers/libc/C/populate.sh
1542        headers/libc/constants.cdb
1543        headers/libc/functions.cdb
1544        headers/libc/records.cdb
1545        headers/libc/objc-classes.cdb
1546        headers/libc/objc-methods.cdb
1547        headers/libc/types.cdb
1548      </programlisting>
1549      <para>e.g, as a set of parallel subdirectories, each with a
1550        lowercase name and each of which contains a set of 6 database
1551        files and a "C" subdirectory which contains a shell script
1552        used in the database creation process.</para>
1553      <para>As one might assume, the database files in each of these
1554        subdirectories contain foreign type, constant, and function
1555        definitions - as well as Objective-C class and method info -that
1556        correspond (roughly) to the information contained in the
1557        header files associated with a "-dev" package in a Linux
1558        distribution.  "libc" corresponds pretty closely to the
1559        interfaces associated with "glibc/libc6" header files, "gl"
1560        corresponds to an "openGL+GLUT" development package, "gtk"
1561        and "gnome" contain interface information from the GTK+1.2 and
1562        GNOME libraries, respectively.</para>
1563      <para>For Darwin, the "ccl:darwin-headers" directory contains
1564        a "libc" subdirectory, whose contents roughly correspond to
1565        those of "/usr/include" under Darwin, as well as
1566        subdirectories corresponding to the MacOSX Carbon and Cocoa
1567        frameworks.</para>
1568      <para>To see the precise set of .h files used to generate the
1569        database files in a given interface directory, consult the
1570        corresponding "populate.sh" shell script (in the interface
1571        directory's "C" subdirectory.)</para>
1572      <para>The intent is that this initial set can be augmented to
1573        meet local needs, and that this can be done in a fairly
1574        incremental fashion: one needn't have unrelated header files
1575        installed in order to generate interface databases for a
1576        package of interest.</para>
1577      <para>Hopefully, this scheme will also make it easier to
1578        distribute patches and bug fixes.</para>
1579      <para>&CCL; maintains a list of directories; when looking
1580        for a foreign type, constant, function, or record definition,
1581        it'll consult the database files in each directory on that
1582        list. Initially, the list contains an entry for the "libc"
1583        interface directory. &CCL; needs to be explicitly told to
1584        look in other interface directories should it need to do
1585        so.</para>
1586    </sect2>
1587
1588    <sect2 id="Creating-new-interface-directories">
1589          <title>Creating new interface directories</title>
1590      <para>This example refers to "ccl:headers;", which is
1591        appropriate for LinuxPPC. The procedure's analogous under
1592        Darwin, where the "ccl:darwin-headers;" directory would be
1593        used instead.</para>
1594      <para>To create a new interface directory, "foo", and a set of
1595        database files in that directory:</para>
1596          <orderedlist continuation="restarts" inheritnum="ignore">
1597            <listitem>
1598              <para>Create a subdirectory of &#34;ccl:headers;&#34; named
1599                &#34;foo&#34;.</para>
1600            </listitem>
1601
1602            <listitem>
1603              <para>Create a subdirectory of &#34;ccl:headers;foo;&#34; named
1604                &#34;C&#34;.</para>
1605            </listitem>
1606
1607            <listitem>
1608              <para>Create a file in &#34;ccl:headers;foo;C;&#34; named
1609                &#34;populate.sh&#34;.</para>
1610
1611              <para>One way of accomplishing the above steps is:</para>
1612
1613              <programlisting format="linespecific">
1614            ? (close (open &#34;ccl:headers;foo;C;populate.sh&#34; :direction :output :
1615                           if-does-not-exist :create :if-exists :overwrite))
1616          </programlisting>
1617            </listitem>
1618
1619            <listitem>
1620              <para>Edit the file created above, using the &#34;populate.sh&#34;
1621                files in the distribution as guidelines.</para>
1622
1623              <para>The file might wind up looking something like:</para>
1624
1625              <programlisting format="linespecific">#/bin/sh
1626            h-to-ffi.sh `foo-config -cflags` /usr/include/foo/foo.h</programlisting>
1627            </listitem>
1628          </orderedlist>
1629
1630      <para>Refer to <xref linkend="The-Interface-Translator"/> for
1631        information about running the interface translator and .ffi
1632        parser.</para>
1633      <para>Assuming that all went well, there should now be .cdb
1634        files in "ccl:headers;foo;". You can then do
1635        <programlisting>
1636          ? (use-interface-dir :foo)
1637            </programlisting> 
1638            whenever you need to
1639        access the foreign type information in those database
1640        files.</para>
1641    </sect2>
1642  </sect1>
1643
1644  <!-- ******************************************  -->
1645  <sect1 id="Using-Shared-Libraries">
1646    <title>Using Shared Libraries</title>
1647
1648    <sect2 id="Shared-Library-Overview">
1649          <title>Overview</title>
1650
1651      <para>&CCL; provides facilities to open and close shared
1652        libraries.</para>
1653      <para>"Opening" a shared library, which is done with <xref
1654                                                              linkend="f_open-shared-library"/>, maps the library's code and
1655        data into &CCL;'s address space and makes its exported
1656        symbols accessible to &CCL;.</para>
1657      <para>"Closing" a shared library, which is done with <xref
1658                                                              linkend="f_close-shared-library"/>, unmaps the library's code
1659        and and removes the library's symbols from the global
1660        namespace.</para>
1661      <para>A small number of shared libraries (including libc,
1662        libm, libdl under Linux, and the "system" library under
1663        Darwin) are opened by the lisp kernel and can't be
1664        closed.</para>
1665      <para>&CCL; uses data structures of type
1666        EXTERNAL-ENTRY-POINT to map a foreign function name (string)
1667        to that foreign function's <emphasis>current</emphasis>
1668        address. (A function's address may vary from session to
1669        session as different versions of shared libraries may load at
1670        different addresses; it may vary within a session for similar
1671        reasons.)</para>
1672      <para>An EXTERNAL-ENTRY-POINT whose address is known is said
1673        to be <emphasis>resolved</emphasis>. When an external entry
1674        point is resolved, the shared library which defines that entry
1675        point is noted; when a shared library is closed, the entry
1676        points that it defines are made unresolved.  An
1677        EXTERNAL-ENTRY-POINT must be in the resolved state in order to
1678        be FF-CALLed; calling an unresolved entry point causes a "last
1679        chance" attempt to resolve it. Attempting to resolve an
1680        entrypoint that was defined in a closed library will cause an
1681        attempt to reopen that library.</para>
1682      <para>&CCL; keeps track of all libraries that have been
1683        opened in a lisp session. When a saved application is first
1684        started, an attempt is made to reopen all libraries that were
1685        open when the image was saved, and an attempt is made to
1686        resolve all entry points that had been referenced when the
1687        image was saved. Either of these attempts can fail "quietly",
1688        leaving some entry points in an unresolved state.</para>
1689      <para>Linux shared libraries can be referred to either by a
1690        string which describes their full pathname or by their
1691        <emphasis>soname</emphasis>, a shorter string that can be
1692        defined when the library is created. The dynamic linker
1693        mechanisms used in Linux make it possible (through a series of
1694        filesystem links and other means) to refer to a library via
1695        several names; the library's soname is often the most
1696        appropriate identifier.</para>
1697      <para>so names are often less version-specific than other names
1698        for libraries; a program that refers to a library by the name
1699        "libc.so.6" is more portable than one which refers to
1700        "libc-2.1.3.so" or to "libc-2.2.3.so", even though the latter
1701        two names might each be platform-specific aliases of the
1702        first.</para>
1703      <para>All of the global symbols described below are exported
1704        from the CCL package.</para>
1705    </sect2>
1706
1707    <sect2 id="Limitations-and-known-bugs--1-">
1708      <title>Limitations and known bugs</title>
1709          <itemizedlist>
1710        <listitem>
1711              <para>Don't get me started.</para>
1712            </listitem>
1713        <listitem>
1714              <para>The underlying functionality has a poor notion of
1715                dependency;it's not always possible to open libraries that
1716                depend on unopened libraries, but it's possible to close
1717                libraries on which other libraries depend. It
1718                <emphasis>may</emphasis> be possible to generate
1719                more explicit dependency information by parsing the output
1720                of the Linux ldd and ldconfig programs.</para>
1721            </listitem>
1722       
1723          </itemizedlist>
1724    </sect2>
1725
1726    <sect2 id="Darwin-Notes">
1727      <title>>Darwin Notes</title>
1728          <para>Darwin shared libraries come in two (basic) flavors:</para>
1729          <itemizedlist>
1730        <listitem>
1731              <para>"dylibs" (which often have the extension".dylib") are
1732                primarily intended to be linked against at compile/link
1733                time. They can be loaded dynamically,<emphasis>but can't
1734                be unloaded</emphasis>. Accordingly,OPEN-SHARED-LIBRARY
1735                can be used to open a .dylib-style library;calling
1736                CLOSE-SHARED-LIBRARY on the result of such a call produces
1737                a warning, and has no other effect. It appears that (due
1738                to an OS bug) attempts to open .dylib shared-libraries
1739                that are already open can cause memory corruption unless
1740                the full pathname of the .dylib file is specified on the
1741                first and all subsequent calls.</para>
1742            </listitem>
1743        <listitem>
1744              <para>"bundles" are intended to serve as application
1745                extensions; they can be opened multiple times (creating
1746                multiple instances of the library!) and closed
1747                properly.</para>
1748        </listitem>
1749          </itemizedlist>
1750      <para>Thanks to Michael Klingbeil for getting both kinds of
1751        Darwin shared libraries working in &CCL;.</para>
1752    </sect2>
1753  </sect1>
1754
1755  <!-- ******************************************  -->
1756  <sect1 id="The-Interface-Translator">
1757    <title>The Interface Translator</title>
1758
1759    <sect2 id="Interface-translator-overview">
1760          <title>Overview</title>
1761          <para>&CCL; uses an interface translation system based on the FFIGEN
1762            system, which is described at
1763            http://www.ccs.neu.edu/home/lth/ffigen/.
1764            The interface translator makes
1765            the constant, type, structure, and function definitions in a set of
1766            C-language header files available to lisp code.</para>
1767      <para>The basic idea of the FFIGEN scheme is to use the C
1768        compiler's frontend and parser to translate .h files into
1769        semantically equivalent .ffi files, which represent the
1770        definitions from the headers using a syntax based on
1771        S-expressions.  Lisp code can then concentrate on the .ffi
1772        representation, without having to concern itself with the
1773        semantics of header file inclusion or the arcana of C
1774        parsing.</para>
1775      <para>The original FFIGEN system used a modified version of
1776        the LCC C compiler to produce .ffi files. Since many LinuxPPC
1777        header files contain GCC-specific constructs, &CCL;'s
1778        translation system uses a modified version of GCC (called,
1779        somewhat confusingly, ffigen.)</para>
1780      <para>A version of ffigen based on GCC-4.0 was developed
1781        during the spring and summer of 2005.  Sources (diffs relative
1782        to the GCC-4.0 release) are available here, and binaries are
1783        available for DarwinPPC and for LinuxPPC.  These versions
1784        should be insensitive to to the version of GCC (and its
1785        preprocessor) installed on the system.</para>
1786      <para>An older version was developed in 2001-2002; it depended
1787        on the installed version of GCC being 2.95.  It may still be
1788        of interest for people unable to run the GCC-4.0-based version
1789        for whatever reason.</para>
1790      <para>A LinuxPPC binary of this older version is available at
1791        ftp://clozure.com/pub/ffigen-0.1.tar.gz, and LinuxPPC source
1792        differences are at
1793        ftp://clozure.com/pub/ffigen-src.tar.gz.</para>
1794      <para>For Darwin, the binary of the older FFIGEN is available
1795        at ftp://clozure.com/pub/ffigen-darwin.tar.gz, and the source
1796        differences are at
1797        ftp://clozure.com/pub/ffigen-darwin-src.tar.gz.</para>
1798      <para>A shell script (distributed with the source and binary
1799        packages) called h-to-ffi.sh reads a specified .h file (and
1800        optional preprocessor arguments) and writes a (hopefully)
1801        equivalent .ffi file to standard output, calling the installed
1802        C preprocessor and the ffigen program with appropriate
1803        arguments.</para>
1804      <para>For each interface directory (see FIXTHIS)
1805        <emphasis>subdir</emphasis> distributed with &CCL;, a shell
1806        script (distributed with &CCL; as
1807        "ccl:headers;<emphasis>subdir</emphasis>;C;populate.sh"
1808        ("ccl:darwin-headers;<emphasis>subdir</emphasis>;C;populate.sh"
1809        for Darwin)) calls h-to-ffi.sh on a large number of the header
1810        files in /usr/include (or some other <emphasis>system header
1811          path</emphasis>) and creates a parallel directory tree in
1812        "ccl:headers;<emphasis>subdir</emphasis>;C;<emphasis>system</emphasis>;<emphasis>header</emphasis>;<emphasis>path</emphasis>;"
1813        (or
1814        "ccl:darwin-headers;<emphasis>subdir</emphasis>;C;<emphasis>system</emphasis>;<emphasis>header</emphasis>;<emphasis>path</emphasis>;"),
1815        populating that directory with .ffi files.</para>
1816      <para>A lisp function defined in "ccl:library;parse-ffi.lisp"
1817        reads the .ffi files in a specified interface directory
1818        <emphasis>subdir</emphasis> and generates new versions of the
1819        databases (files with the extension .cdb).</para>
1820      <para>The CDB databases are used by the #$ and #_ reader
1821        macros and are used in the expansion of RREF, RLET, and
1822        related macros.</para>
1823    </sect2>
1824
1825    <sect2 id="Details--rebuilding-the-CDB-databases--step-by-step">
1826      <title>Details: rebuilding the CDB databases, step by step</title>
1827          <orderedlist>
1828            <listitem>
1829              <para>Ensure that the FFIGEN program is installed. See
1830                the"README" file in the source or binary archive for
1831                specific installation instructions.This example assumes
1832                LinuxPPC; for 32-bit DarwinPPC,
1833                substitute"ccl:darwin-headers;" for "ccl:headers;".  For
1834                64-bit DarwinPPC,substitute
1835                "ccl:darwin-headers64;".</para>
1836            </listitem>
1837        <listitem>
1838              <para>Edit the
1839                "ccl:headers;<emphasis>subdir</emphasis>;C;populate.sh"shell
1840                script. When you're confident that the files
1841                and preprocessor options match your environment, cd to
1842                the"ccl:headers;<emphasis>subdir</emphasis>;C;" directory
1843                and invoke ./populate.sh. Repeat this step until you're
1844                able to cleanly translate all files referenced in the shell
1845                script.</para>
1846            </listitem>
1847            <listitem>
1848              <para>Run &CCL;:
1849            <programlisting>
1850              ? (require "PARSE-FFI")
1851              PARSE-FFI
1852
1853              ? (parse-standard-ffi-files :SUBDIR)
1854              ;;; lots of output ... after a while, shiny new .cdb files should
1855              ;;; appear in "ccl:headers;subdir;"
1856              ;;; (or "ccl:darwin-headers;subdir;" under Darwin)
1857          </programlisting></para>
1858            </listitem>
1859          </orderedlist>
1860    </sect2>
1861  </sect1>
1862
1863  <!-- ******************************************  -->
1864  <sect1 id="Case-sensitivity-of-foreign-names-in-CCL">
1865    <title>Case-sensitivity of foreign names in &CCL;</title>
1866
1867    <sect2 id="Case-sensitivity-overview">
1868          <title>Overview</title>
1869          <para>As of release 0.11, &CCL; addresses the fact that
1870            foreign type, constant, record, field, and function nams are
1871            case-sensitive and provides mechanisms to refer to these names
1872            via lisp symbols.</para>
1873      <para>Previous versions of &CCL; have tried to ignore that
1874        fact, under the belief that case conflicts were rare and that
1875        many users (and implementors) would prefer not to deal with
1876        case-related issues. The fact that some information in the
1877        interface databases was incomplete or inaccessible because of
1878        this policy made it clearer that the policy was untenable. I
1879        can't claim that the approach described here is aesthetically
1880        pleasing, but I can honestly say that it's less unpleasant
1881        than other approaches that I'd thought of. I'd be interested
1882        to hear alternate proposals.</para>
1883      <para>The issues described here have to do with how lisp
1884        symbols are used to denote foreign functions, constants,
1885        types, records, and fields. It doesn't affect how other lisp
1886        objects are sometimes used to denote foreign objects. For
1887        instance, the first argument to the EXTERNAL-CALL macros is
1888        now and has always been a case-sensitive string.</para>
1889    </sect2>
1890
1891    <sect2 id="Foreign-constant-and-function-names">
1892          <title>Foreign constant and function names</title>
1893      <para>The primary way of referring to foreign constant and
1894        function names in &CCL; is via the #$ and #_ reader
1895        macros. These reader macro functions each read a symbol into
1896        the "OS" package, look up its constant or function definition
1897        in the interface database, and assign the value of the
1898        constant to the symbol or install a macroexpansion function on
1899        the symbol.</para>
1900      <para>In order to observe case-sensitivity, the reader-macros
1901        now read the symbol with (READTABLE-CASE :PRESERVE) in
1902        effect.</para>
1903      <para>This means that it's necessary to type the foreign
1904        constant or function name in correct case, but it isn't
1905        necessary to use any special escaping constructs when writing
1906        the variable name. For instance:</para>
1907      <programlisting>
1908        (#_read fd buf n) ; refers to foreign symbol "read"
1909        (#_READ fd buf n) ; refers to foreign symbol "READ", which may
1910        ; not exist ...
1911        #$o_rdonly ; Probably doesn't exist
1912        #$O_RDONLY ; Exists on most platforms
1913      </programlisting>
1914    </sect2>
1915
1916    <sect2 id="Foreign-type--record--and-field-names">
1917          <title>Foreign type, record, and field names</title>
1918          <para>Constructs like RLET expect a foreign type or record
1919            name to be denoted by a symbol (typically a keyword); RREF
1920            (and PREF) expect an "accessor" form, typically a keyword
1921            formed by concatenating a foreign type or record name with a
1922            sequence of one or more foreign field names, separated by
1923            dots. These names are interned by the reader as other lisp
1924            symbols are, with an arbitrary value of READTABLE-CASE in
1925            effect (typically :UPCASE.) It seems like it would be very
1926            tedious to force users to manually escape (via vertical bar or
1927            backslash syntax) all lowercase characters in symbols used to
1928            specify foreign type, record, and field names (especially
1929            given that many traditional POSIX structure, type, and field
1930            names are entirely lowercase.)</para>
1931      <para>The approach taken by &CCL; is to allow the symbols
1932        (keywords) used to denote foreign type, record, and field
1933        names to contain angle brackets (<literal>&lt;</literal> and
1934        <literal>&gt;</literal>). Such symbols are translated to
1935            foreign names via the following set of conventions:</para>
1936          <itemizedlist>
1937        <listitem>
1938              <para>All instances of &lt; and &gt; in the symbol's pname
1939                are balanced and don't nest.</para>
1940            </listitem>
1941        <listitem>
1942              <para>Any alphabetic characters in the symbol's pname
1943                that aren't enclosed in angle brackets are treated as
1944                lower-case,regardless of the value of READTABLE-CASE and
1945                regardless of the case in which they were written.</para>
1946            </listitem>
1947        <listitem>
1948              <para>Alphabetic characters that appear within angle
1949                brackets are mapped to upper-case, again regardless of how
1950                they were written or interned.</para>
1951            </listitem>
1952      </itemizedlist>
1953          <para>There may be many ways of "escaping" (with angle
1954            brackets) sequences of upper-case and non-lower-case
1955            characters in a symbol used to denote a foreign name. When
1956            translating in the other direction, &CCL; always escapes the
1957            longest sequence that starts with an upper-case character and
1958            doesn't contain a lower-case character.</para>
1959      <para>It's often preferable to use this canonical form of a
1960        foreign type name.</para>
1961      <para>The accessor forms used by PREF/RREF should be viewed as
1962        a series of foreign type/record and field names; upper-case
1963        sequences in the component names should be escaped with angle
1964        brackets, but those sequences shouldn't span components. (More
1965        simply, the separating dots shouldn't be enclosed, even if
1966        both surrounding characters need to be.)</para>
1967      <para>Older POSIX code tends to use lower-case exclusively for
1968        type, record, and field names; there are only a few cases in
1969        the &CCL; sources where mixed-case names need to be
1970        escaped.</para>
1971         
1972    </sect2>
1973
1974    <sect2 id="Examples--1-">
1975      <title>Examples</title>
1976      <programlisting>
1977        ;;; Allocate a record of type "window".
1978        (rlet ((w :window)) ...)
1979        ;;; Allocate a record of type "Window", which is probably a
1980        ;;;  different type
1981        (rlet ((w :&lt;w&gt;indow)) ...)
1982        ;;; This is equivalent to the last example
1983        (rlet ((w :&lt;w&gt;INDOW)))
1984      </programlisting>
1985    </sect2>
1986  </sect1>
1987
1988  <!-- ******************************************  -->
1989  <sect1 id="Reading-Foreign-Names">
1990    <title>Reading Foreign Names</title> <para>Clozure Common Lisp
1991      provides several reader macros to make it more convenient to
1992      handle foreign type, function, variable, and constant
1993      names. Each of these reader macros reads symbols preserving the
1994      case of the source text, and selects an appropriate package in
1995      which to intern the resulting symbol. These reader macros are
1996      especially useful when your Lisp code interacts extensively with
1997      a foreign library&mdash;for example, when using Mac OS X's Cocoa
1998      frameworks.</para>
1999
2000    <para>These reader macros include "#_" to read foreign function
2001      names, "#&amp;" to read foreign variable names (note that in
2002      earlier versions of OpenMCL the reader macro "#?" was used for
2003      this same purpose), "#$" to read foreign constant names, "#/" to
2004      read the names of foreign Objective-C methods, and "#>" to read
2005      keywords that can be used as the names of types, records, and
2006      accessors.</para>
2007
2008    <para>All of these reader macros preserve the case of the text
2009      that they read; beyond that similarity, each performs some
2010      additional work, unique to each reader macro, to create symbols
2011      suitable for a particular use. For example, the function,
2012      variable, and constant reader macros intern the resulting symbol
2013      in the "OS" package of the running platform, but the reader
2014      macro for Objective-C method names interns symbols in the
2015      "NEXTSTEP-FUNCTIONS" package.</para>
2016
2017    <para>You are likely to see these reader macros used extensively
2018      in Lisp code that works with foreign libraries; for example,
2019      &CCL; IDE code, which defines numerous Objective-C classes
2020      and methods, uses these reader macros extensively.</para>
2021
2022    <para>For more detailed descriptions of each of these reader
2023      macros, see the Foreign-Function-Interface Dictionary
2024      section.</para>
2025  </sect1>
2026
2027  <!-- ******************************************  -->
2028  <sect1 id="Tutorial--Using-Basic-Calls-and-Types">
2029    <title>Tutorial: Using Basic Calls and Types</title>
2030    <para>This tutorial is meant to cover the basics of &CCL; for
2031      calling external C functions and passing data back and forth.
2032      These basics will provide the foundation for more advanced
2033      techniques which will allow access to the various external
2034      libraries and toolkits.</para>
2035    <para>The first step is to start with a simple C dynamic library
2036      in order to actually observe what is actually passing between
2037      &CCL; and C.  So, some C code is in order:</para>
2038    <para>Create the file typetest.c, and put the following code
2039      into it:</para>
2040    <programlisting>
2041#include &lt;stdio.&gt;
2042
2043void
2044void_void_test(void)
2045{
2046    printf("Entered %s:\n", __FUNCTION__);
2047    printf("Exited  %s:\n", __FUNCTION__);
2048    fflush(stdout);
2049}
2050
2051signed char
2052sc_sc_test(signed char data)
2053{
2054    printf("Entered %s:\n", __FUNCTION__);
2055    printf("Data In: %d\n", (signed int)data);
2056    printf("Exited  %s:\n", __FUNCTION__);
2057    fflush(stdout);
2058    return data;
2059}
2060
2061unsigned char
2062uc_uc_test(unsigned char data)
2063{
2064    printf("Entered %s:\n", __FUNCTION__);
2065    printf("Data In: %d\n", (signed int)data);
2066    printf("Exited  %s:\n", __FUNCTION__);
2067    fflush(stdout);
2068    return data;
2069}
2070    </programlisting>
2071    <para>This defines three functions.  If you're familiar with C,
2072      notice that there's no <literal>main()</literal>, because we're
2073      just building a library, not an executable.</para>
2074    <para>The function <literal>void_void_test()</literal> doesn't
2075      take any parameters, and doesn't return anything, but it prints
2076      two lines to let us know it was called.
2077      <literal>sc_sc_test()</literal> takes a signed char as a
2078      parameter, prints it, and returns it.
2079      <literal>uc_uc_test()</literal> does the same thing, but with an
2080      unsigned char.  Their purpose is just to prove to us that we
2081      really can call C functions, pass them values, and get values
2082      back from them.</para>
2083    <para>This code is compiled into a dynamic library on OS X
2084      10.3.4 with the command:</para>
2085    <programlisting>
2086
2087      gcc -dynamiclib -Wall -o libtypetest.dylib typetest.c \
2088      -install_name ./libtypetest.dylib
2089    </programlisting>
2090    <tip><para>Users of 64-bit platforms may need to pass options such
2091        as "-m64" to gcc, may need to give the output library a different
2092        extension (such as ".so"), and may need to user slightly different
2093        values for other options in order to create an equivalent test
2094        library.</para></tip>
2095
2096    <para>The -dynamiclib tells gcc that we will be compiling this
2097      into a dynamic library and not an executable binary program.
2098      The output filename is "libtypetest.dylib".  Notice that we
2099      chose a name which follows the normal OS X convention, being in
2100      the form "libXXXXX.dylib", so that other programs can link to
2101      the library.  &CCL; doesn't need it to be this way, but it is
2102      a good idea to adhere to existing conventions.</para>
2103    <para>The -install_name flag is primarily used when building OS
2104      X "bundles".  In this case, we are not using it, so we put a
2105      placeholder into it, "./libtypetest.dylib".  If we wanted to use
2106      typetest in a bundle, the -install_name argument would be a
2107      relative path from some "current" directory.</para>
2108    <para>After creating this library, the first step is to tell
2109      &CCL; to open the dynamic library.  This is done by calling
2110      .</para>
2111    <programlisting>
2112
2113      Welcome to &CCL; Version (Beta: Darwin) 0.14.2-040506!
2114
2115      ? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
2116      #&lt;SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E&gt;
2117    </programlisting>
2118    <para>You should use an absolute path here; using a relative
2119      one, such as just "libtypetest.dylib", would appear to work, but
2120      there are subtle problems which occur after reloading it.  See
2121      the Darwin notes on for details.  It would be a bad idea anyway,
2122      because software should never rely on its starting directory
2123      being anything in particular.</para>
2124    <para>This command returns a reference to the opened shared library, and
2125      &CCL; also adds one to the global variable
2126      <literal>ccl::*shared-libraries*</literal>:</para>
2127    <programlisting>
2128
2129      ? ccl::*shared-libraries*
2130      (#&lt;SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
2131       #&lt;SHLIB /usr/lib/libSystem.B.dylib #x606179E>)
2132    </programlisting>
2133    <para>Before we call anything, let's check that the individual
2134      functions can actually be found by the system.  We don't have to
2135      do this, but it helps to know how to find out whether this is
2136      the problem, when something goes wrong.  We use <xref
2137                                                         linkend="m_external-call"/>:</para>
2138    <programlisting>
2139
2140      ? (external "_void_void_test")
2141      #&lt;EXTERNAL-ENTRY-POINT "_void_void_test" (#x000CFDF8) /Users/andewl/openmcl/libtypetest.dylib #x638EDF6>
2142
2143      ? (external "_sc_sc_test")
2144      #&lt;EXTERNAL-ENTRY-POINT "_sc_sc_test" (#x000CFE50) /Users/andewl/openmcl/libtypetest.dylib #x638EB3E>
2145
2146      ? (external "_uc_uc_test")
2147      #&lt;EXTERNAL-ENTRY-POINT "_uc_uc_test" (#x000CFED4) /Users/andewl/openmcl/libtypetest.dylib #x638E626>
2148    </programlisting>
2149    <para>Notice that the actual function names have been "mangled"
2150      by the C linker.  The first function was named "void_void_test"
2151      in typetest.c, but in libtypetest.dylib, it has an underscore (a
2152      "_" symbol) before it: "_void_void_test".  So, this is the name
2153      which you have to use.  The mangling - the way the name is
2154      changed - may be different for other operating systems or other
2155      versions, so you need to "just know" how it's done...</para>
2156    <para>Also, pay particular attention to the fact that a
2157      hexadecimal value appears in the EXTERNAL-ENTRY-POINT.
2158      (#x000CFDF8, for example - but what it is doesn't matter.)
2159      These hex numbers mean that the function can be dereferenced.
2160      Functions which aren't found will not have a hex number.  For
2161      example:</para>
2162    <programlisting>
2163
2164      ? (external "functiondoesnotexist")
2165      #&lt;EXTERNAL-ENTRY-POINT "functiondoesnotexist" {unresolved}  #x638E3F6>
2166    </programlisting>
2167    <para>The "unresolved" tells us that &CCL; wasn't able to find this
2168      function, which means you would get an error, "Can't resolve foreign
2169      symbol," if you tried to call it.</para>
2170    <para>These external function references also are stored in a
2171      hash table which is accessible through a global variable,
2172      <literal>ccl::*eeps*</literal>.</para>
2173    <para>At this point, we are ready to try our first external
2174      function call:</para>
2175    <programlisting>
2176
2177      ? (external-call "_void_void_test" :void)
2178      Entered void_void_test:
2179      Exited  void_void_test:
2180      NIL
2181    </programlisting>
2182    <para>We used , which is is the normal mechanism for accessing
2183      externally linked code.  The "_void_void_test" is the mangled
2184      name of the external function.  The :void refers to the return
2185      type of the function.</para>
2186    <para>The next step is to try passing a value to C, and getting one
2187      back:</para>
2188    <programlisting>
2189
2190      ? (external-call "_sc_sc_test" :signed-byte -128 :signed-byte)
2191      Entered sc_sc_test:
2192      Data In: -128
2193      Exited  sc_sc_test:
2194      -128
2195    </programlisting>
2196    <para>The first :signed-byte gives the type of the first
2197      argument, and then -128 gives the value to pass for it.  The
2198      second :signed-byte gives the return type.  The return type is
2199      always given by the last argument to .</para>
2200    <para>Everything looks good.  Now, let's try a number outside
2201      the range which fits in one byte:</para>
2202    <programlisting>
2203
2204      ? (external-call "_sc_sc_test" :signed-byte -567 :signed-byte)
2205      Entered sc_sc_test:
2206      Data In: -55
2207      Exited  sc_sc_test:
2208      -55
2209    </programlisting>
2210    <para>Hmmmm.  A little odd.  Let's look at the unsigned stuff to
2211      see how it reacts:</para>
2212    <programlisting>
2213
2214      ? (external-call "_uc_uc_test" :unsigned-byte 255 :unsigned-byte)
2215      Entered uc_uc_test:
2216      Data In: 255
2217      Exited  uc_uc_test:
2218      255
2219    </programlisting>
2220    <para>That looks okay.  Now, let's go outside the valid range again:</para>
2221    <programlisting>
2222
2223      ? (external-call "_uc_uc_test" :unsigned-byte 567 :unsigned-byte)
2224      Entered uc_uc_test:
2225      Data In: 55
2226      Exited  uc_uc_test:
2227      55
2228
2229      ? (external-call "_uc_uc_test" :unsigned-byte -567 :unsigned-byte)
2230      Entered uc_uc_test:
2231      Data In: 201
2232      Exited  uc_uc_test:
2233      201
2234    </programlisting>
2235    <para>Since a signed byte can only hold values from -128 through 127, and
2236      an unsigned one can only hold values from 0 through 255, any number
2237      outside that range gets "clipped": only the low eight bits of it
2238      are used.</para>
2239    <para>What is important to remember is that <emphasis>external
2240        function calls have
2241        very few safety checks.</emphasis>
2242      Data outside the valid range for its type will silently do
2243      very strange things; pointers outside the valid range can very well
2244      crash the system.</para>
2245    <para>That's it for our first example library.  If you're still
2246      following along, let's add some more C code to look at the rest
2247      of the primitive types.  Then we'll need to recompile the
2248      dynamic library, load it again, and then we can see what
2249      happens.</para>
2250    <para>Add the following code to typetest.c:</para>
2251    <programlisting>
2252int
2253si_si_test(int data)
2254{
2255    printf("Entered %s:\n", __FUNCTION__);
2256    printf("Data In: %d\n", data);
2257    printf("Exited  %s:\n", __FUNCTION__);
2258    fflush(stdout);
2259    return data;
2260}
2261
2262long
2263sl_sl_test(long data)
2264{
2265    printf("Entered %s:\n", __FUNCTION__);
2266    printf("Data In: %ld\n", data);
2267    printf("Exited  %s:\n", __FUNCTION__);
2268    fflush(stdout);
2269    return data;
2270}
2271
2272long long
2273sll_sll_test(long long data)
2274{
2275    printf("Entered %s:\n", __FUNCTION__);
2276    printf("Data In: %lld\n", data);
2277    printf("Exited  %s:\n", __FUNCTION__);
2278    fflush(stdout);
2279    return data;
2280}
2281
2282float
2283f_f_test(float data)
2284{
2285    printf("Entered %s:\n", __FUNCTION__);
2286    printf("Data In: %e\n", data);
2287    printf("Exited  %s:\n", __FUNCTION__);
2288    fflush(stdout);
2289    return data;
2290}
2291
2292double
2293d_d_test(double data)
2294{
2295    printf("Entered %s:\n", __FUNCTION__);
2296    printf("Data In: %e\n", data);
2297    printf("Exited  %s:\n", __FUNCTION__);
2298    fflush(stdout);
2299    return data;
2300}
2301    </programlisting>
2302    <para>The command line to compile the dynamic library is the same as before:</para>
2303    <programlisting>
2304
2305      gcc -dynamiclib -Wall -o libtypetest.dylib typetest.c \
2306      -install_name ./libtypetest.dylib
2307    </programlisting>
2308    <para>Now, restart &CCL;.  This step is required because
2309      &CCL; cannot close and reload a dynamic library on OS
2310      X.</para>
2311    <para>Have you restarted?  Okay, try out the new code:</para>
2312    <programlisting>
2313
2314      Welcome to &CCL; Version (Beta: Darwin) 0.14.2-040506!
2315
2316      ? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
2317      #&lt;SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
2318
2319      ? (external-call "_si_si_test" :signed-fullword -178965 :signed-fullword)
2320      Entered si_si_test:
2321      Data In: -178965
2322      Exited  si_si_test:
2323      -178965
2324
2325      ? ;; long is the same size as int on 32-bit machines.
2326      (external-call "_sl_sl_test" :signed-fullword -178965 :signed-fullword)
2327      Entered sl_sl_test:
2328      Data In: -178965
2329      Exited  sl_sl_test:
2330      -178965
2331
2332      ? (external-call "_sll_sll_test"
2333      :signed-doubleword -973891578912 :signed-doubleword)
2334      Entered sll_sll_test:
2335      Data In: -973891578912
2336      Exited  sll_sll_test:
2337      -973891578912
2338    </programlisting>
2339    <para>Okay, everything seems to be acting as expected.  However,
2340      just to remind you that most of this stuff has no safety net,
2341      here's what happens if somebody mistakes
2342      <literal>sl_sl_test()</literal> for
2343      <literal>sll_sll_test()</literal>, thinking that a long is
2344      actually a doubleword:</para>
2345    <programlisting>
2346
2347      ? (external-call "_sl_sl_test"
2348      :signed-doubleword -973891578912 :signed-doubleword)
2349      Entered sl_sl_test:
2350      Data In: -227
2351      Exited  sl_sl_test:
2352      -974957576192
2353    </programlisting>
2354    <para>Ouch.  The C function changes the value with no warning
2355      that something is wrong.  Even worse, it manages to pass the
2356      original value back to &CCL;, which hides the fact that
2357      something is wrong.</para>
2358    <para>Finally, let's take a look at doing this with
2359      floating-point numbers.</para>
2360    <programlisting>
2361
2362      Welcome to &CCL; Version (Beta: Darwin) 0.14.2-040506!
2363
2364      ? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
2365      #&lt;SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
2366
2367      ? (external-call "_f_f_test" :single-float -1.256791e+11 :single-float)
2368      Entered f_f_test:
2369      Data In: -1.256791e+11
2370      Exited  f_f_test:
2371      -1.256791E+11
2372
2373      ? (external-call "_d_d_test" :double-float -1.256791d+290 :double-float)
2374      Entered d_d_test:
2375      Data In: -1.256791e+290
2376      Exited  d_d_test:
2377      -1.256791D+290
2378    </programlisting>
2379    <para>Notice that the number ends with "...e+11" for the single-float,
2380      and "...d+290" for the
2381      double-float.  Lisp has both of these float types itself, and the
2382      d instead of the e is how you specify which to create.  If
2383      you tried to pass :double-float 1.0e2 to external-call, Lisp would
2384      be nice enough to notice and give you a type error.  Don't get the
2385      :double-float wrong, though, because then there's no protection.</para>
2386    <para>Congratulations!  You now know how to call external C functions from
2387      within &CCL;, and pass numbers back and forth.  Now that the basic
2388      mechanics of calling and passing work, the next step is to examine how
2389      to pass more complex data structures around.</para>
2390
2391    <sect2 id="Acknowledgement">
2392      <title>Acknowledgement</title>
2393          <para>This chapter was generously contributed by Andrew
2394            P. Lentvorski Jr.</para>
2395    </sect2>
2396  </sect1>
2397
2398  <!-- ******************************************  -->
2399  <sect1 id="Tutorial--Allocating-Foreign-Data-on-the-Lisp-Heap">
2400    <title>Tutorial: Allocating Foreign Data on the Lisp Heap </title>
2401    <para>Not every foreign function is so marvelously easy to use
2402      as the ones we saw in the last section.  Some of them require
2403      you to allocate a C struct, fill it in with your own
2404      information, and pass it a pointer to the struct.  Some of them
2405      require you to allocate an empty struct so they can fill it in,
2406      and then you can read the information out of it.</para>
2407    <para>Also, some of them have their own structs and return a
2408      pointer to that same struct every time you call them, but those
2409      are easier to deal with, so they won't be covered in this
2410      section.</para>
2411    <para>You might know that Lisp (and, indeed, most programming
2412      languages) has two separate regions of memory.  There's the
2413      stack, which is where variable bindings are kept.  Memory on the
2414      stack is allocated every time any function is called, and
2415      deallocated when it returns, so it's useful for anything that
2416      doesn't need to last longer than one function call, when there's
2417      only one thread.  If that's all you need, you can do it with
2418      .</para>
2419    <para>Then, there's the heap, which holds everything else, and
2420      is our subject here.  There are two advantages and one big
2421      disadvantage to putting things on the heap rather than the
2422      stack.  First, data allocated on the heap can be passed outside
2423      of the scope in which it was created.  This is useful for data
2424      which may need to be passed between multiple C calls or multiple
2425      threads. Also, some data may be too large to copy multiple times
2426      or may be too large to allocate on the stack.</para>
2427    <para>The second advantage is security.  If incoming data is
2428      being placed directly onto the stack, the input data can cause
2429      stack overflows and underflows.  This is not something which
2430      Lisp users generally worry about since garbage collection
2431      generally handles memory management.  However, "stack smashing"
2432      is one of the classic exploits in C which malicious hackers can
2433      use to gain control of a machine.  Not checking external data is
2434      always a bad idea; however, allocating it into the heap at least
2435      offers more protection than direct stack allocation.</para>
2436    <para>The big disadvantage to allocating data on the heap is
2437      that it must be explicitly deallocated&mdash;you need to "free" it
2438      when you're done with it.  Ordinarily, in Lisp, you wouldn't
2439      allocate memory yourself, and the garbage collector would know
2440      about it, so you wouldn't have to think about it again.  When
2441      you're doing it manually, it's very different.  Memory
2442      management becomes a manual process, just like in C and
2443      C++.</para>
2444    <para>What that means is that, if you allocate something and
2445      then lose track of the pointer to it, there's no way to ever
2446      free that memory.  That's what's called a memory leak, and if
2447      your program leaks enough memory it will eventually use up all
2448      of it!  So, you need to be careful to not lose your
2449      pointers.</para>
2450    <para>That disadvantage, though, is also an advantage for using
2451      foreign functions.  Since the garbage collector doesn't know
2452      about this memory, it will never move it around.  External C
2453      code needs this, because it doesn't know how to follow it to
2454      where it moved, the way that Lisp code does.  If you allocate
2455      data manually, you can pass it to foreign code and know that no
2456      matter what that code needs to do with it, it will be able to,
2457      until you deallocate it.  Of course, you'd better be sure it's
2458      done before you do.  Otherwise, your program will be unstable
2459      and might crash sometime in the future, and you'll have trouble
2460      figuring out what caused the trouble, because there won't be
2461      anything pointing back and saying "you deallocated this too
2462      soon."</para>
2463    <para>And, so, on to the code...</para>
2464    <para>As in the last tutorial, our first step
2465      is to create a local dynamic library in order to help show
2466      what is actually going on between &CCL; and C.  So, create the file
2467      ptrtest.c, with the following code:</para>
2468    <programlisting>
2469#include &lt;stdio.h&gt;
2470
2471void reverse_int_array(int * data, unsigned int dataobjs)
2472{
2473    int i, t;
2474   
2475    for(i=0; i&lt;dataobjs/2; i++)
2476        {
2477            t = *(data+i);
2478            *(data+i) = *(data+dataobjs-1-i);
2479            *(data+dataobjs-1-i) = t;
2480        }
2481}
2482
2483void reverse_int_ptr_array(int **ptrs, unsigned int ptrobjs)
2484{
2485    int *t;
2486    int i;
2487   
2488    for(i=0; i&lt;ptrobjs/2; i++)
2489        {
2490            t = *(ptrs+i);
2491            *(ptrs+i) = *(ptrs+ptrobjs-1-i);
2492            *(ptrs+ptrobjs-1-i) = t;
2493        }
2494}
2495
2496void
2497reverse_int_ptr_ptrtest(int **ptrs)
2498{
2499    reverse_int_ptr_array(ptrs, 2);
2500   
2501    reverse_int_array(*(ptrs+0), 4);
2502    reverse_int_array(*(ptrs+1), 4);
2503}
2504    </programlisting>
2505    <para>This defines three functions.
2506      <literal>reverse_int_array</literal> takes a pointer to an array
2507      of <literal>int</literal>s, and a count telling how many items
2508      are in the array, and loops through it putting the elements in
2509      reverse.  <literal>reverse_int_ptr_array</literal> does the same
2510      thing, but with an array of pointers to <literal>int</literal>s.
2511      It only reverses the order the pointers are in; each pointer
2512      still points to the same thing.
2513      <literal>reverse_int_ptr_ptrtest</literal> takes an array of
2514      pointers to arrays of <literal>int</literal>s.  (With me?)  It
2515      doesn't need to be told their sizes; it just assumes that the
2516      array of pointers has two items, and that both of those are
2517      arrays which have four items.  It reverses the array of
2518      pointers, then it reverses each of the two arrays of
2519      <literal>int</literal>s.</para>
2520    <para>Now, compile ptrtest.c into a dynamic library using the
2521      command:</para>
2522    <programlisting>
2523      gcc -dynamiclib -Wall -o libptrtest.dylib ptrtest.c -install_name ./libptrtest.dylib
2524    </programlisting>
2525    <para>If that command doesn't make sense to you, feel free to go back
2526      and read about it at .</para>
2527    <para>Now, start &CCL; and enter:</para>
2528    <programlisting>
2529      ? ;; make-heap-ivector courtesy of Gary Byers
2530      (defun make-heap-ivector (element-count element-type)
2531       (let* ((subtag (ccl::element-type-subtype element-type)))
2532        (unless (= (logand subtag target::fulltagmask)
2533                 target::fulltag-immheader)
2534         (error "~s is not an ivector subtype." element-type))
2535        (let* ((size-in-bytes (ccl::subtag-bytes subtag element-count)))
2536         (ccl::%make-heap-ivector subtag size-in-bytes element-count))))     
2537      MAKE-HEAP-IVECTOR
2538
2539      ? ;; dispose-heap-ivector created for symmetry
2540      (defmacro dispose-heap-ivector (a mp)
2541       `(progn
2542         (ccl::%dispose-heap-ivector ,a)
2543         ;; Demolish the arguments for safety
2544         (setf ,a nil)
2545         (setf ,mp nil)))
2546      DISPOSE-HEAP-IVECTOR
2547    </programlisting>
2548    <para>If you don't understand how those functions do what they do.
2549      That's okay; it gets into very fine detail which really doesn't
2550      matter, because you don't need to change them.</para>
2551    <para>The function <literal>make-heap-ivector</literal> is the
2552      primary tool for allocating objects in heap memory.  It
2553      allocates a fixed-size &CCL; object in heap memory.  It
2554      returns both an array reference, which can be used directly from
2555      &CCL;, and a <literal>macptr</literal>, which can be used to
2556      access the underlying memory directly.  For example:</para>
2557    <programlisting>
2558      ? ;; Create an array of 3 4-byte-long integers
2559      (multiple-value-bind (la lap)
2560          (make-heap-ivector 3 '(unsigned-byte 32))
2561        (setq a la)
2562        (setq ap lap))
2563      ;Compiler warnings :
2564      ;   Undeclared free variable A, in an anonymous lambda form.
2565      ;   Undeclared free variable AP, in an anonymous lambda form.
2566      #&lt;A Mac Pointer #x10217C>
2567
2568      ? a
2569      #(1396 2578 97862649)
2570
2571      ? ap
2572      #&lt;A Mac Pointer #x10217C>
2573    </programlisting>
2574    <para>It's important to realize that the contents of the
2575      <literal>ivector</literal> we've just created haven't been
2576      initialized, so their values are unpredictable, and you should
2577      be sure not to read from them before you set them, to avoid
2578      confusing results.</para>
2579    <para>At this point, <literal>a</literal> references an object
2580      which works just like a normal array.  You can refer to any item
2581      of it with the standard <literal>aref</literal> function, and
2582      set them by combining that with <literal>setf</literal>.  As
2583      noted above, the <literal>ivector</literal>'s contents haven't
2584      been initialized, so that's the next order of business:</para>
2585    <programlisting>
2586      ? a
2587      #(1396 2578 97862649)
2588
2589      ? (aref a 2)
2590      97862649
2591
2592      ? (setf (aref a 0) 3)
2593      3
2594
2595      ? (setf (aref a 1) 4)
2596      4
2597
2598      ? (setf (aref a 2) 5)
2599      5
2600
2601      ? a
2602      #(3 4 5)
2603    </programlisting>
2604    <para>In addition, the <literal>macptr</literal> allows direct
2605      access to the same memory:</para>
2606    <programlisting>
2607      ? (setq *byte-length-of-long* 4)
2608      4
2609
2610      ? (%get-signed-long ap (* 2 *byte-length-of-long*))
2611      5
2612
2613      ? (%get-signed-long ap (* 0 *byte-length-of-long*))
2614      3
2615
2616      ? (setf (%get-signed-long ap (* 0 *byte-length-of-long*)) 6)
2617      6
2618
2619      ? (setf (%get-signed-long ap (* 2 *byte-length-of-long*)) 7)
2620      7
2621
2622      ? ;; Show that a actually got changed through ap
2623      a
2624      #(6 4 7)
2625    </programlisting>
2626    <para>So far, there is nothing about this object that could not
2627      be done much better with standard Lisp.  However, the
2628      <literal>macptr</literal> can be used to pass this chunk of
2629      memory off to a C function.  Let's use the C code to reverse the
2630      elements in the array:</para>
2631    <programlisting>
2632      ? ;; Insert the full path to your copy of libptrtest.dylib
2633      (open-shared-library "/Users/andrewl/openmcl/openmcl/gtk/libptrtest.dylib")
2634      #&lt;SHLIB /Users/andrewl/openmcl/openmcl/gtk/libptrtest.dylib #x639D1E6>
2635
2636      ? a
2637      #(6 4 7)
2638
2639      ? ap
2640      #&lt;A Mac Pointer #x10217C>
2641
2642      ? (external-call "_reverse_int_array" :address ap :unsigned-int (length a) :address)
2643      #&lt;A Mac Pointer #x10217C>
2644
2645      ? a
2646      #(7 4 6)
2647
2648      ? ap
2649      #&lt;A Mac Pointer #x10217C>
2650    </programlisting>
2651    <para>The array gets passed correctly to the C function,
2652      <literal>reverse_int_array</literal>.  The C function reverses
2653      the contents of the array in-place; that is, it doesn't make a
2654      new array, just keeps the same one and reverses what's in it.
2655      Finally, the C function passes control back to &CCL;.  Since
2656      the allocated array memory has been directly modified, &CCL;
2657      reflects those changes directly in the array as well.</para>
2658    <para>There is one final bit of housekeeping to deal with.
2659      Before moving on, the memory needs to be deallocated:</para>
2660    <programlisting>
2661      ? ;; dispose-heap-ivector created for symmetry
2662      ;; Macro repeated here for pedagogy
2663      (defmacro dispose-heap-ivector (a mp)
2664      `(progn
2665      (ccl::%dispose-heap-ivector ,a)
2666      ;; Demolish the arguments for safety
2667      (setf ,a nil)
2668      (setf ,mp nil)))
2669      DISPOSE-HEAP-IVECTOR
2670
2671      ? (dispose-heap-ivector a ap)
2672      NIL
2673
2674      ? a
2675      NIL
2676
2677      ? ap
2678      NIL
2679    </programlisting>
2680    <para>The <literal>dispose-heap-ivector</literal> macro actually
2681      deallocates the ivector, releasing its memory into the heap for
2682      something else to use.  In addition, it makes sure that the
2683      variables which it was called with are set to nil, because
2684      otherwise they would still be referencing the memory of the
2685      ivector - which is no longer allocated, so that would be a bug.
2686      Making sure there are no other variables set to it is up to
2687      you.</para>
2688    <para>When do you call <literal>dispose-heap-ivector</literal>?
2689      Anytime after you know the ivector will never be used again, but
2690      no sooner.  If you have a lot of ivectors, say, in a hash table,
2691      you need to make sure that when whatever you were doing with the
2692      hash table is done, those ivectors all get freed.  Unless
2693      there's still something somewhere else which refers to them, of
2694      course!  Exactly what strategy to take depends on the situation,
2695      so just try to keep things simple unless you know better.</para>
2696    <para>The simplest situation is when you have things set up so
2697      that a Lisp object "encapsulates" a pointer to foreign data,
2698      taking care of all the details of using it.  In this case, you
2699      don't want those two things to have different lifetimes: You
2700      want to make sure your Lisp object exists as long as the foreign
2701      data does, and no longer; and you want to make sure the foreign
2702      data doesn't get deallocated while your Lisp object still refers
2703      to it.</para>
2704    <para>If you're willing to accept a few limitations, you can
2705      make this easy.  First, you can't let foreign code keep a
2706      permanent pointer to the memory; it has to always finish what
2707      it's doing, then return, and not refer to that memory again.
2708      Second, you can't let any Lisp code that isn't part of your
2709      encapsulating "wrapper" refer to the pointer directly.  Third,
2710      nothing, either foreign code or Lisp code, should explicitly
2711      deallocate the memory.</para>
2712    <para>If you can make sure all of these are true, you can at
2713      least ensure that the foreign pointer is deallocated when the
2714      encapsulating object is about to become garbage, by using
2715      &CCL;'s nonstandard "termination" mechanism, which is
2716      essentially the same as what Java and other languages call
2717      "finalization".</para>
2718    <para>Termination is a way of asking the garbage collector to
2719      let you know when it's about to destroy an object which isn't
2720      used anymore.  Before destroying the object, it calls a function
2721      which you write, called a terminator.</para>
2722    <para>So, you can use termination to find out when a particular
2723      <literal>macptr</literal> is about to become garbage.  That's
2724      not quite as helpful as it might seem: It's not exactly the same
2725      thing as knowing that the block of memory it points to is
2726      unreferenced.  For example, there could be another
2727      <literal>macptr</literal> somewhere to the same block; or, if
2728      it's a struct, there could be a <literal>macptr</literal> to one
2729      of its fields.  Most problematically, if the address of that
2730      memory has been passed to foreign code, it's sometimes hard to
2731      know whether that code has kept the pointer.  Most foreign
2732      functions don't, but it's not hard to think of
2733      exceptions.</para>
2734    <para>You can use code such as this to make all this happen:</para>
2735    <programlisting>
2736      (defclass wrapper (whatever)
2737        ((element-type :initarg :element-type)
2738         (element-count :initarg :element-count)
2739         (ivector)
2740         (macptr)))
2741
2742      (defmethod initialize-instance ((wrapper wrapper) &rest; initargs)
2743        (declare (ignore initargs))
2744        (call-next-method)
2745        (ccl:terminate-when-unreachable wrapper)
2746        (with-slots (ivector macptr element-type element-count) wrapper
2747          (multiple-value-bind (new-ivector new-macptr)
2748              (make-heap-ivector element-count element-type)
2749            (setq ivector new-ivector
2750                  macptr new-macptr))))
2751
2752      (defmethod ccl:terminate ((wrapper wrapper))
2753        (with-slots (ivector macptr) wrapper
2754          (when ivector
2755            (dispose-heap-ivector ivector macptr)
2756            (setq ivector nil
2757                  macptr nil))))
2758    </programlisting>
2759    <para>The <literal>ccl:terminate</literal> method will be called
2760      on some arbitrary thread sometime (hopefully soon) after the GC
2761      has decided that there are no strong references to an object
2762      which has been the argument of a
2763      <literal>ccl:terminate-when-unreachable</literal> call.</para>
2764    <para>If it makes sense to say that the foreign object should
2765      live as long as there's Lisp code that references it (through
2766      the encapsulating object) and no longer, this is one way of doing
2767      that.</para>
2768    <para>Now we've covered passing basic types back and forth with
2769      C, and we've done the same with pointers.  You may think this is
2770      all...  but we've only done pointers to basic types.  Join us
2771      next time for pointers... to pointers.</para>
2772
2773    <sect2 id="Acknowledgement--1-">
2774          <title>Acknowledgement</title>
2775          <para>Much of this chapter was generously contributed by
2776            Andrew P. Lentvorski Jr.</para>
2777    </sect2>
2778  </sect1>
2779
2780  <!-- ******************************************  -->
2781  <sect1>
2782    <title>The Foreign-Function-Interface Dictionary</title>
2783    <anchor id="anchor_Foreign-Function-Interface-Dictionary"/>
2784    <!-- ====================================  -->
2785    <refentry id="rm_sharpsign-underscore">
2786      <indexterm zone="rm_sharpsign-underscore">
2787            <primary>#_</primary>
2788      </indexterm>
2789
2790      <refnamediv>
2791            <refname>#_</refname>
2792            <refpurpose></refpurpose>
2793            <refclass>Reader Macro</refclass>
2794      </refnamediv>
2795
2796      <refsect1>
2797            <title>Description</title>
2798
2799            <para>Reads a symbol from the current input stream, with *PACKAGE*
2800              bound to the &#34;OS&#34; package and with readtable-case preserved.</para>
2801           
2802            <para>Does a lookup on that symbol in <link
2803                                                     linkend="The-Interface-Database">the &CCL; interface
2804                database</link>, signalling an error if no foreign function
2805              information can be found for the symbol in any active <link
2806                                                                       linkend="Using-Interface-Directories">interface
2807                directory</link>.</para>
2808
2809            <para>Notes the foreign function information, including the foreign
2810              function&#39;s return type, the number and type of the foreign
2811              function&#39;s required arguments, and an indication of whether or
2812              not the function accepts additional arguments (via e.g., the
2813              &#34;varargs&#34; mechanism in C).</para>
2814
2815            <para>Defines a macroexpansion function on the symbol, which expand
2816              macro calls involving the symbol into EXTERNAL-CALL forms where
2817              foreign argument type specifiers for required arguments and the
2818              return value specifer are provided from the information in the
2819              database.</para>
2820
2821            <para>Returns the symbol.</para>
2822
2823            <para>The effect of these steps is that it&#39;s possible to call
2824              foreign functions that take fixed numbers of arguments by simply
2825              providing argument values, as in:</para>
2826
2827            <programlisting format="linespecific">(#_isatty fd)
2828          (#_read fd buf n)</programlisting>
2829
2830            <para>and to call foreign functions that take variable numbers of
2831              arguments by specifying the types of non-required args, as in:</para>
2832
2833            <programlisting format="linespecific">(with-cstrs ((format-string &#34;the answer is: %d&#34;))
2834          (#_printf format-string :int answer))</programlisting>
2835
2836        <para>You can query whether a given name is defined in the
2837          interface databases by appending the '?' character to the reader
2838          macro; for example:</para>
2839
2840        <programlisting>
2841          CL-USER&gt; #_?printf
2842          T
2843          CL-USER&gt; #_?foo
2844          NIL
2845        </programlisting>
2846
2847      </refsect1>
2848    </refentry>
2849
2850    <!-- ====================================  -->
2851    <refentry id="rm_sharpsign-ampersand">
2852      <indexterm zone="rm_sharpsign-ampersand">
2853            <primary>#&amp;</primary>
2854      </indexterm>
2855
2856      <refnamediv>
2857            <refname>#&amp;</refname>
2858            <refpurpose></refpurpose>
2859            <refclass>Reader Macro</refclass>
2860      </refnamediv>
2861
2862      <refsect1>
2863            <title>Description</title>
2864
2865            <para>In &CCL; 1.2 and later, the #&amp; reader macro can be used to
2866              access foreign variables; this functionality depends on the presence of
2867              &#34;vars.cdb&#34; files in the interface database. The current behavior
2868              of the #&amp; reader macro is to:</para>
2869
2870            <para>Read a symbol from the current input stream, with *PACKAGE*
2871              bound to the &#34;OS&#34; package and with readtable-case preserved.</para>
2872           
2873            <para>Use that symbol&#39;s pname to access the &CCL; interface
2874              database, signalling an error if no appropriate foreign variable
2875              information can be found with that name in any active interface
2876              directory.</para>
2877
2878            <para>Use type information recorded in the database to construct a
2879              form which can be used to access the foreign variable, and return
2880              that form.</para>
2881
2882            <para>Please note that the set of foreign variables declared in header files
2883              may or may not match the set of foreign variables exported from
2884              libraries (we&#39;re generally talking about C and Unix here ...). When
2885              they do match, the form constructed by the #&amp; reader macro manages the
2886              details of resolving and tracking changes to the foreign variable&#39;s
2887              address.</para>
2888
2889            <para>Future extensions (via prefix arguments to the reader macro) may
2890              offer additional behavior; it might be convenient (for instance) to be
2891              able to access the address of a foreign variable without dereferencing
2892              that address.</para>
2893
2894            <para>Foreign variables in C code tend to be platform- and
2895              package-specific (the canonical example - &#34;errno&#34; - is typically
2896              not a variable when threads are involved. )</para>
2897
2898            <para>In LinuxPPC, </para>
2899
2900            <programlisting>? #&amp;stderr</programlisting>
2901
2902            <para>returns a pointer to the stdio error stream (&#34;stderr&#34; is a
2903              macro under OSX/Darwin).</para>
2904
2905            <para>On both LinuxPPC and DarwinPPC, </para>
2906
2907            <programlisting>? #&amp;sys_errlist</programlisting>
2908
2909            <para>returns a pointer to a C array of C error message strings.</para>
2910
2911        <para>You can query whether a given name is defined in the
2912          interface databases by appending the '?' character to the reader
2913          macro; for example:</para>
2914
2915        <programlisting>
2916          CL-USER&gt; #&amp;?sys_errlist
2917          T
2918          CL-USER&gt; #&amp;?foo
2919          NIL
2920        </programlisting>
2921
2922      </refsect1>
2923    </refentry>
2924
2925    <!-- ====================================  -->
2926    <refentry id="rm_sharpsign-dollarsign">
2927      <indexterm zone="rm_sharpsign-dollarsign">
2928        <primary>#$</primary>
2929      </indexterm>
2930     
2931      <refnamediv>
2932        <refname>#$</refname>
2933        <refpurpose></refpurpose>
2934        <refclass>Reader Macro</refclass>
2935      </refnamediv>
2936     
2937      <refsect1>
2938        <title>Description</title>
2939       
2940            <para>In &CCL; 0.14.2 and later, the #? reader macro can be used
2941              to access foreign constants; this functionality depends on the
2942              presence of &#34;constants.cdb&#34; files in the interface
2943              database. The current behavior of the #$ reader macro is
2944              to:</para>
2945
2946            <para>Read a symbol from the current input stream, with
2947              *PACKAGE* bound to the &#34;OS&#34; package and with
2948              readtable-case preserved.</para>
2949           
2950            <para>Use that symbol&#39;s pname to access the &CCL; interface
2951              database, signalling an error if no appropriate foreign constant
2952              information can be found with that name in any active interface
2953              directory.</para>
2954
2955            <para>Use type information recorded in the database to construct a
2956              form which can be used to access the foreign constant, and return
2957              that form.</para>
2958
2959            <para>Please note that the set of foreign constants declared in
2960              header files may or may not match the set of foreign constants
2961              exported from libraries. When they do match, the form
2962              constructed by the #$ reader macro manages the details of
2963              resolving and tracking changes to the foreign constant's
2964              address.</para>
2965
2966        <para>You can query whether a given name is defined in the
2967          interface databases by appending the '?' character to the reader
2968          macro; for example:</para>
2969
2970        <programlisting>
2971          CL-USER&gt; #$?SO_KEEPALIVE
2972          T
2973          CL-USER&gt; #$?foo
2974          NIL
2975        </programlisting>
2976
2977      </refsect1>
2978    </refentry>
2979
2980    <!-- ====================================  -->
2981    <refentry id="rm_sharpsign-slash">
2982      <indexterm zone="rm_sharpsign-slash">
2983        <primary>#/</primary>
2984      </indexterm>
2985     
2986      <refnamediv>
2987        <refname>#/</refname>
2988        <refpurpose></refpurpose>
2989        <refclass>Reader Macro</refclass>
2990      </refnamediv>
2991     
2992      <refsect1>
2993        <title>Description</title>
2994       
2995            <para>In &CCL; 1.2 and later, the #/ reader macro can be used to
2996              access foreign functions on the Darwin platform. The current
2997              behavior of the #/ reader macro is to:</para>
2998
2999            <para>Read a symbol from the current input stream, with
3000              *PACKAGE* bound to the "NEXTSTEP-FUNCTIONS" package, with
3001              readtable-case preserved, and with any colons included.</para>
3002
3003        <para>Do limited sanity-checking on the resulting symbol; for
3004          example, any name that contains at least one colon is required
3005          also to end with a colon, to conform to Objective-C
3006          method-naming conventions.</para>
3007
3008        <para>Export the resulting symbol from the "NEXTSTEP-FUNCTIONS"
3009          package and return it.</para>
3010
3011        <para>For example, reading "#/alloc" interns and returns
3012          NEXTSTEP-FUNCTIONS:|alloc|. Reading "#/initWithFrame:" interns
3013          and returns NEXTSTEP-FUNCTIONS:|initWithFrame:|.</para>
3014
3015        <para>A symbol read using this macro can be used as an operand
3016          in most places where an Objective-C message name can be used, such as
3017          in the (OBJ:@SELECTOR ...) construct.</para>
3018
3019        <para>Please note: the reader macro is not rigorous about
3020          enforcing Objective-C method-naming conventions. Despite the
3021          simple checking done by the reader macro, it may still be
3022          possible to use it to construct invalid names.</para>
3023
3024        <para>The act of interning a new symbol in the
3025          NEXTSTEP-FUNCTIONS package triggers an interface database lookup
3026          of Objective-C methods with the corresponding message name.  If any
3027          such information is found, a special type of dispatching
3028          function is created and initialized and the new symbol is given
3029          the newly-created dispatching function as its function
3030          definition.</para>
3031
3032        <para>The dispatching knows how to call declared Objective-C methods
3033          defined on the message. In many cases, all methods have the same
3034          foreign type signature, and the dispatching function merely
3035          passes any arguments that it receives to a function that does an
3036          Objective-C message send with the indicated foreign argument and return
3037          types. In other cases, where different Objective-C messages have
3038          different type signatures, the dispatching function tries to
3039          choose a function that handles the right type signature based on
3040          the class of the dispatching function's first argument.</para>
3041           
3042        <para>If new information about Objective-C methods is introduced
3043          (e.g., by using additional interface files or as Objective-C
3044          methods are defined from lisp), the dispatch function is
3045          reinitialized to recognize newly-introduced foreign type
3046          signatures.</para>
3047
3048        <para>The argument and result coercion that the bridge has
3049          traditionally supported is supported by the new mechanism (e.g.,
3050          :&lt;BOOL&gt; arguments can be specified as lisp booleans and :&lt;BOOL&gt;
3051          results are returned as lisp boolean values, and an argument
3052          value of NIL is coerced to a null pointer if the corresponding
3053          argument type is :ID.</para>
3054
3055        <para>Some Objective-C methods accept variable numbers of
3056          arguments; the foreign types of non-required arguments are
3057          determined by the lisp types of those arguments (e.g., integers
3058          are passed as integers, floats as floats, pointers as pointers,
3059          record types by reference.)</para>
3060
3061        <para>Examples:</para>
3062
3063        <programlisting>
3064          ;;; #/alloc is a known message.
3065          ? #'#/alloc
3066          #&lt;OBJC-DISPATCH-FUNCTION NEXTSTEP-FUNCTIONS:|alloc| #x300040E94EBF&gt;
3067          ;;; Sadly, #/foo is not ...
3068          ? #'#/foo
3069          &gt; Error: Undefined function: NEXTSTEP-FUNCTIONS:|foo|
3070
3071          ;;; We can send an "init" message to a newly-allocated instance of
3072          ;;; "NSObject" by:
3073
3074          (send (send ns:ns-object 'alloc) 'init)
3075
3076          ;;; or by
3077
3078          (#/init (#/alloc ns:ns-object))
3079        </programlisting>
3080
3081        <para>Objective-C methods that "return" structures return them
3082          as garbage-collectable pointers when called via dispatch
3083          functions.  For example, if "my-window" is an NS:NS-WINDOW
3084          instance, then</para>
3085
3086        <programlisting>
3087          (#/frame my-window)
3088        </programlisting>
3089
3090        <para>returns a garbage-collectable pointer to a structure that
3091          describes that window's frame rectangle. This convention means
3092          that there's no need to use SLET or special structure-returning
3093          message send syntax; keep in mind, though, that #_malloc,
3094          #_free, and the GC are all involved in the creation and eventual
3095          destruction of structure-typed return values. In some programs
3096          these operations may have an impact on performance.</para>
3097
3098      </refsect1>
3099    </refentry>
3100
3101    <!-- ====================================  -->
3102    <refentry id="rm_sharpsign-greaterthan">
3103      <indexterm zone="rm_sharpsign-greaterthan">
3104        <primary>#&gt;</primary>
3105      </indexterm>
3106     
3107      <refnamediv>
3108        <refname>#&gt;</refname>
3109        <refpurpose></refpurpose>
3110        <refclass>Reader Macro</refclass>
3111      </refnamediv>
3112     
3113      <refsect1>
3114        <title>Description</title>
3115       
3116        <para>In Clozure CL 1.2 and later, the #&gt; reader macro reads
3117          the following text as a keyword, preserving the case of the
3118          text. For example:</para>
3119
3120        <programlisting>
3121          CL-USER&gt; #&gt;FooBar
3122          :&lt;F&gt;OO&lt;B&gt;AR
3123        </programlisting>
3124
3125        <para>The resulting keyword can be used as the name of foreign
3126          types, records, and accessors.</para>
3127       
3128      </refsect1>
3129    </refentry>
3130
3131    <!-- ====================================  -->
3132    <refentry id="f_close-shared-library">
3133          <indexterm zone="f_close-shared-library">
3134            <primary>close-shared-library</primary>
3135          </indexterm>
3136
3137          <refnamediv>
3138            <refname>CLOSE-SHARED-LIBRARY</refname>
3139            <refpurpose>Stops using a shared library, informing the operating
3140              system that it can be unloaded if appropriate.</refpurpose>
3141            <refclass>Function</refclass>
3142          </refnamediv>
3143
3144          <refsynopsisdiv>
3145            <synopsis>
3146              <function>close-shared-library</function> library &key;
3147              completely</synopsis>
3148          </refsynopsisdiv>
3149
3150          <refsect1>
3151            <title>Values</title>
3152
3153            <variablelist>
3154              <varlistentry>
3155                <term>library</term>
3156
3157                <listitem>
3158                      <para>either an object of type SHLIB, or a string which
3159                        designates one by its so-name.</para>
3160                </listitem>
3161              </varlistentry>
3162
3163              <varlistentry>
3164                <term>completely</term>
3165
3166                <listitem>
3167                      <para>a boolean.  The default is T.</para>
3168                </listitem>
3169              </varlistentry>
3170            </variablelist>
3171          </refsect1>
3172
3173          <refsect1>
3174            <title>Description</title>
3175
3176            <para>If <varname>completely</varname> is T, sets the
3177              reference count of <varname>library</varname> to 0.  Otherwise,
3178              decrements it by 1.  In either case, if the reference count
3179              becomes 0, <function>close-shared-library</function>
3180              frees all memory resources consumed <varname>library</varname>
3181              and
3182              causes any EXTERNAL-ENTRY-POINTs known to be defined by it to
3183              become unresolved.</para>
3184          </refsect1>
3185    </refentry>
3186
3187    <!-- ====================================  -->
3188    <refentry id="m_defcallback">
3189          <indexterm zone="m_defcallback">
3190            <primary>defcallback</primary>
3191          </indexterm>
3192
3193          <refnamediv>
3194            <refname>DEFCALLBACK</refname>
3195            <refpurpose></refpurpose>
3196            <refclass>Macro</refclass>
3197          </refnamediv>
3198
3199          <refsynopsisdiv>
3200            <synopsis>
3201              <function>defcallback</function> name
3202              ({arg-type-specifier var}* &optional; result-type-specifier)
3203              &body; body
3204            </synopsis>
3205          </refsynopsisdiv>
3206
3207          <refsect1>
3208            <title>Values</title>
3209
3210            <variablelist>
3211              <varlistentry>
3212                <term>name</term>
3213
3214                <listitem>
3215                      <para>A symbol which can be made into a special variable</para>
3216                </listitem>
3217              </varlistentry>
3218
3219              <varlistentry>
3220                <term>arg-type-specifer</term>
3221
3222                <listitem>
3223                      <para>One of the foreign argument-type keywords,
3224                        described above, or an equivalent <link
3225                                                             linkend="Specifying-And-Using-Foreign-Types">foreign
3226                          type specifier</link>.  In addition, if the keyword
3227                        :WITHOUT-INTERRUPTS is specified, the callback will be
3228                        executed with lisp interrupts disabled if the
3229                        corresponding var is non-NIL. If :WITHOUT-INTERRUPTS
3230                        is specified more than once, the rightmost instance
3231                        wins.</para>
3232                </listitem>
3233              </varlistentry>
3234
3235              <varlistentry>
3236                <term>var</term>
3237
3238                <listitem>
3239                      <para>A symbol (lisp variable), which will be bound to a
3240                        value of the specified type.</para>
3241                </listitem>
3242              </varlistentry>
3243
3244              <varlistentry>
3245                <term>body</term>
3246
3247                <listitem>
3248                      <para>A sequence of lisp forms, which should return a value
3249                        which can be coerced to the specified result-type.</para>
3250                </listitem>
3251              </varlistentry>
3252            </variablelist>
3253          </refsect1>
3254
3255          <refsect1>
3256            <title>Description</title>
3257
3258            <para>Proclaims <varname>name</varname>
3259              to be a special variable; sets its value to a
3260              MACPTR which, when called by foreign code, calls a lisp function
3261              which expects foreign arguments of the specified types and which
3262              returns a foreign value of the specified result type. Any argument
3263              variables which correspond to foreign arguments of type :ADDRESS
3264              are bound to stack-allocated MACPTRs.</para>
3265           
3266            <para>If <varname>name</varname>
3267              is already a callback function pointer, its value is
3268              not changed; instead, it&#39;s arranged
3269              that an
3270              updated version of the lisp callback function will be called.
3271              This feature allows for callback functions to be redefined
3272              incrementally, just like Lisp functions are.</para>
3273
3274            <para><function>defcallback</function>
3275              returns the callback pointer, e.g., the
3276              value of <varname>name</varname>.</para>
3277          </refsect1>
3278    </refentry>
3279
3280    <!-- ====================================  -->
3281    <refentry id="m_def-foreign-type">
3282          <indexterm zone="m_def-foreign-type">
3283            <primary>def-foreign-type</primary>
3284          </indexterm>
3285
3286          <refnamediv>
3287            <refname>DEF-FOREIGN-TYPE</refname>
3288            <refpurpose></refpurpose>
3289            <refclass>Macro</refclass>
3290          </refnamediv>
3291
3292          <refsynopsisdiv>
3293            <synopsis>
3294              <function>def-foreign-type</function> name foreign-type-spec
3295            </synopsis>
3296          </refsynopsisdiv>
3297
3298          <refsect1>
3299            <title>Values</title>
3300
3301            <variablelist>
3302              <varlistentry>
3303                <term>name</term>
3304               
3305                <listitem>
3306                      <para>NIL or a keyword; the keyword may contain
3307                        <link linkend="Case-sensitivity-of-foreign-names-in-CCL" >escaping constructs</link>.</para>
3308                </listitem>
3309              </varlistentry>
3310             
3311              <varlistentry>
3312                <term>foreign-type-spec</term>
3313               
3314                <listitem>
3315                      <para>A foreign type specifier, whose syntax is (loosely)
3316                        defined above.</para>
3317                </listitem>
3318              </varlistentry>
3319            </variablelist>
3320          </refsect1>
3321
3322          <refsect1>
3323            <title>Description</title>
3324
3325            <para>If name is non-NIL, defines name to be an alias for the
3326              foreign type specified by foreign-type-spec. If foreign-type-spec
3327              is a named structure or union type, additionally defines that
3328              structure or union type.</para>
3329           
3330            <para>If name is NIL, foreign-type-spec must be a named foreign
3331              struct or union definition, in which case the foreign structure
3332              or
3333              union definition is put in effect.</para>
3334           
3335            <para>Note that there are two separate namespaces for foreign
3336              type names, one for the names of ordinary types and one for
3337              the names of structs and unions.  Which one
3338              <varname>name</varname> refers to depends on
3339              <varname>foreign-type-spec</varname> in the obvious manner.
3340            </para>
3341          </refsect1>
3342    </refentry>
3343
3344    <!-- ====================================  -->
3345    <refentry id="m_external">
3346          <indexterm zone="m_external">
3347            <primary>external</primary>
3348          </indexterm>
3349         
3350          <refnamediv>
3351            <refname>EXTERNAL</refname>
3352            <refpurpose>Resolves a reference to an external symbol which
3353              is defined in a shared library.</refpurpose>
3354            <refclass>Macro</refclass>
3355          </refnamediv>
3356
3357          <refsynopsisdiv>
3358            <synopsis>
3359              <function>external</function> name => entry
3360            </synopsis>
3361          </refsynopsisdiv>
3362
3363          <refsect1>
3364            <title>Values</title>
3365
3366            <variablelist>
3367              <varlistentry>
3368                <term>name</term>
3369                <listitem>
3370                      <para>
3371                        a simple-string which names an external symbol.
3372                        Case-sensitive.
3373                      </para>
3374                </listitem>
3375              </varlistentry>
3376              <varlistentry>
3377                <term>entry</term>
3378                <listitem>
3379                      <para>
3380                        an object of type EXTERNAL-ENTRY-POINT which maintains
3381                        the address of the foreign symbol named by
3382                        <varname>name</varname>.
3383                      </para>
3384                </listitem>
3385              </varlistentry>
3386            </variablelist>
3387          </refsect1>
3388
3389          <refsect1>
3390            <title>Description</title>
3391
3392            <para>If there is already an EXTERNAL-ENTRY-POINT for
3393              the symbol named by <varname>name</varname>, finds it and
3394              returns it.  If not, creates one and returns it.</para>
3395
3396            <para>Tries to resolve the entry point to a memory address,
3397              and identify the containing library.</para>
3398
3399            <para>Be aware that under Darwin, external functions which
3400              are callable from C have underscores prepended to their names,
3401              as in "_fopen".</para>
3402          </refsect1>
3403    </refentry>
3404
3405    <!-- ====================================  -->
3406    <refentry id="m_external-call">
3407          <indexterm zone="m_external-call">
3408            <primary>external-call</primary>
3409          </indexterm>
3410
3411          <refnamediv>
3412            <refname>EXTERNAL-CALL</refname>
3413            <refpurpose></refpurpose>
3414            <refclass>Macro</refclass>
3415          </refnamediv>
3416
3417          <refsynopsisdiv>
3418            <synopsis>
3419              <function>external-call</function> name
3420              {arg-type-specifier arg}* &optional; result-type-specifier
3421            </synopsis>
3422          </refsynopsisdiv>
3423
3424          <refsect1>
3425            <title>Values</title>
3426
3427            <variablelist>
3428              <varlistentry>
3429                <term>name</term>
3430
3431                <listitem>
3432                      <para>A lisp string. See external, above.</para>
3433                </listitem>
3434              </varlistentry>
3435
3436              <varlistentry>
3437                <term>arg-type-specifer</term>
3438
3439                <listitem>
3440                      <para>One of the foreign argument-type keywords, described
3441                        above, or an equivalent <link linkend="Specifying-And-Using-Foreign-Types">foreign
3442                          type specifier</link>.</para>
3443                </listitem>
3444              </varlistentry>
3445
3446              <varlistentry>
3447                <term>arg</term>
3448
3449                <listitem>
3450                      <para>A lisp value of type indicated by the corresponding
3451                        arg-type-specifier</para>
3452                </listitem>
3453              </varlistentry>
3454
3455              <varlistentry>
3456                <term>result-type-specifier</term>
3457
3458                <listitem>
3459                      <para>One of the foreign argument-type keywords, described
3460                        above, or an equivalent <link linkend="Specifying-And-Using-Foreign-Types">foreign
3461                          type specifier</link>.</para>
3462                </listitem>
3463              </varlistentry>
3464            </variablelist>
3465          </refsect1>
3466
3467          <refsect1>
3468            <title>Description</title>
3469
3470            <para>Calls the foreign function at the address obtained by
3471              resolving the external-entry-point associated with name, passing
3472              the values of each arg as a foreign argument of type indicated by
3473              the corresponding arg-type-specifier. Returns the foreign function
3474              result (coerced to a Lisp object of type indicated by
3475              result-type-specifier), or NIL if result-type-specifer is :VOID or
3476              NIL</para>
3477          </refsect1>
3478    </refentry>
3479
3480    <!-- ====================================  -->
3481    <refentry id="f_Pff-call">
3482          <indexterm zone="f_Pff-call">
3483            <primary>%ff-call</primary>
3484          </indexterm>
3485
3486          <refnamediv>
3487            <refname>%FF-CALL</refname>
3488            <refpurpose></refpurpose>
3489            <refclass>Function</refclass>
3490          </refnamediv>
3491
3492          <refsynopsisdiv>
3493            <synopsis>
3494              <function>%ff-call</function> entrypoint
3495              {arg-type-keyword arg}* &optional; result-type-keyword
3496            </synopsis>
3497          </refsynopsisdiv>
3498
3499          <refsect1>
3500            <title>Values</title>
3501
3502            <variablelist>
3503              <varlistentry>
3504                <term>entrypoint</term>
3505               
3506                <listitem>
3507                      <para>A fixnum or MACPTR</para>
3508                </listitem>
3509              </varlistentry>
3510
3511              <varlistentry>
3512                <term>arg-type-keyword</term>
3513
3514                <listitem>
3515                      <para>One of the foreign argument-type keywords, described
3516                        above</para>
3517                </listitem>
3518              </varlistentry>
3519
3520              <varlistentry>
3521                <term>arg</term>
3522
3523                <listitem>
3524                      <para>A lisp value of type indicated by the corresponding
3525                        arg-type-keyword</para>
3526                </listitem>
3527              </varlistentry>
3528
3529              <varlistentry>
3530                <term>result-type-keyword</term>
3531
3532                <listitem>
3533                      <para>One of the foreign argument-type keywords, described
3534                        above</para>
3535                </listitem>
3536              </varlistentry>
3537            </variablelist>
3538          </refsect1>
3539
3540          <refsect1>
3541            <title>Description</title>
3542
3543            <para>Calls the foreign function at address entrypoint passing the
3544              values of each arg as a foreign argument of type indicated by the
3545              corresponding arg-type-keyword. Returns the foreign function
3546              result (coerced to a Lisp object of type indicated by
3547              result-type-keyword), or NIL if result-type-keyword is :VOID or
3548              NIL</para>
3549          </refsect1>
3550    </refentry>
3551
3552    <!-- ====================================  -->
3553    <refentry id="m_ff-call">
3554          <indexterm zone="m_ff-call">
3555            <primary>ff-call</primary>
3556          </indexterm>
3557
3558          <refnamediv>
3559            <refname>FF-CALL</refname>
3560            <refpurpose></refpurpose>
3561            <refclass>Macro</refclass>
3562          </refnamediv>
3563
3564          <refsynopsisdiv>
3565            <synopsis>
3566              <function>ff-call</function> entrypoint
3567              {arg-type-specifier arg}* &optional; result-type-specifier
3568            </synopsis>
3569          </refsynopsisdiv>
3570
3571          <refsect1>
3572            <title>Values</title>
3573
3574            <variablelist>
3575              <varlistentry>
3576                <term>entrypoint</term>
3577
3578                <listitem>
3579                      <para>A fixnum or MACPTR</para>
3580                </listitem>
3581              </varlistentry>
3582
3583              <varlistentry>
3584                <term>arg-type-specifer</term>
3585
3586                <listitem>
3587                      <para>One of the foreign argument-type keywords, described
3588                        above, or an equivalent <link linkend="Specifying-And-Using-Foreign-Types">foreign
3589                          type specifier</link>.</para>
3590                </listitem>
3591              </varlistentry>
3592
3593              <varlistentry>
3594                <term>arg</term>
3595
3596                <listitem>
3597                      <para>A lisp value of type indicated by the corresponding
3598                        arg-type-specifier</para>
3599                </listitem>
3600              </varlistentry>
3601
3602              <varlistentry>
3603                <term>result-type-specifier</term>
3604
3605                <listitem>
3606                      <para>One of the foreign argument-type keywords, described
3607                        above, or an equivalent <link linkend="Specifying-And-Using-Foreign-Types">foreign
3608                          type specifier</link>.</para>
3609                </listitem>
3610              </varlistentry>
3611            </variablelist>
3612          </refsect1>
3613
3614          <refsect1>
3615            <title>Description</title>
3616
3617            <para>Calls the foreign function at address entrypoint passing the
3618              values of each arg as a foreign argument of type indicated by the
3619              corresponding arg-type-specifier. Returns the foreign function
3620              result (coerced to a Lisp object of type indicated by
3621              result-type-specifier), or NIL if result-type-specifer is :VOID or
3622              NIL</para>
3623          </refsect1>
3624    </refentry>
3625
3626    <!-- ====================================  -->
3627    <refentry id="f_foreign-symbol-address">
3628          <indexterm zone="f_foreign-symbol-address">
3629            <primary>foreign-symbol-address</primary>
3630          </indexterm>
3631
3632          <refnamediv>
3633            <refname>FOREIGN-SYMBOL-ADDRESS</refname>
3634            <refpurpose></refpurpose>
3635            <refclass>Function</refclass>
3636          </refnamediv>
3637
3638          <refsynopsisdiv>
3639            <synopsis>
3640              <function>foreign-symbol-address</function> name
3641            </synopsis>
3642          </refsynopsisdiv>
3643
3644          <refsect1>
3645            <title>Values</title>
3646
3647            <variablelist>
3648              <varlistentry>
3649                <term>name</term>
3650
3651                <listitem>
3652                      <para>A lisp string.</para>
3653                </listitem>
3654              </varlistentry>
3655            </variablelist>
3656          </refsect1>
3657
3658          <refsect1>
3659            <title>Description</title>
3660
3661            <para>Tries to resolve the address of the foreign symbol
3662              name. If successful, returns that address encapsulated in
3663              <link
3664                 linkend="Referencing-and-Using-Foreign-Memory-Addresses">a
3665                MACPTR</link>, else returns NIL.</para>
3666          </refsect1>
3667    </refentry>
3668
3669    <!-- ====================================  -->
3670    <refentry id="f_foreign-symbol-entry">
3671          <indexterm zone="f_foreign-symbol-entry">
3672            <primary>foreign-symbol-entry</primary>
3673          </indexterm>
3674
3675          <refnamediv>
3676            <refname>FOREIGN-SYMBOL-ENTRY</refname>
3677            <refpurpose></refpurpose>
3678            <refclass>Function</refclass>
3679          </refnamediv>
3680
3681          <refsynopsisdiv>
3682            <synopsis>
3683              <function>foreign-symbol-entry</function> name
3684            </synopsis>
3685          </refsynopsisdiv>
3686
3687          <refsect1>
3688            <title>Values</title>
3689
3690            <variablelist>
3691              <varlistentry>
3692                <term>name</term>
3693
3694                <listitem>
3695                      <para>A lisp string.</para>
3696                </listitem>
3697              </varlistentry>
3698            </variablelist>
3699          </refsect1>
3700
3701          <refsect1>
3702            <title>Description</title>
3703
3704            <para>Tries to resolve the address of the foreign symbol name. If
3705              successful, returns a fixnum representation of that address, else
3706              returns NIL.</para>
3707          </refsect1>
3708    </refentry>
3709
3710    <!-- ====================================  -->
3711    <refentry id="f_free">
3712      <indexterm zone="f_free">
3713        <primary>free</primary>
3714      </indexterm>
3715     
3716      <refnamediv>
3717        <refname>FREE</refname>
3718        <refpurpose></refpurpose>
3719        <refclass>Function</refclass>
3720      </refnamediv>
3721     
3722          <refsynopsisdiv>
3723            <synopsis>
3724              <function>free</function> ptr
3725            </synopsis>
3726          </refsynopsisdiv>
3727
3728          <refsect1>
3729            <title>Values</title>
3730
3731            <variablelist>
3732              <varlistentry>
3733                <term>ptr</term>
3734
3735                <listitem>
3736                      <para>A <code>MACPTR</code> that points to a block of
3737                      foreign, heap-allocated memory.</para>
3738                </listitem>
3739              </varlistentry>
3740            </variablelist>
3741          </refsect1>
3742      <refsect1>
3743        <title>Description</title>
3744       
3745        <para>In Clozure CL 1.2 and later, the <code>CCL:FREE</code>
3746          function invokes the foreign <code>free</code> function from
3747          the platform's standard C library to deallocate a block of
3748          foreign memory.</para>
3749
3750        <para>Previous versions of &CCL; implemented this function,
3751          but it was not exported.</para>
3752
3753        <para>If the argument to <code>CCL:FREE</code> is a gcable
3754        pointer (for example, an object returned
3755        by <code>MAKE-GCABLE-RECORD</code>)
3756        then <code>CCL:FREE</code> informs the garbage collector that
3757        the foreign memory has been deallocated before calling the
3758        foreign <code>free</code> function.</para>
3759       
3760      </refsect1>
3761    </refentry>
3762
3763    <!-- ====================================  -->
3764    <refentry id="m_makegcable--record">
3765      <indexterm zone="m_make-record">
3766            <primary>make-gcable-record</primary>
3767      </indexterm>
3768
3769      <refnamediv>
3770            <refname>MAKE-GCABLE-RECORD</refname>
3771            <refpurpose></refpurpose>
3772            <refclass>Macro</refclass>
3773      </refnamediv>
3774
3775      <refsynopsisdiv>
3776            <synopsis>
3777              <function>make-gcable-record</function> typespec
3778              &rest; initforms => result
3779            </synopsis>
3780      </refsynopsisdiv>
3781
3782      <refsect1>
3783            <title>Values</title>
3784
3785            <variablelist>
3786              <varlistentry>
3787                <term>typespec</term>
3788
3789                <listitem>
3790                      <para>A foreign type specifier, or a keyword which is used
3791                        as the name of a foreign struct or union.</para>
3792                </listitem>
3793              </varlistentry>
3794
3795              <varlistentry>
3796                <term>initforms</term>
3797
3798                <listitem>
3799                      <para>If the type denoted by <varname>typespec</varname>
3800                        is scalar, a single value appropriate for that type;
3801                        otherwise, a list of alternating field names and
3802                        values appropriate for the types of those fields.</para>
3803                </listitem>
3804              </varlistentry>
3805
3806              <varlistentry>
3807                <term>result</term>
3808
3809                <listitem>
3810                      <para>
3811                        A <type>macptr</type> which encapsulates the address of a
3812                        newly-allocated record on the foreign heap. The foreign
3813                        object returned by <function>make-gcable-record</function>
3814                        is freed when the garbage collector determines that
3815                        the <code>MACPTR</code> object that describes it is
3816                        unreachable.</para>
3817                </listitem>
3818              </varlistentry>
3819            </variablelist>
3820      </refsect1>
3821
3822      <refsect1>
3823            <title>Description</title>
3824
3825            <para>
3826              Allocates a block of foreign memory suitable to hold the foreign
3827              type described by <code>typespec</code>, in the same manner
3828              as <link linkend="anchor_make-record">MAKE-RECORD</link>. In
3829              addition, <code>MAKE-GCABLE-RECORD</code> marks the
3830              returned object gcable; in other words, it informs the garbage
3831              collector that it may reclaim the object when it becomes
3832              unreachable.
3833            </para>
3834
3835        <para>In all other respects, <code>MAKE-GCABLE-RECORD</code> works
3836          the same way
3837          as <link linkend="anchor_make-record">MAKE-RECORD</link></para>
3838
3839        <para> When using gcable pointers, it's important to remember the
3840          distinction between a <code>MACPTR</code> object (which is a
3841          lisp object, more or less like any other) and the block of
3842          foreign memory that the <code>MACPTR</code> object points to.
3843          If a gcable <code>MACPTR</code> object is the only thing in the
3844          world (lisp world or foreign world) that references the
3845          underlying block of foreign memory, then freeing the foreign
3846          memory when it becomes impossible to reference it is convenient
3847          and sane.  If other lisp <code>MACPTR</code>s reference the
3848          underlying block of foreign memory or if the address of that
3849          foreign memory is passed to and retained by foreign code, having
3850          the GC free the memory may have unpleasant consequences if those
3851          other references are used.</para>
3852
3853        <para>Take care, therefore, not to create a gcable record unless
3854          you are sure that the returned <code>MACPTR</code> will be the
3855          only reference to the allocated memory that will ever be
3856          used.</para>
3857      </refsect1>
3858
3859    </refentry>
3860
3861
3862    <!-- ====================================  -->
3863    <refentry id="m_make-record">
3864          <indexterm zone="m_make-record">
3865            <primary>make-record</primary>
3866          </indexterm>
3867
3868          <refnamediv>
3869        <anchor id="anchor_make-record"/>
3870            <refname>MAKE-RECORD</refname>
3871            <refpurpose></refpurpose>
3872            <refclass>Macro</refclass>
3873          </refnamediv>
3874
3875          <refsynopsisdiv>
3876            <synopsis>
3877              <function>make-record</function> typespec
3878              &rest; initforms => result
3879            </synopsis>
3880          </refsynopsisdiv>
3881
3882          <refsect1>
3883            <title>Values</title>
3884
3885            <variablelist>
3886              <varlistentry>
3887                <term>typespec</term>
3888
3889                <listitem>
3890                      <para>A foreign type specifier, or a keyword which is used
3891                        as the name of a foreign struct or union.</para>
3892                </listitem>
3893              </varlistentry>
3894
3895              <varlistentry>
3896                <term>initforms</term>
3897
3898                <listitem>
3899                      <para>If the type denoted by <varname>typespec</varname>
3900                        is scalar, a single value appropriate for that type;
3901                        otherwise, a list of alternating field names and
3902                        values appropriate for the types of those fields.</para>
3903                </listitem>
3904              </varlistentry>
3905
3906              <varlistentry>
3907                <term>result</term>
3908
3909                <listitem>
3910                      <para>
3911                        A <type>macptr</type> which encapsulates the address of a
3912                        newly-allocated record on the foreign heap.
3913                      </para>
3914                </listitem>
3915              </varlistentry>
3916            </variablelist>
3917          </refsect1>
3918
3919          <refsect1>
3920            <title>Description</title>
3921
3922            <para>
3923              Expands into code which allocates and initializes
3924              an instance of the type
3925              denoted by <varname>typespec</varname>, on the foreign
3926              heap.  The record is allocated using the C function
3927              <function>malloc</function>, and the user of
3928              <function>make-record</function> must explicitly call
3929              the function <function>CCL:FREE</function> to deallocate the
3930              record, when it is no longer needed.
3931            </para>
3932
3933            <para>
3934              If <varname>initforms</varname> is provided, its value
3935              or values are used in the initialization.  When the type
3936              is a scalar, <varname>initforms</varname> is either a single
3937              value which can be coerced to that type, or no value, in which
3938              case binary 0 is used.  When the type is a <type>struct</type>,
3939              <varname>initforms</varname> is a list, giving field names
3940              and the values for each.  Each field is treated in the same way
3941              as a scalar is: If a value for it is given, it must be
3942              coerceable to the field's type; if not, binary 0 is used.
3943            </para>
3944
3945            <para>
3946              When the type is an array, <varname>initforms</varname> may
3947              not be provided, because <function>make-record</function>
3948              cannot initialize its values.  <function>make-record</function>
3949              is also unable to initialize fields of a <type>struct</type>
3950              which are themselves
3951              <type>struct</type>s.  The user of
3952              <function>make-record</function> should set these values
3953              by another means.
3954            </para>
3955
3956            <para>
3957              A possibly-significant limitation is that it must be possible to
3958              find the foreign type at the time the macro is expanded;
3959              <function>make-record</function> signals an error if this is
3960              not the case.
3961            </para>
3962          </refsect1>
3963
3964          <refsect1>
3965            <title>Notes</title>
3966
3967            <para>
3968              It is inconvenient that <function>make-record</function> is a
3969              macro, because this means that <varname>typespec</varname>
3970              cannot be a variable; it must be an immediate value.
3971            </para>
3972           
3973            <para>
3974              If it weren't for this requirement,
3975              <function>make-record</function> could be a function.  However,
3976              that would mean that any stand-alone application using it would
3977              have to include a copy of the interface database
3978              (see <xref linkend="The-Interface-Database"/>), which is undesirable
3979              because it's large.
3980            </para>
3981          </refsect1>
3982    </refentry>
3983
3984    <!-- ====================================  -->
3985    <refentry id="f_open-shared-library">
3986          <indexterm zone="f_open-shared-library">
3987            <primary>open-shared-library</primary>
3988          </indexterm>
3989         
3990          <refnamediv>
3991            <refname>OPEN-SHARED-LIBRARY</refname>
3992            <refpurpose>Asks the operating system to load a shared library
3993              for &CCL; to use.</refpurpose>
3994            <refclass>Function</refclass>
3995          </refnamediv>
3996         
3997          <refsynopsisdiv>
3998            <synopsis>
3999              <function>open-shared-library</function> name => library
4000            </synopsis>
4001          </refsynopsisdiv>
4002         
4003          <refsect1>
4004            <title>Values</title>
4005           
4006            <variablelist>
4007              <varlistentry>
4008                <term>name</term>       
4009                <listitem>
4010                      <para>A SIMPLE-STRING which is presumed to be the so-name of
4011                        or a filesystem path to the library.</para>
4012                </listitem>
4013              </varlistentry>
4014             
4015              <varlistentry>
4016                <term>library</term>
4017                <listitem>
4018                      <para>An object of type SHLIB which describes the
4019                        library denoted by <varname>name</varname>.</para>
4020                </listitem>
4021              </varlistentry>
4022            </variablelist>
4023          </refsect1>
4024         
4025          <refsect1>
4026            <title>Description</title>
4027
4028            <para>If the library denoted by <varname>name</varname> can
4029              be loaded by the
4030              operating system, returns an object of type SHLIB that describes
4031              the library; if the library is already open, increments a
4032              reference count. If the library can&#39;t be loaded, signals a
4033              SIMPLE-ERROR which contains an often-cryptic message from the
4034              operating system.</para>
4035          </refsect1>
4036
4037          <refsect1>
4038            <title>Examples</title>
4039
4040            <programlisting format="linespecific">;;; Try to do something simple.
4041          ? (open-shared-library &#34;libgtk.so&#34;)
4042          &#62; Error: Error opening shared library &#34;libgtk.so&#34;: /usr/lib/libgtk.so: undefined symbol: gdk_threads_mutex
4043          &#62; While executing: OPEN-SHARED-LIBRARY
4044
4045          ;;; Grovel around, curse, and try to find out where &#34;gdk_threads_mutex&#34;
4046          ;;; might be defined. Then try again:
4047
4048          ? (open-shared-library &#34;libgdk.so&#34;)
4049          #&#60;SHLIB libgdk.so #x3046DBB6&#62;
4050
4051          ? (open-shared-library &#34;libgtk.so&#34;)
4052          #&#60;SHLIB libgtk.so #x3046DC86&#62;
4053
4054          ;;; Reference an external symbol defined in one of those libraries.
4055
4056          ? (external &#34;gtk_main&#34;)
4057          #&#60;EXTERNAL-ENTRY-POINT &#34;gtk_main&#34; (#x012C3004) libgtk.so #x3046FE46&#62;
4058
4059          ;;; Close those libraries.
4060
4061          ? (close-shared-library &#34;libgtk.so&#34;)
4062          T
4063
4064          ? (close-shared-library &#34;libgdk.so&#34;)
4065          T
4066
4067          ;;; Reference the external symbol again.
4068
4069          ? (external &#34;gtk_main&#34;)
4070          #&#60;EXTERNAL-ENTRY-POINT &#34;gtk_main&#34; {unresolved} libgtk.so #x3046FE46&#62;</programlisting>
4071          </refsect1>
4072
4073          <refsect1>
4074            <title>Notes</title>
4075
4076            <para>It would be helpful to describe what an soname is and give
4077              examples of one.</para>
4078
4079            <para>Does the SHLIB still get returned if the library is
4080              already open?</para>
4081          </refsect1>
4082    </refentry>
4083   
4084    <!-- ====================================  -->
4085    <refentry id="m_pref">
4086          <indexterm zone="m_pref">
4087            <primary>pref</primary>
4088          </indexterm>
4089
4090          <refnamediv>
4091            <refname>PREF</refname>
4092            <refpurpose></refpurpose>
4093            <refclass>Macro</refclass>
4094          </refnamediv>
4095
4096          <refsynopsisdiv>
4097            <synopsis>
4098              <function>pref</function> ptr accessor-form
4099            </synopsis>
4100
4101          </refsynopsisdiv>
4102
4103          <refsect1>
4104            <title>Values</title>
4105
4106            <variablelist>
4107              <varlistentry>
4108                <term>ptr</term>
4109
4110                <listitem>
4111                      <para><link linkend="Referencing-and-Using-Foreign-Memory-Addresses">a MACPTR</link>.</para>
4112                </listitem>
4113              </varlistentry>
4114
4115              <varlistentry>
4116                <term>accessor-form</term>
4117
4118                <listitem>
4119                      <para>a keyword which names a foreign type or record, as
4120                        described in <xref linkend="Foreign-type--record--and-field-names"/>.
4121                      </para>
4122                </listitem>
4123              </varlistentry>
4124            </variablelist>
4125          </refsect1>
4126
4127          <refsect1>
4128            <title>Description</title>
4129
4130            <para>References an instance of a foreign type (or a component of
4131              a foreign type) accessible via ptr.</para>
4132           
4133            <para>Expands into code which references the indicated scalar type
4134              or component, or returns a pointer to a composite type.</para>
4135           
4136            <para>PREF can be used with SETF.</para>
4137           
4138            <para>RREF is a deprecated alternative to PREF. It accepts a
4139              :STORAGE keyword and rather loudly ignores it.</para>
4140          </refsect1>
4141    </refentry>
4142
4143    <!-- ====================================  -->
4144    <refentry id="f_Preference-external-entry-point">
4145          <indexterm zone="f_Preference-external-entry-point">
4146            <primary>%reference-external-entry-point</primary>
4147          </indexterm>
4148
4149          <refnamediv>
4150            <refname>%REFERENCE-EXTERNAL-ENTRY-POINT</refname>
4151            <refpurpose></refpurpose>
4152            <refclass>Function</refclass>
4153          </refnamediv>
4154
4155          <refsynopsisdiv>
4156            <synopsis>
4157              <function>%reference-external-entry-point</function> eep
4158            </synopsis>
4159          </refsynopsisdiv>
4160
4161          <refsect1>
4162            <title>Values</title>
4163
4164            <variablelist>
4165              <varlistentry>
4166                <term>eep</term>
4167
4168                <listitem>
4169                      <para>An EXTERNAL-ENTRY-POINT, as obtained by the EXTERNAL
4170                        macro.</para>
4171                </listitem>
4172              </varlistentry>
4173            </variablelist>
4174          </refsect1>
4175
4176          <refsect1>
4177            <title>Description</title>
4178
4179            <para>Tries to resolve the address of the EXTERNAL-ENTRY-POINT
4180              eep; returns a fixnum representation of that address if
4181              successful, else signals an error.</para>
4182          </refsect1>
4183    </refentry>
4184
4185    <!-- ====================================  -->
4186    <anchor id="anchor_rlet"/>
4187    <refentry id="m_rlet">
4188          <indexterm zone="m_rlet">
4189            <primary>rlet</primary>
4190          </indexterm>
4191
4192          <refnamediv>
4193            <refname>RLET</refname>
4194            <refpurpose></refpurpose>
4195            <refclass>Macro</refclass>
4196          </refnamediv>
4197
4198          <refsynopsisdiv>
4199            <synopsis>
4200              <function>rlet</function> (var typespec &rest; initforms)*
4201              &body; body
4202            </synopsis>
4203          </refsynopsisdiv>
4204
4205          <refsect1>
4206            <title>Values</title>
4207
4208            <variablelist>
4209              <varlistentry>
4210                <term>var</term>
4211
4212                <listitem>
4213                      <para>A symbol (a lisp variable)</para>
4214                </listitem>
4215              </varlistentry>
4216
4217              <varlistentry>
4218                <term>typespec</term>
4219
4220                <listitem>
4221                      <para>A foreign type specifier or foreign record name.</para>
4222                </listitem>
4223              </varlistentry>
4224
4225          <varlistentry>
4226                <term>initforms</term>
4227
4228                <listitem>
4229                      <para>As described above, for
4230                        <xref linkend="m_make-record"/></para>
4231                </listitem>
4232              </varlistentry>
4233            </variablelist>
4234          </refsect1>
4235
4236          <refsect1>
4237            <title>Description</title>
4238
4239            <para>Executes <varname>body</varname>
4240              in an environment in which each var is bound
4241              to <link linkend="Referencing-and-Using-Foreign-Memory-Addresses">a MACPTR</link> encapsulating the
4242              address of a stack-allocated foreign memory block, allocated and
4243              initialized from typespec and initforms as per
4244              <xref linkend="m_make-record"/>.
4245              Returns whatever value(s) <varname>body</varname>
4246              returns.</para>
4247           
4248            <para>Record fields that aren&#39;t explicitly initialized have
4249              unspecified contents.</para>
4250          </refsect1>
4251    </refentry>
4252
4253    <!-- ====================================  -->
4254    <refentry id="m_rletz">
4255          <indexterm zone="m_rletz">
4256            <primary>rletz</primary>
4257          </indexterm>
4258
4259          <refnamediv>
4260            <refname>RLETZ</refname>
4261            <refpurpose></refpurpose>
4262            <refclass>Macro</refclass>
4263          </refnamediv>
4264
4265          <refsynopsisdiv>
4266            <synopsis>
4267              <function>rletz</function> (var typespec &rest; initforms)*
4268              &body; body
4269            </synopsis>
4270          </refsynopsisdiv>
4271
4272          <refsect1>
4273            <title>Values</title>
4274
4275            <variablelist>
4276              <varlistentry>
4277                <term>var</term>
4278
4279                <listitem>
4280                      <para>A symbol (a lisp variable)</para>
4281                </listitem>
4282              </varlistentry>
4283
4284              <varlistentry>
4285                <term>typespec</term>
4286
4287                <listitem>
4288                      <para>A foreign type specifier or foreign record name.</para>
4289                </listitem>
4290              </varlistentry>
4291
4292              <varlistentry>
4293                <term>initforms</term>
4294
4295                <listitem>
4296                      <para>As described above, for ccl:make-record</para>
4297                </listitem>
4298              </varlistentry>
4299            </variablelist>
4300          </refsect1>
4301
4302          <refsect1>
4303            <title>Description</title>
4304
4305            <para>Executes body in an environment in which each var is
4306              bound to <link
4307                          linkend="Referencing-and-Using-Foreign-Memory-Addresses">a
4308                MACPTR</link> encapsulating the address of a stack-allocated
4309              foreign memory block, allocated and initialized from
4310              typespec and initforms as ccl:make-record.</para>
4311           
4312            <para>Returns whatever value(s) body returns.</para>
4313
4314            <para>Unlike rlet, record fields that aren&#39;t explicitly
4315              initialized are set to binary 0.</para>
4316          </refsect1>
4317    </refentry>
4318
4319    <!-- ====================================  -->
4320    <refentry id="f_terminate-when-unreachable">
4321          <indexterm zone="f_terminate-when-unreachable">
4322            <primary>terminate-when-unreachable</primary>
4323          </indexterm>
4324
4325          <refnamediv>
4326            <refname>TERMINATE-WHEN-UNREACHABLE</refname>
4327            <refpurpose></refpurpose>
4328            <refclass>Function</refclass>
4329          </refnamediv>
4330
4331          <refsynopsisdiv>
4332            <synopsis>
4333              <function>terminate-when-unreachable</function> object
4334            </synopsis>
4335          </refsynopsisdiv>
4336
4337          <refsect1>
4338            <title>Values</title>
4339
4340            <variablelist>
4341              <varlistentry>
4342                <term>object</term>
4343
4344                <listitem>
4345                      <para>A CLOS object of a class for which there exists
4346                        a method of the generic function
4347                        <function>ccl:terminate</function>.
4348                      </para>
4349                </listitem>
4350              </varlistentry>
4351            </variablelist>
4352          </refsect1>
4353
4354          <refsect1>
4355            <title>Description</title>
4356
4357            <para>
4358              The "termination" mechanism is a way to have the garbage
4359              collector run a function right before an object is about to
4360              become garbage.  It is very similar to the "finalization"
4361              mechanism which Java has.  It is not standard Common Lisp,
4362              although other Lisp implementations have similar features.
4363              It is useful when there is some sort of special cleanup,
4364              deallocation, or releasing of resources which needs to happen
4365              when a certain object is no longer being used.
4366            </para>
4367
4368            <para>
4369              When the garbage collector discovers that an object is no
4370              longer referred to anywhere in the program, it deallocates
4371              that object, freeing its memory.  However, if
4372              <function>ccl:terminate-when-unreachable</function> has been
4373              called on the object at any time, the garbage collector first
4374              invokes the generic function <function>ccl:terminate</function>,
4375              passing it the object as a parameter.
4376            </para>
4377
4378            <para>
4379              Therefore, to make termination do something useful, you need to
4380              define a method on <function>ccl:terminate</function>.
4381            </para>
4382
4383            <para>
4384              Because calling
4385              <function>ccl:terminate-when-unreachable</function> only
4386              affects a single object, rather than all objects of its
4387              class, you
4388              may wish to put a call to it in the
4389              <function>initialize-instance</function> method of a
4390              class.  Of course, this is only appropriate if you do in fact
4391              want to use termination for all objects of a given class.
4392            </para>
4393          </refsect1>
4394
4395          <refsect1>
4396            <title>Example</title>
4397
4398        <programlisting format="linespecific">
4399          (defclass resource-wrapper ()
4400            ((resource :accessor resource)))
4401
4402          (defmethod initialize-instance :after ((x resource-wrapper) &amp;rest initargs)
4403             (ccl:terminate-when-unreachable x))
4404
4405          (defmethod ccl:terminate ((x resource-wrapper))
4406             (when (resource x)
4407                (deallocate (resource x))))</programlisting>
4408          </refsect1>
4409
4410          <refsect1>
4411            <title>See Also</title>
4412
4413            <simplelist type="inline">
4414              <member><xref linkend="Tutorial--Allocating-Foreign-Data-on-the-Lisp-Heap"/></member>
4415            </simplelist>
4416          </refsect1>
4417
4418    </refentry>
4419
4420     <!-- ====================================  -->
4421    <refentry id="f_unuse-interface-dir">
4422          <indexterm zone="f_unuse-interface-dir">
4423            <primary>unuse-interface-dir</primary>
4424          </indexterm>
4425
4426          <refnamediv>
4427            <refname>UNUSE-INTERFACE-DIR</refname>
4428            <refpurpose></refpurpose>
4429            <refclass>Function</refclass>
4430          </refnamediv>
4431
4432          <refsynopsisdiv>
4433            <synopsis>
4434              <function>unuse-interface-dir</function> dir-id
4435            </synopsis>
4436          </refsynopsisdiv>
4437
4438          <refsect1>
4439            <title>Values</title>
4440
4441            <variablelist>
4442              <varlistentry>
4443                <term>dir-id</term>
4444
4445                <listitem>
4446                      <para>A keyword whose pname, mapped to lower case, names a
4447                        subdirectory of &#34;ccl:headers;&#34; (or
4448                        "ccl:darwin-headers;")</para>
4449                </listitem>
4450              </varlistentry>
4451            </variablelist>
4452          </refsect1>
4453
4454          <refsect1>
4455            <title>Description</title>
4456
4457            <para>Tells &CCL; to remove the interface directory denoted by
4458              dir-id from the list of interface directories which are
4459              consulted for
4460              foreign type and function information. Returns T if the directory
4461              was on the search list, NIL otherwise.</para>
4462          </refsect1>
4463    </refentry>
4464
4465   <!-- ====================================  -->
4466    <refentry id="f_use-interface-dir">
4467          <indexterm zone="f_use-interface-dir">
4468            <primary>use-interface-dir</primary>
4469          </indexterm>
4470
4471          <refnamediv>
4472            <refname>USE-INTERFACE-DIR</refname>
4473            <refpurpose></refpurpose>
4474            <refclass>Function</refclass>
4475          </refnamediv>
4476
4477          <refsynopsisdiv>
4478            <synopsis>
4479              <function>use-interface-dir</function> dir-id
4480            </synopsis>
4481          </refsynopsisdiv>
4482
4483          <refsect1>
4484            <title>Values</title>
4485
4486            <variablelist>
4487              <varlistentry>
4488                <term>dir-id</term>
4489
4490                <listitem>
4491                      <para>A keyword whose pname, mapped to lower case, names a
4492                        subdirectory of &#34;ccl:headers;&#34; (or
4493                        "ccl:darwin-headers;")</para>
4494                </listitem>
4495              </varlistentry>
4496            </variablelist>
4497          </refsect1>
4498
4499          <refsect1>
4500            <title>Description</title>
4501
4502            <para>Tells &CCL; to add the interface directory denoted by
4503              dir-id to the list of interface directories which it consults for
4504              foreign type and function information. Arranges that that
4505              directory is searched before any others.</para>
4506
4507            <para>Note that <function>use-interface-dir</function>
4508              merely adds an entry
4509              to a search list.
4510              If the named directory doesn&#39;t exist in the file system
4511              or doesn&#39;t
4512              contain a set of database files, a runtime error may occur
4513              when &CCL;
4514              tries to open some database file in that directory, and it
4515              will try to
4516              open such a database file whenever it needs to find any
4517              foreign type or
4518              function information. <xref linkend="f_unuse-interface-dir"/>
4519              may come in
4520              handy in that case.</para>
4521          </refsect1>
4522
4523          <refsect1>
4524            <title>Examples</title>
4525
4526            <para>One typically wants interface information to be
4527              available at compile-time (or, in many cases, at read-time).
4528              A typical idiom would be:</para>
4529
4530            <programlisting format="linespecific">(eval-when (:compile-toplevel :execute)
4531          (use-interface-dir :GTK))</programlisting>
4532
4533            <para>Using the :GTK interface directory makes available
4534              information on
4535              foreign types, functions, and constants.  It's generally
4536              necessary to
4537              load foreign libraries before actually calling the
4538              foreign code, which for GTK can be done like this:</para>
4539
4540            <programlisting>(load-gtk-libraries)</programlisting>
4541
4542            <para>It should now be possible to do things like:</para>
4543
4544            <programlisting>(#_gtk_widget_destroy w)</programlisting>
4545          </refsect1>
4546    </refentry>
4547
4548  </sect1>
4549</chapter>
Note: See TracBrowser for help on using the repository browser.