source: trunk/source/doc/src/openmcl-documentation.xml @ 8516

Last change on this file since 8516 was 8516, checked in by gb, 13 years ago

new files, work-in-progress

File size: 536.5 KB
Line 
1<?xml version="1.0" encoding="US-ASCII"?>
2<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
3<!ENTITY rest "<varname><property>&amp;rest</property></varname>">
4<!ENTITY key "<varname><property>&amp;key</property></varname>">
5<!ENTITY optional "<varname><property>&amp;optional</property></varname>">
6<!ENTITY body "<varname><property>&amp;body</property></varname>">
7<!ENTITY aux "<varname><property>&amp;aux</property></varname>">
8<!ENTITY allow-other-keys
9         "<varname><property>&amp;allow-other-keys</property></varname>">
10]>
11<book lang="en">
12 <bookinfo>
13  <title>OpenMCL Documentation</title>
14 </bookinfo>
15
16 <chapter><title>Obtaining, Installing, and Running OpenMCL</title>
17 
18  <sect1><title>Releases and System Requirements</title>
19   
20   <para>There are three active versions of OpenMCL.  Version 1.0 was
21   a stable release (released in late 2005); it is no longer under
22   active development.  Version 1.1 has been under active development
23   since shortly after 1.0 was released; it's been distributed as a
24   series of development "snapshots" and CVS updates.  1.1 snapshots
25   have introduced support for x86-64 platforms, internal use of
26   Unicode, and many other features, but have presented something of a
27   moving target.  Version 1.2 (being released in early 2008) is
28   intended to both a more stable and predictable release schedule and
29   to make it a bit easier for users who wish to track the "bleeding
30   edge" of development to do so.</para>
31
32      <para>Version 1.0 is available for three platform configurations:</para>
33      <itemizedlist>
34        <listitem>
35          <para>Linux on PowerPC (32-bit implementation)</para>
36        </listitem>
37        <listitem>
38          <para>Mac OS X on PowerPC (32-bit implementation)</para>
39        </listitem>
40        <listitem>
41          <para>Mac OS X on PowerPC (64-bit implementation)</para>
42        </listitem>
43      </itemizedlist>
44
45      <para>Versions 1.1 and 1.2 are available for five platform
46      configurations:</para>
47      <itemizedlist>
48        <listitem>
49          <para>Linux on PowerPC (32-bit and 64-bit implementations)</para>
50        </listitem>
51        <listitem>
52          <para>>Mac OS X on PowerPC (32-bit and 64-bit implementations)</para>
53        </listitem>
54        <listitem>
55          <para>Linux on X86-64 (64-bit implementation)</para>
56        </listitem>
57        <listitem>
58          <para>Mac OS X on X86-64 (64-bit implementation)</para>
59        </listitem>
60        <listitem><para>FreeBSD on X86-64 (64-bit implementation)</para></listitem>
61      </itemizedlist>
62
63      <para>A 64-bit version of OpenMCL requires a 64-bit processor
64      (obviously) running a 64-bit OS variant.</para>
65     
66      <para>There are ongoing efforts to port OpenMCL to the Windows
67      operating system and 32-bit x86 processors.</para>
68     
69     
70      <para>Additional platform-specific information is given in the
71      following subsections.</para>
72
73      <sect2><title>LinuxPPC</title> 
74       
75        <para>OpenMCL versions 1.0 and later run under relatively
76        recent versions of LinuxPPC. All versions of OpenMCL require
77        version 2.2.13 (or later) of the Linux kernel and version
78        2.1.3 (or later) of the GNU C library (glibc) at a bare
79        minimum.</para>
80      </sect2>
81
82      <sect2><title>LinuxX8664</title> 
83
84        <para>Version 1.1 and later of OpenMCL runs on relatively
85        recent Linux distributions for the x86-64 architecture.  It
86        requires a Linux with Thread Local Storage support in the
87        toolchain and standard libraries, and the New Posix Thread
88        Library (NPTL).  Fortunately, these features seem to be
89        present in all current Linux distributions for x86-64, though
90        there may be some problems with early Linux distributions for
91        x86-64.  (Some GCC versions older than 4.0 on Linux have been
92        known to have problems compiling some of the C code in the
93        kernel, some very old Linux distributions don't follow the
94        current ABI standards wrt segment register usage, some early
95        Linux kernels for x86-64 had problems mapping large regions of
96        the address space ... it's difficult to enumerate exactly what
97        versions of what Linux distributions have what problems.  A
98        rule of thumb is that - since much of the development of
99        OpenMCL for x86-64 took place in that time frame - Linux
100        distributions released earlier than early 2006 may have
101        problems running OpenMCL.) </para>
102      </sect2>
103
104      <sect2><title>FreeBSD-amd64</title>
105        <para>Versions 1.1 and later of OpenMCL runs on FreeBSD on
106        x86-64 (FreeBSD releases generally call the platform "and64")
107        OpenMCL should run under FreeBSD 6.0 or later; as of this
108        writing, FreeBSD 7.0 is about to be released and it may be
109        necessary for FreeBSD 7 users to install the "compat6x"
110        package in order to use a version of OpenMCL built on FreeBSD
111        6.x under FreeBSD 7.  .</para>
112      </sect2>
113
114      <sect2><title>DarwinPPC-MacOS-X</title>
115
116        <para> OpenMCL 1.0 runs on MacOS X versions 10.2, 10.3 and
117        10.4 on the PowerPC.</para>
118
119        <para>Current development on version 1.1 and later takes place
120        under OS X versions 10.4 and 10.5 and requires at least
121        version 10.3.9</para>
122
123        <para>The 64-bit DarwinPPC version of OpenMCL requires
124        functionality introduced in OSX 10.4 (namely, the ability to
125        run 64-bit binaries).  It also, obviously, requires a G5
126        processor.</para>
127
128        <para>OpenMCL hasn't been tested under Darwin proper, but
129        OpenMCL doesn't intentionally use any MacOS X features beyond
130        the Darwin subset and therefore it seems likely that OpenMCL
131        would run on PPC Darwin versions that correspond to recent OSX
132        versions.</para>
133      </sect2>
134
135      <sect2><title>Darwinx8664-MacOS-X</title>
136        <para>Versions 1.1 and later of OpenMCL runs on 64-bit
137        DarwinX86 (MacOS on Intel).</para>
138
139        <para>OpenMCL Darwinx8664/MacOS X requires a 64-bit processor.
140        All Macintoshes currently sold by Apple (as of early 2008) and
141        all Macintoshes introduced by Apple since August 2006 have
142        such processors.  However, the original MacBooks, MacBook Pros
143        and Intel iMacs (models introduced in early 2006) used 32-bit
144        Core Duo processors, and so OpenMCL will not (yet) run on
145        them.</para>
146
147      </sect2>
148    </sect1>
149
150    <sect1><title>Installation</title>
151      <para>Installing OpenMCL consists of</para>
152      <orderedlist> 
153        <listitem>
154          <para>Downloading an appropriate distribution of OpenMCL;</para>
155        </listitem>
156        <listitem>
157          <para>If necessary, extracting the archive somewhere on your
158          computer;</para>
159        </listitem>
160        <listitem>
161          <para>Configuring a shell script so OpenMCL can locate
162          necessary library files and other auxilliary files.</para>
163        </listitem>
164      </orderedlist>
165
166      <sect2><title>Downloading and untarring OpenMCL binary source
167      releases</title> <para> OpenMCL releases and snapshots are
168      distributed as tarballs (compressed tar archives).</para>
169
170      <para>Tarballs of version 1.0 for supported platforms are
171      available from the download page of the OpenMCL website.</para>
172
173      <para>Tarballs of the latest development snapshots of version
174      1.1, along with release notes, are available from the testing
175      directory on Clozure.com.</para>
176      <para>Both the release and snapshot archives contain a directory
177      named <literal>ccl</literal>, which in turn contains a number of
178      files and subdirectories.  The <literal>ccl</literal> directory
179      can reside anywhere in the filesystem, assuming appropriate
180      permissions. If you wanted the <literal>ccl</literal> directory
181      to reside in <literal>``~/openmcl/ccl''</literal> and the
182      directory <literal>``~/openmcl/''</literal> already existed, you
183      could do anything equivalent to:</para>
184      <programlisting>
185shell&gt; cd ~/openmcl
186shell&gt; tar xvzf <emphasis>path-to-downloaded-openmcl-archive.tar.gz</emphasis>
187      </programlisting>
188      <para><emphasis>Important Note for Macintosh Users:</emphasis>
189      Double-clicking the archive in the Finder may work, but it also
190      may have unintended side-effects.  In some versions of the Mac
191      OS double-clicking an archive will invoke Stuffit, which may try
192      to replace linefeeds with carriage returns as it extracts
193      files. Also, tar can be used to merge the archive contents into
194      an existing <literal>ccl</literal> directory, whereas
195      double-clicking in the Finder will create a new directory named
196      <literal>ccl 2</literal> (or <literal>ccl 3</literal>, or...)
197      Bottom line is that you're better off using tar from the
198      shell.</para>
199
200      <para>Once the <literal>ccl</literal> directory is installed,
201      it's necessary to install and configure a shell script
202      distributed with OpenMCL before using it.</para>
203      </sect2>
204
205      <sect2><title>The-openmcl-Shell-Script"</title>
206      <para>OpenMCL needs to be able to find the
207      <literal>ccl</literal> directory in order to support features
208      such as <literal>require</literal> and
209      <literal>provide</literal>, access to foreign interface
210      information (see ) and the lisp build process (see
211      ). Specifically, it needs to set up logical pathname
212      translations for the <literal>"ccl:"</literal> logical host.  If
213      this logical host isn't defined (or isn't defined correctly),
214      some things might work, some things might not... and it'll
215      generally be hard to invoke and use OpenMCL productively.</para>
216
217        <para>OpenMCL uses the value of the environment variable
218        <literal>CCL_DEFAULT_DIRECTORY</literal> to determine the
219        filesystem location of the <literal>ccl</literal> directory;
220        the openmcl shell script is intended to provide a way to
221        invoke OpenMCL with that environment variable set
222        correctly.</para>
223        <para>There are two versions of the shell script:
224        <literal>"ccl/scripts/openmcl"</literal> is used to invoke
225        32-bit implementations of OpenMCL and
226        <literal>"ccl/scripts/openmcl64"</literal> is used to invoke
227        64-bit implementations.</para>
228        <para>To use the script:</para>
229        <orderedlist>
230          <listitem>
231            <para>Edit the definition of
232            <literal>CCL_DEFAULT_DIRECTORY</literal> near the
233            beginning of theshell script so that it refers to your ccl
234            directory.  Alternately,set
235            <literal>CCL_DEFAULT_DIRECTORY</literal> in your .cshrc,
236            .tcshrc, .bashrc,.bash_profile, .MacOSX/environment.plist,
237            or wherever you usually set environment variables.  If
238            there is an existing definition of thevariable, the
239            openmcl script will not override it.The shell script sets
240            a local variable (<literal>OPENMCL_KERNEL</literal>) to
241            the standard name of the OpenMCL kernel approprate for the
242            platform (asdetermined by 'uname -s'.) You might prefer to
243            set this variable manually in the shell script</para>
244          </listitem>
245
246          <listitem>
247            <para>Ensure that the shell script is executable, for
248            example:<literal>$ chmod +x
249            ~/openmcl/ccl/scripts/openmcl64</literal>This command
250            grants execute permission to the named script. If you
251            areusing a 32-bit platform, substitute "openmcl" in place
252            of "openmcl64".
253            <warning>
254              <para>The above command won't work if you are not the
255              owner of the installed copy of OpenMCL. In that case,
256              you can use the "sudo" command like this:</para>
257              <para><literal>$ sudo chmod +x
258              ~/openmcl/ccl/scripts/openmcl64</literal></para>
259              <para>Give your password when prompted.</para>
260              <para>If the "sudo" command doesn't work, then you are
261              not an administrator on the system you're using, and you
262              don't have the appropriate "sudo" permissions. In that
263              case you'll need to get help from the system's
264              administrator.</para>
265            </warning></para>
266          </listitem>
267          <listitem>
268            <para>Install the shell script somewhere on your shell's
269            search path</para>
270          </listitem>
271        </orderedlist>
272
273        <para>Once this is done, it should be possible to invoke
274        OpenMCL by typing <literal>openmcl</literal> at a shell
275        prompt:</para>
276        <programlisting>
277&gt; openmcl [args ...]
278Welcome to OpenMCL Version whatever (DarwinPPC32)!
279?
280</programlisting>
281        <para>The openmcl shell script will pass all of its arguments
282        to the OpenMCL kernel.  See <xref linkend="invocation"/> for
283        more information about "args".  When invoked this way, the
284        lisp should be able to initialize the
285        <literal>"ccl:"</literal> logical host so that its
286        translations refer to the <literal>"ccl"</literal>
287        directory. To test this, you can call
288        <literal>probe-file</literal> in OpenMCL's read-eval-print
289        loop:</para>
290        <programlisting>
291? (probe-file "ccl:level-1;level-1.lisp")  ;will return the physical pathname of the file
292#P"/Users/alms/my_lisp_stuff/ccl/level-1/level-1.lisp"
293</programlisting>
294      </sect2>
295
296      <sect2 id="Personal-Customization-with-the-Init-File">
297        <title>Personal Customization with the Init File</title>
298        <para>By default OpenMCL will try to load the file
299        <literal>"home:openmcl-init.lisp"</literal> or the compiled
300       
301        <literal>"home:openmcl-init.fasl"</literal> upon starting
302        up. It does this by executing <literal>(load
303        "home:openmcl-init")</literal>.  If it is unable to load the
304        file (for example because it does not exist) no action is
305        taken; an error is not signalled.</para>
306        <para>The <literal>"home:"</literal> prefix to the filename is
307        a Common Lisp logical host, which OpenMCL initializes to refer
308        to your home directory, so this looks for either of the files
309        <literal>~/openmcl-init.lisp</literal> or
310        <literal>~/openmcl-init.fasl</literal>.</para>
311        <para>Since the init file is loaded the same way as normal
312        Lisp code is, you can put anything you want in it.  For
313        example, you can change the working directory, and load
314        packages which you use frequently.</para>
315        <para>To suppress the loading of this init-file, invoke OpenMCL with the
316<literal>--no-init</literal> option.</para>
317      </sect2>
318
319      <sect2 id="Some--hopefully--useful-options">
320        <title>Some (hopefully) useful options</title>
321        <para> The exact set of command-line arguments accepted by
322        OpenMCL may vary slightly from release to release;
323        <literal>openmcl --help</literal> will provide a definitive
324        (if somewhat terse) summary of the options accepted by the
325        current implementation and then exit. Some of those options
326        are described below.</para>
327        <itemizedlist>
328          <listitem>
329            <para>-S (or --stack-size). Specify the size of the initial process
330            stack.</para>
331          </listitem>
332
333          <listitem>
334            <para>-b (or --batch). Execute in &#34;batch mode&#34;. End-of-file
335            from *STANDARD-INPUT* will cause OpenMCL to exit, as will attempts to
336            enter a break loop.</para>
337          </listitem>
338
339          <listitem>
340            <para>-n (or --no-init). If this option is given, the init file
341            is not loaded.  This is useful if OpenMCL is being invoked by a
342            shell script which should not be affected by whatever
343            customizations a user might have in place.
344            </para>
345          </listitem>
346
347          <listitem>
348            <para>-e &#60;form&#62; (or --eval &#60;form&#62;). An expression is
349            read (via READ-FROM-STRING from the string &#60;form&#62; and
350            evaluated. If &#60;form&#62; contains shell metacharacters, it may be
351            necessary to escape or quote them to prevent the shell from
352            interpreting them.</para>
353          </listitem>
354
355          <listitem>
356            <para>-l &#62;path&#62; (or --load &#60;path&#62;). Executes (load
357            &#34;&#60;path&#62;&#34;).</para>
358          </listitem>
359        </itemizedlist>
360
361        <para>The <literal>--load</literal> and
362        <literal>--eval</literal> options can each be provided
363        multiple times.  They're executed in the order specified on
364        the command line, after the init file (if there is one) is
365        loaded and before the toplevel read-eval-print loop is
366        entered.</para>
367      </sect2>
368    </sect1>
369
370    <sect1 id="Using-OpenMCL-with-GNU-Emacs-and-SLIME">
371      <title>Using OpenMCL with GNU Emacs and SLIME</title>
372      <para>A very common way to use OpenMCL is to run it within the
373      GNU Emacs editor, using a Lisp interface called SLIME ("Superior
374      Lisp Interaction Mode for Emacs"). SLIME is an Emacs package
375      designed to provide good support within Emacs for any of several
376      Common Lisp implementations; one of the supported
377      implementations is OpenMCL. This page describes how you can
378      download SLIME and set it up to work with your OpenMCL
379      installation.</para>
380      <para>Why use SLIME? With SLIME, you can do the following things from within
381an Emacs editing session:</para>
382      <itemizedlist>
383        <listitem><para>run and control Lisp</para></listitem>
384        <listitem><para>evaluate, compile, and load files or expressions</para></listitem>
385        <listitem><para>macroexpand expressions</para></listitem>
386        <listitem><para>fetch documentation and source code for Lisp symbols</para></listitem>
387        <listitem><para>autocomplete symbols and package names</para></listitem>
388        <listitem><para>cross-reference function calls</para></listitem>
389        <listitem><para>examine stack traces and debug errors</para></listitem>
390     
391      </itemizedlist>
392      <para>There is an excellent SLIME tutorial video available at
393Common-Lisp.net.  For
394complete information about SLIME, see the
395SLIME home page.</para>
396
397      <sect2 id="Assumptions-and-Requirements">
398        <title>Assumptions and Requirements</title>
399        <para>In order to simplify these instructions, we'll make
400        several assumptions about your system. Specifically, we
401        assume:</para>
402        <itemizedlist>
403          <listitem>
404            <para>You have a working installation of GNU Emacs. If you
405            don't have a working copy of GNU Emacs, see the web page on
406            obtaining Emacs.  If you prefer to use XEmacs instead of
407            GNU Emacs,these instructions should still work; SLIME
408            supports XEmacs Version21. Mac OS X includes an Emacs
409            installation.  If you want to look into different versions,
410            you can check out theEmacsWiki, whichmaintains a
411            page,EmacsForMacOS,that provides much more information
412            about using Emacs on the Mac.
413            <warning>
414              <para>A popular version of Emacs among Mac users is
415              Aquamacs. This application is a version of GNU Emacs
416              with a number of customizations meant to make it behave
417              more like a standard Maciontosh application, with
418              windows, a menubar, etc.  Aquamacs includes SLIME; if
419              you like Aquamacs then you can use SLIME right away,
420              without getting and installing it separately. You just
421              need to tell SLIME where to find your installation of
422              OpenMCL. (See FIXTHIS.)</para>
423            </warning>
424          </para>
425          </listitem>
426          <listitem>
427            <para>You have a working copy of OpenMCL, installed in
428            <literal>"~/openmcl/ccl"</literal>If you prefer to install
429            OpenMCL in some directory other
430            than<literal>"~/openmcl/ccl"</literal> then these
431            instructions still work, but you must remember to use your
432            path to your ccl directory instead of theone that we give
433            here.</para>
434          </listitem>
435          <listitem>
436            <para>You install emacs add-ons in the folder
437            <literal>"~/emacs/site/"</literal>If this directory
438            doesn't exist on your system, you can just create it.If
439            you prefer to install Emacs add-ons in some place other
440            than<literal>"~/emacs/site/"</literal> then you must
441            remember to use your path toEmacs add-ons in place of
442            ours.</para>
443          </listitem>
444       
445        </itemizedlist>
446      </sect2>
447
448      <sect2 id="Getting_Slime"><title>Getting SLIME</title>       
449
450        <para>You can get SLIME from the SLIME Home Page. Stable
451        releases and CVS snapshots are available as archive files, or
452        you can follow the instructions on the SLIME Home Page to
453        check out the latest version from their CVS repository.</para>
454
455        <para>It's worth noting that stable SLIME releases happen very
456        seldom, but the SLIME developers often make changes and
457        improvements that are available through CVS updates. If you
458        asked the SLIM developers, they would most likely recommend
459        that you get SLIME from their CVS repository and update it
460        frequently.</para>
461
462        <para>Whether you get it from CVS, or download and unpack one
463        of the available archives, you should end up with a folder
464        named "slime" that contains the SLIME distribution.</para>
465      </sect2>
466
467      <sect2><title>Installing SLIME</title> 
468
469        <para>Once you have the "slime" folder described in the
470        previous section,installation is a simple matter of copying
471        the folder to the proper place. You can drag it into the
472        "~/emacs/site/" folder, or you can use a terminal command to
473        copy it there. For example, assuming your working directory
474        contains the unpacked "slime" folder:</para> <para><literal>$
475        cp -R slime ~/emacs/site/</literal></para> <para>That's all it
476        takes.</para>
477
478      </sect2>
479
480      <sect2 id="Telling-Emacs-About-SLIME">
481        <title>Telling Emacs About SLIME</title>
482        <para> Once SLIME and OpenMCL are installed, you just need to
483        add a line to your "~/.emacs" file that tells SLIME where to
484        find the script that runs OpenMCL:</para>
485        <para><literal>(setq inferior-lisp-program "~/openmcl/ccl/scripts/openmcl64")</literal></para>
486        <para>or</para>
487        <para><literal>(setq inferior-lisp-program "~/openmcl/ccl/scripts/openmcl")</literal></para>
488        <warning>
489          <para>Aquamacs users should add this line to the file "~/Library/Preferences/Aquamacs Emacs/Preferences.el".</para>
490        </warning>
491      </sect2>
492
493      <sect2 id="Running-OpenMCL-with-SLIME">
494        <title>Running OpenMCL with SLIME</title>
495        <para>Once the preparations in the previous section are
496        complete, exit Emacs and restart it, to ensure that it reads
497        the changes you made in your ".emacs" file (alternatively, you
498        could tell Emacs to reload the ".emacs" file). If all went
499        well, you should now be ready to run OpenMCL using
500        SLIME.</para>
501        <para>To run OpenMCL, execute the command "M-x slime". SLIME
502        should start an OpenMCL session in a new buffer.  (If you are
503        unfamiliar with the Emacs notation "M-x command", see the GNU
504        Emacs FAQ; specifically, take a look at questions 1, 2, and
505        128.)</para>
506      </sect2>
507
508      <sect2 id="What-if-a-New-Version-of-OpenMCL-Breaks-SLIME-">
509        <title>What if a New Version of OpenMCL Breaks SLIME?</title>
510        <para>Sometimes you'll get a new version of OpenMCL, set up
511        Emacs to use it with SLIME, and SLIME will fail. Most likely
512        what has happened is that the new version of OpenMCL has a
513        change in the output files produced by the compiler (OpenMCL
514        developers will say "the fasl version has changed." fasl
515        stands for &ldquo;fast load&rdquo; aka compiled files). This
516        problem is easy to fix: just delete the existing SLIME fasl
517        files. The next time you launch Emacs and start SLIME, it will
518        automatically recompile the Lisp files, and that should fix
519        the problem.</para>
520        <para>SLIME's load process stores its fasl files in a hidden
521        folder inside your home folder. The path is</para>
522        <para><literal>~/.slime/fasl</literal></para>
523        <para>You can use a shell command to remove the fasl files, or
524        remove them using your system's file browser.</para>
525        <screen><emphasis role="bold">Note for Macintosh
526        Users:</emphasis> The leading "." character in the ".slime"
527        folder's name prevents the Finder from showing this folder to
528        you. If you use the "Go To Folder" menu item in the Finder's
529        "Go" menu, you can type in "~/.slime" and the Finder will show
530        it to you. You can then drag the "fasl" folder to the trash.
531        </screen>
532      </sect2>
533
534      <sect2 id="Known-Bugs">
535        <title>Known Bugs</title>
536        <para>SLIME has not been updated to account for recent changes
537        made in OpenMCL to support x86-64 processors. You may run into
538        bugs running on those platforms.</para>
539        <para>The SLIME backtrace sometimes shows incorrect information.</para>
540        <para><literal>return-from-frame</literal> and
541        <literal>apply-in-frame</literal> do not work reliably.  (If
542        they work at all, it's pure luck.)</para>
543        <para>Some versions of Emacs on the Macintosh may have trouble
544        finding the shell script that runs OpenMCL unless you specify
545        a full path to it. See the above section "Telling Emacs About
546        SLIME" to learn how to specify the path to the shell
547        script.</para>
548        <para>For more help with OpenMCL on Mac OS X, consult the
549        OpenMCL mailing lists.</para>
550      </sect2>
551    </sect1>
552
553    <sect1 id="Example-Programs">
554      <title>Example Programs</title>
555      <para>Beginning with release 0.9, a number (ok, a
556      <emphasis>small</emphasis> number, at least initially) of
557      example programs are distributed in the "ccl:examples;"
558      directory of the source distribution. See the
559      README-OPENMCL-EXAMPLES text file in that directory for
560      information about prerequisites and usage.</para>
561      <para>Some of the example programs are derived from C examples
562      in textbooks, etc.; in those cases, the original author and work
563      are cited in the source code.</para>
564      <para>It may be a stretch to imply that the examples have a
565      great deal of value as "intellectual property", but I might as
566      well say this: Unless the original author or contributor claims
567      other rights, you're free to incorporate any of this example
568      code or derivative thereof in any of you're own works without
569      restriction. In doing so, you agree that the code was provided
570      "as is", and that no other party is legally or otherwise
571      responsible for any consequences of your decision to use
572      it.</para>
573      <para>If you've developed OpenMCL examples that you'd like to
574      see added to the distribution, please send mail to let me
575      know. Any such contributions would be welcome and appreciated
576      (as would bug fixes and improvements to the existing
577      examples.)</para>
578    </sect1>
579  </chapter>
580
581  <chapter><title>Building OpenMCL from its Source Code</title>
582
583    <para>OpenMCL, like many other Lisp implementations, consists of a
584    kernel and a heap image.  The kernel is an ordinary C program, and
585    is built with a C compiler.  It provides very basic and
586    fundamental facilities, such as memory management, garbage
587    collection, and bootstrapping.  All the higher-level features are
588    written in Lisp, and compiled into the heap image.  Both parts are
589    needed to have a working Lisp implementation; neither the kernel
590    nor the heap image can stand on their own.</para>
591
592    <para>You may already know that, when you have a C compiler which
593    is written in C, to build the compiler, you need to already have a
594    C compiler available.  The heap image includes a Lisp compiler,
595    which is written in Lisp.  Therefore, you need a working Lisp
596    compiler in order to build the heap image!</para>
597   
598    <para>Where will you get a working Lisp compiler?  No worries; you
599    can use a precompiled copy of a (slightly older and compatible)
600    version of OpenMCL.  For help, read on.</para>
601
602    <para>In principle it would be possible to use another Lisp as the
603    host compiler, rather than an old OpenMCL; this would be a
604    challenging and experimental way to build, and is not described
605    here.</para>
606
607    <sect1 id="building-definitions"><title>>building definitions</title>
608      <para>The following terms are used in subsequent sections; it
609      may be helpful to refer to these definitions.</para>
610
611      <para><indexterm><primary>fasl files</primary></indexterm> are
612      the object files produced by<literal>compile-file</literal>.
613      fasl files store the machine codeassociated with function
614      definitions and the external representationof other lisp objects
615      in a compact, machine-readable form. fasl is short for
616      &ldquo;<literal>FAS</literal>t
617      <literal>L</literal>oading&rdquo;.OpenMCL uses different
618      pathname types (extensions) to name faslfiles on different
619      platforms; see <xref
620      linkend="Platform-specific-filename-conventions"/> </para>
621
622      <para>The <indexterm><primary>lisp kernel</primary></indexterm> is
623      a C program (with a fair amount ofplatform-specific assembly
624      language code as well.)  Its basic job isto map a lisp heap
625      image into memory, transfer control to somecompiled lisp code
626      that the image contains, handle any exceptionsthat occur during
627      the execution of that lisp code, and provide variousother forms
628      of runtime support for that code.OpenMCL uses different
629      filenames to name the lisp kernel fileson different platforms;
630      see FIXTHIS.</para>
631
632      <para>A <indexterm><primary>heap image</primary></indexterm> is
633      a file that can be quickly mapped into aprocess's address space.
634      Conceptually, it's not too different from anexecutable file or
635      shared library in the OS's native format (ELF orMach-O/dyld
636      format); for historical reasons, OpenMCL's own heap images are in
637      their own (fairly simple) format.The term <literal>full heap
638      image</literal> refers to a heap image file thatcontains all of
639      the code and data that comprise OpenMCL.OpenMCL uses different
640      filenames to name the standard full heapimage files on different
641      platforms; see FIXTHIS .</para>
642
643      <para>A <indexterm><primary>bootstrapping
644      image</primary></indexterm> is a minimal heap image used in
645      the process of building OpenMCL itself.  The bootstrapping image
646      containsjust enough code to load the rest of OpenMCL from fasl
647      files.  It mayhelp to think of the bootstrapping image as the
648      egg and the full heapimage as the chicken...OpenMCL uses
649      different filenames to name the standardbootstrapping image
650      files on different platforms; see FIXTHIS .</para>
651
652      <para>Each supported platform (and possibly a few
653      as-yet-unsupported ones) has a uniquely named subdirectory of
654      <literal>ccl/lisp-kernel/</literal>; each such
655      <indexterm><primary>kernel build directory</primary></indexterm>
656      contains a Makefile and may contain some auxiliary files (linker
657      scripts, etc.) that are used to build the lispkernel on a
658      particular platform.The platform-specific name of the kernel
659      build directory is described in FIXTHIS.</para>
660
661      <sect2 id="Platform-specific-filename-conventions">
662       <title>Platform-specific filename conventions</title>
663       <table>
664         <title>Platform-specific filename conventions</title>
665         <tgroup cols="6">
666           <thead>
667            <row>
668                <entry>Platform</entry>
669                <entry>kernel</entry>
670                <entry>full-image</entry>
671                <entry>boot-image</entry>
672                <entry>fasl extension</entry>
673                <entry>kernel-build directory</entry>
674            </row>
675           </thead>
676           <tbody>
677             <row>
678               <entry>DarwinPPC32</entry>
679                <entry>dppccl</entry>
680                <entry>dppccl.image</entry>
681                <entry>ppc-boot.image</entry>
682                <entry>.dfsl</entry>
683                <entry>darwinppc</entry>
684             </row>
685             <row>
686               <entry>LinuxPPC32</entry>
687                <entry>ppccl</entry>
688                <entry>PPCCL</entry>
689                <entry>ppc-boot</entry>
690                <entry>.pfsl</entry>
691                <entry>linuxppc</entry>
692             </row>
693             <row>
694               <entry>DarwinPPC64</entry>
695               <entry>dppccl64</entry>
696               <entry>dppccl64.image</entry>
697               <entry>ppc-boot64.image</entry>
698               <entry>.d64fsl</entry>
699               <entry>darwinppc64</entry>
700             </row>
701              <row>
702                <entry>LinuxPPC64</entry>
703                <entry>ppccl64</entry>
704                <entry>PPCCL64</entry>
705                <entry>ppc-boot64</entry>
706                <entry>.p64fsl</entry>
707                <entry>linuxppc64</entry>
708              </row>
709              <row>
710                <entry>LinuxX8664</entry>
711                <entry>lx86cl64</entry>
712                <entry>LX86CL64</entry>
713                <entry>x86-boot64</entry>
714                <entry>.lx64fsl</entry>
715                <entry>linuxx8664</entry>
716              </row>
717              <row>
718                <entry>DarwinX8664</entry>
719                <entry>dx86cl64</entry>
720                <entry>dx86cl64.image</entry>
721                <entry>x86-boot64.image</entry>
722                <entry>.dx64fsl</entry>
723                <entry>darwinx8664</entry>
724              </row>
725              <row>
726                <entry>FreeBSDX8664</entry>
727                <entry>fx86cl64</entry>
728                <entry>FX86CL64</entry>
729                <entry>fx86-boot64</entry>
730                <entry>.fx64fsl</entry>
731                <entry>freebsdx8664</entry>
732              </row>
733           </tbody>
734         </tgroup>
735       </table>
736      </sect2>
737    </sect1>
738
739    <sect1 id="Setting-Up-to-Build">
740      <title>Setting Up to Build</title>
741      <para>There are currently three versions of OpenMCL that you
742      might want to use (and therefore might want to build from
743      source):</para>
744      <itemizedlist>
745        <listitem><para>Version 1.0 - the more stable version</para></listitem>
746        <listitem><para>Version 1.1 - the more recent version, which
747        runs on more platforms (including x86-64 platforms) and
748        supports Unicode</para></listitem>
749        <listitem><para>Version 1.2 - supports (at least) all of the
750        features and platforms as 1.1, but is distributed and updated
751        differently</para></listitem>
752      </itemizedlist>
753      <para>All versions are available for download from the OpenMCL
754      website in the form of archives that contain everything you need
755      to work with OpenMCL, including the complete sources, a full
756      heap image, and the foreign-function interface database.</para>
757      <para>Version 1.0 archives are named
758      <literal>openmcl-</literal><replaceable>platform</replaceable><literal>-all-1.0.tar.gz</literal>,
759      where <replaceable>platform</replaceable> is either
760      <literal>darwinppc</literal>, <literal>darwinppc64</literal>, or
761      <literal>linuxppc</literal>.  Because version 1.0 is no longer
762      undergoing active development, you won't ever need to update
763      these sources.</para>
764      <para>Version 1.1 archives are named
765      <literal>openmcl-</literal><replaceable>platform</replaceable><literal>-snapshot-</literal><replaceable>yymmdd</replaceable><literal>.tar.gz</literal>,
766      where <replaceable>platform</replaceable> is either
767      <literal>darwinppc</literal>, <literal>darwinx8664</literal>,
768      <literal>linuxppc</literal>, <literal>linuxx8664</literal>, or
769      <literal>freebsdx8664</literal>, and where
770      <replaceable>yymmdd</replaceable> is the year, month, and day
771      the snapshot was released.</para>
772      <para>Because version 1.1 is undergoing active development,
773      there may be times when you want to get sources that are more
774      recent than the most recent snapshot and use them to build
775      yourself a new bleeding-edge OpenMCL.  In that case, you should
776      download and install the latest snapshot, and then update your
777      sources via CVS.  At that point you can rebuild and you'll have
778      the latest and greatest OpenMCL.  The snapshot has CVS
779      working-copy information in it, so all you need to do to update
780      is</para>
781      <programlisting>
782$ cd ccl
783$ cvs login             # password is "cvs"
784                        # this step only needs to be done once,
785                        # that'll store the trivially encrypted
786                        # password in ~/.cvspas
787$ cvs update
788      </programlisting>
789      <para>Unless you tell it to, cvs won't delete ("prune") empty
790      directories or create new ones when the repository changes.
791      It's generally a good habit to use</para>
792      <programlisting>
793$ cvs update -d -P      # create dirs as needed, prune empty ones
794      </programlisting>
795      <para>Version 1.2 archives follow naming conventions that are
796      similar to those used by 1.0 (though more platforms are supported.)
797      However, rather than containing CVS working-copy information, the
798      1.2 (and, presumably, later) archives contain metainformation used
799      by the Subversion (svn) source-code control system.</para>
800      <para>Subversion client programs are pre-installed on OSX 10.5 and
801      later and are typically either pre-installed or readily available
802      on Linux and FreeBSD platforms.  The <ulink url="http://subversion.tigris.org">Subversion web page</ulink> contains links to subversion client programs
803      for many platforms; users of OSX versions 10.4 and earlier can also
804      install Subversion clients via Fink or MacPorts.</para>
805     
806    </sect1>
807
808    <sect1 id="Building-Everything">
809      <title>Building Everything</title>
810      <para>Given that you now have everything you need, do the
811      following in a running OpenMCL to bring your Lisp system
812      completely up to date.</para>
813      <programlisting>
814? (ccl:rebuild-ccl :full t)
815      </programlisting>
816      <para>That call to the function <literal>rebuild-ccl</literal>
817      will perform the following steps:</para>
818      <itemizedlist>
819        <listitem>
820          <para>Deletes all fasl files and other object files in the
821          <literal>ccl</literal>directory tree</para>
822        </listitem>
823        <listitem>
824          <para>Runs an external process which does a
825          <literal>make</literal> in the currentplatform's kernel
826          build directory to create a new kernel</para>
827        </listitem>
828        <listitem>
829          <para>Does <literal>(compile-ccl t)</literal> in the running
830          lisp, to produce aset of fasl files from the &ldquo;higher
831          level&rdquo; lisp sources.</para>
832        </listitem>
833        <listitem>
834          <para>Does <literal>(xload-level-0 :force)</literal> in the
835          running lisp, to compile thelisp sources in the
836          &ldquo;ccl:level-0;&rdquo; directory into fasl files and
837          then createa bootstrapping image from those fasl
838          files.</para>
839        </listitem>
840        <listitem>
841          <para>Runs another external process, which causes the newly
842          compiled lispkernel to load the new bootstrapping image.
843          The bootsrtrapping image then loadsthe &ldquo;higher
844          level&rdquo; fasl files and a new copy of the platform's
845          full heap imageis then saved.</para>
846        </listitem>
847      </itemizedlist>
848      <para>If all goes well, it'll all happen without user
849      intervention and with some simple progress messages.  If
850      anything goes wrong during execution of either of the external
851      processes, the process output is displayed as part of a lisp
852      error message.</para>
853      <para><literal>rebuild-ccl</literal> is essentially just a short
854      cut for running all the individual steps involved in rebuilding
855      the system.  You can also execute these steps individually, as
856      described below.</para>
857    </sect1>
858
859    <sect1 id="Building-the-kernel">
860      <title>Building the kernel</title>
861      <para>The Lisp kernel is the executable which you run to use
862      Lisp.  It doesn't actually contain the entire Lisp
863      implementation; rather, it loads a heap image which contains the
864      specifics - the "library", as it might be called if this was a C
865      program.  The kernel also provides runtime support to the heap
866      image, such as garbage collection, memory allocation, exception
867      handling, and the OS interface.</para>
868
869      <para>The Lisp kernel file has different names on different
870      platforms. See FIXTHIS . On all platforms the lisp kernel sources reside
871      in <literal>ccl/lisp-kernel</literal>.</para>
872
873      <para>This section gives directions on how to rebuild the Lisp
874      kernel from its source code.  Most OpenMCL users will rarely
875      have to do this.  You probably will only need to do it if you are
876      attempting to port OpenMCL to a new architecture or extend or enhance
877      its kernel in some way.  As mentioned above, this step happens
878      automatically when you do
879      <programlisting>
880? (rebuild-ccl :full t)
881      </programlisting>
882      </para>
883
884
885      <sect2 id="Kernel-build-prerequisites">
886        <title>Kernel build prerequisites</title>
887        <para>The OpenMCL kernel can be bult with the following widely
888        available tools:</para>
889        <itemizedlist>
890          <listitem><para>cc or gcc- the GNU C compiler</para></listitem>
891          <listitem><para>ld - the GNU linker</para></listitem>
892          <listitem><para>m4 or gm4- the GNU m4 macro processor</para></listitem>
893          <listitem><para>as - the GNU assembler (version 2.10.1 or later)</para></listitem>
894          <listitem><para>make - either GNU make or, on FreeBSD, the default BSD make program</para></listitem>
895        </itemizedlist>
896        <para> In general, the more recent the versions of those
897        tools, the better; some versions of gcc 3.x on Linux have
898        difficulty compiling some of the kernel source code correctly
899        (so gcc 4.0 should be used, if possible.)  On OSX, the
900        versions of the tools distributed with XCode should work fine;
901        on Linux, the versions of the tools installed with the OS (or
902        available through its package management system) should work
903        fine if they're "recent enough".  On FreeBSD, the installed
904        version of the <literal>m4</literal> program doesn't support
905        some features that the kernel build process depends on; the
906        GNU version of the m4 macroprocessor (called
907        <literal>gm4</literal> on FreeBSD) should be installed
908
909        </para>
910      </sect2>
911      <sect2 id="kernel-build-command">
912        <title>Using "make" to build the lisp kernel</title>
913        <para>With those tools in place, do:
914        <programlisting>
915shell> cd ccl/lisp-kernel/<replaceable>PLATFORM</replaceable>
916shell> make
917        </programlisting>
918        </para>
919        <para>That'll assemble several assembly language source files,
920        compile several C source files, and link
921        ../../<replaceable>the kernel</replaceable>.
922        </para>
923      </sect2>
924    </sect1>
925    <sect1 id="Building-the-heap-image">
926      <title>Building the heap image</title>
927      <para>The initial heap image is loaded by the Lisp kernel, and
928      provides most all of the language implementation The heap image
929      captures the entire state of a running Lisp (except for external
930      resources, such as open files and TCP sockets).  After it is
931      loaded, the contents of the new Lisp process's memory are
932      exactly the same as those of the old Lisp process when the image
933      was created.</para>
934      <para>The heap image is how we get around the fact that we can't
935      run Lisp code until we have a working Lisp implementation, and
936      we can't make our Lisp implementation work until we can run Lisp
937      code.  Since the heap image already contains a fully-working
938      implementation, all we need to do is load it into memory and
939      start using it.</para>
940      <para>If you're building a new version of OpenMCL, you need to
941      build a new heap image.</para>
942      <para>(You might also wish to build a heap image if you have a
943      large program which it is very complicated or time-consuming to
944      load, so that you will be able to load it once, save an image,
945      and thenceforth never have to load it again. At any time, a heap
946      image capturing the entire memory state of a running Lisp can be
947      created by calling the function
948      <literal>ccl:save-application</literal>.)</para>
949
950      <sect2 id="Development-cycle">
951        <title>Development cycle</title>
952        <para>Creating a new OpenMCL full heap image consists of the
953        following steps:</para>
954        <orderedlist>
955          <listitem><para>Using your existing OpenMCL, create a
956          bootstrapping image</para></listitem>
957          <listitem><para>Using your existing OpenMCL, recompile your
958          updated OpenMCL sources</para></listitem>
959          <listitem><para>Invoke OpenMCL with the bootstrapping image
960          you just created (rather than with the existing full heap
961          image).</para></listitem>
962        </orderedlist>
963        <para>When you invoke OpenMCL with the bootstrapping image, it
964        will start up, load al of the OpenMCL fasl files, and save out
965        a new full heap image.  Voila.  You've created a new heap
966        image.</para>
967        <para>A few points worth noting:</para>
968        <itemizedlist>
969          <listitem>
970            <para>There's a circular dependency between the full heap
971            image and thebootstrapping image, in that each is used to
972            build the other.</para>
973          </listitem>
974          <listitem>
975            <para>There are some minor implementation
976            differences, but the environment in effect after the
977            bootstrapping image has loaded its fasl files is essentially
978            equivalent to the environment provided by the full heap
979            image; the latter loads a lot faster and is easier to
980            distribute, of course.</para>
981          </listitem>
982          <listitem>
983            <para>If the full heap image doesn't work (because
984            of an OScompatibilty problem or other bug), it's very likely
985            that thebootstrapping image will suffer the same
986            problems.</para>
987          </listitem>
988        </itemizedlist>
989        <para>Given a bootstrapping image and a set of up-to-date fasl
990        files, the development cycle usually involves editing lisp
991        sources (or updating those sources via cvs update),
992        recompiling modified files, and using the bootstrapping image
993        to produce a new heap image.</para>
994      </sect2>
995
996      <sect2 id="Generating-a-bootstrapping-image">
997        <title>Generating a bootstrapping image</title>
998        <para>The bootstrapping image isn't provided in OpenMCL
999        distributions. It can be built from the source code provided
1000        in distributions (using a lisp image and kernel provided in
1001        those distributions) using the procedure described
1002        below.</para>
1003
1004        <para>The bootstrapping image is built by invoking a special
1005        utility inside a running OpenMCL heap image to load files
1006        contained in the <literal>ccl/level-0</literal> directory. The
1007        bootstrapping image loads several dozen fasl files.  After
1008        it's done so, it saves a heap image via
1009        <literal>save-application</literal>. This process is called
1010        "cross-dumping".</para>
1011
1012        <para>Given a source distribution, a lisp kernel, and aheap
1013        image, one can produce a bootstapping image by first invoking
1014        OpenMCL from the shell:</para>
1015        <programlisting>
1016shell&gt; openmcl
1017Welcome to OpenMCL .... !
1018?
1019        </programlisting>
1020        <para>then calling <literal>ccl:xload-level-0</literal> at the
1021        lisp prompt</para>
1022        <programlisting>
1023? (ccl:xload-level-0)
1024        </programlisting>
1025        <para>This will compile the lisp sources in the ccl/level-0
1026        directory if they're newer than the corresponding fasl files
1027        and will then load the resulting fasl files into a simulated
1028        lisp heap contained inside data structures inside the running
1029        lisp. That simulated heap image is then written to
1030        disk.</para>
1031        <para><literal>xload-level-0</literal> should be called
1032        whenever your existing boot image is out-of-date with respect
1033        to the source files in <literal>ccl:level-0;</literal>
1034        :</para>
1035        <programlisting>
1036? (ccl:xload-level-0 :force)
1037</programlisting>
1038        <para>will force recompilation of the level-0 sources.</para>
1039      </sect2>
1040
1041      <sect2 id="Generating-fasl-files">
1042        <title>Generating fasl files</title>
1043        <para> Calling:</para>
1044        <programlisting>
1045? (ccl:compile-ccl)
1046        </programlisting>
1047        <para>at the lisp prompt will compile any fasl files that are
1048        out-of-date with respect to the corresponding lisp sources;
1049        <literal>(ccl:compile-ccl t)</literal> will force
1050        recompilation. <literal>ccl:compile-ccl</literal> will reload
1051        newly-compiled versions of some files;
1052        <literal>ccl:xcompile-ccl</literal> is analogous, but skips
1053        this reloading step.</para>
1054        <para>Unless there are bootstrapping considerations involved,
1055        it usually doesn't matter whether these files reloaded after
1056        they're recompiled.</para>
1057        <para>Calling <literal>compile-ccl</literal> or
1058        <literal>xcompile-ccl</literal> in an environment where fasl
1059        files don't yet exist may produce warnings to that effect
1060        whenever files are <literal>require</literal>d during
1061        compilation; those warnings can be safely ignored. Depending
1062        on the maturity of the OpenMCL release, calling
1063        <literal>compile-ccl</literal> or
1064        <literal>xcompile-ccl</literal> may also produce several
1065        warnings about undefined functions, etc. They should be
1066        cleaned up at some point.</para>
1067      </sect2>
1068
1069      <sect2 id="Building-a-full-image-from-a-bootstrapping-image">
1070        <title>Building a full image from a bootstrapping image</title>
1071        <para>To build a full image from a bootstrapping image, just
1072        invoke the kernel with the bootstrapping image is an
1073        argument</para>
1074        <programlisting>
1075$ cd ccl                        # wherever your ccl directory is
1076$ ./<replaceable><kernel&gt; <boot_image&gt;</replaceable>
1077        </programlisting>
1078        <para>Where <replaceable><kernel&gt;</replaceable> and
1079        <replaceable><boot_image&gt;</replaceable> are the names of
1080        the kernel and boot image appropriate to the platform you are
1081        running on.  See FIXTHIS</para>
1082        <para>That should load a few dozen fasl files (printing a
1083        message as each file is loaded.) If all of these files
1084        successfully load, the lisp will print a prompt. You should be
1085        able to do essentially everything in that environment that you
1086        can in the environment provided by a "real" heap image. If
1087        you're confident that things loaded OK, you can save that
1088        image.</para>
1089        <programlisting>
1090? (ccl:save-application "<replaceable>image_name</replaceable>") ; Overwiting the existing heap image
1091        </programlisting>
1092        <para>Where <replaceable>image_name</replaceable> is the name
1093        of the full heap image for your platform. See FIXTHIS.</para>
1094        <para>If things go wrong in the early stages of the loading
1095        sequence, errors are often difficult to debug; until a fair
1096        amount of code (CLOS, the CL condition system, streams, the
1097        reader, the read-eval-print loop) is loaded, it's generally
1098        not possible for the lisp to report an error.  Errors that
1099        occur during these early stages ("the cold load") sometimes
1100        cause the lisp kernel debugger (see ) to be invoked; it's
1101        primitive, but can sometimes help one to get oriented.</para>
1102      </sect2>
1103    </sect1>
1104  </chapter>
1105
1106  <chapter id="Questions-and-Answers">
1107
1108    <title>Questions and Answers</title>
1109
1110    <sect1 id="How-can-I-do-nonblocking--aka--unbuffered--and--raw---IO-">
1111      <title>How can I do nonblocking (aka "unbuffered" and "raw") IO?</title>
1112      <para>There's some code for manipulating TTY modes in
1113      "ccl:library;pty.lisp".</para>
1114      <programlisting>
1115? (require "PTY")
1116
1117? (ccl::disable-tty-local-modes 0 #$ICANON)
1118T
1119      </programlisting>
1120      <para>will turn off "input canonicalization" on file descriptor
1121      0, which is at least part of what you need to do here.  This
1122      disables the #$ICANON mode, which tells the OS not to do any
1123      line-buffering or line-editing.  Of course, this only has any
1124      effect in situations where the OS ever does that, which means
1125      when stdin is a TTY or PTY.</para>
1126      <para>If the #$ICANON mode is disabled, you can do things like:</para>
1127      <programlisting>
1128? (progn (read-char) (read-char))
1129a
1130#\a
1131      </programlisting>
1132      <para>(where the first READ-CHAR consumes the newline, which
1133      isn't really necessary to make the reader happy anymore.)  So,
1134      you can do:</para>
1135      <programlisting>
1136? (read-char)
1137#\Space
1138</programlisting>
1139      <para>(where there's a space after the close-paren) without
1140      having to type a newline.</para>
1141    </sect1>
1142
1143    <sect1 id="I-m-using-the-graphics-demos--Why-doesn-t-the-menubar-change-">
1144      <title>I'm using the graphics demos. Why doesn't the menubar
1145      change?</title>
1146      <para>When you interact with text-only OpenMCL, you're either
1147      in Terminal or in Emacs, running OpenMCL as a subprocess.  When
1148      you load Cocoa or the graphical environment, the subprocess does
1149      some tricky things that turn it into a full-fledged Application,
1150      as far as the OS is concerned.</para>
1151      <para>So, it gets its own icon in the dock, and its own menubar,
1152      and so on.  It can be confusing, because standard input and
1153      output will still be connected to Terminal or Emacs, so you can
1154      still type commands to OpenMCL from there.  To see the menubar
1155      you loaded, or the windows you opened, just click on the OpenMCL
1156      icon in the dock.</para>
1157    </sect1>
1158
1159    <sect1 id="I-m-using-Slime-and-Cocoa--Why-doesn-t--standard-output--seem-to-work-">
1160      <title>I'm using Slime and Cocoa. Why doesn't *standard-output*
1161      seem to work? </title>
1162      <para>This comes up if you're using the Slime interface
1163      to run OpenMCL under Emacs, and you are doing Cocoa programming
1164      which involves printing to *standard-output*.  It seems as
1165      though the output goes nowhere; no error is reported, but it
1166      doesn't appear in the *slime-repl* buffer.</para>
1167
1168      <para>For the most part, this is only relevant when you are
1169      trying to insert debug code into your event handlers.  The SLIME
1170      listener runs in a thread where the standard stream varaiables
1171      (like <literal>*STANDARD-OUTPUT* and</literal> and
1172      <literal>*TERMINAL-IO*</literal> are bound to the stream used to
1173      communicate with Emacs; the Cocoa event thread has its own
1174      bindings of these standard stream variables, and output to these
1175      streams goes to the *inferior-lisp* buffer instead.  Look for it
1176      there.</para>
1177    </sect1>
1178  </chapter>
1179
1180  <chapter id="Programming-with-Threads">
1181    <title>Programming with Threads</title>
1182
1183    <sect1 id="Threads-overview">
1184      <title>Threads Overview</title>
1185
1186      <para>OpenMCL provides facilities which enable multiple threads
1187      of execution (<emphasis>threads</emphasis>, sometimes called
1188      <emphasis>lightweight processes</emphasis> or just
1189      <emphasis>processes</emphasis>, though the latter term shouldn't
1190      be confused with the OS's notion of a process) within a lisp
1191      session. This document describes those facilities and issues
1192      related to multitheaded programming in OpenMCL.</para>
1193
1194      <para>Wherever possible, I'll try to use the term "thread" to
1195      denote a lisp thread, even though many of the functions in the
1196      API have the word "process" in their name. A
1197      <emphasis>lisp-process</emphasis> is a lisp object (of type
1198      CCL:PROCESS) which is used to control and communicate with an
1199      underlying <emphasis>native thread</emphasis>. Sometimes, the
1200      distinction between these two (quite different) objects can be
1201      blurred; other times, it's important to maintain.</para>
1202      <para>Lisp threads share the same address space, but maintain
1203      their own execution context (stacks and registers) and their own
1204      dynamic binding context.</para>
1205     
1206      <para>Traditionally, OpenMCL's threads have been
1207      <emphasis>cooperatively scheduled</emphasis>: through a
1208      combination of compiler and runtime suppport, the currently
1209      executing lisp thread arranged to be interrrupted at certain
1210      discrete points in its execution (typically on entry to a
1211      function and at the beginning of any looping construct). This
1212      interrupt occurred several dozen times per second; in response,
1213      a handler function might observe that the current thread had
1214      used up its time slice and another function (<emphasis>the lisp
1215      scheduler</emphasis>) would be called to find some other thread
1216      that was in a runnable state, suspend execution of the current
1217      thread, and resume execution of the newly executed thread.  The
1218      process of switching contexts between the outgoing and incoming
1219      threads happened in some mixture of Lisp and assembly language
1220      code; as far as the OS was concerned, there was one native
1221      thread running in the Lisp image and its stack pointer and other
1222      registers just happened to change from time to time.</para>
1223      <para>Under OpenMCL's cooperative scheduling model, it was
1224      possible (via the use of the CCL:WITHOUT-INTERRUPTS construct)
1225      to defer handling of the periodic interrupt that invoked the
1226      lisp scheduler; it was not uncommon to use WITHOUT-INTERRUPTS to
1227      gain safe, exclusive access to global data structures. In some
1228      code (including much of OpenMCL itself) this idiom was very
1229      common: it was (justifiably) believed to be an efficient way of
1230      inhibiting the execution of other threads for a short period of
1231      time.</para>
1232
1233      <para>The timer interrupt that drove the cooperative scheduler
1234      was only able to (pseudo-)preempt lisp code: if any thread
1235      called a blocking OS I/O function, no other thread could be
1236      scheduled until that thread resumed execution of lisp code. Lisp
1237      library functions were generally attuned to this constraint, and
1238      did a complicated mixture of polling and "timed blocking" in an
1239      attempt to work around it. Needless to say, this code is
1240      complicated and less efficient than it might be; it meant that
1241      the lisp was a little busier than it should have been when it
1242      was "doing nothing" (waiting for I/O to be possible.)</para>
1243
1244      <para>For a variety of reasons - better utilization of CPU
1245      resources on single and multiprocessor systems and better
1246      integration with the OS in general - threads in OpenMCL 0.14 and
1247      later are <emphasis>preemptively scheduled. </emphasis>In this
1248      model, lisp threads are native threads and all scheduling
1249      decisions involving them are made by the OS kernel. (Those
1250      decisions might involve scheduling multiple lisp threads
1251      simultaneously on multiple processors on SMP systems.) This
1252      change has a number of subtle effects:</para>
1253
1254      <itemizedlist>
1255        <listitem>
1256          <para>it is possible for two (or more) lisp threads to be
1257          executingsimultaneously, possibly trying to access and/or
1258          modify the same datastructures. Such access really should
1259          have been coordinated throughthe use of synchronization
1260          objects regardless of the scheduling modelin effect;
1261          preemptively scheduled threads increase the chance ofthings
1262          going wrong at the wrong time and do not offer
1263          lightweightalternatives to the use of those synchronization
1264          objects.</para>
1265        </listitem>
1266        <listitem>
1267          <para>even on a single-processor system, a context switch
1268          can happenon any instruction boundary. Since (in general)
1269          other threads mightallocate memory, this means that a GC can
1270          effectively take place atany instruction boundary. That's
1271          mostly an issue for the compilerand runtime system to be
1272          aware of, but it means that certain practices(such as trying
1273          to pass the address of a lisp object to foreign code)that
1274          were always discouraged are now discouraged
1275          ... vehemently.</para>
1276        </listitem>
1277        <listitem>
1278          <para>there is no simple and efficient way to "inhibit the
1279          scheduler"or otherwise gain exclusive access to the entire
1280          CPU.</para>
1281        </listitem>
1282        <listitem>
1283          <para>There are a variety of simple and efficient ways
1284          tosynchronize access to particular data
1285          structures.</para>
1286        </listitem>
1287      </itemizedlist>
1288      <para>As a broad generalization: code that's been aggressively
1289      tuned to the constraints of the cooperative scheduler may need
1290      to be redesigned to work well with the preemptive scheduler (and
1291      code written to run under OpenMCL's interface to the native
1292      scheduler may be less portable to other CL implementations, many
1293      of which offer a cooperative scheduler and an API similar to
1294      OpenMCL (< 0.14) 's.) At the same time, there's a large
1295      overlap in functionality in the two scheduling models, and it'll
1296      hopefully be possible to write interesting and useful MP code
1297      that's largely independent of the underlying scheduling
1298      details.</para>
1299      <para>The keyword :OPENMCL-NATIVE-THREADS is on *FEATURES* in
1300      0.14 and later and can be used for conditionalization where
1301      required.</para>
1302    </sect1>
1303
1304    <sect1 id="Intentionally--Missing-Functionality">
1305      <title>(Intentionally) Missing Functionality</title>
1306      <para>Much of the functionality described above is similar to
1307      that provided by OpenMCL's cooperative scheduler, some other
1308      parts of which make no sense in a native threads
1309      implementation.</para>
1310      <itemizedlist>
1311        <listitem>
1312          <para>PROCESS-RUN-REASONS and PROCESS-ARREST-REASONS were
1313          SETFable process attributes; each was just a list of
1314          arbitrary tokens. A thread was eligible for scheduling
1315          (roughly equivalent to being "enabled") if its arrest-reasons
1316          list was empty and its run-reasons list was not. I don't
1317          think that it's appropriate to encourage a programming style
1318          in which otherwise runnable threads are enabled and disabled
1319          on a regular basis (it's preferable for threads to wait for
1320          some sort of synchronization event to occur if they can't
1321          occupy their time productively.)</para>
1322        </listitem>
1323        <listitem>
1324          <para>There were a number of primitives for maintaining
1325          process queues;that's now the OS's job.</para>
1326        </listitem>
1327        <listitem>
1328          <para>Cooperative threads were based on coroutining
1329          primitivesassociated with objects of type
1330          STACK-GROUP. STACK-GROUPs no longerexist.</para>
1331        </listitem>
1332      </itemizedlist>
1333    </sect1>
1334
1335    <sect1 id="Implementation-Decisions-and-Open-Questions">
1336      <title>Implementation Decisions and Open Questions</title>
1337      <para> As of August 2003:</para>
1338      <itemizedlist>
1339        <listitem>
1340          <para>It's not clear that exposing
1341          PROCESS-SUSPEND/PROCESS-RESUME is a good idea: it's not clear
1342          that they offer ways to win, and it's clear that they offer
1343          ways to lose.</para>
1344        </listitem>
1345        <listitem>
1346          <para>It has traditionally been possible to reset and enable
1347          a process that's "exhausted" . (As used here, the
1348          term"exhausted" means that the process's initial function
1349          hasrun and returned and the underlying native thread has
1350          beendeallocated.) One of the principle uses of PROCESS-RESET
1351          is to "recycle" threads; enabling an exhausted process
1352          involves creating a new native thread (and stacks and
1353          synchronization objects and ...),and this is the sort of
1354          overhead that such a recycling scheme is seeking to avoid. It
1355          might be worth trying to tighten things up and declare that
1356          it's an error to apply PROCESS-ENABLE to an exhausted thread
1357          (and to make PROCESS-ENABLE detect this error.)</para>
1358        </listitem>
1359        <listitem>
1360          <para>When native threads that aren't created by OpenMCL
1361          first call into lisp, a "foreign process" is created, and
1362          that process is given its own set of initial bindings and set
1363          up to look mostly like a process that had been created by
1364          MAKE-PROCESS. The life cycle of a foreign process is
1365          certainly different from that of a lisp-created one: it
1366          doesn't make sense to reset/preset/enable a foreign process,
1367          and attempts to perform these operations should be
1368          detectedand treated as errors.</para>
1369        </listitem>
1370      </itemizedlist>
1371    </sect1>
1372
1373    <sect1 id="Porting-Code-from-the-Old-Thread-Model">
1374      <title>Porting Code from the Old Thread Model</title>
1375      <para>Older versions of OpenMCL used what are often called
1376      "user-mode threads", a less versatile threading model which does
1377      not require specific support from the operating system.  This
1378      section discusses how to port code which was written for that
1379      mode.</para>
1380      <para>It's hard to give step-by-step instructions; there are certainly
1381      a few things that one should look at carefully:</para>
1382      <itemizedlist>
1383        <listitem>
1384          <para>It's wise to be suspicious of most uses
1385          of WITHOUT-INTERRUPTS; there may be exceptions, but
1386          WITHOUT-INTERRUPTS is often used as shorthand for
1387          WITH-APPROPRIATE-LOCKING. Determining what type of locking
1388          is appropriate and writing the code to implement it is
1389          likely to be straightforward and simple most of the
1390          time.</para>
1391        </listitem>
1392        <listitem>
1393          <para>I've only seen one case where a process's "run reasons"
1394          were used to communicate information as well as tocontrol
1395          execution; I don't think that this is a common idiom, but may
1396          be mistaken about that.
1397          </para>
1398        </listitem>
1399        <listitem>
1400          <para>It's certainly possible that programs written
1401          for cooperatively scheduled lisps that have run reliably for
1402          a long timehave done so by accident: resource-contention
1403          issues tend to be timing-sensitive, and decoupling thread
1404          scheduling from lisp program execution affects timing. I know
1405          that there is or was code in both OpenMCL and commercial MCL
1406          that was written under the explicit assumption that certain
1407          sequences of open-coded operations were uninterruptable; it's
1408          certainly possible that the same assumptions have been made
1409          (explicitly or otherwise) by application developers.</para>
1410        </listitem>
1411      </itemizedlist>
1412    </sect1>
1413
1414    <sect1 id="Background-Terminal-Input">
1415      <title>Background Terminal Input</title>
1416
1417      <sect2 id="backgrount-ti-overview">
1418        <title>Overview</title>
1419        <para>
1420          Unless and until OpenMCL provides alternatives (via window
1421          streams, telnet streams, or some other mechanism) all lisp
1422          processes share a common *TERMINAL-IO* stream (and therefore
1423          share *DEBUG-IO*, *QUERY-IO*, and other standard and
1424          internal interactive streams.)</para>
1425          <para>It's anticipated that most lisp processes other than
1426          the "Initial" process run mostly in the background. If a
1427          background process writes to the output side of
1428          *TERMINAL-IO*, that may be a little messy and a little
1429          confusing to the user, but it shouldn't really be
1430          catastrophic. All I/O to OpenMCL's buffered streams goes
1431          thru a locking mechanism that prevents the worst kinds of
1432          resource-contention problems.</para>
1433          <para>Although the problems associated with terminal output
1434          from multiple processes may be mostly cosmetic, the question
1435          of which process receives input from the terminal is likely
1436          to be a great deal more important. The stream locking
1437          mechanisms can make a confusing situation even worse:
1438          competing processes may "steal" terminal input from each
1439          other unless locks are held longer than they otherwise need
1440          to be, and locks can be held longer than they need to be (as
1441          when a process is merely waiting for input to become
1442          available on an underlying file descriptor).</para>
1443          <para>Even if background processes rarely need to
1444          intentionally read input from the terminal, they may still
1445          need to do so in response to errors or other unanticipated
1446          situations. There are tradeoffs involved in any solution to
1447          this problem. The protocol described below allows background
1448          processes which follow it to reliably prompt for and receive
1449          terminal input. Background processes which attempt to
1450          receive terminal input without following this protocol will
1451          likely hang indefinitely while attempting to do so. That's
1452          certainly a harsh tradeoff, but since attempts to read
1453          terminal input without following this protocol only worked
1454          some of the time anyway, it doesn't seem to be an
1455          unreasonable one.</para>
1456          <para>In the solution described here (and introduced in
1457          OpenMCL 0.9), the internal stream used to provide terminal
1458          input is always locked by some process (the "owning"
1459          process.) The initial process (the process that typically
1460          runs the read-eval-print loop) owns that stream when it's
1461          first created. By using the macro WITH-TERMINAL-INPUT,
1462          background processes can temporarily obtain ownership of the
1463          terminal and relinquish ownership to the previous owner when
1464          they're done with it.</para>
1465          <para>In OpenMCL, BREAK, ERROR, CERROR, Y-OR-N-P,
1466          YES-OR-NO-P, and CCL:GET-STRING- FROM-USER are all defined
1467          in terms of WITH-TERMINAL-INPUT, as are the :TTY
1468          user-interfaces to STEP and INSPECT.</para>
1469      </sect2>
1470
1471      <sect2 id="background-terminal-example">
1472        <title>An example</title>
1473        <programlisting>
1474? Welcome to OpenMCL Version (Beta: linux) 0.9!
1475?
1476
1477? (process-run-function "sleeper" #'(lambda () (sleep 5) (break "broken")))
1478#<PROCESS sleeper(1) [Enabled] #x3063B33E&gt;
1479
1480?
1481;;
1482;; Process sleeper(1) needs access to terminal input.
1483;;
1484</programlisting>
1485        <para>This example was run under ILISP; ILISP often gets confused if one
1486        tries to enter input and "point" doesn't follow a prompt.
1487        Entering a "simple" expression at this point gets it back in
1488        synch; that's otherwise not relevant to this example.</para>
1489        <programlisting>
1490()
1491NIL
1492? (:y 1)
1493;;
1494;; process sleeper(1) now controls terminal input
1495;;
1496> Break in process sleeper(1): broken
1497> While executing: #<Anonymous Function #x3063B276&gt;
1498> Type :GO to continue, :POP to abort.
1499> If continued: Return from BREAK.
1500Type :? for other options.
15011 > :b
1502(30C38E30) : 0 "Anonymous Function #x3063B276" 52
1503(30C38E40) : 1 "Anonymous Function #x304984A6" 376
1504(30C38E90) : 2 "RUN-PROCESS-INITIAL-FORM" 340
1505(30C38EE0) : 3 "%RUN-STACK-GROUP-FUNCTION" 768
15061 &gt; :pop
1507;;
1508;; control of terminal input restored to process Initial(0)
1509;;
1510?
1511</programlisting>
1512      </sect2>
1513
1514      <sect2 id="A-more-elaborate-example-">
1515        <title>A more elaborate example.</title>
1516        <para>If a background process ("A") needs access to the terminal
1517        input stream and that stream is owned by another background process
1518        ("B"), process "A" announces that fact, then waits until
1519        the initial process regains control.</para>
1520        <programlisting>
1521? Welcome to OpenMCL Version (Beta: linux) 0.9!
1522?
1523
1524? (process-run-function "sleep-60" #'(lambda () (sleep 60) (break "Huh?")))
1525#<PROCESS sleep-60(1) [Enabled] #x3063BF26&gt;
1526
1527? (process-run-function "sleep-5" #'(lambda () (sleep 5) (break "quicker")))
1528#<PROCESS sleep-5(2) [Enabled] #x3063D0A6&gt;
1529
1530? ;;
1531;; Process sleep-5(2) needs access to terminal input.
1532;;
1533()
1534NIL
1535
1536? (:y 2)
1537;;
1538;; process sleep-5(2) now controls terminal input
1539;;
1540> Break in process sleep-5(2): quicker
1541> While executing: #x3063CFDE>
1542> Type :GO to continue, :POP to abort.
1543> If continued: Return from BREAK.
1544Type :? for other options.
15451 > ;; Process sleep-60(1) will need terminal access when
1546;; the initial process regains control of it.
1547;;
1548()
1549NIL
15501 > :pop
1551;;
1552;; Process sleep-60(1) needs access to terminal input.
1553;;
1554;;
1555;; control of terminal input restored to process Initial(0)
1556;;
1557
1558? (:y 1)
1559;;
1560;; process sleep-60(1) now controls terminal input
1561;;
1562> Break in process sleep-60(1): Huh?
1563> While executing: #x3063BE5E>
1564> Type :GO to continue, :POP to abort.
1565> If continued: Return from BREAK.
1566Type :? for other options.
15671 > :pop
1568;;
1569;; control of terminal input restored to process Initial(0)
1570;;
1571
1572?
1573</programlisting>
1574      </sect2>
1575
1576      <sect2 id="Summary">
1577        <title>Summary</title>
1578        <para>This scheme is certainly not bulletproof: imaginative
1579        use of PROCESS-INTERRUPT and similar functions might be able
1580        to defeat it and deadlock the lisp, and any scenario where
1581        several background processes are clamoring for access to the
1582        shared terminal input stream at the same time is likely to be
1583        confusing and chaotic. (An alternate scheme, where the input
1584        focus was magically granted to whatever thread the user was
1585        thinking about, was considered and rejected due to technical
1586        limitations.)</para>
1587        <para>The longer-term fix would probably involve using network or
1588        window-system streams to give each process unique instances of
1589        *TERMINAL-IO*.</para>
1590        <para>Existing code that attempts to read from *TERMINAL-IO*
1591        from a background process will need to be changed to use
1592        WITH-TERMINAL-INPUT.  Since that code was probably not working
1593        reliably in previous versions of OpenMCL, this requirement
1594        doesn't seem to be too onerous.</para>
1595        <para>Note that WITH-TERMINAL-INPUT both requests ownership of
1596        the terminal input stream and promises to restore that
1597        ownership to the initial process when it's done with it. An ad
1598        hoc use of READ or READ-CHAR doesn't make this promise; this
1599        is the rationale for the restriction on the :Y command.</para>
1600      </sect2>
1601    </sect1>
1602
1603    <sect1 id="The-Threads-which-OpenMCL-Uses-for-Its-Own-Purposes">
1604      <para>The Threads which OpenMCL Uses for Its Own Purposes
1605In the "tty world", OpenMCL starts out with 2 lisp-level threads:</para>
1606      <programlisting>
1607? :proc
16081 : -> listener     [Active]
16090 :    Initial      [Active]
1610</programlisting>
1611      <para>If you look at a running OpenMCL with a debugging tool,
1612      such as GDB, or Apple's Thread Viewer.app, you'll see an
1613      additional kernel-level thread on Darwin; this is used by the
1614      Mach exception-handling mechanism.</para>
1615      <para>The initial thread, conveniently named "initial", is the
1616      one that was created by the operating system when it launched
1617      OpenMCL.  It maps the heap image into memory, does some
1618      Lisp-level initialization, and, when the Cocoa IDE isn't being
1619      used, creates the thread "listener", which runs the top-level
1620      loop that reads input, evaluates it, and prints the
1621      result.</para>
1622      <para>After the listener thread is created, the initial thread
1623      does "housekeeping": it sits in a loop, sleeping most of the
1624      time and waking up occasionally to do "periodic tasks".  These
1625      tasks include forcing output on specified interactive streams,
1626      checking for and handling control-C interrupts, etc.  Currently,
1627      those tasks also include polling for the exit status of external
1628      processes and handling some kinds of I/O to and from those
1629      processes.</para>
1630      <para>In this environment, the initial thread does these
1631      "housekeeping" activities as necessary, until
1632      <literal>ccl:quit</literal> is called;
1633      <literal>quit</literal>ting interrupts the initial thread, which
1634      then ends all other threads in as orderly a fashion as possible
1635      and calls the C function <literal>#_exit</literal>.</para>
1636      <para>The short-term plan is to handle each external-process in
1637      a dedicated thread; the worst-case behavior of the current
1638      scheme can involve busy-waiting and excessive CPU utilization
1639      while waiting for an external process to terminate in some
1640      cases.</para>
1641      <para>The Cocoa features use more threads.  Adding a Cocoa
1642      listener creates two threads:</para>
1643      <programlisting>
1644? :proc
16453 : -> Listener     [Active]
16462 :    housekeeping  [Active]
16471 :    listener     [Active]
16480 :    Initial      [Active]
1649      </programlisting>
1650      <para>The Cocoa event loop has to run in the initial thread;
1651      when the event loop starts up, it creates a new thread to do the
1652      "housekeeping" tasks which the initial thread would do in the
1653      terminal-only mode.  The initial thread then becomes the one to
1654      receive all Cocoa events from the window server; it's the only
1655      thread which can.</para>
1656      <para>It also creates one "Listener" (capital-L) thread for each
1657      listener window, with a lifetime that lasts as long as the
1658      thread does.  So, if you open a second listener, you'll see five
1659      threads all together:</para>
1660      <programlisting>
1661? :proc
16624 : -> Listener-2   [Active]
16633 :    Listener     [Active]
16642 :    housekeeping  [Active]
16651 :    listener     [Active]
16660 :    Initial      [Active]
1667</programlisting>
1668      <para>Unix signals, such as SIGINT (control-C), invoke a handler
1669      installed by the Lisp kernel.  Although the OS doesn't make any
1670      specific guarantee about which thread will receive the signal,
1671      in practice, it seems to be the initial thread.  The handler
1672      just sets a flag and returns; the housekeeping thread (which may
1673      be the initial thread, if Cocoa's not being used) will check for
1674      the flag and take whatever action is appropriate to the
1675      signal.</para>
1676      <para>In the case of SIGINT, the action is to enter a break
1677      loop, by calling on the thread being interrupted.  When there's
1678      more than one Lisp listener active, it's not always clear what
1679      thread that should be, since it really depends on the user's
1680      intentions, which there's no way to divine programmatically.  To
1681      make its best guess, the handler first checks whether the value
1682      of <literal>ccl:*interactive-abort-process*</literal> is a
1683      thread, and, if so, uses it.  If that fails, it chooses the
1684      thread which currently "owns" the default terminal input stream;
1685      see .</para>
1686      <para>In the bleeding-edge version of the Cocoa support which is
1687      based on Hemlock, an Emacs-like editor, each editor window has a
1688      dedicated thread associated with it.  When a keypress event
1689      comes in which affects that specific window the initial thread
1690      sends it to the window's dedicated thread.  The dedicated thread
1691      is responsible for trying to interpret keypresses as Hemlock
1692      commands, applying those commands to the active buffer; it
1693      repeats this in a loop, until the window closes.  The initial
1694      thread handles all other events, such as mouse clicks and
1695      drags.</para>
1696      <para>This thread-per-window scheme makes many things simpler,
1697      including the process of entering a "recursive command loop" in
1698      commands like "Incremental Search Forward", etc.  (It might be
1699      possible to handle all Hemlock commands in the Cocoa event
1700      thread, but these "recursive command loops" would have to
1701      maintain a lot of context/state information; threads are a
1702      straightforward way of maintaining that information.)</para>
1703      <para>Currently (August 2004), when a dedicated thread needs to
1704      alter the contents of the buffer or the selection, it does so by
1705      invoking methods in the initial thread, for synchronization
1706      purposes, but this is probably overkill and will likely be
1707      replaced by a more efficient scheme in the future.</para>
1708      <para>The per-window thread could probably take more
1709      responsibility for drawing and handling the screen than it
1710      currently does; -something- needs to be done to buffer screen
1711      updates a bit better in some cases: you don't need to see
1712      everything that happens during something like indentation; you
1713      do need to see the results...</para>
1714      <para>When Hemlock is being used, listener windows are editor
1715      windows, so in addition to each "Listener" thread, you should
1716      also see a thread which handles Hemlock command
1717      processing.</para>
1718      <para>The Cocoa runtime may make additional threads in certain
1719      special situations; these threads usually don't run lisp code,
1720      and rarely if ever run much of it.</para>
1721    </sect1>
1722
1723    <sect1 id="Threads-Dictionary">
1724      <title>Threads Dictionary</title>
1725
1726      <refentry id="f_all-processes">
1727        <indexterm zone="f_all-processes">
1728          <primary>all-processes</primary>
1729        </indexterm>
1730
1731        <refnamediv>
1732          <refname>ALL-PROCESSES</refname>
1733          <refpurpose>Obtain a fresh list of all known Lisp
1734          threads.</refpurpose>
1735          <refclass>Function</refclass>
1736        </refnamediv>
1737
1738        <refsynopsisdiv>
1739          <synopsis>
1740            <function>all-processes</function> => result
1741          </synopsis>
1742        </refsynopsisdiv>
1743
1744        <refsect1>
1745          <title>Values</title>
1746         
1747          <variablelist>
1748            <varlistentry>
1749              <term>result</term>
1750              <listitem>
1751                <para>a list of all lisp processes (threads)
1752                known to OpenMCL.</para>
1753              </listitem>
1754            </varlistentry>
1755          </variablelist>
1756        </refsect1>
1757        <refsect1>
1758          <title>Description</title>
1759
1760          <para>Returns a list of all lisp processes (threads) known
1761          to OpenMCL as of the precise instant it&#39;s
1762          called. It&#39;s safe to traverse this list and to modify
1763          the cons cells that comprise that list (it&#39;s freshly
1764          consed.) Since other threads can create and kill threads at
1765          any time, there&#39;s generally no way to get an
1766          &#34;accurate&#34; list of all threads, and (generally) no
1767          sense in which such a list can be accurate.</para>
1768        </refsect1>
1769
1770        <refsect1>
1771          <title>See Also</title>
1772         
1773          <simplelist type="inline">
1774            <member><xref linkend="v_current-process"/></member>
1775          </simplelist>
1776        </refsect1>
1777      </refentry>
1778
1779
1780       <refentry id="f_make-process">
1781        <indexterm zone="f_make-process">
1782          <primary>make-process</primary>
1783        </indexterm>
1784
1785        <refnamediv>
1786          <refname>MAKE-PROCESS</refname>
1787          <refpurpose>Creates and returns a new process.</refpurpose>
1788          <refclass>Function</refclass>
1789        </refnamediv>
1790
1791        <refsynopsisdiv>
1792          <synopsis>
1793            <function>make-process</function>
1794            name &key;
1795            persistent priority class stack-size vstack-size
1796            tstack-size initial-bindings use-standard-initial-bindings
1797            => process
1798          </synopsis>
1799        </refsynopsisdiv>
1800
1801        <refsect1>
1802          <title>Arguments and Values</title>
1803         
1804          <variablelist>
1805            <varlistentry>
1806              <term>name</term>
1807             
1808              <listitem>
1809                <para>a string, used to identify the process.</para>
1810              </listitem>
1811            </varlistentry>
1812           
1813            <varlistentry>
1814              <term>persistent</term>
1815             
1816              <listitem>
1817                <para>if true, requests that information about the process
1818                be retained by SAVE-APPLICATION so that an equivalent
1819                process can be restarted when a saved image is run.  The
1820                default is nil.</para>
1821              </listitem>
1822            </varlistentry>
1823           
1824            <varlistentry>
1825              <term>priority</term>
1826             
1827              <listitem>
1828                <para>ignored.  It
1829                shouldn't be ignored of course, but there are
1830                complications on some platforms.  The default is 0.</para>
1831              </listitem>
1832            </varlistentry>
1833           
1834            <varlistentry>
1835              <term>class</term>
1836             
1837              <listitem>
1838                <para>the class of process object to create;
1839                should be a subclass of CCL:PROCESS.  The default is
1840                CCL:PROCESS.</para>
1841              </listitem>
1842            </varlistentry>
1843           
1844            <varlistentry>
1845              <term>stack-size</term>
1846             
1847              <listitem>
1848                <para>the size, in bytes, of the newly-created process's
1849                control stack; used for foreign function calls and to save
1850                function return address context.  The default is
1851                CCL:*DEFAULT-CONTROL-STACK-SIZE*.</para>
1852              </listitem>
1853            </varlistentry>
1854           
1855            <varlistentry>
1856              <term>vstack-size</term>
1857             
1858              <listitem>
1859                <para>the size, in bytes, of the newly-created process's
1860                value stack; used for lisp function arguments, local
1861                variables, and other stack-allocated lisp objects.
1862                The default is CCL:*DEFAULT-VALUE-STACK-SIZE*.</para>
1863              </listitem>
1864            </varlistentry>
1865           
1866            <varlistentry>
1867              <term>tstack-size</term>
1868             
1869              <listitem>
1870                <para>the size, in bytes, of the newly-created process's
1871                temp stack; used for the allocation of dynamic-extent
1872                objects.  The default is CCL:*DEFAULT-TEMP-STACK-SIZE*.</para>
1873              </listitem>
1874            </varlistentry>
1875           
1876            <varlistentry>
1877              <term>use-standard-initial-bindings</term>
1878             
1879              <listitem>
1880                <para>when true, the global "standard initial
1881                bindings" are put into effect in the new thread before. See
1882                DEF-STANDARD-INITIAL-BINDING.  "standard" initial bindings
1883                are put into effect before any bindings specified by
1884                :initial-bindings are.  The default is t.</para>
1885              </listitem>
1886            </varlistentry>
1887           
1888            <varlistentry>
1889              <term>initial-bindings</term>
1890             
1891              <listitem>
1892                <para>an alist of (<varname>symbol</varname> .
1893                <varname>valueform</varname>) pairs, which can be
1894                used to initialize special variable bindings in the new
1895                thread. Each <varname>valueform</varname> is used to
1896                compute the value of a new binding of
1897                <varname>symbol</varname> in the execution environment of
1898                the newly-created thread.  The default is nil.</para>
1899              </listitem>
1900            </varlistentry>
1901           
1902            <varlistentry>
1903              <term>process</term>
1904             
1905              <listitem>
1906                <para>the newly-created process.</para>
1907              </listitem>
1908            </varlistentry>
1909          </variablelist>
1910        </refsect1>
1911
1912        <refsect1>
1913          <title>Description</title>
1914
1915          <para>Creates and returns a new lisp process (thread) with the
1916          specified attributes. <varname>process</varname> will not begin
1917          execution immediately; it will need to be
1918          <emphasis>preset</emphasis> (given
1919          an initial function to run, as by
1920          <xref linkend="f_process-preset"/>) and
1921          <emphasis>enabled</emphasis>
1922          (allowed to execute, as by <xref linkend="f_process-enable"/>)
1923          before it&#39;s able to actually do anything.</para>
1924
1925          <para>If <varname>valueform</varname> is a function, it is
1926          called, with no arguments, in the execution environment of the
1927          newly-created thread; the primary value it returns is used for
1928          the binding of the corresponding <varname>symbol</varname>.</para>
1929
1930          <para>Otherwise, <varname>valueform</varname> is evaluated in the
1931          execution
1932          environment of the newly-created thread, and the resulting value
1933          is used.</para>
1934        </refsect1>
1935
1936        <refsect1>
1937          <title>See Also</title>
1938         
1939          <simplelist type="inline">
1940            <member><xref linkend="f_process-preset"/></member>
1941            <member><xref linkend="f_process-enable"/></member>
1942            <member><xref linkend="f_process-run-function"/></member>
1943          </simplelist>
1944        </refsect1>
1945      </refentry>
1946
1947      <sect2 id="PROCESS-SUSPEND">
1948        <para>PROCESS-SUSPEND</para>
1949        <informalfigure>process-suspend</informalfigure>
1950        <bridgehead renderas="sect3">Name</bridgehead>
1951        <para>PROCESS-SUSPEND &mdash; Suspends a specified process.</para>
1952        <para>Function</para>
1953        <bridgehead renderas="sect3">Synopsis</bridgehead>
1954        <programlisting>
1955process-suspend process
1956          => result
1957</programlisting>
1958        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
1959        <term><indexterm>process
1960            <variablelist>a lisp process (thread).</variablelist>
1961          </indexterm><indexterm>result
1962            <variablelist>T if <literal>process</literal> had been runnableand is now suspended; NIL otherwise.  That is, T if<literal>process</literal>'stransitioned from 0 to 1.</variablelist>
1963          </indexterm>
1964        </term>
1965        <bridgehead renderas="sect3">Description</bridgehead>
1966        <para>Suspends <literal>process</literal>, preventing it from
1967running, and stopping it if it was already running. This is a fairly
1968expensive operation, because it involves a few
1969calls to the OS.  It also risks creating deadlock if used
1970improperly, for instance, if the process being suspended owns a
1971lock or other resource which another process will wait for.</para>
1972        <para>Each
1973call to <literal>process-suspend</literal> must be reversed by
1974a matching call to
1975before <literal>process</literal> is able to run.  What
1976<literal>process-suspend</literal> actually does is increment
1977the  of
1978<literal>process</literal>.</para>
1979        <para>A process cannot suspend itself (although that was possible in
1980some older OpenMCL releases and this documentation claimed that
1981it still was.)</para>
1982        <bridgehead renderas="sect3">See Also</bridgehead>
1983        <para role="continues">,</para>
1984        <bridgehead renderas="sect3">Notes</bridgehead>
1985        <para><literal>process-suspend</literal> was previously called
1986<literal>process-disable</literal>.</para>
1987        <para>now names a function for which there is no
1988obvious inverse, so <literal>process-disable</literal>
1989is no longer
1990defined.</para>
1991      </sect2>
1992
1993      <sect2 id="PROCESS-RESUME">
1994        <para>PROCESS-RESUME</para>
1995        <informalfigure>process-resume</informalfigure>
1996        <bridgehead renderas="sect3">Name</bridgehead>
1997        <para>PROCESS-RESUME &mdash; Resumes a specified process which had previously
1998been suspended by process-suspend.</para>
1999        <para>Function</para>
2000        <bridgehead renderas="sect3">Synopsis</bridgehead>
2001        <programlisting>
2002process-resume process
2003          => result
2004</programlisting>
2005        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2006        <term><indexterm>process
2007            <variablelist>a lisp process (thread).</variablelist>
2008          </indexterm><indexterm>result
2009            <variablelist>T if <literal>process</literal> had been suspendedand is now runnable; NIL otherwise.  That is, T if<literal>process</literal>'stransitioned from  to 0.</variablelist>
2010          </indexterm>
2011        </term>
2012        <bridgehead renderas="sect3">Description</bridgehead>
2013        <para>Undoes the effect of a previous call to
2014; if
2015all such calls are undone, makes the process runnable. Has no
2016effect if the process is not suspended.  What
2017<literal>process-resume</literal> actually does is decrement
2018the  of
2019<literal>process</literal>, to a minimum of 0.</para>
2020        <bridgehead renderas="sect3">See Also</bridgehead>
2021        <para role="continues">,</para>
2022        <bridgehead renderas="sect3">Notes</bridgehead>
2023        <para>This was previously called PROCESS-ENABLE;
2024 now does something slightly
2025different.</para>
2026      </sect2>
2027
2028      <sect2 id="PROCESS-SUSPEND-COUNT">
2029        <para>PROCESS-SUSPEND-COUNT</para>
2030        <informalfigure>process-suspend-count</informalfigure>
2031        <bridgehead renderas="sect3">Name</bridgehead>
2032        <para>PROCESS-SUSPEND-COUNT &mdash; Returns the number of currently-pending suspensions
2033applicable to a given process.</para>
2034        <para>Function</para>
2035        <bridgehead renderas="sect3">Synopsis</bridgehead>
2036        <programlisting>
2037
2038            process-suspend-count
2039            process => result
2040
2041</programlisting>
2042        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2043        <term><indexterm>process
2044            <variablelist>a lisp process (thread).</variablelist>
2045          </indexterm><indexterm>result
2046            <variablelist>The number of "outstanding" calls on<literal>process</literal>, or NIL if<literal>process</literal> has expired.</variablelist>
2047          </indexterm>
2048        </term>
2049        <bridgehead renderas="sect3">Description</bridgehead>
2050        <para>An "outstanding"  call
2051is one which has not yet been reversed by a call to
2052.  A process expires when
2053its initial function returns, although it may later be
2054reset.</para>
2055        <para>A process is <emphasis>runnable</emphasis> when it has a
2056<literal>process-suspend-count</literal> of 0, has been
2057preset as by , and has been
2058enabled as by .  Newly-created
2059processes have a <literal>process-suspend-count</literal> of
20600.</para>
2061        <bridgehead renderas="sect3">See Also</bridgehead>
2062        <para role="continues">,</para>
2063      </sect2>
2064
2065      <sect2 id="PROCESS-PRESET">
2066        <para>PROCESS-PRESET</para>
2067        <informalfigure>process-preset</informalfigure>
2068        <bridgehead renderas="sect3">Name</bridgehead>
2069        <para>PROCESS-PRESET &mdash; Sets the initial function and arguments of a specified
2070process.</para>
2071        <para>Function</para>
2072        <bridgehead renderas="sect3">Synopsis</bridgehead>
2073        <programlisting>
2074process-preset
2075          process function &amp;rest args
2076          => result
2077</programlisting>
2078        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2079        <term><indexterm>process
2080            <variablelist>a lisp process (thread).</variablelist>
2081          </indexterm><indexterm>function
2082            <variablelist>a function, designated by itself or by a symbolwhich names it.</variablelist>
2083          </indexterm><indexterm>args
2084            <variablelist>a list of values, appropriate as arguments to<literal>function</literal>.</variablelist>
2085          </indexterm><indexterm>result
2086            <variablelist>undefined.</variablelist>
2087          </indexterm>
2088        </term>
2089        <bridgehead renderas="sect3">Description</bridgehead>
2090        <para>Typically used to initialize a newly-created or newly-reset
2091process, setting things up so that when <literal>process</literal>
2092becomes enabled, it will begin execution by
2093applying <literal>function</literal> to <literal>args</literal>.
2094<literal>process-preset</literal> does not enable
2095<literal>process</literal>,
2096although a process must be <literal>process-preset</literal>
2097before it can be enabled.  Processes are normally enabled by
2098.</para>
2099        <bridgehead renderas="sect3">See Also</bridgehead>
2100        <para role="continues">, ,</para>
2101      </sect2>
2102
2103      <sect2 id="PROCESS-ENABLE">
2104        <para>PROCESS-ENABLE</para>
2105        <informalfigure>process-enable</informalfigure>
2106        <bridgehead renderas="sect3">Name</bridgehead>
2107        <para>PROCESS-ENABLE &mdash; Begins executing the initial function of a specified
2108process.</para>
2109        <para>Function</para>
2110        <bridgehead renderas="sect3">Synopsis</bridgehead>
2111        <programlisting>
2112process-enable
2113          process &amp;optional timeout
2114
2115</programlisting>
2116        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2117        <term><indexterm>process
2118            <variablelist>a lisp process (thread).</variablelist>
2119          </indexterm><indexterm>timeout
2120            <variablelist>a time interval in seconds.  May be anynon-negative real number the <literal>floor</literal> ofwhich fits in 32 bits.  The default is 1.</variablelist>
2121          </indexterm><indexterm>result
2122            <variablelist>undefined.</variablelist>
2123          </indexterm>
2124        </term>
2125        <bridgehead renderas="sect3">Description</bridgehead>
2126        <para>Tries to begin the execution of <literal>process</literal>.
2127An error is signaled if <literal>process</literal> has never
2128been .  Otherwise,
2129<literal>process</literal> invokes its initial function.</para>
2130        <para><literal>process-enable</literal> attempts to
2131synchronize with <literal>process</literal>, which is presumed
2132to be reset or in the act of resetting itself.  If this attempt
2133is not successful within the time interval specified by
2134<literal>timeout</literal>, a continuable error is signaled,
2135which offers the opportunity to continue waiting.</para>
2136        <para>A process cannot meaningfully attempt to enable itself.</para>
2137        <bridgehead renderas="sect3">See Also</bridgehead>
2138        <para role="continues">, ,</para>
2139        <bridgehead renderas="sect3">Notes</bridgehead>
2140        <para>It would be nice to have more discussion of what it means
2141to synchronize with the process.</para>
2142      </sect2>
2143
2144      <sect2 id="PROCESS-RUN-FUNCTION">
2145        <para>PROCESS-RUN-FUNCTION</para>
2146        <informalfigure>process-run-function</informalfigure>
2147        <bridgehead renderas="sect3">Name</bridgehead>
2148        <para>PROCESS-RUN-FUNCTION &mdash; Creates a process, presets it, and enables it.</para>
2149        <para>Function</para>
2150        <bridgehead renderas="sect3">Synopsis</bridgehead>
2151        <programlisting>
2152process-run-function
2153          process-specifier function &amp;rest args => process
2154</programlisting>
2155        <term><indexterm>process-specifier
2156            <variablelist><literal>name</literal> |(<literal>&amp;key</literal> <literal>name</literal><literal>persistent</literal><literal>priority</literal><literal>class</literal><literal>stack-size</literal><literal>vstack-size</literal><literal>tstack-size</literal>)</variablelist>
2157          </indexterm>
2158        </term>
2159        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2160        <term><indexterm>name
2161            <variablelist>a string, used to identify the process.Passed to <literal>make-process</literal>.</variablelist>
2162          </indexterm><indexterm>function
2163            <variablelist>a function, designated by itself or by a symbolwhich names it.  Passed to<literal>preset-process</literal>.</variablelist>
2164          </indexterm><indexterm>persistent
2165            <variablelist>a boolean, passed to <literal>make-process</literal>.</variablelist>
2166          </indexterm><indexterm>priority
2167            <variablelist>ignored.</variablelist>
2168          </indexterm><indexterm>class
2169            <variablelist>a subclass of CCL:PROCESS.  Passed to<literal>make-process</literal>.</variablelist>
2170          </indexterm><indexterm>stack-size
2171            <variablelist>a size, in bytes.  Passed to<literal>make-process</literal>.</variablelist>
2172          </indexterm><indexterm>vstack-size
2173            <variablelist>a size, in bytes.  Passed to<literal>make-process</literal>.</variablelist>
2174          </indexterm><indexterm>tstack-size
2175            <variablelist>a size, in bytes.  Passed to<literal>make-process</literal>.</variablelist>
2176          </indexterm><indexterm>process
2177            <variablelist>the newly-created process.</variablelist>
2178          </indexterm>
2179        </term>
2180        <bridgehead renderas="sect3">Description</bridgehead>
2181        <para>Creates a lisp process (thread) via
2182,
2183presets it via , and
2184enables it via .  This means
2185that <literal>process</literal> will immediately begin to
2186execute.
2187<literal>process-run-function</literal> is
2188the simplest way to create and run a process.</para>
2189        <bridgehead renderas="sect3">See Also</bridgehead>
2190        <para role="continues">, ,</para>
2191      </sect2>
2192
2193      <sect2 id="PROCESS-INTERRUPT">
2194        <para>PROCESS-INTERRUPT</para>
2195        <informalfigure>process-interrupt</informalfigure>
2196        <bridgehead renderas="sect3">Name</bridgehead>
2197        <para>PROCESS-INTERRUPT &mdash; Arranges for the target process to invoke a
2198specified function at some point in the near future, and then
2199return to what it was doing.</para>
2200        <para>Function</para>
2201        <bridgehead renderas="sect3">Synopsis</bridgehead>
2202        <programlisting>
2203process-interrupt
2204          process function &amp;rest args => result
2205</programlisting>
2206        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2207        <term><indexterm>process
2208            <variablelist>a lisp process (thread).</variablelist>
2209          </indexterm><indexterm>function
2210            <variablelist>a function.</variablelist>
2211          </indexterm><indexterm>args
2212            <variablelist>a list of values, appropriate as arguments to<literal>function</literal>.</variablelist>
2213          </indexterm><indexterm>result
2214            <variablelist>the result of applying <literal>function</literal>to <literal>args</literal> if <literal>proceess</literal>is the <literal>current-process</literal>, otherwiseNIL.</variablelist>
2215          </indexterm>
2216        </term>
2217        <bridgehead renderas="sect3">Description</bridgehead>
2218        <para>Arranges for <literal>process</literal> to apply <literal>function</literal> to <literal>args</literal> at
2219some point in the near future (interrupting whatever <literal>process</literal>
2220was doing.) If <literal>function</literal> returns normally, <literal>process</literal>
2221resumes execution at the point at which it was interrupted.</para>
2222        <para><literal>process</literal> must be in an enabled state in order to respond to a
2223<literal>process-interrupt</literal> request.  It's perfectly legal for a process
2224to call <literal>process-interrupt</literal> on itself.</para>
2225        <para><literal>process-interrupt</literal> uses asynchronous POSIX signals to interrupt
2226threads. If the thread being interrupted is executing lisp code, it
2227can respond to the interrupt almost immediately (as soon as it has
2228finished pseudo-atomic operations like consing and stack-frame
2229initialization.)</para>
2230        <para>If the interrupted thread is blocking in a system call, that system
2231call is aborted by the signal and the interrupt is handled on return.</para>
2232        <para>Beginning with the version 1.1 prereleases of OpenMCL interrupts are
2233disabled in <literal>unwind-protect</literal> cleanup forms and in any stack-unwinding
2234code between the point of the <literal>throw</literal> and the corresponding CATCH
2235target.  If interrupts were enabled at the time that the CATCH was
2236established, then any interrupt that had been deferred during
2237unwinding will be taken just before the transfer to <literal>catch</literal> target
2238(after all of that unwinding is complete.)</para>
2239        <para>If an <literal>unwind-protect</literal> cleanup form actually does something that
2240needs to be interruptible, it's necessary to use
2241<literal>with-interrupts-enabled</literal>.  (This actually happens somewhere in
2242the code which waits for external processes to complete; some of that
2243waiting occurred within an unwind-protect cleanup, and interrupts
2244needed to be explicitly enabled in that case in order to make the wait
2245interruptible.)</para>
2246        <para>Note that <literal>(without-interrupts (throw ...))</literal> wouldn't have the intended
2247effect, since the <literal>throw</literal> would cause execution to exit the extent of
2248the <literal>without-interrupts</literal>.</para>
2249        <para>In versions prior to 1.1, interrupts could occur at arbitrary times
2250during the process of unwinding the stack and executing intervening
2251cleanup forms.</para>
2252        <para>It is still difficult to reliably interrupt arbitrary foreign code
2253(that may be stateful or otherwise non-reentrant); the interrupt
2254request is handled when such foreign code returns to or enters lisp.</para>
2255        <bridgehead renderas="sect3">See Also</bridgehead>
2256        <para role="continues"></para>
2257        <bridgehead renderas="sect3">Notes</bridgehead>
2258        <para>It would probably be better for <literal>result</literal> to always be NIL, since
2259the present behaviour is inconsistent.</para>
2260        <para><literal>Process-interrupt</literal> works by sending signals between threads, via
2261the C function <literal>#_pthread_signal</literal>.  It could be argued that it
2262should be done in one of several possible other ways under Darwin, to
2263make it practical to asynchronously interrupt things which make heavy
2264use of the Mach nanokernel.</para>
2265      </sect2>
2266
2267      <sect2 id="iCURRENT-PROCESS-">
2268        <para>*CURRENT-PROCESS*</para>
2269        <informalfigure>*current-process*</informalfigure>
2270        <bridgehead renderas="sect3">Name</bridgehead>
2271        <para>*CURRENT-PROCESS* &mdash; Bound in each process, to that process
2272itself.</para>
2273        <para>Variable</para>
2274        <bridgehead renderas="sect3">Value Type</bridgehead>
2275        <para>A lisp process (thread).</para>
2276        <bridgehead renderas="sect3">Initial Value</bridgehead>
2277        <para>Bound separately in each process, to that process itself.</para>
2278        <bridgehead renderas="sect3">Description</bridgehead>
2279        <para>Used when lisp code needs to find out what process it is
2280executing in.  Shouldn't be set by user code.</para>
2281        <bridgehead renderas="sect3">See Also</bridgehead>
2282        <para role="continues"></para>
2283      </sect2>
2284
2285      <sect2 id="PROCESS-RESET">
2286        <para>PROCESS-RESET</para>
2287        <informalfigure>process-reset</informalfigure>
2288        <bridgehead renderas="sect3">Name</bridgehead>
2289        <para>PROCESS-RESET &mdash; Causes a specified process to cleanly exit from
2290any ongoing computation.</para>
2291        <para>Function</para>
2292        <bridgehead renderas="sect3">Synopsis</bridgehead>
2293        <programlisting>
2294process-reset
2295          process &amp;optional kill-option => result
2296</programlisting>
2297        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2298        <term><indexterm>process
2299            <variablelist>a lisp process (thread).</variablelist>
2300          </indexterm><indexterm>kill-option
2301            <variablelist>a generalized boolean.  The default is T.</variablelist>
2302          </indexterm><indexterm>result
2303            <variablelist>undefined.</variablelist>
2304          </indexterm>
2305        </term>
2306        <bridgehead renderas="sect3">Description</bridgehead>
2307        <para>Causes <literal>process</literal> to cleanly exit
2308from any ongoing computation.  If <literal>kill-option</literal>
2309is true, <literal>process</literal> then exits.  Otherwise, it
2310enters a state where it can be
2311. This
2312is implemented by signaling a condition of type PROCESS-RESET;
2313user-defined condition handlers should generally refrain from
2314attempting to handle conditions of this type.</para>
2315        <para>A process can meaningfully reset itself.</para>
2316        <para>There is in general no way to know precisely when
2317<literal>process</literal>
2318has completed the act of resetting or killing itself; a process
2319which has either entered the limbo of the reset state or exited
2320has few ways of communicating either fact.</para>
2321        <para>can reliably determine when a process has entered
2322the "limbo of the reset state", but can't predict how long the
2323clean exit from ongoing computation might take: that depends on
2324the behavior of <literal>unwind-protect</literal> cleanup
2325forms, and of the OS scheduler.</para>
2326        <para>Resetting a process other than
2327 involves the
2328use of .</para>
2329        <bridgehead renderas="sect3">See Also</bridgehead>
2330        <para role="continues">,</para>
2331      </sect2>
2332
2333      <sect2 id="PROCESS-KILL">
2334        <para>PROCESS-KILL</para>
2335        <informalfigure>process-kill</informalfigure>
2336        <bridgehead renderas="sect3">Name</bridgehead>
2337        <para>PROCESS-KILL &mdash; Causes a specified process to cleanly exit from any
2338ongoing computation, and then exit.</para>
2339        <para>Function</para>
2340        <bridgehead renderas="sect3">Synopsis</bridgehead>
2341        <programlisting>
2342process-kill process
2343          => result
2344</programlisting>
2345        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2346        <term><indexterm>process
2347            <variablelist>a lisp process (thread).</variablelist>
2348          </indexterm><indexterm>result
2349            <variablelist>undefined.</variablelist>
2350          </indexterm>
2351        </term>
2352        <bridgehead renderas="sect3">Description</bridgehead>
2353        <para>Entirely equivalent to calling
2354(PROCESS-RESET PROCESS T).  Causes <literal>process</literal>
2355to cleanly exit from any ongoing computation, and then exit.</para>
2356        <bridgehead renderas="sect3">See Also</bridgehead>
2357        <para role="continues">,</para>
2358      </sect2>
2359
2360      <sect2 id="PROCESS-ABORT">
2361        <para>PROCESS-ABORT</para>
2362        <informalfigure>process-abort</informalfigure>
2363        <bridgehead renderas="sect3">Name</bridgehead>
2364        <para>PROCESS-ABORT &mdash; Causes a specified process to process an abort
2365condition, as if it had invoked
2366<literal>abort</literal>.</para>
2367        <para>Function</para>
2368        <bridgehead renderas="sect3">Synopsis</bridgehead>
2369        <programlisting>
2370process-abort process
2371          &amp;optional condition
2372          => NIL
2373</programlisting>
2374        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2375        <term><indexterm>process
2376            <variablelist>a lisp process (thread).</variablelist>
2377          </indexterm><indexterm>condition
2378            <variablelist>a lisp condition.  The default is NIL.</variablelist>
2379          </indexterm>
2380        </term>
2381        <bridgehead renderas="sect3">Description</bridgehead>
2382        <para>Entirely equivalent to calling
2383( <literal>process</literal>
2384(<literal>lambda</literal> ()
2385(<literal>abort</literal> <literal>condition</literal>))).
2386Causes <literal>process</literal> to transfer control to the
2387applicable handler or restart for <literal>abort</literal>.</para>
2388        <para>If <literal>condition</literal> is non-NIL,
2389<literal>process-abort</literal> does not consider any
2390handlers which are explicitly bound to conditions other than
2391<literal>condition</literal>.</para>
2392        <bridgehead renderas="sect3">See Also</bridgehead>
2393        <para role="continues">,</para>
2394      </sect2>
2395
2396      <sect2 id="iTICKS-PER-SECOND-">
2397        <para>*TICKS-PER-SECOND*</para>
2398        <informalfigure>*ticks-per-second*</informalfigure>
2399        <bridgehead renderas="sect3">Name</bridgehead>
2400        <para>*TICKS-PER-SECOND* &mdash; Bound to the clock resolution of the OS
2401scheduler.</para>
2402        <para>Variable</para>
2403        <bridgehead renderas="sect3">Value Type</bridgehead>
2404        <para>A positive integer.</para>
2405        <bridgehead renderas="sect3">Initial Value</bridgehead>
2406        <para>The clock resoluton of the OS scheduler.  Currently,
2407both LinuxPPC and DarwinPPC yeild an initial value of 100.</para>
2408        <bridgehead renderas="sect3">Description</bridgehead>
2409        <para>This value is ordinarily of marginal interest at best,
2410but, for backward compatibility, some functions accept timeout
2411values expressed in "ticks".  This value gives the number of
2412ticks per second.</para>
2413        <bridgehead renderas="sect3">See Also</bridgehead>
2414        <para role="continues"></para>
2415      </sect2>
2416
2417      <sect2 id="PROCESS-WHOSTATE">
2418        <para>PROCESS-WHOSTATE</para>
2419        <informalfigure>process-whostate</informalfigure>
2420        <bridgehead renderas="sect3">Name</bridgehead>
2421        <para>PROCESS-WHOSTATE &mdash; Returns a string which describes the status of
2422a specified process.</para>
2423        <para>Function</para>
2424        <bridgehead renderas="sect3">Synopsis</bridgehead>
2425        <programlisting>
2426process-whostate process
2427          => whostate
2428</programlisting>
2429        <term><indexterm>process
2430            <variablelist>a lisp process (thread).</variablelist>
2431          </indexterm><indexterm>whostate
2432            <variablelist>a string which describes the "state" of<literal>process</literal>.</variablelist>
2433          </indexterm>
2434        </term>
2435        <bridgehead renderas="sect3">Description</bridgehead>
2436        <para>This information is primarily for the benefit of
2437debugging tools.  <literal>whostate</literal> is a terse report
2438on what <literal>process</literal> is doing, or not doing,
2439and why.</para>
2440        <para>If the process is currently waiting in a call to
2441 or
2442, its
2443<literal>process-whostate</literal> will be the value
2444which was passed to that function as <literal>whostate</literal>.</para>
2445        <bridgehead renderas="sect3">See Also</bridgehead>
2446        <para role="continues">, ,</para>
2447        <bridgehead renderas="sect3">Notes</bridgehead>
2448        <para>This should arguably be SETFable, but doesn't seem to
2449ever have been.</para>
2450      </sect2>
2451
2452      <sect2 id="PROCESS-ALLOW-SCHEDULE">
2453        <para>PROCESS-ALLOW-SCHEDULE</para>
2454        <informalfigure>process-allow-schedule</informalfigure>
2455        <bridgehead renderas="sect3">Name</bridgehead>
2456        <para>PROCESS-ALLOW-SCHEDULE &mdash; Used for cooperative multitasking; probably never
2457necessary.</para>
2458        <para>Function</para>
2459        <bridgehead renderas="sect3">Synopsis</bridgehead>
2460        <programlisting>
2461process-allow-schedule
2462</programlisting>
2463        <bridgehead renderas="sect3">Description</bridgehead>
2464        <para>Advises the OS scheduler that the current thread has nothing
2465useful to do and that it should try to find some other thread to
2466schedule in its place. There's almost always a better
2467alternative, such as waiting for some specific event to
2468occur.  For example, you could use a lock or semaphore.</para>
2469        <bridgehead renderas="sect3">See Also</bridgehead>
2470        <para role="continues">, , , , ,</para>
2471        <bridgehead renderas="sect3">Notes</bridgehead>
2472        <para>This is a holdover from the days of cooperative
2473multitasking.  All modern general-purpose operating systems use
2474preemptive multitasking.</para>
2475      </sect2>
2476
2477      <sect2 id="PROCESS-WAIT">
2478        <para>PROCESS-WAIT</para>
2479        <informalfigure>process-wait</informalfigure>
2480        <bridgehead renderas="sect3">Name</bridgehead>
2481        <para>PROCESS-WAIT &mdash; Causes the current lisp process (thread) to wait for
2482a given
2483predicate to return true.</para>
2484        <para>Function</para>
2485        <bridgehead renderas="sect3">Synopsis</bridgehead>
2486        <programlisting>
2487process-wait
2488          whostate function &amp;rest args => result
2489</programlisting>
2490        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2491        <term><indexterm>whostate
2492            <variablelist>a string, which will be the value ofwhile the process is waiting.</variablelist>
2493          </indexterm><indexterm>function
2494            <variablelist>a function, designated by itself or by a symbolwhich names it.</variablelist>
2495          </indexterm><indexterm>args
2496            <variablelist>a list of values, appropriate as arguments to<literal>function</literal>.</variablelist>
2497          </indexterm><indexterm>result
2498            <variablelist>NIL.</variablelist>
2499          </indexterm>
2500        </term>
2501        <bridgehead renderas="sect3">Description</bridgehead>
2502        <para>Causes the current lisp process (thread) to repeatedly
2503apply <literal>function</literal> to
2504<literal>args</literal> until the call returns a true result, then
2505returns NIL. After
2506each failed call, yields the CPU as if by
2507.</para>
2508        <para>As with , it's almost
2509always more efficient to wait for some
2510specific event to occur; this isn't exactly busy-waiting, but
2511the OS scheduler can do a better job of scheduling if it's given
2512the relevant information.  For example, you could use a lock
2513or semaphore.</para>
2514        <bridgehead renderas="sect3">See Also</bridgehead>
2515        <para role="continues">, , , , , , ,</para>
2516      </sect2>
2517
2518      <sect2 id="PROCESS-WAIT-WITH-TIMEOUT">
2519        <para>PROCESS-WAIT-WITH-TIMEOUT</para>
2520        <informalfigure>process-wait-with-timeout</informalfigure>
2521        <bridgehead renderas="sect3">Name</bridgehead>
2522        <para>PROCESS-WAIT-WITH-TIMEOUT &mdash; Causes the current thread to wait for a given
2523predicate to return true, or for a timeout to expire.</para>
2524        <para>Function</para>
2525        <bridgehead renderas="sect3">Synopsis</bridgehead>
2526        <programlisting>
2527process-wait-with-timeout
2528          whostate ticks function args => result
2529</programlisting>
2530        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2531        <term><indexterm>whostate
2532            <variablelist>a string, which will be the value ofwhile the process is waiting.</variablelist>
2533          </indexterm><indexterm>ticks
2534            <variablelist>either a positive integer expressing a durationin "ticks" (see ),or NIL.</variablelist>
2535          </indexterm><indexterm>function
2536            <variablelist>a function, designated by itself or by a symbolwhich names it.</variablelist>
2537          </indexterm><indexterm>args
2538            <variablelist>a list of values, appropriate as arguments to<literal>function</literal>.</variablelist>
2539          </indexterm><indexterm>result
2540            <variablelist>T if <literal>process-wait-with-timeout</literal>returned because its <literal>function</literal> returnedtrue, or NIL if it returned because the duration<literal>ticks</literal> has been exceeded.</variablelist>
2541          </indexterm>
2542        </term>
2543        <bridgehead renderas="sect3">Description</bridgehead>
2544        <para>If <literal>ticks</literal> is NIL, behaves exactly like
2545, except for returning T.
2546Otherwise, <literal>function</literal> will be tested repeatedly,
2547in the same
2548kind of test/yield loop as in >
2549until either <literal>function</literal> returns true,
2550or the duration <literal>ticks</literal> has been exceeded.</para>
2551        <para>Having already read the descriptions of
2552 and
2553, the
2554astute reader has no doubt anticipated the observation that
2555better alternatives should be used whenever possible.</para>
2556        <bridgehead renderas="sect3">See Also</bridgehead>
2557        <para role="continues">, , , , , , , ,</para>
2558      </sect2>
2559
2560      <sect2 id="WITHOUT-INTERRUPTS">
2561        <para>WITHOUT-INTERRUPTS</para>
2562        <informalfigure>without-interrupts</informalfigure>
2563        <bridgehead renderas="sect3">Name</bridgehead>
2564        <para>WITHOUT-INTERRUPTS &mdash; Evaluates its body in an environment in which
2565process-interrupt requests are deferred.</para>
2566        <para>Macro</para>
2567        <bridgehead renderas="sect3">Synopsis</bridgehead>
2568        <programlisting>
2569without-interrupts
2570          &amp;body body => result
2571</programlisting>
2572        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2573        <term><indexterm>body
2574            <variablelist>an implicit progn.</variablelist>
2575          </indexterm><indexterm>result
2576            <variablelist>the primary value returned by<literal>body</literal>.</variablelist>
2577          </indexterm>
2578        </term>
2579        <bridgehead renderas="sect3">Description</bridgehead>
2580        <para>Executes <literal>body</literal> in an environment in which
2581 requests are deferred. As noted in the
2582description of , this has nothing to do with
2583the scheduling of other threads; it may be necessary to inhibit
2584 handling when (for instance) modifying some
2585data structure (for which the current thread holds an appropriate
2586lock) in some manner that's not reentrant.</para>
2587        <para>Beginning with the version 1.1 prereleases of OpenMCL interrupts are
2588disabled in <literal>unwind-protect</literal> cleanup forms and in any
2589stack-unwinding code between the point of the <literal>throw</literal> and the
2590corresponding CATCH target. If an <literal>unwind-protect</literal> cleanup form
2591actually does something that needs to be interruptible, it's necessary
2592to use <literal>with-interrupts-enabled</literal>. In versions prior to 1.1,
2593interrupts can occur at arbitrary times during the process of
2594unwinding the stack and executing intervening cleanup forms.</para>
2595        <bridgehead renderas="sect3">See Also</bridgehead>
2596        <para role="continues"></para>
2597      </sect2>
2598
2599      <sect2 id="MAKE-LOCK">
2600        <para>MAKE-LOCK</para>
2601        <informalfigure>make-lock</informalfigure>
2602        <bridgehead renderas="sect3">Name</bridgehead>
2603        <para>MAKE-LOCK &mdash; Creates and returns a lock object, which can
2604be used for synchronization between threads.</para>
2605        <para>Function</para>
2606        <bridgehead renderas="sect3">Synopsis</bridgehead>
2607        <programlisting>
2608make-lock &amp;optional
2609          name => lock
2610</programlisting>
2611        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2612        <term><indexterm>name
2613            <variablelist>any lisp object; saved as part of<literal>lock</literal>.  Typically a string or symbolwhich may appear in the sof threads which are waiting for <literal>lock</literal>.</variablelist>
2614          </indexterm><indexterm>lock
2615            <variablelist>a newly-allocated object of type CCL:LOCK.</variablelist>
2616          </indexterm>
2617        </term>
2618        <bridgehead renderas="sect3">Description</bridgehead>
2619        <para>Creates and returns a lock object, which can
2620be used to synchronize access to some shared resource.
2621<literal>lock</literal> is
2622initially in a "free" state; a lock can also be
2623"owned" by a
2624thread.</para>
2625        <bridgehead renderas="sect3">See Also</bridgehead>
2626        <para role="continues">, , , , , , , ,</para>
2627      </sect2>
2628
2629      <sect2 id="WITH-LOCK-GRABBED">
2630        <para>WITH-LOCK-GRABBED</para>
2631        <informalfigure>with-lock-grabbed</informalfigure>
2632        <bridgehead renderas="sect3">Name</bridgehead>
2633        <para>WITH-LOCK-GRABBED &mdash; Waits until a given lock can be obtained, then
2634evaluates its body with the lock held.</para>
2635        <para>Macro</para>
2636        <bridgehead renderas="sect3">Synopsis</bridgehead>
2637        <programlisting>
2638with-lock-grabbed
2639          (lock) &amp;body body
2640</programlisting>
2641        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2642        <term><indexterm>lock
2643            <variablelist>an object of type CCL:LOCK.</variablelist>
2644          </indexterm><indexterm>body
2645            <variablelist>an implicit progn.</variablelist>
2646          </indexterm><indexterm>result
2647            <variablelist>the primary value returned by<literal>body</literal>.</variablelist>
2648          </indexterm>
2649        </term>
2650        <bridgehead renderas="sect3">Description</bridgehead>
2651        <para>Waits until <literal>lock</literal> is either free or
2652owned by the calling
2653thread, then excutes <literal>body</literal> with the
2654lock owned by the calling thread. If <literal>lock</literal>
2655was free when <literal>with-lock-grabbed</literal> was called,
2656it is restored to a free state after <literal>body</literal>
2657is executed.</para>
2658        <bridgehead renderas="sect3">See Also</bridgehead>
2659        <para role="continues">, , , , , , , ,</para>
2660      </sect2>
2661
2662      <sect2 id="GRAB-LOCK">
2663        <para>GRAB-LOCK</para>
2664        <informalfigure>grab-lock</informalfigure>
2665        <bridgehead renderas="sect3">Name</bridgehead>
2666        <para>GRAB-LOCK &mdash; Waits until a given lock can be obtained, then
2667obtains it.</para>
2668        <para>Function</para>
2669        <bridgehead renderas="sect3">Synopsis</bridgehead>
2670        <programlisting>
2671grab-lock lock
2672</programlisting>
2673        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2674        <term><indexterm>lock
2675            <variablelist>an object of type CCL:LOCK.</variablelist>
2676          </indexterm>
2677        </term>
2678        <bridgehead renderas="sect3">Description</bridgehead>
2679        <para>Blocks until <literal>lock</literal> is owned by the
2680calling thread.</para>
2681        <para>The macro
2682<emphasis>could</emphasis> be defined in
2683terms of <literal>grab-lock</literal> and
2684, but it is actually
2685implemented at a slightly lower level.</para>
2686        <bridgehead renderas="sect3">See Also</bridgehead>
2687        <para role="continues">, , , , , , , ,</para>
2688      </sect2>
2689
2690      <sect2 id="RELEASE-LOCK">
2691        <para>RELEASE-LOCK</para>
2692        <informalfigure>release-lock</informalfigure>
2693        <bridgehead renderas="sect3">Name</bridgehead>
2694        <para>RELEASE-LOCK &mdash; Relinquishes ownership of a given lock.</para>
2695        <para>Function</para>
2696        <bridgehead renderas="sect3">Synopsis</bridgehead>
2697        <programlisting>
2698release-lock lock
2699</programlisting>
2700        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2701        <term><indexterm>lock
2702            <variablelist>an object of type CCL:LOCK.</variablelist>
2703          </indexterm>
2704        </term>
2705        <bridgehead renderas="sect3">Description</bridgehead>
2706        <para>Signals an error of type CCL:LOCK-NOT-OWNER if
2707<literal>lock</literal>
2708is not already owned by the calling thread; otherwise, undoes the
2709effect of one previous
2710.  If this means that
2711<literal>release-lock</literal> has now been called on
2712<literal>lock</literal> the same number of times as
2713 has, <literal>lock</literal>
2714becomes free.</para>
2715        <bridgehead renderas="sect3">See Also</bridgehead>
2716        <para role="continues">, , , , , , , ,</para>
2717      </sect2>
2718
2719      <sect2 id="TRY-LOCK">
2720        <para>TRY-LOCK</para>
2721        <informalfigure>try-lock</informalfigure>
2722        <bridgehead renderas="sect3">Name</bridgehead>
2723        <para>TRY-LOCK &mdash; Obtains the given lock, but only if it is not
2724necessary to wait for it.</para>
2725        <para>Function</para>
2726        <bridgehead renderas="sect3">Synopsis</bridgehead>
2727        <programlisting>
2728try-lock lock => result
2729</programlisting>
2730        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2731        <term><indexterm>lock
2732            <variablelist>an object of type CCL:LOCK.</variablelist>
2733          </indexterm><indexterm>result
2734            <variablelist>T if <literal>lock</literal> has been obtained,or NIL if it has not.</variablelist>
2735          </indexterm>
2736        </term>
2737        <bridgehead renderas="sect3">Description</bridgehead>
2738        <para>Tests whether <literal>lock</literal>
2739can be obtained without blocking - that is, either
2740<literal>lock</literal> is already free, or it is already owned
2741by .  If it can,
2742causes it to
2743be owned by the calling lisp process (thread) and returns T.
2744Otherwise, the lock
2745is already owned by another thread and cannot be obtained without
2746blocking; NIL is returned in this case.</para>
2747        <bridgehead renderas="sect3">See Also</bridgehead>
2748        <para role="continues">, , , , , , , ,</para>
2749      </sect2>
2750
2751      <sect2 id="MAKE-READ-WRITE-LOCK">
2752        <para>MAKE-READ-WRITE-LOCK</para>
2753        <informalfigure>make-read-write-lock</informalfigure>
2754        <bridgehead renderas="sect3">Name</bridgehead>
2755        <para>MAKE-READ-WRITE-LOCK &mdash; Creates and returns a read-write lock, which can
2756be used for synchronization between threads.</para>
2757        <para>Function</para>
2758        <bridgehead renderas="sect3">Synopsis</bridgehead>
2759        <programlisting>
2760make-read-write-lock
2761          => read-write-lock
2762</programlisting>
2763        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2764        <term><indexterm>read-write-lock
2765            <variablelist>a newly-allocated object of typeCCL:READ-WRITE-LOCK.</variablelist>
2766          </indexterm>
2767        </term>
2768        <bridgehead renderas="sect3">Description</bridgehead>
2769        <para>Creates and returns an object of type CCL::READ-WRITE-LOCK.
2770A read-write lock may, at any given time, belong to any number
2771of lisp processes (threads) which act as "readers"; or, it may
2772belong to at most one process which acts as a "writer".  A
2773read-write lock may never be held by a reader at the same time as
2774a writer.  Intially, <literal>read-write-lock</literal> has
2775no readers and no writers.</para>
2776        <bridgehead renderas="sect3">See Also</bridgehead>
2777        <para role="continues">, , , , , ,</para>
2778        <bridgehead renderas="sect3">Notes</bridgehead>
2779        <para>There probably should be some way to
2780atomically "promote" a reader, making it a writer without
2781releasing the lock, which could otherwise cause delay.</para>
2782      </sect2>
2783
2784      <sect2 id="WITH-READ-LOCK">
2785        <para>WITH-READ-LOCK</para>
2786        <informalfigure>with-read-lock</informalfigure>
2787        <bridgehead renderas="sect3">Name</bridgehead>
2788        <para>WITH-READ-LOCK &mdash; Waits until a given lock is available for
2789read-only access, then evaluates its body with the lock
2790held.</para>
2791        <para>Macro</para>
2792        <bridgehead renderas="sect3">Synopsis</bridgehead>
2793        <programlisting>
2794with-read-lock
2795          (read-write-lock) &amp;body body => result
2796</programlisting>
2797        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2798        <term><indexterm>read-write-lock
2799            <variablelist>an object of typeCCL:READ-WRITE-LOCK.</variablelist>
2800          </indexterm><indexterm>body
2801            <variablelist>an implicit progn.</variablelist>
2802          </indexterm><indexterm>result
2803            <variablelist>the primary value returned by<literal>body</literal>.</variablelist>
2804          </indexterm>
2805        </term>
2806        <bridgehead renderas="sect3">Description</bridgehead>
2807        <para>Waits until <literal>read-write-lock</literal> has no
2808writer,
2809ensures that  is a
2810reader of it, then executes <literal>body</literal>.</para>
2811        <para>After executing <literal>body</literal>, if
2812 was not a reader of
2813<literal>read-write-lock</literal> before
2814<literal>with-read-lock</literal> was called, the lock is
2815released.  If it was already a reader, it remains one.</para>
2816        <bridgehead renderas="sect3">See Also</bridgehead>
2817        <para role="continues">, , , , , ,</para>
2818      </sect2>
2819
2820      <sect2 id="WITH-WRITE-LOCK">
2821        <para>WITH-WRITE-LOCK</para>
2822        <informalfigure>with-write-lock</informalfigure>
2823        <bridgehead renderas="sect3">Name</bridgehead>
2824        <para>WITH-WRITE-LOCK &mdash; Waits until the given lock is available for write
2825access, then executes its body with the lock held.</para>
2826        <para>Macro</para>
2827        <bridgehead renderas="sect3">Synopsis</bridgehead>
2828        <programlisting>
2829with-write-lock
2830          (read-write-lock) &amp;body body
2831</programlisting>
2832        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2833        <term><indexterm>read-write-lock
2834            <variablelist>an object of typeCCL:READ-WRITE-LOCK.</variablelist>
2835          </indexterm><indexterm>body
2836            <variablelist>an implicit progn.</variablelist>
2837          </indexterm><indexterm>result
2838            <variablelist>the primary value returned by<literal>body</literal>.</variablelist>
2839          </indexterm>
2840        </term>
2841        <bridgehead renderas="sect3">Description</bridgehead>
2842        <para>Waits until <literal>read-write-lock</literal> has no
2843readers and no writer other than ,
2844then ensures that  is the
2845writer of it.  With the lock held, executes <literal>body</literal>.</para>
2846        <para>After executing <literal>body</literal>, if
2847 was not the writer of
2848<literal>read-write-lock</literal> before
2849<literal>with-write-lock</literal> was called, the lock is
2850released.  If it was already the writer, it remains the
2851writer.</para>
2852        <bridgehead renderas="sect3">See Also</bridgehead>
2853        <para role="continues">, , , , , ,</para>
2854      </sect2>
2855
2856      <sect2 id="MAKE-SEMAPHORE">
2857        <para>MAKE-SEMAPHORE</para>
2858        <informalfigure>make-semaphore</informalfigure>
2859        <bridgehead renderas="sect3">Name</bridgehead>
2860        <para>MAKE-SEMAPHORE &mdash; Creates and returns a semaphore, which can be used
2861for synchronization between threads.</para>
2862        <para>Function</para>
2863        <bridgehead renderas="sect3">Synopsis</bridgehead>
2864        <programlisting>
2865make-semaphore
2866          => semaphore
2867</programlisting>
2868        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2869        <term><indexterm>semaphore
2870            <variablelist>a newly-allocated object of type CCL:SEMAPHORE.</variablelist>
2871          </indexterm>
2872        </term>
2873        <bridgehead renderas="sect3">Description</bridgehead>
2874        <para>Creates and returns an object of type CCL:SEMAPHORE.
2875A semaphore has an associated "count" which may be incremented
2876and decremented atomically; incrementing it represents sending
2877a signal, and decrementing it represents handling that signal.
2878<literal>semaphore</literal> has an initial count of 0.</para>
2879        <bridgehead renderas="sect3">See Also</bridgehead>
2880        <para role="continues">, , , , , , ,</para>
2881      </sect2>
2882
2883      <sect2 id="SIGNAL-SEMAPHORE">
2884        <para>SIGNAL-SEMAPHORE</para>
2885        <informalfigure>signal-semaphore</informalfigure>
2886        <bridgehead renderas="sect3">Name</bridgehead>
2887        <para>SIGNAL-SEMAPHORE &mdash; Atomically increments the count of a given
2888semaphore.</para>
2889        <para>Function</para>
2890        <bridgehead renderas="sect3">Synopsis</bridgehead>
2891        <programlisting>
2892signal-semaphore
2893          semaphore => result
2894</programlisting>
2895        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2896        <term><indexterm>semaphore
2897            <variablelist>an object of type CCL:SEMAPHORE.</variablelist>
2898          </indexterm><indexterm>result
2899            <variablelist>an integer representing an error identifierwhich was returned by the underlying OS call.</variablelist>
2900          </indexterm>
2901        </term>
2902        <bridgehead renderas="sect3">Description</bridgehead>
2903        <para>Atomically increments <literal>semaphore</literal>'s
2904"count" by 1; this
2905may enable a waiting thread to resume execution.</para>
2906        <bridgehead renderas="sect3">See Also</bridgehead>
2907        <para role="continues">, , , , , , ,</para>
2908        <bridgehead renderas="sect3">Notes</bridgehead>
2909        <para><literal>result</literal> should probably be interpreted
2910and acted on by <literal>signal-semaphore</literal>, because
2911it is not likely to be meaningful to a lisp program, and the
2912most common cause of failure is a type error.</para>
2913      </sect2>
2914
2915      <sect2 id="WAIT-ON-SEMAPHORE">
2916        <para>WAIT-ON-SEMAPHORE</para>
2917        <informalfigure>wait-on-semaphore</informalfigure>
2918        <bridgehead renderas="sect3">Name</bridgehead>
2919        <para>WAIT-ON-SEMAPHORE &mdash; Waits until the given semaphore has a positive
2920count which can be atomically decremented.</para>
2921        <para>Function</para>
2922        <bridgehead renderas="sect3">Synopsis</bridgehead>
2923        <programlisting>
2924wait-on-semaphore
2925          semaphore => result
2926</programlisting>
2927        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2928        <term><indexterm>semaphore
2929            <variablelist>an object of type CCL:SEMAPHORE.</variablelist>
2930          </indexterm><indexterm>result
2931            <variablelist>WAIT-ON-SEMAPHORE always returns T.</variablelist>
2932          </indexterm>
2933        </term>
2934        <bridgehead renderas="sect3">Description</bridgehead>
2935        <para>Waits until <literal>semaphore</literal>
2936has a positive count that can be
2937atomically decremented; this will succeed exactly once for each
2938corresponding call to SIGNAL-SEMAPHORE.</para>
2939        <bridgehead renderas="sect3">See Also</bridgehead>
2940        <para role="continues">, , , , , , ,</para>
2941        <bridgehead renderas="sect3">Notes</bridgehead>
2942      </sect2>
2943
2944      <sect2 id="TIMED-WAIT-ON-SEMAPHORE">
2945        <para>TIMED-WAIT-ON-SEMAPHORE</para>
2946        <informalfigure>timed-wait-on-semaphore</informalfigure>
2947        <bridgehead renderas="sect3">Name</bridgehead>
2948        <para>TIMED-WAIT-ON-SEMAPHORE &mdash; Waits until the given semaphore has a postive
2949count which can be atomically decremented, or until a timeout
2950expires.</para>
2951        <para>Function</para>
2952        <bridgehead renderas="sect3">Synopsis</bridgehead>
2953        <programlisting>
2954timed-wait-on-semaphore
2955          semaphore timeout => result
2956</programlisting>
2957        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2958        <term><indexterm>semaphore
2959            <variablelist>An object of type CCL:SEMAPHORE.</variablelist>
2960          </indexterm><indexterm>timeout
2961            <variablelist>a time interval in seconds.  May be anynon-negative real number the <literal>floor</literal> ofwhich fits in 32 bits.  The default is 1.</variablelist>
2962          </indexterm><indexterm>result
2963            <variablelist>T if <literal>timed-wait-on-semaphore</literal>returned because it was able to decrement the count of<literal>semaphore</literal>; NIL if it returned becausethe duration <literal>timeout</literal> has beenexceeded.</variablelist>
2964          </indexterm>
2965        </term>
2966        <bridgehead renderas="sect3">Description</bridgehead>
2967        <para>Waits until <literal>semaphore</literal>
2968has a positive count that can be
2969atomically decremented, or until the duration
2970<literal>timeout</literal> has
2971elapsed.</para>
2972        <bridgehead renderas="sect3">See Also</bridgehead>
2973        <para role="continues">, , , , , ,</para>
2974      </sect2>
2975
2976      <sect2 id="PROCESS-INPUT-WAIT">
2977        <para>PROCESS-INPUT-WAIT</para>
2978        <informalfigure>process-input-wait</informalfigure>
2979        <bridgehead renderas="sect3">Name</bridgehead>
2980        <para>PROCESS-INPUT-WAIT &mdash; Waits until input is available on a given
2981file-descriptor.</para>
2982        <para>Function</para>
2983        <bridgehead renderas="sect3">Synopsis</bridgehead>
2984        <programlisting>
2985process-input-wait
2986          fd &amp;optional timeout
2987</programlisting>
2988        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
2989        <term><indexterm>fd
2990            <variablelist>a file descriptor, which is a non-negative integerused by the OS to refer to an open file, socket, or similarI/O connection.  See CCL::STREAM-DEVICE.</variablelist>
2991          </indexterm><indexterm>timeout
2992            <variablelist>either NIL, or a time interval in seconds.  May be anynon-negative real number the <literal>floor</literal> ofwhich fits in 32 bits.  The default is NIL.</variablelist>
2993          </indexterm>
2994        </term>
2995        <bridgehead renderas="sect3">Description</bridgehead>
2996        <para>Wait until input is available on <literal>fd</literal>.
2997This uses the <literal>select()</literal> system call, and is
2998generally a fairly
2999efficient way of blocking while waiting for input. More
3000accurately, <literal>process-input-wait</literal>
3001waits until it's possible to read
3002from fd without blocking, or until <literal>timeout</literal>, if
3003it is not NIL, has been exceeded.</para>
3004        <para>Note that it's possible to read without blocking if
3005the file is at its end - although, of course, the read will
3006return zero bytes.</para>
3007        <bridgehead renderas="sect3">See Also</bridgehead>
3008        <para role="continues">, , , ,</para>
3009        <bridgehead renderas="sect3">Notes</bridgehead>
3010        <para><literal>process-input-wait</literal> has a timeout parameter,
3011and
3012 does not.  This
3013inconsistency should probably be corrected.</para>
3014      </sect2>
3015
3016      <sect2 id="PROCESS-OUTPUT-WAIT">
3017        <para>PROCESS-OUTPUT-WAIT</para>
3018        <informalfigure>process-output-wait</informalfigure>
3019        <bridgehead renderas="sect3">Name</bridgehead>
3020        <para>PROCESS-OUTPUT-WAIT &mdash; Waits until output is possible on a given file
3021descriptor.</para>
3022        <para>Function</para>
3023        <bridgehead renderas="sect3">Synopsis</bridgehead>
3024        <programlisting>
3025process-output-wait fd
3026</programlisting>
3027        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3028        <term><indexterm>fd
3029            <variablelist>a file descriptor, which is a non-negative integerused by the OS to refer to an open file, socket, or similarI/O connection.  See CCL::STREAM-DEVICE.</variablelist>
3030          </indexterm>
3031        </term>
3032        <bridgehead renderas="sect3">Description</bridgehead>
3033        <para>Wait until output is possible on <literal>fd</literal>.
3034This uses the <literal>select()</literal> system call, and is
3035generally a fairly
3036efficient way of blocking while waiting to output.</para>
3037        <para>If <literal>process-output-wait</literal> is called on
3038a network socket which has not yet established a connection, it
3039will wait until the connection is established.  This is an
3040important use, often overlooked.</para>
3041        <bridgehead renderas="sect3">See Also</bridgehead>
3042        <para role="continues">, , , ,</para>
3043        <bridgehead renderas="sect3">Notes</bridgehead> 
3044        <para>has a timeout parameter,
3045and
3046<literal>process-output-wait</literal> does not.  This
3047inconsistency should probably be corrected.</para>
3048      </sect2>
3049
3050      <sect2 id="WITH-TERMINAL-INPUT">
3051        <para>WITH-TERMINAL-INPUT</para>
3052        <informalfigure>with-terminal-input</informalfigure>
3053        <bridgehead renderas="sect3">Name</bridgehead>
3054        <para>WITH-TERMINAL-INPUT &mdash; Executes its body in an environment with exclusive
3055read access to the terminal.</para>
3056        <para>Macro</para>
3057        <bridgehead renderas="sect3">Synopsis</bridgehead>
3058        <programlisting>
3059with-terminal-input
3060          &amp;body body => result
3061</programlisting>
3062        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3063        <term><indexterm>body
3064            <variablelist>an implicit progn.</variablelist>
3065          </indexterm><indexterm>result
3066            <variablelist>the primary value returned by<literal>body</literal>.</variablelist>
3067          </indexterm>
3068        </term>
3069        <bridgehead renderas="sect3">Description</bridgehead>
3070        <para>Requests exclusive read access to the standard terminal
3071stream, <literal>*terminal-io*</literal>.  Executes
3072<literal>body</literal> in an environment with that access.</para>
3073        <bridgehead renderas="sect3">See Also</bridgehead>
3074        <para role="continues">, :Y, , , , ,</para>
3075      </sect2>
3076
3077      <sect2 id="iREQUEST-TERMINAL-INPUT-VIA-BREAK-">
3078        <para>*REQUEST-TERMINAL-INPUT-VIA-BREAK*</para>
3079        <informalfigure>request-terminal-input-via-break</informalfigure>
3080        <bridgehead renderas="sect3">Name</bridgehead>
3081        <para>*REQUEST-TERMINAL-INPUT-VIA-BREAK* &mdash; Controls how attempts to obtain ownership of
3082terminal input are made.</para>
3083        <para>Variable</para>
3084        <bridgehead renderas="sect3">Value Type</bridgehead>
3085        <para>A boolean.</para>
3086        <bridgehead renderas="sect3">Initial Value</bridgehead>
3087        <para>NIL.</para>
3088        <bridgehead renderas="sect3">Description</bridgehead>
3089        <para>Controls how attempts to obtain ownership of terminal input
3090are made. When NIL, a message is printed on *TERMINAL-IO*;
3091it's expected that the user will later yield
3092control of the terminal via the :Y toplevel command. When T, a
3093BREAK condition is signaled in the owning process; continuing from
3094the break loop will yield the terminal to the requesting process
3095(unless the :Y command was already used to do so in the break
3096loop.)</para>
3097        <bridgehead renderas="sect3">See Also</bridgehead>
3098        <para role="continues">, :Y, , , , ,</para>
3099      </sect2>
3100
3101      <sect2 id="iY">
3102        <para>:Y</para>
3103        <informalfigure>:y</informalfigure>
3104        <bridgehead renderas="sect3">Name</bridgehead>
3105        <para>:Y &mdash; Yields control of terminal input to a specified
3106lisp process (thread).</para>
3107        <para>Toplevel Command</para>
3108        <bridgehead renderas="sect3">Synopsis</bridgehead>
3109        <programlisting>
3110(:y p)
3111</programlisting>
3112        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3113        <term><indexterm>p
3114            <variablelist>a lisp process (thread), designated either byan integer which matches its<literal>process-serial-number</literal>,or by a string which is <literal>equal</literal> toits <literal>process-name</literal>.</variablelist>
3115          </indexterm>
3116        </term>
3117        <bridgehead renderas="sect3">Description</bridgehead>
3118        <para>:Y is a toplevel command, not a function.  As such, it
3119can only be used interactively, and only from the initial
3120process.</para>
3121        <para>The command yields control of terminal input to the
3122process <literal>p</literal>, which must have used
3123 to request access to the
3124terminal input stream.</para>
3125        <bridgehead renderas="sect3">See Also</bridgehead>
3126        <para role="continues">, , , , , ,</para>
3127      </sect2>
3128    </sect1>
3129  </chapter>
3130
3131  <chapter id="Programming-with-Sockets">
3132    <title>Programming with Sockets</title>
3133
3134    <sect1 id="Sockets-Overview">
3135      <para>OverviewOpenMCL supports the socket abstraction for
3136      interprocess communication. A socket represents a connection to
3137      another process, typically (but not necessarily) a TCP/IP
3138      network connection to a client or server running on some other
3139      machine on the network.</para>
3140      <para>All symbols mentioned in this chapter are exported from
3141      the CCL package. As of version 0.13, these symbols are
3142      additionally exported from the OPENMCL-SOCKET package.</para>
3143      <para>OpenMCL supports three types of sockets: TCP sockets, UDP
3144      sockets, and Unix-domain sockets.  This should be enough for all
3145      but the most esoteric network situations.  All sockets are
3146      created by .  The type of socket depends on the arguments to it,
3147      as follows:</para>
3148      <term><indexterm>tcp-stream
3149          <variablelist>A buffered bi-directional stream over a TCP/IP connection.tcp-stream is a subclass of stream, and you can read and write to itusing all the usual stream functions. Created by (make-socket:addess-family :internet :type :stream :connect :active ...) or by(accept-connection ...).</variablelist>
3150        </indexterm><indexterm>file-socket-stream
3151          <variablelist>A buffered bi-directional stream over a "UNIX domain"connection. file-socket-stream is a subclass of stream, and you canread and write to it using all the usual stream functions. Createdby (make-socket :address-family :file :type :stream :connect :active...) or by (accept-connection ...),</variablelist>
3152        </indexterm><indexterm>listener-socket
3153          <variablelist>A passive socket used to listen for incoming TCP/IPconnections on a particular port. A listener-socket is not a stream.It doesn't support I/O. It can only be used to create newtcp-streams by accept-connection. Created by (make-socket :type:stream :connect :passive ...)</variablelist>
3154        </indexterm><indexterm>file-listener-socket
3155          <variablelist>A passive socket used to listen for incoming UNIX domainconnections named by a file in the local filesystem. Alistener-socket is not a stream. It doesn't support I/O. It canonly be used to create new file-socket-streams by accept-connection.Created by (make-socket :address-family :file :type :stream :connect:passive ...)</variablelist>
3156        </indexterm><indexterm>udp-socket
3157          <variablelist>A socket representing a packet-based UDP/IP connection. Audp-socket supports I/O but it is not a stream. Instead, you mustuse the special functions send-to and receive-from to read and writeto it. Created by (make-socket :type :datagram ...)</variablelist>
3158        </indexterm>
3159      </term>
3160    </sect1>
3161
3162    <sect1 id="Sockets-Dictionary">
3163      <title>Sockets Dictionary</title>
3164
3165      <sect2 id="MAKE-SOCKET">
3166        <para>MAKE-SOCKET</para>
3167        <informalfigure>make-socket</informalfigure>
3168        <bridgehead renderas="sect3">Name</bridgehead>
3169        <para>MAKE-SOCKET &mdash;</para>
3170        <para>Function</para>
3171        <bridgehead renderas="sect3">Synopsis</bridgehead>
3172        <programlisting>
3173make-socket
3174          &amp;key address-family type connect eol format
3175          remote-host remote-port local-host local-port local-filename
3176          remote-filename keepalive reuse-address nodelay broadcast linger
3177          backlog
3178</programlisting>
3179        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3180        <term><indexterm>address-family
3181            <variablelist>The address/protocol family of this socket. Currentlyonly :internet (the default), meaning IP, and :file,referring to UNIX domain addresses, are supported.</variablelist>
3182          </indexterm><indexterm>type
3183            <variablelist>One of :stream (the default) to request aconnection-oriented socket, or :datagram to request aconnectionless socket. The default is :stream.</variablelist>
3184          </indexterm><indexterm>connect
3185            <variablelist>This argument is only relevant to sockets of type:stream. One of :active (the default) to request a :passiveto request a file or TCP listener socket.</variablelist>
3186          </indexterm><indexterm>eol
3187            <variablelist>This argument is currently ignored (it is accepted forcompatibility with Franz Allegro).</variablelist>
3188          </indexterm><indexterm>format
3189            <variablelist>One of :text (the default), :binary, or :bivalent.This argument is ignored for :stream sockets for now, as:stream sockets are currently always bivalent (i.e. theysupport both character and byte I/O). For :datagram sockets,the format determines the type of buffer created byreceive-from.</variablelist>
3190          </indexterm><indexterm>remote-host
3191            <variablelist>Required for TCP streams, it specifies the host toconnect to (in any format acceptable to lookup-hostname).Ignored for listener sockets. For UDP sockets, it can beused to specify a default host for subsequent calls tosend-to or receive-from.</variablelist>
3192          </indexterm><indexterm>remote-port
3193            <variablelist>Required for TCP streams, it specifies the port toconnect to (in any format acceptable to lookup-port).Ignored for listener sockets. For UDP sockets, it can beused to specify a default port for subsequent calls to forsubsequent calls to send-to or receive-from.</variablelist>
3194          </indexterm><indexterm>remote-filename
3195            <variablelist>Required for file-socket streams, it specifies thename of a file in the local filesystem (e.g., NOT mountedvia NFS, AFP, SMB, ...) which names and controls access to aUNIX-domain socket.</variablelist>
3196          </indexterm><indexterm>local-host
3197            <variablelist>Allows you to specify a local host address for alistener or UDP socket, for the rare case where you want torestrict connections to those coming to a specific localaddress for security reasons.</variablelist>
3198          </indexterm><indexterm>local-port
3199            <variablelist>Specify a local port for a socket. Most useful forlistener sockets, where it is the port on which the socketwill listen for connections.</variablelist>
3200          </indexterm><indexterm>local-filename
3201            <variablelist>Required for file-listener-sockets. Specifies the nameof a file in the local filesystem which is used to name aUNIX-domain socket. The actual filesystem file should notpreviously exist when the file-listener-socket is created;its parent directory should exist and be writable by thecaller. The file used to name the socket will be deletedwhen the file-listener-socket is closed.</variablelist>
3202          </indexterm><indexterm>keepalive
3203            <variablelist>If true, enables the periodic transmission of"keepalive" messages.</variablelist>
3204          </indexterm><indexterm>reuse-address
3205            <variablelist>If true, allows the reuse of local ports in listenersockets, overriding some TCP/IP protocol specifications. Youwill need this if you are debugging a server..</variablelist>
3206          </indexterm><indexterm>nodelay
3207            <variablelist>If true, disables Nagle's algorithm, which triesto minimize TCP packet fragmentation by introducingtransmission delays in the absence of replies. Try settingthis if you are using a protocol which involves sending asteady stream of data with no replies and are seeingsignificant degradations in throughput.</variablelist>
3208          </indexterm><indexterm>broadcast
3209            <variablelist>If true, requests permission to broadcast datagrams ona UDP socket.</variablelist>
3210          </indexterm><indexterm>linger
3211            <variablelist>If specified and non-nil, should be the number ofseconds the OS is allowed to wait for data to be pushedthrough when a close is done. Only relevant for TCP sockets.</variablelist>
3212          </indexterm><indexterm>backlog
3213            <variablelist>For a listener socket, specifies the number ofconnections which can be pending but not accepted. Thedefault is 5, which is also the maximum on some operatingsystems.</variablelist>
3214          </indexterm>
3215        </term>
3216        <bridgehead renderas="sect3">Description</bridgehead>
3217        <para>Creates and returns a new socket</para>
3218      </sect2>
3219
3220      <sect2 id="ACCEPT-CONNECTION">
3221        <para>ACCEPT-CONNECTION</para>
3222        <informalfigure>accept-connection</informalfigure>
3223        <bridgehead renderas="sect3">Name</bridgehead>
3224        <para>ACCEPT-CONNECTION &mdash;</para>
3225        <para>Function</para>
3226        <bridgehead renderas="sect3">Synopsis</bridgehead>
3227        <programlisting>
3228accept-connection
3229          (socket listener-socket) &amp;key wait
3230</programlisting>
3231        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3232        <term><indexterm>socket
3233            <variablelist>The listener-socket to listen on.</variablelist>
3234          </indexterm><indexterm>wait
3235            <variablelist>If true (the default), and there are no connectionswaiting to be accepted, waits until one arrives. If false,returns NIL immediately.</variablelist>
3236          </indexterm>
3237        </term>
3238        <bridgehead renderas="sect3">Description</bridgehead>
3239        <para>Extracts the first connection on the queue of pending
3240connections, accepts it (i.e. completes the connection startup
3241protocol) and returns a new tcp-stream or file-socket-stream
3242representing the newly established connection. The tcp stream
3243inherits any properties of the listener socket that are relevant
3244(e.g. :keepalive, :nodelay, etc.) The original listener socket
3245continues to be open listening for more connections, so you can
3246call accept-connection on it again.</para>
3247      </sect2>
3248
3249      <sect2 id="DOTTED-TO-IPADDR">
3250        <para>DOTTED-TO-IPADDR</para>
3251        <informalfigure>dotted-to-ipaddr</informalfigure>
3252        <bridgehead renderas="sect3">Name</bridgehead>
3253        <para>DOTTED-TO-IPADDR &mdash;</para>
3254        <para>Function</para>
3255        <bridgehead renderas="sect3">Synopsis</bridgehead>
3256        <programlisting>
3257dotted-to-ipaddr
3258          dotted &amp;key errorp
3259</programlisting>
3260        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3261        <term><indexterm>dotted
3262            <variablelist>A string representing an IP address in the"nn.nn.nn.nn" format</variablelist>
3263          </indexterm><indexterm>errorp
3264            <variablelist>If true (the default) an error is signaled if dottedis invalid. If false, NIL is returned.</variablelist>
3265          </indexterm>
3266        </term>
3267        <bridgehead renderas="sect3">Description</bridgehead>
3268        <para>Converts a dotted-string representation of a host address to
3269a 32-bit unsigned IP address.</para>
3270      </sect2>
3271
3272      <sect2 id="IPADDR-TO-DOTTED">
3273        <para>IPADDR-TO-DOTTED</para>
3274        <informalfigure>ipaddr-to-dotted</informalfigure>
3275        <bridgehead renderas="sect3">Name</bridgehead>
3276        <para>IPADDR-TO-DOTTED &mdash;</para>
3277        <para>Function</para>
3278        <bridgehead renderas="sect3">Synopsis</bridgehead>
3279        <programlisting>
3280ipaddr-to-dotted
3281          ipaddr &amp;key values
3282</programlisting>
3283        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3284        <term><indexterm>ipaddr
3285            <variablelist>A 32-bit integer representing an internet host address</variablelist>
3286          </indexterm><indexterm>values
3287            <variablelist>If false (the default), returns a string in the form"nn.nn.nn.nn". If true, returns four valuesrepresenting the four octets of the address as unsigned8-bit integers.</variablelist>
3288          </indexterm>
3289        </term>
3290        <bridgehead renderas="sect3">Description</bridgehead>
3291        <para>Converts a 32-bit unsigned IP address into octets.</para>
3292      </sect2>
3293
3294      <sect2 id="IPADDR-TO-HOSTNAME">
3295        <para>IPADDR-TO-HOSTNAME</para>
3296        <informalfigure>ipaddr-to-hostname</informalfigure>
3297        <bridgehead renderas="sect3">Name</bridgehead>
3298        <para>IPADDR-TO-HOSTNAME &mdash;</para>
3299        <para>Function</para>
3300        <bridgehead renderas="sect3">Synopsis</bridgehead>
3301        <programlisting>
3302ipaddr-to-hostname
3303          ipaddr &amp;key ignore-cache
3304</programlisting>
3305        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3306        <term><indexterm>ipaddr
3307            <variablelist>a 32-bit integer representing an internet host address</variablelist>
3308          </indexterm><indexterm>ignore-cache
3309            <variablelist>This argument is ignored (it is accepted forcompatibility with Franz Allegro)</variablelist>
3310          </indexterm>
3311        </term>
3312        <bridgehead renderas="sect3">Description</bridgehead>
3313        <para>Converts a 32-bit unsigned IP address into a host name
3314string</para>
3315      </sect2>
3316
3317      <sect2 id="LOOKUP-HOSTNAME">
3318        <para>LOOKUP-HOSTNAME</para>
3319        <informalfigure>lookup-hostname</informalfigure>
3320        <bridgehead renderas="sect3">Name</bridgehead>
3321        <para>LOOKUP-HOSTNAME &mdash;</para>
3322        <para>Function</para>
3323        <bridgehead renderas="sect3">Synopsis</bridgehead>
3324        <programlisting>
3325lookup-hostname
3326          host
3327</programlisting>
3328        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3329        <term><indexterm>host
3330            <variablelist>Specifies the host. It can be either a host namestring such as "clozure.com", or a dotted addressstring such as "192.168.0.1", or a 32-bit unsignedIP address such as 3232235521.</variablelist>
3331          </indexterm>
3332        </term>
3333        <bridgehead renderas="sect3">Description</bridgehead>
3334        <para>Converts a host spec in any of the acceptable formats into a
333532-bit unsigned IP address</para>
3336      </sect2>
3337
3338      <sect2 id="LOOKUP-PORT">
3339        <para>LOOKUP-PORT</para>
3340        <informalfigure>lookup-port</informalfigure>
3341        <bridgehead renderas="sect3">Name</bridgehead>
3342        <para>LOOKUP-PORT &mdash;</para>
3343        <para>Function</para>
3344        <bridgehead renderas="sect3">Synopsis</bridgehead>
3345        <programlisting>
3346lookup-port
3347          port protocol
3348</programlisting>
3349        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3350        <term><indexterm>port
3351            <variablelist>Specifies the port. It can be either a string, such as"http" or a symbol, such as :http, or an unsignedport number. Note that a string is case-sensitive. A symbolis lowercased before lookup.</variablelist>
3352          </indexterm><indexterm>protocol
3353            <variablelist>Must be one of "tcp" or "udp".</variablelist>
3354          </indexterm>
3355        </term>
3356        <bridgehead renderas="sect3">Description</bridgehead>
3357        <para>Finds the port number for the specified port and protocol</para>
3358      </sect2>
3359
3360      <sect2 id="RECEIVE-FROM">
3361        <para>RECEIVE-FROM</para>
3362        <informalfigure>receive-from</informalfigure>
3363        <bridgehead renderas="sect3">Name</bridgehead>
3364        <para>RECEIVE-FROM &mdash;</para>
3365        <para>Function</para>
3366        <bridgehead renderas="sect3">Synopsis</bridgehead>
3367        <programlisting>
3368receive-from
3369          (socket udp-socket) size &amp;key buffer
3370          extract offset
3371</programlisting>
3372        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3373        <term><indexterm>socket
3374            <variablelist>The socket to read from</variablelist>
3375          </indexterm><indexterm>size
3376            <variablelist>Maximum number of bytes to read. If the packet islarger than this, any extra bytes are discarded.</variablelist>
3377          </indexterm><indexterm>buffer
3378            <variablelist>If specified, must be either a string or a byte vectorwhich will be used to read in the data. If not specified, anew buffer will be created (of type determined bysocket-format).</variablelist>
3379          </indexterm><indexterm>extract
3380            <variablelist>If true, the subsequence of the buffer correspondingonly to the data read in is extracted and returned as thefirst value. If false (the default) the original buffer isreturned even if it is only partially filled.</variablelist>
3381          </indexterm><indexterm>offset
3382            <variablelist>Specifies the start offset into the buffer at whichdata is to be stored. The default is 0.</variablelist>
3383          </indexterm>
3384        </term>
3385        <bridgehead renderas="sect3">Description</bridgehead>
3386        <para>Reads a UDP packet from a socket. If no packets are
3387available, waits for a packet to arrive. Returns four values:</para>
3388        <varlistentry numeration="arabic">
3389          <variablelist>The buffer with the data</variablelist>
3390          <variablelist>The number of bytes read</variablelist>
3391          <variablelist>The 32-bit unsigned IP address of the sender of the data</variablelist>
3392          <variablelist>The port number of the sender of the data</variablelist>
3393        </varlistentry>
3394      </sect2>
3395
3396      <sect2 id="SEND-TO">
3397        <para>SEND-TO</para>
3398        <informalfigure>send-to</informalfigure>
3399        <bridgehead renderas="sect3">Name</bridgehead>
3400        <para>SEND-TO &mdash;</para>
3401        <para>Function</para>
3402        <bridgehead renderas="sect3">Synopsis</bridgehead>
3403        <programlisting>
3404send-to
3405          (socket udp-socket) buffer size &amp;key remote-host
3406          remote-port offset
3407</programlisting>
3408        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3409        <term><indexterm>socket
3410            <variablelist>The socket to write to</variablelist>
3411          </indexterm><indexterm>buffer
3412            <variablelist>A vector containing the data to send. It must beeither a string or a byte vector (either one is acceptableregardless of the stream format).</variablelist>
3413          </indexterm><indexterm>size
3414            <variablelist>Number of bytes to send</variablelist>
3415          </indexterm><indexterm>remote-host
3416            <variablelist>The host to send the packet to, in any formatacceptable to lookup-hostname. The default is the remotehost specified in the call to make-socket.</variablelist>
3417          </indexterm><indexterm>remote-port
3418            <variablelist>The port to send the packet to, in any formatacceptable to lookup-port. The default is the remote portspecified in the call to make-socket.</variablelist>
3419          </indexterm><indexterm>offset
3420            <variablelist>The offset in the buffer where the packet data starts</variablelist>
3421          </indexterm>
3422        </term>
3423        <bridgehead renderas="sect3">Description</bridgehead>
3424        <para>Send a UDP packet over a socket.</para>
3425      </sect2>
3426
3427      <sect2 id="SHUTDOWN">
3428        <para>SHUTDOWN</para>
3429        <informalfigure>shutdown</informalfigure>
3430        <bridgehead renderas="sect3">Name</bridgehead>
3431        <para>SHUTDOWN &mdash;</para>
3432        <para>Function</para>
3433        <bridgehead renderas="sect3">Synopsis</bridgehead>
3434        <programlisting>
3435shutdown
3436          socket &amp;key direction
3437</programlisting>
3438        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3439        <term><indexterm>socket
3440            <variablelist>The socket to shut down (typically a tcp-stream)</variablelist>
3441          </indexterm><indexterm>direction
3442            <variablelist>One of :input to disallow further input, or :output todisallow further output.</variablelist>
3443          </indexterm>
3444        </term>
3445        <bridgehead renderas="sect3">Description</bridgehead>
3446        <para>Shuts down part of a bidirectional connection. This is
3447useful if e.g. you need to read responses after sending an
3448end-of-file signal.</para>
3449      </sect2>
3450
3451      <sect2 id="SOCKET-OS-FD">
3452        <para>SOCKET-OS-FD</para>
3453        <informalfigure>socket-os-fd</informalfigure>
3454        <bridgehead renderas="sect3">Name</bridgehead>
3455        <para>SOCKET-OS-FD &mdash;</para>
3456        <para>Function</para>
3457        <bridgehead renderas="sect3">Synopsis</bridgehead>
3458        <programlisting>
3459socket-os-fd
3460          socket
3461</programlisting>
3462        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3463        <term><indexterm>socket
3464            <variablelist>The socket</variablelist>
3465          </indexterm>
3466        </term>
3467        <bridgehead renderas="sect3">Description</bridgehead>
3468        <para>Returns the native OS's representation of the socket, or
3469NIL if the socket is closed. On Unix, this is the Unix 'file
3470descriptor', a small non-negative integer. Note that it is
3471rather dangerous to mess around with tcp-stream fd's, as there
3472is all sorts of buffering and asynchronous I/O going on above the
3473OS level. listener-socket and udp-socket fd's are safer to
3474mess with directly as there is less magic going on.</para>
3475      </sect2>
3476
3477      <sect2 id="REMOTE-HOST">
3478        <para>REMOTE-HOST</para>
3479        <informalfigure>remote-host</informalfigure>
3480        <bridgehead renderas="sect3">Name</bridgehead>
3481        <para>REMOTE-HOST &mdash;</para>
3482        <para>Function</para>
3483        <bridgehead renderas="sect3">Synopsis</bridgehead>
3484        <programlisting>
3485remote-host
3486          socket
3487</programlisting>
3488        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3489        <term><indexterm>socket
3490            <variablelist>The socket</variablelist>
3491          </indexterm>
3492        </term>
3493        <bridgehead renderas="sect3">Description</bridgehead>
3494        <para>Returns the 32-bit unsigned IP address of the remote host,
3495or NIL if the socket is not connected.</para>
3496      </sect2>
3497
3498      <sect2 id="REMOTE-PORT">
3499        <para>REMOTE-PORT</para>
3500        <informalfigure>remote-port</informalfigure>
3501        <bridgehead renderas="sect3">Name</bridgehead>
3502        <para>REMOTE-PORT &mdash;</para>
3503        <para>Function</para>
3504        <bridgehead renderas="sect3">Synopsis</bridgehead>
3505        <programlisting>
3506remote-port
3507          socket
3508</programlisting>
3509        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3510        <term><indexterm>socket
3511            <variablelist>The socket</variablelist>
3512          </indexterm>
3513        </term>
3514        <bridgehead renderas="sect3">Description</bridgehead>
3515        <para>Returns the remote port number, or NIL if the socket is not
3516connected.</para>
3517      </sect2>
3518
3519      <sect2 id="LOCAL-HOST">
3520        <para>LOCAL-HOST</para>
3521        <informalfigure>local-host</informalfigure>
3522        <bridgehead renderas="sect3">Name</bridgehead>
3523        <para>LOCAL-HOST &mdash;</para>
3524        <para>Function</para>
3525        <bridgehead renderas="sect3">Synopsis</bridgehead>
3526        <programlisting>
3527local-host
3528          socket
3529</programlisting>
3530        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3531        <term><indexterm>socket
3532            <variablelist>The socket</variablelist>
3533          </indexterm>
3534        </term>
3535        <bridgehead renderas="sect3">Description</bridgehead>
3536        <para>Returns 32-bit unsigned IP address of the local host.</para>
3537      </sect2>
3538
3539      <sect2 id="LOCAL-PORT">
3540        <para>LOCAL-PORT</para>
3541        <informalfigure>local-port</informalfigure>
3542        <bridgehead renderas="sect3">Name</bridgehead>
3543        <para>LOCAL-PORT &mdash;</para>
3544        <para>Function</para>
3545        <bridgehead renderas="sect3">Synopsis</bridgehead>
3546        <programlisting>
3547local-port
3548          socket
3549</programlisting>
3550        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3551        <term><indexterm>socket
3552            <variablelist>The socket</variablelist>
3553          </indexterm>
3554        </term>
3555        <bridgehead renderas="sect3">Description</bridgehead>
3556        <para>Returns the local port number</para>
3557      </sect2>
3558
3559      <sect2 id="SOCKET-ADDRESS-FAMILY">
3560        <para>SOCKET-ADDRESS-FAMILY</para>
3561        <informalfigure>socket-address-family</informalfigure>
3562        <bridgehead renderas="sect3">Name</bridgehead>
3563        <para>SOCKET-ADDRESS-FAMILY &mdash;</para>
3564        <para>Function</para>
3565        <bridgehead renderas="sect3">Synopsis</bridgehead>
3566        <programlisting>
3567socket-address-family
3568          socket
3569</programlisting>
3570        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3571        <term><indexterm>socket
3572            <variablelist>The socket</variablelist>
3573          </indexterm>
3574        </term>
3575        <bridgehead renderas="sect3">Description</bridgehead>
3576        <para>Returns :internet or :file, as appropriate.</para>
3577      </sect2>
3578
3579      <sect2 id="SOCKET-CONNECT">
3580        <para>SOCKET-CONNECT</para>
3581        <informalfigure>socket-connect</informalfigure>
3582        <bridgehead renderas="sect3">Name</bridgehead>
3583        <para>SOCKET-CONNECT &mdash;</para>
3584        <para>Function</para>
3585        <bridgehead renderas="sect3">Synopsis</bridgehead>
3586        <programlisting>
3587socket-connect
3588          socket
3589</programlisting>
3590        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3591        <term><indexterm>socket
3592            <variablelist>The socket</variablelist>
3593          </indexterm>
3594        </term>
3595        <bridgehead renderas="sect3">Description</bridgehead>
3596        <para>Returns :active for tcp-stream, :passive for
3597listener-socket, and NIL for udp-socket</para>
3598      </sect2>
3599
3600      <sect2 id="SOCKET-FORMAT">
3601        <para>SOCKET-FORMAT</para>
3602        <informalfigure>socket-format</informalfigure>
3603        <bridgehead renderas="sect3">Name</bridgehead>
3604        <para>SOCKET-FORMAT &mdash;</para>
3605        <para>Function</para>
3606        <bridgehead renderas="sect3">Synopsis</bridgehead>
3607        <programlisting>
3608socket-format
3609          socket
3610</programlisting>
3611        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3612        <term><indexterm>socket
3613            <variablelist>The socket</variablelist>
3614          </indexterm>
3615        </term>
3616        <bridgehead renderas="sect3">Description</bridgehead>
3617        <para>Returns the socket format as specified by the :format
3618argument to make-socket.</para>
3619      </sect2>
3620
3621      <sect2 id="SOCKET-TYPE">
3622        <para>SOCKET-TYPE</para>
3623        <informalfigure>socket-type</informalfigure>
3624        <bridgehead renderas="sect3">Name</bridgehead>
3625        <para>SOCKET-TYPE &mdash;</para>
3626        <para>Function</para>
3627        <bridgehead renderas="sect3">Synopsis</bridgehead>
3628        <programlisting>
3629socket-type
3630          socket
3631</programlisting>
3632        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3633        <term><indexterm>socket
3634            <variablelist>The socket</variablelist>
3635          </indexterm>
3636        </term>
3637        <bridgehead renderas="sect3">Description</bridgehead>
3638        <para>returns :stream for tcp-stream and listener-socket, and
3639:datagram for udp-socket.</para>
3640      </sect2>
3641
3642      <sect2 id="SOCKET-ERROR">
3643        <para>SOCKET-ERROR</para>
3644        <informalfigure>socket-error</informalfigure>
3645        <bridgehead renderas="sect3">Name</bridgehead>
3646        <para>SOCKET-ERROR &mdash;</para>
3647        <para>Class</para>
3648        <bridgehead renderas="sect3">Description</bridgehead>
3649        <para>The class of OS errors signaled by socket functions</para>
3650        <bridgehead renderas="sect3">Superclasses</bridgehead>
3651        <para>simple-error</para>
3652      </sect2>
3653
3654      <sect2 id="SOCKET-ERROR-CODE">
3655        <para>SOCKET-ERROR-CODE</para>
3656        <informalfigure>socket-error-code</informalfigure>
3657        <bridgehead renderas="sect3">Name</bridgehead>
3658        <para>SOCKET-ERROR-CODE &mdash;</para>
3659        <para>Function</para>
3660        <bridgehead renderas="sect3">Synopsis</bridgehead>
3661        <programlisting>
3662socket-error-code
3663          socket-error
3664</programlisting>
3665        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3666        <term><indexterm>socket-error
3667            <variablelist>the condition</variablelist>
3668          </indexterm>
3669        </term>
3670        <bridgehead renderas="sect3">Description</bridgehead>
3671        <para>The OS error code of the error</para>
3672      </sect2>
3673
3674      <sect2 id="SOCKET-ERROR-IDENTIFIER">
3675        <para>SOCKET-ERROR-IDENTIFIER</para>
3676        <informalfigure>socket-error-identifier</informalfigure>
3677        <bridgehead renderas="sect3">Name</bridgehead>
3678        <para>SOCKET-ERROR-IDENTIFIER &mdash;</para>
3679        <para>Function</para>
3680        <bridgehead renderas="sect3">Synopsis</bridgehead>
3681        <programlisting>
3682socket-error-identifier
3683          socket-error
3684</programlisting>
3685        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3686        <term><indexterm>socket-error
3687            <variablelist>the condition</variablelist>
3688          </indexterm>
3689        </term>
3690        <bridgehead renderas="sect3">Description</bridgehead>
3691        <para>A symbol representing the error code in a more
3692OS-independent way.</para>
3693        <para>One of: :address-in-use :connection-aborted :no-buffer-space
3694:connection-timed-out :connection-refused :host-unreachable
3695:host-down :network-down :address-not-available :network-reset
3696:connection-reset :shutdown :access-denied or :unknown.</para>
3697      </sect2>
3698
3699      <sect2 id="SOCKET-ERROR-SITUATION">
3700        <para>SOCKET-ERROR-SITUATION</para>
3701        <informalfigure>socket-error-situation</informalfigure>
3702        <bridgehead renderas="sect3">Name</bridgehead>
3703        <para>SOCKET-ERROR-SITUATION &mdash;</para>
3704        <para>Function</para>
3705        <bridgehead renderas="sect3">Synopsis</bridgehead>
3706        <programlisting>
3707socket-error-situation
3708          socket-error
3709</programlisting>
3710        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3711        <term><indexterm>socket-error
3712            <variablelist>the condition</variablelist>
3713          </indexterm>
3714        </term>
3715        <bridgehead renderas="sect3">Description</bridgehead>
3716        <para>A string describing the context where the error happened. On
3717Linux, this is the name of the system call which returned the
3718error.</para>
3719      </sect2>
3720
3721      <sect2 id="CLOSE">
3722        <para>CLOSE</para>
3723        <informalfigure>close</informalfigure>
3724        <bridgehead renderas="sect3">Name</bridgehead>
3725        <para>CLOSE &mdash;</para>
3726        <para>Method</para>
3727        <bridgehead renderas="sect3">Synopsis</bridgehead>
3728        <programlisting>
3729close
3730          (socket socket) &amp;key abort
3731</programlisting>
3732        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3733        <term><indexterm>socket
3734            <variablelist>The socket to close</variablelist>
3735          </indexterm><indexterm>abort
3736            <variablelist>If false (the default), closes the socket in anorderly fashion, finishing up any buffered pending I/O,before closing the connection. If true, aborts/ignorespending I/O. (For listener and udp sockets, this argument iseffectively ignored since there is never any buffered I/O toclean up).</variablelist>
3737          </indexterm>
3738        </term>
3739        <bridgehead renderas="sect3">Description</bridgehead>
3740        <para>The close generic function can be applied to sockets. It
3741releases the operating system resources associated with the
3742socket.</para>
3743      </sect2>
3744
3745      <sect2 id="WITH-OPEN-SOCKET">
3746        <para>WITH-OPEN-SOCKET</para>
3747        <informalfigure>with-open-socket</informalfigure>
3748        <bridgehead renderas="sect3">Name</bridgehead>
3749        <para>WITH-OPEN-SOCKET &mdash;</para>
3750        <para>Macro</para>
3751        <bridgehead renderas="sect3">Synopsis</bridgehead>
3752        <programlisting>
3753with-open-socket
3754          (var . make-socket-args) &amp;body body
3755</programlisting>
3756        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3757        <term><indexterm>var
3758            <variablelist>variable to bind</variablelist>
3759          </indexterm><indexterm>make-socket-args
3760            <variablelist>arguments suitable for passing to make-socket</variablelist>
3761          </indexterm><indexterm>body
3762            <variablelist>body to execute</variablelist>
3763          </indexterm>
3764        </term>
3765        <bridgehead renderas="sect3">Description</bridgehead>
3766        <para>executes body with var bound to the result of applying
3767make-socket to make-socket-args. The socket gets closed on exit.</para>
3768      </sect2>
3769    </sect1>
3770  </chapter>
3771
3772  <chapter id="Running-Other-Programs-as-Subprocesses">
3773    <title>Running Other Programs as Subprocesses</title>
3774
3775    <sect1 id="Subprocess-Overview">
3776      <title>Overview</title>
3777      <para>OpenMCL provides primitives to run external Unix programs,
3778      to select and connect Lisp streams to their input and output
3779      sources, to (optionally) wait for their completion and to check
3780      their execution and exit status.</para>
3781      <para>All of the global symbols described below are exported
3782      from the CCL package.</para>
3783      <para>This implementation is modeled on - and uses some code
3784      from - similar facilities in CMUCL.</para>
3785    </sect1>
3786
3787    <sect1 id="Subprocess-Examples">
3788      <title>Examples</title>
3789      <programlisting>
3790;;; Capture the output of the "uname" program in a lisp string-stream
3791;;; and return the generated string (which will contain a trailing
3792;;; newline.)
3793? (with-output-to-string (stream)
3794    (run-program "uname" '("-r") :output stream))
3795;;; Write a string to *STANDARD-OUTPUT*, the hard way.
3796? (run-program "cat" () :input (make-string-input-stream "hello") :output t)
3797;;; Find out that "ls" doesn't expand wildcards.
3798? (run-program "ls" '("*.lisp") :output t)
3799;;; Let the shell expand wildcards.
3800? (run-program "sh" '("-c" "ls *.lisp") :output t)
3801</programlisting>
3802      <para>These last examples will only produce output if OpenMCL's
3803      current directory contains .lisp files, of course.</para>
3804    </sect1>
3805
3806    <sect1 id="Limitations-and-known-bugs">
3807      <para>Limitations and known bugs</para>
3808      <itemizedlist>
3809        <listitem><para>OpenMCL and the external processs may get
3810        confused about whoowns which streams when input, output, or
3811        error are specified as T and wait is specified as
3812        NIL.</para></listitem>
3813        <listitem><para>External processes that need to talk to a
3814        terminal device may not work properly; the environment (SLIME,
3815        ILISP) under which OpenMCL is run can affect
3816        this.</para></listitem>
3817     
3818      </itemizedlist>
3819    </sect1>
3820
3821    <sect1 id="External-Program-Dictionary">
3822      <title>External-Program Dictionary</title>
3823
3824      <sect2 id="RUN-PROGRAM">
3825        <para>RUN-PROGRAM</para>
3826        <informalfigure>run-program</informalfigure>
3827        <bridgehead renderas="sect3">Name</bridgehead>
3828        <para>RUN-PROGRAM &mdash; Invokes an external program as an OS subprocess
3829of lisp.</para>
3830        <para>Function</para>
3831        <bridgehead renderas="sect3">Synopsis</bridgehead>
3832        <programlisting>
3833run-program
3834            program args &amp;key (wait t) pty input
3835            if-input-does-not-exist output (if-output-exists :error) (error
3836            :output) (if-error-exists :error) status-hook
3837</programlisting>
3838        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3839        <term><indexterm>program
3840            <variablelist>A string or pathname which denotes an executable file.The PATH environment variable is used to find programs whosename doesn't contain a directory component.</variablelist>
3841          </indexterm><indexterm>args
3842            <variablelist>A list of simple-strings</variablelist>
3843          </indexterm><indexterm>wait
3844            <variablelist>Indicates whether or not run-program should wait forthe EXTERNAL-PROCESS to complete or should returnimmediately.</variablelist>
3845          </indexterm><indexterm>pty
3846            <variablelist>This option is accepted but currently ignored;it's intended to make it easier to run external programsthat need to interact with a terminal device.</variablelist>
3847          </indexterm><indexterm>input
3848            <variablelist>Selects the input source used by the EXTERNAL-PROCESS.May be any of the following:
3849              <listitem mark="bullet">
3850                <variablelist>NIL Specifies that a null input stream (e.g.,/dev/null) should be used.</variablelist>
3851                <variablelist>T Specifies that the EXTERNAL-PROCESS should usethe input source with which OpenMCL was invoked.</variablelist>
3852                <variablelist>A string or pathname. Specifies that theEXTERNAL-PROCESS should receive its input from the namedexisting file.</variablelist>
3853                <variablelist>:STREAM Creates a Lisp stream opened for characteroutput. Any data written to this stream (accessible asthe EXTERNAL-PROCESS-INPUT-STREAM of theEXTERNAL-PROCESS object) appears as input to theexternal process.</variablelist>
3854                <variablelist>A stream. Specifies that the lisp stream shouldprovide input to the EXTERNAL-PROCESS.</variablelist>
3855             
3856              </listitem>
3857</variablelist>
3858          </indexterm><indexterm>if-input-does-not-exist
3859            <variablelist>If the input argument specifies the name of anexisting file, this argument is used as theif-does-not-exist argument to OPEN when that file is opened.</variablelist>
3860          </indexterm><indexterm>output
3861            <variablelist>Specifies where standard output from the externalprocess should be sent. Analogous to input above.</variablelist>
3862          </indexterm><indexterm>if-output-exists
3863            <variablelist>If output is specified as a string or pathname, thisargument is used as the if-exists argument to OPEN when thatfile is opened.</variablelist>
3864          </indexterm><indexterm>error
3865            <variablelist>Specifies where error output from the external processshould be sent. In addition to the values allowed foroutput, the keyword :OUTPUT can be used to indicate thaterror output should be sent where standard output goes.</variablelist>
3866          </indexterm><indexterm>if-error-exists
3867            <variablelist>Analogous to if-output-exists.</variablelist>
3868          </indexterm><indexterm>status-hook
3869            <variablelist>A user-defined function of one argument (theEXTERNAL-PROCESS structure.) This function is calledwhenever OpenMCL detects a change in the staus of theEXTERNAL-PROCESS.</variablelist>
3870          </indexterm>
3871        </term>
3872        <bridgehead renderas="sect3">Description</bridgehead>
3873        <para>Runs the specified program in an external (Unix) process,
3874returning an object of type EXTERNAL-PROCESS if successful.</para>
3875      </sect2>
3876
3877      <sect2 id="SIGNAL-EXTERNAL-PROCESS">
3878        <para>SIGNAL-EXTERNAL-PROCESS</para>
3879        <informalfigure>signal-external-process</informalfigure>
3880        <bridgehead renderas="sect3">Name</bridgehead>
3881        <para>SIGNAL-EXTERNAL-PROCESS &mdash;</para>
3882        <para>Function</para>
3883        <bridgehead renderas="sect3">Synopsis</bridgehead>
3884        <programlisting>
3885signal-external-process
3886            proc signal-number
3887</programlisting>
3888        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3889        <term><indexterm>proc
3890            <variablelist>An EXTERNAL-PROCESS, as returned by RUN-PROGRAM.</variablelist>
3891          </indexterm><indexterm>signal
3892            <variablelist>A small integer.</variablelist>
3893          </indexterm>
3894        </term>
3895        <bridgehead renderas="sect3">Description</bridgehead>
3896        <para>Sends the specified "signal" to the specified
3897external process. (Typically, it would only be useful tocall
3898this function if the EXTERNAL-PROCESS was created with :WAIT
3899NIL. ) Returns T if successful; signals an error otherwise.</para>
3900      </sect2>
3901
3902      <sect2 id="EXTERNAL-PROCESS-ID">
3903        <para>EXTERNAL-PROCESS-ID</para>
3904        <informalfigure>external-process-id</informalfigure>
3905        <bridgehead renderas="sect3">Name</bridgehead>
3906        <para>EXTERNAL-PROCESS-ID &mdash; Returns the "process ID" of an OS subprocess,
3907a positive integer which identifies it.</para>
3908        <para>Function</para>
3909        <bridgehead renderas="sect3">Synopsis</bridgehead>
3910        <programlisting>
3911external-process-id
3912            proc
3913</programlisting>
3914        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3915        <term><indexterm>proc
3916            <variablelist>An EXTERNAL-PROCESS, as returned by RUN-PROGRAM.</variablelist>
3917          </indexterm>
3918        </term>
3919        <bridgehead renderas="sect3">Description</bridgehead>
3920        <para>Returns the <emphasis>process id</emphasis> assigned to
3921the external process by the operating system. This is typically
3922a positive, 16-bit number.</para>
3923      </sect2>
3924
3925      <sect2 id="EXTERNAL-PROCESS-INPUT-STREAM">
3926        <para>EXTERNAL-PROCESS-INPUT-STREAM</para>
3927        <informalfigure>external-process-input-stream</informalfigure>
3928        <bridgehead renderas="sect3">Name</bridgehead>
3929        <para>EXTERNAL-PROCESS-INPUT-STREAM &mdash; Returns the lisp stream which is used to write
3930input to a given OS subprocess, if it has one.</para>
3931        <para>Function</para>
3932        <bridgehead renderas="sect3">Synopsis</bridgehead>
3933        <programlisting>
3934external-process-input-stream
3935            proc
3936</programlisting>
3937        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3938        <term><indexterm>proc
3939            <variablelist>An EXTERNAL-PROCESS, as returned by RUN-PROGRAM.</variablelist>
3940          </indexterm>
3941        </term>
3942        <bridgehead renderas="sect3">Description</bridgehead>
3943        <para>Returns the stream created when the input argument to
3944run-program is specified as :STREAM.</para>
3945      </sect2>
3946
3947      <sect2 id="EXTERNAL-PROCESS-OUTPUT-STREAM">
3948        <para>EXTERNAL-PROCESS-OUTPUT-STREAM</para>
3949        <informalfigure>external-process-output-stream</informalfigure>
3950        <bridgehead renderas="sect3">Name</bridgehead>
3951        <para>EXTERNAL-PROCESS-OUTPUT-STREAM &mdash; Returns the lisp stream which is used to read
3952output from an OS subprocess, if there is one.</para>
3953        <para>Function</para>
3954        <bridgehead renderas="sect3">Synopsis</bridgehead>
3955        <programlisting>
3956external-process-output-stream
3957            proc
3958</programlisting>
3959        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3960        <term><indexterm>proc
3961            <variablelist>An EXTERNAL-PROCESS, as returned by RUN-PROGRAM.</variablelist>
3962          </indexterm>
3963        </term>
3964        <bridgehead renderas="sect3">Description</bridgehead>
3965        <para>Returns the stream created when the output argument to
3966run-program is specified as :STREAM.</para>
3967      </sect2>
3968
3969      <sect2 id="EXTERNAL-PROCESS-ERROR-STREAM">
3970        <para>EXTERNAL-PROCESS-ERROR-STREAM</para>
3971        <informalfigure>external-process-error-stream</informalfigure>
3972        <bridgehead renderas="sect3">Name</bridgehead>
3973        <para>EXTERNAL-PROCESS-ERROR-STREAM &mdash; Returns the stream which is used to read
3974"error" output from a given OS subprocess, if it has
3975one.</para>
3976        <para>Function</para>
3977        <bridgehead renderas="sect3">Synopsis</bridgehead>
3978        <programlisting>
3979external-process-error-stream
3980            proc
3981</programlisting>
3982        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
3983        <term><indexterm>proc
3984            <variablelist>An EXTERNAL-PROCESS, as returned by RUN-PROGRAM.</variablelist>
3985          </indexterm>
3986        </term>
3987        <bridgehead renderas="sect3">Description</bridgehead>
3988        <para>Returns the stream created when the error argument to
3989run-program is specified as :STREAM.</para>
3990      </sect2>
3991
3992      <sect2 id="EXTERNAL-PROCESS-STATUS">
3993        <para>EXTERNAL-PROCESS-STATUS</para>
3994        <informalfigure>external-process-status</informalfigure>
3995        <bridgehead renderas="sect3">Name</bridgehead>
3996        <para>EXTERNAL-PROCESS-STATUS &mdash; Returns information about whether an OS
3997subprocess is running; or, if not, why not; and what its
3998result code was if it completed.</para>
3999        <para>Function</para>
4000        <bridgehead renderas="sect3">Synopsis</bridgehead>
4001        <programlisting>
4002external-process-status
4003            proc
4004</programlisting>
4005        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
4006        <term><indexterm>proc
4007            <variablelist>An EXTERNAL-PROCESS, as returned by RUN-PROGRAM.</variablelist>
4008          </indexterm>
4009        </term>
4010        <bridgehead renderas="sect3">Description</bridgehead>
4011        <para>Returns, as multiple values, a keyword denoting the status
4012of the external process (one of :running, :stopped, :signaled, or
4013:exited), and the exit code or terminating signal if the first
4014value is other than :running.</para>
4015      </sect2>
4016    </sect1>
4017  </chapter>
4018
4019  <chapter id="Creating-Your-Own-Stream-Classes-with-Gray-Streams">
4020    <title>Creating Your Own Stream Classes with Gray Streams</title>
4021
4022    <sect1 id="Streams-Overview">
4023      <title>Overview</title>
4024      <para>This chapter is still being written and revised, because
4025      it is woefully incomplete.  The dictionary section currently
4026      only lists a couple functions.  Caveat lector.</para>
4027      <para>Gray streams are an extension to Common Lisp.  They were
4028      proposed for standardization by David Gray (the astute reader
4029      now understands their name) quite some years ago, but not
4030      accepted, because they had not been tried sufficiently to find
4031      conceptual problems with them.</para>
4032      <para>They have since been implemented by quite a few modern
4033      Lisp implementations.  However, they do indeed have some
4034      inadequacies, and each implementation has addressed these in
4035      different ways.  The situation today is that it's difficult to
4036      even find out how to get started using Gray streams.  This is
4037      why standards are important.</para>
4038      <para>Here's a list of some classes which you might wish for
4039      your new stream class to inherit from:</para>
4040      <colspec>
4041        <thead cols="1">
4042          <tbody colwidth="100*"></tbody>
4043          <row>
4044            <abstract>
4045              <entry>fundamental-stream</entry>
4046           
4047            </abstract>
4048            <abstract>
4049              <entry>fundamental-input-stream</entry>
4050           
4051            </abstract>
4052            <abstract>
4053              <entry>fundamental-output-stream</entry>
4054           
4055            </abstract>
4056            <abstract>
4057              <entry>fundamental-character-stream</entry>
4058           
4059            </abstract>
4060            <abstract>
4061              <entry>fundamental-binary-stream</entry>
4062           
4063            </abstract>
4064            <abstract>
4065              <entry>fundamental-character-input-stream</entry>
4066           
4067            </abstract>
4068            <abstract>
4069              <entry>fundamental-character-output-stream</entry>
4070           
4071            </abstract>
4072            <abstract>
4073              <entry>fundamental-binary-input-stream</entry>
4074           
4075            </abstract>
4076            <abstract>
4077              <entry>fundamental-binary-output-stream</entry>
4078           
4079            </abstract>
4080            <abstract>
4081              <entry>ccl::buffered-stream-mixin</entry>
4082           
4083            </abstract>
4084            <abstract>
4085              <entry>ccl::buffered-input-stream-mixin</entry>
4086           
4087            </abstract>
4088            <abstract>
4089              <entry>ccl::buffered-output-stream-mixin</entry>
4090           
4091            </abstract>
4092            <abstract>
4093              <entry>ccl::buffered-io-stream-mixin</entry>
4094           
4095            </abstract>
4096            <abstract>
4097              <entry>ccl::buffered-character-input-stream-mixin</entry>
4098           
4099            </abstract>
4100            <abstract>
4101              <entry>ccl::buffered-character-output-stream-mixin</entry>
4102           
4103            </abstract>
4104            <abstract>
4105              <entry>ccl::buffered-character-io-stream-mixin</entry>
4106           
4107            </abstract>
4108            <abstract>
4109              <entry>ccl::buffered-binary-input-stream-mixin</entry>
4110           
4111            </abstract>
4112            <abstract>
4113              <entry>ccl::buffered-binary-output-stream-mixin</entry>
4114           
4115            </abstract>
4116            <abstract>
4117              <entry>ccl::buffered-binary-io-stream-mixin</entry>
4118           
4119            </abstract>
4120            <abstract>
4121              <entry>file-stream</entry>
4122           
4123            </abstract>
4124            <abstract>
4125              <entry>file-input-stream</entry>
4126           
4127            </abstract>
4128            <abstract>
4129              <entry>file-output-stream</entry>
4130           
4131            </abstract>
4132            <abstract>
4133              <entry>file-io-stream</entry>
4134           
4135            </abstract>
4136            <abstract>
4137              <entry>file-character-input-stream</entry>
4138           
4139            </abstract>
4140            <abstract>
4141              <entry>file-character-output-stream</entry>
4142           
4143            </abstract>
4144            <abstract>
4145              <entry>file-charcter-io-stream</entry>
4146           
4147            </abstract>
4148            <abstract>
4149              <entry>file-binary-input-stream</entry>
4150           
4151            </abstract>
4152            <abstract>
4153              <entry>file-binary-output-stream</entry>
4154           
4155            </abstract>
4156            <abstract>
4157              <entry>file-binary-io-stream</entry>
4158           
4159            </abstract>
4160            <abstract>
4161              <entry>ccl::fd-stream</entry>
4162           
4163            </abstract>
4164            <abstract>
4165              <entry>ccl::fd-input-stream</entry>
4166           
4167            </abstract>
4168            <abstract>
4169              <entry>ccl::fd-output-stream</entry>
4170           
4171            </abstract>
4172            <abstract>
4173              <entry>ccl::fd-io-stream</entry>
4174           
4175            </abstract>
4176            <abstract>
4177              <entry>ccl::fd-character-input-stream</entry>
4178           
4179            </abstract>
4180            <abstract>
4181              <entry>ccl::fd-character-output-stream</entry>
4182           
4183            </abstract>
4184            <abstract>
4185              <entry>ccl::fd-character-io-stream</entry>
4186           
4187            </abstract>
4188            <abstract>
4189              <entry>ccl::fd-binary-input-stream</entry>
4190           
4191            </abstract>
4192            <abstract>
4193              <entry>ccl::fd-binary-output-stream</entry>
4194           
4195            </abstract>
4196            <abstract>
4197              <entry>ccl::fd-binary-io-stream</entry>
4198           
4199            </abstract>
4200          </row>
4201        </thead>
4202      </colspec>
4203      <para>All of these are defined in ccl/level-1/l1-streams.lisp, except for
4204the ccl:file-* ones, which are in ccl/level-1/l1-sysio.lisp.</para>
4205      <para>According to the original Gray streams proposal, you should inherit
4206from the most specific of the
4207fundamental-* classes which applies.  Using OpenMCL, though, if you
4208want
4209buffering for better performance, which, unless you know of some
4210reason you wouldn't, you do,
4211you should instead inherit from the appropriate ccl::buffered-* class
4212The buffering you get this way is exactly the same as the
4213buffering which is used on ordinary, non-Gray streams, and force-output
4214will work properly on it.</para>
4215      <para>Notice that -mixin suffix in the names of all the ccl::buffered-*
4216classes?
4217The suffix means that this class
4218is not "complete"
4219by itself; you still need to inherit from a fundamental-* stream,
4220even if you also inherit from a *-mixin stream.  You might consider
4221making your own class like this.  ....  Except that they do
4222inherit from the fundamental-* streams, that's weird.</para>
4223      <para>If you want to be able to create an instance of your class with the
4224:class argument to (open) and (with-open-file), you should make it
4225inherit from one of the file-* classes.  If you do this, it's not
4226necessary to inherit from any of the other classes (though it won't
4227hurt anything), since the file-* classes already do.</para>
4228      <para>When you inherit from the file-* classes, you can use
4229(call-next-method) in any of your methods to get the standard
4230behaviour.  This is especially useful if you want to create a class
4231which performs some simple filtering operation, such
4232as changing everything to uppercase or to a different character
4233encoding.  If you do this, you will definitely need to specialize
4234ccl::select-stream-class.  Your method on ccl::stream-select-class
4235should accept an instance of the class, but pay no attention to its
4236contents, and return a symbol naming the
4237class to actually be instantiated.</para>
4238      <para>If you need to make your functionality generic across all the
4239different types of stream, probably the best way
4240to implement it is to make it a mixin, define classes with all the
4241variants of input, output, io, character, and binary, which inherit
4242both from your mixin and from the appropriate other class, then
4243define a method on ccl::select-stream-class which chooses from among
4244those classes.</para>
4245      <para>Note that some of these classes are internal
4246to the CLL package.  If you try to inherit from those ones without
4247the ccl:: prefix, you'll get an error which may confuse you, calling
4248them "forward-referenced classes".  That just means you used the
4249wrong symbol, so add the prefix.</para>
4250      <para>Here's a list of some generic functions which you might wish to
4251specialize for your new stream class, and which ought to be
4252documented at some point.</para>
4253      <colspec>
4254        <thead cols="1">
4255          <tbody colwidth="100*"></tbody>
4256          <row>
4257            <abstract>
4258              <entry>stream-direction stream =></entry>
4259           
4260            </abstract>
4261            <abstract>
4262              <entry>stream-device stream direction =></entry>
4263           
4264            </abstract>
4265            <abstract>
4266              <entry>stream-length stream <literal>&amp;optional</literal>new =></entry>
4267           
4268            </abstract>
4269            <abstract>
4270              <entry>stream-position stream <literal>&amp;optional</literal>new =></entry>
4271           
4272            </abstract>
4273            <abstract>
4274              <entry>streamp stream => boolean</entry>
4275           
4276            </abstract>
4277            <abstract>
4278              <entry>stream-write-char output-stream char =></entry>
4279           
4280            </abstract>
4281            <abstract>
4282              <entry>stream-write-entire-string output-stream string =></entry>
4283           
4284            </abstract>
4285            <abstract>
4286              <entry>stream-read-char input-stream =></entry>
4287           
4288            </abstract>
4289            <abstract>
4290              <entry>stream-unread-char input-stream char =></entry>
4291           
4292            </abstract>
4293            <abstract>
4294              <entry>stream-force-output output-stream => nil</entry>
4295           
4296            </abstract>
4297            <abstract>
4298              <entry>stream-maybe-force-output output-stream => nil</entry>
4299           
4300            </abstract>
4301            <abstract>
4302              <entry>stream-finish-output output-stream => nil</entry>
4303           
4304            </abstract>
4305            <abstract>
4306              <entry>stream-clear-output output-stream => nil</entry>
4307           
4308            </abstract>
4309            <abstract>
4310              <entry>close stream <literal>&amp;key</literal>abort => boolean</entry>
4311           
4312            </abstract>
4313            <abstract>
4314              <entry>stream-fresh-line stream => t</entry>
4315           
4316            </abstract>
4317            <abstract>
4318              <entry>stream-line-length stream => length</entry>
4319           
4320            </abstract>
4321            <abstract>
4322              <entry>interactive-stream-p stream => boolean</entry>
4323           
4324            </abstract>
4325            <abstract>
4326              <entry>stream-clear-input input-stream => nil</entry>
4327           
4328            </abstract>
4329            <abstract>
4330              <entry>stream-listen input-stream => boolean</entry>
4331           
4332            </abstract>
4333            <abstract>
4334              <entry>stream-filename stream => string</entry>
4335           
4336            </abstract>
4337            <abstract>
4338              <entry>ccl::select-stream-class instance in-p out-p char-p =>class</entry>
4339           
4340            </abstract>
4341          </row>
4342        </thead>
4343      </colspec>
4344      <para>The following functions are standard parts of Common Lisp, but
4345behave in special ways with regard to Gray streams.</para>
4346      <colspec>
4347        <thead cols="1">
4348          <tbody colwidth="100*"></tbody>
4349          <row>
4350            <abstract>
4351              <entry>open-stream-p stream => generalized-boolean</entry>
4352           
4353            </abstract>
4354            <abstract>
4355              <entry>input-stream-p stream => generalized-boolean</entry>
4356           
4357            </abstract>
4358            <abstract>
4359              <entry>output-stream-p stream => generalized-boolean</entry>
4360           
4361            </abstract>
4362            <abstract>
4363              <entry>stream-element-type stream =></entry>
4364           
4365            </abstract>
4366            <abstract>
4367              <entry>stream-error-stream =></entry>
4368           
4369            </abstract>
4370            <abstract>
4371              <entry>open</entry>
4372           
4373            </abstract>
4374            <abstract>
4375              <entry>close</entry>
4376           
4377            </abstract>
4378            <abstract>
4379              <entry>with-open-file</entry>
4380           
4381            </abstract>
4382          </row>
4383        </thead>
4384      </colspec>
4385      <para>Specifically, (open) and (with-open-file) accept a new keyword
4386argument, :class, which may be a symbol naming a class; the class
4387itself; or an instance of it.  The class so given must be a subtype
4388of 'stream, and an instance of it with no particular contents will
4389be passed to ccl::select-stream-class to determine what class to
4390actually instantiate.</para>
4391      <para>The following are standard, and do not behave specially with
4392regard to Gray streams, but probably should.</para>
4393      <colspec>
4394        <thead cols="1">
4395          <tbody colwidth="100*"></tbody>
4396          <row>
4397            <abstract>
4398              <entry>stream-external-format</entry>
4399           
4400            </abstract>
4401          </row>
4402        </thead>
4403      </colspec>
4404    </sect1>
4405
4406    <sect1 id="Extending-READ-SEQUENCE-and-WRITE-SEQUENCE">
4407      <title>Extending READ-SEQUENCE and WRITE-SEQUENCE</title>
4408
4409      <sect2 id="extending-read-write-overview">
4410        <title>Overview</title>
4411        <para>The "Gray Streams" API is based on an informal proposal that was
4412        made before ANSI CL adopted the READ-SEQUENCE and WRITE-SEQUENCE
4413        functions; as such, there is no "standard" way for the author of a Gray
4414        stream class to improve the performance of these functions by exploiting
4415        knowledge of the stream's internals (e.g., the buffering mechanism it
4416        uses.)</para>
4417        <para>In the absence of any such knowledge, READ-SEQUENCE and
4418        WRITE-SEQUENCE are effectively just convenient shorthand for a
4419        loop which calls READ-CHAR/READ-BYTE/WRITE-CHAR/WRITE-BYTE as
4420        appropriate. The mechanism described below allows subclasses
4421        of FUNDAMENTAL-STREAM to define more specialized (and
4422        presumably more efficient) behavior.</para>
4423      </sect2>
4424
4425      <sect2 id="Notes">
4426        <title>Notes</title>
4427        <para>READ-SEQUENCE and WRITE-SEQUENCE do a certain amount of
4428        sanity-checking and normalization of their arguments before
4429        dispatching to one of the methods above. If an individual
4430        method can't do anything particularly clever, CALL-NEXT-METHOD
4431        can be used to handle the general case.</para>
4432      </sect2>
4433
4434      <sect2 id="Example">
4435        <title>Example</title>
4436        <programlisting>
4437(defclass my-string-input-stream (fundamental-character-input-stream)
4438  ((string :initarg :string :accessor my-string-input-stream-string)
4439   (index :initform 0 :accessor my-string-input-stream-index)
4440   (length)))
4441
4442(defmethod stream-read-vector ((stream my-string-input-stream) vector start end)
4443  (if (not (typep vector 'simple-base-string))
4444      (call-next-method)
4445      (with-slots (string index length)
4446        (do* ((outpos start (1+ outpos)))
4447             ((or (= outpos end)
4448                  (= index length))
4449              outpos))
4450          (setf (schar vector outpos)
4451                (schar string index))
4452          (incf index)))))
4453</programlisting>
4454      </sect2>
4455    </sect1>
4456
4457    <sect1 id="Multibyte-I-O">
4458      <title>Multibyte I/O</title>
4459      <para>All heap-allocated objects in OpenMCL that cannot contain
4460      pointers to lisp objects are represented as
4461      <emphasis>ivectors</emphasis>. OpenMCL provides low-level
4462      functions, and , to efficiently transfer data between buffered
4463      streams and ivectors. There's some overlap in functionality
4464      between the functions described here and the ANSI CL
4465      READ-SEQUENCE and WRITE-SEQUENCE functions.</para>
4466      <para>As used here, the term "octet" means roughly the same
4467      thing as the term "8-bit byte". The functions described below
4468      transfer a specified sequence of octets between a buffered
4469      stream and an ivector, and don't really concern themselves with
4470      higher-level issues (like whether that octet sequence is within
4471      bounds or how it relates to the logical contents of the
4472      ivector.) For these reasons, these functions are generally less
4473      safe and more flexible than their ANSI counterparts.</para>
4474    </sect1>
4475
4476    <sect1 id="Gray-Streams-Dictionary">
4477      <title>Gray Streams Dictionary</title>
4478
4479      <sect2 id="CCL-STREAM-READ-LIST">
4480        <para>CCL:STREAM-READ-LIST</para>
4481        <informalfigure>stream-read-list</informalfigure>
4482        <bridgehead renderas="sect3">Name</bridgehead>
4483        <para>CCL:STREAM-READ-LIST &mdash;</para>
4484        <para>Generic Function</para>
4485        <bridgehead renderas="sect3">Synopsis</bridgehead>
4486        <programlisting>
4487stream-read-list
4488          stream list count
4489</programlisting>
4490        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
4491        <term><indexterm>stream
4492            <variablelist>a stream, presumably a fundamental-input-stream.</variablelist>
4493          </indexterm><indexterm>list
4494            <variablelist>a list. When a STREAM-READ-LIST method is called byREAD-SEQUENCE, this argument is guaranteed to be a properlist.</variablelist>
4495          </indexterm><indexterm>count
4496            <variablelist>a non-negative integer. When a STREAM-READ-LIST methodis called by READ-SEQUENCE, this argument is guaranteed notto be greater than the length of the list.</variablelist>
4497          </indexterm>
4498        </term>
4499        <bridgehead renderas="sect3">Description</bridgehead>
4500        <para>Should try to read up to count elements from stream into the
4501list list, returning the number of elements actually read (which
4502may be less than count in case of a premature end-of-file.)</para>
4503      </sect2>
4504
4505      <sect2 id="CCL-STREAM-WRITE-LIST">
4506        <para>CCL:STREAM-WRITE-LIST</para>
4507        <informalfigure>stream-write-list</informalfigure>
4508        <bridgehead renderas="sect3">Name</bridgehead>
4509        <para>CCL:STREAM-WRITE-LIST &mdash;</para>
4510        <para>Generic Function</para>
4511        <bridgehead renderas="sect3">Synopsis</bridgehead>
4512        <programlisting>
4513stream-write-list
4514          stream list count
4515</programlisting>
4516        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
4517        <term><indexterm>stream
4518            <variablelist>a stream, presumably a fundamental-ouput-stream.</variablelist>
4519          </indexterm><indexterm>list
4520            <variablelist>a list. When a STREAM-WRITE-LIST method is called byWRITE-SEQUENCE, this argument is guaranteed to be a properlist.</variablelist>
4521          </indexterm><indexterm>count
4522            <variablelist>a non-negative integer. When a STREAM-WRITE-LISTmethod is called by WRITE-SEQUENCE, this argument isguaranteed not to be greater than the length of the list.</variablelist>
4523          </indexterm>
4524        </term>
4525        <bridgehead renderas="sect3">Description</bridgehead>
4526        <para>should try to write the first count elements of list to
4527stream. The return value of this method is ignored.</para>
4528      </sect2>
4529
4530      <sect2 id="CCL-STREAM-READ-VECTOR">
4531        <para>CCL:STREAM-READ-VECTOR</para>
4532        <informalfigure>stream-read-vector</informalfigure>
4533        <bridgehead renderas="sect3">Name</bridgehead>
4534        <para>CCL:STREAM-READ-VECTOR &mdash;</para>
4535        <para>Generic Function</para>
4536        <bridgehead renderas="sect3">Synopsis</bridgehead>
4537        <programlisting>
4538stream-read-vector
4539          stream vector start end
4540</programlisting>
4541        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
4542        <term><indexterm>stream
4543            <variablelist>a stream, presumably a fundamental-input-stream</variablelist>
4544          </indexterm><indexterm>vector
4545            <variablelist>a vector. When a STREAM-READ-VECTOR method is calledby READ-SEQUENCE, this argument is guaranteed to be a simpleone-dimensional array.</variablelist>
4546          </indexterm><indexterm>start
4547            <variablelist>a non-negative integer. When a STREAM-READ-VECTORmethod is called by READ-SEQUENCE, this argument isguaranteed to be no greater than end and not greater thanthe length of vector.</variablelist>
4548          </indexterm><indexterm>end
4549            <variablelist>a non-negative integer. When a STREAM-READ-VECTORmethod is called by READ-SEQUENCE, this argument isguaranteed to be no less than end and not greater than thelength of vector.</variablelist>
4550          </indexterm>
4551        </term>
4552        <bridgehead renderas="sect3">Description</bridgehead>
4553        <para>should try to read successive elements from stream into
4554vector, starting at element start (inclusive) and continuing
4555through element end (exclusive.) Should return the index of the
4556vector element beyond the last one stored into, which may be less
4557than end in case of premature end-of-file.</para>
4558      </sect2>
4559
4560      <sect2 id="CCL-STREAM-WRITE-VECTOR">
4561        <para>CCL:STREAM-WRITE-VECTOR</para>
4562        <informalfigure>stream-write-vector</informalfigure>
4563        <bridgehead renderas="sect3">Name</bridgehead>
4564        <para>CCL:STREAM-WRITE-VECTOR &mdash;</para>
4565        <para>Generic Function</para>
4566        <bridgehead renderas="sect3">Synopsis</bridgehead>
4567        <programlisting>
4568stream-write-vector
4569          stream vector start end
4570</programlisting>
4571        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
4572        <term><indexterm>stream
4573            <variablelist>a stream, presumably a fundamental-output-stream</variablelist>
4574          </indexterm><indexterm>vector
4575            <variablelist>a vector. When a STREAM-WRITE-VECTOR method is calledby WRITE-SEQUENCE, this argument is guaranteed to be asimple one-dimensional array.</variablelist>
4576          </indexterm><indexterm>start
4577            <variablelist>a non-negative integer. When a STREAM-WRITE-VECTORmethod is called by WRITE-SEQUENCE, this argument isguaranteed to be no greater than end and not greater thanthe length of vector.</variablelist>
4578          </indexterm><indexterm>end
4579            <variablelist>a non-negative integer. When a STREAM-WRITE-VECTORmethod is called by WRITE-SEQUENCE, this argument isguaranteed to be no less than end and not greater than thelength of vector.</variablelist>
4580          </indexterm>
4581        </term>
4582        <bridgehead renderas="sect3">Description</bridgehead>
4583        <para>should try to write successive elements of vector to stream,
4584starting at element start (inclusive) and continuing through
4585element end (exclusive.)</para>
4586      </sect2>
4587
4588      <sect2 id="CCL--STREAM-DEVICE">
4589        <para>CCL::STREAM-DEVICE</para>
4590        <informalfigure>stream-device</informalfigure>
4591        <bridgehead renderas="sect3">Name</bridgehead>
4592        <para>CCL::STREAM-DEVICE &mdash; Returns the OS file descriptor associated with a
4593given lisp stream.</para>
4594        <para>Generic Function</para>
4595        <bridgehead renderas="sect3">Synopsis</bridgehead>
4596        <programlisting>
4597ccl::stream-device
4598          s direction
4599</programlisting>
4600        <bridgehead renderas="sect3">Method Signatures</bridgehead>
4601        <programlisting>
4602ccl::stream-device
4603          (s stream) direction => fd
4604</programlisting>
4605        <bridgehead renderas="sect3">Arguments and Values</bridgehead>
4606        <term><indexterm>s
4607            <variablelist>a stream.</variablelist>
4608          </indexterm><indexterm>direction
4609            <variablelist>either :INPUT or :OUTPUT.</variablelist>
4610          </indexterm><indexterm>fd
4611            <variablelist>a file descriptor, which is a non-negative integerused by the OS to refer to an open file, socket, or similarI/O connection.  NIL if there is no file descriptor associatedwith <literal>s</literal> in the direction given by<literal>direction</literal>.</variablelist>
4612          </indexterm>
4613        </term>
4614        <bridgehead renderas="sect3">Description</bridgehead>
4615        <para>Returns the file descriptor associated with
4616<literal>s</literal> in the direction given by
4617<literal>direction</literal>.  It is necessary to specify
4618<literal>direction</literal> because the input and output
4619file descriptors may be different; the most common case is when
4620one of them has been redirected by the Unix shell.</para>
4621      </sect2>
4622
4623      <sect2 id="STREAM-READ-IVECTOR">
4624        <para>STREAM-READ-IVECTOR</para>
4625        <informalfigure>stream-read-ivector</informalfigure>
4626        <bridgehead renderas="sect3">Name</bridgehead>
4627        <para>STREAM-READ-IVECTOR &mdash;</para>
4628        <para>Generic Function</para>
4629        <bridgehead renderas="sect3">Synopsis</bridgehead>
4630        <programlisting>
4631stream-read-ivector
4632          stream ivector start-octet max-octets
4633</programlisting>
4634        <bridgehead renderas="sect3">Description</bridgehead>
4635        <para>Reads up to max-octets octets from stream into ivector,
4636storing them at start-octet. Returns the number of octets actually
4637read.</para>
4638        <bridgehead renderas="sect3">Arguments</bridgehead>
4639        <term><indexterm>stream
4640            <variablelist>An input stream. The method defined onBUFFERED-INPUT-STREAMs requires that the size in octets ofan instance of the stream's element type is 1.</variablelist>
4641          </indexterm><indexterm>ivector
4642            <variablelist>Any ivector.</variablelist>
4643          </indexterm><indexterm>start-octet
4644            <variablelist>A non-negative integer.</variablelist>
4645          </indexterm><indexterm>max-octets
4646            <variablelist>A non-negative integer. The return value may be lessthan the value of this parameter if EOF was encountered.</variablelist>
4647          </indexterm>
4648        </term>
4649      </sect2>
4650
4651      <sect2 id="STREAM-WRITE-IVECTOR">
4652        <para>STREAM-WRITE-IVECTOR</para>
4653        <informalfigure>stream-write-ivector</informalfigure>
4654        <bridgehead renderas="sect3">Name</bridgehead>
4655        <para>STREAM-WRITE-IVECTOR &mdash;</para>
4656        <para>Generic Function</para>
4657        <bridgehead renderas="sect3">Synopsis</bridgehead>
4658        <programlisting>
4659stream-write-ivector stream
4660          ivector start-octet max-octets
4661</programlisting>
4662        <bridgehead renderas="sect3">Description</bridgehead>
4663        <para>Writes max-octets octets to stream from ivector, starting at
4664start-octet. Returns max-octets.</para>
4665        <bridgehead renderas="sect3">Arguments</bridgehead>
4666        <term><indexterm>stream
4667            <variablelist>An input stream. The method defined onBUFFERED-OUTPUT-STREAMs requires that the size in octets ofan instance of the stream's element type is 1.</variablelist>
4668          </indexterm><indexterm>ivector
4669            <variablelist>Any ivector</variablelist>
4670          </indexterm><indexterm>start-octet
4671            <variablelist>A non-negative integer.</variablelist>
4672          </indexterm><indexterm>max-octet
4673            <variablelist>A non-negative integer.</variablelist>
4674          </indexterm>
4675        </term>
4676        <bridgehead renderas="sect3">Examples</bridgehead>
4677        <programlisting>
4678;;; Write the contents of a (SIMPLE-ARRAY(UNSIGNED-BYTE 16) 3)
4679;;; to a character file stream. Read back the characters.
4680(let* ((a (make-array 3
4681                     :element-type '(unsigned-byte 16)
4682                     :initial-contents '(26725 27756 28449))))
4683  (with-open-file (s "junk"
4684                   :element-type 'character
4685                   :direction :io
4686                   :if-does-not-exist :create
4687                   :if-exists :supersede)
4688    ;; Write six octets (three elements).
4689    (stream-write-ivector s a 0 6)
4690    ;; Rewind, then read a line
4691    (file-position s 0)
4692    (read-line s)))
4693
4694;;; Write a vector of DOUBLE-FLOATs. Note that (to maintain
4695;;; alignment) there are 4 octets of padding before the 0th
4696;;; element of a (VECTOR DOUBLE-FLOAT) in 32-bit OpenMCL.
4697;;; (Note that (= (- ppc32::misc-dfloat-offset
4698;;;                  ppc32::misc-data-offset) 4))
4699(defun write-double-float-vector
4700  (stream vector &amp;key (start 0) (end (length vector)))
4701  (check-type vector (vector double-float))
4702  (let* ((start-octet (+ (* start 8)
4703                         (- target::misc-dfloat-offset
4704                         target::misc-data-offset)))
4705         (num-octets (* 8 (- end start))))
4706    (stream-write-ivector stream vector start-octet num-octets)))
4707</programlisting>
4708      </sect2>
4709    </sect1>
4710  </chapter>
4711
4712  <chapter id="Writing-Portable-Extensions-to-the-Object-System-using-the-MetaObject-Protocol">
4713    <title>Writing Portable Extensions to the Object System  using the MetaObject Protocol</title>
4714
4715    <sect1 id="MOP-Overview">
4716      <title>Overview</title>
4717      <para>OpenMCL supports a fairly large subset of the
4718      semi-standard MetaObject Protocol (MOP) for CLOS, as defined in
4719      chapters 5 and 6 of "The Art Of The Metaobject Protocol",
4720      (Kiczales et al, MIT Press 1991, ISBN 0-262-61074-4); this
4721      specification is also available online at
4722      http://www.alu.org/mop/index.html.</para>
4723    </sect1>
4724
4725    <sect1 id="MOP-Implementation-status">
4726      <title>Implementation status</title>
4727      <para>The keyword :openmcl-partial-mop is on *FEATURES* to
4728      indicate the presence of this functionality.</para>
4729
4730      <para>All of the symbols defined in the MOP specification
4731      (whether implemented or not) are exported from the "CCL" package
4732      and from an "OPENMCL-MOP" package.</para>
4733      <para><informaltable><tgroup cols="2" colsep="1"
4734      rowsep="1"><colspec align="center" colname="col0" /><colspec
4735      align="center" colname="col1" /><thead><row><entry
4736      align="center" valign="top"><para>construct</para></entry><entry
4737      align="center"
4738      valign="top"><para>status</para></entry></row></thead><tbody><row><entry
4739      align="center"
4740      valign="top"><para>accessor-method-slot-definition</para></entry><entry
4741      align="center"
4742      valign="top"><para>+</para></entry></row><row><entry
4743      align="center"
4744      valign="top"><para>add-dependent</para></entry><entry
4745      align="center"
4746      valign="top"><para>+</para></entry></row><row><entry
4747      align="center"
4748      valign="top"><para>add-direct-method</para></entry><entry
4749      align="center"
4750      valign="top"><para>+</para></entry></row><row><entry
4751      align="center"
4752      valign="top"><para>add-direct-subclass</para></entry><entry
4753      align="center"
4754      valign="top"><para>+</para></entry></row><row><entry
4755      align="center"
4756      valign="top"><para>add-method</para></entry><entry
4757      align="center"
4758      valign="top"><para>+</para></entry></row><row><entry
4759      align="center"
4760      valign="top"><para>class-default-initargs</para></entry><entry
4761      align="center"
4762      valign="top"><para>+</para></entry></row><row><entry
4763      align="center"
4764      valign="top"><para>class-direct-default-initargs</para></entry><entry
4765      align="center"
4766      valign="top"><para>+</para></entry></row><row><entry
4767      align="center"
4768      valign="top"><para>class-direct-slots</para></entry><entry
4769      align="center"
4770      valign="top"><para>+</para></entry></row><row><entry
4771      align="center"
4772      valign="top"><para>class-direct-subclasses</para></entry><entry
4773      align="center"
4774      valign="top"><para>+</para></entry></row><row><entry
4775      align="center"
4776      valign="top"><para>class-direct-superclasses</para></entry><entry
4777      align="center"
4778      valign="top"><para>+</para></entry></row><row><entry
4779      align="center"
4780      valign="top"><para>class-finalized-p</para></entry><entry
4781      align="center"
4782      valign="top"><para>+</para></entry></row><row><entry
4783      align="center"
4784      valign="top"><para>class-prototype</para></entry><entry
4785      align="center"
4786      valign="top"><para>+</para></entry></row><row><entry
4787      align="center"
4788      valign="top"><para>class-slots</para></entry><entry
4789      align="center"
4790      valign="top"><para>+</para></entry></row><row><entry
4791      align="center"
4792      valign="top"><para>compute-applicable-methods</para></entry><entry
4793      align="center"
4794      valign="top"><para>-</para></entry></row><row><entry
4795      align="center"
4796      valign="top"><para>compute-applicable-methods-using-classes</para></entry><entry
4797      align="center"
4798      valign="top"><para>-</para></entry></row><row><entry
4799      align="center"
4800      valign="top"><para>compute-class-precedence-list</para></entry><entry
4801      align="center"
4802      valign="top"><para>+</para></entry></row><row><entry
4803      align="center"
4804      valign="top"><para>compute-direct-initargs</para></entry><entry
4805      align="center"
4806      valign="top"><para>+</para></entry></row><row><entry
4807      align="center"
4808      valign="top"><para>compute-discriminating-function</para></entry><entry
4809      align="center"
4810      valign="top"><para>-</para></entry></row><row><entry
4811      align="center"
4812      valign="top"><para>compute-effective-method</para></entry><entry
4813      align="center"
4814      valign="top"><para>+</para></entry></row><row><entry
4815      align="center"
4816      valign="top"><para>compute-effective-slot-definition</para></entry><entry
4817      align="center"
4818      valign="top"><para>+</para></entry></row><row><entry
4819      align="center"
4820      valign="top"><para>compute-slots</para></entry><entry
4821      align="center"
4822      valign="top"><para>+</para></entry></row><row><entry
4823      align="center"
4824      valign="top"><para>direct-slot-definition-class</para></entry><entry
4825      align="center"
4826      valign="top"><para>+</para></entry></row><row><entry
4827      align="center"
4828      valign="top"><para>effective-slot-definition-class</para></entry><entry
4829      align="center"
4830      valign="top"><para>+</para></entry></row><row><entry
4831      align="center"
4832      valign="top"><para>ensure-class</para></entry><entry
4833      align="center"
4834      valign="top"><para>+</para></entry></row><row><entry
4835      align="center"
4836      valign="top"><para>ensure-class-using-class</para></entry><entry
4837      align="center"
4838      valign="top"><para>+</para></entry></row><row><entry
4839      align="center"
4840      valign="top"><para>ensure-generic-function-using-class</para></entry><entry
4841      align="center"
4842      valign="top"><para>+</para></entry></row><row><entry
4843      align="center"
4844      valign="top"><para>eql-specializer-object</para></entry><entry
4845      align="center"
4846      valign="top"><para>+</para></entry></row><row><entry
4847      align="center"
4848      valign="top"><para>extract-lambda-list</para></entry><entry
4849      align="center"
4850      valign="top"><para>+</para></entry></row><row><entry
4851      align="center"
4852      valign="top"><para>extract-specializer-names</para></entry><entry
4853      align="center"
4854      valign="top"><para>+</para></entry></row><row><entry
4855      align="center"
4856      valign="top"><para>finalize-inheritance</para></entry><entry
4857      align="center"
4858      valign="top"><para>+</para></entry></row><row><entry
4859      align="center"
4860      valign="top"><para>find-method-combination</para></entry><entry
4861      align="center"
4862      valign="top"><para>+</para></entry></row><row><entry
4863      align="center"
4864      valign="top"><para>funcallable-standard-instance-access</para></entry><entry
4865      align="center"
4866      valign="top"><para>+</para></entry></row><row><entry
4867      align="center"
4868      valign="top"><para>generic-function-argument-precedence-order</para></entry><entry
4869      align="center"
4870      valign="top"><para>+</para></entry></row><row><entry
4871      align="center"
4872      valign="top"><para>generic-function-declarations</para></entry><entry
4873      align="center"
4874      valign="top"><para>+</para></entry></row><row><entry
4875      align="center"
4876      valign="top"><para>generic-function-lambda-list</para></entry><entry
4877      align="center"
4878      valign="top"><para>+</para></entry></row><row><entry
4879      align="center"
4880      valign="top"><para>generic-function-method-class</para></entry><entry
4881      align="center"
4882      valign="top"><para>+</para></entry></row><row><entry
4883      align="center"
4884      valign="top"><para>generic-function-method-combination</para></entry><entry
4885      align="center"
4886      valign="top"><para>+</para></entry></row><row><entry
4887      align="center"
4888      valign="top"><para>generic-function-methods</para></entry><entry
4889      align="center"
4890      valign="top"><para>+</para></entry></row><row><entry
4891      align="center"
4892      valign="top"><para>generic-function-name</para></entry><entry
4893      align="center"
4894      valign="top"><para>+</para></entry></row><row><entry
4895      align="center"
4896      valign="top"><para>intern-eql-specializer</para></entry><entry
4897      align="center"
4898      valign="top"><para>+</para></entry></row><row><entry
4899      align="center"
4900      valign="top"><para>make-method-lambda</para></entry><entry
4901      align="center"
4902      valign="top"><para>-</para></entry></row><row><entry
4903      align="center"
4904      valign="top"><para>map-dependents</para></entry><entry
4905      align="center"
4906      valign="top"><para>+</para></entry></row><row><entry
4907      align="center"
4908      valign="top"><para>method-function</para></entry><entry
4909      align="center"
4910      valign="top"><para>+</para></entry></row><row><entry
4911      align="center"
4912      valign="top"><para>method-generic-function</para></entry><entry
4913      align="center"
4914      valign="top"><para>+</para></entry></row><row><entry
4915      align="center"
4916      valign="top"><para>method-lambda-list</para></entry><entry
4917      align="center"
4918      valign="top"><para>+</para></entry></row><row><entry
4919      align="center"
4920      valign="top"><para>method-qualifiers</para></entry><entry
4921      align="center"
4922      valign="top"><para>+</para></entry></row><row><entry
4923      align="center"
4924      valign="top"><para>method-specializers</para></entry><entry
4925      align="center"
4926      valign="top"><para>+</para></entry></row><row><entry
4927      align="center"
4928      valign="top"><para>reader-method-class</para></entry><entry
4929      align="center"
4930      valign="top"><para>+</para></entry></row><row><entry
4931      align="center"
4932      valign="top"><para>remove-dependent</para></entry><entry
4933      align="center"
4934      valign="top"><para>+</para></entry></row><row><entry
4935      align="center"
4936      valign="top"><para>remove-direct-method</para></entry><entry
4937      align="center"
4938      valign="top"><para>+</para></entry></row><row><entry
4939      align="center"
4940      valign="top"><para>remove-direct-subclass</para></entry><entry
4941      align="center"
4942      valign="top"><para>+</para></entry></row><row><entry
4943      align="center"
4944      valign="top"><para>remove-method</para></entry><entry
4945      align="center"
4946      valign="top"><para>+</para></entry></row><row><entry
4947      align="center"
4948      valign="top"><para>set-funcallable-instance-function</para></entry><entry
4949      align="center"
4950      valign="top"><para>-</para></entry></row><row><entry
4951      align="center"
4952      valign="top"><para>slot-boundp-using-class</para></entry><entry
4953      align="center"
4954      valign="top"><para>+</para></entry></row><row><entry
4955      align="center"
4956      valign="top"><para>slot-definition-allocation</para></entry><entry
4957      align="center"
4958      valign="top"><para>+</para></entry></row><row><entry
4959      align="center"
4960      valign="top"><para>slot-definition-initargs</para></entry><entry
4961      align="center"
4962      valign="top"><para>+</para></entry></row><row><entry
4963      align="center"
4964      valign="top"><para>slot-definition-initform</para></entry><entry
4965      align="center"
4966      valign="top"><para>+</para></entry></row><row><entry
4967      align="center"
4968      valign="top"><para>slot-definition-initfunction</para></entry><entry
4969      align="center"
4970      valign="top"><para>+</para></entry></row><row><entry
4971      align="center"
4972      valign="top"><para>slot-definition-location</para></entry><entry
4973      align="center"
4974      valign="top"><para>+</para></entry></row><row><entry
4975      align="center"
4976      valign="top"><para>slot-definition-name</para></entry><entry
4977      align="center"
4978      valign="top"><para>+</para></entry></row><row><entry
4979      align="center"
4980      valign="top"><para>slot-definition-readers</para></entry><entry
4981      align="center"
4982      valign="top"><para>+</para></entry></row><row><entry
4983      align="center"
4984      valign="top"><para>slot-definition-type</para></entry><entry
4985      align="center"
4986      valign="top"><para>+</para></entry></row><row><entry
4987      align="center"
4988      valign="top"><para>slot-definition-writers</para></entry><entry
4989      align="center"
4990      valign="top"><para>+</para></entry></row><row><entry
4991      align="center"
4992      valign="top"><para>slot-makunbound-using-class</para></entry><entry
4993      align="center"
4994      valign="top"><para>+</para></entry></row><row><entry
4995      align="center"
4996      valign="top"><para>slot-value-using-class</para></entry><entry
4997      align="center"
4998      valign="top"><para>+</para></entry></row><row><entry
4999      align="center"
5000      valign="top"><para>specializer-direct-generic-functions</para></entry><entry
5001      align="center"
5002      valign="top"><para>+</para></entry></row><row><entry
5003      align="center"
5004      valign="top"><para>specializer-direct-methods</para></entry><entry
5005      align="center"
5006      valign="top"><para>+</para></entry></row><row><entry
5007      align="center"
5008      valign="top"><para>standard-instance-access</para></entry><entry
5009      align="center"
5010      valign="top"><para>+</para></entry></row><row><entry
5011      align="center"
5012      valign="top"><para>update-dependent</para></entry><entry
5013      align="center"
5014      valign="top"><para>+</para></entry></row><row><entry
5015      align="center"
5016      valign="top"><para>validate-superclass</para></entry><entry
5017      align="center"
5018      valign="top"><para>+</para></entry></row><row><entry
5019      align="center"
5020      valign="top"><para>writer-method-class</para></entry><entry
5021      align="center"
5022      valign="top"><para>+</para></entry></row></tbody></tgroup></informaltable></para>
5023     
5024      <para>Note that those generic functions whose status is "-" in
5025      the table above deal with the internals of generic function
5026      dispatch and method invocation (the "Generic Function Invocation
5027      Protocol".) Method functions are implemented a bit differently
5028      in OpenMCL from what the MOP expects, and it's not yet clear if
5029      or how this subprotocol can be well-supported.</para>
5030      <para>Those constructs that are marked as "+" in the table above
5031      are nominally implemented as the MOP document specifies
5032      (deviations from the specification should be considered bugs;
5033      please report them as such.) Note that some CLOS implementations
5034      in widespread use (e.g., PCL) implement some things
5035      (ENSURE-CLASS-USING-CLASS comes to mind) a bit differently from
5036      what the MOP specifies.</para>
5037    </sect1>
5038
5039    <sect1 id="Concurrency-issues">
5040      <title>Concurrency issues</title>
5041      <para>The entire CLOS class and generic function hierarchy is
5042      effectively a (large, complicated) shared data structure; it's
5043      not generally practical for a thread to request exclusive access
5044      to all of CLOS, and the effects of volitional modification of
5045      the CLOS hierarchy (via clas redefinition, CHANGE-CLASS, etc) in
5046      a multithreaded environment aren't always tractable.</para>
5047      <para>Native threads exacerbate this problem (in that they
5048      increase the opportunities for concurrent modification and
5049      access.) The implementation should try to ensure that a thread's
5050      view of any subset of the CLOS hierarchy is consistent (to the
5051      extent that that's possible) and should try to ensure that
5052      incidental modifications of the hierarchy (cache updates, etc.)
5053      happen atomically; it's not generally possible for the
5054      implementation to guarantee that a thread's view of things is
5055      correct and current.</para>
5056      <para>If you are loading code and defining classes in the most
5057      usual way, which is to say, via the compiler, using only a
5058      single thread, these issues are probably not going to affect you
5059      much.</para>
5060      <para>If, however, you are making finicky changes to the class
5061      hierarchy while you're running multiple threads which manipulate
5062      objects related to each other, more care is required.  Before
5063      doing such a thing, you should know what you're doing and
5064      already be aware of what precautions to take, without being
5065      told.  That said, if you do it, you should seriously consider
5066      what your application's critical data is, and use locks for
5067      critical code sections.</para>
5068    </sect1>
5069  </chapter>
5070
5071  <chapter id="The-Foreign-Function-Interface">
5072    <title>The Foreign-Function Interface</title>
5073
5074    <sect1 id="Specifying-And-Using-Foreign-Types">
5075      <title>Specifying And Using Foreign Types</title>
5076
5077      <sect2 id="Overview-foreign-types">
5078        <title>Overview</title>
5079        <para>OpenMCL provides a fairly rich language for defining and
5080        specifying foreign data types (this language is derived from
5081        CMUCL's "alien type" system.)</para>
5082        <para>In practice, most foreign type definitions are
5083        introduced into OpenMCL via its interface database (see ),
5084        though it's also possible to define foreign types
5085        interactively and/or programmatically.</para>
5086        <para>OpenMCL's foreign type system is "evolving" (a polite
5087        word for not-quite-complete): there are some inconsistencies
5088        involving package usage, for instance. Symbols used in foreign
5089        type specifiers <emphasis>should</emphasis> be keywords, but
5090        this convention isn't always enforced.</para>
5091        <para>Foreign type, record, and field names are
5092        case-sensitive; OpenMCL uses some escaping conventions (see )
5093        to allow keywords to be used to denote these names.</para>
5094      </sect2>
5095
5096      <sect2 id="Syntax-of-Foreign-Type-Specifiers">
5097        <title>Syntax of Foreign Type Specifiers</title>
5098        <itemizedlist>
5099          <listitem>
5100            <para>Some foreign types are builtin: keywords denote
5101            primitive,builtin types such as the IEEE-double-float type
5102            (denoted:DOUBLE-FLOAT), in much the same way as certain
5103            symbols(CONS, FIXNUM,etc.) define primitive CL
5104            types.</para>
5105          </listitem>
5106          <listitem>
5107            <para>Constructors such as :SIGNED and :UNSIGNED can be
5108            used to denotesigned and unsigned integer subtypes
5109            (analogous to the CL typespecifiers SIGNED-BYTE and
5110            UNSIGNED-BYTE.) :SIGNED is shorthand for(:SIGNED 32) and
5111            :UNSIGNED is shorthand for (:UNSIGNED 32).</para>
5112          </listitem>
5113          <listitem>
5114            <para>Aliases for other (perhaps more complicated) types
5115            can bedefined via CCL:DEF-FOREIGN-TYPE (sort of like
5116            CL:DEFTYPE or the Ctypedef facility). The type :CHAR is
5117            defined as an alias for (:SIGNED8) on some platforms, as
5118            (:UNSIGNED 8) on others.</para>
5119          </listitem>
5120          <listitem>
5121            <para>The construct (:STRUCT <emphasis>name</emphasis>)
5122            can be used torefer to a named structure type; (:UNION
5123            <emphasis>name</emphasis>)can be used to refer to a named
5124            union type. It isn't necessary toenumerate a structure or
5125            union type's fields in order to refer tothe type.</para>
5126          </listitem>
5127          <listitem>
5128            <para>If <emphasis>X</emphasis> is a valid foreign type
5129            reference,then (:* <emphasis>X</emphasis>) denotes the
5130            foreign type "pointerto<emphasis> X</emphasis>". By
5131            convention, (:* T) denotes ananonymous pointer type,
5132            vaguely equivalent to "void*" in C.</para>
5133          </listitem>
5134          <listitem>
5135            <para>If a fieldlist is a list of lists, each of whose CAR
5136            is a foreign field name (keyword) and whose CADR is a
5137            foreign type specifier, then (:STRUCT
5138            <emphasis>name</emphasis> ,@fieldlist) is adefinition of
5139            the structure type <emphasis>name</emphasis>,
5140            and (:UNION<emphasis> name</emphasis> ,@fieldlist) is a
5141            definition of theunion type
5142            <emphasis>name</emphasis>. Note that it's necessary
5143            todefine a structure or union type in order to include
5144            that type in a structure, union, or array, but only
5145            necessary to "refer to" a strucure or union type in order
5146            to define a type alias or a pointer type.</para>
5147          </listitem>
5148          <listitem>
5149            <para>If <emphasis>X</emphasis> is a defined foreign type
5150            , then (:array <emphasis>X</emphasis> &amp;rest dims)
5151            denotes the foreigntype "array of
5152            <emphasis>X</emphasis>". Although multiplearray dimensions
5153            are allowed by the :array constructor,
5154            only single-dimensioned arrays are (at all) well-supported
5155            in OpenMCL.</para>
5156          </listitem>
5157        </itemizedlist>
5158      </sect2>
5159    </sect1>
5160
5161    <sect1 id="Foreign-Function-Calls">
5162      <title>Foreign Function Calls</title>
5163
5164      <sect2 id="Overview-foreign-calls">
5165        <title>Overview</title>
5166        <para>OpenMCL provides a number of constructs for calling
5167        foreign functions from Lisp code (all of them based on the
5168        function CCL:%FF-CALL).  In many cases, OpenMCL's interface
5169        translator (see ) provides information about the foreign
5170        function's entrypoint name and argument and return types; this
5171        enables the use of the #_ reader macro (described below),
5172        which may be more concise and/or more readable than other
5173        constructs.</para>
5174        <para>OpenMCL also provides a mechanism for defining
5175        <emphasis>callbacks</emphasis>: lisp functions which can be
5176        called from foreign code.</para>
5177        <para>There's no supported way to directly pass lisp data to
5178        foreign functions: scalar lisp data must be coerced to an
5179        equivalent foreign representatation, and lisp arrays (notably
5180        strings) must be copied to non-GCed memory.</para>
5181
5182        <sect3 id="Type-Designators-for-Arguments-and-Return-Values">
5183          <title>Type Designators for Arguments and Return Values</title>
5184          <para>The types of foreign argument and return values in foreign
5185          function calls and callbacks can be specified by any of the following
5186keywords:</para>
5187          <variablelist>
5188            <varlistentry>
5189              <term>:UNSIGNED-BYTE</term>
5190
5191              <listitem>
5192                <para>The argument/return value is of type (UNSIGNED-BYTE 8)</para>
5193              </listitem>
5194            </varlistentry>
5195
5196            <varlistentry>
5197              <term>:SIGNED-BYTE</term>
5198
5199              <listitem>
5200                <para>The argument/return value is of type (SIGNED-BYTE 8)</para>
5201              </listitem>
5202            </varlistentry>
5203
5204            <varlistentry>
5205              <term>:UNSIGNED-HALFWORD</term>
5206
5207              <listitem>
5208                <para>The argument/return value is of type (UNSIGNED-BYTE 16)</para>
5209              </listitem>
5210            </varlistentry>
5211
5212            <varlistentry>
5213              <term>:SIGNED-HALFWORD</term>
5214
5215              <listitem>
5216                <para>The argument/return value is of type (SIGNED-BYTE 16)</para>
5217              </listitem>
5218            </varlistentry>
5219
5220            <varlistentry>
5221              <term>:UNSIGNED-FULLWORD</term>
5222
5223              <listitem>
5224                <para>The argument/return value is of type (UNSIGNED-BYTE 32)</para>
5225              </listitem>
5226            </varlistentry>
5227
5228            <varlistentry>
5229              <term>:SIGNED-FULLWORD</term>
5230
5231              <listitem>
5232                <para>The argument/return value is of type (SIGNED-BYTE 32)</para>
5233              </listitem>
5234            </varlistentry>
5235
5236            <varlistentry>
5237              <term>:UNSIGNED-DOUBLEWORD</term>
5238
5239              <listitem>
5240                <para>The argument/return value is of type (UNSIGNED-BYTE 64)</para>
5241              </listitem>
5242            </varlistentry>
5243
5244            <varlistentry>
5245              <term>:SIGNED-DOUBLEWORD</term>
5246
5247              <listitem>
5248                <para>The argument/return value is of type (SIGNED-BYTE 64)</para>
5249              </listitem>
5250            </varlistentry>
5251
5252            <varlistentry>
5253              <term>:SINGLE-FLOAT</term>
5254
5255              <listitem>
5256                <para>The argument/return value is of type SINGLE-FLOAT</para>
5257              </listitem>
5258            </varlistentry>
5259
5260            <varlistentry>
5261              <term>:DOUBLE-FLOAT</term>
5262
5263              <listitem>
5264                <para>The argument/return value is of type DOUBLE-FLOAT</para>
5265              </listitem>
5266            </varlistentry>
5267
5268            <varlistentry>
5269              <term>:ADDRESS</term>
5270
5271              <listitem>
5272                <para>The argument/return values
5273                is <link linkend="arb24">a MACPTR</link>.</para>
5274              </listitem>
5275            </varlistentry>
5276
5277            <varlistentry>
5278              <term>:VOID</term>
5279
5280              <listitem>
5281                <para>or NIL Not valid as an argument type specifier; specifies
5282                that there is no meaningful return value</para>
5283              </listitem>
5284            </varlistentry>
5285          </variablelist>
5286
5287          <para>On some platforms, a small positive integer
5288          <emphasis>N</emphasis> can also be used as an argument
5289          specifier; it indicates that the corresponding argument is a
5290          pointer to an <emphasis>N</emphasis>-word structure or union
5291          which should be passed by value to the foreign
5292          function.  Exactly which foreign structures are passed
5293          by value and how is very dependent on the Application
5294          Binary Interface (ABI) of the platform; unless you're
5295          very familar with ABI detatils (some of which are quite
5296          baroque), it's often easier to let higher-level constructs
5297          deal with these details.</para>
5298        </sect3>
5299
5300        <sect3 id="External-Entrypoints-and-Named-External-Entrypoints">
5301          <title>External Entrypoints and Named External Entrypoints</title>
5302          <para>PowerPC machine instructions are always aligned on
5303          32-bit boundaries, so the two least significant bits of the
5304          first instruction ("entrypoint") of a foreign function are
5305          always 0. OpenMCL often represents an entrypoint address as
5306          a fixnum that's binary-equivalent to the entrypoint address:
5307          if<emphasis> E</emphasis> is an entrypoint address expressed
5308          as a signed 32-bit integer, then (ash <emphasis>E</emphasis>
5309          -2) is an equivalent fixnum representation of that
5310          address. An entrypoint address can also be encapsulated in a
5311          MACPTR (see FIXTHIS), but that's somewhat less efficient.</para>
5312          <para>Although it's possible to use fixnums or macptrs to
5313          represent entrypoint addresses, it's somewhat cumbersome to
5314          do so. OpenMCL can cache the addresses of named external
5315          functions in structure-like objects of type
5316          CCL:EXTERNAL-ENTRY-POINT (sometimes abbreviated as EEP).
5317          Through the use of LOAD-TIME-VALUE, compiled lisp functions
5318          are able to reference EEPs as constants; the use of an
5319          indirection allows OpenMCL runtime system to ensure that the
5320          EEP's address is current and correct.</para>
5321        </sect3>
5322      </sect2>
5323
5324      <sect2 id="Return-Conventions-for-C-Structures">
5325        <title>Return Conventions for C Structures</title>
5326        <para> On some platforms, C functions that are defined to
5327        return structures do so by reference: they actually
5328        accept a first parameter of type "pointer to returned
5329        struct/union" - which must be allocated by the caller - and
5330        don't return a meaningful value.</para>
5331        <para><emphasis>Exactly</emphasis> how a C function that's
5332        defined to return a foreign structure does so is dependent on
5333        the ABI (and on the size ad composition of the structure/union
5334        in many cases.)</para>
5335      </sect2>
5336    </sect1>
5337
5338    <sect1 id="Referencing-and-Using-Foreign-Memory-Addresses">
5339      <title>Referencing and Using Foreign Memory Addresses</title>
5340
5341      <sect2 id="Overview-memory-addresses">
5342        <title>Overview</title>
5343
5344        <sect3 id="Basics">
5345          <title>Basics</title>
5346          <para>For a variety of technical reasons, it isn't generally
5347          possible to directly reference arbitrary absolute addresses
5348          (such as those returned by the C library function malloc(),
5349          for instance) in OpenMCL. In OpenMCL (and in MCL), such
5350          addresses need to be <emphasis>encapsulated</emphasis> in
5351          objects of type CCL:MACPTR; one can think of a MACPTR as
5352          being a specialized type of structure whose sole purpose is
5353          to provide a way of referring to an underlying "raw"
5354          address.</para>
5355          <para>It's sometimes convenient to blur the distinction
5356          between a MACPTR and the address it represents; it's
5357          sometimes necessary to maintain that distiction. It's
5358          important to remember that a MACPTR is (generally) a
5359          first-class Lisp object in the same sense that a CONS cell
5360          is: it'll get GCed when it's no longer possible to reference
5361          it. The "lifetime" of a MACPTR doesn't generally have
5362          anything to do with the lifetime of the block of memory its
5363          address points to.</para>
5364          <para>It might be tempting to ask "How does one obtain the
5365          address encapsulated by a MACPTR ?". The answer to that
5366          question is that one doesn't do that (and there's no way to
5367          do that): addresses aren't first-class objects, and there's
5368          no way to refer to one.</para>
5369          <para>Two MACPTRs that encapsulate the same address are EQL
5370          to each other.</para>
5371          <para>There are a small number of ways to directly create a
5372          MACPTR (and there's a fair amount of syntactic sugar built
5373          on top of of those primitives.) These primitives will be
5374          discussed in greater detail below, but they include:</para>
5375
5376          <itemizedlist>
5377            <listitem>
5378            <para>Creating a MACPTR with a specified address, usually
5379            via thefunction CCL:%INT-TO-PTR.</para>
5380            </listitem>
5381            <listitem>
5382              <para>Referencing the return valueof a foreign function
5383              call (see )that's specified to return an address.</para>
5384            </listitem>
5385            <listitem>
5386              <para>Referencing a memory location that's specified to
5387              contain an address.</para>
5388            </listitem>
5389          </itemizedlist>
5390
5391          <para>All of these primitive MACPTR-creating operations are
5392          usually open-coded by the compiler; it has a fairly good
5393          notion of what low-level operations "produce" MACPTRs and
5394          which operations "consume" the addresses that the
5395          encapsulate, and will usually optimize out the introduction
5396          of intermediate MACPTRs in a simple expression.</para>
5397          <para>One consequence of the use of MACPTR objects to
5398          encapsulate foreign addresses is that (naively)
5399          <emphasis>every reference to a foreign address causes a
5400          MACPTR to be allocated.</emphasis></para>
5401          <para>Consider a code fragment like the following:</para>
5402          <programlisting>
5403(defun get-next-event ()
5404  "get the next event from a hypothetical window system"
5405  (loop
5406    (let* ((event (#_get_next_window_system_event))) ; via an FF-CALL
5407          (unless (null-event-p event)
5408            (handle-event event)))))
5409</programlisting>
5410          <para>As this is written, each call to the (hypothetical)
5411          foreign function #_get_next_window_system_event will return
5412          a new MACPTR object.  Ignoring for the sake of argument the
5413          question of whether this code fragment exhibits a good way
5414          to poll for external events (it doesn't), it's not hard to
5415          imagine that this loop could execute several millon times
5416          per second (producing several million MACPTRs per second.)
5417          Clearly, the "naive" approach is impractical in many
5418          cases.</para>
5419        </sect3>
5420
5421        <sect3 id="Stack-allocation-of---and-destructive-operations-on---MACPTRs-">
5422          <title>Stack allocation of - and destructive operations on - MACPTRs.</title>
5423          <para>If certain conditions held in the environment in which
5424          GET-NEXT-EVENT ran - namely, if it was guaranteed that
5425          neither NULL-EVENT-P nor HANDLE-EVENT cached or otherwise
5426          retained their arguments (the "event" pointer) - there'd be
5427          a few alternatives to the naive approach. One of those
5428          approaches would be to use the primitive function
5429          %SETF-MACPTR (described in greater detail below) to
5430          destructively modify a MACPTR (to change the value of the
5431          address it encapsulates.) The GET-NEXT-EVENT example could
5432          be re-written as:</para>
5433          <programlisting>
5434(defun get-next-event ()
5435  (let* ((event (%int-to-ptr 0))) ; create a MACPTR with address 0
5436    (loop
5437      (%setf-macptr event (#_get_next_window_system_event)) ; re-use it
5438      (unless (null-event-p event)
5439        (handle-event event)))))
5440</programlisting>
5441          <para>That version's a bit more realistic: it allocates a
5442          single MACPTR outside if the loop, then changes its address
5443          to point to the current address of the hypothetical event
5444          structure on each loop iteration. If there are a million
5445          loop iterations per call to GET-NEXT-EVENT, we're allocating
5446          a million times fewer MACPTRs per call; that sounds like a
5447          Good Thing.</para>
5448          <para>An Even Better Thing would be to advise the compiler
5449          that the initial value (the null MACPTR) bound to the
5450          variable event has dynamic extent (that value won't be
5451          referenced once control leaves the extent of the binding of
5452          that variable.) Common Lisp allows us to make such an
5453          assertion via a DYNAMIC-EXTENT declaration; OpenMCL's
5454          compiler can recognize the "primitive MACPTR-creating
5455          operation" involved and can replace it with an equivalent
5456          operation that stack-allocates the MACPTR object. If we're
5457          not worried about the cost of allocating that MACPTR on
5458          every iteration (the cost is small and there's no hidden GC
5459          cost), we could move the binding back inside the
5460          loop:</para>
5461          <programlisting>
5462(defun get-next-event ()
5463  (loop
5464    (let* ((event (%null-ptr))) ; (%NULL-PTR) is shorthand for (%INT-TO-PTR 0)
5465      (declare (dynamic-extent event))
5466      (%setf-macptr event (#_get_next_window_system_event))
5467      (unless (null-event-p event)
5468        (handle-event event)))))
5469</programlisting>
5470          <para>The idiom of binding one or more variables to
5471          stack-allocated MACPTRs, then destructively modifying those
5472          MACPTRs before executing a body of code is common enough
5473          that OpenMCL provides a macro (WITH-MACPTRS) that handles
5474          all of the gory details. The following version of
5475          GET-NEXT-EVENT is semantically equivalent to the previous
5476          version, but hopefully a bit more concise:</para>
5477          <programlisting>
5478(defun get-next-event ()
5479  (loop
5480    (with-macptrs ((event (#_get_next_window_system_event)))
5481      (unless (null-event-p event)
5482        (handle-event event)))))
5483</programlisting>
5484        </sect3>
5485
5486        <sect3 id="Stack-allocated-memory--and-stack-allocated-pointers-to-it--">
5487          <title>Stack-allocated memory (and stack-allocated pointers to it.)</title>
5488          <para>Fairly often, the blocks of foreign memory (obtained
5489          by malloc or something similar) have well-defined lifetimes
5490          (they can safely be freed at some point when it's known that
5491          they're no longer needed and it's known that they're no
5492          longer referenced.) A common idiom might be:</para>
5493          <programlisting>
5494(with-macptrs (p (#_allocate_foreign_memory size))
5495  (unwind-protect
5496    (use-foreign-memory p)
5497    (#_deallocate_foreign_memory p)))
5498</programlisting>
5499          <para>That's not unreasonable code, but it's fairly
5500          expensive for a number of reasons: foreign functions calls
5501          are themselves fairly expensive (as is UNWIND-PROTECT), and
5502          most library routines for allocating and deallocating
5503          foreign memory (things like malloc and free) can be fairly
5504          expensive in their own right.</para>
5505          <para>In the idiomatic code above, both the MACPTR P and the
5506          block of memory that's being allocated and freed have
5507          dynamic extent and are therefore good candidates for stack
5508          allocation. OpenMCL provides the %STACK-BLOCK macro, which
5509          executes a body of code with one or more variables bound to
5510          stack-allocated MACPTRs which encapsulate the addresses of
5511          stack-allocated blocks of foreign memory. Using
5512          %STACK-BLOCK, the idiomatic code is:</para>
5513          <programlisting>
5514(%stack-block ((p size))
5515  (use-foreign-memory p))
5516</programlisting>
5517          <para>which is a bit more efficient and a bit more concise
5518          than the version presented earlier.</para>
5519          <para>%STACK-BLOCK is used as the basis for slightly
5520          higher-level things like RLET. (See FIXTHIS for more information
5521          about RLET.)</para>
5522        </sect3>
5523
5524        <sect3 id="Caveats-">
5525          <title>Caveats.</title>
5526          <para>Reading from, writing to, allocating, and freeing
5527          foreign memory are all potentially dangerous operations;
5528          this is no less true when these operations are performed in
5529          OpenMCL than when they're done in C or some other
5530          lower-level language. In addition, destructive operations on
5531          Lisp objects be dangerous, as can stack allocation if it's
5532          abused (if DYNAMIC-EXTENT declarations are violated.)
5533          Correct use of the constructs and primitives described here
5534          is reliable and safe; slightly incorrect use of these
5535          constructs and primitives can crash OpenMCL.</para>
5536        </sect3>
5537      </sect2>
5538
5539      <sect2 id="Foreign-Memory-Addresses-Dictionary">
5540        <title>Foreign-Memory-Addresses Dictionary</title>
5541        <para>Unless otherwise noted, all of the symbols mentioned
5542        below are exported from the CCL package.</para>
5543
5544        <sect3 id="Scalar-memory-reference">
5545          <title>Scalar memory reference</title>
5546          <variablelist>
5547            <varlistentry>
5548              <term>Syntax</term>
5549
5550              <listitem>
5551                <para>%get-signed-byte ptr &#38;optional (offset 0)</para>
5552
5553                <para>%get-unsigned-byte ptr &#38;optional (offset 0)</para>
5554
5555                <para>%get-signed-word ptr &#38;optional (offset 0)</para>
5556
5557                <para>%get-unsigned-word ptr &#38;optional (offset 0)</para>
5558
5559                <para>%get-signed-long ptr &#38;optional (offset 0)</para>
5560
5561                <para>%get-unsigned-long ptr &#38;optional (offset 0)</para>
5562
5563                <para>%%get-signed-longlong ptr &#38;optional (offset 0)</para>
5564
5565                <para>%%get-unsigned-longlong ptr &#38;optional (offset 0)</para>
5566
5567                <para>%get-ptr ptr &#38;optional (offset 0)</para>
5568
5569                <para>%get-single-float ptr &#38;optional (offset 0)</para>
5570
5571                <para>%get-double-float ptr &#38;optional (offset 0)</para>
5572              </listitem>
5573            </varlistentry>
5574            <varlistentry>
5575              <term>Description</term>
5576
5577              <listitem>
5578                <para>References and returns the signed or unsigned 8-bit byte,
5579                signed or unsigned 16-bit word, signed or unsigned 32-bit long
5580                word, signed or unsigned 64-bit long long word, 32-bit address,
5581                32-bit single-float, or 64-bit double-float at the effective byte
5582                address formed by adding offset to the address encapsulated by
5583                ptr.</para>
5584              </listitem>
5585            </varlistentry>
5586
5587            <varlistentry>
5588              <term>Arguments</term>
5589
5590              <listitem>
5591
5592                <variablelist>
5593                  <varlistentry>
5594                    <term>ptr</term>
5595
5596                    <listitem>
5597                      <para>A MACPTR</para>
5598                    </listitem>
5599                  </varlistentry>
5600
5601                  <varlistentry>
5602                    <term>offset</term>
5603
5604                    <listitem>
5605                      <para>A fixnum</para>
5606                    </listitem>
5607                  </varlistentry>
5608                </variablelist>
5609              </listitem>
5610            </varlistentry>
5611          </variablelist>
5612
5613          <para>All of the memory reference primitives described above can be</para>
5614          <para>used with SETF.</para>
5615        </sect3>
5616
5617        <sect3 id="iget-bit--Function-">
5618          <para>%get-bit [Function]</para>
5619          <term><indexterm>Syntax
5620              <variablelist>%get-bit ptr bit-offset</variablelist>
5621            </indexterm><indexterm>Description
5622              <variablelist>References and returns the bit-offsetth bit at the addressencapsulated by ptr. (Bit 0 at a given address is the mostsignificant bit of the byte at that address.) Can be used withSETF.</variablelist>
5623            </indexterm><indexterm>Arguments
5624              <variablelist> 
5625                <term><indexterm>ptr
5626                    <variablelist>A MACPTR</variablelist>
5627                  </indexterm><indexterm>bit-offset
5628                    <variablelist>A fixnum</variablelist>
5629                  </indexterm>
5630                </term></variablelist>
5631            </indexterm>
5632          </term>
5633        </sect3>
5634
5635        <sect3 id="iget-bitfield--Function-">
5636          <para>%get-bitfield [Function]</para>
5637          <term><indexterm>Syntax
5638              <variablelist>%get-bitfield ptr bit-offset width</variablelist>
5639            </indexterm><indexterm>Description
5640              <variablelist>References and returns an unsigned integer composed from thewidth bits found bit-offsetbits from the address encapsulated byptr. (The least significant bit of the result is the value of(%get-bit ptr (1- (+ bit-offset width)))). Can be used with SETF.</variablelist>
5641            </indexterm><indexterm>Arguments
5642              <variablelist> </variablelist>
5643            </indexterm><indexterm>ptr
5644              <variablelist>A MACPTR
5645                <term><indexterm>bit-offset
5646                    <variablelist>A fixnum</variablelist>
5647                  </indexterm><indexterm>width
5648                    <variablelist>A positive fixnum</variablelist>
5649                  </indexterm>
5650                </term></variablelist>
5651            </indexterm>
5652          </term>
5653        </sect3>
5654
5655        <sect3 id="iint-to-ptr--Function-">
5656          <para>%int-to-ptr [Function]</para>
5657          <term><indexterm>Syntax
5658              <variablelist>%int-to-ptr int</variablelist>
5659            </indexterm><indexterm>Description
5660              <variablelist>Creates and returns a MACPTR whose address matches int.</variablelist>
5661            </indexterm><indexterm>Arguments
5662              <variablelist> 
5663                <term><indexterm>int
5664                    <variablelist>An (unsigned-byte 32)</variablelist>
5665                  </indexterm>
5666                </term></variablelist>
5667            </indexterm>
5668          </term>
5669        </sect3>
5670
5671        <sect3 id="iinc-ptr--Function-">
5672          <para>%inc-ptr [Function]</para>
5673          <term><indexterm>Syntax
5674              <variablelist>%inc-ptr ptr &amp;optional (delta 1)</variablelist>
5675            </indexterm><indexterm>Description
5676              <variablelist>Creates and returns a MACPTR whose address is the address ofptr plus delta. The idiom (%inc-ptr ptr 0) is sometimes used tocopy a MACPTR, e.g., to create a new MACPTR encapsulating the sameaddress as ptr.</variablelist>
5677            </indexterm><indexterm>Arguments
5678              <variablelist> 
5679                <term><indexterm>ptr
5680                    <variablelist>A MACPTR</variablelist>
5681                  </indexterm><indexterm>delta
5682                    <variablelist>A fixnum</variablelist>
5683                  </indexterm>
5684                </term></variablelist>
5685            </indexterm>
5686          </term>
5687        </sect3>
5688
5689        <sect3 id="iptr-to-int--Function-">
5690          <para>%ptr-to-int [Function]</para>
5691          <term><indexterm>Syntax
5692              <variablelist>%ptr-to-int ptr</variablelist>
5693            </indexterm><indexterm>Description
5694              <variablelist>Returns the address encapsulated by ptr, as an(unsigned-byte 32).</variablelist>
5695            </indexterm><indexterm>Arguments
5696              <variablelist> 
5697                <term><indexterm>ptr
5698                    <variablelist>A MACPTR</variablelist>
5699                  </indexterm>
5700                </term></variablelist>
5701            </indexterm>
5702          </term>
5703        </sect3>
5704
5705        <sect3 id="inull-ptr--Macro-">
5706          <para>%null-ptr [Macro]</para>
5707          <term><indexterm>Syntax
5708              <variablelist>%null-ptr</variablelist>
5709            </indexterm><indexterm>Description
5710              <variablelist>Equivalent to (%ptr-to-int 0).</variablelist>
5711            </indexterm>
5712          </term>
5713        </sect3>
5714
5715        <sect3 id="inull-ptr-p--Function-">
5716          <para>%null-ptr-p [Function]</para>
5717          <term><indexterm>Syntax
5718              <variablelist>%null-ptr-p ptr</variablelist>
5719            </indexterm><indexterm>Description
5720              <variablelist>Returns T If ptr is a MACPTR encapsulating the address 0,NIL if ptr encapsulates some other address.</variablelist>
5721            </indexterm><indexterm>Arguments
5722              <variablelist> 
5723                <term><indexterm>ptr
5724                    <variablelist>A MACPTR</variablelist>
5725                  </indexterm>
5726                </term></variablelist>
5727            </indexterm>
5728          </term>
5729        </sect3>
5730
5731        <sect3 id="isetf-macptr--Function-">
5732          <para>%setf-macptr [Function]</para>
5733          <term><indexterm>Syntax
5734              <variablelist>%setf-macptr dest-ptr src-ptr</variablelist>
5735            </indexterm><indexterm>Description
5736              <variablelist>Causes dest-ptr to encapsulate the same address that src-ptrdoes, then returns dest-ptr.</variablelist>
5737            </indexterm><indexterm>Arguments
5738              <variablelist> 
5739                <term><indexterm>dest-ptr
5740                    <variablelist>A MACPTR</variablelist>
5741                  </indexterm><indexterm>src-ptr
5742                    <variablelist>A MACPTR</variablelist>
5743                  </indexterm>
5744                </term></variablelist>
5745            </indexterm>
5746          </term>
5747        </sect3>
5748
5749        <sect3 id="iincf-ptr--Macro-">
5750          <para>%incf-ptr [Macro]</para>
5751          <term><indexterm>Syntax
5752              <variablelist>%incf-ptr ptr &amp;optional (delta 1)</variablelist>
5753            </indexterm><indexterm>Description
5754              <variablelist>Destructively modifies ptr, by adding delta to the addressit encapsulates. Returns ptr.</variablelist>
5755            </indexterm><indexterm>Arguments
5756              <variablelist> 
5757                <term><indexterm>ptr
5758                    <variablelist>A MACPTR</variablelist>
5759                  </indexterm><indexterm>delta
5760                    <variablelist>A fixnum</variablelist>
5761                  </indexterm>
5762                </term></variablelist>
5763            </indexterm>
5764          </term>
5765        </sect3>
5766
5767        <sect3 id="with-macptrs--Macro-">
5768          <para>with-macptrs [Macro]</para>
5769          <term><indexterm>Syntax
5770              <variablelist>with-macptrs (var expr)* &amp;body body</variablelist>
5771            </indexterm><indexterm>Description
5772              <variablelist>Executes body in an environment in which each var is boundto a stack-allocated macptr which encapsulates the foreign addressyielded by the corresponding expr. Returns whatever value(s) bodyreturns.</variablelist>
5773            </indexterm><indexterm>Arguments
5774              <variablelist> 
5775                <term><indexterm>var
5776                    <variablelist>A symbol (variable name)</variablelist>
5777                  </indexterm><indexterm>expr
5778                    <variablelist>A MACPTR-valued expression</variablelist>
5779                  </indexterm>
5780                </term></variablelist>
5781            </indexterm>
5782          </term>
5783        </sect3>
5784
5785        <sect3 id="istack-block--Macro-">
5786          <para>%stack-block [Macro]</para>
5787          <term><indexterm>Syntax
5788              <variablelist>%stack-block (var expr)* &amp;body body</variablelist>
5789            </indexterm><indexterm>Description
5790              <variablelist>Executes body in an environment in which each var is boundto a stack-allocated macptr which encapsulates the address of astack-allocated region of size expr bytes. Returns whatevervalue(s) body returns.</variablelist>
5791            </indexterm><indexterm>Arguments
5792              <variablelist> 
5793                <term><indexterm>var
5794                    <variablelist>A symbol (variable name)</variablelist>
5795                  </indexterm><indexterm>expr
5796                    <variablelist>An expression which should evaluate to a non-negativefixnum</variablelist>
5797                  </indexterm>
5798                </term></variablelist>
5799            </indexterm>
5800          </term>
5801        </sect3>
5802
5803        <sect3 id="make-cstring--Function-">
5804          <para>make-cstring [Function]</para>
5805          <term><indexterm>Syntax
5806              <variablelist>make-cstring string</variablelist>
5807            </indexterm><indexterm>Description
5808              <variablelist>Allocates a block of memory (via malloc) of length (1+(length string)). Copies the string to this block and appends atrailing NUL byte; returns a MACPTR to the block.</variablelist>
5809            </indexterm><indexterm>Arguments
5810              <variablelist> 
5811                <term><indexterm>string
5812                    <variablelist>A lisp string</variablelist>
5813                  </indexterm>
5814                </term></variablelist>
5815            </indexterm>
5816          </term>
5817        </sect3>
5818
5819        <sect3 id="with-cstrs--Macro-">
5820          <para>with-cstrs [Macro]</para>
5821          <term><indexterm>Syntax
5822              <variablelist>with-cstrs (var string)* &amp;body body</variablelist>
5823            </indexterm><indexterm>Description
5824              <variablelist>Executes body in an environment in which each var is boundto a stack-allocated macptr which encapsulates the %address of astack-allocated region of into which each string (and a trailingNUL byte) has been copied. Returns whatever value(s) body returns.</variablelist>
5825            </indexterm><indexterm>Arguments
5826              <variablelist> 
5827                <term><indexterm>var
5828                    <variablelist>A symbol (variable name)</variablelist>
5829                  </indexterm><indexterm>string
5830                    <variablelist>An expression which should evaluate to a lisp string</variablelist>
5831                  </indexterm>
5832                </term></variablelist>
5833            </indexterm>
5834          </term>
5835        </sect3>
5836
5837        <sect3 id="iget-cstring--Function-">
5838          <para>%get-cstring [Function]</para>
5839          <term><indexterm>Syntax
5840              <variablelist>%get-cstring ptr</variablelist>
5841            </indexterm><indexterm>Description
5842              <variablelist>Interprets ptr as a pointer to a (NUL -terminated) C string;returns an equivalent lisp string.</variablelist>
5843            </indexterm><indexterm>Arguments
5844              <variablelist>
5845                <term><indexterm>ptr
5846                    <variablelist>A MACPTR</variablelist>
5847                  </indexterm>
5848                </term></variablelist>
5849            </indexterm>
5850          </term>
5851        </sect3>
5852
5853        <sect3 id="istr-from-ptr--Function-">
5854          <para>%str-from-ptr [Function]</para>
5855          <term><indexterm>Syntax
5856              <variablelist>%str-from-ptr ptr length</variablelist>
5857            </indexterm><indexterm>Description
5858              <variablelist>Returns a lisp string of length <literal>length</literal>,whose contents are initialized from the bytes at<literal> ptr.</literal></variablelist>
5859            </indexterm><indexterm>Arguments
5860              <variablelist>
5861                <term><indexterm>ptr
5862                    <variablelist>AMACPTR</variablelist>
5863                  </indexterm><indexterm>length
5864                    <variablelist>anon-negative fixnum</variablelist>
5865                  </indexterm>
5866                </term></variablelist>
5867            </indexterm>
5868          </term>
5869        </sect3>
5870      </sect2>
5871    </sect1>
5872
5873    <sect1 id="The-Interface-Database">
5874      <title>The Interface Database</title>
5875
5876      <sect2 id="interface-database-Overview">
5877        <title>Overview</title>
5878        <para>OpenMCL uses a set of database files which contain
5879        foreign type, record, constant, and function definitions
5880        derived from the operating system's header files, be that
5881        Linux or Darwin.  An archive containing these database files
5882        (and the shell scripts which were used in their creation) is
5883        available; see the Distributions page for information about
5884        obtaining current interface database files.</para>
5885        <para>Not surprisingly, different platforms use different database files.</para>
5886        <para>OpenMCL defines reader macros that consult these databases:</para>
5887        <listitem mark="bullet">
5888          <variablelist>#$foo looks up the value of the constant definition of FIXTHIS</variablelist>
5889          <variablelist>#_foo looks up the foreign function definition for FIXTHIS</variablelist>
5890       
5891        </listitem>
5892        <para>In both cases, the symbol foo is interned in the "OS"
5893        package. The #$ reader macro has the side-effect of defining
5894        foo as a constant (as if via DEFCONSTANT); the #_ reader macro
5895        has the side effect of defining foo as a macro which will
5896        expand into an (EXTERNAL-CALL form.)</para>
5897        <para>It's important to remember that the side-effect happens
5898        when the form containing the reader macro is
5899        read. Macroexpansion functions that expand into forms which
5900        contain instances of those reader macros don't do what one
5901        might think that they do, unless the macros are expanded in
5902        the same lisp session as the reader macro was read in.</para>
5903        <para>In addition, references to foreign type,
5904        structure/union, and field names (when used in the RREF/PREF
5905        and RLET macros) will cause these database files to be
5906        consulted.</para>
5907        <para>Since the OpenMCL sources contain instances of these
5908        reader macros (and references to foreign record types and
5909        fields), compiling OpenMCL from those sources depends on the
5910        ability to find and use (see <xref
5911        linkend="Building-the-heap-image"/>).</para>
5912      </sect2>
5913
5914      <sect2 id="Other-issues">
5915        <title>Other issues:</title>
5916        <itemizedlist>
5917          <listitem>
5918            <para>OpenMCL now preserves the case of external symbols
5919            in itsdatabase files. See for information about case in
5920            foreign symbol names.</para>
5921          </listitem>
5922          <listitem>
5923            <para>The Linux databases are derived from a somewhat
5924            arbitrary set of Linux header files. Linux is enough of a
5925            moving target that it may be difficult to define a standard,
5926            reference set of interfaces from which to derive a standard,
5927            reference set of database files.This seems to be less of
5928            an issue with Darwin and FreeBSD.</para>
5929          </listitem>
5930        </itemizedlist>
5931        <para>For information about building the database files,
5932        see <xref linkend="The-Interface-Translator"/>.</para>
5933      </sect2>
5934    </sect1>
5935
5936    <sect1 id="Using-Interface-Directories">
5937      <title>Using Interface Directories</title>
5938
5939      <sect2 id="Interface-Directory-Overview">
5940        <title>Overview</title>
5941        <para>As distributed, the "ccl:headers;" (for LinuxPPC)
5942        directory is organized like:</para>
5943        <programlisting>
5944headers/
5945headers/gl/
5946headers/gl/C/
5947headers/gl/C/populate.sh
5948headers/gl/constants.cdb
5949headers/gl/functions.cdb
5950headers/gl/records.cdb
5951headers/gl/objc-classes.cdb
5952headers/gl/objc-methods.cdb
5953headers/gl/types.cdb
5954headers/gnome/
5955headers/gnome/C/
5956headers/gnome/C/populate.sh
5957headers/gnome/constants.cdb
5958headers/gnome/functions.cb
5959headers/gnome/records.cdb
5960headers/gnome/objc-classes.cdb
5961headers/gnome/objc-methods.cdb
5962headers/gnome/types.cdb
5963headers/gtk/
5964headers/gtk/C/
5965headers/gtk/C/populate.sh
5966headers/gtk/constants.cdb
5967headers/gtk/functions.cdb
5968headers/gtk/records.cdb
5969headers/gtk/objc-classes.cdb
5970headers/gtk/objc-methods.cdb
5971headers/gtk/types.cdb
5972headers/libc/
5973headers/libc/C/
5974headers/libc/C/populate.sh
5975headers/libc/constants.cdb
5976headers/libc/functions.cdb
5977headers/libc/records.cdb
5978headers/libc/objc-classes.cdb
5979headers/libc/objc-methods.cdb
5980headers/libc/types.cdb
5981</programlisting>
5982        <para>e.g, as a set of parallel subdirectories, each with a
5983        lowercase name and each of which contains a set of 6 database
5984        files and a "C" subdirectory which contains a shell script
5985        used in the database creation process.</para>
5986        <para>As one might assume, the database files in each of these
5987        subdirectories contain foreign type, constant, and function
5988        definitions - as well as ObjC class and method info -that
5989        correspond (roughly) to the information contained in the
5990        header files associated with a "-dev" package in a Linux
5991        distribution.  "libc" corresponds pretty closely to the
5992        interfaces associated with "glibc/libc6" header files, "gl"
5993        corresponds to an "openGL+GLUT" developmnent package, "gtk"
5994        and "gnome" contain interface information from the GTK+1.2 and
5995        GNOME libraries, respectively.</para>
5996        <para>For Darwin, the "ccl:darwin-headers" directory contains
5997        a "libc" subdirectory, whose contents roughly correspond to
5998        those of "/usr/include" under Darwin, as well as
5999        subdirectories corresponding to the MacOSX Carbon and Cocoa
6000        frameworks.</para>
6001        <para>To see the precise set of .h files used to generate the
6002        database files in a given interface directory, consult the
6003        corresponding "populate.sh" shell script (in the interface
6004        directory's "C" subdirectory.)</para>
6005        <para>The intent is that this initial set can be augmented to
6006        meet local needs, and that this can be done in a fairly
6007        incremental fashion: one needn't have unrelated header files
6008        installed in order to generate interface databases for a
6009        package of interest.</para>
6010        <para>Hopefully, this scheme will also make it easier to
6011        distribute patches and bug fixes.</para>
6012        <para>OpenMCL maintains a list of directories; when looking
6013        for a foreign type, constant, function, or record definition,
6014        it'll consult the database files in each directory on that
6015        list. Initially, the list contains an entry for the "libc"
6016        interface directory. OpenMCL needs to be explicitly told to
6017        look in other interface directories should it need to do
6018        so.</para>
6019      </sect2>
6020
6021      <sect2 id="Creating-new-interface-directories">
6022        <title>Creating new interface directories</title>
6023        <para>This example refers to "ccl:headers;", which is
6024        appropriate for LinuxPPC. The procedure's analogous under
6025        Darwin, where the "ccl:darwin-headers;" directory would be
6026        used instead.</para>
6027        <para>To create a new interface directory, "foo", and a set of
6028        database files in that directory:</para>
6029        <varlistentry numeration="arabic">
6030          <variablelist>Create a subdirectory of "ccl:headers;" named"foo".</variablelist>
6031          <variablelist>Create a subdirectory of "ccl:headers;foo;" named"C".</variablelist>
6032          <variablelist>Create a file in "ccl:headers;foo;C;" named"populate.sh".One way of accomplishing the above steps is:
6033            <programlisting>
6034? (close (open "ccl:headers;foo;C;populate.sh" :direction :output :
6035               if-does-not-exist :create :if-exists :overwrite))
6036</programlisting></variablelist>
6037          <variablelist>Edit the file created above, using the "populate.sh"files in the distribution as guidelines.The file might wind up looking something like:
6038            <programlisting>
6039#/bin/sh
6040h-to-ffi.sh `foo-config -cflags` /usr/include/foo/foo.h
6041</programlisting></variablelist>
6042        </varlistentry>
6043        <para>Refer to
6044for information about running the
6045interface translator and .ffi parser.</para>
6046        <para>Assuming that all went well, there should now be .cdb files in
6047"ccl:headers;foo;". You can then do
6048<code>(use-interface-dir :foo)</code> whenever you need to
6049access the foreign type information in those database files.</para>
6050      </sect2>
6051    </sect1>
6052
6053    <sect1 id="Using-Shared-Libraries">
6054      <para>Using Shared Libraries</para>
6055
6056      <sect2 id="Overview--13-">
6057        <para>Overview
6058OpenMCL provides facilities to open and close shared libraries.</para>
6059        <para>"Opening" a shared library, which is done with
6060, maps the library's
6061code and
6062data into OpenMCL's address space and makes its exported symbols
6063accessible to OpenMCL.</para>
6064        <para>"Closing" a shared library, which is done with
6065, unmaps the
6066library's code and
6067data and removes the library's symbols from the global namespace.</para>
6068        <para>A small number of shared libraries (including libc, libm, libdl
6069under Linux, and the "system" library under Darwin) are opened by
6070the lisp kernel and can't be closed.</para>
6071        <para>OpenMCL uses data structures of type EXTERNAL-ENTRY-POINT to map a
6072foreign function name (string) to that foreign function's
6073<emphasis>current<</emphasis> address. (A function's address may
6074vary from session to session as different versions of shared libraries may
6075load at different addresses; it may vary within a session for similar
6076reasons.)</para>
6077        <para>An EXTERNAL-ENTRY-POINT whose address is known is said to be
6078<emphasis>resolved</emphasis>. When an external entry point is resolved,
6079the shared library which defines that entry point is noted; when a shared
6080library is closed, the entry points that it defines are made unresolved.
6081An EXTERNAL-ENTRY-POINT must be in the resolved state in order to be
6082FF-CALLed; calling an unresolved entry point causes a "last
6083chance" attempt to resolve it. Attempting to resolve an entrypoint
6084that was defined in a closed library will cause an attempt to reopen that
6085library.</para>
6086        <para>OpenMCL keeps track of all libraries that have been opened in a lisp
6087session. When a saved application is first started, an attempt is made to
6088reopen all libraries that were open when the image was saved, and an
6089attempt is made to resolve all entrypoints that had been referenced when
6090the image was saved. Either of these attempts can fail "quietly",
6091leaving some entry points in an unresolved state.</para>
6092        <para>Linux shared libraries can be referred to either by a string which
6093describes their full pathname or by their <emphasis>soname</emphasis>, a
6094shorter string that can be defined when the library is created. The
6095dynamic linker mechanisms used in Linux make it possible (through a series
6096of filesystem links and other means) to refer to a library via several
6097names; the library's soname is often the most appropriate identifier.</para>
6098        <para>sonames are often less version-specific than other names for
6099libraries; a program that refers to a library by the name
6100"libc.so.6" is more portable than one which refers to
6101"libc-2.1.3.so" or to "libc-2.2.3.so", even though the
6102latter two names might each be platform-specific aliases of the first.</para>
6103        <para>All of the global symbols described below are exported from the CCL
6104package.</para>
6105      </sect2>
6106
6107      <sect2 id="Limitations-and-known-bugs--1-">
6108        <para>Limitations and known bugs</para>
6109        <listitem mark="bullet">
6110          <variablelist>Don't get me started.</variablelist>
6111          <variablelist>The underlying functionality has a poor notion of dependency;it's not always possible to open libraries that depend on unopenedlibraries, but it's possible to close libraries on which otherlibraries depend.It <emphasis>may</emphasis> be possible to generate moreexplicit dependency information by parsing the output of the Linux lddand ldconfig programs.</variablelist>
6112       
6113        </listitem>
6114      </sect2>
6115
6116      <sect2 id="Darwin-Notes">
6117        <para>Darwin Notes
6118Darwin shared libraries come in two (basic) flavors:</para>
6119        <listitem mark="bullet">
6120          <variablelist>"dylibs" (which often have the extension".dylib") are primarily intended to be linked against atcompile/link time. They can be loaded dynamically,<emphasis>but can't be unloaded</emphasis>. Accordingly,OPEN-SHARED-LIBRARY can be used to open a .dylib-style library;calling CLOSE-SHARED-LIBRARY on the result of such a callproduces a warning, and has no other effect.It appears that (due to an OS bug) attempts to open .dylibshared-libraries that are already open can cause memory corruptionunless the full pathname of the .dylib file is specified on thefirst and all subsequent calls.</variablelist>
6121          <variablelist>"bundles" are intended to serve as applicationextensions; they can be opened multiple times (creating multipleinstances of the library!) and closed properly.</variablelist>
6122       
6123        </listitem>
6124        <para>Thanks to Michael Klingbeil for getting both kinds of Darwin shared
6125libraries working in OpenMCL.</para>
6126      </sect2>
6127    </sect1>
6128
6129    <sect1 id="The-Interface-Translator">
6130      <para>The Interface Translator</para>
6131
6132      <sect2 id="Overview--14-">
6133        <para>Overview
6134OpenMCL uses an interface translation system based on the FFIGEN
6135system, which is described at
6136http://www.ccs.neu.edu/home/lth/ffigen/.
6137The interface translator makes
6138the constant, type, structure, and function definitions in a set of
6139C-language header files available to lisp code.</para>
6140        <para>The basic idea of the FFIGEN scheme is to use the C compiler's
6141frontend and parser to translate .h files into semantically equivalent
6142.ffi files, which represent the definitions from the headers using a
6143syntax based on S-expressions.
6144Lisp code can then concentrate on
6145the .ffi representation, without having to concern itself with the
6146semantics of header file inclusion or the arcana of C parsing.</para>
6147        <para>The original FFIGEN system used a modified version of the LCC C
6148compiler to produce .ffi files. Since many LinuxPPC header files contain
6149GCC-specific constructs, OpenMCL's translation system uses a modified
6150version of GCC (called, somewhat confusingly, ffigen.)</para>
6151        <para>A version of ffigen based on GCC-4.0 was developed during the spring
6152and summer of 2005.  Sources (diffs relative to the GCC-4.0 release)
6153are available here, and
6154binaries are available for
6155DarwinPPC
6156and for
6157LinuxPPC.
6158These versions should be insensitive to to the version of GCC (and its
6159preprocessor) installed on the system.</para>
6160        <para>An older version was developed in 2001-2002; it depended on the installed
6161version of GCC being 2.95.  It may still be of interest for people unable
6162to run the GCC-4.0-based version for whatever reason.</para>
6163        <para>A LinuxPPC binary of this older version is available at ftp://clozure.com/pub/ffigen-0.1.tar.gz,
6164and LinuxPPC source differences are at ftp://clozure.com/pub/ffigen-src.tar.gz.</para>
6165        <para>For Darwin, the binary of the older FFIGEN is available at ftp://clozure.com/pub/ffigen-darwin.tar.gz,
6166and the source differences are at ftp://clozure.com/pub/ffigen-darwin-src.tar.gz.</para>
6167        <para>A shell script (distributed with the source and binary packages)
6168called h-to-ffi.sh  reads a specified .h file (and optional
6169preprocessor arguments) and writes a (hopefully) equivalent .ffi file to
6170standard output, calling the installed C preprocessor and the ffigen
6171program with appropriate arguments.</para>
6172        <para>For each interface directory (see )
6173<emphasis>subdir</emphasis> distributed with OpenMCL, a shell script
6174(distributed with OpenMCL as "ccl:headers;<emphasis>subdir</emphasis>;C;populate.sh"
6175("ccl:darwin-headers;<emphasis>subdir</emphasis>;C;populate.sh"
6176for Darwin)) calls h-to-ffi.sh on a large number of the header files in
6177/usr/include (or some other <emphasis>system header path</emphasis>) and
6178creates a parallel directory tree in "ccl:headers;<emphasis>subdir</emphasis>;C;<emphasis>system</emphasis>;<emphasis>header</emphasis>;<emphasis>path</emphasis>;"
6179(or "ccl:darwin-headers;<emphasis>subdir</emphasis>;C;<emphasis>system</emphasis>;<emphasis>header</emphasis>;<emphasis>path</emphasis>;"),
6180populating that directory with .ffi files.</para>
6181        <para>A lisp function defined in "ccl:library;parse-ffi.lisp"
6182reads the .ffi files in a specified interface directory
6183<emphasis>subdir</emphasis> and generates new versions of the databases
6184(files with the extension .cdb).</para>
6185        <para>The CDB databases are used by the #$ and #_ reader macros and are
6186used in the expansion of RREF, RLET, and related macros.</para>
6187      </sect2>
6188
6189      <sect2 id="Details--rebuilding-the-CDB-databases--step-by-step">
6190        <para>Details: rebuilding the CDB databases, step by step</para>
6191        <varlistentry numeration="arabic">
6192          <variablelist>Ensure that the FFIGEN program is installed. See the"README" file in the source or binary archive for specificinstallation instructions.This example assumes LinuxPPC; for 32-bit DarwinPPC, substitute"ccl:darwin-headers;" for "ccl:headers;".   For 64-bit DarwinPPC,substitute "ccl:darwin-headers64;".</variablelist>
6193          <variablelist>Edit the "ccl:headers;<emphasis>subdir</emphasis>;C;populate.sh"shell script. When you're confident that the files andpreprocessor options match your environment, cd to the"ccl:headers;<emphasis>subdir</emphasis>;C;" directory andinvoke ./populate.sh. Repeat this step until you're able tocleanly translate all files refrenced in the shell script.</variablelist>
6194          <variablelist>Run OpenMCL:
6195            <programlisting>
6196? (require "PARSE-FFI")
6197PARSE-FFI
6198
6199? (parse-standard-ffi-files :SUBDIR)
6200;;; lots of output ... after a while, shiny new .cdb files should
6201;;; appear in "ccl:headers;subdir;"
6202;;; (or "ccl:darwin-headers;subdir;" under Darwin)
6203</programlisting></variablelist>
6204        </varlistentry>
6205        <para>PARSE-STANDARD-FFI-FILES accepts a :PREPEND-UNDERSCORES keyword
6206argument. Darwin (and some other platforms) use a convention wherein the
6207symbols associated with C-visible external function and variables have
6208underscore characters prepended to their names. When this argument is
6209true, PARSE-STANDARD-FFI-FILES will prepend underscores to all foreign
6210function names written to the database, so that (#_foo ...) expands into
6211an EXTERNAL-CALL to "_foo".</para>
6212      </sect2>
6213    </sect1>
6214
6215    <sect1 id="Case-sensitivity-of-foreign-names-in-OpenMCL">
6216      <para>Case-sensitivity of foreign names in OpenMCL</para>
6217
6218      <sect2 id="Overview--15-">
6219        <para>Overview
6220As of release 0.11, OpenMCL addresses the fact that foreign type,
6221constant, record, field, and function nams are case-sensitive and provides
6222mechanisms to refer to these names via lisp symbols.</para>
6223        <para>Previous versions of OpenMCL have tried to ignore that fact, under
6224the belief that case conflicts were rare and that many users (and
6225implementors) would prefer not to deal with case-related issues. The fact
6226that some information in the interface databases was incomplete or
6227inaccessable because of this policy made it clearer that the policy was
6228untenable. I can't claim that the approach described here is
6229aesthetically pleasing, but I can honestly say that it's less
6230unpleasant than other approaches that I'd thought of. I'd be
6231interested to hear alternate proposals.</para>
6232        <para>The issues described here have to do with how lisp symbols are used
6233to denote foreign functions, constants, types, records, and fields. It
6234doesn't affect how other lisp objects are sometimes used to denote
6235foreign objects. For instance, the first argument to the EXTERNAL-CALL
6236macros is now and has always been a case-sensitive string.</para>
6237      </sect2>
6238
6239      <sect2 id="Foreign-constant-and-function-names">
6240        <para>Foreign constant and function names
6241The primary way of referring to foreign constant and function names
6242in OpenMCL is via the #$ and #_ reader macros. These reader macro
6243functions each read a symbol into the "OS" package, look up its
6244constant or function definition in the interface database, and assign the
6245value of the constant to the symbol or install a macroexpansion function
6246on the symbol.</para>
6247        <para>In order to observe case-sensitivity, the reader-macros now read the
6248symbol with (READTABLE-CASE :PRESERVE) in effect.</para>
6249        <para>This means that it's necessary to type the foreign constant or
6250function name in correct case, but it isn't necessary to use any
6251special escaping constructs when writing the variable name. For instance:</para>
6252        <programlisting>
6253(#_read fd buf n) ; refers to foreign symbol "read"
6254(#_READ fd buf n) ; refers to foreign symbol "READ", which may
6255; not exist ...
6256#$o_rdonly ; Probably doesn't exist
6257#$O_RDONLY ; Exists on most platforms
6258</programlisting>
6259      </sect2>
6260
6261      <sect2 id="Foreign-type--record--and-field-names">
6262        <para>Foreign type, record, and field names
6263Constructs like RLET expect a foreign type or record name to be
6264denoted by a symbol (typically a keyword); RREF (and PREF) expect an
6265"accessor" form, typically a keyword formed by concatenating a
6266foreign type or record name with a sequence of one or more foreign field
6267names, separated by dots. These names are interned by the reader as other
6268lisp symbols are, with an arbitrary value of READTABLE-CASE in effect
6269(typically :UPCASE.) It seems like it would be very tedious to force users
6270to manually escape (via vertical bar or backslash syntax) all lowercase
6271characters in symbols used to specify foreign type, record, and field
6272names (especially given that many traditional POSIX structure, type, and
6273field names are entirely lowercase.)</para>
6274        <para>The approach taken by OpenMCL is to allow the symbols (keywords)
6275used to denote foreign type, record, and field names to contain angle
6276brackets (< and >). Such symbols are translated to foreign names
6277via the following set of conventions:</para>
6278        <listitem mark="bullet">
6279          <variablelist>All instances of < and > in the symbol's pname arebalanced and don't nest.</variablelist>
6280          <variablelist>Any alphabetic characters in the symbol's pname thataren't enclosed in angle brackets are treated as lower-case,regardless of the value of READTABLE-CASE and regardless of the casein which they were written.</variablelist>
6281          <variablelist>Alphabetic characters that appear within angle brackets aremapped to upper-case, again regardless of how they were written orinterned.</variablelist>
6282       
6283        </listitem>
6284        <para>There may be many ways of "escaping" (with angle brackets)
6285sequences of upper-case and non-lower-case characters in a symbol used to
6286denote a foreign name. When translating in the other direction, OpenMCL
6287always escapes the longest sequence that starts with an upper-case
6288character and doesn't contain a lower-case character.</para>
6289        <para>It's often preferable to use this canonical form of a foreign
6290type name.</para>
6291        <para>The accessor forms used by PREF/RREF should be viewed as a series of
6292foreign type/record and field names; upper-case sequences in the component
6293names should be escaped with angle brackets, but those sequences
6294shouldn't span components. (More simply, the separating dots
6295shouldn't be enclosed, even if both surrounding characters need to
6296be.)</para>
6297        <para>Older POSIX code tends to use lower-case exclusively for type,
6298record, and field names; there are only a few cases in the OpenMCL sources
6299where mixed-case names need to be escaped.</para>
6300      </sect2>
6301
6302      <sect2 id="Examples--1-">
6303        <para>Examples</para>
6304        <programlisting>
6305;;; Allocate a record of type "window".
6306(rlet ((w :window)) ...)
6307;;; Allocate a record of type "Window", which is probably a
6308;;;  different type
6309(rlet ((w :<w>indow)) ...)
6310;;; This is equivalent to the last example
6311(rlet ((w :<w>INDOW)))
6312</programlisting>
6313      </sect2>
6314    </sect1>
6315
6316    <sect1 id="Tutorial--Using-Basic-Calls-and-Types">
6317      <para>Tutorial: Using Basic Calls and Types
6318This tutorial is meant to cover the basics of OpenMCL for calling
6319external C functions and passing data back and forth.  These basics
6320will provide the foundation for more advanced techniques which will
6321allow access to the various external libraries and toolkits.</para>
6322      <para>The first step is to start with a simple C dynamic library in order to
6323actually observe what is actually passing between OpenMCL and C.  So,
6324some C code is in order:</para>
6325      <para>Create the file typetest.c, and put the following code into it:</para>
6326      <programlisting>
6327
6328#include <stdio.h>
6329
6330void
6331void_void_test(void)
6332{
6333    printf("Entered %s:\n", __FUNCTION__);
6334    printf("Exited  %s:\n", __FUNCTION__);
6335}
6336
6337signed char
6338sc_sc_test(signed char data)
6339{
6340    printf("Entered %s:\n", __FUNCTION__);
6341    printf("Data In: %d\n", (signed int)data);
6342    printf("Exited  %s:\n", __FUNCTION__);
6343    return data;
6344}
6345
6346unsigned char
6347uc_uc_test(unsigned char data)
6348{
6349    printf("Entered %s:\n", __FUNCTION__);
6350    printf("Data In: %d\n", (signed int)data);
6351    printf("Exited  %s:\n", __FUNCTION__);
6352    return data;
6353}
6354</programlisting>
6355      <para>This defines three functions.  If you're familiar with C, notice
6356that there's no <literal>main()</literal>, because we're just
6357building a library, not an executable.</para>
6358      <para>The function <literal>void_void_test()</literal> doesn't take
6359any parameters, and doesn't return anything, but it prints two
6360lines to let us know it was called.
6361<literal>sc_sc_test()</literal> takes a signed char as a
6362parameter, prints it, and returns it.
6363<literal>uc_uc_test()</literal> does the same thing, but with
6364an unsigned char.  Their purpose is just to prove to us
6365that we really can call C functions, pass them values, and get
6366values back from them.</para>
6367      <para><inlinemediaobject name="arb36"></inlinemediaobject></para>
6368      <para>This code is compiled into a dynamic library on OS X 10.3.4 with the
6369command:</para>
6370      <programlisting>
6371
6372gcc -dynamiclib -Wall -o libtypetest.dylib typetest.c \
6373    -install_name ./libtypetest.dylib
6374</programlisting>
6375      <para>The -dynamiclib tells gcc that we will be compiling this into a
6376dynamic library and not an executable binary program.  The output
6377filename is "libtypetest.dylib".  Notice that we chose a name which
6378follows the normal OS X convention, being in the form
6379"libXXXXX.dylib", so that other programs can link to the library.
6380OpenMCL doesn't need it to be this way, but it is a good idea to
6381adhere to existing conventions.</para>
6382      <para>The -install_name flag is primarily used when
6383building OS X "bundles".  In this case, we are not using it, so we
6384put a placeholder into it, "./libtypetest.dylib".  If we wanted
6385to use typetest in a bundle, the -install_name argument would
6386be a relative path from some "current" directory.</para>
6387      <para>After creating this library, the first step is to tell OpenMCL to open
6388the dynamic library.  This is done by calling
6389.</para>
6390      <programlisting>
6391
6392Welcome to OpenMCL Version (Beta: Darwin) 0.14.2-040506!
6393
6394? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
6395#<SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
6396</programlisting>
6397      <para>You should use an absolute path here; using a relative one, such
6398as just "libtypetest.dylib", would appear to work, but there are
6399subtle problems which occur after reloading it.  See the Darwin
6400notes on
6401 for details.  It would be
6402a bad idea anyway, because software should never rely on its
6403starting directory being anything in particular.</para>
6404      <para>This command returns a reference to the opened shared library, and
6405OpenMCL also adds one to the global variable
6406<literal>ccl::*shared-libraries*</literal>:</para>
6407      <programlisting>
6408
6409? ccl::*shared-libraries*
6410(#<SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
6411 #<SHLIB /usr/lib/libSystem.B.dylib #x606179E>)
6412</programlisting>
6413      <para>Before we call anything, let's check that the individual functions can
6414actually be found by the system.  We don't have to do this, but
6415it helps to know how to find out whether this is the problem,
6416when something goes wrong.
6417We use :</para>
6418      <programlisting>
6419
6420? (external "_void_void_test")
6421#<EXTERNAL-ENTRY-POINT "_void_void_test" (#x000CFDF8) /Users/andewl/openmcl/libtypetest.dylib #x638EDF6>
6422
6423? (external "_sc_sc_test")
6424#<EXTERNAL-ENTRY-POINT "_sc_sc_test" (#x000CFE50) /Users/andewl/openmcl/libtypetest.dylib #x638EB3E>
6425
6426? (external "_uc_uc_test")
6427#<EXTERNAL-ENTRY-POINT "_uc_uc_test" (#x000CFED4) /Users/andewl/openmcl/libtypetest.dylib #x638E626>
6428</programlisting>
6429      <para>Notice that the actual function names have
6430been "mangled" by the C linker.  The first function was named
6431"void_void_test" in typetest.c, but in
6432libtypetest.dylib, it has an underscore (a "_" symbol) before
6433it: "_void_void_test".  So, this is the name which you have to use.
6434The mangling - the way the name is changed - may be different for
6435other operating systems or other versions, so you need to "just know"
6436how it's done...</para>
6437      <para>Also, pay particular attention to the fact
6438that a hexadecimal value appears in the EXTERNAL-ENTRY-POINT.
6439(#x000CFDF8, for example - but what it is doesn't matter.)
6440These hex numbers mean that the
6441function can be dereferenced.  Functions which aren't found will
6442not have a hex number.  For example:</para>
6443      <programlisting>
6444
6445? (external "functiondoesnotexist")
6446#<EXTERNAL-ENTRY-POINT "functiondoesnotexist" {unresolved}  #x638E3F6>
6447</programlisting>
6448      <para>The "unresolved" tells us that OpenMCL wasn't able to find this
6449function, which means you would get an error, "Can't resolve foreign
6450symbol," if you tried to call it.</para>
6451      <para>These
6452external function references also are stored in a hash table which is
6453accessible through a global variable, <literal>ccl::*eeps*</literal>.</para>
6454      <para>At this point, we are ready to try our first external function
6455call:</para>
6456      <programlisting>
6457
6458? (external-call "_void_void_test" :void)
6459Entered void_void_test:
6460Exited  void_void_test:
6461NIL
6462</programlisting>
6463      <para>We used , which is
6464is the normal mechanism for
6465accessing externally linked code.  The "_void_void_test"
6466is the mangled name of the external function.
6467The :void
6468refers to the return type of the function.</para>
6469      <para>If you're using ILISP to run OpenMCL inside of Emacs, you won't
6470see the "Entered" and "Exited" lines until you quit (as of
6471July 2004).  It's not
6472clear why this is, but it's a pity.  If you want to see them, run
6473OpenMCL from Terminal.app or in some other way.</para>
6474      <para>The next step is to try passing a value to C, and getting one
6475back:</para>
6476      <programlisting>
6477
6478? (external-call "_sc_sc_test" :signed-byte -128 :signed-byte)
6479Entered sc_sc_test:
6480Data In: -128
6481Exited  sc_sc_test:
6482-128
6483</programlisting>
6484      <para>The first :signed-byte gives the type of the first argument,
6485and then -128 gives the value to pass for it.  The second
6486:signed-byte
6487gives the return type.  The return type is always given by the
6488last argument to .</para>
6489      <para>Everything looks good.  Now, let's try a number outside
6490the range which fits in one byte:</para>
6491      <programlisting>
6492
6493? (external-call "_sc_sc_test" :signed-byte -567 :signed-byte)
6494Entered sc_sc_test:
6495Data In: -55
6496Exited  sc_sc_test:
6497-55
6498</programlisting>
6499      <para>Hmmmm.  A little odd.  Let's look at the unsigned stuff
6500to see how it reacts:</para>
6501      <programlisting>
6502
6503? (external-call "_uc_uc_test" :unsigned-byte 255 :unsigned-byte)
6504Entered uc_uc_test:
6505Data In: 255
6506Exited  uc_uc_test:
6507255
6508</programlisting>
6509      <para>That looks okay.  Now, let's go outside the valid range again:</para>
6510      <programlisting>
6511
6512? (external-call "_uc_uc_test" :unsigned-byte 567 :unsigned-byte)
6513Entered uc_uc_test:
6514Data In: 55
6515Exited  uc_uc_test:
651655
6517
6518? (external-call "_uc_uc_test" :unsigned-byte -567 :unsigned-byte)
6519Entered uc_uc_test:
6520Data In: 201
6521Exited  uc_uc_test:
6522201
6523</programlisting>
6524      <para>Since a signed byte can only hold values from -128 through 127, and
6525an unsigned one can only hold values from 0 through 255, any number
6526outside that range gets "clipped": only the low eight bits of it
6527are used.</para>
6528      <para>What is important to remember is that <emphasis>external
6529function calls have
6530very few safety checks.</emphasis>
6531Data outside the valid range for its type will silently do
6532very strange things; pointers outside the valid range can very well
6533crash the system.</para>
6534      <para>That's it for our first example library.  If you're still following
6535along, let's add some more C code to look at the rest of the
6536primitive types.  Then we'll need to recompile the dynamic library,
6537load it again, and then we can see what happens.</para>
6538      <para>Add the following code to typetest.c:</para>
6539      <programlisting>
6540
6541int
6542si_si_test(int data)
6543{
6544    printf("Entered %s:\n", __FUNCTION__);
6545    printf("Data In: %d\n", data);
6546    printf("Exited  %s:\n", __FUNCTION__);
6547    return data;
6548}
6549
6550long
6551sl_sl_test(long data)
6552{
6553    printf("Entered %s:\n", __FUNCTION__);
6554    printf("Data In: %ld\n", data);
6555    printf("Exited  %s:\n", __FUNCTION__);
6556    return data;
6557}
6558
6559long long
6560sll_sll_test(long long data)
6561{
6562    printf("Entered %s:\n", __FUNCTION__);
6563    printf("Data In: %lld\n", data);
6564    printf("Exited  %s:\n", __FUNCTION__);
6565    return data;
6566}
6567
6568float
6569f_f_test(float data)
6570{
6571    printf("Entered %s:\n", __FUNCTION__);
6572    printf("Data In: %e\n", data);
6573    printf("Exited  %s:\n", __FUNCTION__);
6574    return data;
6575}
6576
6577double
6578d_d_test(double data)
6579{
6580    printf("Entered %s:\n", __FUNCTION__);
6581    printf("Data In: %e\n", data);
6582    printf("Exited  %s:\n", __FUNCTION__);
6583    return data;
6584}
6585</programlisting>
6586      <para>The command line to compile the dynamic library is the same as before:</para>
6587      <programlisting>
6588
6589gcc -dynamiclib -Wall -o libtypetest.dylib typetest.c \
6590    -install_name ./libtypetest.dylib
6591</programlisting>
6592      <para>Now, restart OpenMCL.  This step is required because OpenMCL cannot
6593close and reload a dynamic library on OS X.</para>
6594      <para>Have you restarted?  Okay, try out the new code:</para>
6595      <programlisting>
6596
6597Welcome to OpenMCL Version (Beta: Darwin) 0.14.2-040506!
6598
6599? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
6600#<SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
6601
6602? (external-call "_si_si_test" :signed-fullword -178965 :signed-fullword)
6603Entered si_si_test:
6604Data In: -178965
6605Exited  si_si_test:
6606-178965
6607
6608? ;; long is the same size as int
6609(external-call "_sl_sl_test" :signed-fullword -178965 :signed-fullword)
6610Entered sl_sl_test:
6611Data In: -178965
6612Exited  sl_sl_test:
6613-178965
6614
6615? (external-call "_sll_sll_test"
6616                 :signed-doubleword -973891578912 :signed-doubleword)
6617Entered sll_sll_test:
6618Data In: -973891578912
6619Exited  sll_sll_test:
6620-973891578912
6621</programlisting>
6622      <para>Okay, everything seems to be acting as expected.  However, just to
6623remind you that most of this stuff has no safety net, here's
6624what happens if somebody mistakes <literal>sl_sl_test()</literal>
6625for <literal>sll_sll_test()</literal>, thinking that a long is
6626actually a doubleword:</para>
6627      <programlisting>
6628
6629? (external-call "_sl_sl_test"
6630                 :signed-doubleword -973891578912 :signed-doubleword)
6631Entered sl_sl_test:
6632Data In: -227
6633Exited  sl_sl_test:
6634-974957576192
6635</programlisting>
6636      <para>Ouch.  The C function changes the value with no warning that something
6637is wrong.  Even worse, it manages to pass the original value back to
6638OpenMCL, which hides the fact that something is wrong.</para>
6639      <para>Finally, let's take a look at doing this with floating-point numbers.</para>
6640      <programlisting>
6641
6642Welcome to OpenMCL Version (Beta: Darwin) 0.14.2-040506!
6643
6644? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib")
6645#<SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E>
6646
6647? (external-call "_f_f_test" :single-float -1.256791e+11 :single-float)
6648Entered f_f_test:
6649Data In: -1.256791e+11
6650Exited  f_f_test:
6651-1.256791E+11
6652
6653? (external-call "_d_d_test" :double-float -1.256791d+290 :double-float)
6654Entered d_d_test:
6655Data In: -1.256791e+290
6656Exited  d_d_test:
6657-1.256791D+290
6658</programlisting>
6659      <para>Notice that the number ends with "...e+11" for the single-float,
6660and "...d+290" for the
6661double-float.  Lisp has both of these float types itself, and the
6662d instead of the e is how you specify which to create.  If
6663you tried to pass :double-float 1.0e2 to external-call, Lisp would
6664be nice enough to notice and give you a type error.  Don't get the
6665:double-float wrong, though, because then there's no protection.</para>
6666      <para>Congratulations!  You now know how to call external C functions from
6667within OpenMCL, and pass numbers back and forth.  Now that the basic
6668mechanics of calling and passing work, the next step is to examine how
6669to pass more complex data structures around.</para>
6670
6671      <sect2 id="Acknowledgement">
6672        <para>Acknowledgement
6673This chapter was generously contributed by
6674Andrew P. Lentvorski Jr.</para>
6675      </sect2>
6676    </sect1>
6677
6678    <sect1 id="Tutorial--Allocating-Foreign-Data-on-the-Lisp-Heap">
6679      <para>Tutorial: Allocating Foreign Data on the Lisp Heap
6680Not every foreign function is so marvelously easy to use as the
6681ones we saw in the last section.  Some of them require you to
6682allocate a C struct, fill it in with your own information, and
6683pass it a pointer to the struct.  Some of them require you to
6684allocate an empty struct so they can fill it in, and then you can
6685read the information out of it.</para>
6686      <para>Also, some of them have their own structs and return a pointer to
6687that same struct every time you call them, but those are easier to
6688deal with,
6689so they won't
6690be covered in this section.</para>
6691      <para>You might know that Lisp (and, indeed, most programming languages)
6692has two separate regions of memory.  There's the stack, which is
6693where variable bindings are kept.  Memory on the stack is allocated
6694every time any function is called, and deallocated when it returns,
6695so it's useful for anything that doesn't need to last longer than
6696one function call, when there's only one thread.  If that's all
6697you need, you can do it with
6698.</para>
6699      <para>Then, there's the heap, which holds everything else, and is our
6700subject here.
6701There are two advantages and one big disadvantage to putting things on
6702the heap rather than the stack.  First, data allocated on the heap can
6703be passed outside of the scope in which it was created.  This is
6704useful for data which may need to be passed between multiple C calls
6705or multiple threads. Also, some data may be too large to copy multiple
6706times or may be too large to allocate on the stack.</para>
6707      <para>The second advantage is security.  If incoming data is being placed
6708directly onto the stack, the input data can cause stack overflows and
6709underflows.  This is not something which Lisp users generally worry
6710about since garbage collection generally handles memory management.
6711However, "stack smashing" is one of the classic exploits in C which
6712malicious hackers can use to gain control of a machine.  Not checking
6713external
6714data is always a bad idea; however, allocating it into the heap at
6715least offers more protection than direct stack allocation.</para>
6716      <para>The big disadvantage to allocating data on the heap is that it must be
6717explicitly deallocated - you need to "free" it when you're done with
6718it.  Ordinarily, in Lisp, you wouldn't allocate memory yourself, and
6719the garbage collector would know about it, so you wouldn't have to
6720think about it again.  When you're doing it manually, it's very
6721different.
6722Memory management becomes a manual process, just like in C and
6723C++.</para>
6724      <para>What that means is that, if you allocate something and then lose
6725track of the pointer to it,
6726there's no way to ever free that
6727memory.  That's what's called a memory leak, and if your program
6728leaks enough memory it will eventually use up all of it!  So, you
6729need to be careful to not lose your pointers.</para>
6730      <para>That disadvantage, though, is also an advantage for using
6731foreign functions.  Since the garbage collector doesn't know about
6732this memory, it will never move it around.  External C code
6733needs this, because it doesn't know how to follow it to where it
6734moved, the way that Lisp code does.  If you allocate data manually,
6735you can pass it to foreign code and know that no matter what that
6736code needs to do with it, it will be able to, until you
6737deallocated it.  Of course, you'd better be sure it's done before
6738you do.  Otherwise, your program will be unstable and might crash
6739sometime in the future, and you'll have trouble figuring out what
6740caused
6741the trouble, because there won't be anything pointing
6742back and saying "you deallocated this too soon."</para>
6743      <para>And, so, on to the code...</para>
6744      <para>As in the last tutorial, our first step
6745is to create a local dynamic library in order to help show
6746what is actually going on between OpenMCL and C.  So, create the file
6747ptrtest.c, with the following code:</para>
6748      <programlisting>
6749#include <stdio.h>
6750
6751void reverse_int_array(int * data, unsigned int dataobjs)
6752{
6753  int i, t;
6754
6755  for(i=0; i<dataobjs/2; i++)
6756    {
6757      t = *(data+i);
6758      *(data+i) = *(data+dataobjs-1-i);
6759      *(data+dataobjs-1-i) = t;
6760    }
6761}
6762
6763void reverse_int_ptr_array(int **ptrs, unsigned int ptrobjs)
6764{
6765  int *t;
6766  int i;
6767
6768  for(i=0; i<ptrobjs/2; i++)
6769    {
6770      t = *(ptrs+i);
6771      *(ptrs+i) = *(ptrs+ptrobjs-1-i);
6772      *(ptrs+ptrobjs-1-i) = t;
6773    }
6774}
6775
6776void
6777reverse_int_ptr_ptrtest(int **ptrs)
6778{
6779  reverse_int_ptr_array(ptrs, 2);
6780
6781  reverse_int_array(*(ptrs+0), 4);
6782  reverse_int_array(*(ptrs+1), 4);
6783}
6784</programlisting>
6785      <para>This defines three functions.  <literal>reverse_int_array</literal>
6786takes a pointer to an array of <literal>int</literal>s, and a count
6787telling how many items are in the array, and loops through it
6788putting the elements in reverse.
6789<literal>reverse_int_ptr_array</literal> does the same thing,
6790but with an array of pointers to <literal>int</literal>s.  It only
6791reverses the order the pointers are in; each pointer still points to
6792the same thing.
6793<literal>reverse_int_ptr_ptrtest</literal>
6794takes an array of pointers to arrays of <literal>int</literal>s.  (With me?)
6795It doesn't need to be told their sizes;
6796it just assumes that the array of pointers has two items,
6797and that both of those are arrays which have four items.  It
6798reverses the array of pointers, then it reverses each of the two
6799arrays of <literal>int</literal>s.</para>
6800      <para>Now, compile ptrtest.c into a dynamic library using the command:</para>
6801      <programlisting>
6802gcc -dynamiclib -Wall -o libptrtest.dylib ptrtest.c -install_name ./libptrtest.dylib
6803</programlisting>
6804      <para>If that command doesn't make sense to you, feel free to go back
6805and read about it at .</para>
6806      <para>Now, start OpenMCL and enter:</para>
6807      <programlisting>
6808? ;; make-heap-ivector courtesy of Gary Byers
6809(defun make-heap-ivector (element-count element-type)
6810  (let* ((subtag (ccl::element-type-subtype element-type)))
6811    (unless (= (logand subtag target::fulltagmask)
6812               target::fulltag-immheader)
6813      (error "~s is not an ivector subtype." element-type))
6814    (let* ((size-in-bytes (ccl::subtag-bytes subtag element-count)))
6815      (ccl::%make-heap-ivector subtag size-in-bytes element-count))))
6816MAKE-HEAP-IVECTOR
6817
6818? ;; dispose-heap-ivector created for symmetry
6819(defmacro dispose-heap-ivector (a mp)
6820  `(progn
6821     (ccl::%dispose-heap-ivector ,a)
6822     ;; Demolish the arguments for safety
6823     (setf ,a nil)
6824     (setf ,mp nil)))
6825DISPOSE-HEAP-IVECTOR
6826</programlisting>
6827      <para>You don't understand how those functions do what they do.  That's
6828okay; it gets into very fine detail which really doesn't matter,
6829because you don't need to change them.</para>
6830      <para>The function <literal>make-heap-ivector</literal> is the primary
6831tool for
6832allocating objects in heap memory.  It allocates a fixed-size OpenMCL
6833object in heap memory.  It returns both an array reference, which can
6834be used directly from OpenMCL, and a <literal>macptr</literal>, which can
6835be used to access the underlying memory directly.  For example:</para>
6836      <programlisting>
6837? ;; Create an array of 3 4-byte-long integers
6838(multiple-value-bind (la lap)
6839    (make-heap-ivector 3 '(unsigned-byte 32))
6840  (setq a la)
6841  (setq ap lap))
6842;Compiler warnings :
6843;   Undeclared free variable A, in an anonymous lambda form.
6844;   Undeclared free variable AP, in an anonymous lambda form.
6845#<A Mac Pointer #x10217C>
6846
6847? a
6848#(1396 2578 97862649)
6849
6850? ap
6851#<A Mac Pointer #x10217C>
6852</programlisting>
6853      <para>It's important to realize that the contents of the
6854<literal>ivector</literal> we've just created
6855haven't been initialized,
6856so their values are unpredictable,
6857and you should be sure not to
6858read from them before you set them, to avoid confusing results.</para>
6859      <para>At this point, <literal>a</literal> references an object which
6860works just
6861like a normal array.  You can refer to any item of it with
6862the standard <literal>aref</literal> function, and set them
6863by combining that with <literal>setf</literal>.
6864As noted
6865above, the <literal>ivector</literal>'s
6866contents haven't been initialized,
6867so that's the next order of business:</para>
6868      <programlisting>
6869? a
6870#(1396 2578 97862649)
6871
6872? (aref a 2)
687397862649
6874
6875? (setf (aref a 0) 3)
68763
6877
6878? (setf (aref a 1) 4)
68794
6880
6881? (setf (aref a 2) 5)
68825
6883
6884? a
6885#(3 4 5)
6886</programlisting>
6887      <para>In addition, the <literal>macptr</literal> allows direct access to the same
6888memory:</para>
6889      <programlisting>
6890? (setq *byte-length-of-long* 4)
68914
6892
6893? (%get-signed-long ap (* 2 *byte-length-of-long*))
68945
6895
6896? (%get-signed-long ap (* 0 *byte-length-of-long*))
68973
6898
6899? (setf (%get-signed-long ap (* 0 *byte-length-of-long*)) 6)
69006
6901
6902? (setf (%get-signed-long ap (* 2 *byte-length-of-long*)) 7)
69037
6904
6905? ;; Show that a actually got changed through ap
6906a
6907#(6 4 7)
6908</programlisting>
6909      <para>So far, there is nothing about this object that could not be done much
6910better with standard Lisp.  However, the <literal>macptr</literal> can be
6911used to pass this chunk of memory off to a C function.  Let's use
6912the C
6913code to reverse the elements in the array:</para>
6914      <programlisting>
6915? ;; Insert the full path to your copy of libptrtest.dylib
6916(open-shared-library "/Users/andrewl/openmcl/openmcl/gtk/libptrtest.dylib")
6917#<SHLIB /Users/andrewl/openmcl/openmcl/gtk/libptrtest.dylib #x639D1E6>
6918
6919? a
6920#(6 4 7)
6921
6922? ap
6923#<A Mac Pointer #x10217C>
6924
6925? (external-call "_reverse_int_array" :address ap :unsigned-int (length a) :address)
6926#<A Mac Pointer #x10217C>
6927
6928? a
6929#(7 4 6)
6930
6931? ap
6932#<A Mac Pointer #x10217C>
6933</programlisting>
6934      <para>The array gets passed correctly to the C function,
6935<literal>reverse_int_array</literal>.  The C function
6936reverses the contents of the array in-place; that is, it doesn't make
6937a new array, just keeps the same one and reverses what's in it.
6938Finally, the C function
6939passes control back to OpenMCL.  Since the allocated array memory has
6940been directly modifed, OpenMCL reflects those changes directly in the
6941array as well.</para>
6942      <para>There is one final bit of housekeeping to deal with.  Before moving
6943on, the memory needs to be deallocated:</para>
6944      <programlisting>
6945? ;; dispose-heap-ivector created for symmetry
6946;; Macro repeated here for pedagogy
6947(defmacro dispose-heap-ivector (a mp)
6948  `(progn
6949     (ccl::%dispose-heap-ivector ,a)
6950     ;; Demolish the arguments for safety
6951     (setf ,a nil)
6952     (setf ,mp nil)))
6953DISPOSE-HEAP-IVECTOR
6954
6955? (dispose-heap-ivector a ap)
6956NIL
6957
6958? a
6959NIL
6960
6961? ap
6962NIL
6963</programlisting>
6964      <para>The <literal>dispose-heap-ivector</literal> macro actually
6965deallocates the
6966ivector, releasing its memory into the heap for something else to
6967use.  In addition, it makes sure
6968that the
6969variables which it was called with are set to nil, because otherwise
6970they would still be referencing the memory of the ivector - which is
6971no longer allocated, so that would be a bug.  Making sure there are
6972no other variables set to it is up to you.</para>
6973      <para>When do you call <literal>dispose-heap-ivector</literal>?
6974Anytime after you know the ivector will never be used again, but no
6975sooner.  If you have a lot of ivectors, say, in a hash table,</