source: trunk/source/doc/src/using.xml @ 13790

Last change on this file since 13790 was 13790, checked in by gb, 9 years ago

Mention (most of) DIRECTORY's keyword arguments. Closes ticket:692.

File size: 112.7 KB
Line 
1<?xml version="1.0" encoding="utf-8"?>
2<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"[
3          <!ENTITY rest "<varname>&amp;rest</varname>">
4          <!ENTITY key "<varname>&amp;key</varname>">
5          <!ENTITY optional "<varname>&amp;optional</varname>">
6          <!ENTITY body "<varname>&amp;body</varname>">
7          <!ENTITY aux "<varname>&amp;aux</varname>">
8          <!ENTITY allow-other-keys "<varname>&amp;allow-other-keys</varname>">
9          <!ENTITY CCL "Clozure CL">
10          ]>
11
12<chapter id="using-ccl"><title>Using &CCL;</title>
13 
14  <!-- ============================================================ -->
15  <sect1 id="using-ccl-introduction"><title>Introduction</title>
16   
17    <para>The Common Lisp standard allows considerable latitude in the
18      details of an implementation, and each particular Common Lisp
19      system has some idiosyncrasies. This chapter describes ordinary
20      user-level features of &CCL;, including features that may be
21      part of the Common Lisp standard, but which may have quirks or
22      details in the &CCL; implementation that are not described by
23      the standard. It also describes extensions to the standard; that
24      is, features of &CCL; that are not part of the Common Lisp
25      standard at all.</para>
26
27  </sect1>
28
29  <!-- ============================================================ -->
30  <sect1 id="Trace"><title>Trace</title>
31
32    <para>
33      &CCL;'s tracing facility is invoked by an extended version of the Common Lisp
34      <varname>trace</varname> macro.  Extensions allow tracing of methods, as well as finer control
35      over tracing actions.
36    </para>
37
38    <para>
39      <indexterm zone="trace"/>
40      <command><varname id="trace">TRACE</varname> {<replaceable>keyword</replaceable>
41        <replaceable>global-value</replaceable>}* {<replaceable>spec</replaceable> |
42        (<replaceable>spec</replaceable> {<replaceable>keyword</replaceable>
43        <replaceable>local-value</replaceable>}*)}* [Macro]</command>
44    </para>
45
46    <para>
47      The <varname>trace</varname> macro encapsulates the functions named by
48      <replaceable>spec</replaceable>s, causing trace actions to take place on entry and
49      exit from each function.  The default actions print a message on function entry and
50      exit. <replaceable>Keyword</replaceable>/<replaceable>value</replaceable> options
51      can be used to specify changes in the default behavior.
52    </para>
53
54    <para>
55      Invoking <varname>(trace)</varname> without arguments returns a list of functions being traced.
56    </para>
57   
58    <para>
59      A <replaceable>spec</replaceable> is either a symbol that is the name of a function, or an
60      expression of the form <varname>(setf <replaceable>symbol</replaceable>)</varname>, or a
61      specific method of a generic function in the form <varname>(:method
62        <replaceable>gf-name</replaceable> {<replaceable>qualifier</replaceable>}*
63        ({<replaceable>specializer</replaceable>}*))</varname>, where a
64      <replaceable>specializer</replaceable> can be the name of a class or an <varname>EQL</varname>
65      specializer.
66    </para>
67
68    <para>
69      A <replaceable>spec</replaceable> can also be a string naming a package, or equivalently a
70      list <varname>(:package <replaceable>package-name</replaceable>)</varname>, in order to
71      request that all functions in the package to be traced.
72    </para>
73
74    <para>
75      By default, whenever a traced function is entered or exited, a short message is
76      printed on <varname>*trace-output*</varname> showing the arguments on entry and
77      values on exit.  Options specified as key/value pairs can be used to modify this
78      behavior.  Options preceding the function <replaceable>spec</replaceable>s apply to
79      all the functions being traced.  Options specified along with a
80      <replaceable>spec</replaceable> apply to that spec only and override any
81      global options. The following options are supported:
82    </para>
83
84    <variablelist>
85
86      <varlistentry>
87            <term><varname>:methods {T | nil}</varname></term>
88            <listitem>
89              <para> If true, and if applied to a <replaceable>spec</replaceable> naming a generic
90                function, arranges to trace all the methods of the generic function in addition to the
91                generic function itself.
92              </para>
93            </listitem>
94      </varlistentry>
95
96      <varlistentry>
97            <term><varname>:inside <replaceable>outside-spec</replaceable>
98                | ({<replaceable>outside-spec</replaceable>}*)</varname></term>
99            <listitem>
100              <para>Inhibits all trace actions unless the current
101                invocation of the function being traced is inside one of the
102                <replaceable>outside-spec</replaceable>'s, i.e. unless a function named by one of the
103                <replaceable>outside-spec</replaceable>'s is currently on the stack.
104                <replaceable>outside-spec</replaceable> can name a function, a
105                method, or a package, as above.
106              </para>
107            </listitem>
108      </varlistentry>
109
110      <varlistentry>
111            <term><varname>:if <replaceable>form</replaceable></varname></term>
112            <term><varname>:condition <replaceable>form</replaceable></varname></term>
113            <listitem>
114              <para> Evaluates <replaceable>form</replaceable> whenever the function being traced is
115                about to be entered, and inhibits all trace actions if <replaceable>form</replaceable>
116                returns nil. The form may reference the lexical variable <varname>ccl::args</varname>,
117                which is a list of the arguments in this call. <varname>:condition</varname> is just a
118                synonym for <varname>:if</varname>, though if both are specified, both must return non-nil.
119              </para>
120            </listitem>
121      </varlistentry>
122
123      <varlistentry>
124            <term><varname>:before-if <replaceable>form</replaceable></varname></term>
125            <listitem>
126              <para> Evaluates <replaceable>form</replaceable> whenever the function being traced is
127                about to be entered, and inhibits the entry trace actions if
128                <replaceable>form</replaceable> returns nil.  The form may reference the lexical variable
129                <varname>ccl::args</varname>, which is a list of the arguments in this call. If both
130                <varname>:if</varname> and <varname>:before-if</varname> are specified, both must return
131                non-nil in order for the before entry actions to happen.
132              </para>
133            </listitem>
134      </varlistentry>
135
136      <varlistentry>
137            <term><varname>:after-if <replaceable>form</replaceable></varname></term>
138            <listitem>
139              <para> Evaluates <replaceable>form</replaceable> whenever the function being traced has
140                just exited, and inhibits the exit trace actions if <replaceable>form</replaceable>
141                returns nil.  The form may reference the lexical variable <varname>ccl::vals</varname>,
142                which is a list of values returned by this call. If both <varname>:if</varname> and
143                <varname>:after-if</varname> are specified, both must return non-nil in order for the
144                after exit actions to happen.
145              </para>
146            </listitem>
147      </varlistentry>
148
149      <varlistentry>
150            <term><varname>:print-before <replaceable>form</replaceable></varname></term>
151            <listitem>
152              <para> Evaluates <replaceable>form</replaceable> whenever the function being traced is
153                about to be entered, and prints the result before printing the standard entry message.
154                The form may reference the lexical variable <varname>ccl::args</varname>, which is a list
155                of the arguments in this call.  To see multiple forms, use <varname>values</varname>:
156                <varname>:print-before (values (one-thing) (another-thing))</varname>.
157              </para>
158            </listitem>
159      </varlistentry>
160
161      <varlistentry>
162            <term><varname>:print-after <replaceable>form</replaceable></varname></term>
163            <listitem>
164              <para> Evaluates <replaceable>form</replaceable> whenever the function being traced has
165                just exited, and prints the result after printing the standard exit message.  The form may
166                reference the lexical variable <varname>ccl::vals</varname>, which is a list of values
167                returned by this call. To see multiple forms, use <varname>values</varname>:
168                <varname>:print-after (values (one-thing) (another-thing))</varname>.
169              </para>
170            </listitem>
171      </varlistentry>
172
173      <varlistentry>
174            <term><varname>:print <replaceable>form</replaceable></varname></term>
175            <listitem>
176              <para>Equivalent to <varname>:print-before <replaceable>form</replaceable> :print-after <replaceable>form</replaceable></varname>.
177              </para>
178            </listitem>
179      </varlistentry>
180
181      <varlistentry>
182            <term><varname>:eval-before <replaceable>form</replaceable></varname></term>
183            <listitem>
184              <para>Evaluates <replaceable>form</replaceable> whenever the function being traced is
185                about to be entered.  The form may reference the lexical variable
186                <varname>ccl::args</varname>, which is a list of the arguments in this call.
187              </para>
188            </listitem>
189      </varlistentry>
190
191      <varlistentry>
192            <term><varname>:eval-after <replaceable>form</replaceable></varname></term>
193            <listitem>
194              <para>Evaluates <replaceable>form</replaceable> whenever the function being has just
195                exited.  The form may reference the lexical variable <varname>ccl::vals</varname>, which
196                is a list of values returned by this call.
197              </para>
198            </listitem>
199      </varlistentry>
200
201      <varlistentry>
202            <term><varname>:eval <replaceable>form</replaceable></varname></term>
203            <listitem>
204              <para>Equivalent to <varname>:eval-before <replaceable>form</replaceable>
205                  :eval-after <replaceable>form</replaceable></varname>.
206              </para>
207            </listitem>
208      </varlistentry>
209
210      <varlistentry>
211            <term><varname>:break-before <replaceable>form</replaceable></varname></term>
212            <listitem>
213              <para>Evaluates <replaceable>form</replaceable> whenever the function being traced is
214                about to be entered, and if the result is non-nil, enters a debugger break loop.  The form
215                may reference the lexical variable <varname>ccl::args</varname>, which is a list of the
216                arguments in this call.
217              </para>
218            </listitem>
219      </varlistentry>
220
221      <varlistentry>
222            <term><varname>:break-after <replaceable>form</replaceable></varname></term>
223            <listitem>
224              <para>Evaluates <replaceable>form</replaceable> whenever the function being traced has
225                just exited, and if the result is non-nil, enters a debugger break loop. The form may
226                reference the lexical variable <varname>ccl::vals</varname>, which is a list of values
227                returned by this call.
228              </para>
229            </listitem>
230      </varlistentry>
231
232      <varlistentry>
233            <term><varname>:break <replaceable>form</replaceable></varname></term>
234            <listitem>
235              <para>Equivalent to <varname>:break-before <replaceable>form</replaceable> :break-after <replaceable>form</replaceable></varname>.
236              </para>
237            </listitem>
238      </varlistentry>
239
240      <varlistentry>
241            <term><varname>:backtrace-before <replaceable>form</replaceable></varname></term>
242            <term><varname>:backtrace <replaceable>form</replaceable></varname></term>
243            <listitem>
244              <para>Evaluates <replaceable>form</replaceable> whenever the function being traced is
245                about to be entered.  The form may reference the lexical variable
246                <varname>ccl::args</varname>, which is a list of the arguments in this call. The value
247                returned by <replaceable>form</replaceable> is intepreted as follows:
248              </para>
249
250              <variablelist>
251
252                <varlistentry>
253                  <term><varname>nil</varname></term>
254                  <listitem><para>does nothing</para></listitem>
255                </varlistentry>
256
257                <varlistentry>
258                  <term><varname>:detailed</varname></term>
259                  <listitem><para>prints a detailed backtrace to
260                      <varname>*trace-output*</varname>.</para></listitem>
261                </varlistentry>
262
263                <varlistentry>
264                  <term><varname>(:detailed <replaceable>integer</replaceable>)</varname></term>
265                  <listitem><para>prints the top <replaceable>integer</replaceable> frames of detailed
266                      backtrace to <varname>*trace-output*</varname>.
267                  </para></listitem>
268                </varlistentry>
269
270                <varlistentry>
271                  <term><replaceable>integer</replaceable></term>
272                  <listitem><para>prints top <replaceable>integer</replaceable> frames of a terse
273                      backtrace to <varname>*trace-output*</varname>.
274                  </para></listitem>
275                </varlistentry>
276
277                <varlistentry>
278                  <term>anything else</term>
279                  <listitem><para>prints a terse backtrace to <varname>*trace-output*</varname>.
280                  </para></listitem>
281                </varlistentry>
282              </variablelist>
283              <para>
284                Note that unlike with the other options, <varname>:backtrace</varname> is equivalent to
285                <varname>:backtrace-before</varname> only, not both before and after, since it's usually
286                not helpful to print the same backtrace both before and after the function call.
287              </para>
288            </listitem>
289      </varlistentry>
290
291      <varlistentry>
292            <term><varname>:backtrace-after <replaceable>form</replaceable></varname></term>
293            <listitem>
294              <para>Evaluates <replaceable>form</replaceable> whenever the function being traced has
295                just exited.  The form may reference the lexical variable <varname>ccl::vals</varname>,
296                which is a list of values returned by this call. The value returned by
297                <replaceable>form</replaceable> is intepreted as follows:
298              </para>
299
300              <variablelist>
301
302                <varlistentry>
303                  <term><varname>nil</varname></term>
304                  <listitem><para>does nothing</para></listitem>
305                </varlistentry>
306
307                <varlistentry>
308                  <term><varname>:detailed</varname></term>
309                  <listitem><para>prints a detailed backtrace to
310                      <varname>*trace-output*</varname>.</para></listitem>
311                </varlistentry>
312
313                <varlistentry>
314                  <term><varname>(:detailed <replaceable>integer</replaceable>)</varname></term>
315                  <listitem><para>prints the top <replaceable>integer</replaceable> frames of detailed
316                      backtrace to <varname>*trace-output*</varname>.
317                  </para></listitem>
318                </varlistentry>
319
320                <varlistentry>
321                  <term><replaceable>integer</replaceable></term>
322                  <listitem><para>prints top <replaceable>integer</replaceable> frames of a terse
323                      backtrace to <varname>*trace-output*</varname>.
324                  </para></listitem>
325                </varlistentry>
326
327                <varlistentry>
328                  <term>anything else</term>
329                  <listitem><para>prints a terse backtrace to <varname>*trace-output*</varname>.
330                  </para></listitem>
331                </varlistentry>
332              </variablelist>
333            </listitem>
334      </varlistentry>
335
336      <varlistentry>
337        <term><varname>:before</varname> <replaceable>action</replaceable></term>
338        <listitem>
339              <para>specifies the action to be taken just before the traced function is entered.  <replaceable>action</replaceable> is one of:</para>
340
341              <variablelist>
342                <varlistentry>
343                  <term><varname>:print</varname></term>
344                  <listitem>
345                        <para>The default, prints a short indented message showing the function name and the invocation arguments</para>
346                  </listitem>
347                </varlistentry>
348
349                <varlistentry>
350                  <term><varname>:break</varname></term>
351                  <listitem>
352                        <para>Equivalent to <varname>:before :print :break-before t</varname></para>
353                  </listitem>
354                </varlistentry>
355
356                <varlistentry>
357                  <term><varname>:backtrace</varname></term>
358                  <listitem>
359                        <para>Equivalent to <varname>:before :print :backtrace-before t</varname></para>
360                  </listitem>
361                </varlistentry>
362
363
364                <varlistentry>
365                  <term><replaceable>function</replaceable></term>
366                  <listitem>
367                        <para>
368                  Any other value is interpreted as a function to call on entry instead of
369                  printing the standard entry message.  It is called with its first
370                  argument being the name of the function being traced, the remaining
371                  arguments being all the arguments to the function being traced, and
372                  <varname>ccl:*trace-level*</varname> bound to the current nesting level
373                  of trace actions. </para>
374                  </listitem>
375                </varlistentry>
376              </variablelist>
377            </listitem>
378      </varlistentry>
379
380      <varlistentry>
381
382        <term><varname>:after</varname> <replaceable>action</replaceable></term>
383        <listitem>
384
385              <para>specifies the action to be taken just after the traced function exits.  <replaceable>action</replaceable> is one of:</para>
386
387              <variablelist>
388                <varlistentry>
389                  <term><varname>:print</varname></term>
390                  <listitem>
391                    <para>The default, prints a short indented message showing the function name and the
392                      returned values </para>
393                  </listitem>
394                </varlistentry>
395
396                <varlistentry>
397                  <term><varname>:break</varname></term>
398                  <listitem>
399                    <para>Equivalent to <varname>:after :print :break-after t</varname></para>
400                  </listitem>
401                </varlistentry>
402
403                    <varlistentry>
404                      <term><varname>:backtrace</varname></term>
405                      <listitem>
406                        <para>Equivalent to <varname>:after :print :backtrace-after t</varname></para>
407                      </listitem>
408                    </varlistentry>
409
410                <varlistentry>
411                  <term><replaceable>function</replaceable></term>
412                  <listitem>
413                    <para>Any other value is interpreted as a function to
414                      call on exit instead of printing the standard exit
415                      message.  It is called with its first argument being
416                      the name of the function being traced, the
417                      remaining arguments being all the values returned by the function
418                      being traced, and ccl:*trace-level* bound to the current
419                      nesting level of trace actions.
420                    </para>
421                  </listitem>
422                </varlistentry>
423              </variablelist>
424        </listitem>
425      </varlistentry>
426
427    </variablelist>
428
429
430    <para>
431      <indexterm zone="trace-level"/>
432      <command><varname id="trace-level">CCL:*TRACE-LEVEL*</varname>    [Variable]</command>
433    </para>
434
435    <para>Variable bound to the current nesting level during execution of before and after trace actions.  The default printing actions use it to determine the amount of indentation.</para>
436
437    <para>
438      <indexterm zone="trace-max-indent"/>
439      <command><varname id="trace-max-indent">CCL:*TRACE-MAX-INDENT*</varname>    [Variable]</command>
440    </para>
441
442    <para>The default before and after print actions will not indent by more than the value of <varname>ccl:*trace-max-indent*</varname> regardless of the current trace level.</para>
443
444    <para>
445      <indexterm zone="trace-function"/>
446      <command><varname id="trace-function">CCL:TRACE-FUNCTION</varname> <replaceable>spec</replaceable> &key; {<replaceable>keyword</replaceable> <replaceable>value</replaceable>}*    [Function]</command>
447    </para>
448   
449    <para>
450      This is a functional version of the TRACE macro.  <replaceable>spec</replaceable> and
451      <replaceable>keyword</replaceable>s are as for TRACE, except that all arguments are evaluated.
452    </para>
453
454    <para>
455      <indexterm zone="trace-print-level"/>
456      <command><varname id="trace-print-level">CCL:*TRACE-PRINT-LEVEL*</varname>   [Variable]</command>
457    </para>
458
459    <para>The default print actions bind <varname>CL:*PRINT-LEVEL*</varname> to this value while
460      printing. Note that this rebinding is only in effect during the default entry and exit messages.
461      It does not apply to printing of <varname>:print-before/:print-after</varname> forms or any
462      explicit printing done by user code.</para>
463
464    <para>
465      <indexterm zone="trace-print-length"/>
466      <command><varname id="trace-print-length">CCL:*TRACE-PRINT-LENGTH*</varname>    [Variable]</command>
467    </para>
468   
469    <para>The default print actions bind <varname>CL:*PRINT-LENGTH*</varname> to this value while
470      printing. Note that this rebinding is only in effect during the default entry and exit messages.
471      It does not apply to printing of <varname>:print-before/:print-after</varname> forms or any
472      explicit printing done by user code.</para>
473
474    <para>
475      <indexterm zone="trace-bar-frequency"/>
476      <command><varname id="trace-bar-frequency">CCL:*TRACE-BAR-FREQUENCY*</varname>    [Variable]</command>
477    </para>
478
479    <para>By default, this is nil. If non-nil it should be a integer,
480    and the default entry and exit messages will print a | instead of
481    space every this many levels of indentation.</para>
482
483
484  </sect1>
485
486  <sect1 id="Directory"><title>Directory</title>
487  <para>&CCL;'s <varname>DIRECTORY</varname> function accepts the following
488implementation-dependent keyword arguments:</para>
489  <variablelist>
490  <varlistentry>
491     <term><varname>:files</varname> <replaceable>boolean</replaceable></term>
492     <listitem>
493       <para> If true, includes regular (non-directory) files in DIRECTORY's output.  Defaults to T.</para>
494     </listitem>
495  </varlistentry>
496  <varlistentry>
497     <term><varname>:directories</varname> <replaceable>boolean</replaceable></term>
498     <listitem>
499       <para> If true, includes directories in DIRECTORY's output.  Defaults to NIL.</para>
500     </listitem>
501  </varlistentry>
502  <varlistentry>
503     <term><varname>:all</varname> <replaceable>boolean</replaceable></term>
504     <listitem>
505       <para> If true, includes files and directories whose names start with a dot character in DIRECTORY's output.  (Entries whose name is "." or ".." are never included.) Defaults to T.</para>
506     </listitem>
507  </varlistentry>
508  <varlistentry>
509     <term><varname>:follow-links</varname> <replaceable>boolean</replaceable></term>
510     <listitem>
511       <para> If true, includes the TRUENAMEs of symbolic or hard links in DIRECTORY's output; if false, includes the link filenames without attempting to resolve them. Defaults to T.</para>
512       <para>Note that legacy HFS alias files are treated as plain files.</para>
513     </listitem>
514  </varlistentry>
515  </variablelist>
516  </sect1>
517  <sect1 id="Unicode"><title>Unicode</title>
518
519    <para>All characters and strings in &CCL; fully support Unicode by
520    using UTF-32. There is only one <literal>CHARACTER</literal> type
521    and one <literal>STRING</literal> type in &CCL;.  There has been a
522    lot of discussion about this decision which can be found by
523    searching the openmcl-devel archives at <ulink
524    url="http://clozure.com/pipermail/openmcl-devel/"/>.  Suffice it
525    to say that we decided that the simplicity and speed advantages of
526    only supporting UTF-32 outweigh the space disadvantage.</para>
527
528
529
530  <sect2 id="unicode-characters"><title>Characters</title>
531    <para>There is one <literal>CHARACTER</literal> type in &CCL;.
532    All <literal>CHARACTER</literal>s are
533    <literal>BASE-CHAR</literal>s.  <varname>CHAR-CODE-LIMIT</varname>
534    is now <literal>#x110000</literal>, which means that all Unicode
535    characters can be directly represented.  As of Unicode 5.0, only
536    about 100,000 of 1,114,112 possible <literal>CHAR-CODE</literal>s
537    are actually defined. The function <function>CODE-CHAR</function>
538    knows that certain ranges of code values (notably
539    <literal>#xd800</literal>-<literal>#xddff</literal>) will never be
540    valid character codes and will return <literal>NIL</literal> for
541    arguments in that range, but may return a
542    non-<literal>NIL</literal> value (an undefined/non-standard
543    <literal>CHARACTER</literal> object) for other unassigned code
544    values.</para>
545
546    <para>&CCL; supports character names of the form
547    <literal>u+xxxx</literal>&mdash;where <literal>x</literal> is a
548    sequence of one or more hex digits.  The value of the hex digits
549    denotes the code of the character.  The <literal>+</literal>
550    character is optional, so <literal>#\u+0020</literal>,
551    <literal>#\U0020</literal>, and <literal>#\U+20</literal> all
552    refer to the <literal>#\Space</literal> character.</para>
553
554    <para>Characters with codes in the range
555    <literal>#xa0</literal>-<literal>#x7ff</literal> also have
556    symbolic names These are the names from the Unicode standard with
557    spaces replaced by underscores.  So
558    <literal>#\Greek_Capital_Letter_Epsilon</literal> can be used to
559    refer to the character whose <function>CHAR-CODE</function> is
560    <literal>#x395</literal>.  To see the complete list of supported
561    character names, look just below the definition for
562    <function>register-character-name</function> in
563    <literal>ccl:level-1;l1-reader.lisp</literal>.</para>
564  </sect2>
565
566
567  <sect2 id="External-Formats"><title>External Formats</title>
568    <para><function>OPEN</function>, <function>LOAD</function>, and
569    <function>COMPILE-FILE</function> all take an
570    <literal>:EXTERNAL-FORMAT</literal> keyword argument.  The value
571    of <literal>:EXTERNAL-FORMAT</literal> can be
572    <literal>:DEFAULT</literal> (the default value), a line
573    termination keyword (see <xref
574    linkend="Line-Termination-Keywords"/>), a character encoding
575    keyword (see <xref linkend="Character-Encodings"/>), an
576    external-format object created using
577    <function>CCL::MAKE-EXTERNAL-FORMAT</function> (see <xref
578    linkend="f_make-external-format"/>), or a plist with keys:
579    <literal>:DOMAIN</literal>, <literal>:CHARACTER-ENCODING</literal>
580    and <literal>:LINE-TERMINATION</literal>.  If
581    <parameter>argument</parameter> is a plist, the result of
582    <literal>(APPLY #'MAKE-EXTERNAL-FORMAT
583    <parameter>argument</parameter>)</literal> will be used.</para>
584
585    <para>If <literal>:DEFAULT</literal> is specified, then the value
586    of <varname>CCL:*DEFAULT-EXTERNAL-FORMAT*</varname> is used.  If
587    no line-termination is specified, then the value of
588    <varname>CCL:*DEFAULT-LINE-TERMINATION*</varname> is used, which
589    defaults to <literal>:UNIX</literal>.  If no character encoding is
590    specified, then
591    <varname>CCL:*DEFAULT-FILE-CHARACTER-ENCODING*</varname> is used
592    for file streams and
593    <varname>CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING*</varname> is used
594    for socket streams.  The default, default character encoding is
595    <literal>NIL</literal> which is a synonym for
596    <literal>:ISO-8859-1</literal>.</para>
597
598    <para>Note that the set of keywords used to denote
599    CHARACTER-ENCODINGs and the set of keywords used to denote
600    line-termination conventions is disjoint: a keyword denotes at
601    most a character encoding or a line termination convention, but
602    never both.</para>
603
604    <para>EXTERNAL-FORMATs are objects (structures) with three
605    read-only fields that can be accessed via the functions:
606    <function>EXTERNAL-FORMAT-DOMAIN</function>,
607    <function>EXTERNAL-FORMAT-LINE-TERMINATION</function> and
608    <function>EXTERNAL-FORMAT-CHARACTER-ENCODING</function>.</para>
609
610 
611    <refentry id="f_make-external-format">
612      <indexterm zone="f_make-external-format">
613        <primary>make-external-format</primary>
614      </indexterm>
615     
616      <refnamediv>
617        <refname>MAKE-EXTERNAL-FORMAT</refname>
618        <refpurpose>Either creates a new external format object, or
619        return an existing one with the same specified slot
620        values.</refpurpose>
621        <refclass>Function</refclass>
622      </refnamediv>
623
624      <refsynopsisdiv>
625        <synopsis>
626          <function>make-external-format</function>
627          &key; domain character-encoding line-termination
628          => external-format
629        </synopsis>
630      </refsynopsisdiv>
631
632      <refsect1>
633        <title>Arguments and Values</title>
634       
635        <variablelist>
636          <varlistentry>
637            <term>domain</term>
638            <listitem>
639              <para>This is used to indicate where the external
640              format is to be used.  Its value can be almost
641              anything.  It defaults to <literal>NIL</literal>.
642              There are two domains that have a pre-defined meaning in
643              &CCL;: <literal>:FILE</literal> indicates
644              encoding for a file in the file system and
645              <literal>:SOCKET</literal> indicates i/o to/from a
646              socket.  The value of <parameter>domain</parameter>
647              affects the default values for
648              <parameter>character-encoding</parameter> and
649              <parameter>line-termination</parameter>.</para>
650            </listitem>
651          </varlistentry>
652          <varlistentry>
653            <term>character-encoding</term>
654            <listitem>
655              <para>A keyword that specifies the character encoding
656              for the external format. <xref
657              linkend="Character-Encodings"/>.  Defaults to
658              <literal>:DEFAULT</literal> which means if
659              <parameter>domain</parameter> is
660              <literal>:FILE</literal> use the value of the variable
661              <varname>CCL:*DEFAULT-FILE-CHARACTER-ENCODING*</varname>
662              and if <parameter>domain</parameter> is
663              <literal>:SOCKET</literal>, use the value of the
664              variable
665              <varname>CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING*</varname>.
666              The initial value of both of these variables is
667              <literal>NIL</literal>, which means the
668              <literal>:ISO-8859-1</literal> encoding.</para>
669            </listitem>
670          </varlistentry>
671          <varlistentry>
672            <term>line-termination</term>
673            <listitem>
674              <para>A keyword that indicates a line termination
675              keyword <xref linkend="Line-Termination-Keywords"/>.
676              Defaults to <literal>:DEFAULT</literal> which means
677              use the value of the variable
678              <varname>CCL:*DEFAULT-LINE-TERMINATION*</varname>.</para>
679            </listitem>
680          </varlistentry>
681          <varlistentry>
682            <term>external-format</term>
683            <listitem>
684              <para>An external-format object as described above.</para>
685            </listitem>
686          </varlistentry>
687        </variablelist>
688      </refsect1>
689     
690      <refsect1>
691        <title>Description</title>
692       
693        <para>Despite the function's name, it doesn't necessarily create a
694        new, unique EXTERNAL-FORMAT object: two calls to
695        MAKE-EXTERNAL-FORMAT with the same arguments made in the same
696        dynamic environment return the same (eq) object.
697        </para>
698      </refsect1>
699    </refentry>
700
701  </sect2>
702
703  <sect2 id="Line-Termination-Keywords"><title>Line Termination Keywords</title>
704  <para>Line termination keywords indicate which characters are used
705  to indicate the end of a line.  On input, the external line
706  termination characters are replaced by <literal>#\Newline</literal>
707  and on output, <literal>#\Newline</literal>s are converted to the
708  external line termination characters.</para>
709  <table id="Line-Termination-Table" frame='all'><title>Line Termination Keywords</title>
710  <tgroup cols='2' align='left' colsep='1' rowsep='1'>
711    <thead>
712      <row>
713        <entry>keyword</entry>
714        <entry>character(s)</entry>
715      </row>
716    </thead>
717    <tbody>
718      <row>
719        <entry><literal>:UNIX</literal></entry>
720        <entry><literal>#\Linefeed</literal></entry>
721      </row>
722      <row>
723        <entry><literal>:MACOS</literal></entry>
724        <entry><literal>#\Return</literal></entry>
725      </row>
726      <row>
727        <entry><literal>:CR</literal></entry>
728        <entry><literal>#\Return</literal></entry>
729      </row>
730      <row>
731        <entry><literal>:CRLF</literal></entry>
732        <entry><literal>#\Return #\Linefeed</literal></entry>
733      </row>
734      <row>
735        <entry><literal>:CP/M</literal></entry>
736        <entry><literal>#\Return #\Linefeed</literal></entry>
737      </row>
738      <row>
739        <entry><literal>:MSDOS</literal></entry>
740        <entry><literal>#\Return #\Linefeed</literal></entry>
741      </row>
742      <row>
743        <entry><literal>:DOS</literal></entry>
744        <entry><literal>#\Return #\Linefeed</literal></entry>
745      </row>
746      <row>
747        <entry><literal>:WINDOWS</literal></entry>
748        <entry><literal>#\Return #\Linefeed</literal></entry>
749      </row>
750      <row>
751        <entry><literal>:INFERRED</literal></entry>
752        <entry>see below</entry>
753      </row>
754      <row>
755        <entry><literal>:UNICODE</literal></entry>
756        <entry><literal>#\Line_Separator</literal></entry>
757      </row>
758    </tbody>
759  </tgroup>
760  </table>
761  <para><literal>:INFERRED</literal> means that a stream's
762  line-termination convention is determined by looking at the contents
763  of a file.  It is only useful for <literal>FILE-STREAM</literal>s
764  that're open for <literal>:INPUT</literal> or
765  <literal>:IO</literal>.  The first buffer full of data is examined,
766  and if a <literal>#\Return</literal> character occurs before any
767  <literal>#\Linefeed</literal> character, then the line termination
768  type is set to <literal>:MACOS</literal>, otherwise it is set to
769  <literal>:UNIX</literal>.</para>
770  </sect2>
771 
772
773
774  <sect2 id="Character-Encodings"><title>Character Encodings</title>
775    <para>Internally, all characters and strings in &CCL; are in
776    UTF-32.  Externally, files or socket streams may encode characters
777    in a wide variety of ways.  The International Organization for
778    Standardization, widely known as ISO, defines many of these
779    character encodings.  &CCL; implements some of these encodings as
780    detailed below.  These encodings are part of the specification of
781    external formats <xref linkend="External-Formats"/>.  When reading
782    from a stream, characters are converted from the specified
783    external character encoding to UTF-32.  When writing to a stream,
784    characters are converted from UTF-32 to the specified character
785    encoding.</para>
786
787    <para>Internally, CHARACTER-ENCODINGs are objects (structures)
788    that are named by character encoding keywords (:ISO-8859-1,
789    :UTF-8, etc.).  The structures contain attributes of the encoding
790    and functions used to encode/decode external data, but unless
791    you're trying to define or debug an encoding there's little reason
792    to know much about the CHARACTER-ENCODING objects and it's usually
793    preferable to refer to a character encoding by its name.
794    </para>
795
796    <para>
797    </para>
798
799    <sect3><title>Encoding Problems</title>
800      <para>On output to streams with character encodings that can
801      encode the full range of Unicode&mdash;and on input from any
802      stream&mdash;"unencodable characters" are represented using the
803      Unicode #\Replacement_Character (= #\U+fffd); the presence of
804      such a character usually indicates that something got lost in
805      translation.  Either data wasn't encoded properly or there was a
806      bug in the decoding process.</para>
807    </sect3>
808
809    <sect3><title>Byte Order Marks</title>
810      <para>The endianness of a character encoding is sometimes
811      explicit, and sometimes not.  For example,
812      <literal>:UTF-16BE</literal> indicates big-endian, but
813      <literal>:UTF-16</literal> does not specify endianness.  A byte
814      order mark is a special character that may appear at the
815      beginning of a stream of encoded characters to specify the
816      endianness of a multi-byte character encoding.  (It may also be
817      used with UTF-8 character encodings, where it is simply used to
818      indicate that the encoding is UTF-8.)</para>
819
820      <para>&CCL; writes a byte order mark as the first character
821      of a file or socket stream when the endianness of the character
822      encoding is not explicit.  &CCL; also expects a byte order
823      mark on input from streams where the endianness is not
824      explicit. If a byte order mark is missing from input data, that
825      data is assumed to be in big-endian order.</para>
826
827      <para>A byte order mark from a UTF-8 encoded input stream is not
828      treated specially and just appears as a normal character from
829      the input stream.  It is probably a good idea to skip over this
830      character.</para>
831    </sect3>
832
833  <sect3><title><function>DESCRIBE-CHARACTER-ENCODINGS</function></title>
834    <para>The set of character encodings supported by &CCL; can be
835    retrieved by calling
836    <function>CCL:DESCRIBE-CHARACTER-ENCODINGS</function>.</para>
837
838
839      <refentry id="f_describe-character-encodings">
840        <indexterm zone="f_describe-character-encodings">
841          <primary>[fn-name]</primary>
842        </indexterm>
843
844        <refnamediv>
845          <refname>DESCRIBE-CHARACTER-ENCODINGS</refname>
846          <refpurpose>Writes descriptions of defined character
847          encodings to <varname>*terminal-io*</varname>.</refpurpose>
848          <refclass>Function</refclass>
849        </refnamediv>
850
851        <refsynopsisdiv>
852          <synopsis>
853            <function>describe-character-encodings</function>
854          </synopsis>
855        </refsynopsisdiv>
856
857        <refsect1>
858          <title>Description</title>
859
860          <para>Writes descriptions of all defined character encodings
861          to <varname>*terminal-io*</varname>.  These descriptions
862          include the names of the encoding's aliases and a doc string
863          which briefly describes each encoding's properties and
864          intended use.</para>
865        </refsect1>
866
867        <refsect1>
868          <title>See Also</title>
869         
870          <simplelist type="inline">
871            <member><xref linkend="Character-Encodings"/></member>
872            <member><xref linkend="External-Formats"/></member>
873            <member><xref linkend="Supported-Character-Encodings"/></member>
874          </simplelist>
875        </refsect1>
876      </refentry>
877  </sect3>
878
879  <sect3 id="Supported-Character-Encodings"><title>Supported Character Encodings</title>
880     <para>The list of supported encodings is reproduced here.  Most
881     encodings have aliases, e.g. the encoding named
882     <literal>:ISO-8859-1</literal> can also be referred to by the
883     names <literal>:LATIN1</literal> and <literal>:IBM819</literal>,
884     among others.  Where possible, the keywordized name of an
885     encoding is equivalent to the preferred MIME charset name (and
886     the aliases are all registered IANA charset names.)</para>
887
888  <variablelist>
889    <varlistentry><term><literal>:ISO-8859-1</literal></term>
890       <listitem><para>An 8-bit, fixed-width character encoding in
891       which all character codes map to their Unicode
892       equivalents. Intended to support most characters used in most
893       Western European languages.</para>
894       <para>&CCL; uses ISO-8859-1 encoding for
895       <varname>*TERMINAL-IO*</varname> and for all streams whose
896       EXTERNAL-FORMAT isn't explicitly specified.  The default for
897       <varname>*TERMINAL-IO*</varname> can be set via the
898       <literal>-K</literal> command-line argument (see <xref
899       linkend="Command-Line-Options"/>).
900       </para>
901       <para>ISO-8859-1 just covers the first 256 Unicode code
902       points, where the first 128 code points are equivalent to
903       US-ASCII.  That should be pretty much equivalent to what
904       earliers versions of &CCL; did that only supported 8-bit characters,
905       but it may not be optimal for users working in a particular
906       locale.</para>
907       <para>Aliases: <literal>:ISO_8859-1, :LATIN1, :L1,
908       :IBM819, :CP819, :CSISOLATIN1</literal></para></listitem>
909     </varlistentry>
910    <varlistentry><term><literal>:ISO-8859-2</literal></term>
911       <listitem><para>An 8-bit, fixed-width character encoding in
912       which codes #x00-#x9f map to their Unicode equivalents and
913       other codes map to other Unicode character values.  Intended to
914       provide most characters found in most languages used in
915       Central/Eastern Europe.</para>
916       <para>Aliases: <literal>:ISO_8859-2, :LATIN-2, :L2,
917       :CSISOLATIN2</literal></para></listitem>
918     </varlistentry>
919    <varlistentry><term><literal>:ISO-8859-3</literal></term>
920       <listitem><para>An 8-bit, fixed-width character encoding in
921       which codes #x00-#x9f map to their Unicode equivalents and
922       other codes map to other Unicode character values.  Intended to
923       provide most characters found in most languages used in
924       Southern Europe.</para>
925       <para>Aliases: <literal>:ISO_8859-3, :LATIN,3 :L3,
926       :CSISOLATIN3</literal></para></listitem>
927     </varlistentry>
928    <varlistentry><term><literal>:ISO-8859-4</literal></term>
929       <listitem><para>An 8-bit, fixed-width character encoding in
930       which codes #x00-#x9f map to their Unicode equivalents and
931       other codes map to other Unicode character values.  Intended to
932       provide most characters found in most languages used in
933       Northern Europe.</para>
934       <para>Aliases: <literal>:ISO_8859-4, :LATIN4, :L4, :CSISOLATIN4</literal></para></listitem>
935     </varlistentry>
936    <varlistentry><term><literal>:ISO-8859-5</literal></term>
937       <listitem><para>An 8-bit, fixed-width character encoding in
938       which codes #x00-#x9f map to their Unicode equivalents and
939       other codes map to other Unicode character values.  Intended to
940       provide most characters found in the Cyrillic
941       alphabet.</para>
942       <para>Aliases: <literal>:ISO_8859-5, :CYRILLIC, :CSISOLATINCYRILLIC,
943       :ISO-IR-144</literal></para></listitem>
944     </varlistentry>
945    <varlistentry><term><literal>:ISO-8859-6</literal></term>
946       <listitem><para>An 8-bit, fixed-width character encoding in
947       which codes #x00-#x9f map to their Unicode equivalents and
948       other codes map to other Unicode character values.  Intended to
949       provide most characters found in the Arabic
950       alphabet.</para>
951       <para>Aliases: <literal>:ISO_8859-6, :ARABIC, :CSISOLATINARABIC,
952       :ISO-IR-127</literal></para></listitem>
953     </varlistentry>
954    <varlistentry><term><literal>:ISO-8859-7</literal></term>
955       <listitem><para>An 8-bit, fixed-width character encoding in
956       which codes #x00-#x9f map to their Unicode equivalents and
957       other codes map to other Unicode character values.  Intended to
958       provide most characters found in the Greek
959       alphabet.</para>
960       <para>Aliases: <literal>:ISO_8859-7, :GREEK, :GREEK8, :CSISOLATINGREEK,
961       :ISO-IR-126, :ELOT_928, :ECMA-118</literal></para></listitem>
962     </varlistentry>
963    <varlistentry><term><literal>:ISO-8859-8</literal></term>
964       <listitem><para>An 8-bit, fixed-width character encoding in
965       which codes #x00-#x9f map to their Unicode equivalents and
966       other codes map to other Unicode character values.  Intended to
967       provide most characters found in the Hebrew
968       alphabet.</para>
969       <para>Aliases: <literal>:ISO_8859-8, :HEBREW, :CSISOLATINHEBREW,
970       :ISO-IR-138</literal></para></listitem>
971     </varlistentry>
972    <varlistentry><term><literal>:ISO-8859-9</literal></term>
973       <listitem><para>An 8-bit, fixed-width character encoding in
974       which codes #x00-#xcf map to their Unicode equivalents and
975       other codes map to other Unicode character values.  Intended to
976       provide most characters found in the Turkish
977       alphabet.</para>
978       <para>Aliases: <literal>:ISO_8859-9, :LATIN5, :CSISOLATIN5,
979       :ISO-IR-148</literal></para></listitem>
980     </varlistentry>
981    <varlistentry><term><literal>:ISO-8859-10</literal></term>
982       <listitem><para>An 8-bit, fixed-width character encoding in
983       which codes #x00-#x9f map to their Unicode equivalents and
984       other codes map to other Unicode character values.  Intended to
985       provide most characters found in Nordic
986       alphabets.</para>
987       <para>Aliases: <literal>:ISO_8859-10, :LATIN6, :CSISOLATIN6,
988       :ISO-IR-157</literal></para></listitem>
989     </varlistentry>
990    <varlistentry><term><literal>:ISO-8859-11</literal></term>
991       <listitem><para>An 8-bit, fixed-width character encoding in
992       which codes #x00-#x9f map to their Unicode equivalents and
993       other codes map to other Unicode character values.  Intended to
994       provide most characters found the Thai
995       alphabet.</para></listitem>
996     </varlistentry>
997    <varlistentry><term><literal>:ISO-8859-13</literal></term>
998       <listitem><para>An 8-bit, fixed-width character encoding in
999       which codes #x00-#x9f map to their Unicode equivalents and
1000       other codes map to other Unicode character values.  Intended to
1001       provide most characters found in Baltic
1002       alphabets.</para></listitem>
1003     </varlistentry>
1004    <varlistentry><term><literal>:ISO-8859-14</literal></term>
1005       <listitem><para>An 8-bit, fixed-width character encoding in
1006       which codes #x00-#x9f map to their Unicode equivalents and
1007       other codes map to other Unicode character values.  Intended to
1008       provide most characters found in Celtic
1009       languages.</para>
1010       <para>Aliases: <literal>:ISO_8859-14, :ISO-IR-199, :LATIN8, :L8,
1011       :ISO-CELTIC</literal></para></listitem>
1012     </varlistentry>
1013    <varlistentry><term><literal>:ISO-8859-15</literal></term>
1014       <listitem><para>An 8-bit, fixed-width character encoding in
1015       which codes #x00-#x9f map to their Unicode equivalents and
1016       other codes map to other Unicode character values.  Intended to
1017       provide most characters found in Western European languages
1018       (including the Euro sign and some other characters missing from
1019       ISO-8859-1.</para>
1020       <para>Aliases: <literal>:ISO_8859-15, :LATIN9</literal></para></listitem>
1021     </varlistentry>
1022    <varlistentry><term><literal>:ISO-8859-16</literal></term>
1023       <listitem><para>An 8-bit, fixed-width character encoding in
1024       which codes #x00-#x9f map to their Unicode equivalents and
1025       other codes map to other Unicode character values.  Intended to
1026       provide most characters found in Southeast European
1027       languages.</para>
1028       <para>Aliases: <literal>:ISO_8859-16, :ISO-IR-199, :LATIN8, :L8,
1029       :ISO-CELTIC</literal></para></listitem>
1030     </varlistentry>
1031    <varlistentry><term><literal>:MACINTOSH</literal></term>
1032       <listitem><para>An 8-bit, fixed-width character encoding in
1033       which codes #x00-#x7f map to their Unicode equivalents and
1034       other codes map to other Unicode character values.
1035       Traditionally used on Classic MacOS to encode characters used
1036       in western languages.</para>
1037       <para>Aliases: <literal>:MACOS-ROMAN, :MACOSROMAN, :MAC-ROMAN,
1038       :MACROMAN</literal></para></listitem>
1039     </varlistentry>
1040    <varlistentry><term><literal>:UCS-2</literal></term>
1041       <listitem><para>A 16-bit, fixed-length encoding in which
1042       characters with CHAR-CODEs less than #x10000 can be encoded in
1043       a single 16-bit word.  The endianness of the encoded data is
1044       indicated by the endianness of a byte-order-mark character
1045       (#u+feff) prepended to the data; in the absence of such a
1046       character on input, the data is assumed to be in big-endian
1047       order.</para></listitem>
1048     </varlistentry>
1049    <varlistentry><term><literal>:UCS-2BE</literal></term>
1050       <listitem><para>A 16-bit, fixed-length encoding in which
1051       characters with CHAR-CODEs less than #x10000 can be encoded in
1052       a single 16-bit big-endian word. The encoded data is implicitly
1053       big-endian; byte-order-mark characters are not interpreted on
1054       input or prepended to output.</para></listitem>
1055     </varlistentry>
1056    <varlistentry><term><literal>:UCS-2LE</literal></term>
1057       <listitem><para>A 16-bit, fixed-length encoding in which
1058       characters with CHAR-CODEs less than #x10000 can be encoded in
1059       a single 16-bit little-endian word. The encoded data is
1060       implicitly little-endian; byte-order-mark characters are not
1061       interpreted on input or prepended to output.</para></listitem>
1062     </varlistentry>
1063    <varlistentry><term><literal>:US-ASCII</literal></term>
1064       <listitem><para>An 7-bit, fixed-width character encoding in
1065       which all character codes map to their Unicode
1066       equivalents. </para>
1067       <para>Aliases: <literal>:CSASCII, :CP63,7 :IBM637, :US,
1068       :ISO646-US, :ASCII, :ISO-IR-6</literal></para></listitem>
1069     </varlistentry>
1070    <varlistentry><term><literal>:UTF-16</literal></term>
1071       <listitem><para>A 16-bit, variable-length encoding in which
1072       characters with CHAR-CODEs less than #x10000 can be encoded in
1073       a single 16-bit word and characters with larger codes can be
1074       encoded in a pair of 16-bit words.  The endianness of the
1075       encoded data is indicated by the endianness of a
1076       byte-order-mark character (#u+feff) prepended to the data; in
1077       the absence of such a character on input, the data is assumed
1078       to be in big-endian order. Output is written in native
1079       byte-order with a leading byte-order mark.</para></listitem>
1080     </varlistentry>
1081    <varlistentry><term><literal>:UTF-16BE</literal></term>
1082       <listitem><para>A 16-bit, variable-length encoding in which
1083       characters with CHAR-CODEs less than #x10000 can be encoded in
1084       a single 16-bit big-endian word and characters with larger
1085       codes can be encoded in a pair of 16-bit big-endian words.  The
1086       endianness of the encoded data is implicit in the encoding;
1087       byte-order-mark characters are not interpreted on input or
1088       prepended to output.</para></listitem>
1089     </varlistentry>
1090    <varlistentry><term><literal>:UTF-16LE</literal></term>
1091       <listitem><para>A 16-bit, variable-length encoding in which
1092       characters with CHAR-CODEs less than #x10000 can be encoded in
1093       a single 16-bit little-endian word and characters with larger
1094       codes can be encoded in a pair of 16-bit little-endian words.
1095       The endianness of the encoded data is implicit in the encoding;
1096       byte-order-mark characters are not interpreted on input or
1097       prepended to output.</para></listitem>
1098     </varlistentry>
1099    <varlistentry><term><literal>:UTF-32</literal></term>
1100       <listitem><para>A 32-bit, fixed-length encoding in which all
1101       Unicode characters can be encoded in a single 32-bit word.  The
1102       endianness of the encoded data is indicated by the endianness
1103       of a byte-order-mark character (#u+feff) prepended to the data;
1104       in the absence of such a character on input, input data is
1105       assumed to be in big-endian order.  Output is written in native
1106       byte order with a leading byte-order mark.</para>
1107       <para>Alias: <literal>:UTF-4</literal></para></listitem>
1108     </varlistentry>
1109    <varlistentry><term><literal>:UTF-32BE</literal></term>
1110       <listitem><para>A 32-bit, fixed-length encoding in which all
1111       Unicode characters encoded in a single 32-bit word. The encoded
1112       data is implicitly big-endian; byte-order-mark characters are
1113       not interpreted on input or prepended to
1114       output.</para>
1115       <para>Alias: <literal>:UCS-4BE</literal></para></listitem>
1116     </varlistentry>
1117    <varlistentry><term><literal>:UTF-8</literal></term>
1118       <listitem><para>An 8-bit, variable-length character encoding in
1119       which characters with CHAR-CODEs in the range #x00-#x7f can be
1120       encoded in a single octet; characters with larger code values
1121       can be encoded in 2 to 4 bytes.</para></listitem>
1122     </varlistentry>
1123    <varlistentry><term><literal>:UTF-32LE</literal></term>
1124       <listitem><para>A 32-bit, fixed-length encoding in which all
1125       Unicode characters can encoded in a single 32-bit word. The
1126       encoded data is implicitly little-endian; byte-order-mark
1127       characters are not interpreted on input or prepended to
1128       output.</para>
1129       <para>Alias: <literal>:UCS-4LE</literal></para></listitem>
1130     </varlistentry>
1131     <varlistentry><term><literal>:Windows-31j</literal></term>
1132     <listitem><para>An 8-bit, variable-length character encoding in
1133     which character code points in the range #x00-#x7f can be encoded
1134     in a single octet; characters with larger code values can be
1135     encoded in 2 bytes.</para>
1136     <para>Aliases: <literal>:CP932, :CSWINDOWS31J</literal></para></listitem>
1137     </varlistentry>
1138     <varlistentry><term><literal>:EUC-JP</literal></term>
1139     <listitem><para>An 8-bit, variable-length character encoding in
1140     which character code points in the range #x00-#x7f can be encoded
1141     in a single octet; characters with larger code values can be
1142     encoded in 2 bytes.</para>
1143     <para>Alias: <literal>:EUCJP</literal></para></listitem>
1144     </varlistentry>
1145   </variablelist>
1146 </sect3>
1147
1148 <sect3><title>Encoding and Decoding Strings</title>
1149 <para>&CCL; provides functions to encode and decode strings
1150to and from vectors of type (simple-array (unsigned-byte 8)).</para>
1151
1152<refentry id="count-characters-in-octet-vector">
1153  <indexterm zone="count-characters-in-octet-vector">
1154    <primary>count-characters-in-octet-vector</primary>
1155  </indexterm>
1156
1157  <refnamediv>
1158    <refname>count-characters-in-octet-vector</refname>
1159    <refpurpose></refpurpose>
1160    <refclass>Function</refclass>
1161  </refnamediv>
1162
1163  <refsynopsisdiv>
1164    <synopsis>
1165        <function>count-characters-in-octet-vector</function>
1166        <parameter>vector</parameter>
1167        <parameter>&key;</parameter>
1168        <parameter>start</parameter>
1169        <parameter>end</parameter>
1170        <parameter>external-format</parameter>
1171    </synopsis>
1172  </refsynopsisdiv>
1173
1174  <refsect1><title>Description</title>
1175  <para>
1176    Returns the number of characters that would be produced by
1177    decoding <varname>vector</varname> (or the subsequence thereof
1178    delimited by <varname>start</varname> and <varname>end</varname>)
1179    according to <varname>external-format</varname>.
1180  </para>
1181  </refsect1>
1182</refentry>
1183
1184<refentry id="decode-string-from-octets">
1185  <indexterm zone="decode-string-from-octets">
1186    <primary>"decode-string-from-octets</primary>
1187  </indexterm>
1188
1189  <refnamediv>
1190    <refname>decode-string-from-octets</refname>
1191    <refpurpose></refpurpose>
1192    <refclass>Function</refclass>
1193  </refnamediv>
1194
1195  <refsynopsisdiv>
1196    <synopsis>
1197        <function>decode-string-from-octets</function>
1198        <parameter>vector</parameter>
1199        <parameter>&key;</parameter>
1200        <parameter>start</parameter>
1201        <parameter>end</parameter>
1202        <parameter>external-format</parameter>
1203        <parameter>string</parameter>
1204    </synopsis>
1205  </refsynopsisdiv>
1206
1207
1208  <refsect1><title>Description</title>
1209  <para>
1210    Decodes the octets in <varname>vector</varname> (or the subsequence
1211    of it delimited by <varname>start</varname> and
1212    <varname>end</varname>) into a string according
1213    to <varname>external-format</varname>.
1214  </para>
1215  <para>
1216    If <varname>string</varname> is supplied, output will be written into it.
1217    It must be large enough to hold the decoded characters.  If <varname>
1218    string</varname> is not supplied, a new string will be allocated to
1219    hold the decoded characters.
1220  </para>
1221  <para>
1222    Returns, as multiple values, the decoded string and the position in
1223    <varname>vector</varname> where the decoding ended.
1224  </para>
1225  </refsect1>
1226</refentry>
1227
1228<refentry id="encode-string-to-octets">
1229  <indexterm zone="encode-string-to-octets">
1230    <primary>encode-string-to-octets</primary>
1231  </indexterm>
1232
1233  <refnamediv>
1234    <refname>encode-string-to-octets</refname>
1235    <refpurpose></refpurpose>
1236    <refclass>Function</refclass>
1237  </refnamediv>
1238
1239  <refsynopsisdiv>
1240    <synopsis>
1241        <function>encode-string-to-octets</function>
1242        <parameter>string</parameter>
1243        <parameter>&key;</parameter>
1244        <parameter>start</parameter>
1245        <parameter>end</parameter>
1246        <parameter>external-format</parameter>
1247        <parameter>use-byte-order-mark</parameter>
1248        <parameter>vector</parameter>
1249        <parameter>vector-offset</parameter>
1250    </synopsis>
1251  </refsynopsisdiv>
1252
1253
1254  <refsect1><title>Description</title>
1255  <para>
1256    Encodes <varname>string</varname> (or the substring delimited by
1257    <varname>start</varname> and <varname>end</varname>)
1258    into <varname>external-format</varname> and returns, as multiple
1259    values, a vector of octets containing the encoded data and an integer
1260    that specifies the offset into the vector where the encoded data ends.
1261  </para>
1262  <para>
1263    When <varname>use-byte-order-mark</varname> is true, a byte-order mark
1264    will be included in the encoded data.
1265  </para>
1266  <para>
1267    If <varname>vector</varname> is supplied, output will be written
1268    to it.  It must be of type (simple-array (unsigned-byte 8)) and be
1269    large enough to hold the encoded data.  If it is not supplied, the function
1270    will allocate a new vector.
1271  </para>
1272  <para>
1273    If <varname>vector-offset</varname> is supplied, data will be written
1274    into the output vector starting at that offset.
1275  </para>
1276  </refsect1>
1277</refentry>
1278
1279<refentry id="string-size-in-octets">
1280  <indexterm zone="string-size-in-octets">
1281    <primary>string-size-in-octets</primary>
1282  </indexterm>
1283
1284  <refnamediv>
1285    <refname>string-size-in-octets</refname>
1286    <refpurpose></refpurpose>
1287    <refclass>Function</refclass>
1288  </refnamediv>
1289
1290  <refsynopsisdiv>
1291    <synopsis>
1292        <function>string-size-in-octets</function>
1293        <parameter>string</parameter>
1294        <parameter>&key;</parameter>
1295        <parameter>start</parameter>
1296        <parameter>end</parameter>
1297        <parameter>external-format</parameter>
1298        <parameter>use-byte-order-mark</parameter>
1299    </synopsis>
1300  </refsynopsisdiv>
1301
1302
1303  <refsect1><title>Description</title>
1304  <para>
1305    Returns the number of octets required to encode
1306    <varname>string</varname> (or the substring delimited by
1307    <varname>start</varname> and <varname>end</varname>) into
1308    <varname>external-format</varname>.
1309  </para>
1310  <para>
1311    When <varname>use-byte-order-mark</varname> is true, the returned
1312    size will include space for a byte-order marker.
1313  </para>
1314  </refsect1>
1315</refentry>
1316
1317 </sect3>
1318
1319
1320  </sect2>
1321
1322  </sect1>
1323
1324  <sect1 id="Pathanmes"><title>Pathnames</title>
1325
1326    <sect2 id="pathname-tilde-expansion">
1327      <title>Pathname Expansion</title>
1328      <para>Leading tilde (~) characters in physical pathname namestrings
1329        are expanded in the way that most shells do:</para>
1330     
1331      <para><literal>"~user/..."</literal> can be used to refer to an absolute pathname rooted
1332        at the home directory of the user named "user".</para>
1333     
1334      <para><literal>"~/..."</literal> can be used to refer to an absolute pathname rooted at
1335        the home directory of the current user.</para>
1336    </sect2>
1337
1338    <sect2 id="Predefined-Logical-Hosts"><title>Predefined Logical Hosts</title>
1339
1340      <para>&CCL; sets up logical pathname translations for logical hosts:  <literal>ccl</literal> and <literal>home</literal></para>
1341
1342      <indexterm><primary>CCL Logical Host</primary></indexterm>
1343      <para>The <literal>CCL</literal> logical host should point to the
1344        <literal>ccl</literal> directory.  It is used for a variety of
1345        purposes by &CCL; including: locating &CCL; source code,
1346        <literal>require</literal> and <literal>provide</literal>, accessing
1347        foreign function information, and the &CCL; build process. It
1348        is set to the value of the environment variable
1349        <varname>CCL_DEFAULT_DIRECTORY</varname>, which is set by the
1350        openmcl shell script <xref linkend="The-ccl-Shell-Script"/>.  If
1351        <varname>CCL_DEFAULT_DIRECTORY</varname> is not set, then it is set
1352        to the directory containing the current heap image.</para>
1353    </sect2>
1354   
1355   
1356    <sect2 id="pathnames-on-darwin">
1357      <title>OS X (Darwin)</title>
1358     
1359      <para>&CCL; assumes that pathname strings are decomposed UTF-8.</para>
1360    </sect2>
1361    <sect2 id="pathnames-on-linux">
1362      <title>Linux</title>
1363     
1364      <para>Pathname strings are treated as null-terminated strings
1365        coded in the encoding named by the value returned by the function
1366      <varname>CCL:PATHNAME-ENCODING-NAME</varname>.  This value may be changed
1367      with <varname>SETF</varname>.</para>
1368    </sect2>
1369    <sect2 id="pathnames-on-freebsd">
1370      <title>FreeBSD</title>
1371     
1372      <para>Pathname strings are treated as null-terminated strings
1373        encoded according to the current locale; a future release may
1374        change this convention to use UTF-8.</para>
1375    </sect2>
1376  </sect1>
1377
1378  <sect1 id="Memory-Mapped-Files">
1379    <title>Memory-mapped Files</title>
1380    <para>In release 1.2 and later, &CCL;
1381      supports <glossterm linkend="memory_mapped_file">memory-mapped
1382        files</glossterm>. On operating systems that support memory-mapped
1383      files (including Mac OS X, Linux, and FreeBSD), the operating
1384      system can arrange for a range of virtual memory addresses to
1385      refer to the contents of an open file. As long as the file remains
1386      open, programs can read values from the file by reading addresses
1387      in the mapped range.</para>
1388
1389    <para>Using memory-mapped files may in some cases be more
1390      efficient than reading the contents of a file into a data
1391      structure in memory.</para>
1392
1393    <para>&CCL; provides the
1394      functions <varname>CCL:MAP-FILE-TO-IVECTOR</varname>
1395      and <varname>CCL:MAP-FILE-TO-OCTET-VECTOR</varname> to support
1396      memory-mapping. These functions return vectors whose contents are
1397      the contents of memory-mapped files. Reading an element of such a
1398      vector returns data from the corresponding position in the
1399      file.</para>
1400
1401    <para>Without memory-mapped files, a common idiom for reading the
1402      contents of files might be something like this:</para>
1403
1404    <programlisting>
1405(let* ((stream (open pathname :direction :input :element-type '(unsigned-byte 8)))
1406       (vector (make-array (file-size-to-vector-size stream)
1407                           :element-type '(unsigned-byte 8))))
1408  (read-sequence vector stream))
1409    </programlisting>
1410
1411    <para>Using a memory-mapped files has a result that is the same in
1412      that, like the above example, it returns a vector whose contents are
1413      the same as the contents of the file. It differs in that the above
1414      example creates a new vector in memory and copies the file's
1415      contents into it; using a memory-mapped file instead arranges for
1416      the vector's elements to point to the file's contents on disk
1417      directly, without copying them into memory first.</para>
1418
1419    <para>The vectors returned by <varname>CCL:MAP-FILE-TO-IVECTOR</varname>
1420      and <varname>CCL:MAP-FILE-TO-OCTET-VECTOR</varname> are read-only; any
1421      attempt to change an element of a vector returned by these
1422      functions results in a memory-access error. &CCL; does not
1423      currently support writing data to memory-mapped files.</para>
1424
1425    <para>Vectors created by <varname>CCL:MAP-FILE-TO-IVECTOR</varname>
1426      and <varname>CCL:MAP-FILE-TO-OCTET-VECTOR</varname> are required to
1427      respect &CCL;'s limit on the total size of an array. That means
1428      that you cannot use these functions to create a vector longer
1429      than <varname>ARRAY-TOTAL-SIZE-LIMIT</varname>, even if the filesystem
1430      supports file sizes that are larger. The value
1431      of <varname>ARRAY-TOTAL-SIZE-LIMIT</varname> is <varname>(EXPT 2 24)</varname>
1432      on 32-but platforms; and <varname>(EXPT 2 56)</varname> on 64-bit
1433      platforms.</para>
1434
1435    <para>
1436      <indexterm zone="map-file-to-ivector"/>
1437      <command><varname id="map-file-to-ivector">CCL:MAP-FILE-TO-IVECTOR</varname>
1438        <parameter>pathname</parameter>
1439        <parameter>element-type</parameter>
1440        [Function]</command>
1441    </para>
1442
1443    <variablelist>
1444      <varlistentry>
1445        <term><varname>pathname</varname></term>
1446        <listitem>
1447          <para>The pathname of the file to be memory-mapped.</para>
1448        </listitem>
1449      </varlistentry>
1450     
1451      <varlistentry>
1452        <term><varname>element-type</varname></term>
1453        <listitem>
1454          <para>The element-type of the vector to be
1455            created. Specified as
1456            a <glossterm linkend="type-specifier">type-specifier</glossterm>
1457            that names a subtype of either <varname>SIGNED-BYTE</varname>
1458            or <varname>UNSIGNED-BYTE</varname>.</para>
1459        </listitem>
1460      </varlistentry>
1461    </variablelist>
1462
1463
1464    <para>
1465      The <varname>map-file-to-ivector</varname> function tries to
1466      open the file at <parameter>pathname</parameter> for reading. If
1467      successful, the function maps the file's contents to a range of
1468      virtual addresses. If successful, it returns a read-only vector
1469      whose element-type is given
1470      by <parameter>element-type</parameter>, and whose contents are
1471      the contents of the memory-mapped file.
1472    </para>
1473
1474    <para>The returned vector is
1475      a <glossterm linkend="displaced-array">displaced-array</glossterm>
1476      whose element-type is <varname>(UPGRADED-ARRAY-ELEMENT-TYPE
1477        element-type)</varname>. The target of the displaced array is a
1478      vector of type <varname>(SIMPLE-ARRAY element-type (*))</varname> whose
1479      elements are the contents of the memory-mapped file.</para>
1480
1481    <para>Because of alignment issues, the mapped file's contents
1482      start a few bytes (4 bytes on 32-bit platforms, 8 bytes on
1483      64-bit platforms) into the vector. The displaced array returned
1484      by <varname>CCL:MAP-FILE-TO-IVECTOR</varname> hides this overhead, but
1485      it's usually more efficient to operate on the underlying simple
1486      1-dimensional array.  Given a displaced array (like the value
1487      returned by <varname>CCL:MAP-FILE-TO-IVECTOR</varname>), the function
1488      <varname>ARRAY-DISPLACEMENT</varname> returns the underlying array and
1489      the displacement index in elements.
1490    </para>
1491
1492    <para>Currently, &CCL; supports only read operations on
1493      memory-mapped files. If you try to change the contents of an array
1494      returned by <varname>map-file-to-ivector</varname>, &CCL; signals
1495      a memory error.</para>
1496
1497    <para>
1498      <indexterm zone="unmap-ivector"/>
1499      <command><varname id="unmap-ivector">CCL:UNMAP-IVECTOR</varname>
1500        <parameter>displaced-array</parameter>
1501        [Function]</command>
1502    </para>
1503
1504    <para>If the argument is a displaced-array returned
1505      by <varname>map-file-to-ivector</varname>, and if it has not yet
1506      been unmapped by this function,
1507      then <varname>unmap-ivector</varname> undoes the memory mapping,
1508      closes the mapped file, and changes the displaced-array so that its
1509      target is an empty vector (of length zero).</para>
1510
1511    <para>
1512      <indexterm zone="map-file-to-octet-vector"/>
1513      <command><varname id="map-file-to-octet-vector">CCL:MAP-FILE-TO-OCTET-VECTOR</varname>
1514        <parameter>pathname</parameter>
1515        [Function]</command>
1516    </para>
1517
1518    <para>This function is a synonym for <varname>(CCL:MAP-FILE-TO-IVECTOR
1519        pathname '(UNSIGNED-BYTE 8))</varname> It is provided as a convenience
1520      for the common case of memory-mapping a file as a vector of
1521      bytes.</para>
1522
1523    <para>
1524      <indexterm zone="unmap-octet-vector"/>
1525      <command><varname id="unmap-octet-vector">CCL:UNMAP-OCTET-VECTOR</varname>
1526        <parameter>displaced-array</parameter>
1527        [Function]</command>
1528    </para>
1529
1530    <para>This function is a synonym
1531      for <varname>(CCL:UNMAP-IVECTOR)</varname></para>
1532  </sect1>
1533
1534  <!-- ============================================================ -->
1535  <sect1 id="Static_Variables">
1536    <title>Static Variables</title>
1537
1538    <para>&CCL; supports the definition
1539      of <glossterm linkend="static_variable">static
1540        variables</glossterm>, whose values are the same across threads,
1541      and which may not be dynamically bound. The value of a static
1542      variable is thus the same across all threads; changing the value
1543      in one thread changes it for all threads.</para> 
1544
1545    <para>Attempting to dynamically rebind a static variable (for
1546      instance, by using <varname>LET</varname>, or using the variable name as
1547      a parameter in a <varname>LAMBDA</varname> form) signals an
1548      error. Static variables are shared global resources; a dynamic
1549      binding is private to a single thread.</para>
1550
1551    <para>Static variables therefore provide a simple way to share
1552      mutable state across threads. They also provide a simple way to
1553      introduce race conditions and obscure bugs into your code, since
1554      every thread reads and writes the same instance of a given static
1555      variable. You must take care, therefore, in how you change the
1556      values of static variables, and use normal multithreaded
1557      programming techniques, such as locks or semaphores, to protect
1558      against race conditions.</para>
1559
1560    <para>In &CCL;, access to a static variable is usually faster than
1561      access to a special variable that has not been declared
1562      static.</para>
1563
1564    <para>
1565      <indexterm zone="defstatic"/>
1566      <command><varname id="defstatic">DEFSTATIC</varname>
1567        <parameter>var</parameter>
1568        <parameter>value</parameter>
1569        &key;
1570        <parameter>doc-string</parameter>
1571        [Macro]</command>
1572    </para>
1573   
1574    <variablelist>
1575      <varlistentry>
1576        <term><varname>var</varname></term>
1577        <listitem>
1578          <para>The name of the new static variable.</para>
1579        </listitem>
1580      </varlistentry>
1581     
1582      <varlistentry>
1583        <term><varname>value</varname></term>
1584        <listitem>
1585          <para>The initial value of the new static variable.</para>
1586        </listitem>
1587      </varlistentry>
1588     
1589      <varlistentry>
1590        <term><varname>doc-string</varname></term>
1591        <listitem>
1592          <para>A documentation string that is assigned to the new
1593            variable.</para>
1594        </listitem>
1595      </varlistentry>
1596    </variablelist>
1597
1598    <para>Proclaims the
1599      variable <glossterm linkend="special_variable">special</glossterm>,
1600      assigns the variable the supplied value, and assigns
1601      the <varname>doc-string</varname> to the
1602      variable's <varname>VARIABLE</varname> documentation. Marks the
1603      variable static, preventing any attempt to dynamically rebind
1604      it. Any attempt to dynamically rebind <varname>var</varname>
1605      signals an error.</para>
1606  </sect1>
1607
1608  <!-- ============================================================ -->
1609  <sect1 id="Saving-Applications">
1610    <title>Saving Applications</title>
1611    <indexterm zone="Saving-Applications">
1612      <primary>save-application</primary>
1613    </indexterm>
1614
1615    <para>&CCL; provides the
1616      function <literal>CCL:SAVE-APPLICATION</literal>, which creates a file
1617      containing an archived Lisp memory image.</para>
1618
1619    <para>&CCL; consists of a small executable called the
1620      Lisp <glossterm linkend="lisp_image">kernel</glossterm>, which
1621      implements the very lowest level features of the Lisp system, and
1622      an <glossterm linkend="lisp_image">image</glossterm>, which
1623      contains the in-memory representation of most of the Lisp system,
1624      including functions, data structures, variables, and so on. When
1625      you start &CCL;, you are launching the kernel, which then locates
1626      and reads an image file, restoring the archived image in
1627      memory. Once the image is fully restored, the Lisp system is
1628      running.</para>
1629
1630    <para>Using <literal>CCL:SAVE-APPLICATION</literal>, you can create a
1631      file that contains a modified image, one that includes any changes
1632      you've made to the running Lisp system. If you later pass your
1633      image file to the &CCL; kernel as a command-line parameter, it
1634      then loads your image file instead of its default one, and &CCL;
1635      starts up with your modifications.</para>
1636
1637    <para>If this scenario seems to you like a convenient way to
1638      create an application, that's just as intended. You can create an
1639      application by modifying the running Lisp until it does what you
1640      want, then use <literal>CCL:SAVE-APPLICATION</literal> to preserve your
1641      changes and later load them for use.</para>
1642
1643    <para>In fact, you can go further than that. You can replace
1644      &CCL;'s <glossterm linkend="toplevel_function">toplevel
1645        function</glossterm> with your own, and then, when the image is
1646      loaded, the Lisp system immediately performs your tasks rather
1647      than the default tasks that make it a Lisp development system. If
1648      you save an image in which you have done this, the resulting Lisp
1649      system is your tool rather than a Lisp development system.</para>
1650
1651    <para>You can go a step further still. You can
1652      tell <literal>CCL:SAVE-APPLICATION</literal> to prepend the Lisp kernel
1653      to the image file. Doing this makes the resulting image into a
1654      self-contained executable binary. When you run the resulting file,
1655      the Lisp kernel immediately loads the attached image file and runs
1656      your saved system. The Lisp system that starts up can have any
1657      behavior you choose. It can be a Lisp development system, but with
1658      your customizations; or it can immediately perform some task of
1659      your design, making it a specialized tool rather than a general
1660      development system.</para>
1661
1662    <para>In other words, you can develop any application you like by
1663      interactively modifying &CCL; until it does what you want, then
1664      using <literal>CCL:SAVE-APPLICATION</literal> to preserve your changes
1665      in an executable image.</para>
1666
1667    <para>On Mac OS X,
1668      the <link linkend="application_builder">application builder</link>
1669      uses <literal>CCL:SAVE-APPLICATION</literal> to create the executable
1670      portion of the <glossterm linkend="application_bundle">application
1671        bundle</glossterm>. Double-clicking the application bundle runs
1672      the executable image created
1673      by <literal>CCL:SAVE-APPLICATION</literal>.</para>
1674
1675    <para>Also on Mac OS X, &CCL; supports an object type
1676      called <literal>MACPTR</literal>, which is the type of pointers into the
1677      foreign (Mac OS) heap. Examples of
1678      commonly-user <literal>MACPTR</literal> objects are Cocoa windows and
1679      other dynamically-allocated Mac OS system objects.</para>
1680
1681    <para>Because a <literal>MACPTR</literal> object is a pointer into a
1682      foreign heap that exists for the lifetime of the running Lisp
1683      process, and because a saved image is used by loading it into a
1684      brand new Lisp process, saved <literal>MACPTR</literal> objects cannot
1685      be relied on to point to the same things when reconstituted from a
1686      saved image. In fact, a restored <literal>MACPTR</literal> object might
1687      point to anything at all&mdash;for example an arbitrary location
1688      in the middle of a block of code, or a completely nonexistent
1689      virtual address.</para> 
1690
1691    <para>For that reason, <literal>CCL:SAVE-APPLICATION</literal> converts
1692      all <literal>MACPTR</literal> objects to <literal>DEAD-MACPTR</literal>
1693      objects when writing them to an image
1694      file. A <literal>DEAD-MACPTR</literal> is functionally identical to
1695      a <literal>MACPTR</literal>, except that code that operates
1696      on <literal>MACPTR</literal> objects distinguishes them
1697      from <literal>DEAD-MACPTR</literal> objects and can handle them
1698      appropriately&mdash;signaling errors, for example.</para>
1699
1700    <para>As of &CCL; 1.2, there is one exception to the conversion
1701      of <literal>MACPTR</literal> to <literal>DEAD-MACPTR</literal> objects:
1702      a <literal>MACPTR</literal> object that points to the address 0 is not
1703      converted, because address 0 can always be relied upon to refer to
1704      the same thing.</para>
1705
1706        <indexterm zone="Saving-Applications">
1707          <primary>+NULL-PTR+</primary>
1708        </indexterm>
1709    <para>As of &CCL; 1.2, the constant <literal>CCL:+NULL-PTR+</literal>
1710      refers to a <literal>MACPTR</literal> object that points to address 0.</para>
1711
1712    <para>On all supported platforms, you can
1713      use <literal>CCL:SAVE-APPLICATION</literal> to create a command-line
1714      tool that runs the same way any command-line program
1715      does. Alternatively, if you choose not to prepend the kernel, you
1716      can save an image and then later run it by passing it as a
1717      command-line parameter to the <literal>opencml</literal>
1718      or <literal>opencml64</literal> script.</para>
1719
1720    <para>
1721      <indexterm zone="save-application"/>
1722      <command><varname id="save-application">SAVE-APPLICATION</varname>
1723        <parameter>filename</parameter>
1724        &key;
1725        <parameter>toplevel-function</parameter>
1726        <parameter>init-file</parameter>
1727        <parameter>error-handler</parameter>
1728        <parameter>application-class</parameter>
1729        <parameter>clear-clos-caches</parameter>
1730        <parameter>(purify t)</parameter>
1731        <parameter>impurify</parameter>
1732        <parameter>(mode #o644)</parameter>
1733        <parameter>prepend-kernel</parameter>
1734        [Function]</command>
1735    </para>
1736   
1737    <variablelist>
1738      <varlistentry>
1739        <term><varname>filename</varname></term>
1740        <listitem>
1741          <para>The pathname of the file to be created when &CCL;
1742            saves the application.</para>
1743        </listitem>
1744      </varlistentry>
1745     
1746      <varlistentry>
1747        <term><varname>toplevel-function</varname></term>
1748        <listitem>
1749          <para>The function to be executed after startup is
1750            complete. The toplevel is a function of no arguments that
1751            performs whatever actions the lisp system should perform
1752            when launched with this image.</para>
1753          <para>If this parameter is not supplied, &CCL; uses its
1754            default toplevel. The default toplevel runs
1755            the <glossterm linkend="REPL">read-eval-print
1756              loop</glossterm>.</para>
1757        </listitem>
1758      </varlistentry>
1759
1760      <varlistentry>
1761        <term><varname>init-file</varname></term>
1762        <listitem>
1763          <para>The pathname of a Lisp file to be loaded when the
1764            image starts up. You can place initialization expressions in
1765            this file, and use it to customize the behavior of the Lisp
1766            system when it starts up.</para>
1767        </listitem>
1768      </varlistentry>
1769     
1770      <varlistentry>
1771        <term><varname>error-handler</varname></term>
1772        <listitem>
1773          <para>The error-handling mode for the saved image. The
1774            supplied value determines what happens when an error is not
1775            handled by the saved image. Valid values
1776            are <literal>:quit</literal> (Lisp exits with an error
1777            message); <literal>:quit-quietly</literal> (Lisp exits without an
1778            error message); or <literal>:listener</literal> (Lisp enters a
1779            break loop, enabling you to debug the problem by interacting
1780            in a listener). If you don't supply this parameter, the
1781            saved image uses the default error handler
1782            (<literal>:listener</literal>).</para>
1783        </listitem>
1784      </varlistentry>
1785
1786      <varlistentry>
1787        <term><varname>application-class</varname></term>
1788        <listitem>
1789          <para>The CLOS class that represents the saved Lisp
1790            application. Normally you don't need to supply this
1791            parameter; <literal>CCL:SAVE-APPLICATION</literal> uses the
1792            class <literal>CCL:LISP-DEVELOPMENT-SYSTEM</literal>. In some
1793            cases you may choose to create a custom application class;
1794            in that case, pass the name of the class as the value for
1795            this parameter.</para>
1796        </listitem>
1797      </varlistentry>
1798     
1799      <varlistentry>
1800        <term><varname>clear-clos-caches</varname></term>
1801        <listitem>
1802          <para>If true, ensures that CLOS caches are emptied before
1803            saving the image. Normally you don't need to supply this
1804            parameter, but if for some reason you want to ensure the
1805            CLOS caches are clear when the image starts up, you can pass
1806            any true value.</para>
1807        </listitem>
1808      </varlistentry>
1809     
1810     
1811    </variablelist>
1812
1813    <para></para>
1814  </sect1>
1815
1816  <sect1 id="concatenating-fasl-files">
1817    <title>Concatenating FASL Files</title>
1818    <para>
1819      Multiple fasl files can be concatenated into a single file.
1820    </para>
1821  <refentry id="f_fasl-concatenate">
1822    <indexterm zone="f_fasl-concatenate">
1823      <primary>fasl-concatenate</primary>
1824    </indexterm>
1825
1826    <refnamediv>
1827      <refname>FASL-CONCATENATE</refname>
1828      <refpurpose>
1829        Concatenate several fasl files, producing a single output file.
1830      </refpurpose>
1831      <refclass>Function</refclass>
1832    </refnamediv>
1833   
1834    <refsynopsisdiv>
1835      <synopsis><function>fasl-concatenate</function> out-file fasl-files &key; (:if-exists :error)</synopsis>
1836    </refsynopsisdiv>
1837
1838    <refsect1>
1839      <title>Arguments and Values</title>
1840      <variablelist>
1841        <varlistentry>
1842          <term>out-file</term>
1843          <listitem>
1844            <para>
1845              Name of the file in which to store the concatenation.
1846            </para>
1847          </listitem>
1848        </varlistentry>
1849        <varlistentry>
1850          <term>fasl-files</term>
1851          <listitem>
1852            <para>
1853              List of names of fasl files to concatenate.
1854            </para>
1855          </listitem>
1856        </varlistentry>
1857        <varlistentry>
1858          <term>:if-exists</term>
1859          <listitem>
1860            <para>
1861              As for <function>OPEN</function>, defaults to <literal>
1862              :error</literal>
1863            </para>
1864          </listitem>
1865        </varlistentry>
1866      </variablelist>
1867    </refsect1>
1868   
1869    <refsect1>
1870      <title>Description</title>
1871      <para>
1872        Creates a fasl file which, when loaded, will have the same
1873        effect as loading the individual input fasl files in the
1874        specified order.  The single file might be easier to
1875        distribute or install, and loading it may be at least a little
1876        faster than loading the individual files (since it avoids the
1877        overhead of opening and closing each file in succession.)
1878      </para>
1879      <para>
1880        The PATHNAME-TYPE of the output file and of each input file
1881        defaults to the current platform's fasl file type (.dx64fsl or
1882        whatever.)  If any of the input files has a different
1883        type/extension an error will be signaled, but it doesn't
1884        otherwise try too hard to verify that the input files are real
1885        fasl files for the current platform.
1886      </para>
1887    </refsect1>
1888  </refentry>
1889  </sect1>
1890  <sect1 id="floating-point">
1891    <title>Floating Point Numbers</title>
1892
1893    <para>
1894      In &CCL;, the Common Lisp types short-float and single-float are
1895      implemented as IEEE single precision values; double-float and
1896      long-float are IEEE double precision values.  On 64-bit
1897      platforms, single-floats are immediate values (like fixnums and
1898      characters).
1899    </para>
1900
1901  <para>
1902    Floating-point exceptions are generally enabled and detected.  By
1903    default, threads start up with overflow, division-by-zero, and
1904    invalid enabled, and the rounding mode is set to nearest. The
1905    functions <varname>SET-FPU-MODE</varname> and
1906    <varname>GET-FPU-MODE</varname> provide user control over
1907    floating-point behavior.
1908  </para>
1909
1910  <refentry id="f_get-fpu-mode">
1911    <indexterm zone="f_get-fpu-mode">
1912      <primary>get-fpu-mode</primary>
1913    </indexterm>
1914
1915    <refnamediv>
1916      <refname>GET-FPU-MODE</refname>
1917      <refpurpose>
1918        Return the state of exception-enable and rounding-mode control
1919        flags for the current thread.
1920      </refpurpose>
1921      <refclass>Function</refclass>
1922    </refnamediv>
1923   
1924    <refsynopsisdiv>
1925      <synopsis><function>get-fpu-mode</function> &optional; mode</synopsis>
1926    </refsynopsisdiv>
1927
1928    <refsect1>
1929      <title>Arguments and Values</title>
1930      <variablelist>
1931        <varlistentry>
1932          <term>mode</term>
1933          <listitem>
1934            <para>
1935              One of the keywords :rounding-mode, :overflow,
1936              :underflow, :division-by-zero, :invalid, :inexact.
1937            </para>
1938          </listitem>
1939        </varlistentry>
1940      </variablelist>
1941    </refsect1>
1942   
1943    <refsect1>
1944      <title>Description</title>
1945      <para>
1946        If <varname>mode</varname> is supplied, returns the value of
1947        the corresponding control flag for the current thread.
1948      </para>
1949      <para>
1950        Otherwise, returns a list of keyword/value pairs which
1951        describe the floating-point exception-enable and rounding-mode
1952        control flags for the current thread.
1953      </para>
1954      <variablelist>
1955        <varlistentry>
1956          <term>rounding-mode</term>
1957          <listitem>
1958            <para>
1959              One of :nearest, :zero, :positive, :negative
1960            </para>
1961          </listitem>
1962        </varlistentry>
1963        <varlistentry>
1964          <term>overflow, underflow, division-by-zero, invalid, inexact
1965          </term>
1966          <listitem>
1967            <para>
1968              If true, the floating-point exception is signaled.
1969              If NIL, it is masked.
1970            </para>
1971          </listitem>
1972        </varlistentry>
1973      </variablelist>
1974    </refsect1>
1975  </refentry>
1976
1977  <refentry id="f_set-fpu-mode">
1978    <indexterm zone="f_set-fpu-mode">
1979      <primary>set-fpu-mode</primary>
1980    </indexterm>
1981
1982    <refnamediv>
1983      <refname>SET-FPU-MODE</refname>
1984      <refpurpose>
1985        Set the state of exception-enable and rounding-mode control
1986        flags for the current thread.
1987      </refpurpose>
1988      <refclass>Function</refclass>
1989    </refnamediv>
1990   
1991    <refsynopsisdiv>
1992      <synopsis><function>set-fpu-mode</function> &key;
1993      rounding-mode overflow underflow division-by-zero
1994      invalid inexact</synopsis>
1995    </refsynopsisdiv>
1996
1997    <refsect1>
1998      <title>Arguments and Values</title>
1999      <variablelist>
2000        <varlistentry>
2001          <term>rounding-mode</term>
2002          <listitem>
2003            <para>
2004              If supplied, must be one of :nearest, :zero, :positive, or
2005              :negative.
2006            </para>
2007          </listitem>
2008        </varlistentry>
2009        <varlistentry>
2010          <term>overflow, underflow, division-by-zero, invalid, inexact</term>
2011          <listitem>
2012            <para>NIL to mask the exception, T to signal it.</para>
2013          </listitem>
2014        </varlistentry>
2015      </variablelist>
2016    </refsect1>
2017   
2018    <refsect1>
2019      <title>Description</title>
2020      <para>
2021        Sets the current thread's exception-enable and rounding-mode
2022        control flags to the indicated values for arguments that are
2023        supplied, and preserves the values assoicated with those
2024        that aren't supplied.
2025      </para>
2026    </refsect1>
2027  </refentry>
2028  </sect1>
2029
2030  <sect1 id="watched-objects"><title>Watched Objects</title>
2031  <para>
2032    As of release 1.4, Clozure CL provides a way for lisp objects to
2033    be watched so that a condition will be signaled when a thread
2034    attempts to write to the watched object. For a certain class of
2035    bugs (someone is changing this value, but I don't know who), this
2036    can be extremely helpful.
2037  </para>
2038  <sect2 id="watched-watch"><title>WATCH</title>
2039  <refentry id="f_watch">
2040    <indexterm zone="f_watch">
2041      <primary>watch</primary>
2042    </indexterm>
2043   
2044    <refnamediv>
2045      <refname>WATCH</refname>
2046      <refpurpose>
2047        Monitor a lisp object for writes.
2048      </refpurpose>
2049      <refclass>Function</refclass>
2050    </refnamediv>
2051   
2052    <refsynopsisdiv>
2053      <synopsis><function>watch</function> &optional; object</synopsis>
2054    </refsynopsisdiv>
2055   
2056    <refsect1>
2057      <title>Arguments and Values</title>
2058      <variablelist>
2059        <varlistentry>
2060          <term>object</term>
2061          <listitem>
2062            <para>
2063              Any memory-allocated lisp object.
2064            </para>
2065          </listitem>
2066        </varlistentry>
2067      </variablelist>
2068    </refsect1>
2069
2070    <refsect1>
2071      <title>Description</title>
2072      <para>
2073        The WATCH function arranges for the specified object to be
2074        monitored for writes. This is accomplished by copying the
2075        object to its own set of virtual memory pages, which are then
2076        write-protected. This protection is enforced by the computer's
2077        memory-management hardware; the write-protection does not slow
2078        down reads at all.
2079      </para>
2080      <para>
2081        When any write to the object is attempted, a
2082        WRITE-TO-WATCHED-OBJECT condition will be signaled.
2083      </para>
2084      <para>
2085        When called with no arguments, WATCH returns a freshly-consed
2086        list of the objects currently being watched.
2087      </para>
2088      <para>
2089        WATCH returns NIL if the object cannot be watched (typically
2090        because the object is in a static or pure memory area).
2091      </para>
2092    </refsect1>
2093    <refsect1 id="watch-dwim"><title>DWIM</title>
2094    <para>
2095      WATCH operates at a fairly low level; it is not possible to
2096      avoid the details of the internal representation of objects.
2097      Nevertheless, as a convenience, WATCHing a standard-instance,
2098      a hash-table, or a multi-dimensional or non-simple CL array
2099      will watch the underlying slot-vector, hash-table-vector, or
2100      data-vector, respectively.
2101      </para>
2102    </refsect1>
2103    <refsect1 id="watch-discuss"><title>Discussion</title>
2104    <para>
2105      WATCH can monitor any memory-allocated lisp object.
2106    </para>
2107    <para>
2108      In Clozure CL, a memory-allocated object is either a cons cell
2109      or a uvector.
2110    </para>
2111    <para>
2112      WATCH operates on cons cells, not lists. In order to watch a
2113      chain of cons cells, each cons cell must be watched
2114      individually. Because each watched cons cell takes up its own
2115      own virtual memory page (4 Kbytes), it's only feasible to watch
2116      relatively short lists.
2117    </para>
2118    <para>
2119      If a memory-allocated object isn't a cons cell, then it is a
2120      vector-like object called a uvector. A uvector is a
2121      memory-allocated lisp object whose first word is a header that
2122      describes the object's type and the number of elements that it
2123      contains.
2124    </para>
2125    <para>
2126      So, a hash table is a uvector, as is a string, a standard
2127      instance, a double-float, a CL array or vector, and so forth.
2128    </para>
2129    <para>
2130      Some CL objects, like strings and other simple vectors, map in a
2131      straightforward way onto the uvector representation. It is easy
2132      to understand what happens in such cases. The uvector index
2133      corresponds directly to the vector index:
2134    </para>
2135    <programlisting>
2136<![CDATA[
2137? (defvar *s* "xxxxx")
2138*S*
2139? (watch *s*)
2140"xxxxx"
2141? (setf (char *s* 3) #\o)
2142> Error: Write to watched uvector "xxxxx" at index 3
2143>        Faulting instruction: (movl (% eax) (@ -5 (% r15) (% rcx)))
2144> While executing: SET-CHAR, in process listener(1).
2145> Type :POP to abort, :R for a list of available restarts.
2146> Type :? for other options.
2147]]>
2148    </programlisting>
2149    <para>
2150      In the case of more complicated objects (e.g., a hash-table, a
2151      standard-instance, a package, etc.), the elements of the uvector
2152      are like slots in a structure. It's necessary to know which one
2153      of those "slots" contains the data that will be changed when the
2154      object is written to.
2155    </para>
2156    <para>
2157      As mentioned above, watch knows about arrays, hash-tables, and
2158      standard-instances, and will automatically watch the appropriate
2159      data-containing element.
2160    </para>
2161    <para>
2162      An example might make this clearer.
2163    </para>
2164    <programlisting>
2165<![CDATA[
2166? (defclass foo ()
2167    (slot-a slot-b slot-c))
2168#<STANDARD-CLASS FOO>
2169? (defvar *a-foo* (make-instance 'foo))
2170*A-FOO*
2171? (watch *a-foo*)
2172#<SLOT-VECTOR #xDB00D>
2173;;; Note that WATCH has watched the internal slot-vector object
2174? (setf (slot-value *a-foo* 'slot-a) 'foo)
2175> Error: Write to watched uvector #<SLOT-VECTOR #xDB00D> at index 1
2176>        Faulting instruction: (movq (% rsi) (@ -5 (% r8) (% rdi)))
2177> While executing: %MAYBE-STD-SETF-SLOT-VALUE-USING-CLASS, in process listener(1).
2178> Type :POP to abort, :R for a list of available restarts.
2179> Type :? for other options.
2180]]>
2181    </programlisting>
2182    <para>
2183      Looking at a backtrace would presumably show what object and
2184      slot name were written.
2185    </para>
2186    <para>
2187      Note that even though the write was to slot-a, the uvector index
2188      was 1 (not 0). This is because the first element of a
2189      slot-vector is a pointer to the instance that owns the slots. We
2190      can retrieve that to look at the object that was modified:
2191    </para>
2192    <programlisting>
2193<![CDATA[
21941 > (uvref (write-to-watched-object-object *break-condition*) 0)
2195#<FOO #x30004113502D>
21961 > (describe *)
2197#<FOO #x30004113502D>
2198Class: #<STANDARD-CLASS FOO>
2199Wrapper: #<CLASS-WRAPPER FOO #x300041135EBD>
2200Instance slots
2201SLOT-A: #<Unbound>
2202SLOT-B: #<Unbound>
2203SLOT-C: #<Unbound>
22041 >
2205]]> 
2206    </programlisting>
2207    </refsect1>
2208  </refentry>
2209</sect2>
2210<sect2 id="watched-unwatch"><title>UNWATCH</title>
2211<refentry id="f_unwatch">
2212  <indexterm zone="f_unwatch">
2213    <primary>unwatch</primary>
2214  </indexterm>
2215 
2216  <refnamediv>
2217    <refname>UNWATCH</refname>
2218      <refpurpose>
2219        Stop monitoring a lisp object for writes.
2220      </refpurpose>
2221      <refclass>Function</refclass>
2222  </refnamediv>
2223 
2224  <refsynopsisdiv>
2225    <synopsis><function>unwatch</function> object</synopsis>
2226  </refsynopsisdiv>
2227 
2228  <refsect1><title>Description</title>
2229  <para>
2230    The UNWATCH function ensures that the specified object is in
2231    normal, non-monitored memory. If the object is not currently
2232    being watched, UNWATCH does nothing and returns NIL. Otherwise,
2233    the newly unwatched object is returned.
2234  </para>
2235  </refsect1>
2236</refentry>
2237</sect2>
2238<sect2 id="watched-write-to-watched-object">
2239  <title>WRITE-TO-WATCHED-OBJECT</title>
2240  <refentry id="c_write-to-watched-object">
2241    <indexterm zone="c_write-to-watched-object">
2242      <primary>write-to-watched-object</primary>
2243    </indexterm>
2244   
2245    <refnamediv>
2246      <refname>WRITE-TO-WATCHED-OBJECT</refname>
2247      <refpurpose>
2248        Condition signaled when a write to a watched object is attempted.
2249      </refpurpose>
2250      <refclass>Condition</refclass>
2251    </refnamediv>
2252
2253    <refsect1><title>Discussion</title>
2254    <para>
2255      This condition is signaled when a watched object is written
2256      to. There are three slots of interest:
2257    </para>
2258    <variablelist>
2259      <varlistentry>
2260        <term>object</term>
2261        <listitem>
2262          <para>
2263            The actual object that was the destination of the write.
2264          </para>
2265        </listitem>
2266      </varlistentry>
2267      <varlistentry>
2268        <term>offset</term>
2269        <listitem>
2270          <para>
2271            The byte offset from the tagged object pointer to the
2272            address of the write.
2273          </para>
2274        </listitem>
2275      </varlistentry>
2276      <varlistentry>
2277        <term>instruction</term>
2278        <listitem>
2279          <para>
2280            The disassembled machine instruction that attempted the write.
2281          </para>
2282        </listitem>
2283      </varlistentry>
2284    </variablelist>
2285    </refsect1>
2286
2287    <refsect1><title>Restarts</title>
2288    <para>
2289      A few restarts are provided: one will skip over the faulting
2290      write instruction and proceed; another offers to unwatch the
2291      object and continue.
2292    </para>
2293    <para>
2294      There is also an emulate restart. In some common cases, the
2295      faulting write instruction can be emulated, enabling the write
2296      to be performed without having to unwatch the object (and
2297      therefore let other threads potentially write to it). If the
2298      faulting instruction isn't recognized, the emulate restart will
2299      not be offered.
2300    </para>
2301    </refsect1>
2302  </refentry>
2303</sect2>
2304<sect2 id="watch-notes"><title>Notes</title>
2305<para>
2306  Although some care has been taken to minimize potential problems
2307  arising from watching and unwatching objects from multiple
2308  threads, there may well be subtle race conditions present that
2309  could cause bad behavior.
2310</para>
2311<para>
2312  For example, suppose that a thread attempts to write to a watched
2313  object. This causes the operating system to generate an
2314  exception. The lisp kernel figures out what the exception is, and
2315  calls back into lisp to signal the write-to-watched-object
2316  condition and perhaps handle the error.
2317</para>
2318<para>
2319  Now, as soon lisp code starts running again (for the callback),
2320  it's possible that some other thread could unwatch the very
2321  watched object that caused the exception, perhaps before we even
2322  have a chance to signal the condition, much less respond to it.
2323</para>
2324<para>
2325  Having the object unwatched out from underneath a handler may at
2326  least confuse it, if not cause deeper trouble. Use caution with
2327  unwatch.
2328</para>
2329</sect2>
2330<sect2 id="watch-examples"><title>Examples</title>
2331<para>
2332  Here are a couple more examples in addition to the above examples
2333  of watching a string and a standard-instance.
2334</para>
2335<sect3><title>Fancy arrays</title>
2336<programlisting>
2337?  (defvar *f* (make-array '(2 3) :element-type 'double-float))
2338*F*
2339? (watch *f*)
2340#(0.0D0 0.0D0 0.0D0 0.0D0 0.0D0 0.0D0)
2341;;; Note that the above vector is the underlying data-vector for the array
2342? (setf (aref *f* 1 2) pi)
2343> Error: Write to watched uvector #&lt;VECTOR 6 type DOUBLE-FLOAT, simple> at index 5
2344>        Faulting instruction: (movq (% rax) (@ -5 (% r8) (% rdi)))
2345> While executing: ASET, in process listener(1).
2346> Type :POP to abort, :R for a list of available restarts.
2347> Type :? for other options.
23481 >
2349  </programlisting>
2350  <para>
2351    In this case, uvector index in the report is the row-major index
2352    of the element that was written to.
2353  </para>
2354  </sect3>
2355  <sect3><title>Hash tables</title>
2356  <para>
2357    Hash tables are surprisingly complicated. The representation of a
2358    hash table includes an element called a hash-table-vector. The
2359    keys and values of the elements are stored pairwise in this
2360    vector.
2361  </para>
2362  <para>
2363    One problem with trying to monitor hash tables for writes is that
2364    the underlying hash-table-vector is replaced with an entirely new
2365    one when the hash table is rehashed. A previously-watched
2366    hash-table-vector will not be the used by the hash table after
2367    rehashing, and writes to the new vector will not be caught.
2368  </para>
2369  <programlisting>
2370? (defvar *h* (make-hash-table))
2371*H*
2372? (setf (gethash 'noise *h*) 'feep)
2373FEEP
2374? (watch *h*)
2375#&lt;HASH-TABLE-VECTOR #xDD00D>
2376;;; underlying hash-table-vector
2377? (setf (gethash 'noise *h*) 'ding)
2378> Error: Write to watched uvector #&lt;HASH-TABLE-VECTOR #xDD00D> at index 35
2379>        Faulting instruction: (lock)
2380>          (cmpxchgq (% rsi) (@ (% r8) (% rdx)))
2381> While executing: %STORE-NODE-CONDITIONAL, in process listener(1).
2382> Type :POP to abort, :R for a list of available restarts.
2383> Type :? for other options.
2384;;; see what value is being replaced...
23851 > (uvref (write-to-watched-object-object *break-condition*) 35)
2386FEEP
2387;;; backtrace shows useful context
23881 > :b
2389*(1A109F8) : 0 (%STORE-NODE-CONDITIONAL ???) NIL
2390 (1A10A50) : 1 (LOCK-FREE-PUTHASH NOISE #&lt;HASH-TABLE :TEST EQL size 1/60 #x30004117D47D> DING) 653
2391 (1A10AC8) : 2 (CALL-CHECK-REGS PUTHASH NOISE #&lt;HASH-TABLE :TEST EQL size 1/60 #x30004117D47D> DING) 229
2392 (1A10B00) : 3 (TOPLEVEL-EVAL (SETF (GETHASH # *H*) 'DING) NIL) 709
2393 ...
2394  </programlisting>
2395  </sect3>
2396  <sect3><title>Lists</title>
2397  <para>
2398    As previously mentioned, WATCH only watches individual cons cells.
2399  </para>
2400  <programlisting>
2401? (defun watch-list (list)
2402    (maplist #'watch list))
2403WATCH-LIST
2404? (defvar *l* (list 1 2 3))
2405*L*
2406? (watch-list *l*)
2407((1 2 3) (2 3) (3))
2408? (setf (nth 2 *l*) 'foo)
2409> Error: Write to the CAR of watched cons cell (3)
2410>        Faulting instruction: (movq (% rsi) (@ 5 (% rdi)))
2411> While executing: %SETNTH, in process listener(1).
2412> Type :POP to abort, :R for a list of available restarts.
2413> Type :? for other options.
2414  </programlisting>
2415  </sect3>
2416  </sect2>
2417</sect1>
2418
2419<sect1 id="code-coverage"><title>Code Coverage</title>
2420<sect2 id="code-coverage-overview"><title>Overview</title>
2421<para>
2422  In Clozure CL 1.4 and later, code coverage provides information
2423  about which paths through generated code have been executed and
2424  which haven't. For each source form, it can report one of three
2425  possible outcomes:
2426</para>
2427<itemizedlist>
2428  <listitem>
2429    <para>
2430      Not covered: this form was never entered.
2431    </para>
2432  </listitem>
2433  <listitem>
2434    <para>
2435      Partly covered: This form was entered, and some parts were
2436      executed and some weren't.
2437    </para>
2438  </listitem>
2439  <listitem>
2440    <para>
2441      Fully covered: Every bit of code generated from this form was
2442      executed.
2443    </para>
2444  </listitem>
2445</itemizedlist>
2446</sect2>
2447
2448<sect2 id="code-coverage-limitations"><title>Limitations</title>
2449<para>
2450  While the information gathered for coverage of generated code is
2451  complete and precise, the mapping back to source forms is of
2452  necessity heuristic, and depends a great deal on the behavior of
2453  macros and the path of the source forms through compiler
2454  transforms. Source information is not recorded for variables, which
2455  further limits the source mapping. In practice, there is often
2456  enough information scattered about a partially covered function to
2457  figure out which logical path through the code was taken and which
2458  wasn't. If that doesn't work, you can try disassembling to see which
2459  parts of the compiled code were not executed: in the disassembled
2460  code there will be references to #&lt;CODE-NOTE [xxx] ...> where xxx
2461  is NIL if the code that follows was never executed and non-NIL if it
2462  was.
2463</para>
2464<para>
2465  Sometimes the situation can be improved by modifying macros to try
2466  to preserve more of the input forms, rather than destructuring and
2467  rebuilding them.
2468</para>
2469<para>
2470  Because the code coverage information is associated with compiled
2471  functions, load-time toplevel expressions do not get reported
2472  on. You can work around this by creating a function and calling
2473  it. I.e. instead of
2474  <programlisting>
2475(progn
2476  (do-this)
2477  (setq that ...) ...))
2478  </programlisting>
2479  do:
2480  <programlisting>
2481(defun init-this-and-that ()
2482  (do-this)
2483  (setq that ...)  ...)
2484(init-this-and-that)
2485  </programlisting>
2486Then you can see the coverage information in the definition of
2487init-this-and-that.
2488</para>
2489</sect2>
2490
2491<sect2 id="code-coverage-usage"><title>Usage</title>
2492<para>
2493  In order to gather code coverage information, you first have to
2494  recompile all your code to include code coverage
2495  instrumentation. Compiling files will generate code coverage
2496  instrumentation if <literal>CCL:*COMPILE-CODE-COVERAGE*</literal>
2497  is true:
2498  <programlisting>
2499(setq ccl:*compile-code-coverage* t)
2500(recompile-all-your-files)
2501  </programlisting>
2502</para>
2503<para>
2504  The compilation process will be many times slower than normal, and
2505  the fasl files will be many times bigger.
2506</para>
2507<para>
2508  When you execute function loaded from instrumented fasl files, they
2509  will record coverage information every time they are executed. The
2510  system keeps track of which instrumented files have been loaded.
2511</para>
2512<para>
2513  The following functions can be used to manage the coverage data:
2514</para>
2515
2516<refentry id="f_report-coverage">
2517  <indexterm zone="f_report-coverage">
2518    <primary>report-coverage</primary>
2519  </indexterm>
2520 
2521  <refnamediv>
2522    <refname>REPORT-COVERAGE</refname>
2523    <refpurpose>Generate code coverage report</refpurpose>
2524    <refclass>Function</refclass>
2525  </refnamediv>
2526
2527  <refsynopsisdiv>
2528    <synopsis><function>report-coverage</function> &key;
2529    (external-format :default) (statistics t) (html t)
2530    </synopsis>
2531  </refsynopsisdiv>
2532
2533  <refsect1><title>Arguments and Values</title>
2534  <variablelist>
2535    <varlistentry>
2536      <term>html</term>
2537      <listitem>
2538        <para>
2539          If non-nil, this will generate an HTML report, consisting of
2540          an index file and one html file for each instrumented source
2541          file that has been loaded in the current session. The
2542          individual source file reports are stored in the same
2543          directory as the index file.
2544        </para>
2545      </listitem>
2546    </varlistentry>
2547    <varlistentry>
2548      <term>external-format</term>
2549      <listitem>
2550        <para>
2551          Controls the external format of the html files.
2552        </para>
2553      </listitem>
2554    </varlistentry>
2555    <varlistentry>
2556      <term>statistics</term>
2557      <listitem>
2558        <para>
2559          If :statistics is non-nil, a comma-separated file is also
2560          generated with the summary of statistics. You can specify a
2561          filename for the statistics argument, otherwise
2562          "statistics.csv" is created in the output directory. See
2563          documentation of ccl:coverage-statistics below for a
2564          description of the values in the statistics file.
2565        </para>
2566      </listitem>
2567    </varlistentry>
2568  </variablelist>
2569  </refsect1>
2570  <refsect1><title>Example</title>
2571    <para>
2572      If you've loaded <filename>foo.lx64fsl</filename> and
2573      <filename>bar.lx64fsl</filename>, and have run some tests, you could
2574      do
2575    <programlisting>
2576(CCL:REPORT-COVERAGE "/my/dir/coverage/report.html")
2577    </programlisting>
2578    and this would generate <filename>report.html</filename>,
2579    <filename>foo_lisp.html</filename> and
2580    <filename>bar_lisp.html</filename>, and
2581    <filename>statistics.csv</filename> all in
2582    <filename>/my/dir/coverage/</filename>.
2583    </para>
2584  </refsect1>
2585</refentry>
2586
2587<refentry id="f_reset-coverage">
2588  <indexterm zone="f_reset-coverage">
2589    <primary>reset-coverage</primary>
2590  </indexterm>
2591 
2592  <refnamediv>
2593    <refname>reset-coverage</refname>
2594    <refpurpose>
2595      Resets all coverage data back to the "Not Executed" state
2596    </refpurpose>
2597    <refclass>Function</refclass>
2598  </refnamediv>
2599
2600  <refsect1><title>Summary</title>
2601    <para>
2602      Resets all coverage data back to the "Not Executed" state
2603    </para>
2604  </refsect1>
2605</refentry>
2606
2607<refentry id="f_clear-coverage">
2608  <indexterm zone="f_clear-coverage">
2609    <primary>clear-coverage</primary>
2610  </indexterm>
2611 
2612  <refnamediv>
2613    <refname>clear-coverage</refname>
2614    <refpurpose>
2615      Forget about all instrumented files that have been loaded.
2616    </refpurpose>
2617    <refclass>Function</refclass>
2618  </refnamediv>
2619
2620  <refsect1><title>Summary</title>
2621    <para>
2622      Gets rid of the information about which instrumented files have
2623      been loaded, so ccl:report-coverage will not report any files,
2624      and ccl:save-coverage-in-file will not save any info, until more
2625      instrumented files are loaded.
2626    </para>
2627  </refsect1>
2628</refentry>
2629
2630<refentry id="f_save-coverage-in-file">
2631  <indexterm zone="f_save-coverage-in-file">
2632    <primary>save-coverage-in-file</primary>
2633  </indexterm>
2634 
2635  <refnamediv>
2636    <refname>save-coverage-in-file</refname>
2637    <refpurpose>
2638      Save all coverage into to a file so you can restore it later.
2639    </refpurpose>
2640    <refclass>Function</refclass>
2641  </refnamediv>
2642
2643  <refsynopsisdiv>
2644    <synopsis><function>save-coverage-in-file</function> pathname
2645    </synopsis>
2646  </refsynopsisdiv>
2647
2648  <refsect1><title>Summary</title>
2649    <para>
2650      Saves all coverage info in a file, so you can restore the
2651      coverage state later. This allows you to combine multiple runs
2652      or continue in a later session. Equivalent to
2653      (ccl:write-coverage-to-file (ccl:save-coverage) pathname).
2654    </para>
2655  </refsect1>
2656</refentry>
2657
2658<refentry id="f_restore-coverage-from-file">
2659  <indexterm zone="f_restore-coverage-from-file">
2660    <primary>restore-coverage-from-file</primary>
2661  </indexterm>
2662 
2663  <refnamediv>
2664    <refname>restore-coverage-from-file</refname>
2665    <refpurpose>
2666      Load coverage state from a file.
2667    </refpurpose>
2668    <refclass>Function</refclass>
2669  </refnamediv>
2670
2671  <refsynopsisdiv>
2672    <synopsis><function>restore-coverage-from-file</function> pathname
2673    </synopsis>
2674  </refsynopsisdiv>
2675
2676  <refsect1><title>Summary</title>
2677    <para>
2678      Restores the coverage data previously saved with
2679      CCL:SAVE-COVERAGE-IN-FILE, for the set of instrumented fasls
2680      that were loaded both at save and restore time. I.e. coverage
2681      info is only restored for files that have been loaded in this
2682      session. For example if in a previous session you had loaded
2683      "foo.lx86fsl" and then saved the coverage info, in this session
2684      you must load the same "foo.lx86fsl" before calling
2685      ccl:restore-coverage-from-file in order to retrieve the stored
2686      coverage info for "foo".  Equivalent to (ccl:restore-coverage
2687      (ccl:read-coverage-from-file pathname)).
2688    </para>
2689  </refsect1>
2690</refentry>
2691
2692<refentry id="f_save-coverage">
2693  <indexterm zone="f_save-coverage">
2694    <primary>save-coverage</primary>
2695  </indexterm>
2696 
2697  <refnamediv>
2698    <refname>save-coverage</refname>
2699    <refpurpose>
2700      Returns a snapshot of the current coverage data.
2701    </refpurpose>
2702    <refclass>Function</refclass>
2703  </refnamediv>
2704
2705  <refsect1><title>Summary</title>
2706    <para>
2707      Returns a snapshot of the current coverage data. A snapshot is a
2708      copy of the current coverage state. It can be saved in a file
2709      with ccl:write-coverage-to-file, reinstated back as the current
2710      state with ccl:restore-coverage, or combined with other
2711      snapshots with ccl:combine-coverage.
2712    </para>
2713  </refsect1>
2714</refentry>
2715
2716<refentry id="f_restore-coverage">
2717  <indexterm zone="f_restore-coverage">
2718    <primary>restore-coverage</primary>
2719  </indexterm>
2720 
2721  <refnamediv>
2722    <refname>RESTORE-COVERAGE</refname>
2723    <refpurpose>
2724      Reinstalls a coverage snapshot as the current coverage state.
2725    </refpurpose>
2726    <refclass>Function</refclass>
2727  </refnamediv>
2728
2729  <refsynopsisdiv>
2730    <synopsis><function>restore-coverage</function> snapshot
2731    </synopsis>
2732  </refsynopsisdiv>
2733
2734  <refsect1><title>Summary</title>
2735    <para>
2736      Reinstalls a coverage snapshot as the current coverage state.
2737    </para>
2738  </refsect1>
2739</refentry>
2740
2741<refentry id="f_write-coverage-to-file">
2742  <indexterm zone="f_write-coverage-to-file">
2743    <primary>write-coverage-to-file</primary>
2744  </indexterm>
2745 
2746  <refnamediv>
2747    <refname>WRITE-COVERAGE-TO-FILE</refname>
2748    <refpurpose>
2749      Save a coverage snapshot in a file.
2750    </refpurpose>
2751    <refclass>Function</refclass>
2752  </refnamediv>
2753
2754  <refsynopsisdiv>
2755    <synopsis><function>write-coverage-to-file</function> snapshot pathname
2756    </synopsis>
2757  </refsynopsisdiv>
2758
2759  <refsect1><title>Summary</title>
2760    <para>
2761      Saves the coverage snapshot in a file. The snapshot can be
2762      loaded back with ccl:read-coverage-from-file or loaded and
2763      restored with ccl:restore-coverage-from-file. Note that the file
2764      created is actually a lisp source file and can be compiled for
2765      faster loading.
2766    </para>
2767  </refsect1>
2768</refentry>
2769
2770<refentry id="f_read-coverage-from-file">
2771  <indexterm zone="f_read-coverage-from-file">
2772    <primary>read-coverage-from-file</primary>
2773  </indexterm>
2774 
2775  <refnamediv>
2776    <refname>READ-COVERAGE-FROM-FILE</refname>
2777    <refpurpose>
2778      Return the coverage snapshot saved in a file.
2779    </refpurpose>
2780    <refclass>Function</refclass>
2781  </refnamediv>
2782
2783  <refsynopsisdiv>
2784    <synopsis><function>read-coverage-from-file</function> pathname
2785    </synopsis>
2786  </refsynopsisdiv>
2787
2788  <refsect1><title>Summary</title>
2789    <para>
2790      Returns the snapshot saved in pathname. Doesn't affect the
2791      current coverage state. pathname can be the file previously
2792      created with ccl:write-coverage-to-file or
2793      ccl:save-coverage-in-file, or it can be the name of the fasl
2794      created from compiling such a file.
2795    </para>
2796  </refsect1>
2797</refentry>
2798
2799<refentry id="f_coverage-statistics">
2800  <indexterm zone="f_coverage-statistics">
2801    <primary>coverage-statistics</primary>
2802  </indexterm>
2803 
2804  <refnamediv>
2805    <refname>COVERAGE-STATISTICS</refname>
2806    <refpurpose>
2807      Returns a sequence of coverage-statistics objects, one per source file.
2808    </refpurpose>
2809    <refclass>Function</refclass>
2810  </refnamediv>
2811
2812  <refsynopsisdiv>
2813    <synopsis><function>coverage-statistics</function>
2814    </synopsis>
2815  </refsynopsisdiv>
2816
2817  <refsect1><title>Summary</title>
2818    <para>
2819      Returns a sequence ccl:coverage-statistics objects, one for each
2820      source file, containing the same information as that written to
2821      the statistics file by ccl:report-coverage. The following
2822      accessors are defined for ccl:coverage-statistics objects:
2823      <variablelist>
2824      <varlistentry>
2825        <term><function>ccl:coverage-source-file</function></term>
2826        <listitem>
2827          <para>
2828            the name of the source file corresponding to this information
2829          </para>
2830        </listitem>
2831      </varlistentry>
2832      <varlistentry>
2833        <term><function>ccl:coverage-expressions-total</function></term>
2834        <listitem>
2835          <para>
2836            the total number of expressions
2837          </para>
2838        </listitem>
2839      </varlistentry>
2840      <varlistentry>
2841        <term><function>ccl:coverage-expressions-entered</function></term>
2842        <listitem>
2843          <para>
2844            the number of source expressions that have been entered
2845            (i.e. at least partially covered)
2846          </para>
2847        </listitem>
2848      </varlistentry>
2849      <varlistentry>
2850        <term><function>ccl:coverage-expressions-covered</function></term>
2851        <listitem>
2852          <para>
2853            the number of source expressions that were fully covered
2854          </para>
2855        </listitem>
2856      </varlistentry>
2857      <varlistentry>
2858        <term><function>ccl:coverage-unreached-branches</function></term>
2859        <listitem>
2860          <para>
2861            the number of conditionals with one branch taken and one not taken
2862          </para>
2863        </listitem>
2864      </varlistentry>
2865      <varlistentry>
2866        <term><function>ccl:coverage-code-forms-total</function></term>
2867        <listitem>
2868          <para>
2869            the total number of code forms. A code form is an
2870            expression in the final stage of compilation, after all
2871            macroexpansion and compiler transforms and simplification
2872          </para>
2873        </listitem>
2874      </varlistentry>
2875      <varlistentry>
2876        <term><function>ccl:coverage-code-forms-covered</function></term>
2877        <listitem>
2878          <para>
2879            the number of code forms that have been entered
2880          </para>
2881        </listitem>
2882      </varlistentry>
2883      <varlistentry>
2884        <term><function>ccl:coverage-functions-total</function></term>
2885        <listitem>
2886          <para>
2887            the total number of functions
2888          </para>
2889        </listitem>
2890      </varlistentry>
2891      <varlistentry>
2892        <term><function>ccl:coverage-functions-fully-covered</function></term>
2893        <listitem>
2894          <para>
2895            the number of functions that were fully covered
2896          </para>
2897        </listitem>
2898      </varlistentry>
2899      <varlistentry>
2900        <term><function>ccl:coverage-functions-partly-covered</function></term>
2901        <listitem>
2902          <para>
2903            the number of functions that were partly covered
2904          </para>
2905        </listitem>
2906      </varlistentry>
2907      <varlistentry>
2908        <term><function>ccl:coverage-functions-not-entered</function></term>
2909        <listitem>
2910          <para>
2911            the number of functions never entered
2912          </para>
2913        </listitem>
2914      </varlistentry>
2915      </variablelist>
2916    </para>
2917  </refsect1>
2918</refentry>
2919
2920<refentry id="v_compile-code-coverage">
2921  <indexterm zone="v_compile-code-coverage">
2922    <primary>*COMPILE-CODE-COVERAGE*</primary>
2923  </indexterm>
2924 
2925  <refnamediv>
2926    <refname>*COMPILE-CODE-COVERAGE*</refname>
2927    <refpurpose>
2928      When true, instrument functions for code coverage.
2929    </refpurpose>
2930    <refclass>Variable</refclass>
2931  </refnamediv>
2932
2933  <refsynopsisdiv>
2934    <synopsis><varname>*compile-code-coverage*</varname>
2935    </synopsis>
2936  </refsynopsisdiv>
2937
2938  <refsect1><title>Summary</title>
2939    <para>
2940      This variable controls whether functions are instrumented for
2941      code coverage. Files compiled while this variable is true will
2942      contain code coverage instrumentation.
2943    </para>
2944  </refsect1>
2945</refentry>
2946
2947<refentry id="v_without-compiling-code-coverage">
2948  <indexterm zone="v_without-compiling-code-coverage">
2949    <primary>without-compiling-code-coverage</primary>
2950  </indexterm>
2951 
2952  <refnamediv>
2953    <refname>WITHOUT-COMPILING-CODE-COVERAGE</refname>
2954    <refpurpose>
2955      Don't record code coverange for forms within the body.
2956    </refpurpose>
2957    <refclass>Macro</refclass>
2958  </refnamediv>
2959
2960  <refsynopsisdiv>
2961    <synopsis><function>without-compiling-code-coverage</function>
2962    </synopsis>
2963  </refsynopsisdiv>
2964
2965  <refsect1><title>Summary</title>
2966    <para>
2967      This macro arranges so that body doesn't record internal details
2968      of code coverage. It will be considered totally covered if it's
2969      entered at all. The Common Lisp macros ASSERT and CHECK-TYPE use
2970      this macro.
2971    </para>
2972  </refsect1>
2973</refentry>
2974
2975</sect2>
2976</sect1>
2977</chapter>
Note: See TracBrowser for help on using the repository browser.