source: branches/working-0710/ccl/release-notes.txt @ 7408

Last change on this file since 7408 was 6961, checked in by gb, 14 years ago

Add more stuff.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Author Date Id Revision
File size: 101.9 KB
1OpenMCL 1.1-pre-070722
2- This will hopefully be the last set of snapshots whose version
3  number contains the string "pre-"; whether or not the last
4  20 months worth of "1.1-pre-yymmdd" snapshot releases are
5  more or less stable than something without "pre-" in its name
6  doesn't have too much to do much to do with whether or not "pre-"
7  is in the version number (and has lots to do with other things.)
8  I'd like to move to a model that's mostly similar to how things
9  have been (new version every month or two, old versions become
10  obsolete soon after, sometimes changes introduce binary incompatiblity)
11  but drop the "prerelease" designation and change the name of the
12  "testing" directory to something like "current".
13- The FASL version didn't change (for the first time in a long time.)
14  It's probably a lot easier to bootstrap new sources with a new
15  lisp and it's probably desirable to recompile your own source
16  code with the new lisp, but there shouldn't be any user-visible
17  low-level ABI changes that make that mandatory.
18- CCL::WITH-ENCODED-CSTRS (which has been unexported and somewhat
19  broken) is now exported and somewhat less broken.
21  (ccl:with-encoded-cstrs ENCODING-NAME ((varI stringI)*) &body body)
23  where ENCODING-NAME is a keyword constant that names a character
24  encoding executes BODY in an environment where each variable varI
25  is bound to a nul-terminated, dynamic-extent foreign pointer to
26  an encoded version of the corresponding stringI.
28  (ccl:with-cstrs ((x "x")) (#_puts x))
30  is functionally equivalent to:
32  (ccl:with-encoded-cstrs :iso-8859-1 ((x "x")) (#_puts x))
35  CCL:WITH-ENCODED-CSTRS doesn't automatically prepend byte-order-marks
36  to its output; the size of the terminating #\NUL depends on the
37  number of octets-per-code-unit in the encoding.
39  There are certainly lots of other conventions for expressing
40  the length of foreign strings besides NUL-termination (length in
41  code units, length in octets.)  I'm not sure if it's better to
42  try to come up with high-level interfaces that support those
43  conventions ("with-encoded-string-and-length-in-octets ...")
44  or to try to support mid-level primitives ("number of octets
45  in encoded version of lisp string in specified encoding", etc.)
47- STREAM-ERRORs (and their subclasses, including READER-ERROR)
48  try to describe the context in which they occur a little better
49  (e.g., by referencing the file position if available and
50  by trying to show a few surrounding characters when possible.)
51  Since streams are usually buffered, this context information
52  may be incomplete, but it's often much better than nothing.
54- Hashing (where some objects are hashed by address) and OpenMCL's
55  GC (which often changes the addresses of lisp objects, possibly
56  invalidating hash tables in which those objects are used as keys)
57  have never interacted well; to minimize the negative effects of
58  this interaction, most primitive functions which access hash
59  tables has disabled the GC while performing that access, secure
60  in the knowledge that hash table keys won't be moving around
61  (because of GC activity in other threads) while the hash table
62  lookup is being performed.
64  Disabling and reenabling the GC can be somewhat expensive, both
65  directly (in terms of the primitive operations used to do so)
66  and indirectly (in terms of the cost of - temporarily - not being
67  able to GC when otherwise desirable.)  If the GC runs (and possibly
68  moves a hash-table key) very rarely relative to the frequency of
69  hash-table access - and that's probably true, much of the time -
70  then it seems like it'd be desirable to avoid the overhead of
71  disabling/reenabling the GC on every hash table access, and it'd
72  be correct to do this as long as we're careful about it.
74  I was going to try to change all hash-table primitives to try
75  to make them avoid inhibiting/enabling the GC for as long as
76  possible, but wimped out and only did that for GETHASH.  (If
77  another thread could GC while we're accessing a hash table, there
78  can still be weird intercations between things like the GC's
79  handling of weak objects and code which looks at the hash table,
80  and that weirdness seemed easier to deal with in the GETHASH case
81  than in some others.)
83  If GETHASH's performance has improved without loss of correctness,
84  then it'd likely be worth trying to make similar changes to
85  REMHASH and CCL::PUTHASH (which implements (SETF (GETHASH ...) ...).
86  If problems are observed or performance still hasn't improved, it'd
87  probably be worth re-thinking some of this.
89- Leading tilde (~) characters in physical pathname namestrings
90  are expanded in the way that most shells do:
92  "~user/...." can be used to refer to an absolute pathname rooted
93  at the home directory of the user named "user"
95  "~/..." can be used to refer to an absulte pathname rooted at
96  the home directory of the current user.
98- The break-loop colon commands for showing the contents of
99  stack frames try to present the frame's contents in a way that's
100  (hopefully) more meaningful and useful.  For each stack frame
101  shown in detail, the corresponding function's argument list
102  is printed, followed by the current values of the function's
103  arguments (indented slightly), a blank line, and the current
104  values of the function's local variables (outdented slightly.)
105  The old method of showing a stack frame's "raw" contents is
106  still available as the :RAW break loop command.
108  The new style of presenting a stack-frame's contents is also
109  used in the Cocoa IDE.
111- It's historically been possible to create stacks (for threads
112  other than the original one) whose size exceeds the nominal
113  OS resource limits for a stack's size.  (OpenMCL's threads
114  use multiple stacks; the stack in question is the one that
115  OpenMCL generally refers to as the "control" or "C" stack.)
116  It's not entirely clear what (if anything) the consequences
117  of exceeding these limits have been, but OpenMCL's GC can
118  use all of the available (C) stack space that it thinks it
119  has under some conditions, and, under OSX/Mach/Darwin, there
120  have been reports of excessive page file creation and paging
121  activity that don't seem related to heap behavior in environments
122  where the GC is running on (and possibly using much of) a stack
123  whose size greatly exceeds the hard resource limit on stack
124  size.
126  Trying to determine exactly what was causing the excessive
127  pages got me trapped in a twisty maze of Mach kernel sources,
128  all alike.  I tried to pin C stack size to the hard resource
129  limit on stack size and have not been able to provoke the
130  excessive paging problems since, but am not confident in
131  concluding (yet) that the problems had to do with resource
132  limits being exceeded.
134  The hard resource limits on stack size for the OS versions
135  that I have readily available (in bash, do "ulimit -s -H";
136  in tcsh, it's "limit -h s", don't know offhand about other
137  shells) are:
139  unlimited on Linux
140  ~512M on FreeBSD
141  ~64M on Darwin
143  The effect of observing (rather than exceeding) this limit
144  on the maximum depth of lisp recursion in OpenMCL is:
146  * nothing, on x86-64 (the C stack is not used by lisp code
147    on x86-64)
149  * visible on ppc32, which uses 4 32-bit words on the control
150    stack for each lisp function invocation
152  * more visible on ppc64, which uses 4 64-bit words of control
153    stack for each lisp function invocation.
155  That seems to suggest that (given that the actual stack resource
156  limit is a bit under 64M and that OpenMCL signals stack overflow
157  when the stack pointer gets within a few hundred KB of the actual
158  limit) that ppc64 threads are now limited to a maximum of about
159  2000000 function calls.
161  (All of this only matters if attempts are made to create threads
162  with large stacks; the default stack sizes in OpenMCL are usually
163  1-2 MB.)
165- On a cheerier (and certainly less confusing) note: for the last few
166  years, OpenMCL has shipped with an extended example which provides an
167  integrated development environment (IDE) based on Cocoa; that's often
168  been described as "the demo IDE" and could also be fairly described as
169  "slow", "buggy", "incomplete", and "little more than a proof of
170  concept."
172  I think that it's fair to describe the current state of the IDE as
173  being "less slow", "less buggy", "less incomplete", and "much more
174  than a proof of concept" than it has been (e.g., there's been some
175  actual progress over the last few months and there are plans to
176  try to continue working on the IDE and related tools.)  It'd probably
177  be optimistic to call it "usable" in its current state (that may
178  depend on how low one's threshold of usability is), but I hope that
179  people who've been discouraged by the lack of IDE progress over the
180  last few years will see reason to be encouraged (and that anyone
181  interested will submit bug reports, patches, feature requests, code ...)
183- There are now "objc-bridge" and "cocoa-ide" subdirectories; by default,
184  REQUIRE will look in these directories for files whose name matches
185  a module name.  Several files were moved from the "examples" directory
186  to "objc-bridge"; other example files, the "" skeleton
187  bundle, and the "hemlock" directory were moved to "cocoa-ide".
190OpenMCL 1.1-pre-070512
191- The FASL version changed (old FASL files won't work with this
192  lisp version), as did the version information which tries to
193  keep the kernel in sync with heap images.  Note that it's generally
194  a lot easier to recompile recent sources with recent images, e.g.,
195  trying to compile 070512 sources with an 070407 image is unlikely
196  to work without tricky bootstrapping.
197- Most of the changes in this release involve the calling sequence
198  used on x86-64.  In very general terms, some kinds of function-call
199  intensive code may see a significant performance boost, most code
200  should see a slight improvement, some code might see a (hopefully
201  very slight) degradation, and anything significantly slower than
202  previous releases should be reported as a bug.
203  It is -possible- that some of these changes may cause errors to
204  be reported differently (the function reported as the function
205  executing when the error ocurred might be different/wrong).  I
206  have not seen as many cases of this as I expected to when making
207  the change, but am also not sure that I fixed all possible cases.
208- The FFI-related reader macros #_, #$, and #& all read a case-sensitive
209  foreign function, constant, or variable name from the input stream
210  and try to find the corresponding definition in the interface files.
211  If the name is prefixed with a #\? - as in #_?foo - the macros
212  return true if the definition could be found and false otherwise.
213  (The general idea is that this might be useful for conditionalizing
214  code in some cases, and there should be -some- way of quietly testing
215  that something's defined.)
216- There is now support for making the contents of (possibly very large)
217  files accessible as lisp vectors.  (This may be many times faster
218  than something like
220  (let* ((stream (open pathname :direction :input :element-type 'whatever))
221         (vector (make-array (file-size-to-vector-size stream)
222                             :element-type 'whatever)))
223    (read-sequence vector stream))
225  but has the similar effect of making the contents of VECTOR match the
226  contents of the file.)
228  CCL:MAP-FILE-TO-IVECTOR pathname element-type [Function]
230  "element-type" should be a type specifier such that
231  (UPGRADED-ARRAY-ELEMENT-TYPE element-type) is a subtype
232  of either SIGNED-BYTE or UNSIGNED-BYTE.
234  Tries to open the file named by "pathname" for reading and to
235  map its contents into the process's address space via #_mmap;
236  if successful, returns a lisp vector of element-type
237  (UPGRADED-ARRAY-ELEMENT-TYPE element-type) which is displaced
238  to an underlying (SIMPLE-ARRAY element-type (*)) whose contents
239  match the mapped file's.
241  Because of alignment issues, the mapped file's contents will
242  start a few bytes (4 bytes on 32-bit platforms, 8 bytes on 64-bit
243  platforms) "into" the vector; the displaced array returned by
244  CCL:MAP-FILE-TO-IVECTOR hides this overhead, but its usually
245  more efficient to operate on the underlying simple 1-dimensional
246  array.  Given a displaced array (like the value returned by
248  returns the underlying array and the displacement index in elements.
250  Currently, only read-only file mapping is supported; the underlying
251  vector will be allocated in read-only memory, and attempts to use
252  (e.g.) (SETF (AREF ...) ...) to modify the mapped vector's contents
253  will result in memory faults.
255  CCL:MAP-FILE-TO-OCTET-VECTOR pathname [Function]
257  Equivalent to (CCL:MAP-FILE-TO-IVECTOR pathname '(UNSIGNED-BYTE 8)).
259  CCL:UNMAP-IVECTOR displaced-vector
261  If the argument is a mapped vector (as returned by
262  MAP-FILE-TO-IVECTOR) that has not yet been "unmapped" by this
263  function, undoes the memory mapping, closes the mapped file, and
264  adjusts its argument so that it's displaced to a 0-length vector.
268  Note that whether a vector's created by MAKE-ARRAY or by mapping
269  a file's contents, it can't have ARRAY-TOTAL-SIZE-LIMIT or more
270  elements.  (ARRAY-TOTAL-SIZE-LIMIT is (EXPT 2 24) in 32-bit OpenMCL
271  and (EXPT 2 56) in 64-bit versions.
273- The lisp kernel now tries to signal memory faults that occur when
274  running lisp code as lisp errors.  As a silly example:
276  ? (defun foo (x)
277     "Crash and burn if X is not a list"
278     (declare (optimize (speed 3) (safety 0)) (list x))
279     (car x))
280  FOO
281  ? (foo 0)
282  > Error: Fault during read of memory address #x4
283  > While executing: FOO, in process listener(1).
285  The fact that things are handled this way (rather than going
286  into the kernel debugger with no easy way of recovering) makes
287  it possible to continue a session without losing work in many
288  cases.  In a trivial example like the one above, it's relatively
289  easy to see that no harm has been done and the error should
290  not be hard to recover from.  In some other cases, it may be
291  true that a buggy function has been scribbling ofer memory for
292  a while before that scribbling resulted in a machine exception.
294  Moral: if you get an unexpected "memory fault" error (the
295  condition type is actually CCL::INVALID-MEMORY-ACCESS) and
296  don't understand why the fault occurred and the consequences
297  of continuing in the lisp session where the fault occurred,
298  you should view the state of that session with some suspicion.
300  Faults in foreign code (should) still trap into the kernel
301  debugger.  (It'd be nice to be able to treat these as lisp
302  errors with the same caveats as described above, but that
303  is more complicated in some cases and isn't yet implemented.)
305- An obscure kernel debugger command - (A), which tries to advance
306  the program counter by one instruction - is now disabled on x86-64.
307  (On the PPC, "one instruction" always meant "4 bytes"; implementing
308  this correctly on x86-64 would require the ability to at least
309  partially disassemble arbitrary x86-64 instructions.)
311  On the other hand, the kernel debugger should be able to show
312  FPU registers on x86-64.
315OpenMCL 1.1-pre-070408
316- The FASL version changed (old FASL files won't work with this
317  lisp version), as did the version information which tries to
318  keep the kernel in sync with heap images.  Note that it's generally
319  a lot easier to recompile recent sources with recent images, e.g.,
320  trying to compile 070408 sources with an 070214 image is unlikely
321  to work without tricky bootstrapping.
322- There's now a Trac bug-tracking/wiki site for OpenMCL at
323  <>.  It needs bug reports; please
324  visit that site and use the features there to report any bugs
325  that you find.
327  (CCL:DEFSTATIC var value &optional doc-string)
328  is like DEFPARAMETER in that it proclaims the variable "var" to
329  be special, sets its value to "value", and sets the symbol's
330  VARIABLE documentation to the optional doc-string.  It differs
331  from DEFPARAMETER in that it further asserts that the variable
332  should never be bound dynamically in any thread (via LET/LAMBDA/etc.);
333  the compiler treats any attempts to bind a "static" variable as an
334  error.
335  It is legal to change the value of a "static" variable, but since
336  all threads see the same (static) binding of that variable it may
337  be necessary to synchronize assignments made from multiple threads.
338  (A "static" variable binding is effectively a shared, global resource;
339  a dynamic binding is thread-private.)
340  Access to the value of a static variable is typically faster than
341  is access to the value a special variable that's not proclaimed to
342  be "static".
343  This functionality has been in MCL/OpenMCL for a long time under
344  the name CCL:DEFGLOBAL; CCL:DEFGLOBAL is an alias for CCL:DEFSTATIC,
345  but the latter seemed to be a better name.
346- The type of foreign object that a MACPTR points to can now be
347  asserted (this means that a MACPTR object can contain a small
348  integer which identifies the alleged FOREIGN-TYPE of the object that
349  the points to.  RLET, MAKE-RECORD, and MAKE-GCABLE-RECORD (see below)
350  assert the foreign type of the object that the MACPTR object they
351  create (as do some new features of the ObjC bridge, described further
352  below.)
353  PRINT-OBJECT on a MACPTR will try to print information about the
354  asserted type of that pointer, as well as information about where
355  the pointer was allocated (heap, stack) and whether it's scheduled
356  for automatic reclamation by the GC.
357  A few constructs that conceivable should assert the type of the
358  pointers they create (e.g., some flavor of PREF, SLOT-VALUE in
359  the ObjC bridge) don't yet do so.
360  A rather obvious way of exploiting typed pointers - namely, extending
361  DESCRIBE and INSPECT to show the contents of foreign records - is
362  not yet implemented.
363- MAKE-GCABLE-RECORD is like MAKE-RECORD, in that it "makes an instance
364  of a foreign record type".  (Or, to be more banal about it, uses
365  #_malloc to allocate a block of foreign memory of the size of the
366  foreign record type named by its argument.)  MAKE-GCABLE-RECORD
367  additionally tells the lisp garbage collector that it should free
368  the foreign memory when the MACPTR object that describes it becomes
369  garbage.
370  When using "gcable pointers", it's important to remember the
371  distinction between a MACPTR object (which is a lisp object, more-
372  or-less like any other) and the block of foreign memory that the
373  MACPTR object points to.  If a gcable MACPTR is the only thing
374  in the world ("lisp world" or "foreign world") that references
375  the underlying block of foreign memory, then freeing the foreign
376  memory when it becomes impossible to reference it is convenient
377  and sane.  If other lisp MACPTRs reference the underlying block
378  of foreign memory or if the address of that foreign memory is
379  passed to and retained by foreign code, having the GC free the
380  memory may have unpleasant consequences if those other references
381  are used.
382- CCL:FREE (which is mostly just a wrapper around #_free that allows
383  #_free to be called early in the bootstrapping process) is now
384  exported; if its argument is a gcable pointer (e.g., created via
385  MAKE-GCABLE-POINTER), it will tell the GC that the pointer's
386  foreign memory has been freed "manually" before calling #_free.
387- The mechanisms used to implement locks has changed (for the curious,
388  the changes involve the use of spinlocks rather than a sequence
389  of atomic additions.)  Someone demonstrated a case of deadlock
390  (where a thread was waiting for a lock that was available) under
391  the old implementation.  I'm not sure that I fully understand how
392  that could have happened, but the new implementation at least has
393  the advantage of being a little easier to understand and might be
394  a tiny bit faster.  Please let me know if either of these assumptions
395  was incorrect.
396- An EOF (control-d) in the REPL (when standard input is a tty or pty
397  device) has traditionally caused an exit to the outer break loop
398  (or had no effect if the REPL was not in a break loop).  If
399  CCL:*QUIT-ON-EOF* is set, an EOF causes the lisp to quit.  (It
400  actually invokes a listener-specific method, so in a multi-listener
401  window system environemt, it might simply cause the listener which
402  receives the EOF to exit.)
403  None of this has any effect when running under environments like
404  SLIME, and (as mentioned above) only matters if the standard input
405  devices is a tty or pseudo-tty (where it's possible to continue
406  reading after an EOF has been read.)  If running under an xterm
407  or OSX, standard input is probably a pty; if running
408  in an Emacs shell buffer or under other means under emacs, different
409  types of IPC mechanisms (pipes, sockets) might be used.
410- SAVE-APPLICATION has historically changed the type of all MACPTR
411  objects (turning them into "dead macptrs", since it's generally
412  meaningless to refer to a foreign pointer from a previous session
413  and generally better to get a type error than some more mysterious
414  or severe failure).  This no longer happens for null pointers (pointers
415  to address 0); COMPILE-FILE also now allows null pointers to be referenced
416  as constants in compiled code.
417- Not entirely coincidentally, CCL:+NULL-PTR+ is now defined as a constant
418  (whose value is a null pointer.)  In some cases, it may be more
419  efficient or convenient to pass CCL:+NULL-PTR+ to foreign code than
420  it would be to call (CCL:%NULL-PTR) to "produce" one.
421- Historically, OpenMCL (and MCL) have maintained a list of open file
422  streams in the value of CCL:*OPEN-FILE-STREAMS*; maintaining this
423  list helps to ensure that streams get closed in as orderly a manner
424  as possible when the lisp exits.  The code which accessed this list
425  didn't do so in a thread-safe manner.
426  The list is now maintained in a lexical variable; the function
427  CCL:OPEN-FILE-STREAMS returns a copy of that list,
428  CCL:NOTE-OPEN-FILE-STREAM adds its argument (a file stream) to the
429  list, and CCL:REMOVE-OPEN-FILE-STREAM removes its argument (a file stream)
430  from the list.  (All of these functions use appropriate locking.)
431- There were a number of timing-related problems related to PROCESS-INTERRUPT
432  (usually involving rapidly and repeatedly interrupting a thread over
433  a long period of time.)  This should be a lot more reliable now
434  (exactly what could go wrong and why and how is all a little hard to
435  describe.)
436- Some Linux distributions may initialize the user's environment in
437  a way that imposes a soft limit on the amount of virtual memory that
438  a process is allowed to map.  OpenMCL now tries to raise this limit
439  before reserving what may be a very large amount of address space,
440  thanks to a patch from Andi Kleen.
441- There were a number of problems with UTF-16 streams, found and
442  fixed by Takehiko Abe.
443- Takehiko Abe also provided fixes for some code in "ccl:lib;xref.lisp"
444  and in source-file recording/reporting that (still) didn't understand
445  the concept of EQL-SPECIALIZER metaobjects.
446- ObjC bridge and ObjC examples
447  The ObjC bridge provides a few new mechanisms for defining ObjC
448  methods, for calling ObjC "generic functions" (e.g., message sending),
449  and for dealing with frequently-used record types and with differences
450  between 32-bit and (forthcoming) 64-bit ObjC/Cocoa implementations.
452  A lot of macros/functions/other things that really should have been
453  exported from some package for the last few years finally have been
454  exported from the OBJC or NS packages (and a lot of things that have
455  historically been internal to CCL are re-imported into CCL).
457  Cocoa (and the underlying Core Graphics libraries) have historically
458  used 32-bit floats and 32-bit integers in data structures that describe
459  geometry, font sizes and metrics, and elsewhere.  64-bit Cocoa will
460  use 64-bit floats and 64-bit integers in many cases.
462  The bridge defines the type NS:CGFLOAT as the lisp type of the
463  preferred float type on the platform, and the constant NS:+CGFLOAT+.
464  On DarwinPPC32, the foreign types :cgfloat, :<NSUI>nteger, and
465  :<NSI>nteger are defined by the bridge (as 32-bit float, 32-bit
466  unsigned integer, and 32-bit signed integer, respectively.); these
467  types are defined (as 64-bit variants) in the 64-bit interfaces.
469  All ObjC classes are properly named, either with a name exported
470  from the NS package (in the case of a predefined class declared in
471  the interface files) or with the name provided in the DEFCLASS
472  form (with :METACLASS NS:+NS-OBJECT) which defines the class from
473  lisp.  The class's lisp name is now proclaimed to be a "static"
474  variable (as if by DEFSTATIC, as described above) and given the
475  class object as its value.  In other words:
477(send (find-class 'ns:ns-application) 'shared-application)
479  and
481(send ns:ns-application 'shared-application)
483  are equivalent.  (Since it's not legal to bind a "static" variable,
484  it may be necessary to rename some things so that unrelated
485  variables whose names coincidentally conflict with ObjC class names
486  don't do so.)
488- A new reader macro - #/ - reads a sequence of "constituent" characters
489  (including colons) from the stream on which it appears and interns
490  that sequence - with case preserved and colons intact - in a new package
491  whose name is NEXTSTEP-FUNCTIONS, exporting the symbol from that package.
492  This means that the act of reading "#/alloc" returns the the symbol
493  NEXTSTEP-FUNCTIONS:|alloc|, and the act of reading "#/initWithFrame:"
494  returns the symbol NEXTSTEP-FUNCTIONS:|initWithFrame:|.  The intent
495  is that the reader macro can be used to construct symbols whose names
496  match ObjC message names; the reader macro tries to do some sanity
497  checks (such as insisting that a name that contains at least one
498  colon ends in a colon), but isn't totally rigourous about enforcing
499  ObjC message name conventions.
501  A symbol read using this macro can be used as an operand in
502  most places where an ObjC message name can be used, such as
503  in the (@SELECTOR ...) construct (which is now OBJC:@SELECTOR,
504  btw.)
506  Marco Baringer proposed the idea of using a reader macro to
507  construct lisp symbols which matched ObjC message names.
509- The act of interning a new symbol in the NEXTSTEP-FUNCTIONS
510  package triggers an interface database lookup of Objc methods
511  with the corresponding message name.  If any such information
512  is found, a special type of dispatching function is created
513  and initialized and the weird-looking symbol is given that
514  dispatching function as its function definition.
516  The dispatching knows how to call declared ObjC methods defined on
517  the message.  In many cases, all methods have the same foreign type
518  signature, and the dispatching function merely passes any arguments
519  that it receives to a function that does an ObjC message send with
520  the indicated foreign argument and return types.  In other cases,
521  where different ObjC messages have different type signatures, the
522  dispatching function tries to choose a function that handles the
523  right type signature based on the class of the dispatching function's
524  first argument.
526  If new information about ObjC methods is introduced (e.g., by
527  using additional interface files or as ObjC methods are defined
528  from lisp), the dispatch function is reinitialized to recognize
529  newly-introduced foreign type signatures.
531  The argument and result coercion that the bridge has tradionally
532  supported is supported by the new mechanism (e.g., :<BOOL> arguments
533  can be specified as lisp booleans and :<BOOL> results are returned
534  as lisp boolean values, and an argument value of NIL is coerced to
535  a null pointer if the corresponding argument type is :ID.
537  Some ObjC methods accept variable numbers of arguments; the
538  foreign types of non-required arguments are determined by the
539  lisp types of those arguments (e.g., integers are passed as
540  integers, floats as floats, pointers as pointers, record types
541  by reference.)
543  Some examples:
545;;; #/alloc is a known message.
546? #'#/alloc
548;;; Sadly, #/foo is not ...
549? #'#/foo
550> Error: Undefined function: NEXTSTEP-FUNCTIONS:|foo|
552;;; We can send an "init" message to a newly-allocated instance of
553;;; "NSObject" by:
555(send (send ns:ns-object 'alloc) 'init)
557;;; or by
559(#/init (#/alloc ns:ns-object))
561  ObjC methods that "return" structures return them as gcable pointers
562  when called via dispatch functions.  E.g., if "my-window" is an
563  NS:NS-WINDOW instance, then
565(#/frame my-window)
567  will return a gcable pointer to a structure that describes that window's
568  frame rectangle.  (The good news is that there's no need to use SLET
569  or special structure-returning message send syntax; the bad news is
570  that #_malloc, #_free, and the GC are all involved in the creation
571  and eventual destruction of structure-typed return values.  Unless
572  and until those factors negatively affect performance, the advantages
573  seem to outweigh the disadvantages.)
575- Since foreign pointers are now (sometimes, somewhat) typed, it's
576  possible to treat pointers to some foreign types as "instances of
577  built-in classes."  Specifically, a pointer to an :<NSR>ect is
578  recognized as an instance of the built-in class NS:NS-RECT, a
579  pointer to an <NSS>ize is treated as an instance of NS:NS-SIZE,
580  <NSP>oint is recognized as NS:NS-POINT, and <NSR>ange maps to
581  NS:NS-RANGE.  (There are a few other more obscure structure
582  types that get this treatment, and with a little more work the
583  mechanism could be made extensible.)
585  For each of these built-in classes:
587  - a PRINT-OBJECT method is defined
589  - a foreign type name derived from the class name (e.g., :NS-RECT
590    for NS:NS-RECT) is made an alias for the corresponding type
591    (so it's possible to say (RLET ((R :NS-RECT)) ...)).
593  - the class is is integrated into the type system (so that
594    (TYPEP R 'NS:NS-RECT) is fairly efficently implemented.)
596  - inlined accessor and setf inverses are defined for the structure
597    type's fields.  In the case of an :<NSR>ect, the fields in question
598    are the fields of the embedded point and size, so NS:NS-RECT-X,
600    are defined.  The accessors and setter functions typecheck their
601    arguments and the setters handle coercion to the approprate type
602    of CGFLOAT where applicable.
604  - an initialization function is defined; (NS:INIT-NS-SIZE s w h) is
605    roughly equivalent to (SETF (NS:NS-SIZE-WIDTH s) w
606    (NS:NS-SIZE-HEIGHT s) h), but might be a little more efficient.
608  - a creation function is defined: (NS:NS-MAKE-POINT x y) is basically
609    equivalent to:
611      (NS:INIT-NS-POINT P X Y)
612      p)
614  - a macro is defined which (much like RLET) stack-allocates an
615    instance of the foreign record type, optionally iniitializes
616    that instance, and executes a body of code with a variable
617    bound to that instance.  E.g.
619    (ns:with-ns-range (r loc len)
620      (format t "~& range has location ~s, length ~s"
621         (ns:ns-range-location r) (ns:ns-range-length r)))
623    which is probably not the world's most realistic example.
625   Note that it's possible to construct a record
626   instance that has a very short useful lifetime:
628   (#/initWithFrame: new-view (ns:ns-make-rect 100 100 200 200))
630   The rectangle above will -eventually- get reclaimed by the GC;
631   if you don't want to give the GC so much work to do, you might
632   prefer to do:
634   (ns:with-ns-rect (r 100 100 200 200)
635     (#/initWithFrame: new-view r))
638 - The macro OBJC:DEFMETHOD can be used to define ObjC methods.
639   It looks superficially like CL:DEFMETHOD in some respects.
640   The syntax is:
642   (OBC:DEFMETHOD name-and-result-type ((receiver-arg-and-class) &rest other-args) &body body)
644   where:
646   "name-and-result-type" is either an ObjC message name (use #/ !)
647   for methods that return a value of type :ID, or a list of an ObjC
648   message name and a foreign type specifier for methods with a different
649   foreign result type
651   "receiver-type-and-class" is a two-element list whose CAR is
652   a variable name and whose CADR is the lisp name of an ObjC class
653   or metaclass.  The receiver variable name can be any bindable
654   lisp variable name, but SELF (in some package) might be a reasonable
655   choice.  The receiver variable is declared to be "unsettable", i.e.,
656   it is an error to try to change the value of the receiver in the
657   body of the method definition.
659   "other-args" are either variable names (denoting parameters of type
660   :ID) or 2-element lists whose first element is a variable name and
661    whose second element is a foreign type specifier.
663  For example:
665(objc:defmethod (#/characterAtIndex: :unichar)
666    ((self hemlock-buffer-string) (index :<NSUI>nteger))
667  ...)
669  The method "characterAtIndex:", when invoked on an object of class
670  HEMLOCK-BUFFER-STRING with an additional argument of type :<NSU>integer
671  returns a value of type :unichar.)
673  Arguments that wind up as some non-:ID pointer type (pointers,
674  records passed by value) are represented as typed foreign pointers
675  (so the higher-level, type-checking accessors can be used on
676  arguments of type :ns-rect, :ns-pointe, etc.)
678  Within the body of methods defined via OBJC:DEFMETHOD, the local
679  function CL:CALL-NEXT-METHOD is defined.  It isn't quite as
680  general as CL:CALL-NEXT-METHOD is when used in a CLOS method,
681  but it has some of the same semantics.  It accepts as many arguments
682  as are present in the containing method's "other args" list and
683  invokes version of the containing method that would have been
684  invoked on instances of the receiver's class's superclass with
685  the receiver and other provided arguments.  (The idiom of passing
686  the current method's arguments to the next method is common enough
687  that the CALL-NEXT-METHOD in OBJC:DEFMETHODs should probably do
688  this if it receives no arguments.)
690  A method defined via OBJC:DEFMETHOD that returns a structure "by value"
691  can do so by returning a record created via MAKE-GCABLE-RECORD, by
692  returning the value returned via CALL-NEXT-METHOD, or by other similar
693  means.  Behind the scenes, there may be a pre-allocated instance of
694  the record type (used to support native structure-return conventions),
695  and any value returned by the method body will be copied to this
696  internal record instance.  Within the body of a method defined with
697  OBJC:DEFMETHOD that's declared to return a structure type, the local
698  macro OBJC:RETURNING-FOREIGN-STRUCT can be used to access the internal
699  structure:
701  (objc:defmethod (#/reallyTinyRectangleAtPoint: :ns-rect)
702    ((self really-tiny-view) (where :ns-point))
703    (objc:returning-foreign-struct (r)
704      (ns:init-ns-rect r (ns:ns-point-x where) (ns:ns-point-y where)
705                          single-float-epsilon single-float-epsilon)
706      r))
708 - If OBJC:DEFMETHOD introduces a new ObjC message, a ... message
709   to that effect.  Sometimes, these messages are merely informative
710   (and barely informative, at that ...), but they may also indicate
711   that a message name is misspelled (or possibly missing a trailing
712   colon.)  If a method is redefined in such a way that it's type
713   signature changes, a continuable error is signaled.
715 - there used to be some fairly obscure reasons that led to
716   MAKE-OBJC-INSTANCE being a bit more efficient than MAKE-INSTANCE
717   in some cases (some of the methods invoked by MAKE-INSTANCE did
718   some extra work to handle Lisp slots even if the class didn't
719   define any Lisp slots.  This work isn't done anymore, and consequently
720   there's less reason to prefer MAKE-OBJC-INSTANCE.  (MAKE-OBJC-INSTANCE
721   is still defined and exported from the OBJC:PACKAGE).
723 - the preferred means of loading an add-on framework and processing
724   the declarations in its interfaces has changed several times over
725   the last several months.  The currently preferred (new) way to
726   do that is via the new function OBJC:LOAD-FRAMEWORK
728   (OBJC:LOAD-FRAMEWORK framework-name interface-dir)
730   where "framework-name" is a string which names the framework and
731   "interface-dir" is a keyword that names the associated set of
732   interfaces.  OBJC:LOAD-FRAMEWORK should find and initialize the
733   framework bundle (looking in standard framework search paths),
734   introduce new ObjC classes to CLOS, update information about
735   declared messages and their methods' type signatures, adjust
736   affected dispatch functions, and make the interfaces other
737   definitions available.  The order in which it does these
738   things isn't specified, and may change in the future.
740 - Most Cocoa-related examples (the demo IDE, the Rubix and Webkit
741   examples) have been rewritten to use the new bridge features.
742   (I may have missed some contributed examples; if people want
743   to convert these, that'd be great.)  It's too early to say
744   whether the new approach is better or worse than the old, but
745   I've (so far) found some of the code easier to read and maintain.
746   We might find that some things that (for instance) SEND does
747   more efficiently could and should be done via SEND (I'm thinking
748   mostly of struct-return stuff), but so far I haven't seen the
749   new stuff keel over.
751   The converted code looks like "lisp code with strange-looking
752   function names" at first glance, and that seems about right.
753   The function names might get to look more familiar as the
754   reader becomes more familiar with Cocoa; as someone here pointed
755   out, it's arguably good that the function names are distinctive
756   in that that helps to remind the reader that these are likely
757   lower-level functions that are less tolerant of type- and other
758   errors than the typical lisp function would be.
762OpenMCL 1.1-pre-070214
763- The FASL version changed (old FASL files won't work with this
764  lisp version), as did the version information which tries to
765  keep the kernel in sync with heap images.
766- There are new interface files for all platforms.  These files
767  encode some foreign type information a little differently
768  than older ones did (notably information about foreign functions
769  that return structures or accept structure args by value.)  The
770  new .cdb files can't be used by older versions of OpenMCL; using
771  older .cdb files with this version is "allowed, but not supported
772  or recommended."
773- Almost all of the changes in functionality since the last (061231)
774  snapshots and since the CVS freeze on 070117 have to do with
775  relatively obscure issues having to do with passing structures
776  to foreign functions by value and/or returning structures from foreign
777  function calls.
779  These idioms are fairly rare in traditional C code (though it's
780  fairly common to pass -pointers- to structures by reference
781  and sometimes to return pointers to structures.  There are
782  a few C compiler runtime routines that perform some flavor
783  of integer division and return a two-element structure that
784  contains "quotient" and "remainder" fields, but that's typically
785  about the extent of the use of this idiom.)  The idioms are used
786  much more often in Apple's Carbon and Cooca libraries and in
787  some of the frameworks (CoreGraphics, CoreFoundation) that those
788  libraries are based on.
790  OpenMCL's FFI has provided some support for this in the past;
791  notably, it's provided support for (most of the) structure-returning
792  and struct-by-value conventions used on 32-bit PPC Darwin.  In these
793  conventions, a foreign function that returned a structure received
794  a pointer to an instance of that structure type as a first argument,
795  and a function that received a structure argument by value received
796  the structure's contents in 32-bit word-size integer chunks (regardless
797  of the types or sizes of the structure's fields.)  Knowledge of these
798  conventions was hardwired into various parts of the system (e.g.,
799  the interface database), so that it was not generally possible to
800  tell whether a given foreign function returned a structure type
801  (or just happened to take an extra pointer argument.)
803  Unfortunately, there are at least 4 other sets of conventions for
804  dealing with structure arguments/return values on the platforms
805  that OpenMCL runs on (and even the DarwinPPC32 conventions weren't
806  fully/correctly implemented.)  OpenMCL's FFI is generally pretty
807  low-level, but to the extent that it's reasonable to talk about
808  "higher level" constructs (EXTERNAL-CALL, SEND, FF-CALL, #_), those
809  higher-level constructs try to enforce uniform syntax and try
810  to hide the platform-specific details in backend-specific functions.
812  The impact of these changes should generally be pretty minimal.
813  In a "higher-level" construct used to call a foreign function that
814  returns a structure type, the first parameter in the call should
815  be a pointer to an instance of that structure type.
817  For example, if a :rect structure is defined as:
819  (def-foreign-type nil
820    (:struct :rect
821      (:width :int)
822      (:height :int)
823      (:x :int)  ; x coordinate of origin
824      (:y :int)))
826  and a foreign function named "inset_rect" takes a rect and an integer
827  delta and returns a new :rect "inset" by that delta, a call to that
828  foreign function might look like:
830  (rlet ((result :rect))
831    (ff-call *address-of-inset-rect* result (:struct :rect) r :int delta :(:struct rect))
832    ;; or, if "inset_rect" was declared in the interface database:
833    (#_inset_rect result r delta))
836  A callback that returns a :rect likewise should accept a pointer
837  to an instance of the :rect type as a first (unqualified) argument
838  and explicitly declare that it returns a (:STRUCT :RECT).
840  (defcallback *address-of-inset-rect (result (:struct :rect) r :int delta (:struct :rect))
841    (setf (pref result :rect.x) (+ (pref r :rect.x) delta)
842          (pref result :rect.y) (+ (pref r :rect.y) delta)
843          (pref result :rect.width) (- (pref r :rect.width) (* 2 delta))
844          (pref result :rect.height) (- (pref r :rect.height) (* 2 delta))))
846  Note that this is very similar to what's been (implicitly) supported
847  on DarwinPPC32; the basic difference is that the return type
848  ("(:STRUCT :RECT)") is explicitly specified (or, in the case of #_,
849  specified in the interface database).  Whether the "result" pointer
850  is actually passed as an argument or not is platform-dependent (on
851  DarwinPPC64, the :rect structure would be "returned" by returning
852  4 :int values in 4 different machine registers), but the same syntax
853  can be used (and hides those details) on all platforms.
855  In the examples above, we said that the (presumed source) rectangle
856  was passed by value as a value of type (:struct :rect), and we let
857  the FFI deal with the details.  Historically, this parameter could
858  have been specified as a small unsigned integer N (denoting the
859  DarwinPPC32 convention of passing the structure value a N
860  native-word-size integer arguments.)  Again, there are several
861  different conventions for passing and receiving structure values,
862  and it's best to let the FFI decide how to follow those conventions.
863  (Some of those conventions are quite complicated, and depend on
864  the size of the structure as well as the types of its fields.)
866  In all cases, a callback which declares a parameter to be of a
867  structure type can treat that parameter as a pointer an instance of
868  that structure type with fields initialized by the caller (as in
869  the case of "r" in the example above.)
871  In the ObjC bridge, the DEFINE-OBJC-METHOD macro has always provided
872  syntax for specifiying that the method "returns" a structure. (That
873  syntax is (:struct <struct-type> <parameter-name>). That continues
874  to be supported.
876  Apple's ObjC runtime provides different functions (#_objc_msgSend and
877  #_objc_msgSend_stret) to handle the cases of sending messages which
878  return non-structure and structure results.  These low-level functions
879  are very sensitive to whether the structure is actually returned via
880  an "invisible" first argument or not (this is only one of a few different
881  conventions on some platforms.)  OpenMCL's ObjC bridge makes similar
882  distinctions, but uses simple, consistent rules: a message that returns
883  a structure should always be sent via SEND/STRET (or some variant of
884  SEND/STRET) and should have a first parameter of type "pointer to
885  returned structure type", regardless of whether or not that pointer
886  is actually passed to the method implementation or just used as by
887  some platform-specific code to transfer register values.)
889  The end result of all of this (several weeks of bootstrapping) is
890  that most things are pretty much the same, at least on DarwinPPC32;
891  only foreign function calls/callbacks that involve passing structures
892  by value or returning structures need change at all, and the changes
893  generally involve being more explicit/declarative about what's going
894  on.  These changes -do- allow these idioms to be used on other
895  (64-bit) platforms, and since they're heavily used in Apple GUI
896  libraries and since 64-bit versions of Carbon and Cocoa are announced
897  features of Leopard, it seemed appropriate to get support for this
898  stuff into the FFI on those platforms and to try to do it in a way
899  that hid the platform-dependent details.  (I didn't expect all of
900  this to take so long.)
902- The initial listener PROCESS now persists across SAVE-APPLICATION.
903  This means that (for instance):
905  ? (defvar *listener-process* (current-process))
907  ? (save-application "new.image")
908  shell> openmcl new.image
909  ? (eq (current-process) *listener-process*)
910  T
911  ;; though of course the underlying OS thread, stacks, etc are unlikely
912  ;; to be "equal" in any sense.
914  The current process is sometimes used to mark "ownership" of thread-private
915  hash-tables and streams.  (Even though it doesn't make much sense for
916  STREAMs to persist across SAVE-APPLICATION, it does make sense for
917  HASH-TABLEs to do so; HASH-TABLES created with the :PRIVATE T option
918  and "owned" by the initial listener process continue to be owned by
919  that the current listener process in the new image.)
921- All of the FFI changes above do seem to allow the Cocoa IDE example
922  to run on ppc64/x86-64 (as well as ppc32) under Leopard, and
923  hopefully that'll soon be true of applications generated via Mikel
924  Evins' Bosco system as well.  The bridge and demo code have been
925  conditionalized to support ObjC 2.0 on 64-bit systems, to avoid
926  deprecated functions and methods, and to support 64-bit Cocoa
927  changes.  Hopefully, this has been done in a way that doesn't break
928  PPC32 Cocoa under Tiger (he said, quickly rushing to the nearest
929  PPC32 Tiger machine and breathing a sigh of relief when the Cocoa
930  listener appeared ..)  64-bit Cocoa sometimes used 64-bit signed and
931  unsigned integers in place of 32-bit integers; accordingly, the
932  foreign types :<NSI>nteger and :<NSUI>nteger are defined (as 32-bit
933  signed/unsigned integers) on 32-bit platforms, and these types are
934  used in some method and type definitions.  (Those integer types are
935  predefined in Objc 2.0, and are 64 bits wide on 64-bit platforms.)
937  More pervasively (and a little more problematically), CoreGraphics
938  (and things built on top of it, including Cocoa) uses double-floats
939  instead of single-floats for many things on 64-bit hardware; the
940  difference is abstracted (a little) via the new CGFloat type.
941  This means that (for instance) code which initializes a constant-sized
942  NSRect on a 32-bit machines and has traditionally done so via
943  something like:
945  (ns-make-rect 0.0 0.0 500.0 200.0)
947  now needs to do something like:
949  (ns-make-rect (float 0.0 ccl::+cgfloat-zero+) ..)
951  in order to compile and run on both 32-bit and 64-bit platforms.
953  where ccl::+cgfloat-zero+ is defined as 1.0f0 on 32-bit platforms
954  and as 1.0d0 on 64-bit machines.  Cases involving constants won't
955  incur any runtime overhead and the occasional runtime overhead in
956  other cases -probably- isn't that great in context (compared to
957  initializing a view hierarchy ...)  but it's certainly ugly to
958  look at.  It's possible that some of this ugliness could be
959  hidden in the bridge/FFI (by making them do the necessary coercions
960  for you), but there are tradeoffs there.
962- The ObjC bridge has had a long-standing bug whereby a standalone
963  Cocoa application may have needed to find the interface databases
964  at runtime in order for MAKE-OBJC-INSTANCE and MAKE-INSTANCE of
965  an ObjC class to work.  (These functions needed to be able to
966  send an "init" message to the newly-allocated instance, and needed
967  to know the type signature of that init message in order to do that.)
968  The current scheme tries to avoid this by pre-compiling helper
969  functions to enable calling all known "init" message signatures.
970  (More accurately, all fixed-argument "init" message signatures.)
971  This scheme avoids the need to send messages whose argument
972  and result types are computed at runtime (via %SEND), and %SEND
973  (a) was known to be inefficient and (b) would have a lot of
974  difficulty handling all known structure return/passing conventions
975  on supported platforms.  Accordingly, %SEND has been deprecated
976  (with extreme prejudice, e.g., removed.)
978- a couple of little functions are defined (but their names are
979  not yet exported) on x86-64: ccl::rdtsc and ccl::rdtsc64 provide
980  access to the values returned by on-chip cycle counting instructions.
981  For instance:
983? (let* ((start (ccl::rdtsc)))
984    (sleep 1)
985    (- (ccl::rdtsc) start))
988  Hmm.  Apparently, the 2.0GHz MacBook I tried that on is actually
989  a 1.995GHz MacBook.
991  There are all kinds of ways for rdtsc to lose (and return
992  inaccurate or misleading results): the cycle counters for
993  each CPU core in a multi-core system aren't necessarily
994  kept in sync, and many modern systems allow CPU clock rates
995  to vary (for power-management reasons) and/or allow the CPU
996  to sleep/hibernate.  OSes seem to offer some support for
997  compensating for these effects, and it seems like ccl::rdtsc
998  and ccl::rdtsc64 can be used to obtain interesting results.
1000  The RDTSC instruction actually returns an unsigned 64-bit
1001  result; apparently, some Intel documentation claims that this
1002  value will not "wrap around" to 0 at contemporary clock rates
1003  for at least 10 years after the system was booted.  (If you can
1004  keep an Intel system running for 9 years between reboots, you
1005  might consider telling Intel that the RDTSC counter wrapped around
1006  a year early; they might give you a refund.  Or maybe not.)
1007  A non-negative OpenMCL64 fixnum is limited to 60 bits; the
1008  ccl::rdtsc function truncates the 64-bit counter value so
1009  that it fits in a non-negative fixnum; if the 10 year limit
1010  for the 64-bit value is accurate, the 60-bit value would
1011  wrap around after about 223 days of uptime.
1013  ccl::rdtsc64 returns the full 64-bit counter value, but
1014  may return a bignum after 223 days of uptime.
1016- lots of bug fixes (not all of which involved the FFI or ObjC
1017  bridge.) 
1021openmcl 1.1-pre-061231
1022- The FASL version changed (old FASL files won't work with this
1023  lisp version), as did the version information which tries to
1024  keep the kernel in sync with heap images.
1025  The binary incompatibility has to do with how a few pages of
1026  low memory in the lisp kernel's address space are mapped and
1027  used.  OpenMCL was generally assuming that these pages were
1028  otherwise unused and could be used to map a small static data
1029  area from the heap image file; on some platforms, the dynamic
1030  linker may have already allocated data in those unused pages
1031  before the lisp kernel even starts to run.  Fixing this involved
1032  changing the address of that small static data area slightly,
1033  and this caused the addresses of some objects contained within
1034  that static data area - notably NIL - to change, as well.
1035- This snapshot is otherwise just a set of bug fixes/work-in-progress
1036  changes.
1037- Even though no supported filesystem actually supports versioned files,
1038  OpenMCL now tries to retain PATHNAME-VERSION informaton for physical
1039  pathnames.  (The fact that it didn't caused several ANSI test failures.)
1040  This change introduced/exposed a few other bugs; I think that I've
1041  caught at least the most obvious ones, but it's possible that some
1042  new pathname-related bugs have been introduced.
1043- The cron job that runs on and updates the ChangeLog from
1044  CVS commit info stopped running as of a system upgrade in late November.
1045  The problem was fixed a couple of weeks ago, so it's once again meaningful
1046  to refer to the ChangeLog for details of bug fixes.
1047- FSQRT and FSQRTS instructions are "optional" on the PPC.  In practice,
1048  that often meant that they are implemented on chips made by IBM and
1049  not on chips made by Motorola/FreeScale.  This version of OpenMCL
1050  assumes that they're implemented and emulates them if they aren't.
1051- OSX 10.2 (Jaguar) and earlier versions are officially no longer
1052  supported.  (I honestly don't know if things have actually worked
1053  on Jaguar in a while, but some recent changes are known not to
1054  work on Jaguar and the kernel now inists on at least Panther on
1055  startup.
1056OpenMCL 1.1-pre-061205
1057- This release is intended to package up the bug fixes since
1058  the 061110 tarballs.  There aren't too many changes in
1059  functionality or any deep architectural changes since 061110,
1060  and it should be easy to bootstrap from current sources with
1061  061110 images.
1062  (It'd still be a good idea to recompile your code with
1063  up-to-date images, whether you download those images or
1064  build them yourself from CVS.)
1065- The one (barely) notable change in functionality has to do
1066  with how the lisp sets up pathname translations for the
1067  "ccl" logical host when the "CCL_DEFAULT_DIRECTORY" environment
1068  variable isn't set (e.g., when a shell script isn't used to
1069  invoke the lisp.)  Previous versions just used the current
1070  directory; this version tries to use the directory containing
1071  the current heap image.  The new scheme might get fooled by
1072  symbolic links (either following them or not following them
1073  could be wrong), but it's more likely to work for people
1074  who don't read or understand the discussion of the shell script
1075  in the documentation.
1076- All (knock wood) bugs that have been reported since the 061110
1077  images were released should be fixed.  Well, almost all.  The
1078  fixes include:
1080  - a typo (wrong register) in the "generic" version of the
1081    code which implements (SETF AREF) on 2-dimensional arrays
1082    on x86-64
1083  - incorrect bounds checking on vector references on x86-64,
1084    which caused some invalid indices to be treated as valid
1085    (usually leading to a segfault).  IIRC, the invalid indices
1086    that were erroneously accepted were fixnums whose absolute
1087    value was > (expt 2 56).  (More or less.).
1088  - Missing stream methods (especially involving string streams)
1089    affecting all platforms.
1090  - Several bugs involving GCD, some of which were specific to
1091    64-bit platforms and some of which affected all platforms.
1092    (These bugs sometimes affected results returned by #'/,
1093    LCM, and other funtions.)
1095  - OpenMCL has only ever supported an ELEMENT-TYPE argument of
1096   ([signed,unsigned]-byte 8|16|32|64) on binary file streams (with
1097   64-bit types supported only on 64-bit platforms.)  It has not
1098   previously tried to upgrade a supplied element-type to a supported
1099   one (it does now) and any errors that resulted from supplying an
1100   element-type that was not supported (and could not be upgraded) were
1101   either obscure side-effects or quiet misbehavior; an error (a
1102   SIMPLE-ERROR complaining about the unsupported element type) is now
1103   signaled as soon as attempts to upgrade to a supported element type
1104   fail.  I believe that the current behavior is both compliant and
1105   reasonable; it's probably better to discuss that issue on
1106   openmcl-devel than to do so here.
1109OpenMCL 1.1-pre-061110
1110- The FASL version changed (old FASL files won't work with this
1111  lisp version), as did the version information which tries to
1112  keep the kernel in sync with heap images.
1113- Several bug fixes (see ChangeLog), and modest-to-moderate
1114  performance improvements.  Notably, AREF and (SETF AREF)
1115  of 2- and 3-dimensional arrays are open-coded in more cases
1116  and are usually at least 5x faster than in previous versions.
1117  If the compiler knows that the array in question is a
1118  SIMPLE-ARRAY of appropiate dimensionality and knows the
1119  array's element-type, the speedup can be much greater.
1120  There are certainly opportunities for further improvements
1121  here, both in breadth (handling more cases) and depth
1122  (eliminating some type-and-bounds checking in safe code,
1123  doing parts of index calculations at compile-time when
1124  bounds and indices are constants ...), but things are
1125  generally improved.
1126- QUIT and SAVE-APPLICATION work a little differently; in
1127  particular, SAVE-APPLICATION sometimes runs after #_exit
1128  is called (via the #_atexit mechanism).
1129  The motivation for this change has to do with how some
1130  environments (Cocoa, to name one) conflate the ideas of
1131  "shutting down the GUI" with "exiting the application".
1132  Previous versions of OpenMCL tried to work around this
1133  by overriding some internal Cocoa methods; that approach
1134  was never particularly attractive and (predictably) it'll
1135  break in future OSX releases.
1136  The new scheme (which involves letting code run after #_exit
1137  has been called) certainly offers other ways to lose; so
1138  far, I haven't seen evidence of such lossage.
1139- For historical reasons (forgotten historical reasons, in fact)
1140  the PPC versions of OpenMCL run with floating-point underflow
1141  exceptions disabled (the x86-64 versions enable these exceptions
1142  by default.)  This should change soon (as soon as I remember
1143  to change it ...); it's unlikely that this will affect much
1144  user code, but it's possible that it'll do so.
1145OpenMCL 1.1-pre-061024
1146- The FASL version changed (old FASL files won't work with this
1147  lisp version), as did the version information which tries to
1148  keep the kernel in sync with heap images.
1149- Linux users: it's possible (depending on the distribution that
1150  you use) that the lisp kernel will claim to depend on newer
1151  versions of some shared libraries than the versions that you
1152  have installed.  This is mostly just an artifact of the GNU
1153  linker, which adds version information to dependent library
1154  references even though no strong dependency exists.  If you
1155  run into this, you should be able to simply cd to the appropriate
1156  build directory under ccl/lisp-kernel and do a "make".
1157- There's now a port of OpenMCL to FreeBSD/amd64; it claims to be
1158  of beta quality.  (The problems that made it too unstable
1159  to release as of a few months ago have been fixed;  I stil run
1160  into occasional FreeBSD-specific issues, and some such issues
1161  may remain.)
1162- The Darwin X8664 port is a bit more stable (no longer generates
1163  obscure "Trace/BKPT trap" exits or spurious-looking FP exceptions.)
1164  I'd never want to pass up a chance to speak ill of Mach, but both
1165  of these bugs seemed to be OpenMCL problems rather than Mach kernel
1166  problems, as I'd previously more-or-less assumed.
1167- I generally don't use SLIME with OpenMCL, but limited testing
1168  with the 2006-04-20 verson of SLIME seems to indicate that no
1169  changes to SLIME are necessary to work with this version.
1170- CHAR-CODE-LIMIT is now #x110000, which means that all Unicode
1171  characters can be directly represented.  There is one CHARACTER
1172  type (all CHARACTERs are BASE-CHARs) and one string type (all
1173  STRINGs are BASE-STRINGs.)  This change (and some other changes
1174  in the compiler and runtime) made the heap images a few MB larger
1175  than in previous versions.
1176- As of Unicode 5.0, only about 100,000 of 1114112./#x110000 CHAR-CODEs
1177  are actually defined; the function CODE-CHAR knows that certain
1178  ranges of code values (notably #xd800-#xddff) will never be valid
1179  character codes and will return NIL for arguments in that range,
1180  but may return a non-NIL value (an undefined/non-standard CHARACTER
1181  object) for other unassigned code values.
1182- The :EXTERNAL-FORMAT argument to OPEN/LOAD/COMPILE-FILE has been
1183  extended to allow the stream's character encoding scheme (as well
1184  as line-termination conventions) to be specified; see more
1185  details below.  MAKE-SOCKET has been extended to allow an
1186  :EXTERNAL-FORMAT argument with similar semantics.
1187- Strings of the form "u+xxxx" - where "x" is a sequence of one
1188  or more hex digits- can be used as as character names to denote
1189  the character whose code is the value of the string of hex digits.
1190  (The +  character is actually optional, so  #\u+0020, #\U0020, and
1191  #\U+20 all refer to the #\Space character.)  Characters with codes
1192  in the range #xa0-#x7ff (IIRC) also have symbolic names (the
1193  names from the Unicode standard with spaces replaced with underscores),
1194  so #\Greek_Capital_Letter_Epsilon can be used to refer to the character
1195  whose CHAR-CODE is #x395.
1196- The line-termination convention popularized with the CP/M operating
1197  system (and used in its descendants) - e.g., CRLF - is now supported,
1198  as is the use of Unicode #\Line_Separator (#\u+2028).
1199- About 15-20 character encoding schemes are defined (so far); these
1200  include UTF-8/16/32 and the big-endian/little-endian variants of
1201  the latter two and ISO-8859-* 8-bit encodings.  (There is not
1202  yet any support for traditional (non-Unicode) ways of externally
1203  encoding characters used in Asian languages, support for legacy
1204  MacOS encodings, legacy Windows/DOS/IBM encodings, ...)  It's hoped
1205  that the existing infrastructure will handle most (if not all) of
1206  what's missing; that may not be the case for "stateful" encodings
1207  (where the way that a given character is encoded/decoded depend
1208  on context, like the value of the preceding/following character.)
1209- There isn't yet any support for Unicode-aware collation (CHAR>
1210  and related CL functions just compare character codes, which
1211  can give meaningless results for non-STANDARD-CHARs), case-inversion,
1212  or normalization/denormalization.  There's generally good support
1213  for this sort of thing in OS-provided libraries (e.g., CoreFoundation
1214  on MacOSX), and it's not yet clear whether it'd be best to duplicate
1215  that in lisp or leverage library support.
1216- Unicode-aware FFI functions and macros are still in a sort of
1217  embryonic state if they're there at all; things like WITH-CSTRs
1218  continue to exist (and continue to assume an 8-bit character
1219  encoding.)
1220- Characters that can't be represented in a fixed-width 8-bit
1221  character encoding are replaced with #\Sub (= (code-char 26) =
1222  ^Z) on output, so if you do something like:
1224? (format t "~a" #\u+20a0)
1226  you might see a #\Sub character (however that's displayed on
1227  the terminal device/Emacs buffer) or a Euro currency sign or
1228  practically anything else (depending on how lisp is configured
1229  to encode output to *TERMINAL-IO* and on how the terminal/Emacs
1230  is configured to decode its input.
1232  On output to streams with character encodings that can encode
1233  the full range of Unicode - and on input from any stream -
1234  "unencodable characters" are represented using the Unicode
1235  #\Replacement_Character (= #\U+fffd); the presence of such a
1236  character usually indicates that something got lost in translation
1237  (data wasn't encoded properly or there was a bug in the decoding
1238  process.)
1239- Streams encoded in schemes which use more than one octet per code unit
1240  (UTF-16, UTF-32, ...) and whose endianness is not explicit will be
1241  written with a leading byte-order-mark character on (new) output and
1242  will expect a BOM on input; if a BOM is missing from input data,
1243  that data will be assumed to have been serialized in big-endian order.
1244  Streams encoded in variants of these schemes whose endianness is
1245  explicit (UTF-16BE, UCS-4LE, ...) will not have byte-order-marks written
1246  on output or expected on input.  (UTF-8 streams might also contain
1247  encoded byte-order-marks; even though UTF-8 uses a single octet per
1248  code unit - and possibly more than one code unit per character - this
1249  convention is sometimes used to advertise that the stream is UTF-8-
1250  encoded.  The current implementation doesn't skip over/ignore leading
1251  BOMs on UTF8-encoded input, but it probably should.)
1253  If the preceding paragraph made little sense, a shorter version is
1254  that sometimes the endianness of encoded data matters and there
1255  are conventions for expressing the endianness of encoded data; I
1256  think that OpenMCL gets it mostly right, but (even if that's true)
1257  the real world may be messier.
1258- By default, OpenMCL uses ISO-8859-1 encoding for *TERMINAL-IO*
1259  and for all streams whose EXTERNAL-FORMAT isn't explicitly specified.
1260  (ISO-8859-1 just covers the first 256 Unicode code points, where
1261  the first 128 code points are equivalent to US-ASCII.)  That should
1262  be pretty much equivalent to what previous versions (that only
1263  supported 8-bit characters) did, but it may not be optimal for
1264  users working in a particular locale.  The default for *TERMINAL-IO*
1265  can be set via a command-line argument (see below) and this setting
1266  persists across calls to SAVE-APPLICATION, but it's not clear that
1267  there's a good way of setting it automatically (e.g., by checking
1268  the POSIX "locale" settings on startup.)  Thing like POSIX locales
1269  aren't always set correctly (even if they're set correctly for
1270  the shell/terminal, they may not be set correctly when running
1271  under Emacs ...) and in general, *TERMINAL-IO*'s notion of the
1272  character encoding it's using and the "terminal device"/Emacs subprocess's
1273  notion need to agree (and fonts need to contain glyphs for the
1274  right set of characters) in order for everything to "work".  Using
1275  ISO-8859-1 as the default seemed to increase the likelyhood that
1276  most things would work even if things aren't quite set up ideally
1277  (since no character translation occurs for 8-bit characters in
1278  ISO-8859-1.)
1279- In non-Unicode-related news: the rewrite of OpenMCL's stream code
1280  that was started a few months ago should now be complete (no more
1281  "missing method for BASIC-STREAM" errors, or at least there shouldn't
1282  be any.)
1283- I haven't done anything with the Cocoa bridge/demos lately, besides
1284  a little bit of smoke-testing.
1286Some implementation/usage details:
1288Character encodings.
1290CHARACTER-ENCODINGs are objects (structures) that're named by keywords
1291(:ISO-8859-1, :UTF-8, etc.).  The structures contain attributes of
1292the encoding and functions used to encode/decode external data, but
1293unless you're trying to define or debug an encoding there's little
1294reason to know much about the CHARACTER-ENCODING objects and it's
1295generally desirable (and sometimes necessary) to refer to the encoding
1296via its name.
1298Most encodings have "aliases"; the encoding named :ISO-8859-1 can
1299also be referred to by the names :LATIN1 and :IBM819, among others.
1300Where possible, the keywordized name of an encoding is equivalent
1301to the preferred MIME charset name (and the aliases are all registered
1302IANA charset names.)
1304NIL is an alias for the :ISO-8859-1 encoding; it's treated a little
1305specially by the I/O system.
1307The function CCL:DESCRIBE-CHARACTER-ENCODINGS will write descriptions
1308of all defined character encodings to *terminal-io*; these descriptions
1309include the names of the encoding's aliases and a doc string which
1310briefly describes each encoding's properties and intended use.
1312Line-termination conventions.
1314As noted in the <=1.0 documentation, the keywords :UNIX, :MACOS, and
1315:INFERRED can be used to denote a stream's line-termination conventions.
1316(:INFERRED is only useful for FILE-STREAMs that're open for :INPUT or
1317:IO.)  In this release, the keyword :CR can also be used to indicate
1318that a stream uses #\Return characters for line-termination (equivalent
1319to :MACOS), the keyword :UNICODE denotes that the stream uses Unicode
1320#\Line_Separator characters to terminate lines, and the keywords :CRLF,
1321:CP/M, :MSDOS, :DOS, and :WINDOWS all indicate that lines are terminated
1322via a #\Return #\Linefeed sequence.
1324In some contexts (when specifying EXTERNAL-FORMATs), the keyword :DEFAULT
1325can also be used; in this case, it's equivalent to specifying the value
1326of the variable CCL:*DEFAULT-LINE-TERMINATION*.  The initial value of
1327this variable is :UNIX.
1329Note that the set of keywords used to denote CHARACTER-ENCODINGs and
1330the set of keywords used to denote line-termination conventions is
1331disjoint: a keyword denotes at most a character encoding or a line
1332termination convention, but never both.
1336EXTERNAL-FORMATs are also objects (structures) with two read-only
1337fields that can be accessed via the functions EXTERNAL-FORMAT-LINE-TERMINATION
1338and EXTERNAL-FORMAT-CHARACTER-ENCODING; the values of these fields are
1339line-termination-convention-names and character-encoding names as described
1342An EXTERNAL-FORMAT object via the function MAKE-EXTERNAL-FORMAT:
1344MAKE-EXTERNAL-FORMAT &key domain character-encoding line-termination
1346(Despite the function's name, it doesn't necessarily create a new,
1347unique EXTERNAL-FORMAT object: two calls to MAKE-EXTERNAL-FORMAT
1348with the same arguments made in the same dynamic environment will
1349return the same (eq) object.)
1351Both the :LINE-TERMINATION and :CHARACTER-ENCODING arguments default
1352to :DEFAULT; if :LINE-TERMINATION is specified as or defaults to
1353:DEFAULT, the value of CCL:*DEFAULT-LINE-TERMINATION* is used to
1354provide a concrete value.
1356When the :CHARACTER-ENCODING argument is specifed as/defaults to
1357:DEFAULT, the concrete character encoding name that's actually used
1358depends on the value of the :DOMAIN argument to MAKE-EXTERNAL-FORMAT.
1359The :DOMAIN-ARGUMENT's value can be practically anything; when it's
1360the keyword :FILE and the :CHARACTER-ENCODING argument's value is
1361:DEFAULT, the concrete character encoding name that's used will be
1362the value of the variable CCL:*DEFAULT-FILE-CHARACTER-ENCODING*; the
1363initial value of this variable is NIL (which is an alias for :ISO-8859-1).
1364If the value of the :DOMAIN argument is :SOCKET and the :CHARACTER-ENCODING
1365argument's value is :DEFAULT, the value of
1366CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING* is used as a concrete character
1367encoding name.  The initial value of CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING*
1368is NIL, again denoting the :ISO-8859-1 encoding.
1369If the value of the :DOMAIN argument is anything else, :ISO-8859-1 is
1370also used (but there's no way to override this.) 
1372The result of a call to MAKE-EXTERNAL-FORMAT can be used as the value
1373of the :EXTERNAL-FORMAT argument to OPEN, LOAD, COMPILE-FILE, and
1374MAKE-SOCKET; it's also possible to use a few shorthand constructs
1375in these contexts:
1377* if ARG is unspecified or specified as :DEFAULT, the value of the
1378  variable CCL:*DEFAULT-EXTERNAL-FORMAT* is used.  Since the value
1379  of this variable has historically been used to name a default
1380  line-termination convention, this case effectively falls into
1381  the next one:
1382* if ARG is a keyword which names a concrete line-termination convention,
1383  an EXTERNAL-FORMAT equivalent to the result of calling
1384  (MAKE-EXTERNAL-FORMAT :line-termination ARG)
1385   will be used
1386* if ARG is a keyword which names a character encoding, an EXTERNAL-FORMAT
1387  equvalent to the result of calling
1388  (MAKE-EXTERNAL-FORMAT :character-encoding ARG)
1389  will be used
1390* if ARG is a list, the result of (APPLY #'MAKE-EXTERNAL-FORMAT ARG)
1391  will be used
1393(When MAKE-EXTERNAL-FORMAT is called to create an EXTERNAL-FORMAT
1394object from one of these shorthand designators, the value of the
1395:DOMAIN keyword argument is :FILE for OPEN,LOAD, and COMPILE-FILE
1396and :SOCKET for MAKE-SOCKET.)
1399The CL function STREAM-EXTERNAL-FORMAT - which is portably defined
1400on FILE-STREAMs - can be applied to any open stream in this release
1401and will return an EXTERNAL-FORMAT object when applied to an open
1403SETF can be used with STREAM-EXTERNAL-FORMAT to change the stream's
1404character encoding, line-termination, or both.
1406If a "shorthand" external-format designator is used in a call to
1407(SETF STREAM-EXTERNAL-FORMAT), the "domain" used to construct an
1408EXTERNAL-FORMAT is derived from the class of the stream in the
1409obvious way (:FILE for FILE-STREAMs, :SOCKET for ... well, for
1410sockets ...)
1412Note that the effect or doing something like:
1414(let* ((s (open "foo" ... :external-format :utf-8)))
1415  ...
1416  (unread-char ch s)
1417  (eetf (stream-external-format s) :us-ascii)
1418  (read-char s))
1420might or might not be what was intended.  The current behavior is
1421that the call to READ-CHAR will return the previously unread character
1422CH, which might surprise any code which assumes that the READ-CHAR
1423will return something encodable in 7 or 8 bits.  Since functions
1424like READ may call UNREAD-CHAR "behind your back", it may or may
1425not be obvious that this has even occurred; the best approach to
1426dealing with this issue might be to avoid using READ or explicit
1427calls to UNREAD-CHAR when processing content encoded in multiple
1428external formats.
1430There's a similar issue with "bivalent" streams (sockets) which
1431can do both character and binary I/O with an :ELEMENT-TYPE of
1432(UNSIGNED-BYTE 8).  Historically, the sequence:
1434   (unread-char ch s)
1435   (read-byte s)
1437caused the READ-BYTE to return (CHAR-CODE CH); that made sense
1438when everything was implicitly encoded as :ISO-8859-1, but may not
1439make any sense anymore.  (The only thing that seems to make sense
1440in that case is to clear the unread character and read the next
1441octet; that's implemented in some cases but I don't think that
1442things are always handled consistently.)
1444Command-line argument for specifying the character encoding to
1445be used for *TERMINAL-IO*.
1447Shortly after a saved lisp image starts up, it creates the standard
1448CL streams (like *STANDARD-OUTPUT*, *TERMINAL-IO*, *QUERY-IO*, etc.);
1449most of these streams are usually SYNONYM-STREAMS which reference
1450the TWO-WAY-STREAM *TERMINAL-IO*, which is itself comprised of
1451a pair of CHARACTER-STREAMs.  The character encoding used for
1452any CHARACTER-STREAMs created during this process is the one
1453named by the value of the variable CCL:*TERMINAL-CHARACTER-ENCODING-NAME*;
1454this value is initially NIL.
1456The -K or --terminal-encoding command-line argument can be used to
1457set the value of this variable (the argument is processed before the
1458standard streams are created.)  The string which is the value of
1459the -K/--terminal-encoding argument is uppercased and interned in
1460the KEYWORD package; if an encoding named by that keyword exists,
1461CCL:*TERMINAL-CHARACTER-ENCODING-NAME* is set to the name of that
1462encoding.  For example:
1464shell> openmcl -K utf-8
1466will have the effect of making the standard CL streams use :UTF-8
1467as their character encoding.
1469(It's probably possible - but a bit awkward - to use (SETF EXTERNAL-FORMAT)
1470from one's init file or --eval arguments or similar to change existing
1471streams' character encodings; the hard/awkward parts of doing so include
1472the difficulty of determining which standard streams are "real" character
1473streams and which are aliases/composite streams.)
1475OpenMCL 1.1-pre-069826
1476- There's an (alpha-quality, maybe) port to x86-64 Darwin (e.g., the
1477  Mac Pro.)  Some known problems include:
1479  * infrequently (but not infrequently enough) the lisp dies on
1480    startup with a spurious "Trace/BKPT trap" error message.  This
1481    seems to be timing-dependent and (very generally) seems to
1482    involve the Mach exception thread not recognizing an exception
1483    used to effect exception return.  Sometimes, this shows up
1484    as a (:SIGNALED 5) error when REBUILD-CCL runs the lisp to
1485    create a new image.
1487  * some math library primitives (#_asin, for one) generate
1488    spurious incidental FP exceptions that have nothing to
1489    do with the validity of the arguments or result.  To work around
1490    this, the lisp ignores FP exceptions which might have occurred
1491    during a call into the math library; that means that it doesn't
1492    detect -real- FP exceptions when they're signaled.  (This bug
1493    only affects things that call into the system math library;
1494    lisp arithmetic operations that're done inline are not affected.)
1496  * The version of OSX/Darwin that shipped with the Mac Pro is missing
1497    some functionality that from OpenMCL's point of view is highly
1498    desirable (namely, the ability to keep application-level thread-
1499    specific data in a per-thread block of memory addressed by an
1500    otherwise unused segment register.)  To get things working (as
1501    well as they are), the lisp "shares" the segment register that
1502    the pthreads library uses to access thread data.  This scheme
1503    isn't intended to be long-lived (and negatively affects
1504    performance of things like foreign-function calls, callbacks,
1505    and exception handling).
1507  * The .cdb files (libc only for Tiger) in ccl:darwin-x86-headers64;
1508    were cross-developed on a Linux x86-64 system, since Apple
1509    has not yet released the sources to their x86-64 enabled gcc.
1511- On all platforms, stream code has been rewritten and often offers
1512  better (sometimes substantially better) performance.  OPEN and
1513  MAKE-SOCKET have each been extended to take additional keyword
1514  arguments.
1516  :SHARING, which can have the values :PRIVATE (the default), :LOCK,
1517  or :EXTERNAL (NIL is also accepted as synonym for :EXTERNAL)
1519   :PRIVATE specifies that the stream can only be accessed by
1520   the thread that created it.  (There was some discussion on openmcl-devel
1521   about the idea of "transferring ownership" of a stream; this has
1522   not yet been implemented.)  Attempts to do I/O on a stream with
1523   :PRIVATE sharing from a thread other than the stream's owner yield
1524   an error.
1526   :LOCK specifies that all access to the stream require the calling
1527   thread to obtain a lock; there are separate "read" and "write"
1528   locks for IO streams (so it's possible for one thread to read
1529   from such a stream while another thread writes to it, for instance.)
1530   :LOCK was the implicit default for all streams prior to this change.
1531   (See below - under the discussion of the AUTO-FLUSH mechanism -
1532   for a discussion of one of the implications of this change that
1533   affects SLIME users.)
1535   :EXTERNAL (or NIL) specifies that I/O primitives enforce no
1536   access protocol.  This may be appropriate for some types of application
1537   which can control stream access via application-level protocols.  Note
1538   that since even the act of reading from a stream changes its internal
1539   state (and simultaneous access from multiple threads can therefore
1540   lead to corruption of that state), some care must be taken in the
1541   design of such protocols.
1543  The :BASIC keyword argument influences whether or not the stream
1544  will be an instance of the class FUNDAMENTAL-STREAM (the superclass
1545  from which all Gray stream classes inherit) or a subclass of the
1546  built-in class CCL::BASIC-STREAM.  The default value of :BASIC
1547  is T and this has effect for FILE-STREAMs created via OPEN;
1548  SOCKETs are still always implemented as FUNDAMENTAL (Gray) streams,
1549  though this should change soon.
1551   The tradeoff between FUNDAMENTAL and BASIC streams is entirely
1552   between flexibility and (potential or actual) performance.  I/O
1553   primitives can recognize BASIC-STREAMs and exploit knowledge of
1554   implementation details; FUNDAMENTAL stream classes can be
1555   subclassed in a semi-standard way (the Gray streams protocol.)
1557   For existing stream classes (FILE-STREAMs, SOCKETs, and the
1558   internal CCL::FD-STREAM classes used to implement file streams
1559   and sockets), a lot of code can be shared between the
1560   FUNDAMENTAL and BASIC implementations.  The biggest difference
1561   should be that that code can be reached from I/O primitives
1562   like READ-CHAR without going through some steps that're there
1563   to support generality and extensibility, and skipping those
1564   steps when that support isn't needed can improve I/O performance.
1566   Gray stream methods (STREAM-READ-CHAR) should work on
1567   appropriate BASIC-STREAMs.  (There may still be cases where
1568   such methods are undefined; such cases should be considered
1569   bugs.)  It is not guaranteed that Gray stream methods would
1570   ever be called by I/O primitives to read a character from
1571   a BASIC-STREAM (though there are still cases where this happens.)
1573   A simple loop reading 2M characters from a text file runs about
1574   10X faster when the file is opened the new defaults (:SHARING :PRIVATE
1575   :BASIC T) than it had before these changes were made.  That sounds
1576   good, until one realizes that the "equivalent" C loop can be about
1577   10X faster still ...
1579 - Forcing output to interactive streams.
1581   OpenMCL has long had a (mostly undocumented) mechanism whereby
1582   a mostly idle thread wakes up a few (~3) times per second and
1583   calls FORCE-OUTPUT on specified OUTPUT-STREAMS; this helps to
1584   ensure that streams with which a user would be expected to
1585   interact (the output side of *TERMINAL-IO*, listener windows
1586   in a GUI, etc.) have all buffered output flushed without
1587   requiring application or I/O library code to be concerned about
1588   that.
1590   The SLIME lisp interaction mode for Emacs uses this mechanism,
1591   but the changes described above interfere with SLIMEs use of
1592   it:  in order to be safely accessed from multiple threads (the
1593   SLIME REPL thread and the thread which does the background
1594   periodic flushing of buffered output), a stream must have
1595   been created with :SHARING :LOCK in effect.  This is no longer
1596   the effective default; the code which does the periodic
1597   output flushing ignores streams which do not use locks as an
1598   access/sharing mechanism.  THIS MEANS THAT BUFFERRED OUTPUT
1600   A small change to SLIME's "swank-openmcl.lisp" is required
1601   to restore this functionality.  First,  a brief description of
1602   a couple of new primitives:
1606    Adds "s", which should be a "simple" OUTPUT-STREAM as returned
1607    by OPEN or MAKE-SOCKET, to a list of streams whose buffered
1608    output should be periodically flushed.  If S was not created
1609    with :SHARING :LOCK in effect, the stream will have its
1610    :SHARING mode changed to put :SHARING :LOCK into effect.
1614    Removes S from the internal list of automatically flushed
1615    streams.  Does not restore the stream's :SHARING mode, which
1616    may have been changed by a previous call to ADD-AUTO-FLUSH-STREAM.
1618 - SLIME changes
1619   In slime:swank-openmcl.lisp, around line 182, the method
1621(defmethod make-stream-interactive ((stream ccl:fundamental-output-stream))
1622  (push stream ccl::*auto-flush-streams*))
1624   should be changed to use CCL:ADD-AUTOFLUSH-STREAM if it's defined:
1626(defmethod make-stream-interactive ((stream ccl:fundamental-output-stream))
1627  (if (fboundp 'ccl::add-auto-flush-stream)
1628    (ccl::add-auto-flush-stream stream)
1629    (push stream ccl::*auto-flush-streams*)))
1631   That's adequate for the moment, since sockets are still
1632   FUNDAMENTAL-STREAMs.  When that changes, some more extensive changes
1633   to swank-openmcl.lisp may become necessary.
1635- on x86-64, floating-point-underflow exceptions are now enabled
1636  by default.  (They really should be on ppc as well.)  Again,
1637  this affects FP operations that are done in lisp code and
1638  the results of FP operations that are reported in response
1639  to calls to reasonable (non-Darwin) math libraries.  This
1640  can affect whether or not some "potential number"  reader
1641  tokens are representable as numbers, e.g., whether or not
1642  attempts to read something like "1.0f-50" signal underflow
1643  or are quietly mapped to 0.0f0.
1645- examples: Phil (from the mailing list) has added code which
1646  supports some of the ffi examples from the documentation.
1648- Bug fixes: see ChangeLog
1652OpenMCL 1.1-pre-060705
1653- Bug fixes again.  Some internal changes to support a FreeBSD/AMD64
1654  port that's not quite ready.
1656- :MCL is back on *features*; there seem to be too many packages out
1657  there that expect it to be, and there hasn't been enough advance
1658  notice of its pending removal.
1660OpenMCL 1.1-pre-060623
1661- Mostly bug fixes (a CLOS bug that prevented the ObjC bridge from
1662  working, FIXNUM arrays weren't quite finished on PPC)
1664- Use Tiger inferfaces (from XCode 10.4u SDK) on DarwinPPC32
1666- Add gl, gtk2, gnome2 interfaces for x86-64.  Add a tiny
1667  "gtk2-clock" example, tweak the opengl-ffi (GLUT) example
1668  so that it works on x86-64.
1670- Some changes to the ObjC bridge to support loading additional
1671  frameworks; update the WebKit example to use these new features.
1673- Still an outstanding issue where things like MAKE-OBJC-INSTANCE
1674  need access to the interfaces at runtime (and can crash if they
1675  aren't available.)
1677- Build snapshots for LinuxPPC{32,64}.
1679OpenMCL 1.1-pre-060608
1680- The FASL version changed, as did the version number which pairs
1681  the lisp kernel with heap images.  Images saved with older kernels
1682  can't be loaded on this one; the images/kernels in the 060608
1683  snapshot tarballs should match.
1685  Most of the ABI changes that caused these version changes were
1686  x86-64 specific; some auxiliary stack pointers that had been
1687  kept in MMX registers are now kept in per-thread memory. (Signal/
1688  exception handlers generally need to be able to access these
1689  stack pointers, but at least some versions of the Linux kernel
1690  don't reliably pass correct values of the MMX registers in the
1691  signal contexts passed to signal handlers.  Moral: some kinds
1692  of stack-allocation and foreign-function operations may (or may not)
1693  be a few cycles slower, but OpenMCL should be a bit less prone
1694  to fatal segfault exceptions.)
1696  Other than that, most changes since the 060530 snapshots are
1697  bugfixes (see the ChangeLog for details).  The x86-64 port has been
1698  exercised fairly heavily (if somewhat narrowly) and its welcome
1699  banner now claims that it's a beta release.  I think that that's
1700  probably fair, and hope that anyone who may have been reluctant to
1701  test an alpha release will agree and be less reluctant.
1703- There's still much more to be done, but some preliminary 1.1 documentation
1704  is now online at:
1708  Note that some relative links on "newsite" may be invalid, but the
1709  internal links in the Doc directory should work.
1711  As noted above, it still needs a lot of work; feedback, criticism,
1712  and help would all be appreciated.
1714OpenMCL 1.1-pre-060530
1716- These release notes have gotten woefully out of date.
1718- OpenMCL now runs on x86-64 (AMD64, Intel EM64T) systems under Linux.
1719  It announces itself as an alpha release in the Welcome banner; it should
1720  in fact be very nearly feature-complete (but possibly still buggy.)
1721  There's a chicken-and-egg issue in that it needs more testing before
1722  it can be formally released and some people may be waiting for a more
1723  stable version.
1725  The build process and most user-visible things should behave the same
1726  way as on PPC; using REBUILD-CCL (described below) is generally the
1727  simplest way to rebuild from sources.  A few (intentional) differences:
1729  * the lisp kernel is named "lx86cl64", the default heap image is
1730    named "LX86CL64" (e.g., the kernel name, case-inverted) and the
1731    bootstrapping image is conventionally named "x86-boot64".
1733  * FASL files have the extension "lx64fsl"
1735  * the kernel build directory is "ccl/lisp-kernel/linuxx8664"
1737  * the "openmcl64" shell script can be used to invoke the
1738    lisp, as on 64-bit PPC platforms.
1740Other changes tend to be a little more modest:
1742- there is now a specialized FIXNUM array element type on all platforms.
1743  (distinct from T or (SIGNED-BYTE <machine-word-size>)).  Access to
1744  such vectors is a little cheaper than the SIGNED-BYTE case (since
1745  elements are known to be fixnums) and a little easier on the GC
1746  than the T case (the GC can avoid looking at their contents and
1747  there are no associated EGC write-barrier issues.)
1749- "colon" commands entered into the REPL/break loops don't need to
1750  be parenthesized if the command and all operands are on the same
1751  line. E.g.
17531> :f 0
1755  and
17571> (:f 0)
1759  are equivalent (and have the effect of examining the raw contents of
1760  the 0th stack frame)
1762- the syntax of the :B (backtrace) break-loop has changed; rather
1763  than taking an optional argument which specifies whether or not
1764  frame details should be shown, it now accepts keyword arguments
1765  for specifying:
1767  :start        ; unsigned integer: the index of the first frame to show
1768  :count        ; unsigned integer: the maximum number of frames to show
1769  :detailed-p   ; boolean: whether or not to show frame detail
1771- a new break-loop command :NFRAMES returns the number of stack frames
1772  accessible to backtrace.  (Both this change and the previous
1773  are intended to help deal with deep recursion/stack overflow cases.)
1775- any command-line arguments that follow a "--" pseudo-argument
1776  are not processed by the lisp startup code and are stored
1777  (as a list of strings) in CCL:*UNPROCESSED-COMMAND-LINE-ARGUMENTS*.
1778  E.g.:
1780shell> openmcl -- -foo 17
1783=> ("-foo" "17")
1785OpenMCL 1.1-pre-060226
1787- The --thread-stack-size (or -Z)  command-line argument changes the values
1788  of the variables used to determine the sizes of the listener thread.
1789  The values of these variables will persist accross SAVE-APPLICATION;
1790  these values have no effect on the sizes of stacks in threads created
1791  under explicit user control.
1793- New functions:
1795  (CCL:GC-VERBOSE on-full-gc &optional (on-egc on-full-gc))
1797  Causes the GC to print (or stop printing ...) informational messages
1798  on entry and exit.  The ON-FULL-GC argument controls whether or
1799  not these messages are printed on ... a full GC, and the ON-EGC
1800  argument (which defaults to the value of the ON-FULL-GC argument)
1801  controls whether messages are printed on ephemeral GCs.
1805  Returns two values (corresponding to the arguments of the last call
1806  to CCL:GC-VERBOSE.)
1809                         :RELOAD-ARGUMENTS)
1811  Depending on the values of its arguments, recompiles lisp and/or
1812  kernel sources and optionallly re-generates ("reloads") a heap
1813  image.
1815  Arguments:
1817  clean   deletes FASL and .o files before performing other steps
1818  kernel  rebuilds the lisp kernel
1819  force   forces recompilation, even if binary is newer than source
1820  reload  tries to rebuild a full heap image after other build steps
1821  exit    quits after all other steps
1822  full    equivalent to :CLEAN T :KERNEL T :RELOAD T
1823  reload-arguments a list of strings, passed as additional arguments
1824                   to the reload step.  E.g. '("--thread-stack-size" "128M").
1826  Output from the :KERNEL and :RELOAD steps is ordinarily only displayed
1827  if an error occurs.
1830- Changes
1832  TRACE now prints an integer (corresponding to the level of indentation)
1833  on each line of output.
1835  Tracing callbacks is currently broken (it may be reimplemented; if so,
1836  it'd be implemented somewhat differently ...)
1838- Bugs
1840  Several bugs involving interactions between the GC and (many) active
1841  threads have been fixed; at least one such bug remains (the symptom
1842  involves a recently allocated  array somehow getting trashed or GCed
1843  incorrectly; the  cause has been under investigation for weeks but is
1844  still not known.)
1846OpenMCL 1.1-pre-060125
1848- FASL version changed; delete old FASL (.dfsl, .pfsl, .dfsl64, .pfsl64) files
1850- "kernel ABI version" changed; build a new lisp kernel before trying to load/use
1851   060125 images.
1853-  Changes: (see also ChangeLog)
1855   New variable:
1859   Initialized on application startup to contain the MMU/OS page size in bytes.
1860   This is 4K on PPC platforms (and likely on most? all? x86 platforms).
1862   New functions:
1866   Returns an integer, the value of which is used by the lisp kernel when
1867   mapping heap memory from the OS.  Mapping requests are usually made in
1868   multiples of this value. 
1870   This value is read-only; currently, it's 64KB on 32-bit platforms and
1871   128KB on 64-bit platforms.
1876   Returns the (per-thread) allocation quantum of the process P.  By default,
1877   this is the same value as that returned by CCL:DEFAULT-ALLOCATION-QUANTUM,
1878   but lower values can be specified on a per-process basis (see below.)
1880   This value is read-only.
1885   but can be used with SETF to change the current processes's
1886   allocation quantum to a value which is between *HOST-PAGE-SIZE* and
1887   (DEFAULT-ALLOCATION-QUANTUM), inclusive, and which is a power of 2.
1890   Changes to existing functions:
1893   &key argument, which defaults to the value returned by (DEFAULT-ALLOCATION-QUANTUM).
1894   If provided, the value of the argument should should satisfy the same
1895   constraints that (SETF (CURRENT-PROCESS-ALLOCATION-QUANTUM) is subject to.
1899In general, larger per-thread allocation quanta are appropriate for programs
1900where a relatively small number of threads need to allocate memory frequently
1901and small per-thread quanta are appropriate for larger numbers of threads
1902that are expected to do small, infrequent memory allocations.
1904The worst-case scenarios would involve a large number of threads doing
1905incidental memory allocation with large quanta (that wastes memory and may
1906trigger the GC too frequently) or a small number of threads doing frequent
1907memory allocation with small quanta (since such threads could be expected
1908to fill up their small per-thread memory allocations quickly and frequently
1909and would waste time frequently allocating more small chunks.)
1911All of these values interact with the GC and EGC thresholds; the ability
1912to exercise some control over how much per-threads memory is allocated
1913at a time can help to ensure that those interactions are appropriate.
1914When these mechanisms are insufficient, applications should consider the
1915use of available mechanisms for adjusting GC and EGC thresholds.
1920OpenMCL 1.1-pre-051027
1922- A lot of internal changes in the way that special bindings, UNWIND-PROTECT,
1923  and WITHOUT-INTERRUPTS are implemented (and in how they interact with
1924  each other.
1926  One user-visible aspect of this is that UNWIND-PROTECT cleanup forms
1927  are run with interrupts disabled (the protected form is run with
1928  interrupts enabled if they were enabled on entry to the UNWIND-PROTECT.)
1929  This means that something like:
1931  (unwind-protect
1932      nil
1933    (loop))
1935  will loop uninterruptibly.
1937- CCL:WITH-INTERRUPTS-ENABLED &body body executes the body with interrupts
1938  enabled.  The example above could be rewritten as:
1940  (unwind-protect
1941      nil
1942    (with-interrupts-enabled (loop)))
1944  and the loop would be interruptible.
1946  These changes introduce binary incompatibility (the FASL version changed,
1947  as did an internal version number that tries to keep the kernel and
1948  heap image in synch.)
1950  Things basically work, but there may be lingering bugs (e.g., as of
1951  a little while ago, QUIT didn't work because the initial process
1952  was running with interrupts disabled.)
1956  argument; processes have a PROCESS-TERMINATION-SEMAPHORE accessor
1957  method.  If the argument is specified and non-null, its value should
1958  of type SEMAPHORE.
1960  If a process dies by any means after it's been successfully enabled
1961  and it has a non-null termination semaphore "at the time of its death",
1962  that semaphore will be signaled just before the underlying OS thread
1963  is destroyed.
1965  SETF can be used with PROCESS-TERMINATION-SEMAPHORE to change or
1966  clear a the termination semaphore of a process.  If the target
1967  process is not the current process when this happens, it's possible
1968  that the process could die before the SETF takes effect; this
1969  possibility must be addressed at the application level (i.e., the
1970  implementation doesn't try to synchronize the calling thread and
1971  the target in any way.
1973  A simple example:
1975  (let* ((s (make-semaphore)))
1976    (process-run-function `(:name "sleepy" :termination-semaphore ,s)
1977                           #'(lambda () (sleep 10)))
1978    (wait-on-semaphore s))
1980  The calling thread will wait for (roughly) 10 seconds (until the
1981  "sleepy" thread has had its nap and signals its termination semaphore.)
1983- A change that was introduced prior to 0.14.3 led to strange, usually
1984  fatal crashes (usually an unhandled bus error, occasionally a cryptic
1985  "can't find active area" message and a trip to the kernel debugger)
1986  under Darwin.  This was caused by an attempt to use certain Mach
1987  primitives to suspend and resume threads (the way that those
1988  primitives were used, Mach exception messages were sometimes sent
1989  twice if the first send was interrupted, and the second send occurred
1990  after the exception had already been handled (because the first send
1991  was recieved but not replied to ...)
1993  1.0 backed out of this change, and used signal handling primitives
1994  (instead of Mach primitives) to suspend and resume threads.  I -think-
1995  that I understand the issue with the Mach primitives
1996  (#_thread_abort_safely isn't necessary and caused the duplicate
1997  exception messages to be sent) and have tried to revert to using
1998  the Mach thread suspension mechanisms.  (If unhandled bus errors -
1999  that exit to the shell - or cryptic "can't find active area" messages
2000  reappear, this experiment will be shown to be a failure.)
2002  There are some obscure but good reasons for favoring the Mach
2003  primiitves, so it'd be good to know if the problem with using them
2004  has indeed been identified.
2006  (The test case involves bad luck and bad timing: two or more
2007  threads having pending exceptions at the same time and the thread
2008  whose exception is handled first tries to suspend the others, typically
2009  on behalf of the GC.  It was possible to run stress tests for many
2010  hours in 0.14.3 without encountering the bug, and possible to
2011  encounter it under seemingly light loads.)
2013- INCF and DECF argument order and fixnum arithmetic.
2015  Bryan fixed some ANSI test failures related to the order in which INCF
2016  and DECF evaluate their args.  (One example is:
2018  (let* ((x 3))
2019    (incf x (setq x 5)))
2021  where the correct answer is 10, not 8.)  We both found that fixing
2022  some cases involving INCF caused some OpenMCL code to compile
2023  incorrectly and were nervous about introducing these changes fairly
2024  late in the development cycle, so we backed out of them prior to
2025  the 1.0 code freeze.
2027  The reasons for the miscompiled code have to do with how the
2028  compiler interprets fixnum declarations under typical optimization
2029  settings.  If A and B are both declared to be FIXNUMS, then
2030  the expression
2032  (setq a (+ a b))
2034  will usually compile to a simple ADD instruction (with no overflow
2035  checking); if A and B are fixnums, the result will be a fixnum,
2036  though if an undetected overflow occurred in the addition, the
2037  result might be missing a significant bit.
2039  There was code in OpenMCL that assumed that
2041  (incf a b)
2043  was exactly the same as
2045  (setq a (+ a b))
2047  and in fact that was true under the old (incorrect) definition of
2048  INCF.  The new definition introduced some temporary bindings:
2050  (let* ((...)
2051         (#:temp (+ a b))
2052         (...))
2053     (setq a #:temp))
2055  In this case, the addition was allowed to generate an overflow
2056  (no type declaration on #:temp), and the SETQ quietly violated
2057  a type declaration (assigning a non-FIXNUM value to A), leading
2058  to further problems.
2060  So far, I found a couple of cases of this in the OpenMCL sources.
2061  (FWIW, both functions were originally transliterated from C code
2062  and were trying to mimic C's silent overflow behavior.)
2064  Moral: if you have code that assumes that INCF or DECF expand
2065  into simple assignments and are trying to exploit the ways that
2066  those assignments interact with type declarations, you may
2067  want to review those assumptions.  If you write code that has
2068  side effects in the DELTA arguments of INCF or DECF rorms,
2069  you'll (hopefully) be pleased to see that Bryan's changes
2070  allow these side-effects to be handled correctly (at the
2071  right time.)  If you don't fall into either of these categories,
2072  you probably won't notice any difference ...
2074- 64-bit Linux support
2076  There's a 64-bit LinuxPPC heap image and some rudimentary (libc-only)
2077  64-bit Linux interfaces in the testing directory.
2079  (Unlike 64-bit Darwin, 64-bit Linux distributions typically provide
2080  64-bit versions of "all" standard libraries; I haven't gotten around
2081  to building 64-bit gnome/gtk/X11/... interfaces yet, but wouldn't
2082  expect there to be a problem.)
2084  The 64-bit Linux OpenMCL seems to basically work, but ... OpenMCL
2085  likes to map its kernel into low addresses (around #x5000); this
2086  allows compiled lisp code to use conditional branches to "short"
2087  (16-bit) absolute addresses.  Newer Linux kernels provide a
2088  "vdso" shared library that's intended to simply communication
2089  between the OS kernel and userspace libraries and programs; when
2090  a program is mapped at "non-standard" addresses, the vdso gets
2091  mapped at address 0.
2093  I don't fully understand the imlications of this (beyond the fact that
2094  indirecting through a NULL pointer will access bits and pieces
2095  of the vdso instead of segfaulting.)  As far as I know, this is
2096  seen as a minor bug in the Linux kernel, and I -think- that I've
2097  seen kernel ChangeLog entries that indicate that the problem's been
2098  fixed in the relatively recent past (and will likely start to
2099  make it into Linux distributions in the near future.)
2101  That said - and seeing a library at address 0 certainly makes me a
2102  little nervous - the LinuxPPC64 port seems to work at least as
2103  well as the DarwinPPC64 port does (i.e., there may be word-size
2104  or other bugs lurking around or hiding in plain sight, but it's
2105  not usually easy to encounter them.)
2107- As documented (and as hasn't been true in a long time), EOF
2108  from *STANDARD-INPUT* terminates the REPL when the --batch argument
2109  is in effect (even if *STANDARD-INPUT* is a tty.)
2111- QUIT does a FRESH-LINE on and FORCE-OUTPUT to the standard output
2112  stream (people had reported that output wasn't always flushed
2113  when --batch or --eval was used; 1.0 was better about this than
2114  previous versions were, but it still wasn't reliable.)
2116OpenMCL 1.1-pre-051028
2117I had been doing development on G5s, and hadn't noticed that the
211832-bit lisp had been using a 64-bit instruction.  (I'm a little
2119confused about how that could have worked; perhaps the 64-bit
2120instruction gets emulated by the OS, or perhaps my model of
2121whether 64-bit instructions can be executed in 32-bit mode
2122is simply incorrect.)
2124In any case, the 32-bit images produced yesterday don't run on
2125G4s (or presumably G3s or older systems.)  Ooops.  New images.
2127OpenMCL 1.1-pre-051029
2128 A function used by both SET-USER-ENVIRONMENT
2129and SET-DEVELOPMENT-ENVIRONMENT wasn't properly changing saved bindings
2130of *PACKAGE*; the last few 1.1-pre releases have come up in the CCL
2131package, as a result.  Ooops again; new images, again.
2134OpenMCL 1.1-pre-051204
2135Not a lot of user-visible changes, but the changes that're there
2136are a little hard to bootstrap.
2137Note that new kernel build directories (darwinppc, darwinppc64,
2138linuxppc, linuxppc64, ...) repace the old versions that don't
2139have "ppc" in their names.  CVS may not prune the old directories,
2140especially if they contain files (.o, random junk).
Note: See TracBrowser for help on using the repository browser.