source: trunk/ccl/release-notes.txt @ 6958

Last change on this file since 6958 was 6958, checked in by gb, 12 years ago

Update for 1.1-pre-070722.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Author Date Id Revision
File size: 99.7 KB
1OpenMCL 1.1-pre-070722
2- This will hopefully be the last set of snapshots whose version
3  number contains the string "pre-"; whether or not the last
4  20 months worth of "1.1-pre-yymmdd" snapshot releases are
5  more or less stable than something without "pre-" in its name
6  doesn't have too much to do much to do with whether or not "pre-"
7  is in the version number (and has lots to do with other things.)
8  I'd like to move to a model that's mostly similar to how things
9  have been (new version every month or two, old versions become
10  obsolete soon after, sometimes changes introduce binary incompatiblity)
11  but drop the "prerelease" designation and change the name of the
12  "testing" directory to something like "current".
13- The FASL version didn't change (for the first time in a long time.)
14  It's probably a lot easier to bootstrap new sources with a new
15  lisp and it's probably desirable to recompile your own source
16  code with the new lisp, but there shouldn't be any user-visible
17  low-level ABI changes that make that mandatory.
18- CCL::WITH-ENCODED-CSTRS (which has been unexported and somewhat
19  broken) is now exported and somewhat less broken.
21  (ccl:with-encoded-cstrs ENCODING-NAME ((varI stringI)*) &body body)
23  where ENCODING-NAME is a keyword constant that names a character
24  encoding executes BODY in an environment where each variable varI
25  is bound to a nul-terminated, dynamic-extent foreign pointer to
26  an encoded version of the corresponding stringI.
28  (ccl:with-cstrs ((x "x")) (#_puts x))
30  is functionally equivalent to:
32  (ccl:with-encoded-cstrs :iso-8859-1 ((x "x")) (#_puts x))
35  CCL:WITH-ENCODED-CSTRS doesn't automatically prepend byte-order-marks
36  to its output; the size of the terminating #\NUL depends on the
37  number of octets-per-code-unit in the encoding.
39  There are certainly lots of other conventions for expressing
40  the length of foreign strings besides NUL-termination (length in
41  code units, length in octets.)  I'm not sure if it's better to
42  try to come up with high-level interfaces that support those
43  conventions ("with-encoded-string-and-length-in-octets ...")
44  or to try to support mid-level primitives ("number of octets
45  in encoded version of lisp string in specified encoding", etc.)
47- STREAM-ERRORs (and their subclasses, including READER-ERROR)
48  try to describe the context in which they occur a little better
49  (e.g., by referencing the file position if available and
50  by trying to show a few surrounding characters when possible.)
51  Since streams are usually buffered, this context information
52  may be incomplete, but it's often much better than nothing.
54- Leading tilde (~) characters in physical pathname namestrings
55  are expanded in the way that most shells do:
57  "~user/...." can be used to refer to an absolute pathname rooted
58  at the home directory of the user named "user"
60  "~/..." can be used to refer to an absulte pathname rooted at
61  the home directory of the current user.
63- The break-loop colon commands for showing the contents of
64  stack frames try to present the frame's contents in a way that's
65  (hopefully) more meaningful and useful.  For each stack frame
66  shown in detail, the corresponding function's argument list
67  is printed, followed by the current values of the function's
68  arguments (indented slightly), a blank line, and the current
69  values of the function's local variables (outdented slightly.)
70  The old method of showing a stack frame's "raw" contents is
71  still available as the :RAW break loop command.
73  The new style of presenting a stack-frame's contents is also
74  used in the Cocoa IDE.
76- It's historically been possible to create stacks (for threads
77  other than the original one) whose size exceeds the nominal
78  OS resource limits for a stack's size.  (OpenMCL's threads
79  use multiple stacks; the stack in question is the one that
80  OpenMCL generally refers to as the "control" or "C" stack.)
81  It's not entirely clear what (if anything) the consequences
82  of exceeding these limits have been, but OpenMCL's GC can
83  use all of the available (C) stack space that it thinks it
84  has under some conditions, and, under OSX/Mach/Darwin, there
85  have been reports of excessive page file creation and paging
86  activity that don't seem related to heap behavior in environments
87  where the GC is running on (and possibly using much of) a stack
88  whose size greatly exceeds the hard resource limit on stack
89  size.
91  Trying to determine exactly what was causing the excessive
92  pages got me trapped in a twisty maze of Mach kernel sources,
93  all alike.  I tried to pin C stack size to the hard resource
94  limit on stack size and have not been able to provoke the
95  excessive paging problems since, but am not confident in
96  concluding (yet) that the problems had to do with resource
97  limits being exceeded.
99  The hard resource limits on stack size for the OS versions
100  that I have readily available (in bash, do "ulimit -s -H";
101  in tcsh, it's "limit -h s", don't know offhand about other
102  shells) are:
104  unlimited on Linux
105  ~512M on FreeBSD
106  ~64M on Darwin
108  The effect of observing (rather than exceeding) this limit
109  on the maximum depth of lisp recursion in OpenMCL is:
111  * nothing, on x86-64 (the C stack is not used by lisp code
112    on x86-64)
114  * visible on ppc32, which uses 4 32-bit words on the control
115    stack for each lisp function invocation
117  * more visible on ppc64, which uses 4 64-bit words of control
118    stack for each lisp function invocation.
120  That seems to suggest that (given that the actual stack resource
121  limit is a bit under 64M and that OpenMCL signals stack overflow
122  when the stack pointer gets within a few hundred KB of the actual
123  limit) that ppc64 threads are now limited to a maximum of about
124  2000000 function calls.
126  (All of this only matters if attempts are made to create threads
127  with large stacks; the default stack sizes in OpenMCL are usually
128  1-2 MB.)
130- On a cheerier (and certainly less confusing) note: for the last few
131  years, OpenMCL has shipped with an extended example which provides an
132  integrated development environment (IDE) based on Cocoa; that's often
133  been described as "the demo IDE" and could also be fairly described as
134  "slow", "buggy", "incomplete", and "little more than a proof of
135  concept."
137  I think that it's fair to describe the current state of the IDE as
138  being "less slow", "less buggy", "less incomplete", and "much more
139  than a proof of concept" than it has been (e.g., there's been some
140  actual progress over the last few months and there are plans to
141  try to continue working on the IDE and related tools.)  It'd probably
142  be optimistic to call it "usable" in its current state (that may
143  depend on how low one's threshold of usability is), but I hope that
144  people who've been discouraged by the lack of IDE progress over the
145  last few years will see reason to be encouraged (and that anyone
146  interested will submit bug reports, patches, feature requests, code ...)
149OpenMCL 1.1-pre-070512
150- The FASL version changed (old FASL files won't work with this
151  lisp version), as did the version information which tries to
152  keep the kernel in sync with heap images.  Note that it's generally
153  a lot easier to recompile recent sources with recent images, e.g.,
154  trying to compile 070512 sources with an 070407 image is unlikely
155  to work without tricky bootstrapping.
156- Most of the changes in this release involve the calling sequence
157  used on x86-64.  In very general terms, some kinds of function-call
158  intensive code may see a significant performance boost, most code
159  should see a slight improvement, some code might see a (hopefully
160  very slight) degradation, and anything significantly slower than
161  previous releases should be reported as a bug.
162  It is -possible- that some of these changes may cause errors to
163  be reported differently (the function reported as the function
164  executing when the error ocurred might be different/wrong).  I
165  have not seen as many cases of this as I expected to when making
166  the change, but am also not sure that I fixed all possible cases.
167- The FFI-related reader macros #_, #$, and #& all read a case-sensitive
168  foreign function, constant, or variable name from the input stream
169  and try to find the corresponding definition in the interface files.
170  If the name is prefixed with a #\? - as in #_?foo - the macros
171  return true if the definition could be found and false otherwise.
172  (The general idea is that this might be useful for conditionalizing
173  code in some cases, and there should be -some- way of quietly testing
174  that something's defined.)
175- There is now support for making the contents of (possibly very large)
176  files accessible as lisp vectors.  (This may be many times faster
177  than something like
179  (let* ((stream (open pathname :direction :input :element-type 'whatever))
180         (vector (make-array (file-size-to-vector-size stream)
181                             :element-type 'whatever)))
182    (read-sequence vector stream))
184  but has the similar effect of making the contents of VECTOR match the
185  contents of the file.)
187  CCL:MAP-FILE-TO-IVECTOR pathname element-type [Function]
189  "element-type" should be a type specifier such that
190  (UPGRADED-ARRAY-ELEMENT-TYPE element-type) is a subtype
191  of either SIGNED-BYTE or UNSIGNED-BYTE.
193  Tries to open the file named by "pathname" for reading and to
194  map its contents into the process's address space via #_mmap;
195  if successful, returns a lisp vector of element-type
196  (UPGRADED-ARRAY-ELEMENT-TYPE element-type) which is displaced
197  to an underlying (SIMPLE-ARRAY element-type (*)) whose contents
198  match the mapped file's.
200  Because of alignment issues, the mapped file's contents will
201  start a few bytes (4 bytes on 32-bit platforms, 8 bytes on 64-bit
202  platforms) "into" the vector; the displaced array returned by
203  CCL:MAP-FILE-TO-IVECTOR hides this overhead, but its usually
204  more efficient to operate on the underlying simple 1-dimensional
205  array.  Given a displaced array (like the value returned by
207  returns the underlying array and the displacement index in elements.
209  Currently, only read-only file mapping is supported; the underlying
210  vector will be allocated in read-only memory, and attempts to use
211  (e.g.) (SETF (AREF ...) ...) to modify the mapped vector's contents
212  will result in memory faults.
214  CCL:MAP-FILE-TO-OCTET-VECTOR pathname [Function]
216  Equivalent to (CCL:MAP-FILE-TO-IVECTOR pathname '(UNSIGNED-BYTE 8)).
218  CCL:UNMAP-IVECTOR displaced-vector
220  If the argument is a mapped vector (as returned by
221  MAP-FILE-TO-IVECTOR) that has not yet been "unmapped" by this
222  function, undoes the memory mapping, closes the mapped file, and
223  adjusts its argument so that it's displaced to a 0-length vector.
227  Note that whether a vector's created by MAKE-ARRAY or by mapping
228  a file's contents, it can't have ARRAY-TOTAL-SIZE-LIMIT or more
229  elements.  (ARRAY-TOTAL-SIZE-LIMIT is (EXPT 2 24) in 32-bit OpenMCL
230  and (EXPT 2 56) in 64-bit versions.
232- The lisp kernel now tries to signal memory faults that occur when
233  running lisp code as lisp errors.  As a silly example:
235  ? (defun foo (x)
236     "Crash and burn if X is not a list"
237     (declare (optimize (speed 3) (safety 0)) (list x))
238     (car x))
239  FOO
240  ? (foo 0)
241  > Error: Fault during read of memory address #x4
242  > While executing: FOO, in process listener(1).
244  The fact that things are handled this way (rather than going
245  into the kernel debugger with no easy way of recovering) makes
246  it possible to continue a session without losing work in many
247  cases.  In a trivial example like the one above, it's relatively
248  easy to see that no harm has been done and the error should
249  not be hard to recover from.  In some other cases, it may be
250  true that a buggy function has been scribbling ofer memory for
251  a while before that scribbling resulted in a machine exception.
253  Moral: if you get an unexpected "memory fault" error (the
254  condition type is actually CCL::INVALID-MEMORY-ACCESS) and
255  don't understand why the fault occurred and the consequences
256  of continuing in the lisp session where the fault occurred,
257  you should view the state of that session with some suspicion.
259  Faults in foreign code (should) still trap into the kernel
260  debugger.  (It'd be nice to be able to treat these as lisp
261  errors with the same caveats as described above, but that
262  is more complicated in some cases and isn't yet implemented.)
264- An obscure kernel debugger command - (A), which tries to advance
265  the program counter by one instruction - is now disabled on x86-64.
266  (On the PPC, "one instruction" always meant "4 bytes"; implementing
267  this correctly on x86-64 would require the ability to at least
268  partially disassemble arbitrary x86-64 instructions.)
270  On the other hand, the kernel debugger should be able to show
271  FPU registers on x86-64.
274OpenMCL 1.1-pre-070408
275- The FASL version changed (old FASL files won't work with this
276  lisp version), as did the version information which tries to
277  keep the kernel in sync with heap images.  Note that it's generally
278  a lot easier to recompile recent sources with recent images, e.g.,
279  trying to compile 070408 sources with an 070214 image is unlikely
280  to work without tricky bootstrapping.
281- There's now a Trac bug-tracking/wiki site for OpenMCL at
282  <>.  It needs bug reports; please
283  visit that site and use the features there to report any bugs
284  that you find.
286  (CCL:DEFSTATIC var value &optional doc-string)
287  is like DEFPARAMETER in that it proclaims the variable "var" to
288  be special, sets its value to "value", and sets the symbol's
289  VARIABLE documentation to the optional doc-string.  It differs
290  from DEFPARAMETER in that it further asserts that the variable
291  should never be bound dynamically in any thread (via LET/LAMBDA/etc.);
292  the compiler treats any attempts to bind a "static" variable as an
293  error.
294  It is legal to change the value of a "static" variable, but since
295  all threads see the same (static) binding of that variable it may
296  be necessary to synchronize assignments made from multiple threads.
297  (A "static" variable binding is effectively a shared, global resource;
298  a dynamic binding is thread-private.)
299  Access to the value of a static variable is typically faster than
300  is access to the value a special variable that's not proclaimed to
301  be "static".
302  This functionality has been in MCL/OpenMCL for a long time under
303  the name CCL:DEFGLOBAL; CCL:DEFGLOBAL is an alias for CCL:DEFSTATIC,
304  but the latter seemed to be a better name.
305- The type of foreign object that a MACPTR points to can now be
306  asserted (this means that a MACPTR object can contain a small
307  integer which identifies the alleged FOREIGN-TYPE of the object that
308  the points to.  RLET, MAKE-RECORD, and MAKE-GCABLE-RECORD (see below)
309  assert the foreign type of the object that the MACPTR object they
310  create (as do some new features of the ObjC bridge, described further
311  below.)
312  PRINT-OBJECT on a MACPTR will try to print information about the
313  asserted type of that pointer, as well as information about where
314  the pointer was allocated (heap, stack) and whether it's scheduled
315  for automatic reclamation by the GC.
316  A few constructs that conceivable should assert the type of the
317  pointers they create (e.g., some flavor of PREF, SLOT-VALUE in
318  the ObjC bridge) don't yet do so.
319  A rather obvious way of exploiting typed pointers - namely, extending
320  DESCRIBE and INSPECT to show the contents of foreign records - is
321  not yet implemented.
322- MAKE-GCABLE-RECORD is like MAKE-RECORD, in that it "makes an instance
323  of a foreign record type".  (Or, to be more banal about it, uses
324  #_malloc to allocate a block of foreign memory of the size of the
325  foreign record type named by its argument.)  MAKE-GCABLE-RECORD
326  additionally tells the lisp garbage collector that it should free
327  the foreign memory when the MACPTR object that describes it becomes
328  garbage.
329  When using "gcable pointers", it's important to remember the
330  distinction between a MACPTR object (which is a lisp object, more-
331  or-less like any other) and the block of foreign memory that the
332  MACPTR object points to.  If a gcable MACPTR is the only thing
333  in the world ("lisp world" or "foreign world") that references
334  the underlying block of foreign memory, then freeing the foreign
335  memory when it becomes impossible to reference it is convenient
336  and sane.  If other lisp MACPTRs reference the underlying block
337  of foreign memory or if the address of that foreign memory is
338  passed to and retained by foreign code, having the GC free the
339  memory may have unpleasant consequences if those other references
340  are used.
341- CCL:FREE (which is mostly just a wrapper around #_free that allows
342  #_free to be called early in the bootstrapping process) is now
343  exported; if its argument is a gcable pointer (e.g., created via
344  MAKE-GCABLE-POINTER), it will tell the GC that the pointer's
345  foreign memory has been freed "manually" before calling #_free.
346- The mechanisms used to implement locks has changed (for the curious,
347  the changes involve the use of spinlocks rather than a sequence
348  of atomic additions.)  Someone demonstrated a case of deadlock
349  (where a thread was waiting for a lock that was available) under
350  the old implementation.  I'm not sure that I fully understand how
351  that could have happened, but the new implementation at least has
352  the advantage of being a little easier to understand and might be
353  a tiny bit faster.  Please let me know if either of these assumptions
354  was incorrect.
355- An EOF (control-d) in the REPL (when standard input is a tty or pty
356  device) has traditionally caused an exit to the outer break loop
357  (or had no effect if the REPL was not in a break loop).  If
358  CCL:*QUIT-ON-EOF* is set, an EOF causes the lisp to quit.  (It
359  actually invokes a listener-specific method, so in a multi-listener
360  window system environemt, it might simply cause the listener which
361  receives the EOF to exit.)
362  None of this has any effect when running under environments like
363  SLIME, and (as mentioned above) only matters if the standard input
364  devices is a tty or pseudo-tty (where it's possible to continue
365  reading after an EOF has been read.)  If running under an xterm
366  or OSX, standard input is probably a pty; if running
367  in an Emacs shell buffer or under other means under emacs, different
368  types of IPC mechanisms (pipes, sockets) might be used.
369- SAVE-APPLICATION has historically changed the type of all MACPTR
370  objects (turning them into "dead macptrs", since it's generally
371  meaningless to refer to a foreign pointer from a previous session
372  and generally better to get a type error than some more mysterious
373  or severe failure).  This no longer happens for null pointers (pointers
374  to address 0); COMPILE-FILE also now allows null pointers to be referenced
375  as constants in compiled code.
376- Not entirely coincidentally, CCL:+NULL-PTR+ is now defined as a constant
377  (whose value is a null pointer.)  In some cases, it may be more
378  efficient or convenient to pass CCL:+NULL-PTR+ to foreign code than
379  it would be to call (CCL:%NULL-PTR) to "produce" one.
380- Historically, OpenMCL (and MCL) have maintained a list of open file
381  streams in the value of CCL:*OPEN-FILE-STREAMS*; maintaining this
382  list helps to ensure that streams get closed in as orderly a manner
383  as possible when the lisp exits.  The code which accessed this list
384  didn't do so in a thread-safe manner.
385  The list is now maintained in a lexical variable; the function
386  CCL:OPEN-FILE-STREAMS returns a copy of that list,
387  CCL:NOTE-OPEN-FILE-STREAM adds its argument (a file stream) to the
388  list, and CCL:REMOVE-OPEN-FILE-STREAM removes its argument (a file stream)
389  from the list.  (All of these functions use appropriate locking.)
390- There were a number of timing-related problems related to PROCESS-INTERRUPT
391  (usually involving rapidly and repeatedly interrupting a thread over
392  a long period of time.)  This should be a lot more reliable now
393  (exactly what could go wrong and why and how is all a little hard to
394  describe.)
395- Some Linux distributions may initialize the user's environment in
396  a way that imposes a soft limit on the amount of virtual memory that
397  a process is allowed to map.  OpenMCL now tries to raise this limit
398  before reserving what may be a very large amount of address space,
399  thanks to a patch from Andi Kleen.
400- There were a number of problems with UTF-16 streams, found and
401  fixed by Takehiko Abe.
402- Takehiko Abe also provided fixes for some code in "ccl:lib;xref.lisp"
403  and in source-file recording/reporting that (still) didn't understand
404  the concept of EQL-SPECIALIZER metaobjects.
405- ObjC bridge and ObjC examples
406  The ObjC bridge provides a few new mechanisms for defining ObjC
407  methods, for calling ObjC "generic functions" (e.g., message sending),
408  and for dealing with frequently-used record types and with differences
409  between 32-bit and (forthcoming) 64-bit ObjC/Cocoa implementations.
411  A lot of macros/functions/other things that really should have been
412  exported from some package for the last few years finally have been
413  exported from the OBJC or NS packages (and a lot of things that have
414  historically been internal to CCL are re-imported into CCL).
416  Cocoa (and the underlying Core Graphics libraries) have historically
417  used 32-bit floats and 32-bit integers in data structures that describe
418  geometry, font sizes and metrics, and elsewhere.  64-bit Cocoa will
419  use 64-bit floats and 64-bit integers in many cases.
421  The bridge defines the type NS:CGFLOAT as the lisp type of the
422  preferred float type on the platform, and the constant NS:+CGFLOAT+.
423  On DarwinPPC32, the foreign types :cgfloat, :<NSUI>nteger, and
424  :<NSI>nteger are defined by the bridge (as 32-bit float, 32-bit
425  unsigned integer, and 32-bit signed integer, respectively.); these
426  types are defined (as 64-bit variants) in the 64-bit interfaces.
428  All ObjC classes are properly named, either with a name exported
429  from the NS package (in the case of a predefined class declared in
430  the interface files) or with the name provided in the DEFCLASS
431  form (with :METACLASS NS:+NS-OBJECT) which defines the class from
432  lisp.  The class's lisp name is now proclaimed to be a "static"
433  variable (as if by DEFSTATIC, as described above) and given the
434  class object as its value.  In other words:
436(send (find-class 'ns:ns-application) 'shared-application)
438  and
440(send ns:ns-application 'shared-application)
442  are equivalent.  (Since it's not legal to bind a "static" variable,
443  it may be necessary to rename some things so that unrelated
444  variables whose names coincidentally conflict with ObjC class names
445  don't do so.)
447- A new reader macro - #/ - reads a sequence of "constituent" characters
448  (including colons) from the stream on which it appears and interns
449  that sequence - with case preserved and colons intact - in a new package
450  whose name is NEXTSTEP-FUNCTIONS, exporting the symbol from that package.
451  This means that the act of reading "#/alloc" returns the the symbol
452  NEXTSTEP-FUNCTIONS:|alloc|, and the act of reading "#/initWithFrame:"
453  returns the symbol NEXTSTEP-FUNCTIONS:|initWithFrame:|.  The intent
454  is that the reader macro can be used to construct symbols whose names
455  match ObjC message names; the reader macro tries to do some sanity
456  checks (such as insisting that a name that contains at least one
457  colon ends in a colon), but isn't totally rigourous about enforcing
458  ObjC message name conventions.
460  A symbol read using this macro can be used as an operand in
461  most places where an ObjC message name can be used, such as
462  in the (@SELECTOR ...) construct (which is now OBJC:@SELECTOR,
463  btw.)
465  Marco Baringer proposed the idea of using a reader macro to
466  construct lisp symbols which matched ObjC message names.
468- The act of interning a new symbol in the NEXTSTEP-FUNCTIONS
469  package triggers an interface database lookup of Objc methods
470  with the corresponding message name.  If any such information
471  is found, a special type of dispatching function is created
472  and initialized and the weird-looking symbol is given that
473  dispatching function as its function definition.
475  The dispatching knows how to call declared ObjC methods defined on
476  the message.  In many cases, all methods have the same foreign type
477  signature, and the dispatching function merely passes any arguments
478  that it receives to a function that does an ObjC message send with
479  the indicated foreign argument and return types.  In other cases,
480  where different ObjC messages have different type signatures, the
481  dispatching function tries to choose a function that handles the
482  right type signature based on the class of the dispatching function's
483  first argument.
485  If new information about ObjC methods is introduced (e.g., by
486  using additional interface files or as ObjC methods are defined
487  from lisp), the dispatch function is reinitialized to recognize
488  newly-introduced foreign type signatures.
490  The argument and result coercion that the bridge has tradionally
491  supported is supported by the new mechanism (e.g., :<BOOL> arguments
492  can be specified as lisp booleans and :<BOOL> results are returned
493  as lisp boolean values, and an argument value of NIL is coerced to
494  a null pointer if the corresponding argument type is :ID.
496  Some ObjC methods accept variable numbers of arguments; the
497  foreign types of non-required arguments are determined by the
498  lisp types of those arguments (e.g., integers are passed as
499  integers, floats as floats, pointers as pointers, record types
500  by reference.)
502  Some examples:
504;;; #/alloc is a known message.
505? #'#/alloc
507;;; Sadly, #/foo is not ...
508? #'#/foo
509> Error: Undefined function: NEXTSTEP-FUNCTIONS:|foo|
511;;; We can send an "init" message to a newly-allocated instance of
512;;; "NSObject" by:
514(send (send ns:ns-object 'alloc) 'init)
516;;; or by
518(#/init (#/alloc ns:ns-object))
520  ObjC methods that "return" structures return them as gcable pointers
521  when called via dispatch functions.  E.g., if "my-window" is an
522  NS:NS-WINDOW instance, then
524(#/frame my-window)
526  will return a gcable pointer to a structure that describes that window's
527  frame rectangle.  (The good news is that there's no need to use SLET
528  or special structure-returning message send syntax; the bad news is
529  that #_malloc, #_free, and the GC are all involved in the creation
530  and eventual destruction of structure-typed return values.  Unless
531  and until those factors negatively affect performance, the advantages
532  seem to outweigh the disadvantages.)
534- Since foreign pointers are now (sometimes, somewhat) typed, it's
535  possible to treat pointers to some foreign types as "instances of
536  built-in classes."  Specifically, a pointer to an :<NSR>ect is
537  recognized as an instance of the built-in class NS:NS-RECT, a
538  pointer to an <NSS>ize is treated as an instance of NS:NS-SIZE,
539  <NSP>oint is recognized as NS:NS-POINT, and <NSR>ange maps to
540  NS:NS-RANGE.  (There are a few other more obscure structure
541  types that get this treatment, and with a little more work the
542  mechanism could be made extensible.)
544  For each of these built-in classes:
546  - a PRINT-OBJECT method is defined
548  - a foreign type name derived from the class name (e.g., :NS-RECT
549    for NS:NS-RECT) is made an alias for the corresponding type
550    (so it's possible to say (RLET ((R :NS-RECT)) ...)).
552  - the class is is integrated into the type system (so that
553    (TYPEP R 'NS:NS-RECT) is fairly efficently implemented.)
555  - inlined accessor and setf inverses are defined for the structure
556    type's fields.  In the case of an :<NSR>ect, the fields in question
557    are the fields of the embedded point and size, so NS:NS-RECT-X,
559    are defined.  The accessors and setter functions typecheck their
560    arguments and the setters handle coercion to the approprate type
561    of CGFLOAT where applicable.
563  - an initialization function is defined; (NS:INIT-NS-SIZE s w h) is
564    roughly equivalent to (SETF (NS:NS-SIZE-WIDTH s) w
565    (NS:NS-SIZE-HEIGHT s) h), but might be a little more efficient.
567  - a creation function is defined: (NS:NS-MAKE-POINT x y) is basically
568    equivalent to:
570      (NS:INIT-NS-POINT P X Y)
571      p)
573  - a macro is defined which (much like RLET) stack-allocates an
574    instance of the foreign record type, optionally iniitializes
575    that instance, and executes a body of code with a variable
576    bound to that instance.  E.g.
578    (ns:with-ns-range (r loc len)
579      (format t "~& range has location ~s, length ~s"
580         (ns:ns-range-location r) (ns:ns-range-length r)))
582    which is probably not the world's most realistic example.
584   Note that it's possible to construct a record
585   instance that has a very short useful lifetime:
587   (#/initWithFrame: new-view (ns:ns-make-rect 100 100 200 200))
589   The rectangle above will -eventually- get reclaimed by the GC;
590   if you don't want to give the GC so much work to do, you might
591   prefer to do:
593   (ns:with-ns-rect (r 100 100 200 200)
594     (#/initWithFrame: new-view r))
597 - The macro OBJC:DEFMETHOD can be used to define ObjC methods.
598   It looks superficially like CL:DEFMETHOD in some respects.
599   The syntax is:
601   (OBC:DEFMETHOD name-and-result-type ((receiver-arg-and-class) &rest other-args) &body body)
603   where:
605   "name-and-result-type" is either an ObjC message name (use #/ !)
606   for methods that return a value of type :ID, or a list of an ObjC
607   message name and a foreign type specifier for methods with a different
608   foreign result type
610   "receiver-type-and-class" is a two-element list whose CAR is
611   a variable name and whose CADR is the lisp name of an ObjC class
612   or metaclass.  The receiver variable name can be any bindable
613   lisp variable name, but SELF (in some package) might be a reasonable
614   choice.  The receiver variable is declared to be "unsettable", i.e.,
615   it is an error to try to change the value of the receiver in the
616   body of the method definition.
618   "other-args" are either variable names (denoting parameters of type
619   :ID) or 2-element lists whose first element is a variable name and
620    whose second element is a foreign type specifier.
622  For example:
624(objc:defmethod (#/characterAtIndex: :unichar)
625    ((self hemlock-buffer-string) (index :<NSUI>nteger))
626  ...)
628  The method "characterAtIndex:", when invoked on an object of class
629  HEMLOCK-BUFFER-STRING with an additional argument of type :<NSU>integer
630  returns a value of type :unichar.)
632  Arguments that wind up as some non-:ID pointer type (pointers,
633  records passed by value) are represented as typed foreign pointers
634  (so the higher-level, type-checking accessors can be used on
635  arguments of type :ns-rect, :ns-pointe, etc.)
637  Within the body of methods defined via OBJC:DEFMETHOD, the local
638  function CL:CALL-NEXT-METHOD is defined.  It isn't quite as
639  general as CL:CALL-NEXT-METHOD is when used in a CLOS method,
640  but it has some of the same semantics.  It accepts as many arguments
641  as are present in the containing method's "other args" list and
642  invokes version of the containing method that would have been
643  invoked on instances of the receiver's class's superclass with
644  the receiver and other provided arguments.  (The idiom of passing
645  the current method's arguments to the next method is common enough
646  that the CALL-NEXT-METHOD in OBJC:DEFMETHODs should probably do
647  this if it receives no arguments.)
649  A method defined via OBJC:DEFMETHOD that returns a structure "by value"
650  can do so by returning a record created via MAKE-GCABLE-RECORD, by
651  returning the value returned via CALL-NEXT-METHOD, or by other similar
652  means.  Behind the scenes, there may be a pre-allocated instance of
653  the record type (used to support native structure-return conventions),
654  and any value returned by the method body will be copied to this
655  internal record instance.  Within the body of a method defined with
656  OBJC:DEFMETHOD that's declared to return a structure type, the local
657  macro OBJC:RETURNING-FOREIGN-STRUCT can be used to access the internal
658  structure:
660  (objc:defmethod (#/reallyTinyRectangleAtPoint: :ns-rect)
661    ((self really-tiny-view) (where :ns-point))
662    (objc:returning-foreign-struct (r)
663      (ns:init-ns-rect r (ns:ns-point-x where) (ns:ns-point-y where)
664                          single-float-epsilon single-float-epsilon)
665      r))
667 - If OBJC:DEFMETHOD introduces a new ObjC message, a ... message
668   to that effect.  Sometimes, these messages are merely informative
669   (and barely informative, at that ...), but they may also indicate
670   that a message name is misspelled (or possibly missing a trailing
671   colon.)  If a method is redefined in such a way that it's type
672   signature changes, a continuable error is signaled.
674 - there used to be some fairly obscure reasons that led to
675   MAKE-OBJC-INSTANCE being a bit more efficient than MAKE-INSTANCE
676   in some cases (some of the methods invoked by MAKE-INSTANCE did
677   some extra work to handle Lisp slots even if the class didn't
678   define any Lisp slots.  This work isn't done anymore, and consequently
679   there's less reason to prefer MAKE-OBJC-INSTANCE.  (MAKE-OBJC-INSTANCE
680   is still defined and exported from the OBJC:PACKAGE).
682 - the preferred means of loading an add-on framework and processing
683   the declarations in its interfaces has changed several times over
684   the last several months.  The currently preferred (new) way to
685   do that is via the new function OBJC:LOAD-FRAMEWORK
687   (OBJC:LOAD-FRAMEWORK framework-name interface-dir)
689   where "framework-name" is a string which names the framework and
690   "interface-dir" is a keyword that names the associated set of
691   interfaces.  OBJC:LOAD-FRAMEWORK should find and initialize the
692   framework bundle (looking in standard framework search paths),
693   introduce new ObjC classes to CLOS, update information about
694   declared messages and their methods' type signatures, adjust
695   affected dispatch functions, and make the interfaces other
696   definitions available.  The order in which it does these
697   things isn't specified, and may change in the future.
699 - Most Cocoa-related examples (the demo IDE, the Rubix and Webkit
700   examples) have been rewritten to use the new bridge features.
701   (I may have missed some contributed examples; if people want
702   to convert these, that'd be great.)  It's too early to say
703   whether the new approach is better or worse than the old, but
704   I've (so far) found some of the code easier to read and maintain.
705   We might find that some things that (for instance) SEND does
706   more efficiently could and should be done via SEND (I'm thinking
707   mostly of struct-return stuff), but so far I haven't seen the
708   new stuff keel over.
710   The converted code looks like "lisp code with strange-looking
711   function names" at first glance, and that seems about right.
712   The function names might get to look more familiar as the
713   reader becomes more familiar with Cocoa; as someone here pointed
714   out, it's arguably good that the function names are distinctive
715   in that that helps to remind the reader that these are likely
716   lower-level functions that are less tolerant of type- and other
717   errors than the typical lisp function would be.
721OpenMCL 1.1-pre-070214
722- The FASL version changed (old FASL files won't work with this
723  lisp version), as did the version information which tries to
724  keep the kernel in sync with heap images.
725- There are new interface files for all platforms.  These files
726  encode some foreign type information a little differently
727  than older ones did (notably information about foreign functions
728  that return structures or accept structure args by value.)  The
729  new .cdb files can't be used by older versions of OpenMCL; using
730  older .cdb files with this version is "allowed, but not supported
731  or recommended."
732- Almost all of the changes in functionality since the last (061231)
733  snapshots and since the CVS freeze on 070117 have to do with
734  relatively obscure issues having to do with passing structures
735  to foreign functions by value and/or returning structures from foreign
736  function calls.
738  These idioms are fairly rare in traditional C code (though it's
739  fairly common to pass -pointers- to structures by reference
740  and sometimes to return pointers to structures.  There are
741  a few C compiler runtime routines that perform some flavor
742  of integer division and return a two-element structure that
743  contains "quotient" and "remainder" fields, but that's typically
744  about the extent of the use of this idiom.)  The idioms are used
745  much more often in Apple's Carbon and Cooca libraries and in
746  some of the frameworks (CoreGraphics, CoreFoundation) that those
747  libraries are based on.
749  OpenMCL's FFI has provided some support for this in the past;
750  notably, it's provided support for (most of the) structure-returning
751  and struct-by-value conventions used on 32-bit PPC Darwin.  In these
752  conventions, a foreign function that returned a structure received
753  a pointer to an instance of that structure type as a first argument,
754  and a function that received a structure argument by value received
755  the structure's contents in 32-bit word-size integer chunks (regardless
756  of the types or sizes of the structure's fields.)  Knowledge of these
757  conventions was hardwired into various parts of the system (e.g.,
758  the interface database), so that it was not generally possible to
759  tell whether a given foreign function returned a structure type
760  (or just happened to take an extra pointer argument.)
762  Unfortunately, there are at least 4 other sets of conventions for
763  dealing with structure arguments/return values on the platforms
764  that OpenMCL runs on (and even the DarwinPPC32 conventions weren't
765  fully/correctly implemented.)  OpenMCL's FFI is generally pretty
766  low-level, but to the extent that it's reasonable to talk about
767  "higher level" constructs (EXTERNAL-CALL, SEND, FF-CALL, #_), those
768  higher-level constructs try to enforce uniform syntax and try
769  to hide the platform-specific details in backend-specific functions.
771  The impact of these changes should generally be pretty minimal.
772  In a "higher-level" construct used to call a foreign function that
773  returns a structure type, the first parameter in the call should
774  be a pointer to an instance of that structure type.
776  For example, if a :rect structure is defined as:
778  (def-foreign-type nil
779    (:struct :rect
780      (:width :int)
781      (:height :int)
782      (:x :int)  ; x coordinate of origin
783      (:y :int)))
785  and a foreign function named "inset_rect" takes a rect and an integer
786  delta and returns a new :rect "inset" by that delta, a call to that
787  foreign function might look like:
789  (rlet ((result :rect))
790    (ff-call *address-of-inset-rect* result (:struct :rect) r :int delta :(:struct rect))
791    ;; or, if "inset_rect" was declared in the interface database:
792    (#_inset_rect result r delta))
795  A callback that returns a :rect likewise should accept a pointer
796  to an instance of the :rect type as a first (unqualified) argument
797  and explicitly declare that it returns a (:STRUCT :RECT).
799  (defcallback *address-of-inset-rect (result (:struct :rect) r :int delta (:struct :rect))
800    (setf (pref result :rect.x) (+ (pref r :rect.x) delta)
801          (pref result :rect.y) (+ (pref r :rect.y) delta)
802          (pref result :rect.width) (- (pref r :rect.width) (* 2 delta))
803          (pref result :rect.height) (- (pref r :rect.height) (* 2 delta))))
805  Note that this is very similar to what's been (implicitly) supported
806  on DarwinPPC32; the basic difference is that the return type
807  ("(:STRUCT :RECT)") is explicitly specified (or, in the case of #_,
808  specified in the interface database).  Whether the "result" pointer
809  is actually passed as an argument or not is platform-dependent (on
810  DarwinPPC64, the :rect structure would be "returned" by returning
811  4 :int values in 4 different machine registers), but the same syntax
812  can be used (and hides those details) on all platforms.
814  In the examples above, we said that the (presumed source) rectangle
815  was passed by value as a value of type (:struct :rect), and we let
816  the FFI deal with the details.  Historically, this parameter could
817  have been specified as a small unsigned integer N (denoting the
818  DarwinPPC32 convention of passing the structure value a N
819  native-word-size integer arguments.)  Again, there are several
820  different conventions for passing and receiving structure values,
821  and it's best to let the FFI decide how to follow those conventions.
822  (Some of those conventions are quite complicated, and depend on
823  the size of the structure as well as the types of its fields.)
825  In all cases, a callback which declares a parameter to be of a
826  structure type can treat that parameter as a pointer an instance of
827  that structure type with fields initialized by the caller (as in
828  the case of "r" in the example above.)
830  In the ObjC bridge, the DEFINE-OBJC-METHOD macro has always provided
831  syntax for specifiying that the method "returns" a structure. (That
832  syntax is (:struct <struct-type> <parameter-name>). That continues
833  to be supported.
835  Apple's ObjC runtime provides different functions (#_objc_msgSend and
836  #_objc_msgSend_stret) to handle the cases of sending messages which
837  return non-structure and structure results.  These low-level functions
838  are very sensitive to whether the structure is actually returned via
839  an "invisible" first argument or not (this is only one of a few different
840  conventions on some platforms.)  OpenMCL's ObjC bridge makes similar
841  distinctions, but uses simple, consistent rules: a message that returns
842  a structure should always be sent via SEND/STRET (or some variant of
843  SEND/STRET) and should have a first parameter of type "pointer to
844  returned structure type", regardless of whether or not that pointer
845  is actually passed to the method implementation or just used as by
846  some platform-specific code to transfer register values.)
848  The end result of all of this (several weeks of bootstrapping) is
849  that most things are pretty much the same, at least on DarwinPPC32;
850  only foreign function calls/callbacks that involve passing structures
851  by value or returning structures need change at all, and the changes
852  generally involve being more explicit/declarative about what's going
853  on.  These changes -do- allow these idioms to be used on other
854  (64-bit) platforms, and since they're heavily used in Apple GUI
855  libraries and since 64-bit versions of Carbon and Cocoa are announced
856  features of Leopard, it seemed appropriate to get support for this
857  stuff into the FFI on those platforms and to try to do it in a way
858  that hid the platform-dependent details.  (I didn't expect all of
859  this to take so long.)
861- The initial listener PROCESS now persists across SAVE-APPLICATION.
862  This means that (for instance):
864  ? (defvar *listener-process* (current-process))
866  ? (save-application "new.image")
867  shell> openmcl new.image
868  ? (eq (current-process) *listener-process*)
869  T
870  ;; though of course the underlying OS thread, stacks, etc are unlikely
871  ;; to be "equal" in any sense.
873  The current process is sometimes used to mark "ownership" of thread-private
874  hash-tables and streams.  (Even though it doesn't make much sense for
875  STREAMs to persist across SAVE-APPLICATION, it does make sense for
876  HASH-TABLEs to do so; HASH-TABLES created with the :PRIVATE T option
877  and "owned" by the initial listener process continue to be owned by
878  that the current listener process in the new image.)
880- All of the FFI changes above do seem to allow the Cocoa IDE example
881  to run on ppc64/x86-64 (as well as ppc32) under Leopard, and
882  hopefully that'll soon be true of applications generated via Mikel
883  Evins' Bosco system as well.  The bridge and demo code have been
884  conditionalized to support ObjC 2.0 on 64-bit systems, to avoid
885  deprecated functions and methods, and to support 64-bit Cocoa
886  changes.  Hopefully, this has been done in a way that doesn't break
887  PPC32 Cocoa under Tiger (he said, quickly rushing to the nearest
888  PPC32 Tiger machine and breathing a sigh of relief when the Cocoa
889  listener appeared ..)  64-bit Cocoa sometimes used 64-bit signed and
890  unsigned integers in place of 32-bit integers; accordingly, the
891  foreign types :<NSI>nteger and :<NSUI>nteger are defined (as 32-bit
892  signed/unsigned integers) on 32-bit platforms, and these types are
893  used in some method and type definitions.  (Those integer types are
894  predefined in Objc 2.0, and are 64 bits wide on 64-bit platforms.)
896  More pervasively (and a little more problematically), CoreGraphics
897  (and things built on top of it, including Cocoa) uses double-floats
898  instead of single-floats for many things on 64-bit hardware; the
899  difference is abstracted (a little) via the new CGFloat type.
900  This means that (for instance) code which initializes a constant-sized
901  NSRect on a 32-bit machines and has traditionally done so via
902  something like:
904  (ns-make-rect 0.0 0.0 500.0 200.0)
906  now needs to do something like:
908  (ns-make-rect (float 0.0 ccl::+cgfloat-zero+) ..)
910  in order to compile and run on both 32-bit and 64-bit platforms.
912  where ccl::+cgfloat-zero+ is defined as 1.0f0 on 32-bit platforms
913  and as 1.0d0 on 64-bit machines.  Cases involving constants won't
914  incur any runtime overhead and the occasional runtime overhead in
915  other cases -probably- isn't that great in context (compared to
916  initializing a view hierarchy ...)  but it's certainly ugly to
917  look at.  It's possible that some of this ugliness could be
918  hidden in the bridge/FFI (by making them do the necessary coercions
919  for you), but there are tradeoffs there.
921- The ObjC bridge has had a long-standing bug whereby a standalone
922  Cocoa application may have needed to find the interface databases
923  at runtime in order for MAKE-OBJC-INSTANCE and MAKE-INSTANCE of
924  an ObjC class to work.  (These functions needed to be able to
925  send an "init" message to the newly-allocated instance, and needed
926  to know the type signature of that init message in order to do that.)
927  The current scheme tries to avoid this by pre-compiling helper
928  functions to enable calling all known "init" message signatures.
929  (More accurately, all fixed-argument "init" message signatures.)
930  This scheme avoids the need to send messages whose argument
931  and result types are computed at runtime (via %SEND), and %SEND
932  (a) was known to be inefficient and (b) would have a lot of
933  difficulty handling all known structure return/passing conventions
934  on supported platforms.  Accordingly, %SEND has been deprecated
935  (with extreme prejudice, e.g., removed.)
937- a couple of little functions are defined (but their names are
938  not yet exported) on x86-64: ccl::rdtsc and ccl::rdtsc64 provide
939  access to the values returned by on-chip cycle counting instructions.
940  For instance:
942? (let* ((start (ccl::rdtsc)))
943    (sleep 1)
944    (- (ccl::rdtsc) start))
947  Hmm.  Apparently, the 2.0GHz MacBook I tried that on is actually
948  a 1.995GHz MacBook.
950  There are all kinds of ways for rdtsc to lose (and return
951  inaccurate or misleading results): the cycle counters for
952  each CPU core in a multi-core system aren't necessarily
953  kept in sync, and many modern systems allow CPU clock rates
954  to vary (for power-management reasons) and/or allow the CPU
955  to sleep/hibernate.  OSes seem to offer some support for
956  compensating for these effects, and it seems like ccl::rdtsc
957  and ccl::rdtsc64 can be used to obtain interesting results.
959  The RDTSC instruction actually returns an unsigned 64-bit
960  result; apparently, some Intel documentation claims that this
961  value will not "wrap around" to 0 at contemporary clock rates
962  for at least 10 years after the system was booted.  (If you can
963  keep an Intel system running for 9 years between reboots, you
964  might consider telling Intel that the RDTSC counter wrapped around
965  a year early; they might give you a refund.  Or maybe not.)
966  A non-negative OpenMCL64 fixnum is limited to 60 bits; the
967  ccl::rdtsc function truncates the 64-bit counter value so
968  that it fits in a non-negative fixnum; if the 10 year limit
969  for the 64-bit value is accurate, the 60-bit value would
970  wrap around after about 223 days of uptime.
972  ccl::rdtsc64 returns the full 64-bit counter value, but
973  may return a bignum after 223 days of uptime.
975- lots of bug fixes (not all of which involved the FFI or ObjC
976  bridge.) 
980openmcl 1.1-pre-061231
981- The FASL version changed (old FASL files won't work with this
982  lisp version), as did the version information which tries to
983  keep the kernel in sync with heap images.
984  The binary incompatibility has to do with how a few pages of
985  low memory in the lisp kernel's address space are mapped and
986  used.  OpenMCL was generally assuming that these pages were
987  otherwise unused and could be used to map a small static data
988  area from the heap image file; on some platforms, the dynamic
989  linker may have already allocated data in those unused pages
990  before the lisp kernel even starts to run.  Fixing this involved
991  changing the address of that small static data area slightly,
992  and this caused the addresses of some objects contained within
993  that static data area - notably NIL - to change, as well.
994- This snapshot is otherwise just a set of bug fixes/work-in-progress
995  changes.
996- Even though no supported filesystem actually supports versioned files,
997  OpenMCL now tries to retain PATHNAME-VERSION informaton for physical
998  pathnames.  (The fact that it didn't caused several ANSI test failures.)
999  This change introduced/exposed a few other bugs; I think that I've
1000  caught at least the most obvious ones, but it's possible that some
1001  new pathname-related bugs have been introduced.
1002- The cron job that runs on and updates the ChangeLog from
1003  CVS commit info stopped running as of a system upgrade in late November.
1004  The problem was fixed a couple of weeks ago, so it's once again meaningful
1005  to refer to the ChangeLog for details of bug fixes.
1006- FSQRT and FSQRTS instructions are "optional" on the PPC.  In practice,
1007  that often meant that they are implemented on chips made by IBM and
1008  not on chips made by Motorola/FreeScale.  This version of OpenMCL
1009  assumes that they're implemented and emulates them if they aren't.
1010- OSX 10.2 (Jaguar) and earlier versions are officially no longer
1011  supported.  (I honestly don't know if things have actually worked
1012  on Jaguar in a while, but some recent changes are known not to
1013  work on Jaguar and the kernel now inists on at least Panther on
1014  startup.
1015OpenMCL 1.1-pre-061205
1016- This release is intended to package up the bug fixes since
1017  the 061110 tarballs.  There aren't too many changes in
1018  functionality or any deep architectural changes since 061110,
1019  and it should be easy to bootstrap from current sources with
1020  061110 images.
1021  (It'd still be a good idea to recompile your code with
1022  up-to-date images, whether you download those images or
1023  build them yourself from CVS.)
1024- The one (barely) notable change in functionality has to do
1025  with how the lisp sets up pathname translations for the
1026  "ccl" logical host when the "CCL_DEFAULT_DIRECTORY" environment
1027  variable isn't set (e.g., when a shell script isn't used to
1028  invoke the lisp.)  Previous versions just used the current
1029  directory; this version tries to use the directory containing
1030  the current heap image.  The new scheme might get fooled by
1031  symbolic links (either following them or not following them
1032  could be wrong), but it's more likely to work for people
1033  who don't read or understand the discussion of the shell script
1034  in the documentation.
1035- All (knock wood) bugs that have been reported since the 061110
1036  images were released should be fixed.  Well, almost all.  The
1037  fixes include:
1039  - a typo (wrong register) in the "generic" version of the
1040    code which implements (SETF AREF) on 2-dimensional arrays
1041    on x86-64
1042  - incorrect bounds checking on vector references on x86-64,
1043    which caused some invalid indices to be treated as valid
1044    (usually leading to a segfault).  IIRC, the invalid indices
1045    that were erroneously accepted were fixnums whose absolute
1046    value was > (expt 2 56).  (More or less.).
1047  - Missing stream methods (especially involving string streams)
1048    affecting all platforms.
1049  - Several bugs involving GCD, some of which were specific to
1050    64-bit platforms and some of which affected all platforms.
1051    (These bugs sometimes affected results returned by #'/,
1052    LCM, and other funtions.)
1054  - OpenMCL has only ever supported an ELEMENT-TYPE argument of
1055   ([signed,unsigned]-byte 8|16|32|64) on binary file streams (with
1056   64-bit types supported only on 64-bit platforms.)  It has not
1057   previously tried to upgrade a supplied element-type to a supported
1058   one (it does now) and any errors that resulted from supplying an
1059   element-type that was not supported (and could not be upgraded) were
1060   either obscure side-effects or quiet misbehavior; an error (a
1061   SIMPLE-ERROR complaining about the unsupported element type) is now
1062   signaled as soon as attempts to upgrade to a supported element type
1063   fail.  I believe that the current behavior is both compliant and
1064   reasonable; it's probably better to discuss that issue on
1065   openmcl-devel than to do so here.
1068OpenMCL 1.1-pre-061110
1069- The FASL version changed (old FASL files won't work with this
1070  lisp version), as did the version information which tries to
1071  keep the kernel in sync with heap images.
1072- Several bug fixes (see ChangeLog), and modest-to-moderate
1073  performance improvements.  Notably, AREF and (SETF AREF)
1074  of 2- and 3-dimensional arrays are open-coded in more cases
1075  and are usually at least 5x faster than in previous versions.
1076  If the compiler knows that the array in question is a
1077  SIMPLE-ARRAY of appropiate dimensionality and knows the
1078  array's element-type, the speedup can be much greater.
1079  There are certainly opportunities for further improvements
1080  here, both in breadth (handling more cases) and depth
1081  (eliminating some type-and-bounds checking in safe code,
1082  doing parts of index calculations at compile-time when
1083  bounds and indices are constants ...), but things are
1084  generally improved.
1085- QUIT and SAVE-APPLICATION work a little differently; in
1086  particular, SAVE-APPLICATION sometimes runs after #_exit
1087  is called (via the #_atexit mechanism).
1088  The motivation for this change has to do with how some
1089  environments (Cocoa, to name one) conflate the ideas of
1090  "shutting down the GUI" with "exiting the application".
1091  Previous versions of OpenMCL tried to work around this
1092  by overriding some internal Cocoa methods; that approach
1093  was never particularly attractive and (predictably) it'll
1094  break in future OSX releases.
1095  The new scheme (which involves letting code run after #_exit
1096  has been called) certainly offers other ways to lose; so
1097  far, I haven't seen evidence of such lossage.
1098- For historical reasons (forgotten historical reasons, in fact)
1099  the PPC versions of OpenMCL run with floating-point underflow
1100  exceptions disabled (the x86-64 versions enable these exceptions
1101  by default.)  This should change soon (as soon as I remember
1102  to change it ...); it's unlikely that this will affect much
1103  user code, but it's possible that it'll do so.
1104OpenMCL 1.1-pre-061024
1105- The FASL version changed (old FASL files won't work with this
1106  lisp version), as did the version information which tries to
1107  keep the kernel in sync with heap images.
1108- Linux users: it's possible (depending on the distribution that
1109  you use) that the lisp kernel will claim to depend on newer
1110  versions of some shared libraries than the versions that you
1111  have installed.  This is mostly just an artifact of the GNU
1112  linker, which adds version information to dependent library
1113  references even though no strong dependency exists.  If you
1114  run into this, you should be able to simply cd to the appropriate
1115  build directory under ccl/lisp-kernel and do a "make".
1116- There's now a port of OpenMCL to FreeBSD/amd64; it claims to be
1117  of beta quality.  (The problems that made it too unstable
1118  to release as of a few months ago have been fixed;  I stil run
1119  into occasional FreeBSD-specific issues, and some such issues
1120  may remain.)
1121- The Darwin X8664 port is a bit more stable (no longer generates
1122  obscure "Trace/BKPT trap" exits or spurious-looking FP exceptions.)
1123  I'd never want to pass up a chance to speak ill of Mach, but both
1124  of these bugs seemed to be OpenMCL problems rather than Mach kernel
1125  problems, as I'd previously more-or-less assumed.
1126- I generally don't use SLIME with OpenMCL, but limited testing
1127  with the 2006-04-20 verson of SLIME seems to indicate that no
1128  changes to SLIME are necessary to work with this version.
1129- CHAR-CODE-LIMIT is now #x110000, which means that all Unicode
1130  characters can be directly represented.  There is one CHARACTER
1131  type (all CHARACTERs are BASE-CHARs) and one string type (all
1132  STRINGs are BASE-STRINGs.)  This change (and some other changes
1133  in the compiler and runtime) made the heap images a few MB larger
1134  than in previous versions.
1135- As of Unicode 5.0, only about 100,000 of 1114112./#x110000 CHAR-CODEs
1136  are actually defined; the function CODE-CHAR knows that certain
1137  ranges of code values (notably #xd800-#xddff) will never be valid
1138  character codes and will return NIL for arguments in that range,
1139  but may return a non-NIL value (an undefined/non-standard CHARACTER
1140  object) for other unassigned code values.
1141- The :EXTERNAL-FORMAT argument to OPEN/LOAD/COMPILE-FILE has been
1142  extended to allow the stream's character encoding scheme (as well
1143  as line-termination conventions) to be specified; see more
1144  details below.  MAKE-SOCKET has been extended to allow an
1145  :EXTERNAL-FORMAT argument with similar semantics.
1146- Strings of the form "u+xxxx" - where "x" is a sequence of one
1147  or more hex digits- can be used as as character names to denote
1148  the character whose code is the value of the string of hex digits.
1149  (The +  character is actually optional, so  #\u+0020, #\U0020, and
1150  #\U+20 all refer to the #\Space character.)  Characters with codes
1151  in the range #xa0-#x7ff (IIRC) also have symbolic names (the
1152  names from the Unicode standard with spaces replaced with underscores),
1153  so #\Greek_Capital_Letter_Epsilon can be used to refer to the character
1154  whose CHAR-CODE is #x395.
1155- The line-termination convention popularized with the CP/M operating
1156  system (and used in its descendants) - e.g., CRLF - is now supported,
1157  as is the use of Unicode #\Line_Separator (#\u+2028).
1158- About 15-20 character encoding schemes are defined (so far); these
1159  include UTF-8/16/32 and the big-endian/little-endian variants of
1160  the latter two and ISO-8859-* 8-bit encodings.  (There is not
1161  yet any support for traditional (non-Unicode) ways of externally
1162  encoding characters used in Asian languages, support for legacy
1163  MacOS encodings, legacy Windows/DOS/IBM encodings, ...)  It's hoped
1164  that the existing infrastructure will handle most (if not all) of
1165  what's missing; that may not be the case for "stateful" encodings
1166  (where the way that a given character is encoded/decoded depend
1167  on context, like the value of the preceding/following character.)
1168- There isn't yet any support for Unicode-aware collation (CHAR>
1169  and related CL functions just compare character codes, which
1170  can give meaningless results for non-STANDARD-CHARs), case-inversion,
1171  or normalization/denormalization.  There's generally good support
1172  for this sort of thing in OS-provided libraries (e.g., CoreFoundation
1173  on MacOSX), and it's not yet clear whether it'd be best to duplicate
1174  that in lisp or leverage library support.
1175- Unicode-aware FFI functions and macros are still in a sort of
1176  embryonic state if they're there at all; things like WITH-CSTRs
1177  continue to exist (and continue to assume an 8-bit character
1178  encoding.)
1179- Characters that can't be represented in a fixed-width 8-bit
1180  character encoding are replaced with #\Sub (= (code-char 26) =
1181  ^Z) on output, so if you do something like:
1183? (format t "~a" #\u+20a0)
1185  you might see a #\Sub character (however that's displayed on
1186  the terminal device/Emacs buffer) or a Euro currency sign or
1187  practically anything else (depending on how lisp is configured
1188  to encode output to *TERMINAL-IO* and on how the terminal/Emacs
1189  is configured to decode its input.
1191  On output to streams with character encodings that can encode
1192  the full range of Unicode - and on input from any stream -
1193  "unencodable characters" are represented using the Unicode
1194  #\Replacement_Character (= #\U+fffd); the presence of such a
1195  character usually indicates that something got lost in translation
1196  (data wasn't encoded properly or there was a bug in the decoding
1197  process.)
1198- Streams encoded in schemes which use more than one octet per code unit
1199  (UTF-16, UTF-32, ...) and whose endianness is not explicit will be
1200  written with a leading byte-order-mark character on (new) output and
1201  will expect a BOM on input; if a BOM is missing from input data,
1202  that data will be assumed to have been serialized in big-endian order.
1203  Streams encoded in variants of these schemes whose endianness is
1204  explicit (UTF-16BE, UCS-4LE, ...) will not have byte-order-marks written
1205  on output or expected on input.  (UTF-8 streams might also contain
1206  encoded byte-order-marks; even though UTF-8 uses a single octet per
1207  code unit - and possibly more than one code unit per character - this
1208  convention is sometimes used to advertise that the stream is UTF-8-
1209  encoded.  The current implementation doesn't skip over/ignore leading
1210  BOMs on UTF8-encoded input, but it probably should.)
1212  If the preceding paragraph made little sense, a shorter version is
1213  that sometimes the endianness of encoded data matters and there
1214  are conventions for expressing the endianness of encoded data; I
1215  think that OpenMCL gets it mostly right, but (even if that's true)
1216  the real world may be messier.
1217- By default, OpenMCL uses ISO-8859-1 encoding for *TERMINAL-IO*
1218  and for all streams whose EXTERNAL-FORMAT isn't explicitly specified.
1219  (ISO-8859-1 just covers the first 256 Unicode code points, where
1220  the first 128 code points are equivalent to US-ASCII.)  That should
1221  be pretty much equivalent to what previous versions (that only
1222  supported 8-bit characters) did, but it may not be optimal for
1223  users working in a particular locale.  The default for *TERMINAL-IO*
1224  can be set via a command-line argument (see below) and this setting
1225  persists across calls to SAVE-APPLICATION, but it's not clear that
1226  there's a good way of setting it automatically (e.g., by checking
1227  the POSIX "locale" settings on startup.)  Thing like POSIX locales
1228  aren't always set correctly (even if they're set correctly for
1229  the shell/terminal, they may not be set correctly when running
1230  under Emacs ...) and in general, *TERMINAL-IO*'s notion of the
1231  character encoding it's using and the "terminal device"/Emacs subprocess's
1232  notion need to agree (and fonts need to contain glyphs for the
1233  right set of characters) in order for everything to "work".  Using
1234  ISO-8859-1 as the default seemed to increase the likelyhood that
1235  most things would work even if things aren't quite set up ideally
1236  (since no character translation occurs for 8-bit characters in
1237  ISO-8859-1.)
1238- In non-Unicode-related news: the rewrite of OpenMCL's stream code
1239  that was started a few months ago should now be complete (no more
1240  "missing method for BASIC-STREAM" errors, or at least there shouldn't
1241  be any.)
1242- I haven't done anything with the Cocoa bridge/demos lately, besides
1243  a little bit of smoke-testing.
1245Some implementation/usage details:
1247Character encodings.
1249CHARACTER-ENCODINGs are objects (structures) that're named by keywords
1250(:ISO-8859-1, :UTF-8, etc.).  The structures contain attributes of
1251the encoding and functions used to encode/decode external data, but
1252unless you're trying to define or debug an encoding there's little
1253reason to know much about the CHARACTER-ENCODING objects and it's
1254generally desirable (and sometimes necessary) to refer to the encoding
1255via its name.
1257Most encodings have "aliases"; the encoding named :ISO-8859-1 can
1258also be referred to by the names :LATIN1 and :IBM819, among others.
1259Where possible, the keywordized name of an encoding is equivalent
1260to the preferred MIME charset name (and the aliases are all registered
1261IANA charset names.)
1263NIL is an alias for the :ISO-8859-1 encoding; it's treated a little
1264specially by the I/O system.
1266The function CCL:DESCRIBE-CHARACTER-ENCODINGS will write descriptions
1267of all defined character encodings to *terminal-io*; these descriptions
1268include the names of the encoding's aliases and a doc string which
1269briefly describes each encoding's properties and intended use.
1271Line-termination conventions.
1273As noted in the <=1.0 documentation, the keywords :UNIX, :MACOS, and
1274:INFERRED can be used to denote a stream's line-termination conventions.
1275(:INFERRED is only useful for FILE-STREAMs that're open for :INPUT or
1276:IO.)  In this release, the keyword :CR can also be used to indicate
1277that a stream uses #\Return characters for line-termination (equivalent
1278to :MACOS), the keyword :UNICODE denotes that the stream uses Unicode
1279#\Line_Separator characters to terminate lines, and the keywords :CRLF,
1280:CP/M, :MSDOS, :DOS, and :WINDOWS all indicate that lines are terminated
1281via a #\Return #\Linefeed sequence.
1283In some contexts (when specifying EXTERNAL-FORMATs), the keyword :DEFAULT
1284can also be used; in this case, it's equivalent to specifying the value
1285of the variable CCL:*DEFAULT-LINE-TERMINATION*.  The initial value of
1286this variable is :UNIX.
1288Note that the set of keywords used to denote CHARACTER-ENCODINGs and
1289the set of keywords used to denote line-termination conventions is
1290disjoint: a keyword denotes at most a character encoding or a line
1291termination convention, but never both.
1295EXTERNAL-FORMATs are also objects (structures) with two read-only
1296fields that can be accessed via the functions EXTERNAL-FORMAT-LINE-TERMINATION
1297and EXTERNAL-FORMAT-CHARACTER-ENCODING; the values of these fields are
1298line-termination-convention-names and character-encoding names as described
1301An EXTERNAL-FORMAT object via the function MAKE-EXTERNAL-FORMAT:
1303MAKE-EXTERNAL-FORMAT &key domain character-encoding line-termination
1305(Despite the function's name, it doesn't necessarily create a new,
1306unique EXTERNAL-FORMAT object: two calls to MAKE-EXTERNAL-FORMAT
1307with the same arguments made in the same dynamic environment will
1308return the same (eq) object.)
1310Both the :LINE-TERMINATION and :CHARACTER-ENCODING arguments default
1311to :DEFAULT; if :LINE-TERMINATION is specified as or defaults to
1312:DEFAULT, the value of CCL:*DEFAULT-LINE-TERMINATION* is used to
1313provide a concrete value.
1315When the :CHARACTER-ENCODING argument is specifed as/defaults to
1316:DEFAULT, the concrete character encoding name that's actually used
1317depends on the value of the :DOMAIN argument to MAKE-EXTERNAL-FORMAT.
1318The :DOMAIN-ARGUMENT's value can be practically anything; when it's
1319the keyword :FILE and the :CHARACTER-ENCODING argument's value is
1320:DEFAULT, the concrete character encoding name that's used will be
1321the value of the variable CCL:*DEFAULT-FILE-CHARACTER-ENCODING*; the
1322initial value of this variable is NIL (which is an alias for :ISO-8859-1).
1323If the value of the :DOMAIN argument is :SOCKET and the :CHARACTER-ENCODING
1324argument's value is :DEFAULT, the value of
1325CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING* is used as a concrete character
1326encoding name.  The initial value of CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING*
1327is NIL, again denoting the :ISO-8859-1 encoding.
1328If the value of the :DOMAIN argument is anything else, :ISO-8859-1 is
1329also used (but there's no way to override this.) 
1331The result of a call to MAKE-EXTERNAL-FORMAT can be used as the value
1332of the :EXTERNAL-FORMAT argument to OPEN, LOAD, COMPILE-FILE, and
1333MAKE-SOCKET; it's also possible to use a few shorthand constructs
1334in these contexts:
1336* if ARG is unspecified or specified as :DEFAULT, the value of the
1337  variable CCL:*DEFAULT-EXTERNAL-FORMAT* is used.  Since the value
1338  of this variable has historically been used to name a default
1339  line-termination convention, this case effectively falls into
1340  the next one:
1341* if ARG is a keyword which names a concrete line-termination convention,
1342  an EXTERNAL-FORMAT equivalent to the result of calling
1343  (MAKE-EXTERNAL-FORMAT :line-termination ARG)
1344   will be used
1345* if ARG is a keyword which names a character encoding, an EXTERNAL-FORMAT
1346  equvalent to the result of calling
1347  (MAKE-EXTERNAL-FORMAT :character-encoding ARG)
1348  will be used
1349* if ARG is a list, the result of (APPLY #'MAKE-EXTERNAL-FORMAT ARG)
1350  will be used
1352(When MAKE-EXTERNAL-FORMAT is called to create an EXTERNAL-FORMAT
1353object from one of these shorthand designators, the value of the
1354:DOMAIN keyword argument is :FILE for OPEN,LOAD, and COMPILE-FILE
1355and :SOCKET for MAKE-SOCKET.)
1358The CL function STREAM-EXTERNAL-FORMAT - which is portably defined
1359on FILE-STREAMs - can be applied to any open stream in this release
1360and will return an EXTERNAL-FORMAT object when applied to an open
1362SETF can be used with STREAM-EXTERNAL-FORMAT to change the stream's
1363character encoding, line-termination, or both.
1365If a "shorthand" external-format designator is used in a call to
1366(SETF STREAM-EXTERNAL-FORMAT), the "domain" used to construct an
1367EXTERNAL-FORMAT is derived from the class of the stream in the
1368obvious way (:FILE for FILE-STREAMs, :SOCKET for ... well, for
1369sockets ...)
1371Note that the effect or doing something like:
1373(let* ((s (open "foo" ... :external-format :utf-8)))
1374  ...
1375  (unread-char ch s)
1376  (eetf (stream-external-format s) :us-ascii)
1377  (read-char s))
1379might or might not be what was intended.  The current behavior is
1380that the call to READ-CHAR will return the previously unread character
1381CH, which might surprise any code which assumes that the READ-CHAR
1382will return something encodable in 7 or 8 bits.  Since functions
1383like READ may call UNREAD-CHAR "behind your back", it may or may
1384not be obvious that this has even occurred; the best approach to
1385dealing with this issue might be to avoid using READ or explicit
1386calls to UNREAD-CHAR when processing content encoded in multiple
1387external formats.
1389There's a similar issue with "bivalent" streams (sockets) which
1390can do both character and binary I/O with an :ELEMENT-TYPE of
1391(UNSIGNED-BYTE 8).  Historically, the sequence:
1393   (unread-char ch s)
1394   (read-byte s)
1396caused the READ-BYTE to return (CHAR-CODE CH); that made sense
1397when everything was implicitly encoded as :ISO-8859-1, but may not
1398make any sense anymore.  (The only thing that seems to make sense
1399in that case is to clear the unread character and read the next
1400octet; that's implemented in some cases but I don't think that
1401things are always handled consistently.)
1403Command-line argument for specifying the character encoding to
1404be used for *TERMINAL-IO*.
1406Shortly after a saved lisp image starts up, it creates the standard
1407CL streams (like *STANDARD-OUTPUT*, *TERMINAL-IO*, *QUERY-IO*, etc.);
1408most of these streams are usually SYNONYM-STREAMS which reference
1409the TWO-WAY-STREAM *TERMINAL-IO*, which is itself comprised of
1410a pair of CHARACTER-STREAMs.  The character encoding used for
1411any CHARACTER-STREAMs created during this process is the one
1412named by the value of the variable CCL:*TERMINAL-CHARACTER-ENCODING-NAME*;
1413this value is initially NIL.
1415The -K or --terminal-encoding command-line argument can be used to
1416set the value of this variable (the argument is processed before the
1417standard streams are created.)  The string which is the value of
1418the -K/--terminal-encoding argument is uppercased and interned in
1419the KEYWORD package; if an encoding named by that keyword exists,
1420CCL:*TERMINAL-CHARACTER-ENCODING-NAME* is set to the name of that
1421encoding.  For example:
1423shell> openmcl -K utf-8
1425will have the effect of making the standard CL streams use :UTF-8
1426as their character encoding.
1428(It's probably possible - but a bit awkward - to use (SETF EXTERNAL-FORMAT)
1429from one's init file or --eval arguments or similar to change existing
1430streams' character encodings; the hard/awkward parts of doing so include
1431the difficulty of determining which standard streams are "real" character
1432streams and which are aliases/composite streams.)
1434OpenMCL 1.1-pre-069826
1435- There's an (alpha-quality, maybe) port to x86-64 Darwin (e.g., the
1436  Mac Pro.)  Some known problems include:
1438  * infrequently (but not infrequently enough) the lisp dies on
1439    startup with a spurious "Trace/BKPT trap" error message.  This
1440    seems to be timing-dependent and (very generally) seems to
1441    involve the Mach exception thread not recognizing an exception
1442    used to effect exception return.  Sometimes, this shows up
1443    as a (:SIGNALED 5) error when REBUILD-CCL runs the lisp to
1444    create a new image.
1446  * some math library primitives (#_asin, for one) generate
1447    spurious incidental FP exceptions that have nothing to
1448    do with the validity of the arguments or result.  To work around
1449    this, the lisp ignores FP exceptions which might have occurred
1450    during a call into the math library; that means that it doesn't
1451    detect -real- FP exceptions when they're signaled.  (This bug
1452    only affects things that call into the system math library;
1453    lisp arithmetic operations that're done inline are not affected.)
1455  * The version of OSX/Darwin that shipped with the Mac Pro is missing
1456    some functionality that from OpenMCL's point of view is highly
1457    desirable (namely, the ability to keep application-level thread-
1458    specific data in a per-thread block of memory addressed by an
1459    otherwise unused segment register.)  To get things working (as
1460    well as they are), the lisp "shares" the segment register that
1461    the pthreads library uses to access thread data.  This scheme
1462    isn't intended to be long-lived (and negatively affects
1463    performance of things like foreign-function calls, callbacks,
1464    and exception handling).
1466  * The .cdb files (libc only for Tiger) in ccl:darwin-x86-headers64;
1467    were cross-developed on a Linux x86-64 system, since Apple
1468    has not yet released the sources to their x86-64 enabled gcc.
1470- On all platforms, stream code has been rewritten and often offers
1471  better (sometimes substantially better) performance.  OPEN and
1472  MAKE-SOCKET have each been extended to take additional keyword
1473  arguments.
1475  :SHARING, which can have the values :PRIVATE (the default), :LOCK,
1476  or :EXTERNAL (NIL is also accepted as synonym for :EXTERNAL)
1478   :PRIVATE specifies that the stream can only be accessed by
1479   the thread that created it.  (There was some discussion on openmcl-devel
1480   about the idea of "transferring ownership" of a stream; this has
1481   not yet been implemented.)  Attempts to do I/O on a stream with
1482   :PRIVATE sharing from a thread other than the stream's owner yield
1483   an error.
1485   :LOCK specifies that all access to the stream require the calling
1486   thread to obtain a lock; there are separate "read" and "write"
1487   locks for IO streams (so it's possible for one thread to read
1488   from such a stream while another thread writes to it, for instance.)
1489   :LOCK was the implicit default for all streams prior to this change.
1490   (See below - under the discussion of the AUTO-FLUSH mechanism -
1491   for a discussion of one of the implications of this change that
1492   affects SLIME users.)
1494   :EXTERNAL (or NIL) specifies that I/O primitives enforce no
1495   access protocol.  This may be appropriate for some types of application
1496   which can control stream access via application-level protocols.  Note
1497   that since even the act of reading from a stream changes its internal
1498   state (and simultaneous access from multiple threads can therefore
1499   lead to corruption of that state), some care must be taken in the
1500   design of such protocols.
1502  The :BASIC keyword argument influences whether or not the stream
1503  will be an instance of the class FUNDAMENTAL-STREAM (the superclass
1504  from which all Gray stream classes inherit) or a subclass of the
1505  built-in class CCL::BASIC-STREAM.  The default value of :BASIC
1506  is T and this has effect for FILE-STREAMs created via OPEN;
1507  SOCKETs are still always implemented as FUNDAMENTAL (Gray) streams,
1508  though this should change soon.
1510   The tradeoff between FUNDAMENTAL and BASIC streams is entirely
1511   between flexibility and (potential or actual) performance.  I/O
1512   primitives can recognize BASIC-STREAMs and exploit knowledge of
1513   implementation details; FUNDAMENTAL stream classes can be
1514   subclassed in a semi-standard way (the Gray streams protocol.)
1516   For existing stream classes (FILE-STREAMs, SOCKETs, and the
1517   internal CCL::FD-STREAM classes used to implement file streams
1518   and sockets), a lot of code can be shared between the
1519   FUNDAMENTAL and BASIC implementations.  The biggest difference
1520   should be that that code can be reached from I/O primitives
1521   like READ-CHAR without going through some steps that're there
1522   to support generality and extensibility, and skipping those
1523   steps when that support isn't needed can improve I/O performance.
1525   Gray stream methods (STREAM-READ-CHAR) should work on
1526   appropriate BASIC-STREAMs.  (There may still be cases where
1527   such methods are undefined; such cases should be considered
1528   bugs.)  It is not guaranteed that Gray stream methods would
1529   ever be called by I/O primitives to read a character from
1530   a BASIC-STREAM (though there are still cases where this happens.)
1532   A simple loop reading 2M characters from a text file runs about
1533   10X faster when the file is opened the new defaults (:SHARING :PRIVATE
1534   :BASIC T) than it had before these changes were made.  That sounds
1535   good, until one realizes that the "equivalent" C loop can be about
1536   10X faster still ...
1538 - Forcing output to interactive streams.
1540   OpenMCL has long had a (mostly undocumented) mechanism whereby
1541   a mostly idle thread wakes up a few (~3) times per second and
1542   calls FORCE-OUTPUT on specified OUTPUT-STREAMS; this helps to
1543   ensure that streams with which a user would be expected to
1544   interact (the output side of *TERMINAL-IO*, listener windows
1545   in a GUI, etc.) have all buffered output flushed without
1546   requiring application or I/O library code to be concerned about
1547   that.
1549   The SLIME lisp interaction mode for Emacs uses this mechanism,
1550   but the changes described above interfere with SLIMEs use of
1551   it:  in order to be safely accessed from multiple threads (the
1552   SLIME REPL thread and the thread which does the background
1553   periodic flushing of buffered output), a stream must have
1554   been created with :SHARING :LOCK in effect.  This is no longer
1555   the effective default; the code which does the periodic
1556   output flushing ignores streams which do not use locks as an
1557   access/sharing mechanism.  THIS MEANS THAT BUFFERRED OUTPUT
1559   A small change to SLIME's "swank-openmcl.lisp" is required
1560   to restore this functionality.  First,  a brief description of
1561   a couple of new primitives:
1565    Adds "s", which should be a "simple" OUTPUT-STREAM as returned
1566    by OPEN or MAKE-SOCKET, to a list of streams whose buffered
1567    output should be periodically flushed.  If S was not created
1568    with :SHARING :LOCK in effect, the stream will have its
1569    :SHARING mode changed to put :SHARING :LOCK into effect.
1573    Removes S from the internal list of automatically flushed
1574    streams.  Does not restore the stream's :SHARING mode, which
1575    may have been changed by a previous call to ADD-AUTO-FLUSH-STREAM.
1577 - SLIME changes
1578   In slime:swank-openmcl.lisp, around line 182, the method
1580(defmethod make-stream-interactive ((stream ccl:fundamental-output-stream))
1581  (push stream ccl::*auto-flush-streams*))
1583   should be changed to use CCL:ADD-AUTOFLUSH-STREAM if it's defined:
1585(defmethod make-stream-interactive ((stream ccl:fundamental-output-stream))
1586  (if (fboundp 'ccl::add-auto-flush-stream)
1587    (ccl::add-auto-flush-stream stream)
1588    (push stream ccl::*auto-flush-streams*)))
1590   That's adequate for the moment, since sockets are still
1591   FUNDAMENTAL-STREAMs.  When that changes, some more extensive changes
1592   to swank-openmcl.lisp may become necessary.
1594- on x86-64, floating-point-underflow exceptions are now enabled
1595  by default.  (They really should be on ppc as well.)  Again,
1596  this affects FP operations that are done in lisp code and
1597  the results of FP operations that are reported in response
1598  to calls to reasonable (non-Darwin) math libraries.  This
1599  can affect whether or not some "potential number"  reader
1600  tokens are representable as numbers, e.g., whether or not
1601  attempts to read something like "1.0f-50" signal underflow
1602  or are quietly mapped to 0.0f0.
1604- examples: Phil (from the mailing list) has added code which
1605  supports some of the ffi examples from the documentation.
1607- Bug fixes: see ChangeLog
1611OpenMCL 1.1-pre-060705
1612- Bug fixes again.  Some internal changes to support a FreeBSD/AMD64
1613  port that's not quite ready.
1615- :MCL is back on *features*; there seem to be too many packages out
1616  there that expect it to be, and there hasn't been enough advance
1617  notice of its pending removal.
1619OpenMCL 1.1-pre-060623
1620- Mostly bug fixes (a CLOS bug that prevented the ObjC bridge from
1621  working, FIXNUM arrays weren't quite finished on PPC)
1623- Use Tiger inferfaces (from XCode 10.4u SDK) on DarwinPPC32
1625- Add gl, gtk2, gnome2 interfaces for x86-64.  Add a tiny
1626  "gtk2-clock" example, tweak the opengl-ffi (GLUT) example
1627  so that it works on x86-64.
1629- Some changes to the ObjC bridge to support loading additional
1630  frameworks; update the WebKit example to use these new features.
1632- Still an outstanding issue where things like MAKE-OBJC-INSTANCE
1633  need access to the interfaces at runtime (and can crash if they
1634  aren't available.)
1636- Build snapshots for LinuxPPC{32,64}.
1638OpenMCL 1.1-pre-060608
1639- The FASL version changed, as did the version number which pairs
1640  the lisp kernel with heap images.  Images saved with older kernels
1641  can't be loaded on this one; the images/kernels in the 060608
1642  snapshot tarballs should match.
1644  Most of the ABI changes that caused these version changes were
1645  x86-64 specific; some auxiliary stack pointers that had been
1646  kept in MMX registers are now kept in per-thread memory. (Signal/
1647  exception handlers generally need to be able to access these
1648  stack pointers, but at least some versions of the Linux kernel
1649  don't reliably pass correct values of the MMX registers in the
1650  signal contexts passed to signal handlers.  Moral: some kinds
1651  of stack-allocation and foreign-function operations may (or may not)
1652  be a few cycles slower, but OpenMCL should be a bit less prone
1653  to fatal segfault exceptions.)
1655  Other than that, most changes since the 060530 snapshots are
1656  bugfixes (see the ChangeLog for details).  The x86-64 port has been
1657  exercised fairly heavily (if somewhat narrowly) and its welcome
1658  banner now claims that it's a beta release.  I think that that's
1659  probably fair, and hope that anyone who may have been reluctant to
1660  test an alpha release will agree and be less reluctant.
1662- There's still much more to be done, but some preliminary 1.1 documentation
1663  is now online at:
1667  Note that some relative links on "newsite" may be invalid, but the
1668  internal links in the Doc directory should work.
1670  As noted above, it still needs a lot of work; feedback, criticism,
1671  and help would all be appreciated.
1673OpenMCL 1.1-pre-060530
1675- These release notes have gotten woefully out of date.
1677- OpenMCL now runs on x86-64 (AMD64, Intel EM64T) systems under Linux.
1678  It announces itself as an alpha release in the Welcome banner; it should
1679  in fact be very nearly feature-complete (but possibly still buggy.)
1680  There's a chicken-and-egg issue in that it needs more testing before
1681  it can be formally released and some people may be waiting for a more
1682  stable version.
1684  The build process and most user-visible things should behave the same
1685  way as on PPC; using REBUILD-CCL (described below) is generally the
1686  simplest way to rebuild from sources.  A few (intentional) differences:
1688  * the lisp kernel is named "lx86cl64", the default heap image is
1689    named "LX86CL64" (e.g., the kernel name, case-inverted) and the
1690    bootstrapping image is conventionally named "x86-boot64".
1692  * FASL files have the extension "lx64fsl"
1694  * the kernel build directory is "ccl/lisp-kernel/linuxx8664"
1696  * the "openmcl64" shell script can be used to invoke the
1697    lisp, as on 64-bit PPC platforms.
1699Other changes tend to be a little more modest:
1701- there is now a specialized FIXNUM array element type on all platforms.
1702  (distinct from T or (SIGNED-BYTE <machine-word-size>)).  Access to
1703  such vectors is a little cheaper than the SIGNED-BYTE case (since
1704  elements are known to be fixnums) and a little easier on the GC
1705  than the T case (the GC can avoid looking at their contents and
1706  there are no associated EGC write-barrier issues.)
1708- "colon" commands entered into the REPL/break loops don't need to
1709  be parenthesized if the command and all operands are on the same
1710  line. E.g.
17121> :f 0
1714  and
17161> (:f 0)
1718  are equivalent (and have the effect of examining the raw contents of
1719  the 0th stack frame)
1721- the syntax of the :B (backtrace) break-loop has changed; rather
1722  than taking an optional argument which specifies whether or not
1723  frame details should be shown, it now accepts keyword arguments
1724  for specifying:
1726  :start        ; unsigned integer: the index of the first frame to show
1727  :count        ; unsigned integer: the maximum number of frames to show
1728  :detailed-p   ; boolean: whether or not to show frame detail
1730- a new break-loop command :NFRAMES returns the number of stack frames
1731  accessible to backtrace.  (Both this change and the previous
1732  are intended to help deal with deep recursion/stack overflow cases.)
1734- any command-line arguments that follow a "--" pseudo-argument
1735  are not processed by the lisp startup code and are stored
1736  (as a list of strings) in CCL:*UNPROCESSED-COMMAND-LINE-ARGUMENTS*.
1737  E.g.:
1739shell> openmcl -- -foo 17
1742=> ("-foo" "17")
1744OpenMCL 1.1-pre-060226
1746- The --thread-stack-size (or -Z)  command-line argument changes the values
1747  of the variables used to determine the sizes of the listener thread.
1748  The values of these variables will persist accross SAVE-APPLICATION;
1749  these values have no effect on the sizes of stacks in threads created
1750  under explicit user control.
1752- New functions:
1754  (CCL:GC-VERBOSE on-full-gc &optional (on-egc on-full-gc))
1756  Causes the GC to print (or stop printing ...) informational messages
1757  on entry and exit.  The ON-FULL-GC argument controls whether or
1758  not these messages are printed on ... a full GC, and the ON-EGC
1759  argument (which defaults to the value of the ON-FULL-GC argument)
1760  controls whether messages are printed on ephemeral GCs.
1764  Returns two values (corresponding to the arguments of the last call
1765  to CCL:GC-VERBOSE.)
1768                         :RELOAD-ARGUMENTS)
1770  Depending on the values of its arguments, recompiles lisp and/or
1771  kernel sources and optionallly re-generates ("reloads") a heap
1772  image.
1774  Arguments:
1776  clean   deletes FASL and .o files before performing other steps
1777  kernel  rebuilds the lisp kernel
1778  force   forces recompilation, even if binary is newer than source
1779  reload  tries to rebuild a full heap image after other build steps
1780  exit    quits after all other steps
1781  full    equivalent to :CLEAN T :KERNEL T :RELOAD T
1782  reload-arguments a list of strings, passed as additional arguments
1783                   to the reload step.  E.g. '("--thread-stack-size" "128M").
1785  Output from the :KERNEL and :RELOAD steps is ordinarily only displayed
1786  if an error occurs.
1789- Changes
1791  TRACE now prints an integer (corresponding to the level of indentation)
1792  on each line of output.
1794  Tracing callbacks is currently broken (it may be reimplemented; if so,
1795  it'd be implemented somewhat differently ...)
1797- Bugs
1799  Several bugs involving interactions between the GC and (many) active
1800  threads have been fixed; at least one such bug remains (the symptom
1801  involves a recently allocated  array somehow getting trashed or GCed
1802  incorrectly; the  cause has been under investigation for weeks but is
1803  still not known.)
1805OpenMCL 1.1-pre-060125
1807- FASL version changed; delete old FASL (.dfsl, .pfsl, .dfsl64, .pfsl64) files
1809- "kernel ABI version" changed; build a new lisp kernel before trying to load/use
1810   060125 images.
1812-  Changes: (see also ChangeLog)
1814   New variable:
1818   Initialized on application startup to contain the MMU/OS page size in bytes.
1819   This is 4K on PPC platforms (and likely on most? all? x86 platforms).
1821   New functions:
1825   Returns an integer, the value of which is used by the lisp kernel when
1826   mapping heap memory from the OS.  Mapping requests are usually made in
1827   multiples of this value. 
1829   This value is read-only; currently, it's 64KB on 32-bit platforms and
1830   128KB on 64-bit platforms.
1835   Returns the (per-thread) allocation quantum of the process P.  By default,
1836   this is the same value as that returned by CCL:DEFAULT-ALLOCATION-QUANTUM,
1837   but lower values can be specified on a per-process basis (see below.)
1839   This value is read-only.
1844   but can be used with SETF to change the current processes's
1845   allocation quantum to a value which is between *HOST-PAGE-SIZE* and
1846   (DEFAULT-ALLOCATION-QUANTUM), inclusive, and which is a power of 2.
1849   Changes to existing functions:
1852   &key argument, which defaults to the value returned by (DEFAULT-ALLOCATION-QUANTUM).
1853   If provided, the value of the argument should should satisfy the same
1854   constraints that (SETF (CURRENT-PROCESS-ALLOCATION-QUANTUM) is subject to.
1858In general, larger per-thread allocation quanta are appropriate for programs
1859where a relatively small number of threads need to allocate memory frequently
1860and small per-thread quanta are appropriate for larger numbers of threads
1861that are expected to do small, infrequent memory allocations.
1863The worst-case scenarios would involve a large number of threads doing
1864incidental memory allocation with large quanta (that wastes memory and may
1865trigger the GC too frequently) or a small number of threads doing frequent
1866memory allocation with small quanta (since such threads could be expected
1867to fill up their small per-thread memory allocations quickly and frequently
1868and would waste time frequently allocating more small chunks.)
1870All of these values interact with the GC and EGC thresholds; the ability
1871to exercise some control over how much per-threads memory is allocated
1872at a time can help to ensure that those interactions are appropriate.
1873When these mechanisms are insufficient, applications should consider the
1874use of available mechanisms for adjusting GC and EGC thresholds.
1879OpenMCL 1.1-pre-051027
1881- A lot of internal changes in the way that special bindings, UNWIND-PROTECT,
1882  and WITHOUT-INTERRUPTS are implemented (and in how they interact with
1883  each other.
1885  One user-visible aspect of this is that UNWIND-PROTECT cleanup forms
1886  are run with interrupts disabled (the protected form is run with
1887  interrupts enabled if they were enabled on entry to the UNWIND-PROTECT.)
1888  This means that something like:
1890  (unwind-protect
1891      nil
1892    (loop))
1894  will loop uninterruptibly.
1896- CCL:WITH-INTERRUPTS-ENABLED &body body executes the body with interrupts
1897  enabled.  The example above could be rewritten as:
1899  (unwind-protect
1900      nil
1901    (with-interrupts-enabled (loop)))
1903  and the loop would be interruptible.
1905  These changes introduce binary incompatibility (the FASL version changed,
1906  as did an internal version number that tries to keep the kernel and
1907  heap image in synch.)
1909  Things basically work, but there may be lingering bugs (e.g., as of
1910  a little while ago, QUIT didn't work because the initial process
1911  was running with interrupts disabled.)
1915  argument; processes have a PROCESS-TERMINATION-SEMAPHORE accessor
1916  method.  If the argument is specified and non-null, its value should
1917  of type SEMAPHORE.
1919  If a process dies by any means after it's been successfully enabled
1920  and it has a non-null termination semaphore "at the time of its death",
1921  that semaphore will be signaled just before the underlying OS thread
1922  is destroyed.
1924  SETF can be used with PROCESS-TERMINATION-SEMAPHORE to change or
1925  clear a the termination semaphore of a process.  If the target
1926  process is not the current process when this happens, it's possible
1927  that the process could die before the SETF takes effect; this
1928  possibility must be addressed at the application level (i.e., the
1929  implementation doesn't try to synchronize the calling thread and
1930  the target in any way.
1932  A simple example:
1934  (let* ((s (make-semaphore)))
1935    (process-run-function `(:name "sleepy" :termination-semaphore ,s)
1936                           #'(lambda () (sleep 10)))
1937    (wait-on-semaphore s))
1939  The calling thread will wait for (roughly) 10 seconds (until the
1940  "sleepy" thread has had its nap and signals its termination semaphore.)
1942- A change that was introduced prior to 0.14.3 led to strange, usually
1943  fatal crashes (usually an unhandled bus error, occasionally a cryptic
1944  "can't find active area" message and a trip to the kernel debugger)
1945  under Darwin.  This was caused by an attempt to use certain Mach
1946  primitives to suspend and resume threads (the way that those
1947  primitives were used, Mach exception messages were sometimes sent
1948  twice if the first send was interrupted, and the second send occurred
1949  after the exception had already been handled (because the first send
1950  was recieved but not replied to ...)
1952  1.0 backed out of this change, and used signal handling primitives
1953  (instead of Mach primitives) to suspend and resume threads.  I -think-
1954  that I understand the issue with the Mach primitives
1955  (#_thread_abort_safely isn't necessary and caused the duplicate
1956  exception messages to be sent) and have tried to revert to using
1957  the Mach thread suspension mechanisms.  (If unhandled bus errors -
1958  that exit to the shell - or cryptic "can't find active area" messages
1959  reappear, this experiment will be shown to be a failure.)
1961  There are some obscure but good reasons for favoring the Mach
1962  primiitves, so it'd be good to know if the problem with using them
1963  has indeed been identified.
1965  (The test case involves bad luck and bad timing: two or more
1966  threads having pending exceptions at the same time and the thread
1967  whose exception is handled first tries to suspend the others, typically
1968  on behalf of the GC.  It was possible to run stress tests for many
1969  hours in 0.14.3 without encountering the bug, and possible to
1970  encounter it under seemingly light loads.)
1972- INCF and DECF argument order and fixnum arithmetic.
1974  Bryan fixed some ANSI test failures related to the order in which INCF
1975  and DECF evaluate their args.  (One example is:
1977  (let* ((x 3))
1978    (incf x (setq x 5)))
1980  where the correct answer is 10, not 8.)  We both found that fixing
1981  some cases involving INCF caused some OpenMCL code to compile
1982  incorrectly and were nervous about introducing these changes fairly
1983  late in the development cycle, so we backed out of them prior to
1984  the 1.0 code freeze.
1986  The reasons for the miscompiled code have to do with how the
1987  compiler interprets fixnum declarations under typical optimization
1988  settings.  If A and B are both declared to be FIXNUMS, then
1989  the expression
1991  (setq a (+ a b))
1993  will usually compile to a simple ADD instruction (with no overflow
1994  checking); if A and B are fixnums, the result will be a fixnum,
1995  though if an undetected overflow occurred in the addition, the
1996  result might be missing a significant bit.
1998  There was code in OpenMCL that assumed that
2000  (incf a b)
2002  was exactly the same as
2004  (setq a (+ a b))
2006  and in fact that was true under the old (incorrect) definition of
2007  INCF.  The new definition introduced some temporary bindings:
2009  (let* ((...)
2010         (#:temp (+ a b))
2011         (...))
2012     (setq a #:temp))
2014  In this case, the addition was allowed to generate an overflow
2015  (no type declaration on #:temp), and the SETQ quietly violated
2016  a type declaration (assigning a non-FIXNUM value to A), leading
2017  to further problems.
2019  So far, I found a couple of cases of this in the OpenMCL sources.
2020  (FWIW, both functions were originally transliterated from C code
2021  and were trying to mimic C's silent overflow behavior.)
2023  Moral: if you have code that assumes that INCF or DECF expand
2024  into simple assignments and are trying to exploit the ways that
2025  those assignments interact with type declarations, you may
2026  want to review those assumptions.  If you write code that has
2027  side effects in the DELTA arguments of INCF or DECF rorms,
2028  you'll (hopefully) be pleased to see that Bryan's changes
2029  allow these side-effects to be handled correctly (at the
2030  right time.)  If you don't fall into either of these categories,
2031  you probably won't notice any difference ...
2033- 64-bit Linux support
2035  There's a 64-bit LinuxPPC heap image and some rudimentary (libc-only)
2036  64-bit Linux interfaces in the testing directory.
2038  (Unlike 64-bit Darwin, 64-bit Linux distributions typically provide
2039  64-bit versions of "all" standard libraries; I haven't gotten around
2040  to building 64-bit gnome/gtk/X11/... interfaces yet, but wouldn't
2041  expect there to be a problem.)
2043  The 64-bit Linux OpenMCL seems to basically work, but ... OpenMCL
2044  likes to map its kernel into low addresses (around #x5000); this
2045  allows compiled lisp code to use conditional branches to "short"
2046  (16-bit) absolute addresses.  Newer Linux kernels provide a
2047  "vdso" shared library that's intended to simply communication
2048  between the OS kernel and userspace libraries and programs; when
2049  a program is mapped at "non-standard" addresses, the vdso gets
2050  mapped at address 0.
2052  I don't fully understand the imlications of this (beyond the fact that
2053  indirecting through a NULL pointer will access bits and pieces
2054  of the vdso instead of segfaulting.)  As far as I know, this is
2055  seen as a minor bug in the Linux kernel, and I -think- that I've
2056  seen kernel ChangeLog entries that indicate that the problem's been
2057  fixed in the relatively recent past (and will likely start to
2058  make it into Linux distributions in the near future.)
2060  That said - and seeing a library at address 0 certainly makes me a
2061  little nervous - the LinuxPPC64 port seems to work at least as
2062  well as the DarwinPPC64 port does (i.e., there may be word-size
2063  or other bugs lurking around or hiding in plain sight, but it's
2064  not usually easy to encounter them.)
2066- As documented (and as hasn't been true in a long time), EOF
2067  from *STANDARD-INPUT* terminates the REPL when the --batch argument
2068  is in effect (even if *STANDARD-INPUT* is a tty.)
2070- QUIT does a FRESH-LINE on and FORCE-OUTPUT to the standard output
2071  stream (people had reported that output wasn't always flushed
2072  when --batch or --eval was used; 1.0 was better about this than
2073  previous versions were, but it still wasn't reliable.)
2075OpenMCL 1.1-pre-051028
2076I had been doing development on G5s, and hadn't noticed that the
207732-bit lisp had been using a 64-bit instruction.  (I'm a little
2078confused about how that could have worked; perhaps the 64-bit
2079instruction gets emulated by the OS, or perhaps my model of
2080whether 64-bit instructions can be executed in 32-bit mode
2081is simply incorrect.)
2083In any case, the 32-bit images produced yesterday don't run on
2084G4s (or presumably G3s or older systems.)  Ooops.  New images.
2086OpenMCL 1.1-pre-051029
2087 A function used by both SET-USER-ENVIRONMENT
2088and SET-DEVELOPMENT-ENVIRONMENT wasn't properly changing saved bindings
2089of *PACKAGE*; the last few 1.1-pre releases have come up in the CCL
2090package, as a result.  Ooops again; new images, again.
2093OpenMCL 1.1-pre-051204
2094Not a lot of user-visible changes, but the changes that're there
2095are a little hard to bootstrap.
2096Note that new kernel build directories (darwinppc, darwinppc64,
2097linuxppc, linuxppc64, ...) repace the old versions that don't
2098have "ppc" in their names.  CVS may not prune the old directories,
2099especially if they contain files (.o, random junk).
Note: See TracBrowser for help on using the repository browser.