Index: /trunk/source/doc/src/openmcl-documentation.xml =================================================================== --- /trunk/source/doc/src/openmcl-documentation.xml (revision 8519) +++ /trunk/source/doc/src/openmcl-documentation.xml (revision 8520) @@ -1,11 +1,10 @@ &rest"> -&key"> -&optional"> -&body"> -&aux"> -&allow-other-keys"> +&rest"> +&key"> +&optional"> +&body"> +&aux"> +&allow-other-keys"> ]> @@ -1074,8 +1073,8 @@ $ cd ccl # wherever your ccl directory is -$ ./ +$ ./KERNEL BOOT_IMAGE - Where and - are the names of + Where KERNEL and + BOOT_IMAGE are the names of the kernel and boot image appropriate to the platform you are running on. See FIXTHIS @@ -1292,5 +1291,5 @@ scheduler may be less portable to other CL implementations, many of which offer a cooperative scheduler and an API similar to - OpenMCL (< 0.14) 's.) At the same time, there's a large + OpenMCL (< 0.14) 's.) At the same time, there's a large overlap in functionality in the two scheduling models, and it'll hopefully be possible to write interesting and useful MP code @@ -1476,5 +1475,5 @@ ? (process-run-function "sleeper" #'(lambda () (sleep 5) (break "broken"))) -# Break in process sleeper(1): broken -> While executing: # While executing: #<Anonymous Function #x3063B276> > Type :GO to continue, :POP to abort. > If continued: Return from BREAK. Type :? for other options. -1 > :b +1 > :b (30C38E30) : 0 "Anonymous Function #x3063B276" 52 (30C38E40) : 1 "Anonymous Function #x304984A6" 376 @@ -1523,8 +1522,8 @@ ? (process-run-function "sleep-60" #'(lambda () (sleep 60) (break "Huh?"))) -# - The Threads which OpenMCL Uses for Its Own Purposes + The Threads which OpenMCL Uses for Its Own Purposes + In the "tty world", OpenMCL starts out with 2 lisp-level threads: @@ -1723,5 +1723,4 @@ Threads Dictionary - @@ -1759,9 +1758,9 @@ Returns a list of all lisp processes (threads) known - to OpenMCL as of the precise instant it's - called. It's safe to traverse this list and to modify - the cons cells that comprise that list (it's freshly - consed.) Since other threads can create and kill threads at - any time, there's generally no way to get an + to OpenMCL as of + the precise instant it's called. It's safe to traverse + this list and to modify the cons cells that comprise that list + (it's freshly consed.) Since other threads can create and kill + threads at any time, there's generally no way to get an "accurate" list of all threads, and (generally) no sense in which such a list can be accurate. @@ -1777,6 +1776,5 @@ - - + make-process @@ -1945,1185 +1943,2442 @@ - - PROCESS-SUSPEND - process-suspend - Name - PROCESS-SUSPEND — Suspends a specified process. - Function - Synopsis - -process-suspend process - => result - - Arguments and Values - process - a lisp process (thread). - result - T if process had been runnableand is now suspended; NIL otherwise. That is, T ifprocess'stransitioned from 0 to 1. - - - Description - Suspends process, preventing it from -running, and stopping it if it was already running. This is a fairly -expensive operation, because it involves a few -calls to the OS. It also risks creating deadlock if used -improperly, for instance, if the process being suspended owns a -lock or other resource which another process will wait for. - Each -call to process-suspend must be reversed by -a matching call to -before process is able to run. What -process-suspend actually does is increment -the of -process. - A process cannot suspend itself (although that was possible in -some older OpenMCL releases and this documentation claimed that -it still was.) - See Also - , - Notes - process-suspend was previously called -process-disable. - now names a function for which there is no -obvious inverse, so process-disable -is no longer -defined. - - - - PROCESS-RESUME - process-resume - Name - PROCESS-RESUME — Resumes a specified process which had previously -been suspended by process-suspend. - Function - Synopsis - -process-resume process - => result - - Arguments and Values - process - a lisp process (thread). - result - T if process had been suspendedand is now runnable; NIL otherwise. That is, T ifprocess'stransitioned from to 0. - - - Description - Undoes the effect of a previous call to -; if -all such calls are undone, makes the process runnable. Has no -effect if the process is not suspended. What -process-resume actually does is decrement -the of -process, to a minimum of 0. - See Also - , - Notes - This was previously called PROCESS-ENABLE; - now does something slightly -different. - - - - PROCESS-SUSPEND-COUNT - process-suspend-count - Name - PROCESS-SUSPEND-COUNT — Returns the number of currently-pending suspensions -applicable to a given process. - Function - Synopsis - - - process-suspend-count + + + process-suspend + + + + PROCESS-SUSPEND + Suspends a specified process. + Function + + + + process-suspend process + => result + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + result + + T if process had been runnable + and is now suspended; NIL otherwise. That is, T if + process's + + transitioned from 0 to 1. + + + + + + + Description + + Suspends process, preventing it from + running, and stopping it if it was already running. This is a fairly + expensive operation, because it involves a few + calls to the OS. It also risks creating deadlock if used + improperly, for instance, if the process being suspended owns a + lock or other resource which another process will wait for. + + + Each + call to process-suspend must be reversed by + a matching call to + before process is able to run. What + process-suspend actually does is increment + the of + process. + + + A process can suspend itself; it it's successful in doing + so, then it can obviously only be resumed by some other + process. + + + + See Also + + + + + + + + + Notes + process-suspend was previously called + process-disable. + + now names a function for which there is no + obvious inverse, so process-disable + is no longer + defined. + + + + + + process-resume + + + + PROCESS-RESUME + Resumes a specified process which had previously + been suspended by process-suspend. + Function + + + + process-resume process + => result + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + result + + T if process had been suspended + and is now runnable; NIL otherwise. That is, T if + process's + + transitioned from to 0. + + + + + + + + Description + + Undoes the effect of a previous call to + ; if + all such calls are undone, makes the process runnable. Has no + effect if the process is not suspended. What + process-resume actually does is decrement + the of + process, to a minimum of 0. + + + + See Also + + + + + + + + + Notes + + + This was previously called PROCESS-ENABLE; + now does something slightly + different. + + + + + + + process-suspend-count + + + + PROCESS-SUSPEND-COUNT + Returns the number of currently-pending suspensions + applicable to a given process. + Function + + + + + process-suspend-count process => result - - - Arguments and Values - process - a lisp process (thread). - result - The number of "outstanding" calls onprocess, or NIL ifprocess has expired. - - - Description - An "outstanding" call -is one which has not yet been reversed by a call to -. A process expires when -its initial function returns, although it may later be -reset. - A process is runnable when it has a -process-suspend-count of 0, has been -preset as by , and has been -enabled as by . Newly-created -processes have a process-suspend-count of -0. - See Also - , - - - - PROCESS-PRESET - process-preset - Name - PROCESS-PRESET — Sets the initial function and arguments of a specified -process. - Function - Synopsis - -process-preset - process function &rest args - => result - - Arguments and Values - process - a lisp process (thread). - function - a function, designated by itself or by a symbolwhich names it. - args - a list of values, appropriate as arguments tofunction. - result - undefined. - - - Description - Typically used to initialize a newly-created or newly-reset -process, setting things up so that when process -becomes enabled, it will begin execution by -applying function to args. -process-preset does not enable -process, -although a process must be process-preset -before it can be enabled. Processes are normally enabled by -. - See Also - , , - - - - PROCESS-ENABLE - process-enable - Name - PROCESS-ENABLE — Begins executing the initial function of a specified -process. - Function - Synopsis - -process-enable - process &optional timeout - - - Arguments and Values - process - a lisp process (thread). - timeout - a time interval in seconds. May be anynon-negative real number the floor ofwhich fits in 32 bits. The default is 1. - result - undefined. - - - Description - Tries to begin the execution of process. -An error is signaled if process has never -been . Otherwise, -process invokes its initial function. - process-enable attempts to -synchronize with process, which is presumed -to be reset or in the act of resetting itself. If this attempt -is not successful within the time interval specified by -timeout, a continuable error is signaled, -which offers the opportunity to continue waiting. - A process cannot meaningfully attempt to enable itself. - See Also - , , - Notes - It would be nice to have more discussion of what it means -to synchronize with the process. - - - - PROCESS-RUN-FUNCTION - process-run-function - Name - PROCESS-RUN-FUNCTION — Creates a process, presets it, and enables it. - Function - Synopsis - -process-run-function - process-specifier function &rest args => process - - process-specifier - name |(&key namepersistentpriorityclassstack-sizevstack-sizetstack-size) - - - Arguments and Values - name - a string, used to identify the process.Passed to make-process. - function - a function, designated by itself or by a symbolwhich names it. Passed topreset-process. - persistent - a boolean, passed to make-process. - priority - ignored. - class - a subclass of CCL:PROCESS. Passed tomake-process. - stack-size - a size, in bytes. Passed tomake-process. - vstack-size - a size, in bytes. Passed tomake-process. - tstack-size - a size, in bytes. Passed tomake-process. - process - the newly-created process. - - - Description - Creates a lisp process (thread) via -, -presets it via , and -enables it via . This means -that process will immediately begin to -execute. -process-run-function is -the simplest way to create and run a process. - See Also - , , - - - - PROCESS-INTERRUPT - process-interrupt - Name - PROCESS-INTERRUPT — Arranges for the target process to invoke a -specified function at some point in the near future, and then -return to what it was doing. - Function - Synopsis - -process-interrupt - process function &rest args => result - - Arguments and Values - process - a lisp process (thread). - function - a function. - args - a list of values, appropriate as arguments tofunction. - result - the result of applying functionto args if proceessis the current-process, otherwiseNIL. - - - Description - Arranges for process to apply function to args at -some point in the near future (interrupting whatever process -was doing.) If function returns normally, process -resumes execution at the point at which it was interrupted. - process must be in an enabled state in order to respond to a -process-interrupt request. It's perfectly legal for a process -to call process-interrupt on itself. - process-interrupt uses asynchronous POSIX signals to interrupt -threads. If the thread being interrupted is executing lisp code, it -can respond to the interrupt almost immediately (as soon as it has -finished pseudo-atomic operations like consing and stack-frame -initialization.) - If the interrupted thread is blocking in a system call, that system -call is aborted by the signal and the interrupt is handled on return. - Beginning with the version 1.1 prereleases of OpenMCL interrupts are -disabled in unwind-protect cleanup forms and in any stack-unwinding -code between the point of the throw and the corresponding CATCH -target. If interrupts were enabled at the time that the CATCH was -established, then any interrupt that had been deferred during -unwinding will be taken just before the transfer to catch target -(after all of that unwinding is complete.) - If an unwind-protect cleanup form actually does something that -needs to be interruptible, it's necessary to use -with-interrupts-enabled. (This actually happens somewhere in -the code which waits for external processes to complete; some of that -waiting occurred within an unwind-protect cleanup, and interrupts -needed to be explicitly enabled in that case in order to make the wait -interruptible.) - Note that (without-interrupts (throw ...)) wouldn't have the intended -effect, since the throw would cause execution to exit the extent of -the without-interrupts. - In versions prior to 1.1, interrupts could occur at arbitrary times -during the process of unwinding the stack and executing intervening -cleanup forms. - It is still difficult to reliably interrupt arbitrary foreign code -(that may be stateful or otherwise non-reentrant); the interrupt -request is handled when such foreign code returns to or enters lisp. - See Also - - Notes - It would probably be better for result to always be NIL, since -the present behaviour is inconsistent. - Process-interrupt works by sending signals between threads, via -the C function #_pthread_signal. It could be argued that it -should be done in one of several possible other ways under Darwin, to -make it practical to asynchronously interrupt things which make heavy -use of the Mach nanokernel. - - - - *CURRENT-PROCESS* - *current-process* - Name - *CURRENT-PROCESS* — Bound in each process, to that process -itself. - Variable - Value Type - A lisp process (thread). - Initial Value - Bound separately in each process, to that process itself. - Description - Used when lisp code needs to find out what process it is -executing in. Shouldn't be set by user code. - See Also - - - - - PROCESS-RESET - process-reset - Name - PROCESS-RESET — Causes a specified process to cleanly exit from -any ongoing computation. - Function - Synopsis - -process-reset - process &optional kill-option => result - - Arguments and Values - process - a lisp process (thread). - kill-option - a generalized boolean. The default is T. - result - undefined. - - - Description - Causes process to cleanly exit -from any ongoing computation. If kill-option -is true, process then exits. Otherwise, it -enters a state where it can be -. This -is implemented by signaling a condition of type PROCESS-RESET; -user-defined condition handlers should generally refrain from -attempting to handle conditions of this type. - A process can meaningfully reset itself. - There is in general no way to know precisely when -process -has completed the act of resetting or killing itself; a process -which has either entered the limbo of the reset state or exited -has few ways of communicating either fact. - can reliably determine when a process has entered -the "limbo of the reset state", but can't predict how long the -clean exit from ongoing computation might take: that depends on -the behavior of unwind-protect cleanup -forms, and of the OS scheduler. - Resetting a process other than - involves the -use of . - See Also - , - - - - PROCESS-KILL - process-kill - Name - PROCESS-KILL — Causes a specified process to cleanly exit from any -ongoing computation, and then exit. - Function - Synopsis - -process-kill process - => result - - Arguments and Values - process - a lisp process (thread). - result - undefined. - - - Description - Entirely equivalent to calling -(PROCESS-RESET PROCESS T). Causes process -to cleanly exit from any ongoing computation, and then exit. - See Also - , - - - - PROCESS-ABORT - process-abort - Name - PROCESS-ABORT — Causes a specified process to process an abort -condition, as if it had invoked -abort. - Function - Synopsis - -process-abort process - &optional condition - => NIL - - Arguments and Values - process - a lisp process (thread). - condition - a lisp condition. The default is NIL. - - - Description - Entirely equivalent to calling -( process -(lambda () -(abort condition))). -Causes process to transfer control to the -applicable handler or restart for abort. - If condition is non-NIL, -process-abort does not consider any -handlers which are explicitly bound to conditions other than -condition. - See Also - , - - - - *TICKS-PER-SECOND* - *ticks-per-second* - Name - *TICKS-PER-SECOND* — Bound to the clock resolution of the OS -scheduler. - Variable - Value Type - A positive integer. - Initial Value - The clock resoluton of the OS scheduler. Currently, -both LinuxPPC and DarwinPPC yeild an initial value of 100. - Description - This value is ordinarily of marginal interest at best, -but, for backward compatibility, some functions accept timeout -values expressed in "ticks". This value gives the number of -ticks per second. - See Also - - - - - PROCESS-WHOSTATE - process-whostate - Name - PROCESS-WHOSTATE — Returns a string which describes the status of -a specified process. - Function - Synopsis - -process-whostate process - => whostate - - process - a lisp process (thread). - whostate - a string which describes the "state" ofprocess. - - - Description - This information is primarily for the benefit of -debugging tools. whostate is a terse report -on what process is doing, or not doing, -and why. - If the process is currently waiting in a call to - or -, its -process-whostate will be the value -which was passed to that function as whostate. - See Also - , , - Notes - This should arguably be SETFable, but doesn't seem to -ever have been. - - - - PROCESS-ALLOW-SCHEDULE - process-allow-schedule - Name - PROCESS-ALLOW-SCHEDULE — Used for cooperative multitasking; probably never -necessary. - Function - Synopsis - -process-allow-schedule - - Description - Advises the OS scheduler that the current thread has nothing -useful to do and that it should try to find some other thread to -schedule in its place. There's almost always a better -alternative, such as waiting for some specific event to -occur. For example, you could use a lock or semaphore. - See Also - , , , , , - Notes - This is a holdover from the days of cooperative -multitasking. All modern general-purpose operating systems use -preemptive multitasking. - - - - PROCESS-WAIT - process-wait - Name - PROCESS-WAIT — Causes the current lisp process (thread) to wait for -a given -predicate to return true. - Function - Synopsis - -process-wait - whostate function &rest args => result - - Arguments and Values - whostate - a string, which will be the value ofwhile the process is waiting. - function - a function, designated by itself or by a symbolwhich names it. - args - a list of values, appropriate as arguments tofunction. - result - NIL. - - - Description - Causes the current lisp process (thread) to repeatedly -apply function to -args until the call returns a true result, then -returns NIL. After -each failed call, yields the CPU as if by -. - As with , it's almost -always more efficient to wait for some -specific event to occur; this isn't exactly busy-waiting, but -the OS scheduler can do a better job of scheduling if it's given -the relevant information. For example, you could use a lock -or semaphore. - See Also - , , , , , , , - - - - PROCESS-WAIT-WITH-TIMEOUT - process-wait-with-timeout - Name - PROCESS-WAIT-WITH-TIMEOUT — Causes the current thread to wait for a given -predicate to return true, or for a timeout to expire. - Function - Synopsis - -process-wait-with-timeout - whostate ticks function args => result - - Arguments and Values - whostate - a string, which will be the value ofwhile the process is waiting. - ticks - either a positive integer expressing a durationin "ticks" (see ),or NIL. - function - a function, designated by itself or by a symbolwhich names it. - args - a list of values, appropriate as arguments tofunction. - result - T if process-wait-with-timeoutreturned because its function returnedtrue, or NIL if it returned because the durationticks has been exceeded. - - - Description - If ticks is NIL, behaves exactly like -, except for returning T. -Otherwise, function will be tested repeatedly, -in the same -kind of test/yield loop as in > -until either function returns true, -or the duration ticks has been exceeded. - Having already read the descriptions of - and -, the -astute reader has no doubt anticipated the observation that -better alternatives should be used whenever possible. - See Also - , , , , , , , , - - - - WITHOUT-INTERRUPTS - without-interrupts - Name - WITHOUT-INTERRUPTS — Evaluates its body in an environment in which -process-interrupt requests are deferred. - Macro - Synopsis - -without-interrupts - &body body => result - - Arguments and Values - body - an implicit progn. - result - the primary value returned bybody. - - - Description - Executes body in an environment in which - requests are deferred. As noted in the -description of , this has nothing to do with -the scheduling of other threads; it may be necessary to inhibit - handling when (for instance) modifying some -data structure (for which the current thread holds an appropriate -lock) in some manner that's not reentrant. - Beginning with the version 1.1 prereleases of OpenMCL interrupts are -disabled in unwind-protect cleanup forms and in any -stack-unwinding code between the point of the throw and the -corresponding CATCH target. If an unwind-protect cleanup form -actually does something that needs to be interruptible, it's necessary -to use with-interrupts-enabled. In versions prior to 1.1, -interrupts can occur at arbitrary times during the process of -unwinding the stack and executing intervening cleanup forms. - See Also - - - - - MAKE-LOCK - make-lock - Name - MAKE-LOCK — Creates and returns a lock object, which can -be used for synchronization between threads. - Function - Synopsis - -make-lock &optional - name => lock - - Arguments and Values - name - any lisp object; saved as part oflock. Typically a string or symbolwhich may appear in the sof threads which are waiting for lock. - lock - a newly-allocated object of type CCL:LOCK. - - - Description - Creates and returns a lock object, which can -be used to synchronize access to some shared resource. -lock is -initially in a "free" state; a lock can also be -"owned" by a -thread. - See Also - , , , , , , , , - - - - WITH-LOCK-GRABBED - with-lock-grabbed - Name - WITH-LOCK-GRABBED — Waits until a given lock can be obtained, then -evaluates its body with the lock held. - Macro - Synopsis - -with-lock-grabbed - (lock) &body body - - Arguments and Values - lock - an object of type CCL:LOCK. - body - an implicit progn. - result - the primary value returned bybody. - - - Description - Waits until lock is either free or -owned by the calling -thread, then excutes body with the -lock owned by the calling thread. If lock -was free when with-lock-grabbed was called, -it is restored to a free state after body -is executed. - See Also - , , , , , , , , - - - - GRAB-LOCK - grab-lock - Name - GRAB-LOCK — Waits until a given lock can be obtained, then -obtains it. - Function - Synopsis - -grab-lock lock - - Arguments and Values - lock - an object of type CCL:LOCK. - - - Description - Blocks until lock is owned by the -calling thread. - The macro -could be defined in -terms of grab-lock and -, but it is actually -implemented at a slightly lower level. - See Also - , , , , , , , , - - - - RELEASE-LOCK - release-lock - Name - RELEASE-LOCK — Relinquishes ownership of a given lock. - Function - Synopsis - -release-lock lock - - Arguments and Values - lock - an object of type CCL:LOCK. - - - Description - Signals an error of type CCL:LOCK-NOT-OWNER if -lock -is not already owned by the calling thread; otherwise, undoes the -effect of one previous -. If this means that -release-lock has now been called on -lock the same number of times as - has, lock -becomes free. - See Also - , , , , , , , , - - - - TRY-LOCK - try-lock - Name - TRY-LOCK — Obtains the given lock, but only if it is not -necessary to wait for it. - Function - Synopsis - -try-lock lock => result - - Arguments and Values - lock - an object of type CCL:LOCK. - result - T if lock has been obtained,or NIL if it has not. - - - Description - Tests whether lock -can be obtained without blocking - that is, either -lock is already free, or it is already owned -by . If it can, -causes it to -be owned by the calling lisp process (thread) and returns T. -Otherwise, the lock -is already owned by another thread and cannot be obtained without -blocking; NIL is returned in this case. - See Also - , , , , , , , , - - - - MAKE-READ-WRITE-LOCK - make-read-write-lock - Name - MAKE-READ-WRITE-LOCK — Creates and returns a read-write lock, which can -be used for synchronization between threads. - Function - Synopsis - -make-read-write-lock - => read-write-lock - - Arguments and Values - read-write-lock - a newly-allocated object of typeCCL:READ-WRITE-LOCK. - - - Description - Creates and returns an object of type CCL::READ-WRITE-LOCK. -A read-write lock may, at any given time, belong to any number -of lisp processes (threads) which act as "readers"; or, it may -belong to at most one process which acts as a "writer". A -read-write lock may never be held by a reader at the same time as -a writer. Intially, read-write-lock has -no readers and no writers. - See Also - , , , , , , - Notes - There probably should be some way to -atomically "promote" a reader, making it a writer without -releasing the lock, which could otherwise cause delay. - - - - WITH-READ-LOCK - with-read-lock - Name - WITH-READ-LOCK — Waits until a given lock is available for -read-only access, then evaluates its body with the lock -held. - Macro - Synopsis - -with-read-lock - (read-write-lock) &body body => result - - Arguments and Values - read-write-lock - an object of typeCCL:READ-WRITE-LOCK. - body - an implicit progn. - result - the primary value returned bybody. - - - Description - Waits until read-write-lock has no -writer, -ensures that is a -reader of it, then executes body. - After executing body, if - was not a reader of -read-write-lock before -with-read-lock was called, the lock is -released. If it was already a reader, it remains one. - See Also - , , , , , , - - - - WITH-WRITE-LOCK - with-write-lock - Name - WITH-WRITE-LOCK — Waits until the given lock is available for write -access, then executes its body with the lock held. - Macro - Synopsis - -with-write-lock - (read-write-lock) &body body - - Arguments and Values - read-write-lock - an object of typeCCL:READ-WRITE-LOCK. - body - an implicit progn. - result - the primary value returned bybody. - - - Description - Waits until read-write-lock has no -readers and no writer other than , -then ensures that is the -writer of it. With the lock held, executes body. - After executing body, if - was not the writer of -read-write-lock before -with-write-lock was called, the lock is -released. If it was already the writer, it remains the -writer. - See Also - , , , , , , - - - - MAKE-SEMAPHORE - make-semaphore - Name - MAKE-SEMAPHORE — Creates and returns a semaphore, which can be used -for synchronization between threads. - Function - Synopsis - -make-semaphore - => semaphore - - Arguments and Values - semaphore - a newly-allocated object of type CCL:SEMAPHORE. - - - Description - Creates and returns an object of type CCL:SEMAPHORE. -A semaphore has an associated "count" which may be incremented -and decremented atomically; incrementing it represents sending -a signal, and decrementing it represents handling that signal. -semaphore has an initial count of 0. - See Also - , , , , , , , - - - - SIGNAL-SEMAPHORE - signal-semaphore - Name - SIGNAL-SEMAPHORE — Atomically increments the count of a given -semaphore. - Function - Synopsis - -signal-semaphore - semaphore => result - - Arguments and Values - semaphore - an object of type CCL:SEMAPHORE. - result - an integer representing an error identifierwhich was returned by the underlying OS call. - - - Description - Atomically increments semaphore's -"count" by 1; this -may enable a waiting thread to resume execution. - See Also - , , , , , , , - Notes - result should probably be interpreted -and acted on by signal-semaphore, because -it is not likely to be meaningful to a lisp program, and the -most common cause of failure is a type error. - - - - WAIT-ON-SEMAPHORE - wait-on-semaphore - Name - WAIT-ON-SEMAPHORE — Waits until the given semaphore has a positive -count which can be atomically decremented. - Function - Synopsis - -wait-on-semaphore - semaphore => result - - Arguments and Values - semaphore - an object of type CCL:SEMAPHORE. - result - WAIT-ON-SEMAPHORE always returns T. - - - Description - Waits until semaphore -has a positive count that can be -atomically decremented; this will succeed exactly once for each -corresponding call to SIGNAL-SEMAPHORE. - See Also - , , , , , , , - Notes - - - - TIMED-WAIT-ON-SEMAPHORE - timed-wait-on-semaphore - Name - TIMED-WAIT-ON-SEMAPHORE — Waits until the given semaphore has a postive -count which can be atomically decremented, or until a timeout -expires. - Function - Synopsis - -timed-wait-on-semaphore - semaphore timeout => result - - Arguments and Values - semaphore - An object of type CCL:SEMAPHORE. - timeout - a time interval in seconds. May be anynon-negative real number the floor ofwhich fits in 32 bits. The default is 1. - result - T if timed-wait-on-semaphorereturned because it was able to decrement the count ofsemaphore; NIL if it returned becausethe duration timeout has beenexceeded. - - - Description - Waits until semaphore -has a positive count that can be -atomically decremented, or until the duration -timeout has -elapsed. - See Also - , , , , , , - - - - PROCESS-INPUT-WAIT - process-input-wait - Name - PROCESS-INPUT-WAIT — Waits until input is available on a given -file-descriptor. - Function - Synopsis - -process-input-wait - fd &optional timeout - - Arguments and Values - fd - a file descriptor, which is a non-negative integerused by the OS to refer to an open file, socket, or similarI/O connection. See CCL::STREAM-DEVICE. - timeout - either NIL, or a time interval in seconds. May be anynon-negative real number the floor ofwhich fits in 32 bits. The default is NIL. - - - Description - Wait until input is available on fd. -This uses the select() system call, and is -generally a fairly -efficient way of blocking while waiting for input. More -accurately, process-input-wait -waits until it's possible to read -from fd without blocking, or until timeout, if -it is not NIL, has been exceeded. - Note that it's possible to read without blocking if -the file is at its end - although, of course, the read will -return zero bytes. - See Also - , , , , - Notes - process-input-wait has a timeout parameter, -and - does not. This -inconsistency should probably be corrected. - - - - PROCESS-OUTPUT-WAIT - process-output-wait - Name - PROCESS-OUTPUT-WAIT — Waits until output is possible on a given file -descriptor. - Function - Synopsis - -process-output-wait fd - - Arguments and Values - fd - a file descriptor, which is a non-negative integerused by the OS to refer to an open file, socket, or similarI/O connection. See CCL::STREAM-DEVICE. - - - Description - Wait until output is possible on fd. -This uses the select() system call, and is -generally a fairly -efficient way of blocking while waiting to output. - If process-output-wait is called on -a network socket which has not yet established a connection, it -will wait until the connection is established. This is an -important use, often overlooked. - See Also - , , , , - Notes - has a timeout parameter, -and -process-output-wait does not. This -inconsistency should probably be corrected. - - - - WITH-TERMINAL-INPUT - with-terminal-input - Name - WITH-TERMINAL-INPUT — Executes its body in an environment with exclusive -read access to the terminal. - Macro - Synopsis - -with-terminal-input - &body body => result - - Arguments and Values - body - an implicit progn. - result - the primary value returned bybody. - - - Description - Requests exclusive read access to the standard terminal -stream, *terminal-io*. Executes -body in an environment with that access. - See Also - , :Y, , , , , - - - - *REQUEST-TERMINAL-INPUT-VIA-BREAK* - request-terminal-input-via-break - Name - *REQUEST-TERMINAL-INPUT-VIA-BREAK* — Controls how attempts to obtain ownership of -terminal input are made. - Variable - Value Type - A boolean. - Initial Value - NIL. - Description - Controls how attempts to obtain ownership of terminal input -are made. When NIL, a message is printed on *TERMINAL-IO*; -it's expected that the user will later yield -control of the terminal via the :Y toplevel command. When T, a -BREAK condition is signaled in the owning process; continuing from -the break loop will yield the terminal to the requesting process -(unless the :Y command was already used to do so in the break -loop.) - See Also - , :Y, , , , , - - - - :Y - :y - Name - :Y — Yields control of terminal input to a specified -lisp process (thread). - Toplevel Command - Synopsis - -(:y p) - - Arguments and Values - p - a lisp process (thread), designated either byan integer which matches itsprocess-serial-number,or by a string which is equal toits process-name. - - - Description - :Y is a toplevel command, not a function. As such, it -can only be used interactively, and only from the initial -process. - The command yields control of terminal input to the -process p, which must have used - to request access to the -terminal input stream. - See Also - , , , , , , - + + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + result + + The number of "outstanding" + calls on + process, or NIL if + process has expired. + + + + + + + + Description + + An "outstanding" call + is one which has not yet been reversed by a call to + . A process expires when + its initial function returns, although it may later be + reset. + + A process is runnable when it has a + process-suspend-count of 0, has been + preset as by , and has been + enabled as by . Newly-created + processes have a process-suspend-count of + 0. + + + + See Also + + + + + + + + + + + process-preset + + + + PROCESS-PRESET + Sets the initial function and arguments of a specified + process. + Function + + + + process-preset + process function &rest; args + => result + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + function + + a function, designated by itself or by a symbol + which names it. + + + + + args + + a list of values, appropriate as arguments to + function. + + + + result + + undefined. + + + + + + + Description + + Typically used to initialize a newly-created or newly-reset + process, setting things up so that when process + becomes enabled, it will begin execution by + applying function to args. + process-preset does not enable + process, + although a process must be process-preset + before it can be enabled. Processes are normally enabled by + . + + + + + See Also + + + + + + + + + + + + process-enable + + + + PROCESS-ENABLE + Begins executing the initial function of a specified + process. + Function + + + + process-enable + process &optional; timeout + + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + timeout + + a time interval in seconds. May be any + non-negative real number the floor of + which fits in 32 bits. The default is 1. + + + + result + + undefined. + + + + + + + Description + + Tries to begin the execution of process. + An error is signaled if process has never + been . Otherwise, + process invokes its initial function. + + + process-enable attempts to + synchronize with process, which is presumed + to be reset or in the act of resetting itself. If this attempt + is not successful within the time interval specified by + timeout, a continuable error is signaled, + which offers the opportunity to continue waiting. + + + A process cannot meaningfully attempt to enable itself. + + + + See Also + + + + + + + + + + Notes + + It would be nice to have more discussion of what it means + to synchronize with the process. + + + + + + process-run-function + + + + PROCESS-RUN-FUNCTION + Creates a process, presets it, and enables it. + + Function + + + + process-run-function + process-specifier function &rest; args => process + + + + process-specifier + + + name | + (&key; name + persistent + priority + class + stack-size + vstack-size + tstack-size) + + + + + + + + Arguments and Values + + + + name + + a string, used to identify the process. + Passed to make-process. + + + + function + + a function, designated by itself or by a symbol + which names it. Passed to + preset-process. + + + + + persistent + + + a boolean, passed to make-process. + + + + + + priority + + + ignored. + + + + + class + + + a subclass of CCL:PROCESS. Passed to + make-process. + + + + + stack-size + + + a size, in bytes. Passed to + make-process. + + + + + vstack-size + + + a size, in bytes. Passed to + make-process. + + + + + tstack-size + + + a size, in bytes. Passed to + make-process. + + + + process + + the newly-created process. + + + + + + + Description + + Creates a lisp process (thread) via + , + presets it via , and + enables it via . This means + that process will immediately begin to + execute. + process-run-function is + the simplest way to create and run a process. + + + + + See Also + + + + + + + + + + + + process-interrupt + + + + PROCESS-INTERRUPT + Arranges for the target process to invoke a + specified function at some point in the near future, and then + return to what it was doing. + Function + + + + process-interrupt + process function &rest; args => result + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + function + + a function. + + + + + args + + a list of values, appropriate as arguments to + function. + + + + result + + the result of applying function + to args if proceess + is the current-process, otherwise + NIL. + + + + + + + Description + + Arranges for process + to apply function to args at + some point in the near future (interrupting whatever + process + was doing.) If function returns normally, + process resumes + execution at the point at which it was interrupted. + + process must be in an enabled state in + order to respond + to a process-interrupt request. It's + perfectly legal for a process to call + process-interrupt on itself. + + process-interrupt + uses asynchronous POSIX signals to interrupt threads. If the + thread being interrupted is executing lisp code, it can + respond to the interrupt almost immediately (as soon as it + has finished pseudo-atomic operations like consing and + stack-frame initialization.) + + If the interrupted thread is + blocking in a system call, that system call is aborted by + the signal and the interrupt is handled on return. + + + It is + still difficult to reliably interrupt arbitrary foreign code + (that may be stateful or otherwise non-reentrant); the + interrupt request is handled when such foreign code returns + to or enters lisp. + + + + See Also + + + + + + + + Notes + + It would probably be better for result + to always be NIL, since the present behaviour is inconsistent. + + + + Process-interrupt works by sending signals + between threads, via the C function + #_pthread_signal. It could be argued + that it should be done in one of several possible other ways + under + Darwin, to make it practical to asynchronously interrupt + things which make heavy use of the Mach nanokernel. + + + + + + + *current-process* + + + + *CURRENT-PROCESS* + Bound in each process, to that process + itself. + Variable + + + + Value Type + + A lisp process (thread). + + + + Initial Value + + Bound separately in each process, to that process itself. + + + + + Description + + Used when lisp code needs to find out what process it is + executing in. Shouldn't be set by user code. + + + + See Also + + + + + + + + + + process-reset + + + + PROCESS-RESET + Causes a specified process to cleanly exit from + any ongoing computation. + Function + + + + process-reset + process &optional; kill-option => result + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + kill-option + + a generalized boolean. The default is T. + + + + result + + undefined. + + + + + + + Description + + Causes process to cleanly exit + from any ongoing computation. If kill-option + is true, process then exits. Otherwise, it + enters a state where it can be + . This + is implemented by signaling a condition of type PROCESS-RESET; + user-defined condition handlers should generally refrain from + attempting to handle conditions of this type. + + A process can meaningfully reset itself. + + There is in general no way to know precisely when + process + has completed the act of resetting or killing itself; a process + which has either entered the limbo of the reset state or exited + has few ways of communicating either fact. + + can reliably determine when a process has entered + the "limbo of the reset state", but can't predict how long the + clean exit from ongoing computation might take: that depends on + the behavior of unwind-protect cleanup + forms, and of the OS scheduler. + + Resetting a process other than + involves the + use of . + + + + See Also + + + + + + + + + + + process-kill + + + + PROCESS-KILL + Causes a specified process to cleanly exit from any + ongoing computation, and then exit. + Function + + + + process-kill process + => result + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + result + + undefined. + + + + + + + Description + + Entirely equivalent to calling + (PROCESS-RESET PROCESS T). Causes process + to cleanly exit from any ongoing computation, and then exit. + + + + See Also + + + + + + + + + + + process-abort + + + + PROCESS-ABORT + Causes a specified process to process an abort + condition, as if it had invoked + abort. + Function + + + + process-abort process + &optional; condition + => NIL + + + + Arguments and Values + + + + process + + a lisp process (thread). + + + + condition + + a lisp condition. The default is NIL. + + + + + + + Description + + Entirely equivalent to calling + ( process + (lambda () + (abort condition))). + Causes process to transfer control to the + applicable handler or restart for abort. + + If condition is non-NIL, + process-abort does not consider any + handlers which are explicitly bound to conditions other than + condition. + + + + See Also + + + + + + + + + + + *ticks-per-second* + + + + *TICKS-PER-SECOND* + Bound to the clock resolution of the OS + scheduler. + Variable + + + + Value Type + + A positive integer. + + + + Initial Value + + The clock resoluton of the OS scheduler. Currently, + both LinuxPPC and DarwinPPC yeild an initial value of 100. + + + + + Description + + This value is ordinarily of marginal interest at best, + but, for backward compatibility, some functions accept timeout + values expressed in "ticks". This value gives the number of + ticks per second. + + + + See Also + + + + + + + + + + process-whostate + + + + PROCESS-WHOSTATE + Returns a string which describes the status of + a specified process. + Function + + + + process-whostate process + => whostate + + + process + + a lisp process (thread). + + + + whostate + + a string which describes the "state" of + process. + + + + + + + Description + + This information is primarily for the benefit of + debugging tools. whostate is a terse report + on what process is doing, or not doing, + and why. + + If the process is currently waiting in a call to + or + , its + process-whostate will be the value + which was passed to that function as whostate. + + + + + See Also + + + + + + + + + + Notes + + This should arguably be SETFable, but doesn't seem to + ever have been. + + + + + + process-allow-schedule + + + + PROCESS-ALLOW-SCHEDULE + Used for cooperative multitasking; probably never + necessary. + Function + + + + process-allow-schedule + + + + Description + + Advises the OS scheduler that the current thread has nothing + useful to do and that it should try to find some other thread to + schedule in its place. There's almost always a better + alternative, such as waiting for some specific event to + occur. For example, you could use a lock or semaphore. + + + + See Also + + + + + + + + + + + + + Notes + + This is a holdover from the days of cooperative + multitasking. All modern general-purpose operating systems use + preemptive multitasking. + + + + + + process-wait + + + + PROCESS-WAIT + Causes the current lisp process (thread) to wait for + a given + predicate to return true. + Function + + + + process-wait + whostate function &rest; args => result + + + + Arguments and Values + + + + whostate + + + a string, which will be the value of + + while the process is waiting. + + + + function + + a function, designated by itself or by a symbol + which names it. + + + + + args + + a list of values, appropriate as arguments to + function. + + + + result + + NIL. + + + + + + + Description + + Causes the current lisp process (thread) to repeatedly + apply function to + args until the call returns a true result, then + returns NIL. After + each failed call, yields the CPU as if by + . + + + As with , it's almost + always more efficient to wait for some + specific event to occur; this isn't exactly busy-waiting, but + the OS scheduler can do a better job of scheduling if it's given + the relevant information. For example, you could use a lock + or semaphore. + + + + See Also + + + + + + + + + + + + + + + + + process-wait-with-timeout + + + + PROCESS-WAIT-WITH-TIMEOUT + Causes the current thread to wait for a given + predicate to return true, or for a timeout to expire. + Function + + + + process-wait-with-timeout + whostate ticks function args => result + + + + Arguments and Values + + + + whostate + + a string, which will be the value of + + while the process is waiting. + + + + ticks + + either a positive integer expressing a duration + in "ticks" (see ), + or NIL. + + + + function + + a function, designated by itself or by a symbol + which names it. + + + + args + + a list of values, appropriate as arguments to + function. + + + + result + + T if process-wait-with-timeout + returned because its function returned + true, or NIL if it returned because the duration + ticks has been exceeded. + + + + + + + Description + + If ticks is NIL, behaves exactly like + , except for returning T. + Otherwise, function will be tested repeatedly, + in the same + kind of test/yield loop as in > + until either function returns true, + or the duration ticks has been exceeded. + + + Having already read the descriptions of + and + , the + astute reader has no doubt anticipated the observation that + better alternatives should be used whenever possible. + + + + See Also + + + + + + + + + + + + + + + + + + without-interrupts + + + + WITHOUT-INTERRUPTS + Evaluates its body in an environment in which + process-interrupt requests are deferred. + Macro + + + + without-interrupts + &body; body => result + + + + Arguments and Values + + + + body + + an implicit progn. + + + + result + + the primary value returned by + body. + + + + + + + Description + + Executes body + in an environment in which + requests are + deferred. As noted in the description of + , this has nothing to do + with the + scheduling of other threads; it may be necessary to inhibit + handling when + (for instance) modifying some data + structure (for which the current thread holds an appropriate lock) + in some manner that's not reentrant. + + + + See Also + + + + + + + + + + make-lock + + + + MAKE-LOCK + Creates and returns a lock object, which can + be used for synchronization between threads. + Function + + + + make-lock &optional; + name => lock + + + + Arguments and Values + + + + name + + any lisp object; saved as part of + lock. Typically a string or symbol + which may appear in the s + of threads which are waiting for lock. + + + + + lock + + a newly-allocated object of type CCL:LOCK. + + + + + + + Description + + Creates and returns a lock object, which can + be used to synchronize access to some shared resource. + lock is + initially in a "free" state; a lock can also be + "owned" by a + thread. + + + + See Also + + + + + + + + + + + + + + + + + + with-lock-grabbed + + + + WITH-LOCK-GRABBED + Waits until a given lock can be obtained, then + evaluates its body with the lock held. + Macro + + + + with-lock-grabbed + (lock) &body; body + + + + Arguments and Values + + + + lock + + an object of type CCL:LOCK. + + + + body + + an implicit progn. + + + + result + + the primary value returned by + body. + + + + + + + Description + + Waits until lock is either free or + owned by the calling + thread, then excutes body with the + lock owned by the calling thread. If lock + was free when with-lock-grabbed was called, + it is restored to a free state after body + is executed. + + + + See Also + + + + + + + + + + + + + + + + + + grab-lock + + + + GRAB-LOCK + Waits until a given lock can be obtained, then + obtains it. + Function + + + + grab-lock lock + + + + Arguments and Values + + + + lock + + an object of type CCL:LOCK. + + + + + + + Description + + Blocks until lock is owned by the + calling thread. + + The macro + could be defined in + terms of grab-lock and + , but it is actually + implemented at a slightly lower level. + + + + See Also + + + + + + + + + + + + + + + + + + release-lock + + + + RELEASE-LOCK + Relinquishes ownership of a given lock. + Function + + + + release-lock lock + + + + Arguments and Values + + + + lock + + an object of type CCL:LOCK. + + + + + + + Description + + Signals an error of type CCL:LOCK-NOT-OWNER if + lock + is not already owned by the calling thread; otherwise, undoes the + effect of one previous + . If this means that + release-lock has now been called on + lock the same number of times as + has, lock + becomes free. + + + + See Also + + + + + + + + + + + + + + + + + + try-lock + + + + TRY-LOCK + Obtains the given lock, but only if it is not + necessary to wait for it. + Function + + + + try-lock lock => result + + + + Arguments and Values + + + + lock + + an object of type CCL:LOCK. + + + + result + + T if lock has been obtained, + or NIL if it has not. + + + + + + + Description + + Tests whether lock + can be obtained without blocking - that is, either + lock is already free, or it is already owned + by . If it can, + causes it to + be owned by the calling lisp process (thread) and returns T. + Otherwise, the lock + is already owned by another thread and cannot be obtained without + blocking; NIL is returned in this case. + + + + See Also + + + + + + + + + + + + + + + + + + make-read-write-lock + + + + MAKE-READ-WRITE-LOCK + Creates and returns a read-write lock, which can + be used for synchronization between threads. + Function + + + + make-read-write-lock + => read-write-lock + + + + Arguments and Values + + + + read-write-lock + + a newly-allocated object of type + CCL:READ-WRITE-LOCK. + + + + + + + Description + + Creates and returns an object of type CCL::READ-WRITE-LOCK. + A read-write lock may, at any given time, belong to any number + of lisp processes (threads) which act as "readers"; or, it may + belong to at most one process which acts as a "writer". A + read-write lock may never be held by a reader at the same time as + a writer. Intially, read-write-lock has + no readers and no writers. + + + + See Also + + + + + + + + + + + + + + Notes + + There probably should be some way to + atomically "promote" a reader, making it a writer without + releasing the lock, which could otherwise cause delay. + + + + + + with-read-lock + + + + WITH-READ-LOCK + Waits until a given lock is available for + read-only access, then evaluates its body with the lock + held. + Macro + + + + with-read-lock + (read-write-lock) &body; body => result + + + + Arguments and Values + + + + read-write-lock + + an object of type + CCL:READ-WRITE-LOCK. + + + + body + + an implicit progn. + + + + result + + the primary value returned by + body. + + + + + + + Description + + Waits until read-write-lock has no + writer, + ensures that is a + reader of it, then executes body. + + + After executing body, if + was not a reader of + read-write-lock before + with-read-lock was called, the lock is + released. If it was already a reader, it remains one. + + + + See Also + + + + + + + + + + + + + + + + with-write-lock + + + + WITH-WRITE-LOCK + Waits until the given lock is available for write + access, then executes its body with the lock held. + Macro + + + + with-write-lock + (read-write-lock) &body; body + + + + Arguments and Values + + + + read-write-lock + + an object of type + CCL:READ-WRITE-LOCK. + + + + body + + an implicit progn. + + + + result + + the primary value returned by + body. + + + + + + + Description + + Waits until read-write-lock has no + readers and no writer other than , + then ensures that is the + writer of it. With the lock held, executes body. + + + After executing body, if + was not the writer of + read-write-lock before + with-write-lock was called, the lock is + released. If it was already the writer, it remains the + writer. + + + + See Also + + + + + + + + + + + + + + + + make-semaphore + + + + MAKE-SEMAPHORE + Creates and returns a semaphore, which can be used + for synchronization between threads. + Function + + + + make-semaphore + => semaphore + + + + Arguments and Values + + + + semaphore + + a newly-allocated object of type CCL:SEMAPHORE. + + + + + + + Description + + Creates and returns an object of type CCL:SEMAPHORE. + A semaphore has an associated "count" which may be incremented + and decremented atomically; incrementing it represents sending + a signal, and decrementing it represents handling that signal. + semaphore has an initial count of 0. + + + + See Also + + + + + + + + + + + + + + + + + signal-semaphore + + + + SIGNAL-SEMAPHORE + Atomically increments the count of a given + semaphore. + Function + + + + signal-semaphore + semaphore => result + + + + Arguments and Values + + + + semaphore + + an object of type CCL:SEMAPHORE. + + + + result + + an integer representing an error identifier + which was returned by the underlying OS call. + + + + + + + Description + + Atomically increments semaphore's + "count" by 1; this + may enable a waiting thread to resume execution. + + + + See Also + + + + + + + + + + + + + + + Notes + + result should probably be interpreted + and acted on by signal-semaphore, because + it is not likely to be meaningful to a lisp program, and the + most common cause of failure is a type error. + + + + + + wait-on-semaphore + + + + WAIT-ON-SEMAPHORE + Waits until the given semaphore has a positive + count which can be atomically decremented. + Function + + + + wait-on-semaphore + semaphore => result + + + + Arguments and Values + + + + semaphore + + an object of type CCL:SEMAPHORE. + + + + result + + an integer representing an error identifier + which was returned by the underlying OS call. + + + + + + + Description + + Waits until semaphore + has a positive count that can be + atomically decremented; this will succeed exactly once for each + corresponding call to SIGNAL-SEMAPHORE. + + + + See Also + + + + + + + + + + + + + + + Notes + + result should probably be interpreted + and acted on by wait-on-semaphore, because + it is not likely to be meaningful to a lisp program, and the + most common cause of failure is a type error. + + + + + + timed-wait-on-semaphore + + + + TIMED-WAIT-ON-SEMAPHORE + Waits until the given semaphore has a postive + count which can be atomically decremented, or until a timeout + expires. + Function + + + + timed-wait-on-semaphore + semaphore timeout => result + + + + Arguments and Values + + + + semaphore + + An object of type CCL:SEMAPHORE. + + + + timeout + + a time interval in seconds. May be any + non-negative real number the floor of + which fits in 32 bits. The default is 1. + + + + result + + T if timed-wait-on-semaphore + returned because it was able to decrement the count of + semaphore; NIL if it returned because + the duration timeout has been + exceeded. + + + + + + + Description + + Waits until semaphore + has a positive count that can be + atomically decremented, or until the duration + timeout has + elapsed. + + + + See Also + + + + + + + + + + + + + + + + process-input-wait + + + + PROCESS-INPUT-WAIT + Waits until input is available on a given + file-descriptor. + Function + + + + process-input-wait + fd &optional; timeout + + + + Arguments and Values + + + + fd + + a file descriptor, which is a non-negative integer + used by the OS to refer to an open file, socket, or similar + I/O connection. See . + + + + timeout + + either NIL, or a time interval in seconds. May be any + non-negative real number the floor of + which fits in 32 bits. The default is NIL. + + + + + + + Description + + Wait until input is available on fd. + This uses the select() system call, and is + generally a fairly + efficient way of blocking while waiting for input. More + accurately, process-input-wait + waits until it's possible to read + from fd without blocking, or until timeout, if + it is not NIL, has been exceeded. + + + Note that it's possible to read without blocking if + the file is at its end - although, of course, the read will + return zero bytes. + + + + See Also + + + + + + + + + + + + Notes + + + process-input-wait has a timeout parameter, + and + does not. This + inconsistency should probably be corrected. + + + + + + + process-output-wait + + + + PROCESS-OUTPUT-WAIT + Waits until output is possible on a given file + descriptor. + Function + + + + process-output-wait fd + + + + Arguments and Values + + + + fd + + a file descriptor, which is a non-negative integer + used by the OS to refer to an open file, socket, or similar + I/O connection. See . + + + + + + + Description + + Wait until output is possible on fd. + This uses the select() system call, and is + generally a fairly + efficient way of blocking while waiting to output. + + If process-output-wait is called on + a network socket which has not yet established a connection, it + will wait until the connection is established. This is an + important use, often overlooked. + + + + See Also + + + + + + + + + + + + Notes + + + has a timeout parameter, + and + process-output-wait does not. This + inconsistency should probably be corrected. + + + + + + + with-terminal-input + + + + WITH-TERMINAL-INPUT + Executes its body in an environment with exclusive + read access to the terminal. + Macro + + + + with-terminal-input + &body; body => result + + + + Arguments and Values + + + + body + + an implicit progn. + + + + result + + the primary value returned by + body. + + + + + + + Description + + Requests exclusive read access to the standard terminal + stream, *terminal-io*. Executes + body in an environment with that access. + + + + + See Also + + + + + + + + + + + + + + + + request-terminal-input-via-break + + + + *REQUEST-TERMINAL-INPUT-VIA-BREAK* + Controls how attempts to obtain ownership of + terminal input are made. + Variable + + + + Value Type + + A boolean. + + + + Initial Value + + NIL. + + + + Description + + Controls how attempts to obtain ownership of terminal input + are made. When NIL, a message is printed on *TERMINAL-IO*; + it's expected that the user will later yield + control of the terminal via the :Y toplevel command. When T, a + BREAK condition is signaled in the owning process; continuing from + the break loop will yield the terminal to the requesting process + (unless the :Y command was already used to do so in the break + loop.) + + + + See Also + + + + + + + + + + + + + + + + :y + + + + :Y + Yields control of terminal input to a specified + lisp process (thread). + Toplevel Command + + + + (:y p) + + + + Arguments and Values + + + + p + + a lisp process (thread), designated either by + an integer which matches its + process-serial-number, + or by a string which is equal to + its process-name. + + + + + + + Description + + :Y is a toplevel command, not a function. As such, it + can only be used interactively, and only from the initial + process. + + The command yields control of terminal input to the + process p, which must have used + to request access to the + terminal input stream. + + + + See Also + + + + + + + + + + + + + @@ -3133,5 +4388,7 @@ - OverviewOpenMCL supports the socket abstraction for + Overview + + OpenMCL supports the socket abstraction for interprocess communication. A socket represents a connection to another process, typically (but not necessarily) a TCP/IP @@ -3144,627 +4401,1370 @@ sockets, and Unix-domain sockets. This should be enough for all but the most esoteric network situations. All sockets are - created by . The type of socket depends on the arguments to it, - as follows: - tcp-stream - A buffered bi-directional stream over a TCP/IP connection.tcp-stream is a subclass of stream, and you can read and write to itusing all the usual stream functions. Created by (make-socket:addess-family :internet :type :stream :connect :active ...) or by(accept-connection ...). - file-socket-stream - A buffered bi-directional stream over a "UNIX domain"connection. file-socket-stream is a subclass of stream, and you canread and write to it using all the usual stream functions. Createdby (make-socket :address-family :file :type :stream :connect :active...) or by (accept-connection ...), - listener-socket - A passive socket used to listen for incoming TCP/IPconnections on a particular port. A listener-socket is not a stream.It doesn't support I/O. It can only be used to create newtcp-streams by accept-connection. Created by (make-socket :type:stream :connect :passive ...) - file-listener-socket - A passive socket used to listen for incoming UNIX domainconnections named by a file in the local filesystem. Alistener-socket is not a stream. It doesn't support I/O. It canonly be used to create new file-socket-streams by accept-connection.Created by (make-socket :address-family :file :type :stream :connect:passive ...) - udp-socket - A socket representing a packet-based UDP/IP connection. Audp-socket supports I/O but it is not a stream. Instead, you mustuse the special functions send-to and receive-from to read and writeto it. Created by (make-socket :type :datagram ...) - - + created by . The type of socket + depends on the arguments to it, as follows: + + + + tcp-stream + + + A buffered bi-directional stream over a TCP/IP connection. + tcp-stream is a subclass of stream, and you can read and write to it + using all the usual stream functions. Created by (make-socket + :addess-family :internet :type :stream :connect :active ...) or by + (accept-connection ...). + + + + + file-socket-stream + + + A buffered bi-directional stream over a "UNIX domain" + connection. file-socket-stream is a subclass of stream, and you can + read and write to it using all the usual stream functions. Created + by (make-socket :address-family :file :type :stream :connect :active + ...) or by (accept-connection ...), + + + + + listener-socket + + + A passive socket used to listen for incoming TCP/IP + connections on a particular port. A listener-socket is not a stream. + It doesn't support I/O. It can only be used to create new + tcp-streams by accept-connection. Created by (make-socket :type + :stream :connect :passive ...) + + + + + file-listener-socket + + + A passive socket used to listen for incoming UNIX domain + connections named by a file in the local filesystem. A + listener-socket is not a stream. It doesn't support I/O. It can + only be used to create new file-socket-streams by accept-connection. + Created by (make-socket :address-family :file :type :stream :connect + :passive ...) + + + + + udp-socket + + + A socket representing a packet-based UDP/IP connection. A + udp-socket supports I/O but it is not a stream. Instead, you must + use the special functions send-to and receive-from to read and write + to it. Created by (make-socket :type :datagram ...) + + + Sockets Dictionary - - - MAKE-SOCKET - make-socket - Name - MAKE-SOCKET — - Function - Synopsis - -make-socket - &key address-family type connect eol format + + + make-socket + + + MAKE-SOCKET + + Function + + + + make-socket + &key; address-family type connect eol format remote-host remote-port local-host local-port local-filename remote-filename keepalive reuse-address nodelay broadcast linger - backlog - - Arguments and Values - address-family - The address/protocol family of this socket. Currentlyonly :internet (the default), meaning IP, and :file,referring to UNIX domain addresses, are supported. - type - One of :stream (the default) to request aconnection-oriented socket, or :datagram to request aconnectionless socket. The default is :stream. - connect - This argument is only relevant to sockets of type:stream. One of :active (the default) to request a :passiveto request a file or TCP listener socket. - eol - This argument is currently ignored (it is accepted forcompatibility with Franz Allegro). - format - One of :text (the default), :binary, or :bivalent.This argument is ignored for :stream sockets for now, as:stream sockets are currently always bivalent (i.e. theysupport both character and byte I/O). For :datagram sockets,the format determines the type of buffer created byreceive-from. - remote-host - Required for TCP streams, it specifies the host toconnect to (in any format acceptable to lookup-hostname).Ignored for listener sockets. For UDP sockets, it can beused to specify a default host for subsequent calls tosend-to or receive-from. - remote-port - Required for TCP streams, it specifies the port toconnect to (in any format acceptable to lookup-port).Ignored for listener sockets. For UDP sockets, it can beused to specify a default port for subsequent calls to forsubsequent calls to send-to or receive-from. - remote-filename - Required for file-socket streams, it specifies thename of a file in the local filesystem (e.g., NOT mountedvia NFS, AFP, SMB, ...) which names and controls access to aUNIX-domain socket. - local-host - Allows you to specify a local host address for alistener or UDP socket, for the rare case where you want torestrict connections to those coming to a specific localaddress for security reasons. - local-port - Specify a local port for a socket. Most useful forlistener sockets, where it is the port on which the socketwill listen for connections. - local-filename - Required for file-listener-sockets. Specifies the nameof a file in the local filesystem which is used to name aUNIX-domain socket. The actual filesystem file should notpreviously exist when the file-listener-socket is created;its parent directory should exist and be writable by thecaller. The file used to name the socket will be deletedwhen the file-listener-socket is closed. - keepalive - If true, enables the periodic transmission of"keepalive" messages. - reuse-address - If true, allows the reuse of local ports in listenersockets, overriding some TCP/IP protocol specifications. Youwill need this if you are debugging a server.. - nodelay - If true, disables Nagle's algorithm, which triesto minimize TCP packet fragmentation by introducingtransmission delays in the absence of replies. Try settingthis if you are using a protocol which involves sending asteady stream of data with no replies and are seeingsignificant degradations in throughput. - broadcast - If true, requests permission to broadcast datagrams ona UDP socket. - linger - If specified and non-nil, should be the number ofseconds the OS is allowed to wait for data to be pushedthrough when a close is done. Only relevant for TCP sockets. - backlog - For a listener socket, specifies the number ofconnections which can be pending but not accepted. Thedefault is 5, which is also the maximum on some operatingsystems. - - - Description - Creates and returns a new socket - - - - ACCEPT-CONNECTION - accept-connection - Name - ACCEPT-CONNECTION — - Function - Synopsis - -accept-connection - (socket listener-socket) &key wait - - Arguments and Values - socket - The listener-socket to listen on. - wait - If true (the default), and there are no connectionswaiting to be accepted, waits until one arrives. If false,returns NIL immediately. - - - Description - Extracts the first connection on the queue of pending -connections, accepts it (i.e. completes the connection startup -protocol) and returns a new tcp-stream or file-socket-stream -representing the newly established connection. The tcp stream -inherits any properties of the listener socket that are relevant -(e.g. :keepalive, :nodelay, etc.) The original listener socket -continues to be open listening for more connections, so you can -call accept-connection on it again. - - - - DOTTED-TO-IPADDR - dotted-to-ipaddr - Name - DOTTED-TO-IPADDR — - Function - Synopsis - -dotted-to-ipaddr - dotted &key errorp - - Arguments and Values - dotted - A string representing an IP address in the"nn.nn.nn.nn" format - errorp - If true (the default) an error is signaled if dottedis invalid. If false, NIL is returned. - - - Description - Converts a dotted-string representation of a host address to -a 32-bit unsigned IP address. - - - - IPADDR-TO-DOTTED - ipaddr-to-dotted - Name - IPADDR-TO-DOTTED — - Function - Synopsis - -ipaddr-to-dotted - ipaddr &key values - - Arguments and Values - ipaddr - A 32-bit integer representing an internet host address - values - If false (the default), returns a string in the form"nn.nn.nn.nn". If true, returns four valuesrepresenting the four octets of the address as unsigned8-bit integers. - - - Description - Converts a 32-bit unsigned IP address into octets. - - - - IPADDR-TO-HOSTNAME - ipaddr-to-hostname - Name - IPADDR-TO-HOSTNAME — - Function - Synopsis - -ipaddr-to-hostname - ipaddr &key ignore-cache - - Arguments and Values - ipaddr - a 32-bit integer representing an internet host address - ignore-cache - This argument is ignored (it is accepted forcompatibility with Franz Allegro) - - - Description - Converts a 32-bit unsigned IP address into a host name -string - - - - LOOKUP-HOSTNAME - lookup-hostname - Name - LOOKUP-HOSTNAME — - Function - Synopsis - -lookup-hostname - host - - Arguments and Values - host - Specifies the host. It can be either a host namestring such as "clozure.com", or a dotted addressstring such as "192.168.0.1", or a 32-bit unsignedIP address such as 3232235521. - - - Description - Converts a host spec in any of the acceptable formats into a -32-bit unsigned IP address - - - - LOOKUP-PORT - lookup-port - Name - LOOKUP-PORT — - Function - Synopsis - -lookup-port - port protocol - - Arguments and Values - port - Specifies the port. It can be either a string, such as"http" or a symbol, such as :http, or an unsignedport number. Note that a string is case-sensitive. A symbolis lowercased before lookup. - protocol - Must be one of "tcp" or "udp". - - - Description - Finds the port number for the specified port and protocol - - - - RECEIVE-FROM - receive-from - Name - RECEIVE-FROM — - Function - Synopsis - -receive-from - (socket udp-socket) size &key buffer - extract offset - - Arguments and Values - socket - The socket to read from - size - Maximum number of bytes to read. If the packet islarger than this, any extra bytes are discarded. - buffer - If specified, must be either a string or a byte vectorwhich will be used to read in the data. If not specified, anew buffer will be created (of type determined bysocket-format). - extract - If true, the subsequence of the buffer correspondingonly to the data read in is extracted and returned as thefirst value. If false (the default) the original buffer isreturned even if it is only partially filled. - offset - Specifies the start offset into the buffer at whichdata is to be stored. The default is 0. - - - Description - Reads a UDP packet from a socket. If no packets are -available, waits for a packet to arrive. Returns four values: - - The buffer with the data - The number of bytes read - The 32-bit unsigned IP address of the sender of the data - The port number of the sender of the data - - - - - SEND-TO - send-to - Name - SEND-TO — - Function - Synopsis - -send-to - (socket udp-socket) buffer size &key remote-host - remote-port offset - - Arguments and Values - socket - The socket to write to - buffer - A vector containing the data to send. It must beeither a string or a byte vector (either one is acceptableregardless of the stream format). - size - Number of bytes to send - remote-host - The host to send the packet to, in any formatacceptable to lookup-hostname. The default is the remotehost specified in the call to make-socket. - remote-port - The port to send the packet to, in any formatacceptable to lookup-port. The default is the remote portspecified in the call to make-socket. - offset - The offset in the buffer where the packet data starts - - - Description - Send a UDP packet over a socket. - - - - SHUTDOWN - shutdown - Name - SHUTDOWN — - Function - Synopsis - -shutdown - socket &key direction - - Arguments and Values - socket - The socket to shut down (typically a tcp-stream) - direction - One of :input to disallow further input, or :output todisallow further output. - - - Description - Shuts down part of a bidirectional connection. This is -useful if e.g. you need to read responses after sending an -end-of-file signal. - - - - SOCKET-OS-FD - socket-os-fd - Name - SOCKET-OS-FD — - Function - Synopsis - -socket-os-fd - socket - - Arguments and Values - socket - The socket - - - Description - Returns the native OS's representation of the socket, or -NIL if the socket is closed. On Unix, this is the Unix 'file -descriptor', a small non-negative integer. Note that it is -rather dangerous to mess around with tcp-stream fd's, as there -is all sorts of buffering and asynchronous I/O going on above the -OS level. listener-socket and udp-socket fd's are safer to -mess with directly as there is less magic going on. - - - - REMOTE-HOST - remote-host - Name - REMOTE-HOST — - Function - Synopsis - -remote-host - socket - - Arguments and Values - socket - The socket - - - Description - Returns the 32-bit unsigned IP address of the remote host, -or NIL if the socket is not connected. - - - - REMOTE-PORT - remote-port - Name - REMOTE-PORT — - Function - Synopsis - -remote-port - socket - - Arguments and Values - socket - The socket - - - Description - Returns the remote port number, or NIL if the socket is not -connected. - - - - LOCAL-HOST - local-host - Name - LOCAL-HOST — - Function - Synopsis - -local-host - socket - - Arguments and Values - socket - The socket - - - Description - Returns 32-bit unsigned IP address of the local host. - - - - LOCAL-PORT - local-port - Name - LOCAL-PORT — - Function - Synopsis - -local-port - socket - - Arguments and Values - socket - The socket - - - Description - Returns the local port number - - - - SOCKET-ADDRESS-FAMILY - socket-address-family - Name - SOCKET-ADDRESS-FAMILY — - Function - Synopsis - -socket-address-family - socket - - Arguments and Values - socket - The socket - - - Description - Returns :internet or :file, as appropriate. - - - - SOCKET-CONNECT - socket-connect - Name - SOCKET-CONNECT — - Function - Synopsis - -socket-connect - socket - - Arguments and Values - socket - The socket - - - Description - Returns :active for tcp-stream, :passive for -listener-socket, and NIL for udp-socket - - - - SOCKET-FORMAT - socket-format - Name - SOCKET-FORMAT — - Function - Synopsis - -socket-format - socket - - Arguments and Values - socket - The socket - - - Description - Returns the socket format as specified by the :format -argument to make-socket. - - - - SOCKET-TYPE - socket-type - Name - SOCKET-TYPE — - Function - Synopsis - -socket-type - socket - - Arguments and Values - socket - The socket - - - Description - returns :stream for tcp-stream and listener-socket, and -:datagram for udp-socket. - - - - SOCKET-ERROR - socket-error - Name - SOCKET-ERROR — - Class - Description - The class of OS errors signaled by socket functions - Superclasses - simple-error - - - - SOCKET-ERROR-CODE - socket-error-code - Name - SOCKET-ERROR-CODE — - Function - Synopsis - -socket-error-code - socket-error - - Arguments and Values - socket-error - the condition - - - Description - The OS error code of the error - - - - SOCKET-ERROR-IDENTIFIER - socket-error-identifier - Name - SOCKET-ERROR-IDENTIFIER — - Function - Synopsis - -socket-error-identifier - socket-error - - Arguments and Values - socket-error - the condition - - - Description - A symbol representing the error code in a more -OS-independent way. - One of: :address-in-use :connection-aborted :no-buffer-space -:connection-timed-out :connection-refused :host-unreachable -:host-down :network-down :address-not-available :network-reset -:connection-reset :shutdown :access-denied or :unknown. - - - - SOCKET-ERROR-SITUATION - socket-error-situation - Name - SOCKET-ERROR-SITUATION — - Function - Synopsis - -socket-error-situation - socket-error - - Arguments and Values - socket-error - the condition - - - Description - A string describing the context where the error happened. On -Linux, this is the name of the system call which returned the -error. - - - - CLOSE - close - Name - CLOSE — - Method - Synopsis - -close - (socket socket) &key abort - - Arguments and Values - socket - The socket to close - abort - If false (the default), closes the socket in anorderly fashion, finishing up any buffered pending I/O,before closing the connection. If true, aborts/ignorespending I/O. (For listener and udp sockets, this argument iseffectively ignored since there is never any buffered I/O toclean up). - - - Description - The close generic function can be applied to sockets. It -releases the operating system resources associated with the -socket. - - - - WITH-OPEN-SOCKET - with-open-socket - Name - WITH-OPEN-SOCKET — - Macro - Synopsis - -with-open-socket - (var . make-socket-args) &body body - - Arguments and Values - var - variable to bind - make-socket-args - arguments suitable for passing to make-socket - body - body to execute - - - Description - executes body with var bound to the result of applying -make-socket to make-socket-args. The socket gets closed on exit. - + backlog + + + + Arguments and Values + + + + address-family + + + The address/protocol family of this socket. Currently + only :internet (the default), meaning IP, and :file, + referring to UNIX domain addresses, are supported. + + + + + type + + + One of :stream (the default) to request a + connection-oriented socket, or :datagram to request a + connectionless socket. The default is :stream. + + + + + connect + + + This argument is only relevant to sockets of type + :stream. One of :active (the default) to request a :passive + to request a file or TCP listener socket. + + + + + eol + + + This argument is currently ignored (it is accepted for + compatibility with Franz Allegro). + + + + + format + + + One of :text (the default), :binary, or :bivalent. + This argument is ignored for :stream sockets for now, as + :stream sockets are currently always bivalent (i.e. they + support both character and byte I/O). For :datagram sockets, + the format determines the type of buffer created by + receive-from. + + + + + remote-host + + + Required for TCP streams, it specifies the host to + connect to (in any format acceptable to lookup-hostname). + Ignored for listener sockets. For UDP sockets, it can be + used to specify a default host for subsequent calls to + send-to or receive-from. + + + + + remote-port + + + Required for TCP streams, it specifies the port to + connect to (in any format acceptable to lookup-port). + Ignored for listener sockets. For UDP sockets, it can be + used to specify a default port for subsequent calls to for + subsequent calls to send-to or receive-from. + + + + + remote-filename + + + Required for file-socket streams, it specifies the + name of a file in the local filesystem (e.g., NOT mounted + via NFS, AFP, SMB, ...) which names and controls access to a + UNIX-domain socket. + + + + + local-host + + + Allows you to specify a local host address for a + listener or UDP socket, for the rare case where you want to + restrict connections to those coming to a specific local + address for security reasons. + + + + + local-port + + + Specify a local port for a socket. Most useful for + listener sockets, where it is the port on which the socket + will listen for connections. + + + + + local-filename + + + Required for file-listener-sockets. Specifies the name + of a file in the local filesystem which is used to name a + UNIX-domain socket. The actual filesystem file should not + previously exist when the file-listener-socket is created; + its parent directory should exist and be writable by the + caller. The file used to name the socket will be deleted + when the file-listener-socket is closed. + + + + + keepalive + + + If true, enables the periodic transmission of + "keepalive" messages. + + + + + reuse-address + + + If true, allows the reuse of local ports in listener + sockets, overriding some TCP/IP protocol specifications. You + will need this if you are debugging a server.. + + + + + nodelay + + + If true, disables Nagle's algorithm, which tries + to minimize TCP packet fragmentation by introducing + transmission delays in the absence of replies. Try setting + this if you are using a protocol which involves sending a + steady stream of data with no replies and are seeing + significant degradations in throughput. + + + + + broadcast + + + If true, requests permission to broadcast datagrams on + a UDP socket. + + + + + linger + + + If specified and non-nil, should be the number of + seconds the OS is allowed to wait for data to be pushed + through when a close is done. Only relevant for TCP sockets. + + + + + backlog + + + For a listener socket, specifies the number of + connections which can be pending but not accepted. The + default is 5, which is also the maximum on some operating + systems. + + + + + + + Description + + Creates and returns a new socket + + + + + + accept-connection + + + ACCEPT-CONNECTION + + Function + + + + accept-connection + (socket listener-socket) &key; wait + + + + Arguments and Values + + + + socket + + + The listener-socket to listen on. + + + + + wait + + + If true (the default), and there are no connections + waiting to be accepted, waits until one arrives. If false, + returns NIL immediately. + + + + + + + Description + + Extracts the first connection on the queue of pending + connections, accepts it (i.e. completes the connection startup + protocol) and returns a new tcp-stream or file-socket-stream + representing the newly established connection. The tcp stream + inherits any properties of the listener socket that are relevant + (e.g. :keepalive, :nodelay, etc.) The original listener socket + continues to be open listening for more connections, so you can + call accept-connection on it again. + + + + + + dotted-to-ipaddr + + + DOTTED-TO-IPADDR + + Function + + + + dotted-to-ipaddr + dotted &key; errorp + + + + Arguments and Values + + + + dotted + + + A string representing an IP address in the + "nn.nn.nn.nn" format + + + + + errorp + + + If true (the default) an error is signaled if dotted + is invalid. If false, NIL is returned. + + + + + + + Description + + Converts a dotted-string representation of a host address to + a 32-bit unsigned IP address. + + + + + + ipaddr-to-dotted + + + IPADDR-TO-DOTTED + + Function + + + + ipaddr-to-dotted + ipaddr &key; values + + + + Arguments and Values + + + + ipaddr + + + A 32-bit integer representing an internet host address + + + + + values + + + If false (the default), returns a string in the form + "nn.nn.nn.nn". If true, returns four values + representing the four octets of the address as unsigned + 8-bit integers. + + + + + + + Description + + Converts a 32-bit unsigned IP address into octets. + + + + + + ipaddr-to-hostname + + + IPADDR-TO-HOSTNAME + + Function + + + + ipaddr-to-hostname + ipaddr &key; ignore-cache + + + + Arguments and Values + + + + ipaddr + + + a 32-bit integer representing an internet host address + + + + + ignore-cache + + + This argument is ignored (it is accepted for + compatibility with Franz Allegro) + + + + + + + Description + + Converts a 32-bit unsigned IP address into a host name + string + + + + + + lookup-hostname + + + LOOKUP-HOSTNAME + + Function + + + + lookup-hostname + host + + + + Arguments and Values + + + + host + + + Specifies the host. It can be either a host name + string such as "clozure.com", or a dotted address + string such as "192.168.0.1", or a 32-bit unsigned + IP address such as 3232235521. + + + + + + + Description + + Converts a host spec in any of the acceptable formats into a + 32-bit unsigned IP address + + + + + + lookup-port + + + LOOKUP-PORT + + Function + + + + lookup-port + port protocol + + + + Arguments and Values + + + + port + + + Specifies the port. It can be either a string, such as + "http" or a symbol, such as :http, or an unsigned + port number. Note that a string is case-sensitive. A symbol + is lowercased before lookup. + + + + + protocol + + + Must be one of "tcp" or "udp". + + + + + + + Description + + Finds the port number for the specified port and protocol + + + + + + receive-from + + + RECEIVE-FROM + + Function + + + + receive-from + (socket udp-socket) size &key; buffer + extract offset + + + + Arguments and Values + + + + socket + + + The socket to read from + + + + + size + + + Maximum number of bytes to read. If the packet is + larger than this, any extra bytes are discarded. + + + + + buffer + + + If specified, must be either a string or a byte vector + which will be used to read in the data. If not specified, a + new buffer will be created (of type determined by + socket-format). + + + + + extract + + + If true, the subsequence of the buffer corresponding + only to the data read in is extracted and returned as the + first value. If false (the default) the original buffer is + returned even if it is only partially filled. + + + + + offset + + + Specifies the start offset into the buffer at which + data is to be stored. The default is 0. + + + + + + + Description + + Reads a UDP packet from a socket. If no packets are + available, waits for a packet to arrive. Returns four values: + + + + The buffer with the data + + + + The number of bytes read + + + + The 32-bit unsigned IP address of the sender of the data + + + + The port number of the sender of the data + + + + + + + + send-to + + + SEND-TO + + Function + + + + send-to + (socket udp-socket) buffer size &key; remote-host + remote-port offset + + + + Arguments and Values + + + + socket + + + The socket to write to + + + + + buffer + + + A vector containing the data to send. It must be + either a string or a byte vector (either one is acceptable + regardless of the stream format). + + + + + size + + + Number of bytes to send + + + + + remote-host + + + The host to send the packet to, in any format + acceptable to lookup-hostname. The default is the remote + host specified in the call to make-socket. + + + + + remote-port + + + The port to send the packet to, in any format + acceptable to lookup-port. The default is the remote port + specified in the call to make-socket. + + + + + offset + + + The offset in the buffer where the packet data starts + + + + + + + Description + + Send a UDP packet over a socket. + + + + + + shutdown + + + SHUTDOWN + + Function + + + + shutdown + socket &key; direction + + + + Arguments and Values + + + + socket + + + The socket to shut down (typically a tcp-stream) + + + + + direction + + + One of :input to disallow further input, or :output to + disallow further output. + + + + + + + Description + + Shuts down part of a bidirectional connection. This is + useful if e.g. you need to read responses after sending an + end-of-file signal. + + + + + + socket-os-fd + + + SOCKET-OS-FD + + Function + + + + socket-os-fd + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + Returns the native OS's representation of the socket, or + NIL if the socket is closed. On Unix, this is the Unix 'file + descriptor', a small non-negative integer. Note that it is + rather dangerous to mess around with tcp-stream fd's, as there + is all sorts of buffering and asynchronous I/O going on above the + OS level. listener-socket and udp-socket fd's are safer to + mess with directly as there is less magic going on. + + + + + + remote-host + + + REMOTE-HOST + + Function + + + + remote-host + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + Returns the 32-bit unsigned IP address of the remote host, + or NIL if the socket is not connected. + + + + + + remote-port + + + REMOTE-PORT + + Function + + + + remote-port + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + Returns the remote port number, or NIL if the socket is not + connected. + + + + + + local-host + + + LOCAL-HOST + + Function + + + + local-host + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + Returns 32-bit unsigned IP address of the local host. + + + + + + local-port + + + LOCAL-PORT + + Function + + + + local-port + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + Returns the local port number + + + + + + socket-address-family + + + SOCKET-ADDRESS-FAMILY + + Function + + + + socket-address-family + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + Returns :internet or :file, as appropriate. + + + + + + socket-connect + + + SOCKET-CONNECT + + Function + + + + socket-connect + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + Returns :active for tcp-stream, :passive for + listener-socket, and NIL for udp-socket + + + + + + socket-format + + + SOCKET-FORMAT + + Function + + + + socket-format + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + Returns the socket format as specified by the :format + argument to make-socket. + + + + + + socket-type + + + SOCKET-TYPE + + Function + + + + socket-type + socket + + + + Arguments and Values + + + + socket + + + The socket + + + + + + + Description + + returns :stream for tcp-stream and listener-socket, and + :datagram for udp-socket. + + + + + + socket-error + + + SOCKET-ERROR + + Class + + + + Description + + The class of OS errors signaled by socket functions + + + + Superclasses + + simple-error + + + + + + socket-error-code + + + SOCKET-ERROR-CODE + + Function + + + + socket-error-code + socket-error + + + + Arguments and Values + + + + socket-error + + + the condition + + + + + + + Description + + The OS error code of the error + + + + + + socket-error-identifier + + + SOCKET-ERROR-IDENTIFIER + + Function + + + + socket-error-identifier + socket-error + + + + Arguments and Values + + + + socket-error + + + the condition + + + + + + + Description + + A symbol representing the error code in a more + OS-independent way. + + One of: :address-in-use :connection-aborted :no-buffer-space + :connection-timed-out :connection-refused :host-unreachable + :host-down :network-down :address-not-available :network-reset + :connection-reset :shutdown :access-denied or :unknown. + + + + + + socket-error-situation + + + SOCKET-ERROR-SITUATION + + Function + + + + socket-error-situation + socket-error + + + + Arguments and Values + + + + socket-error + + + the condition + + + + + + + Description + + A string describing the context where the error happened. On + Linux, this is the name of the system call which returned the + error. + + + + + + close + + + CLOSE + + Method + + + + close + (socket socket) &key; abort + + + + Arguments and Values + + + + socket + + + The socket to close + + + + + abort + + + If false (the default), closes the socket in an + orderly fashion, finishing up any buffered pending I/O, + before closing the connection. If true, aborts/ignores + pending I/O. (For listener and udp sockets, this argument is + effectively ignored since there is never any buffered I/O to + clean up). + + + + + + + Description + + The close generic function can be applied to sockets. It + releases the operating system resources associated with the + socket. + + + + + + with-open-socket + + + WITH-OPEN-SOCKET + + Macro + + + + with-open-socket + (var . make-socket-args) &body; body + + + + Arguments and Values + + + + var + + + variable to bind + + + + + make-socket-args + + + arguments suitable for passing to make-socket + + + + + body + + + body to execute + + + + + + + Description + + executes body with var bound to the result of applying + make-socket to make-socket-args. The socket gets closed on exit. + + @@ -3805,5 +5805,5 @@ - Limitations and known bugs + Limitations and known bugs OpenMCL and the external processs may get @@ -3821,197 +5821,421 @@ External-Program Dictionary - - - RUN-PROGRAM - run-program - Name - RUN-PROGRAM — Invokes an external program as an OS subprocess -of lisp. - Function - Synopsis - -run-program - program args &key (wait t) pty input + + + run-program + + + + RUN-PROGRAM + Invokes an external program as an OS subprocess + of lisp. + Function + + + + run-program + program args &key; (wait t) pty input if-input-does-not-exist output (if-output-exists :error) (error - :output) (if-error-exists :error) status-hook - - Arguments and Values - program - A string or pathname which denotes an executable file.The PATH environment variable is used to find programs whosename doesn't contain a directory component. - args - A list of simple-strings - wait - Indicates whether or not run-program should wait forthe EXTERNAL-PROCESS to complete or should returnimmediately. - pty - This option is accepted but currently ignored;it's intended to make it easier to run external programsthat need to interact with a terminal device. - input - Selects the input source used by the EXTERNAL-PROCESS.May be any of the following: - - NIL Specifies that a null input stream (e.g.,/dev/null) should be used. - T Specifies that the EXTERNAL-PROCESS should usethe input source with which OpenMCL was invoked. - A string or pathname. Specifies that theEXTERNAL-PROCESS should receive its input from the namedexisting file. - :STREAM Creates a Lisp stream opened for characteroutput. Any data written to this stream (accessible asthe EXTERNAL-PROCESS-INPUT-STREAM of theEXTERNAL-PROCESS object) appears as input to theexternal process. - A stream. Specifies that the lisp stream shouldprovide input to the EXTERNAL-PROCESS. - - - - if-input-does-not-exist - If the input argument specifies the name of anexisting file, this argument is used as theif-does-not-exist argument to OPEN when that file is opened. - output - Specifies where standard output from the externalprocess should be sent. Analogous to input above. - if-output-exists - If output is specified as a string or pathname, thisargument is used as the if-exists argument to OPEN when thatfile is opened. - error - Specifies where error output from the external processshould be sent. In addition to the values allowed foroutput, the keyword :OUTPUT can be used to indicate thaterror output should be sent where standard output goes. - if-error-exists - Analogous to if-output-exists. - status-hook - A user-defined function of one argument (theEXTERNAL-PROCESS structure.) This function is calledwhenever OpenMCL detects a change in the staus of theEXTERNAL-PROCESS. - - - Description - Runs the specified program in an external (Unix) process, -returning an object of type EXTERNAL-PROCESS if successful. - - - - SIGNAL-EXTERNAL-PROCESS - signal-external-process - Name - SIGNAL-EXTERNAL-PROCESS — - Function - Synopsis - -signal-external-process - proc signal-number - - Arguments and Values - proc - An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. - signal - A small integer. - - - Description - Sends the specified "signal" to the specified -external process. (Typically, it would only be useful tocall -this function if the EXTERNAL-PROCESS was created with :WAIT -NIL. ) Returns T if successful; signals an error otherwise. - - - - EXTERNAL-PROCESS-ID - external-process-id - Name - EXTERNAL-PROCESS-ID — Returns the "process ID" of an OS subprocess, -a positive integer which identifies it. - Function - Synopsis - -external-process-id - proc - - Arguments and Values - proc - An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. - - - Description - Returns the process id assigned to -the external process by the operating system. This is typically -a positive, 16-bit number. - - - - EXTERNAL-PROCESS-INPUT-STREAM - external-process-input-stream - Name - EXTERNAL-PROCESS-INPUT-STREAM — Returns the lisp stream which is used to write -input to a given OS subprocess, if it has one. - Function - Synopsis - -external-process-input-stream - proc - - Arguments and Values - proc - An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. - - - Description - Returns the stream created when the input argument to -run-program is specified as :STREAM. - - - - EXTERNAL-PROCESS-OUTPUT-STREAM - external-process-output-stream - Name - EXTERNAL-PROCESS-OUTPUT-STREAM — Returns the lisp stream which is used to read -output from an OS subprocess, if there is one. - Function - Synopsis - -external-process-output-stream - proc - - Arguments and Values - proc - An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. - - - Description - Returns the stream created when the output argument to -run-program is specified as :STREAM. - - - - EXTERNAL-PROCESS-ERROR-STREAM - external-process-error-stream - Name - EXTERNAL-PROCESS-ERROR-STREAM — Returns the stream which is used to read -"error" output from a given OS subprocess, if it has -one. - Function - Synopsis - -external-process-error-stream - proc - - Arguments and Values - proc - An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. - - - Description - Returns the stream created when the error argument to -run-program is specified as :STREAM. - - - - EXTERNAL-PROCESS-STATUS - external-process-status - Name - EXTERNAL-PROCESS-STATUS — Returns information about whether an OS -subprocess is running; or, if not, why not; and what its -result code was if it completed. - Function - Synopsis - -external-process-status - proc - - Arguments and Values - proc - An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. - - - Description - Returns, as multiple values, a keyword denoting the status -of the external process (one of :running, :stopped, :signaled, or -:exited), and the exit code or terminating signal if the first -value is other than :running. - + :output) (if-error-exists :error) status-hook + + + + Arguments and Values + + + + program + + + A string or pathname which denotes an executable file. + The PATH environment variable is used to find programs whose + name doesn't contain a directory component. + + + + + args + + + A list of simple-strings + + + + + wait + + + Indicates whether or not run-program should wait for + the EXTERNAL-PROCESS to complete or should return + immediately. + + + + + pty + + + This option is accepted but currently ignored; + it's intended to make it easier to run external programs + that need to interact with a terminal device. + + + + + input + + + Selects the input source used by the EXTERNAL-PROCESS. + May be any of the following: + + + + NIL Specifies that a null input stream (e.g., + /dev/null) should be used. + + + + T Specifies that the EXTERNAL-PROCESS should use + the input source with which OpenMCL was invoked. + + + + A string or pathname. Specifies that the + EXTERNAL-PROCESS should receive its input from the named + existing file. + + + + :STREAM Creates a Lisp stream opened for character + output. Any data written to this stream (accessible as + the EXTERNAL-PROCESS-INPUT-STREAM of the + EXTERNAL-PROCESS object) appears as input to the + external process. + + + + A stream. Specifies that the lisp stream should + provide input to the EXTERNAL-PROCESS. + + + + + + + if-input-does-not-exist + + + If the input argument specifies the name of an + existing file, this argument is used as the + if-does-not-exist argument to OPEN when that file is opened. + + + + + output + + + Specifies where standard output from the external + process should be sent. Analogous to input above. + + + + + if-output-exists + + + If output is specified as a string or pathname, this + argument is used as the if-exists argument to OPEN when that + file is opened. + + + + + error + + + Specifies where error output from the external process + should be sent. In addition to the values allowed for + output, the keyword :OUTPUT can be used to indicate that + error output should be sent where standard output goes. + + + + + if-error-exists + + + Analogous to if-output-exists. + + + + + status-hook + + + A user-defined function of one argument (the + EXTERNAL-PROCESS structure.) This function is called + whenever OpenMCL detects a change in the staus of the + EXTERNAL-PROCESS. + + + + + + + Description + + Runs the specified program in an external (Unix) process, + returning an object of type EXTERNAL-PROCESS if successful. + + + + + + signal-external-process + + + + SIGNAL-EXTERNAL-PROCESS + + Function + + + + signal-external-process + proc signal-number + + + + Arguments and Values + + + + proc + + + An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. + + + + + signal + + + A small integer. + + + + + + + Description + + Sends the specified "signal" to the specified + external process. (Typically, it would only be useful tocall + this function if the EXTERNAL-PROCESS was created with :WAIT + NIL. ) Returns T if successful; signals an error otherwise. + + + + + + external-process-id + + + + EXTERNAL-PROCESS-ID + Returns the "process ID" of an OS subprocess, + a positive integer which identifies it. + Function + + + + external-process-id + proc + + + + Arguments and Values + + + + proc + + + An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. + + + + + + + Description + + Returns the process id assigned to + the external process by the operating system. This is typically + a positive, 16-bit number. + + + + + + external-process-input-stream + + + + EXTERNAL-PROCESS-INPUT-STREAM + Returns the lisp stream which is used to write + input to a given OS subprocess, if it has one. + Function + + + + external-process-input-stream + proc + + + + Arguments and Values + + + + proc + + + An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. + + + + + + + Description + + Returns the stream created when the input argument to + run-program is specified as :STREAM. + + + + + + external-process-output-stream + + + + EXTERNAL-PROCESS-OUTPUT-STREAM + Returns the lisp stream which is used to read + output from an OS subprocess, if there is one. + Function + + + + external-process-output-stream + proc + + + + Arguments and Values + + + + proc + + + An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. + + + + + + + Description + + Returns the stream created when the output argument to + run-program is specified as :STREAM. + + + + + + external-process-error-stream + + + + EXTERNAL-PROCESS-ERROR-STREAM + Returns the stream which is used to read + "error" output from a given OS subprocess, if it has + one. + Function + + + + external-process-error-stream + proc + + + + Arguments and Values + + + + proc + + + An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. + + + + + + + Description + + Returns the stream created when the error argument to + run-program is specified as :STREAM. + + + + + + external-process-status + + + + EXTERNAL-PROCESS-STATUS + Returns information about whether an OS + subprocess is running; or, if not, why not; and what its + result code was if it completed. + Function + + + + external-process-status + proc + + + + Arguments and Values + + + + proc + + + An EXTERNAL-PROCESS, as returned by RUN-PROGRAM. + + + + + + + Description + + Returns, as multiple values, a keyword denoting the status + of the external process (one of :running, :stopped, :signaled, or + :exited), and the exit code or terminating signal if the first + value is other than :running. + + @@ -4038,368 +6262,145 @@ Here's a list of some classes which you might wish for your new stream class to inherit from: - - - - - - fundamental-stream - - - - fundamental-input-stream - - - - fundamental-output-stream - - - - fundamental-character-stream - - - - fundamental-binary-stream - - - - fundamental-character-input-stream - - - - fundamental-character-output-stream - - - - fundamental-binary-input-stream - - - - fundamental-binary-output-stream - - - - ccl::buffered-stream-mixin - - - - ccl::buffered-input-stream-mixin - - - - ccl::buffered-output-stream-mixin - - - - ccl::buffered-io-stream-mixin - - - - ccl::buffered-character-input-stream-mixin - - - - ccl::buffered-character-output-stream-mixin - - - - ccl::buffered-character-io-stream-mixin - - - - ccl::buffered-binary-input-stream-mixin - - - - ccl::buffered-binary-output-stream-mixin - - - - ccl::buffered-binary-io-stream-mixin - - - - file-stream - - - - file-input-stream - - - - file-output-stream - - - - file-io-stream - - - - file-character-input-stream - - - - file-character-output-stream - - - - file-charcter-io-stream - - - - file-binary-input-stream - - - - file-binary-output-stream - - - - file-binary-io-stream - - - - ccl::fd-stream - - - - ccl::fd-input-stream - - - - ccl::fd-output-stream - - - - ccl::fd-io-stream - - - - ccl::fd-character-input-stream - - - - ccl::fd-character-output-stream - - - - ccl::fd-character-io-stream - - - - ccl::fd-binary-input-stream - - - - ccl::fd-binary-output-stream - - - - ccl::fd-binary-io-stream - - - - - - All of these are defined in ccl/level-1/l1-streams.lisp, except for -the ccl:file-* ones, which are in ccl/level-1/l1-sysio.lisp. - According to the original Gray streams proposal, you should inherit -from the most specific of the -fundamental-* classes which applies. Using OpenMCL, though, if you -want -buffering for better performance, which, unless you know of some -reason you wouldn't, you do, -you should instead inherit from the appropriate ccl::buffered-* class -The buffering you get this way is exactly the same as the -buffering which is used on ordinary, non-Gray streams, and force-output -will work properly on it. - Notice that -mixin suffix in the names of all the ccl::buffered-* -classes? -The suffix means that this class -is not "complete" -by itself; you still need to inherit from a fundamental-* stream, -even if you also inherit from a *-mixin stream. You might consider -making your own class like this. .... Except that they do -inherit from the fundamental-* streams, that's weird. - If you want to be able to create an instance of your class with the -:class argument to (open) and (with-open-file), you should make it -inherit from one of the file-* classes. If you do this, it's not -necessary to inherit from any of the other classes (though it won't -hurt anything), since the file-* classes already do. + + + fundamental-stream + fundamental-input-stream + fundamental-output-stream + fundamental-character-stream + fundamental-binary-stream + fundamental-character-input-stream + fundamental-character-output-stream + fundamental-binary-input-stream + fundamental-binary-output-stream + ccl::buffered-stream-mixin + ccl::buffered-input-stream-mixin + ccl::buffered-output-stream-mixin + ccl::buffered-io-stream-mixin + ccl::buffered-character-input-stream-mixin + ccl::buffered-character-output-stream-mixin + ccl::buffered-character-io-stream-mixin + ccl::buffered-binary-input-stream-mixin + ccl::buffered-binary-output-stream-mixin + ccl::buffered-binary-io-stream-mixin + file-stream + file-input-stream + file-output-stream + file-io-stream + file-character-input-stream + file-character-output-stream + file-charcter-io-stream + file-binary-input-stream + file-binary-output-stream + file-binary-io-stream + ccl::fd-stream + ccl::fd-input-stream + ccl::fd-output-stream + ccl::fd-io-stream + ccl::fd-character-input-stream + ccl::fd-character-output-stream + ccl::fd-character-io-stream + ccl::fd-binary-input-stream + ccl::fd-binary-output-stream + ccl::fd-binary-io-stream + + + All of these are defined in ccl/level-1/l1-streams.lisp, + except for the ccl:file-* ones, which are in + ccl/level-1/l1-sysio.lisp. + According to the original Gray streams proposal, you + should inherit from the most specific of the fundamental-* + classes which applies. Using OpenMCL, though, if you want + buffering for better performance, which, unless you know of some + reason you wouldn't, you do, you should instead inherit from the + appropriate ccl::buffered-* class The buffering you get this way + is exactly the same as the buffering which is used on ordinary, + non-Gray streams, and force-output will work properly on + it. + Notice that -mixin suffix in the names of all the + ccl::buffered-* classes? The suffix means that this class is + not "complete" by itself; you still need to inherit from a + fundamental-* stream, even if you also inherit from a *-mixin + stream. You might consider making your own class like this. + .... Except that they do inherit from the fundamental-* + streams, that's weird. + If you want to be able to create an instance of your class + with the :class argument to (open) and (with-open-file), you + should make it inherit from one of the file-* classes. If you + do this, it's not necessary to inherit from any of the other + classes (though it won't hurt anything), since the file-* + classes already do. When you inherit from the file-* classes, you can use -(call-next-method) in any of your methods to get the standard -behaviour. This is especially useful if you want to create a class -which performs some simple filtering operation, such -as changing everything to uppercase or to a different character -encoding. If you do this, you will definitely need to specialize -ccl::select-stream-class. Your method on ccl::stream-select-class -should accept an instance of the class, but pay no attention to its -contents, and return a symbol naming the -class to actually be instantiated. - If you need to make your functionality generic across all the -different types of stream, probably the best way -to implement it is to make it a mixin, define classes with all the -variants of input, output, io, character, and binary, which inherit -both from your mixin and from the appropriate other class, then -define a method on ccl::select-stream-class which chooses from among -those classes. - Note that some of these classes are internal -to the CLL package. If you try to inherit from those ones without -the ccl:: prefix, you'll get an error which may confuse you, calling -them "forward-referenced classes". That just means you used the -wrong symbol, so add the prefix. - Here's a list of some generic functions which you might wish to -specialize for your new stream class, and which ought to be -documented at some point. - - - - - - stream-direction stream => - - - - stream-device stream direction => - - - - stream-length stream &optionalnew => - - - - stream-position stream &optionalnew => - - - - streamp stream => boolean - - - - stream-write-char output-stream char => - - - - stream-write-entire-string output-stream string => - - - - stream-read-char input-stream => - - - - stream-unread-char input-stream char => - - - - stream-force-output output-stream => nil - - - - stream-maybe-force-output output-stream => nil - - - - stream-finish-output output-stream => nil - - - - stream-clear-output output-stream => nil - - - - close stream &keyabort => boolean - - - - stream-fresh-line stream => t - - - - stream-line-length stream => length - - - - interactive-stream-p stream => boolean - - - - stream-clear-input input-stream => nil - - - - stream-listen input-stream => boolean - - - - stream-filename stream => string - - - - ccl::select-stream-class instance in-p out-p char-p =>class - - - - - + (call-next-method) in any of your methods to get the standard + behaviour. This is especially useful if you want to create a + class which performs some simple filtering operation, such as + changing everything to uppercase or to a different character + encoding. If you do this, you will definitely need to + specialize ccl::select-stream-class. Your method on + ccl::stream-select-class should accept an instance of the class, + but pay no attention to its contents, and return a symbol naming + the class to actually be instantiated. + If you need to make your functionality generic across all + the different types of stream, probably the best way to + implement it is to make it a mixin, define classes with all the + variants of input, output, io, character, and binary, which + inherit both from your mixin and from the appropriate other + class, then define a method on ccl::select-stream-class which + chooses from among those classes. + Note that some of these classes are internal to the CLL + package. If you try to inherit from those ones without the + ccl:: prefix, you'll get an error which may confuse you, calling + them "forward-referenced classes". That just means you used the + wrong symbol, so add the prefix. + Here's a list of some generic functions which you might + wish to specialize for your new stream class, and which ought to + be documented at some point. + + stream-direction stream => + stream-device stream direction => + stream-length stream &optional; new => + stream-position stream &optional; new => + streamp stream => boolean + stream-write-char output-stream char => + stream-write-entire-string output-stream string => + stream-read-char input-stream => + stream-unread-char input-stream char => + stream-force-output output-stream => nil + stream-maybe-force-output output-stream => nil + stream-finish-output output-stream => nil + stream-clear-output output-stream => nil + close stream &key; abort => boolean + stream-fresh-line stream => t + stream-line-length stream => length + interactive-stream-p stream => boolean + stream-clear-input input-stream => nil + stream-listen input-stream => boolean + stream-filename stream => string + ccl::select-stream-class instance in-p out-p char-p => + class + The following functions are standard parts of Common Lisp, but behave in special ways with regard to Gray streams. - - - - - - open-stream-p stream => generalized-boolean - - - - input-stream-p stream => generalized-boolean - - - - output-stream-p stream => generalized-boolean - - - - stream-element-type stream => - - - - stream-error-stream => - - - - open - - - - close - - - - with-open-file - - - - - - Specifically, (open) and (with-open-file) accept a new keyword -argument, :class, which may be a symbol naming a class; the class -itself; or an instance of it. The class so given must be a subtype -of 'stream, and an instance of it with no particular contents will -be passed to ccl::select-stream-class to determine what class to -actually instantiate. - The following are standard, and do not behave specially with -regard to Gray streams, but probably should. - - - - - - stream-external-format - - - - - + + open-stream-p stream => generalized-boolean + input-stream-p stream => generalized-boolean + output-stream-p stream => generalized-boolean + stream-element-type stream => + stream-error-stream => + open + close + with-open-file + + + Specifically, (open) and (with-open-file) accept a new + keyword argument, :class, which may be a symbol naming a class; + the class itself; or an instance of it. The class so given must + be a subtype of 'stream, and an instance of it with no + particular contents will be passed to ccl::select-stream-class + to determine what class to actually instantiate. + The following are standard, and do not behave specially + with regard to Gray streams, but probably should. + + stream-external-format + @@ -4476,207 +6477,465 @@ Gray Streams Dictionary - - - CCL:STREAM-READ-LIST - stream-read-list - Name - CCL:STREAM-READ-LIST — - Generic Function - Synopsis - -stream-read-list - stream list count - - Arguments and Values - stream - a stream, presumably a fundamental-input-stream. - list - a list. When a STREAM-READ-LIST method is called byREAD-SEQUENCE, this argument is guaranteed to be a properlist. - count - a non-negative integer. When a STREAM-READ-LIST methodis called by READ-SEQUENCE, this argument is guaranteed notto be greater than the length of the list. - - - Description - Should try to read up to count elements from stream into the -list list, returning the number of elements actually read (which -may be less than count in case of a premature end-of-file.) - - - - CCL:STREAM-WRITE-LIST - stream-write-list - Name - CCL:STREAM-WRITE-LIST — - Generic Function - Synopsis - -stream-write-list - stream list count - - Arguments and Values - stream - a stream, presumably a fundamental-ouput-stream. - list - a list. When a STREAM-WRITE-LIST method is called byWRITE-SEQUENCE, this argument is guaranteed to be a properlist. - count - a non-negative integer. When a STREAM-WRITE-LISTmethod is called by WRITE-SEQUENCE, this argument isguaranteed not to be greater than the length of the list. - - - Description - should try to write the first count elements of list to -stream. The return value of this method is ignored. - - - - CCL:STREAM-READ-VECTOR - stream-read-vector - Name - CCL:STREAM-READ-VECTOR — - Generic Function - Synopsis - -stream-read-vector - stream vector start end - - Arguments and Values - stream - a stream, presumably a fundamental-input-stream - vector - a vector. When a STREAM-READ-VECTOR method is calledby READ-SEQUENCE, this argument is guaranteed to be a simpleone-dimensional array. - start - a non-negative integer. When a STREAM-READ-VECTORmethod is called by READ-SEQUENCE, this argument isguaranteed to be no greater than end and not greater thanthe length of vector. - end - a non-negative integer. When a STREAM-READ-VECTORmethod is called by READ-SEQUENCE, this argument isguaranteed to be no less than end and not greater than thelength of vector. - - - Description - should try to read successive elements from stream into -vector, starting at element start (inclusive) and continuing -through element end (exclusive.) Should return the index of the -vector element beyond the last one stored into, which may be less -than end in case of premature end-of-file. - - - - CCL:STREAM-WRITE-VECTOR - stream-write-vector - Name - CCL:STREAM-WRITE-VECTOR — - Generic Function - Synopsis - -stream-write-vector - stream vector start end - - Arguments and Values - stream - a stream, presumably a fundamental-output-stream - vector - a vector. When a STREAM-WRITE-VECTOR method is calledby WRITE-SEQUENCE, this argument is guaranteed to be asimple one-dimensional array. - start - a non-negative integer. When a STREAM-WRITE-VECTORmethod is called by WRITE-SEQUENCE, this argument isguaranteed to be no greater than end and not greater thanthe length of vector. - end - a non-negative integer. When a STREAM-WRITE-VECTORmethod is called by WRITE-SEQUENCE, this argument isguaranteed to be no less than end and not greater than thelength of vector. - - - Description - should try to write successive elements of vector to stream, -starting at element start (inclusive) and continuing through -element end (exclusive.) - - - - CCL::STREAM-DEVICE - stream-device - Name - CCL::STREAM-DEVICE — Returns the OS file descriptor associated with a -given lisp stream. - Generic Function - Synopsis - -ccl::stream-device - s direction - - Method Signatures - -ccl::stream-device - (s stream) direction => fd - - Arguments and Values - s - a stream. - direction - either :INPUT or :OUTPUT. - fd - a file descriptor, which is a non-negative integerused by the OS to refer to an open file, socket, or similarI/O connection. NIL if there is no file descriptor associatedwith s in the direction given bydirection. - - - Description - Returns the file descriptor associated with -s in the direction given by -direction. It is necessary to specify -direction because the input and output -file descriptors may be different; the most common case is when -one of them has been redirected by the Unix shell. - - - - STREAM-READ-IVECTOR - stream-read-ivector - Name - STREAM-READ-IVECTOR — - Generic Function - Synopsis - -stream-read-ivector - stream ivector start-octet max-octets - - Description - Reads up to max-octets octets from stream into ivector, -storing them at start-octet. Returns the number of octets actually -read. - Arguments - stream - An input stream. The method defined onBUFFERED-INPUT-STREAMs requires that the size in octets ofan instance of the stream's element type is 1. - ivector - Any ivector. - start-octet - A non-negative integer. - max-octets - A non-negative integer. The return value may be lessthan the value of this parameter if EOF was encountered. - - - - - - STREAM-WRITE-IVECTOR - stream-write-ivector - Name - STREAM-WRITE-IVECTOR — - Generic Function - Synopsis - -stream-write-ivector stream - ivector start-octet max-octets - - Description - Writes max-octets octets to stream from ivector, starting at -start-octet. Returns max-octets. - Arguments - stream - An input stream. The method defined onBUFFERED-OUTPUT-STREAMs requires that the size in octets ofan instance of the stream's element type is 1. - ivector - Any ivector - start-octet - A non-negative integer. - max-octet - A non-negative integer. - - - Examples - -;;; Write the contents of a (SIMPLE-ARRAY(UNSIGNED-BYTE 16) 3) + + + stream-read-list + + + + CCL:STREAM-READ-LIST + + Generic Function + + + + stream-read-list + stream list count + + + + Arguments and Values + + + + stream + + + a stream, presumably a fundamental-input-stream. + + + + + list + + + a list. When a STREAM-READ-LIST method is called by + READ-SEQUENCE, this argument is guaranteed to be a proper + list. + + + + + count + + + a non-negative integer. When a STREAM-READ-LIST method + is called by READ-SEQUENCE, this argument is guaranteed not + to be greater than the length of the list. + + + + + + + Description + + Should try to read up to count elements from stream into the + list list, returning the number of elements actually read (which + may be less than count in case of a premature end-of-file.) + + + + + + stream-write-list + + + + CCL:STREAM-WRITE-LIST + + Generic Function + + + + stream-write-list + stream list count + + + + Arguments and Values + + + + stream + + + a stream, presumably a fundamental-ouput-stream. + + + + + list + + + a list. When a STREAM-WRITE-LIST method is called by + WRITE-SEQUENCE, this argument is guaranteed to be a proper + list. + + + + + count + + + a non-negative integer. When a STREAM-WRITE-LIST + method is called by WRITE-SEQUENCE, this argument is + guaranteed not to be greater than the length of the list. + + + + + + + Description + + should try to write the first count elements of list to + stream. The return value of this method is ignored. + + + + + + stream-read-vector + + + + CCL:STREAM-READ-VECTOR + + Generic Function + + + + stream-read-vector + stream vector start end + + + + Arguments and Values + + + + stream + + + a stream, presumably a fundamental-input-stream + + + + + vector + + + a vector. When a STREAM-READ-VECTOR method is called + by READ-SEQUENCE, this argument is guaranteed to be a simple + one-dimensional array. + + + + + start + + + a non-negative integer. When a STREAM-READ-VECTOR + method is called by READ-SEQUENCE, this argument is + guaranteed to be no greater than end and not greater than + the length of vector. + + + + + end + + + a non-negative integer. When a STREAM-READ-VECTOR + method is called by READ-SEQUENCE, this argument is + guaranteed to be no less than end and not greater than the + length of vector. + + + + + + + Description + + should try to read successive elements from stream into + vector, starting at element start (inclusive) and continuing + through element end (exclusive.) Should return the index of the + vector element beyond the last one stored into, which may be less + than end in case of premature end-of-file. + + + + + + stream-write-vector + + + + CCL:STREAM-WRITE-VECTOR + + Generic Function + + + + stream-write-vector + stream vector start end + + + + Arguments and Values + + + + stream + + + a stream, presumably a fundamental-output-stream + + + + + vector + + + a vector. When a STREAM-WRITE-VECTOR method is called + by WRITE-SEQUENCE, this argument is guaranteed to be a + simple one-dimensional array. + + + + + start + + + a non-negative integer. When a STREAM-WRITE-VECTOR + method is called by WRITE-SEQUENCE, this argument is + guaranteed to be no greater than end and not greater than + the length of vector. + + + + + end + + + a non-negative integer. When a STREAM-WRITE-VECTOR + method is called by WRITE-SEQUENCE, this argument is + guaranteed to be no less than end and not greater than the + length of vector. + + + + + + + Description + + should try to write successive elements of vector to stream, + starting at element start (inclusive) and continuing through + element end (exclusive.) + + + + + + stream-device + + + + CCL::STREAM-DEVICE + Returns the OS file descriptor associated with a + given lisp stream. + Generic Function + + + + ccl::stream-device + s direction + + + + Method Signatures + + ccl::stream-device + (s stream) direction => fd + + + + Arguments and Values + + + + s + + a stream. + + + + direction + + either :INPUT or :OUTPUT. + + + + fd + + a file descriptor, which is a non-negative integer + used by the OS to refer to an open file, socket, or similar + I/O connection. NIL if there is no file descriptor associated + with s in the direction given by + direction. + + + + + + + Description + + Returns the file descriptor associated with + s in the direction given by + direction. It is necessary to specify + direction because the input and output + file descriptors may be different; the most common case is when + one of them has been redirected by the Unix shell. + + + + + + stream-read-ivector + + + + STREAM-READ-IVECTOR + + Generic Function + + + + stream-read-ivector + stream ivector start-octet max-octets + + + + Description + + Reads up to max-octets octets from stream into ivector, + storing them at start-octet. Returns the number of octets actually + read. + + + + Arguments + + + + stream + + + An input stream. The method defined on + BUFFERED-INPUT-STREAMs requires that the size in octets of + an instance of the stream's element type is 1. + + + + + ivector + + + Any ivector. + + + + + start-octet + + + A non-negative integer. + + + + + max-octets + + + A non-negative integer. The return value may be less + than the value of this parameter if EOF was encountered. + + + + + + + + + stream-write-ivector + + + + STREAM-WRITE-IVECTOR + + Generic Function + + + + stream-write-ivector stream + ivector start-octet max-octets + + + + Description + + Writes max-octets octets to stream from ivector, starting at + start-octet. Returns max-octets. + + + + Arguments + + + + stream + + + An input stream. The method defined on + BUFFERED-OUTPUT-STREAMs requires that the size in octets of + an instance of the stream's element type is 1. + + + + + ivector + + + Any ivector + + + + + start-octet + + + A non-negative integer. + + + + + max-octet + + + A non-negative integer. + + + + + + + Examples + +;;; Write the contents of a (SIMPLE-ARRAY(UNSIGNED-BYTE 16) 3) ;;; to a character file stream. Read back the characters. -(let* ((a (make-array 3 +(let* ((a (make-array 3 :element-type '(unsigned-byte 16) :initial-contents '(26725 27756 28449)))) @@ -4693,18 +6952,18 @@ ;;; Write a vector of DOUBLE-FLOATs. Note that (to maintain -;;; alignment) there are 4 octets of padding before the 0th -;;; element of a (VECTOR DOUBLE-FLOAT) in 32-bit OpenMCL. -;;; (Note that (= (- ppc32::misc-dfloat-offset -;;; ppc32::misc-data-offset) 4)) +;;; alignment) there are 4 octets of padding before the 0th +;;; element of a (VECTOR DOUBLE-FLOAT). +;;; (Note that (= (- arch::misc-dfloat-offset +;;; arch::misc-data-offset) 4)) (defun write-double-float-vector - (stream vector &key (start 0) (end (length vector))) + (stream vector &key (start 0) (end (length vector))) (check-type vector (vector double-float)) - (let* ((start-octet (+ (* start 8) - (- target::misc-dfloat-offset - target::misc-data-offset))) + (let* ((start-octet (+ (* start 8) + (- arch::misc-dfloat-offset + arch::misc-data-offset))) (num-octets (* 8 (- end start)))) - (stream-write-ivector stream vector start-octet num-octets))) - - + (stream-write-ivector stream vector start-octet num-octets))) + + @@ -5271,5 +7530,5 @@ The argument/return values - is a MACPTR. + is a MACPTR. @@ -5616,256 +7875,675 @@ - %get-bit [Function] - Syntax - %get-bit ptr bit-offset - Description - References and returns the bit-offsetth bit at the addressencapsulated by ptr. (Bit 0 at a given address is the mostsignificant bit of the byte at that address.) Can be used withSETF. - Arguments - - ptr - A MACPTR - bit-offset - A fixnum - - - - + %get-bit [Function] + + + + Syntax + + + %get-bit ptr bit-offset + + + + + Description + + + References and returns the bit-offsetth bit at the address + encapsulated by ptr. (Bit 0 at a given address is the most + significant bit of the byte at that address.) Can be used with + SETF. + + + + + Arguments + + +   + + + + ptr + + + A MACPTR + + + + + bit-offset + + + A fixnum + + + + + + - %get-bitfield [Function] - Syntax - %get-bitfield ptr bit-offset width - Description - References and returns an unsigned integer composed from thewidth bits found bit-offsetbits from the address encapsulated byptr. (The least significant bit of the result is the value of(%get-bit ptr (1- (+ bit-offset width)))). Can be used with SETF. - Arguments - - ptr - A MACPTR - bit-offset - A fixnum - width - A positive fixnum - - - - + %get-bitfield [Function] + + + Syntax + + + %get-bitfield ptr bit-offset width + + + + + Description + + + References and returns an unsigned integer composed from the + width bits found bit-offsetbits from the address encapsulated by + ptr. (The least significant bit of the result is the value of + (%get-bit ptr (1- (+ bit-offset width))). Can be used with SETF. + + + + + Arguments + + +   + + + + + ptr + + + A MACPTR + + + + bit-offset + + + A fixnum + + + + + width + + + A positive fixnum + + + + + + - - %int-to-ptr [Function] - Syntax - %int-to-ptr int - Description - Creates and returns a MACPTR whose address matches int. - Arguments - - int - An (unsigned-byte 32) - - - - - - - - %inc-ptr [Function] - Syntax - %inc-ptr ptr &optional (delta 1) - Description - Creates and returns a MACPTR whose address is the address ofptr plus delta. The idiom (%inc-ptr ptr 0) is sometimes used tocopy a MACPTR, e.g., to create a new MACPTR encapsulating the sameaddress as ptr. - Arguments - - ptr - A MACPTR - delta - A fixnum - - - - - - - - %ptr-to-int [Function] - Syntax - %ptr-to-int ptr - Description - Returns the address encapsulated by ptr, as an(unsigned-byte 32). - Arguments - - ptr - A MACPTR - - - - - - - - %null-ptr [Macro] - Syntax - %null-ptr - Description - Equivalent to (%ptr-to-int 0). - - - - - - %null-ptr-p [Function] - Syntax - %null-ptr-p ptr - Description - Returns T If ptr is a MACPTR encapsulating the address 0,NIL if ptr encapsulates some other address. - Arguments - - ptr - A MACPTR - - - - - - - - %setf-macptr [Function] - Syntax - %setf-macptr dest-ptr src-ptr - Description - Causes dest-ptr to encapsulate the same address that src-ptrdoes, then returns dest-ptr. - Arguments - - dest-ptr - A MACPTR - src-ptr - A MACPTR - - - - - - - - %incf-ptr [Macro] - Syntax - %incf-ptr ptr &optional (delta 1) - Description - Destructively modifies ptr, by adding delta to the addressit encapsulates. Returns ptr. - Arguments - - ptr - A MACPTR - delta - A fixnum - - - - - - - - with-macptrs [Macro] - Syntax - with-macptrs (var expr)* &body body - Description - Executes body in an environment in which each var is boundto a stack-allocated macptr which encapsulates the foreign addressyielded by the corresponding expr. Returns whatever value(s) bodyreturns. - Arguments - - var - A symbol (variable name) - expr - A MACPTR-valued expression - - - - - - - - %stack-block [Macro] - Syntax - %stack-block (var expr)* &body body - Description - Executes body in an environment in which each var is boundto a stack-allocated macptr which encapsulates the address of astack-allocated region of size expr bytes. Returns whatevervalue(s) body returns. - Arguments - - var - A symbol (variable name) - expr - An expression which should evaluate to a non-negativefixnum - - - - - - - - make-cstring [Function] - Syntax - make-cstring string - Description - Allocates a block of memory (via malloc) of length (1+(length string)). Copies the string to this block and appends atrailing NUL byte; returns a MACPTR to the block. - Arguments - - string - A lisp string - - - - - - - - with-cstrs [Macro] - Syntax - with-cstrs (var string)* &body body - Description - Executes body in an environment in which each var is boundto a stack-allocated macptr which encapsulates the %address of astack-allocated region of into which each string (and a trailingNUL byte) has been copied. Returns whatever value(s) body returns. - Arguments - - var - A symbol (variable name) - string - An expression which should evaluate to a lisp string - - - - - - - - %get-cstring [Function] - Syntax - %get-cstring ptr - Description - Interprets ptr as a pointer to a (NUL -terminated) C string;returns an equivalent lisp string. - Arguments - - ptr - A MACPTR - - - - - - - - %str-from-ptr [Function] - Syntax - %str-from-ptr ptr length - Description - Returns a lisp string of length length,whose contents are initialized from the bytes at ptr. - Arguments - - ptr - AMACPTR - length - anon-negative fixnum - - - - - + + %int-to-ptr [Function] + + + + Syntax + + + %int-to-ptr int + + + + + Description + + + Creates and returns a MACPTR whose address matches int. + + + + + Arguments + + +   + + + + int + + + An (unsigned-byte 32) + + + + + + + + + + %inc-ptr [Function] + + + + Syntax + + + %inc-ptr ptr &optional (delta 1) + + + + + Description + + + Creates and returns a MACPTR whose address is the address of + ptr plus delta. The idiom (%inc-ptr ptr 0) is sometimes used to + copy a MACPTR, e.g., to create a new MACPTR encapsulating the same + address as ptr. + + + + + Arguments + + +   + + + + ptr + + + A MACPTR + + + + + delta + + + A fixnum + + + + + + + + + + %ptr-to-int [Function] + + + + Syntax + + + %ptr-to-int ptr + + + + + Description + + + Returns the address encapsulated by ptr, as an + (unsigned-byte 32). + + + + + Arguments + + +   + + + + ptr + + + A MACPTR + + + + + + + + + + %null-ptr [Macro] + + + + Syntax + + + %null-ptr + + + + + Description + + + Equivalent to (%ptr-to-int 0). + + + + + + + %null-ptr-p [Function] + + + + Syntax + + + %null-ptr-p ptr + + + + + Description + + + Returns T If ptr is a MACPTR encapsulating the address 0, + NIL if ptr encapsulates some other address. + + + + + Arguments + + +   + + + + ptr + + + A MACPTR + + + + + + + + + + %setf-macptr [Function] + + + + Syntax + + + %setf-macptr dest-ptr src-ptr + + + + + Description + + + Causes dest-ptr to encapsulate the same address that src-ptr + does, then returns dest-ptr. + + + + + Arguments + + +   + + + + dest-ptr + + + A MACPTR + + + + + src-ptr + + + A MACPTR + + + + + + + + + + %incf-ptr [Macro] + + + + Syntax + + + %incf-ptr ptr &optional (delta 1) + + + + + Description + + + Destructively modifies ptr, by adding delta to the address + it encapsulates. Returns ptr. + + + + + Arguments + + +   + + + + ptr + + + A MACPTR + + + + + delta + + + A fixnum + + + + + + + + + + with-macptrs [Macro] + + + + Syntax + + + with-macptrs (var expr)* &body body + + + + + Description + + + Executes body in an environment in which each var is bound + to a stack-allocated macptr which encapsulates the foreign address + yielded by the corresponding expr. Returns whatever value(s) body + returns. + + + + + Arguments + + +   + + + + var + + + A symbol (variable name) + + + + + expr + + + A MACPTR-valued expression + + + + + + + + + + %stack-block [Macro] + + + + Syntax + + + %stack-block (var expr)* &body body + + + + + Description + + + Executes body in an environment in which each var is bound + to a stack-allocated macptr which encapsulates the address of a + stack-allocated region of size expr bytes. Returns whatever + value(s) body returns. + + + + + Arguments + + +   + + + + var + + + A symbol (variable name) + + + + + expr + + + An expression which should evaluate to a non-negative + fixnum + + + + + + + + + + make-cstring [Function] + + + + Syntax + + + make-cstring string + + + + + Description + + + Allocates a block of memory (via malloc) of length (1+ + (length string)). Copies the string to this block and appends a + trailing NUL byte; returns a MACPTR to the block. + + + + + Arguments + + +   + + + + string + + + A lisp string + + + + + + + + + + with-cstrs [Macro] + + + + Syntax + + + with-cstrs (var string)* &body body + + + + + Description + + + Executes body in an environment in which each var is bound + to a stack-allocated macptr which encapsulates the %address of a + stack-allocated region of into which each string (and a trailing + NUL byte) has been copied. Returns whatever value(s) body returns. + + + + + Arguments + + +   + + + + var + + + A symbol (variable name) + + + + + string + + + An expression which should evaluate to a lisp string + + + + + + + + + + %get-cstring [Function] + + + + Syntax + + + %get-cstring ptr + + + + + Description + + + Interprets ptr as a pointer to a (NUL -terminated) C string; + returns an equivalent lisp string. + + + + + Arguments + + + + + + + ptr + + + A MACPTR + + + + + + + + + + %str-from-ptr [Function] + + + + Syntax + + + %str-from-ptr ptr length + + + + + Description + + + Returns a lisp string of length length, + whose contents are initialized from the bytes at ptr. + + + + + + Arguments + + + ptrA + MACPTRlengtha + non-negative fixnum + + + + @@ -5885,9 +8563,13 @@ Not surprisingly, different platforms use different database files. OpenMCL defines reader macros that consult these databases: - - #$foo looks up the value of the constant definition of FIXTHIS - #_foo looks up the foreign function definition for FIXTHIS + + + #$foo looks up the value of the constant definition of foo + + + #_foo looks up the foreign function definition for foo + - + In both cases, the symbol foo is interned in the "OS" package. The #$ reader macro has the side-effect of defining @@ -6027,170 +8709,253 @@ To create a new interface directory, "foo", and a set of database files in that directory: - - Create a subdirectory of "ccl:headers;" named"foo". - Create a subdirectory of "ccl:headers;foo;" named"C". - Create a file in "ccl:headers;foo;C;" named"populate.sh".One way of accomplishing the above steps is: - -? (close (open "ccl:headers;foo;C;populate.sh" :direction :output : - if-does-not-exist :create :if-exists :overwrite)) - - Edit the file created above, using the "populate.sh"files in the distribution as guidelines.The file might wind up looking something like: - -#/bin/sh -h-to-ffi.sh `foo-config -cflags` /usr/include/foo/foo.h - - - Refer to -for information about running the -interface translator and .ffi parser. - Assuming that all went well, there should now be .cdb files in -"ccl:headers;foo;". You can then do -(use-interface-dir :foo) whenever you need to -access the foreign type information in those database files. + + + Create a subdirectory of "ccl:headers;" named + "foo". + + + + Create a subdirectory of "ccl:headers;foo;" named + "C". + + + + Create a file in "ccl:headers;foo;C;" named + "populate.sh". + + One way of accomplishing the above steps is: + + ? (close (open "ccl:headers;foo;C;populate.sh" :direction :output : + if-does-not-exist :create :if-exists :overwrite)) + + + + Edit the file created above, using the "populate.sh" + files in the distribution as guidelines. + + The file might wind up looking something like: + + #/bin/sh +h-to-ffi.sh `foo-config -cflags` /usr/include/foo/foo.h + + + + Refer to for + information about running the interface translator and .ffi + parser. + Assuming that all went well, there should now be .cdb + files in "ccl:headers;foo;". You can then do + +? (use-interface-dir :foo) + + whenever you need to + access the foreign type information in those database + files. - Using Shared Libraries - - - Overview -OpenMCL provides facilities to open and close shared libraries. - "Opening" a shared library, which is done with -, maps the library's -code and -data into OpenMCL's address space and makes its exported symbols -accessible to OpenMCL. - "Closing" a shared library, which is done with -, unmaps the -library's code and -data and removes the library's symbols from the global namespace. - A small number of shared libraries (including libc, libm, libdl -under Linux, and the "system" library under Darwin) are opened by -the lisp kernel and can't be closed. - OpenMCL uses data structures of type EXTERNAL-ENTRY-POINT to map a -foreign function name (string) to that foreign function's -current< address. (A function's address may -vary from session to session as different versions of shared libraries may -load at different addresses; it may vary within a session for similar -reasons.) - An EXTERNAL-ENTRY-POINT whose address is known is said to be -resolved. When an external entry point is resolved, -the shared library which defines that entry point is noted; when a shared -library is closed, the entry points that it defines are made unresolved. -An EXTERNAL-ENTRY-POINT must be in the resolved state in order to be -FF-CALLed; calling an unresolved entry point causes a "last -chance" attempt to resolve it. Attempting to resolve an entrypoint -that was defined in a closed library will cause an attempt to reopen that -library. - OpenMCL keeps track of all libraries that have been opened in a lisp -session. When a saved application is first started, an attempt is made to -reopen all libraries that were open when the image was saved, and an -attempt is made to resolve all entrypoints that had been referenced when -the image was saved. Either of these attempts can fail "quietly", -leaving some entry points in an unresolved state. - Linux shared libraries can be referred to either by a string which -describes their full pathname or by their soname, a -shorter string that can be defined when the library is created. The -dynamic linker mechanisms used in Linux make it possible (through a series -of filesystem links and other means) to refer to a library via several -names; the library's soname is often the most appropriate identifier. - sonames are often less version-specific than other names for -libraries; a program that refers to a library by the name -"libc.so.6" is more portable than one which refers to -"libc-2.1.3.so" or to "libc-2.2.3.so", even though the -latter two names might each be platform-specific aliases of the first. - All of the global symbols described below are exported from the CCL -package. + Using Shared Libraries + + + Overview + + OpenMCL provides facilities to open and close shared + libraries. + "Opening" a shared library, which is done with , maps the library's code and + data into OpenMCL's address space and makes its exported + symbols accessible to OpenMCL. + "Closing" a shared library, which is done with , unmaps the library's code + and and removes the library's symbols from the global + namespace. + A small number of shared libraries (including libc, + libm, libdl under Linux, and the "system" library under + Darwin) are opened by the lisp kernel and can't be + closed. + OpenMCL uses data structures of type + EXTERNAL-ENTRY-POINT to map a foreign function name (string) + to that foreign function's current + address. (A function's address may vary from session to + session as different versions of shared libraries may load at + different addresses; it may vary within a session for similar + reasons.) + An EXTERNAL-ENTRY-POINT whose address is known is said + to be resolved. When an external entry + point is resolved, the shared library which defines that entry + point is noted; when a shared library is closed, the entry + points that it defines are made unresolved. An + EXTERNAL-ENTRY-POINT must be in the resolved state in order to + be FF-CALLed; calling an unresolved entry point causes a "last + chance" attempt to resolve it. Attempting to resolve an + entrypoint that was defined in a closed library will cause an + attempt to reopen that library. + OpenMCL keeps track of all libraries that have been + opened in a lisp session. When a saved application is first + started, an attempt is made to reopen all libraries that were + open when the image was saved, and an attempt is made to + resolve all entrypoints that had been referenced when the + image was saved. Either of these attempts can fail "quietly", + leaving some entry points in an unresolved state. + Linux shared libraries can be referred to either by a + string which describes their full pathname or by their + soname, a shorter string that can be + defined when the library is created. The dynamic linker + mechanisms used in Linux make it possible (through a series of + filesystem links and other means) to refer to a library via + several names; the library's soname is often the most + appropriate identifier. + sonames are often less version-specific than other names + for libraries; a program that refers to a library by the name + "libc.so.6" is more portable than one which refers to + "libc-2.1.3.so" or to "libc-2.2.3.so", even though the latter + two names might each be platform-specific aliases of the + first. + All of the global symbols described below are exported + from the CCL package. - Limitations and known bugs - - Don't get me started. - The underlying functionality has a poor notion of dependency;it's not always possible to open libraries that depend on unopenedlibraries, but it's possible to close libraries on which otherlibraries depend.It may be possible to generate moreexplicit dependency information by parsing the output of the Linux lddand ldconfig programs. + Limitations and known bugs + + + Don't get me started. + + + The underlying functionality has a poor notion of + dependency;it's not always possible to open libraries that + depend on unopened libraries, but it's possible to close + libraries on which other libraries depend. It + may be possible to generate + more explicit dependency information by parsing the output + of the Linux ldd and ldconfig programs. + - + - Darwin Notes -Darwin shared libraries come in two (basic) flavors: - - "dylibs" (which often have the extension".dylib") are primarily intended to be linked against atcompile/link time. They can be loaded dynamically,but can't be unloaded. Accordingly,OPEN-SHARED-LIBRARY can be used to open a .dylib-style library;calling CLOSE-SHARED-LIBRARY on the result of such a callproduces a warning, and has no other effect.It appears that (due to an OS bug) attempts to open .dylibshared-libraries that are already open can cause memory corruptionunless the full pathname of the .dylib file is specified on thefirst and all subsequent calls. - "bundles" are intended to serve as applicationextensions; they can be opened multiple times (creating multipleinstances of the library!) and closed properly. - - - Thanks to Michael Klingbeil for getting both kinds of Darwin shared -libraries working in OpenMCL. + >Darwin Notes + Darwin shared libraries come in two (basic) flavors: + + + "dylibs" (which often have the extension".dylib") + are primarily intended to be linked against atcompile/link + time. They can be loaded dynamically,but can't + be unloaded. Accordingly,OPEN-SHARED-LIBRARY + can be used to open a .dylib-style library;calling + CLOSE-SHARED-LIBRARY on the result of such a callproduces + a warning, and has no other effect.It appears that (due to + an OS bug) attempts to open .dylibshared-libraries that + are already open can cause memory corruptionunless the + full pathname of the .dylib file is specified on thefirst + and all subsequent calls. + + + "bundles" are intended to serve as application + extensions; they can be opened multiple times (creating + multiple instances of the library!) and closed + properly. + + + Thanks to Michael Klingbeil for getting both kinds of + Darwin shared libraries working in OpenMCL. - The Interface Translator - - - Overview -OpenMCL uses an interface translation system based on the FFIGEN -system, which is described at -http://www.ccs.neu.edu/home/lth/ffigen/. -The interface translator makes -the constant, type, structure, and function definitions in a set of -C-language header files available to lisp code. - The basic idea of the FFIGEN scheme is to use the C compiler's -frontend and parser to translate .h files into semantically equivalent -.ffi files, which represent the definitions from the headers using a -syntax based on S-expressions. -Lisp code can then concentrate on -the .ffi representation, without having to concern itself with the -semantics of header file inclusion or the arcana of C parsing. - The original FFIGEN system used a modified version of the LCC C -compiler to produce .ffi files. Since many LinuxPPC header files contain -GCC-specific constructs, OpenMCL's translation system uses a modified -version of GCC (called, somewhat confusingly, ffigen.) - A version of ffigen based on GCC-4.0 was developed during the spring -and summer of 2005. Sources (diffs relative to the GCC-4.0 release) -are available here, and -binaries are available for -DarwinPPC -and for -LinuxPPC. -These versions should be insensitive to to the version of GCC (and its -preprocessor) installed on the system. - An older version was developed in 2001-2002; it depended on the installed -version of GCC being 2.95. It may still be of interest for people unable -to run the GCC-4.0-based version for whatever reason. - A LinuxPPC binary of this older version is available at ftp://clozure.com/pub/ffigen-0.1.tar.gz, -and LinuxPPC source differences are at ftp://clozure.com/pub/ffigen-src.tar.gz. - For Darwin, the binary of the older FFIGEN is available at ftp://clozure.com/pub/ffigen-darwin.tar.gz, -and the source differences are at ftp://clozure.com/pub/ffigen-darwin-src.tar.gz. - A shell script (distributed with the source and binary packages) -called h-to-ffi.sh reads a specified .h file (and optional -preprocessor arguments) and writes a (hopefully) equivalent .ffi file to -standard output, calling the installed C preprocessor and the ffigen -program with appropriate arguments. - For each interface directory (see ) -subdir distributed with OpenMCL, a shell script -(distributed with OpenMCL as "ccl:headers;subdir;C;populate.sh" -("ccl:darwin-headers;subdir;C;populate.sh" -for Darwin)) calls h-to-ffi.sh on a large number of the header files in -/usr/include (or some other system header path) and -creates a parallel directory tree in "ccl:headers;subdir;C;system;header;path;" -(or "ccl:darwin-headers;subdir;C;system;header;path;"), -populating that directory with .ffi files. + The Interface Translator + + + Overview + OpenMCL uses an interface translation system based on the FFIGEN + system, which is described at + http://www.ccs.neu.edu/home/lth/ffigen/. + The interface translator makes + the constant, type, structure, and function definitions in a set of + C-language header files available to lisp code. + The basic idea of the FFIGEN scheme is to use the C + compiler's frontend and parser to translate .h files into + semantically equivalent .ffi files, which represent the + definitions from the headers using a syntax based on + S-expressions. Lisp code can then concentrate on the .ffi + representation, without having to concern itself with the + semantics of header file inclusion or the arcana of C + parsing. + The original FFIGEN system used a modified version of + the LCC C compiler to produce .ffi files. Since many LinuxPPC + header files contain GCC-specific constructs, OpenMCL's + translation system uses a modified version of GCC (called, + somewhat confusingly, ffigen.) + A version of ffigen based on GCC-4.0 was developed + during the spring and summer of 2005. Sources (diffs relative + to the GCC-4.0 release) are available here, and binaries are + available for DarwinPPC and for LinuxPPC. These versions + should be insensitive to to the version of GCC (and its + preprocessor) installed on the system. + An older version was developed in 2001-2002; it depended + on the installed version of GCC being 2.95. It may still be + of interest for people unable to run the GCC-4.0-based version + for whatever reason. + A LinuxPPC binary of this older version is available at + ftp://clozure.com/pub/ffigen-0.1.tar.gz, and LinuxPPC source + differences are at + ftp://clozure.com/pub/ffigen-src.tar.gz. + For Darwin, the binary of the older FFIGEN is available + at ftp://clozure.com/pub/ffigen-darwin.tar.gz, and the source + differences are at + ftp://clozure.com/pub/ffigen-darwin-src.tar.gz. + A shell script (distributed with the source and binary + packages) called h-to-ffi.sh reads a specified .h file (and + optional preprocessor arguments) and writes a (hopefully) + equivalent .ffi file to standard output, calling the installed + C preprocessor and the ffigen program with appropriate + arguments. + For each interface directory (see FIXTHIS) + subdir distributed with OpenMCL, a shell + script (distributed with OpenMCL as + "ccl:headers;subdir;C;populate.sh" + ("ccl:darwin-headers;subdir;C;populate.sh" + for Darwin)) calls h-to-ffi.sh on a large number of the header + files in /usr/include (or some other system header + path) and creates a parallel directory tree in + "ccl:headers;subdir;C;system;header;path;" + (or + "ccl:darwin-headers;subdir;C;system;header;path;"), + populating that directory with .ffi files. A lisp function defined in "ccl:library;parse-ffi.lisp" -reads the .ffi files in a specified interface directory -subdir and generates new versions of the databases -(files with the extension .cdb). - The CDB databases are used by the #$ and #_ reader macros and are -used in the expansion of RREF, RLET, and related macros. + reads the .ffi files in a specified interface directory + subdir and generates new versions of the + databases (files with the extension .cdb). + The CDB databases are used by the #$ and #_ reader + macros and are used in the expansion of RREF, RLET, and + related macros. - Details: rebuilding the CDB databases, step by step - - Ensure that the FFIGEN program is installed. See the"README" file in the source or binary archive for specificinstallation instructions.This example assumes LinuxPPC; for 32-bit DarwinPPC, substitute"ccl:darwin-headers;" for "ccl:headers;". For 64-bit DarwinPPC,substitute "ccl:darwin-headers64;". - Edit the "ccl:headers;subdir;C;populate.sh"shell script. When you're confident that the files andpreprocessor options match your environment, cd to the"ccl:headers;subdir;C;" directory andinvoke ./populate.sh. Repeat this step until you're able tocleanly translate all files refrenced in the shell script. - Run OpenMCL: + Details: rebuilding the CDB databases, step by step + + + Ensure that the FFIGEN program is installed. See + the"README" file in the source or binary archive for + specificinstallation instructions.This example assumes + LinuxPPC; for 32-bit DarwinPPC, + substitute"ccl:darwin-headers;" for "ccl:headers;". For + 64-bit DarwinPPC,substitute + "ccl:darwin-headers64;". + + + Edit the + "ccl:headers;subdir;C;populate.sh"shell + script. When you're confident that the files + andpreprocessor options match your environment, cd to + the"ccl:headers;subdir;C;" directory + andinvoke ./populate.sh. Repeat this step until you're + able tocleanly translate all files refrenced in the shell + script. + + + Run OpenMCL: ? (require "PARSE-FFI") @@ -6201,53 +8966,53 @@ ;;; appear in "ccl:headers;subdir;" ;;; (or "ccl:darwin-headers;subdir;" under Darwin) - - - PARSE-STANDARD-FFI-FILES accepts a :PREPEND-UNDERSCORES keyword -argument. Darwin (and some other platforms) use a convention wherein the -symbols associated with C-visible external function and variables have -underscore characters prepended to their names. When this argument is -true, PARSE-STANDARD-FFI-FILES will prepend underscores to all foreign -function names written to the database, so that (#_foo ...) expands into -an EXTERNAL-CALL to "_foo". + + + - Case-sensitivity of foreign names in OpenMCL - - - Overview -As of release 0.11, OpenMCL addresses the fact that foreign type, -constant, record, field, and function nams are case-sensitive and provides -mechanisms to refer to these names via lisp symbols. - Previous versions of OpenMCL have tried to ignore that fact, under -the belief that case conflicts were rare and that many users (and -implementors) would prefer not to deal with case-related issues. The fact -that some information in the interface databases was incomplete or -inaccessable because of this policy made it clearer that the policy was -untenable. I can't claim that the approach described here is -aesthetically pleasing, but I can honestly say that it's less -unpleasant than other approaches that I'd thought of. I'd be -interested to hear alternate proposals. - The issues described here have to do with how lisp symbols are used -to denote foreign functions, constants, types, records, and fields. It -doesn't affect how other lisp objects are sometimes used to denote -foreign objects. For instance, the first argument to the EXTERNAL-CALL -macros is now and has always been a case-sensitive string. + Case-sensitivity of foreign names in OpenMCL + + + Overview + As of release 0.11, OpenMCL addresses the fact that + foreign type, constant, record, field, and function nams are + case-sensitive and provides mechanisms to refer to these names + via lisp symbols. + Previous versions of OpenMCL have tried to ignore that + fact, under the belief that case conflicts were rare and that + many users (and implementors) would prefer not to deal with + case-related issues. The fact that some information in the + interface databases was incomplete or inaccessable because of + this policy made it clearer that the policy was untenable. I + can't claim that the approach described here is aesthetically + pleasing, but I can honestly say that it's less unpleasant + than other approaches that I'd thought of. I'd be interested + to hear alternate proposals. + The issues described here have to do with how lisp + symbols are used to denote foreign functions, constants, + types, records, and fields. It doesn't affect how other lisp + objects are sometimes used to denote foreign objects. For + instance, the first argument to the EXTERNAL-CALL macros is + now and has always been a case-sensitive string. - Foreign constant and function names -The primary way of referring to foreign constant and function names -in OpenMCL is via the #$ and #_ reader macros. These reader macro -functions each read a symbol into the "OS" package, look up its -constant or function definition in the interface database, and assign the -value of the constant to the symbol or install a macroexpansion function -on the symbol. - In order to observe case-sensitivity, the reader-macros now read the -symbol with (READTABLE-CASE :PRESERVE) in effect. - This means that it's necessary to type the foreign constant or -function name in correct case, but it isn't necessary to use any -special escaping constructs when writing the variable name. For instance: + Foreign constant and function names + The primary way of referring to foreign constant and + function names in OpenMCL is via the #$ and #_ reader + macros. These reader macro functions each read a symbol into + the "OS" package, look up its constant or function definition + in the interface database, and assign the value of the + constant to the symbol or install a macroexpansion function on + the symbol. + In order to observe case-sensitivity, the reader-macros + now read the symbol with (READTABLE-CASE :PRESERVE) in + effect. + This means that it's necessary to type the foreign + constant or function name in correct case, but it isn't + necessary to use any special escaping constructs when writing + the variable name. For instance: (#_read fd buf n) ; refers to foreign symbol "read" @@ -6260,46 +9025,63 @@ - Foreign type, record, and field names -Constructs like RLET expect a foreign type or record name to be -denoted by a symbol (typically a keyword); RREF (and PREF) expect an -"accessor" form, typically a keyword formed by concatenating a -foreign type or record name with a sequence of one or more foreign field -names, separated by dots. These names are interned by the reader as other -lisp symbols are, with an arbitrary value of READTABLE-CASE in effect -(typically :UPCASE.) It seems like it would be very tedious to force users -to manually escape (via vertical bar or backslash syntax) all lowercase -characters in symbols used to specify foreign type, record, and field -names (especially given that many traditional POSIX structure, type, and -field names are entirely lowercase.) - The approach taken by OpenMCL is to allow the symbols (keywords) -used to denote foreign type, record, and field names to contain angle -brackets (< and >). Such symbols are translated to foreign names -via the following set of conventions: - - All instances of < and > in the symbol's pname arebalanced and don't nest. - Any alphabetic characters in the symbol's pname thataren't enclosed in angle brackets are treated as lower-case,regardless of the value of READTABLE-CASE and regardless of the casein which they were written. - Alphabetic characters that appear within angle brackets aremapped to upper-case, again regardless of how they were written orinterned. - - - There may be many ways of "escaping" (with angle brackets) -sequences of upper-case and non-lower-case characters in a symbol used to -denote a foreign name. When translating in the other direction, OpenMCL -always escapes the longest sequence that starts with an upper-case -character and doesn't contain a lower-case character. - It's often preferable to use this canonical form of a foreign -type name. - The accessor forms used by PREF/RREF should be viewed as a series of -foreign type/record and field names; upper-case sequences in the component -names should be escaped with angle brackets, but those sequences -shouldn't span components. (More simply, the separating dots -shouldn't be enclosed, even if both surrounding characters need to -be.) - Older POSIX code tends to use lower-case exclusively for type, -record, and field names; there are only a few cases in the OpenMCL sources -where mixed-case names need to be escaped. + Foreign type, record, and field names + Constructs like RLET expect a foreign type or record + name to be denoted by a symbol (typically a keyword); RREF + (and PREF) expect an "accessor" form, typically a keyword + formed by concatenating a foreign type or record name with a + sequence of one or more foreign field names, separated by + dots. These names are interned by the reader as other lisp + symbols are, with an arbitrary value of READTABLE-CASE in + effect (typically :UPCASE.) It seems like it would be very + tedious to force users to manually escape (via vertical bar or + backslash syntax) all lowercase characters in symbols used to + specify foreign type, record, and field names (especially + given that many traditional POSIX structure, type, and field + names are entirely lowercase.) + The approach taken by OpenMCL is to allow the symbols + (keywords) used to denote foreign type, record, and field + names to contain angle brackets (< and + >). Such symbols are translated to + foreign names via the following set of conventions: + + + All instances of < and > in the symbol's pname + are balanced and don't nest. + + + Any alphabetic characters in the symbol's pname + thataren't enclosed in angle brackets are treated as + lower-case,regardless of the value of READTABLE-CASE and + regardless of the case in which they were written. + + + Alphabetic characters that appear within angle + brackets aremapped to upper-case, again regardless of how + they were written orinterned. + + + There may be many ways of "escaping" (with angle + brackets) sequences of upper-case and non-lower-case + characters in a symbol used to denote a foreign name. When + translating in the other direction, OpenMCL always escapes the + longest sequence that starts with an upper-case character and + doesn't contain a lower-case character. + It's often preferable to use this canonical form of a + foreign type name. + The accessor forms used by PREF/RREF should be viewed as + a series of foreign type/record and field names; upper-case + sequences in the component names should be escaped with angle + brackets, but those sequences shouldn't span components. (More + simply, the separating dots shouldn't be enclosed, even if + both surrounding characters need to be.) + Older POSIX code tends to use lower-case exclusively for + type, record, and field names; there are only a few cases in + the OpenMCL sources where mixed-case names need to be + escaped. + - Examples + Examples ;;; Allocate a record of type "window". @@ -6307,7 +9089,7 @@ ;;; Allocate a record of type "Window", which is probably a ;;; different type -(rlet ((w :indow)) ...) +(rlet ((w :<w>indow)) ...) ;;; This is equivalent to the last example -(rlet ((w :INDOW))) +(rlet ((w :<w>INDOW))) @@ -6315,16 +9097,18 @@ - Tutorial: Using Basic Calls and Types -This tutorial is meant to cover the basics of OpenMCL for calling -external C functions and passing data back and forth. These basics -will provide the foundation for more advanced techniques which will -allow access to the various external libraries and toolkits. - The first step is to start with a simple C dynamic library in order to -actually observe what is actually passing between OpenMCL and C. So, -some C code is in order: - Create the file typetest.c, and put the following code into it: + Tutorial: Using Basic Calls and Types + This tutorial is meant to cover the basics of OpenMCL for + calling external C functions and passing data back and forth. + These basics will provide the foundation for more advanced + techniques which will allow access to the various external + libraries and toolkits. + The first step is to start with a simple C dynamic library + in order to actually observe what is actually passing between + OpenMCL and C. So, some C code is in order: + Create the file typetest.c, and put the following code + into it: -#include +#include <stdio.> void @@ -6333,4 +9117,5 @@ printf("Entered %s:\n", __FUNCTION__); printf("Exited %s:\n", __FUNCTION__); + fflush(stdout); } @@ -6341,4 +9126,5 @@ printf("Data In: %d\n", (signed int)data); printf("Exited %s:\n", __FUNCTION__); + fflush(stdout); return data; } @@ -6350,56 +9136,61 @@ printf("Data In: %d\n", (signed int)data); printf("Exited %s:\n", __FUNCTION__); + fflush(stdout); return data; } - - This defines three functions. If you're familiar with C, notice -that there's no main(), because we're just -building a library, not an executable. - The function void_void_test() doesn't take -any parameters, and doesn't return anything, but it prints two -lines to let us know it was called. -sc_sc_test() takes a signed char as a -parameter, prints it, and returns it. -uc_uc_test() does the same thing, but with -an unsigned char. Their purpose is just to prove to us -that we really can call C functions, pass them values, and get -values back from them. - - This code is compiled into a dynamic library on OS X 10.3.4 with the -command: + + This defines three functions. If you're familiar with C, + notice that there's no main(), because we're + just building a library, not an executable. + The function void_void_test() doesn't + take any parameters, and doesn't return anything, but it prints + two lines to let us know it was called. + sc_sc_test() takes a signed char as a + parameter, prints it, and returns it. + uc_uc_test() does the same thing, but with an + unsigned char. Their purpose is just to prove to us that we + really can call C functions, pass them values, and get values + back from them. + This code is compiled into a dynamic library on OS X + 10.3.4 with the command: gcc -dynamiclib -Wall -o libtypetest.dylib typetest.c \ -install_name ./libtypetest.dylib + + Users of 64-bit platforms may need to pass options such + as "-m64" to gcc, may need to give the output library a different + extension (such as ".so"), and may need to user slightly different + values for other options in order to create an equivalent test + library. + + The -dynamiclib tells gcc that we will be compiling this + into a dynamic library and not an executable binary program. + The output filename is "libtypetest.dylib". Notice that we + chose a name which follows the normal OS X convention, being in + the form "libXXXXX.dylib", so that other programs can link to + the library. OpenMCL doesn't need it to be this way, but it is + a good idea to adhere to existing conventions. + The -install_name flag is primarily used when building OS + X "bundles". In this case, we are not using it, so we put a + placeholder into it, "./libtypetest.dylib". If we wanted to use + typetest in a bundle, the -install_name argument would be a + relative path from some "current" directory. + After creating this library, the first step is to tell + OpenMCL to open the dynamic library. This is done by calling + . + + +Welcome to OpenMCL Version (Beta: Darwin) 0.14.2-040506! + +? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib") +#<SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E> - The -dynamiclib tells gcc that we will be compiling this into a -dynamic library and not an executable binary program. The output -filename is "libtypetest.dylib". Notice that we chose a name which -follows the normal OS X convention, being in the form -"libXXXXX.dylib", so that other programs can link to the library. -OpenMCL doesn't need it to be this way, but it is a good idea to -adhere to existing conventions. - The -install_name flag is primarily used when -building OS X "bundles". In this case, we are not using it, so we -put a placeholder into it, "./libtypetest.dylib". If we wanted -to use typetest in a bundle, the -install_name argument would -be a relative path from some "current" directory. - After creating this library, the first step is to tell OpenMCL to open -the dynamic library. This is done by calling -. - - -Welcome to OpenMCL Version (Beta: Darwin) 0.14.2-040506! - -? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib") -# - - You should use an absolute path here; using a relative one, such -as just "libtypetest.dylib", would appear to work, but there are -subtle problems which occur after reloading it. See the Darwin -notes on - for details. It would be -a bad idea anyway, because software should never rely on its -starting directory being anything in particular. + You should use an absolute path here; using a relative + one, such as just "libtypetest.dylib", would appear to work, but + there are subtle problems which occur after reloading it. See + the Darwin notes on for details. It would be a bad idea anyway, + because software should never rely on its starting directory + being anything in particular. This command returns a reference to the opened shared library, and OpenMCL also adds one to the global variable @@ -6408,50 +9199,49 @@ ? ccl::*shared-libraries* -(# - #) - - Before we call anything, let's check that the individual functions can -actually be found by the system. We don't have to do this, but -it helps to know how to find out whether this is the problem, -when something goes wrong. -We use : +(#<SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E> + #<SHLIB /usr/lib/libSystem.B.dylib #x606179E>) + + Before we call anything, let's check that the individual + functions can actually be found by the system. We don't have to + do this, but it helps to know how to find out whether this is + the problem, when something goes wrong. We use : ? (external "_void_void_test") -# +#<EXTERNAL-ENTRY-POINT "_void_void_test" (#x000CFDF8) /Users/andewl/openmcl/libtypetest.dylib #x638EDF6> ? (external "_sc_sc_test") -# +#<EXTERNAL-ENTRY-POINT "_sc_sc_test" (#x000CFE50) /Users/andewl/openmcl/libtypetest.dylib #x638EB3E> ? (external "_uc_uc_test") -# - - Notice that the actual function names have -been "mangled" by the C linker. The first function was named -"void_void_test" in typetest.c, but in -libtypetest.dylib, it has an underscore (a "_" symbol) before -it: "_void_void_test". So, this is the name which you have to use. -The mangling - the way the name is changed - may be different for -other operating systems or other versions, so you need to "just know" -how it's done... - Also, pay particular attention to the fact -that a hexadecimal value appears in the EXTERNAL-ENTRY-POINT. -(#x000CFDF8, for example - but what it is doesn't matter.) -These hex numbers mean that the -function can be dereferenced. Functions which aren't found will -not have a hex number. For example: +#<EXTERNAL-ENTRY-POINT "_uc_uc_test" (#x000CFED4) /Users/andewl/openmcl/libtypetest.dylib #x638E626> + + Notice that the actual function names have been "mangled" + by the C linker. The first function was named "void_void_test" + in typetest.c, but in libtypetest.dylib, it has an underscore (a + "_" symbol) before it: "_void_void_test". So, this is the name + which you have to use. The mangling - the way the name is + changed - may be different for other operating systems or other + versions, so you need to "just know" how it's done... + Also, pay particular attention to the fact that a + hexadecimal value appears in the EXTERNAL-ENTRY-POINT. + (#x000CFDF8, for example - but what it is doesn't matter.) + These hex numbers mean that the function can be dereferenced. + Functions which aren't found will not have a hex number. For + example: ? (external "functiondoesnotexist") -# +#<EXTERNAL-ENTRY-POINT "functiondoesnotexist" {unresolved} #x638E3F6> The "unresolved" tells us that OpenMCL wasn't able to find this -function, which means you would get an error, "Can't resolve foreign -symbol," if you tried to call it. - These -external function references also are stored in a hash table which is -accessible through a global variable, ccl::*eeps*. - At this point, we are ready to try our first external function -call: + function, which means you would get an error, "Can't resolve foreign + symbol," if you tried to call it. + These external function references also are stored in a + hash table which is accessible through a global variable, + ccl::*eeps*. + At this point, we are ready to try our first external + function call: @@ -6460,16 +9250,9 @@ Exited void_void_test: NIL - - We used , which is -is the normal mechanism for -accessing externally linked code. The "_void_void_test" -is the mangled name of the external function. -The :void -refers to the return type of the function. - If you're using ILISP to run OpenMCL inside of Emacs, you won't -see the "Entered" and "Exited" lines until you quit (as of -July 2004). It's not -clear why this is, but it's a pity. If you want to see them, run -OpenMCL from Terminal.app or in some other way. + + We used , which is is the normal mechanism for accessing + externally linked code. The "_void_void_test" is the mangled + name of the external function. The :void refers to the return + type of the function. The next step is to try passing a value to C, and getting one back: @@ -6482,11 +9265,10 @@ -128 - The first :signed-byte gives the type of the first argument, -and then -128 gives the value to pass for it. The second -:signed-byte -gives the return type. The return type is always given by the -last argument to . + The first :signed-byte gives the type of the first + argument, and then -128 gives the value to pass for it. The + second :signed-byte gives the return type. The return type is + always given by the last argument to . Everything looks good. Now, let's try a number outside -the range which fits in one byte: + the range which fits in one byte: @@ -6497,6 +9279,6 @@ -55 - Hmmmm. A little odd. Let's look at the unsigned stuff -to see how it reacts: + Hmmmm. A little odd. Let's look at the unsigned stuff to + see how it reacts: @@ -6521,19 +9303,20 @@ Exited uc_uc_test: 201 - + Since a signed byte can only hold values from -128 through 127, and -an unsigned one can only hold values from 0 through 255, any number -outside that range gets "clipped": only the low eight bits of it -are used. + an unsigned one can only hold values from 0 through 255, any number + outside that range gets "clipped": only the low eight bits of it + are used. What is important to remember is that external -function calls have -very few safety checks. -Data outside the valid range for its type will silently do -very strange things; pointers outside the valid range can very well -crash the system. - That's it for our first example library. If you're still following -along, let's add some more C code to look at the rest of the -primitive types. Then we'll need to recompile the dynamic library, -load it again, and then we can see what happens. + function calls have + very few safety checks. + Data outside the valid range for its type will silently do + very strange things; pointers outside the valid range can very well + crash the system. + That's it for our first example library. If you're still + following along, let's add some more C code to look at the rest + of the primitive types. Then we'll need to recompile the + dynamic library, load it again, and then we can see what + happens. Add the following code to typetest.c: @@ -6545,4 +9328,5 @@ printf("Data In: %d\n", data); printf("Exited %s:\n", __FUNCTION__); + fflush(stdout); return data; } @@ -6554,4 +9338,5 @@ printf("Data In: %ld\n", data); printf("Exited %s:\n", __FUNCTION__); + fflush(stdout); return data; } @@ -6563,4 +9348,5 @@ printf("Data In: %lld\n", data); printf("Exited %s:\n", __FUNCTION__); + fflush(stdout); return data; } @@ -6572,4 +9358,5 @@ printf("Data In: %e\n", data); printf("Exited %s:\n", __FUNCTION__); + fflush(stdout); return data; } @@ -6581,4 +9368,5 @@ printf("Data In: %e\n", data); printf("Exited %s:\n", __FUNCTION__); + fflush(stdout); return data; } @@ -6590,6 +9378,7 @@ -install_name ./libtypetest.dylib - Now, restart OpenMCL. This step is required because OpenMCL cannot -close and reload a dynamic library on OS X. + Now, restart OpenMCL. This step is required because + OpenMCL cannot close and reload a dynamic library on OS + X. Have you restarted? Okay, try out the new code: @@ -6598,5 +9387,5 @@ ? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib") -# +#<SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E> ? (external-call "_si_si_test" :signed-fullword -178965 :signed-fullword) @@ -6606,5 +9395,5 @@ -178965 -? ;; long is the same size as int +? ;; long is the same size as int on 32-bit machines. (external-call "_sl_sl_test" :signed-fullword -178965 :signed-fullword) Entered sl_sl_test: @@ -6620,9 +9409,10 @@ -973891578912 - Okay, everything seems to be acting as expected. However, just to -remind you that most of this stuff has no safety net, here's -what happens if somebody mistakes sl_sl_test() -for sll_sll_test(), thinking that a long is -actually a doubleword: + Okay, everything seems to be acting as expected. However, + just to remind you that most of this stuff has no safety net, + here's what happens if somebody mistakes + sl_sl_test() for + sll_sll_test(), thinking that a long is + actually a doubleword: @@ -6633,9 +9423,11 @@ Exited sl_sl_test: -974957576192 - - Ouch. The C function changes the value with no warning that something -is wrong. Even worse, it manages to pass the original value back to -OpenMCL, which hides the fact that something is wrong. - Finally, let's take a look at doing this with floating-point numbers. + + Ouch. The C function changes the value with no warning + that something is wrong. Even worse, it manages to pass the + original value back to OpenMCL, which hides the fact that + something is wrong. + Finally, let's take a look at doing this with + floating-point numbers. @@ -6643,5 +9435,5 @@ ? (open-shared-library "/Users/andewl/openmcl/libtypetest.dylib") -# +#<SHLIB /Users/andewl/openmcl/libtypetest.dylib #x638EF3E> ? (external-call "_f_f_test" :single-float -1.256791e+11 :single-float) @@ -6656,5 +9448,5 @@ Exited d_d_test: -1.256791D+290 - + Notice that the number ends with "...e+11" for the single-float, and "...d+290" for the @@ -6670,82 +9462,81 @@ - Acknowledgement -This chapter was generously contributed by -Andrew P. Lentvorski Jr. + Acknowledgement + This chapter was generously contributed by Andrew + P. Lentvorski Jr. - Tutorial: Allocating Foreign Data on the Lisp Heap -Not every foreign function is so marvelously easy to use as the -ones we saw in the last section. Some of them require you to -allocate a C struct, fill it in with your own information, and -pass it a pointer to the struct. Some of them require you to -allocate an empty struct so they can fill it in, and then you can -read the information out of it. - Also, some of them have their own structs and return a pointer to -that same struct every time you call them, but those are easier to -deal with, -so they won't -be covered in this section. - You might know that Lisp (and, indeed, most programming languages) -has two separate regions of memory. There's the stack, which is -where variable bindings are kept. Memory on the stack is allocated -every time any function is called, and deallocated when it returns, -so it's useful for anything that doesn't need to last longer than -one function call, when there's only one thread. If that's all -you need, you can do it with -. - Then, there's the heap, which holds everything else, and is our -subject here. -There are two advantages and one big disadvantage to putting things on -the heap rather than the stack. First, data allocated on the heap can -be passed outside of the scope in which it was created. This is -useful for data which may need to be passed between multiple C calls -or multiple threads. Also, some data may be too large to copy multiple -times or may be too large to allocate on the stack. - The second advantage is security. If incoming data is being placed -directly onto the stack, the input data can cause stack overflows and -underflows. This is not something which Lisp users generally worry -about since garbage collection generally handles memory management. -However, "stack smashing" is one of the classic exploits in C which -malicious hackers can use to gain control of a machine. Not checking -external -data is always a bad idea; however, allocating it into the heap at -least offers more protection than direct stack allocation. - The big disadvantage to allocating data on the heap is that it must be -explicitly deallocated - you need to "free" it when you're done with -it. Ordinarily, in Lisp, you wouldn't allocate memory yourself, and -the garbage collector would know about it, so you wouldn't have to -think about it again. When you're doing it manually, it's very -different. -Memory management becomes a manual process, just like in C and -C++. - What that means is that, if you allocate something and then lose -track of the pointer to it, -there's no way to ever free that -memory. That's what's called a memory leak, and if your program -leaks enough memory it will eventually use up all of it! So, you -need to be careful to not lose your pointers. + Tutorial: Allocating Foreign Data on the Lisp Heap + Not every foreign function is so marvelously easy to use + as the ones we saw in the last section. Some of them require + you to allocate a C struct, fill it in with your own + information, and pass it a pointer to the struct. Some of them + require you to allocate an empty struct so they can fill it in, + and then you can read the information out of it. + Also, some of them have their own structs and return a + pointer to that same struct every time you call them, but those + are easier to deal with, so they won't be covered in this + section. + You might know that Lisp (and, indeed, most programming + languages) has two separate regions of memory. There's the + stack, which is where variable bindings are kept. Memory on the + stack is allocated every time any function is called, and + deallocated when it returns, so it's useful for anything that + doesn't need to last longer than one function call, when there's + only one thread. If that's all you need, you can do it with + . + Then, there's the heap, which holds everything else, and + is our subject here. There are two advantages and one big + disadvantage to putting things on the heap rather than the + stack. First, data allocated on the heap can be passed outside + of the scope in which it was created. This is useful for data + which may need to be passed between multiple C calls or multiple + threads. Also, some data may be too large to copy multiple times + or may be too large to allocate on the stack. + The second advantage is security. If incoming data is + being placed directly onto the stack, the input data can cause + stack overflows and underflows. This is not something which + Lisp users generally worry about since garbage collection + generally handles memory management. However, "stack smashing" + is one of the classic exploits in C which malicious hackers can + use to gain control of a machine. Not checking external data is + always a bad idea; however, allocating it into the heap at least + offers more protection than direct stack allocation. + The big disadvantage to allocating data on the heap is + that it must be explicitly deallocated - you need to "free" it + when you're done with it. Ordinarily, in Lisp, you wouldn't + allocate memory yourself, and the garbage collector would know + about it, so you wouldn't have to think about it again. When + you're doing it manually, it's very different. Memory + management becomes a manual process, just like in C and + C++. + What that means is that, if you allocate something and + then lose track of the pointer to it, there's no way to ever + free that memory. That's what's called a memory leak, and if + your program leaks enough memory it will eventually use up all + of it! So, you need to be careful to not lose your + pointers. That disadvantage, though, is also an advantage for using -foreign functions. Since the garbage collector doesn't know about -this memory, it will never move it around. External C code -needs this, because it doesn't know how to follow it to where it -moved, the way that Lisp code does. If you allocate data manually, -you can pass it to foreign code and know that no matter what that -code needs to do with it, it will be able to, until you -deallocated it. Of course, you'd better be sure it's done before -you do. Otherwise, your program will be unstable and might crash -sometime in the future, and you'll have trouble figuring out what -caused -the trouble, because there won't be anything pointing -back and saying "you deallocated this too soon." + foreign functions. Since the garbage collector doesn't know + about this memory, it will never move it around. External C + code needs this, because it doesn't know how to follow it to + where it moved, the way that Lisp code does. If you allocate + data manually, you can pass it to foreign code and know that no + matter what that code needs to do with it, it will be able to, + until you deallocated it. Of course, you'd better be sure it's + done before you do. Otherwise, your program will be unstable + and might crash sometime in the future, and you'll have trouble + figuring out what caused the trouble, because there won't be + anything pointing back and saying "you deallocated this too + soon." And, so, on to the code... As in the last tutorial, our first step -is to create a local dynamic library in order to help show -what is actually going on between OpenMCL and C. So, create the file -ptrtest.c, with the following code: + is to create a local dynamic library in order to help show + what is actually going on between OpenMCL and C. So, create the file + ptrtest.c, with the following code: -#include +#include <stdio.h> void reverse_int_array(int * data, unsigned int dataobjs) @@ -6753,5 +9544,5 @@ int i, t; - for(i=0; i - This defines three functions. reverse_int_array -takes a pointer to an array of ints, and a count -telling how many items are in the array, and loops through it -putting the elements in reverse. -reverse_int_ptr_array does the same thing, -but with an array of pointers to ints. It only -reverses the order the pointers are in; each pointer still points to -the same thing. -reverse_int_ptr_ptrtest -takes an array of pointers to arrays of ints. (With me?) -It doesn't need to be told their sizes; -it just assumes that the array of pointers has two items, -and that both of those are arrays which have four items. It -reverses the array of pointers, then it reverses each of the two -arrays of ints. - Now, compile ptrtest.c into a dynamic library using the command: + + This defines three functions. + reverse_int_array takes a pointer to an array + of ints, and a count telling how many items + are in the array, and loops through it putting the elements in + reverse. reverse_int_ptr_array does the same + thing, but with an array of pointers to ints. + It only reverses the order the pointers are in; each pointer + still points to the same thing. + reverse_int_ptr_ptrtest takes an array of + pointers to arrays of ints. (With me?) It + doesn't need to be told their sizes; it just assumes that the + array of pointers has two items, and that both of those are + arrays which have four items. It reverses the array of + pointers, then it reverses each of the two arrays of + ints. + Now, compile ptrtest.c into a dynamic library using the + command: gcc -dynamiclib -Wall -o libptrtest.dylib ptrtest.c -install_name ./libptrtest.dylib - + If that command doesn't make sense to you, feel free to go back and read about it at . @@ -6825,13 +9617,13 @@ DISPOSE-HEAP-IVECTOR - You don't understand how those functions do what they do. That's -okay; it gets into very fine detail which really doesn't matter, -because you don't need to change them. - The function make-heap-ivector is the primary -tool for -allocating objects in heap memory. It allocates a fixed-size OpenMCL -object in heap memory. It returns both an array reference, which can -be used directly from OpenMCL, and a macptr, which can -be used to access the underlying memory directly. For example: + If you don't understand how those functions do what they do. + That's okay; it gets into very fine detail which really doesn't + matter, because you don't need to change them. + The function make-heap-ivector is the + primary tool for allocating objects in heap memory. It + allocates a fixed-size OpenMCL object in heap memory. It + returns both an array reference, which can be used directly from + OpenMCL, and a macptr, which can be used to + access the underlying memory directly. For example: ? ;; Create an array of 3 4-byte-long integers @@ -6843,5 +9635,5 @@ ; Undeclared free variable A, in an anonymous lambda form. ; Undeclared free variable AP, in an anonymous lambda form. -# +#<A Mac Pointer #x10217C> ? a @@ -6849,21 +9641,17 @@ ? ap -# +#<A Mac Pointer #x10217C> It's important to realize that the contents of the -ivector we've just created -haven't been initialized, -so their values are unpredictable, -and you should be sure not to -read from them before you set them, to avoid confusing results. - At this point, a references an object which -works just -like a normal array. You can refer to any item of it with -the standard aref function, and set them -by combining that with setf. -As noted -above, the ivector's -contents haven't been initialized, -so that's the next order of business: + ivector we've just created haven't been + initialized, so their values are unpredictable, and you should + be sure not to read from them before you set them, to avoid + confusing results. + At this point, a references an object + which works just like a normal array. You can refer to any item + of it with the standard aref function, and + set them by combining that with setf. As + noted above, the ivector's contents haven't + been initialized, so that's the next order of business: ? a @@ -6885,6 +9673,6 @@ #(3 4 5) - In addition, the macptr allows direct access to the same -memory: + In addition, the macptr allows direct + access to the same memory: ? (setq *byte-length-of-long* 4) @@ -6906,14 +9694,14 @@ a #(6 4 7) - - So far, there is nothing about this object that could not be done much -better with standard Lisp. However, the macptr can be -used to pass this chunk of memory off to a C function. Let's use -the C -code to reverse the elements in the array: + + So far, there is nothing about this object that could not + be done much better with standard Lisp. However, the + macptr can be used to pass this chunk of + memory off to a C function. Let's use the C code to reverse the + elements in the array: ? ;; Insert the full path to your copy of libptrtest.dylib (open-shared-library "/Users/andrewl/openmcl/openmcl/gtk/libptrtest.dylib") -# +#<SHLIB /Users/andrewl/openmcl/openmcl/gtk/libptrtest.dylib #x639D1E6> ? a @@ -6921,8 +9709,8 @@ ? ap -# +#<A Mac Pointer #x10217C> ? (external-call "_reverse_int_array" :address ap :unsigned-int (length a) :address) -# +#<A Mac Pointer #x10217C> ? a @@ -6930,16 +9718,15 @@ ? ap -# +#<A Mac Pointer #x10217C> The array gets passed correctly to the C function, -reverse_int_array. The C function -reverses the contents of the array in-place; that is, it doesn't make -a new array, just keeps the same one and reverses what's in it. -Finally, the C function -passes control back to OpenMCL. Since the allocated array memory has -been directly modifed, OpenMCL reflects those changes directly in the -array as well. - There is one final bit of housekeeping to deal with. Before moving -on, the memory needs to be deallocated: + reverse_int_array. The C function reverses + the contents of the array in-place; that is, it doesn't make a + new array, just keeps the same one and reverses what's in it. + Finally, the C function passes control back to OpenMCL. Since + the allocated array memory has been directly modifed, OpenMCL + reflects those changes directly in the array as well. + There is one final bit of housekeeping to deal with. + Before moving on, the memory needs to be deallocated: ? ;; dispose-heap-ivector created for symmetry @@ -6963,60 +9750,57 @@ The dispose-heap-ivector macro actually -deallocates the -ivector, releasing its memory into the heap for something else to -use. In addition, it makes sure -that the -variables which it was called with are set to nil, because otherwise -they would still be referencing the memory of the ivector - which is -no longer allocated, so that would be a bug. Making sure there are -no other variables set to it is up to you. + deallocates the ivector, releasing its memory into the heap for + something else to use. In addition, it makes sure that the + variables which it was called with are set to nil, because + otherwise they would still be referencing the memory of the + ivector - which is no longer allocated, so that would be a bug. + Making sure there are no other variables set to it is up to + you. When do you call dispose-heap-ivector? -Anytime after you know the ivector will never be used again, but no -sooner. If you have a lot of ivectors, say, in a hash table, -you need to make sure that when whatever you were doing with the -hash table is done, those ivectors all get freed. Unless there's -still something somewhere else which refers to them, of course! -Exactly what strategy to take depends on the situation, -so just try to keep things simple unless you know better. - The simplest situation is when you have things set up so that a Lisp -object "encapsulates" a pointer to foreign data, taking care of all -the -details of using it. In this case, you don't want those two things -to have different lifetimes: You want to make sure your Lisp object -exists as long as the foreign data does, and no longer; and you want -to make sure the foreign data doesn't get deallocated while your -Lisp object still refers to it. - If you're willing to accept a few limitations, you can make this -easy. First, you can't let foreign code keep a permanent pointer -to the -memory; it has to always finish what it's doing, then return, and -not refer to that memory again. Second, you can't let any Lisp -code that isn't part of your encapsulating "wrapper" refer to the -pointer directly. Third, nothing, either foreign code or Lisp -code, should explicitly deallocate the memory. - If you can make sure all of these are true, you can at least -ensure that the foreign -pointer is deallocated when the encapsulating object is about to -become -garbage, by using OpenMCL's nonstandard "termination" mechanism, -which is essentially the same as what Java and other languages -call "finialization". - Termination is a way of asking the garbage collector to let you know -when it's about to destroy an object which isn't used anymore. -Before destroying the object, it calls a function which you write, -called a terminator. + Anytime after you know the ivector will never be used again, but + no sooner. If you have a lot of ivectors, say, in a hash table, + you need to make sure that when whatever you were doing with the + hash table is done, those ivectors all get freed. Unless + there's still something somewhere else which refers to them, of + course! Exactly what strategy to take depends on the situation, + so just try to keep things simple unless you know better. + The simplest situation is when you have things set up so + that a Lisp object "encapsulates" a pointer to foreign data, + taking care of all the details of using it. In this case, you + don't want those two things to have different lifetimes: You + want to make sure your Lisp object exists as long as the foreign + data does, and no longer; and you want to make sure the foreign + data doesn't get deallocated while your Lisp object still refers + to it. + If you're willing to accept a few limitations, you can + make this easy. First, you can't let foreign code keep a + permanent pointer to the memory; it has to always finish what + it's doing, then return, and not refer to that memory again. + Second, you can't let any Lisp code that isn't part of your + encapsulating "wrapper" refer to the pointer directly. Third, + nothing, either foreign code or Lisp code, should explicitly + deallocate the memory. + If you can make sure all of these are true, you can at + least ensure that the foreign pointer is deallocated when the + encapsulating object is about to become garbage, by using + OpenMCL's nonstandard "termination" mechanism, which is + essentially the same as what Java and other languages call + "finialization". + Termination is a way of asking the garbage collector to + let you know when it's about to destroy an object which isn't + used anymore. Before destroying the object, it calls a function + which you write, called a terminator. So, you can use termination to find out when a particular -macptr is about to become garbage. -That's not quite -as helpful as it might seem: It's not exactly the same thing as -knowing that the block of memory it points to is unreferenced. -For example, there could be another macptr somewhere -to the -same block; or, if it's a struct, there could be a -macptr to one of its fields. Most problematically, -if the address of that memory has been passed to foreign code, -it's sometimes hard to know whether that code has kept the -pointer. Most foreign functions don't, but it's not hard to think of -exceptions. + macptr is about to become garbage. That's + not quite as helpful as it might seem: It's not exactly the same + thing as knowing that the block of memory it points to is + unreferenced. For example, there could be another + macptr somewhere to the same block; or, if + it's a struct, there could be a macptr to one + of its fields. Most problematically, if the address of that + memory has been passed to foreign code, it's sometimes hard to + know whether that code has kept the pointer. Most foreign + functions don't, but it's not hard to think of + exceptions. You can use code such as this to make all this happen: @@ -7043,789 +9827,1367 @@ (setq ivector nil macptr nil)))) - - The ccl:terminate method will be called on -some arbitrary thread -sometime (hopefully soon) after the GC has decided that there are no -strong references to an object which has been the argument of a -ccl:terminate-when-unreachable call. - If it makes sense to say that the foreign object should live as long -as there's Lisp code that references it (through the encapsulating -obect) and no longer, this is one way of doing that. - Now we've covered passing basic types back and forth with C, and -we've done the same with pointers. You may think this is all... -but we've only done pointers to basic types. Join us next time -for pointers... to pointers. + + The ccl:terminate method will be called + on some arbitrary thread sometime (hopefully soon) after the GC + has decided that there are no strong references to an object + which has been the argument of a + ccl:terminate-when-unreachable call. + If it makes sense to say that the foreign object should + live as long as there's Lisp code that references it (through + the encapsulating obect) and no longer, this is one way of doing + that. + Now we've covered passing basic types back and forth with + C, and we've done the same with pointers. You may think this is + all... but we've only done pointers to basic types. Join us + next time for pointers... to pointers. - Acknowledgement -Much of this chapter was generously contributed by -Andrew P. LentvorskiJr. + Acknowledgement + Much of this chapter was generously contributed by + Andrew P. Lentvorski Jr. - - The Foreign-Function-Interface Dictionary - - - DEF-FOREIGN-TYPE - def-foreign-type - Name - DEF-FOREIGN-TYPE — - Macro - Synopsis - - - def-foreign-type name foreign-type-spec - - - Values - name - NIL or a keyword; the keyword may containescaping constructs (see ). - foreign-type-spec - A foreign type specifier, whose syntax is (loosely)defined above. - - - Description - If name is non-NIL, defines name to be an alias for the -foreign type specified by foreign-type-spec. If foreign-type-spec -is a named structure or union type, additionally defines that -structure or union type. - If name is NIL, foreign-type-spec must be a named foreign -struct or union definition, in which case the foreign structure -or -union definition is put in effect. - Note that there are two separate namespaces for foreign -type names, one for the names of ordinary types and one for -the names of structs and unions. Which one -name refers to depends on -foreign-type-spec in the obvious manner. - - - - MAKE-RECORD - make-record - Name - MAKE-RECORD — - Macro - Synopsis - - - make-record typespec - &rest initforms => result - - - Values - typespec - A foreign type specifier, or a keyword which is usedas the name of a foreign struct or union. - initforms - If the type denoted by typespecis scalar, a single value appropriate for that type;otherwise, a list of alternating field names andvalues appropriate for the types of those fields. - result - A macptr which encapsulates the address of anewly-allocated record on the foreign heap. - - - Description - Expands into code which allocates and initalizes -an instance of the type -denoted by typespec, on the foreign -heap. The record is allocated using the C function -malloc, and the user of -make-record must explicitly call -the C function free to deallocate the -record, when it is no longer needed. - If initforms is provided, its value -or values are used in the initialization. When the type -is a scalar, initforms is either a single -value which can be coerced to that type, or no value, in which -case binary 0 is used. When the type is a struct, -initforms is a list, giving field names -and the values for each. Each field is treated in the same way -as a scalar is: If a value for it is given, it must be -coerceable to the field's type; if not, binary 0 is used. - When the type is an array, initforms may -not be provided, because make-record -cannot initialize its values. make-record -is also unable to initialize fields of a struct -which are themselves -structs. The user of -make-record should set these values -by another means. - A possibly-significant limitation is that it must be possible to -find the foreign type at the time the macro is expanded; -make-record signals an error if this is -not the case. - Notes - It is inconvenient that make-record is a -macro, because this means that typespec -cannot be a variable; it must be an immediate value. - If it weren't for this requirement, -make-record could be a function. However, -that would mean that any stand-alone application using it would -have to include a copy of the interface database -(see The Interface Database), which is undesireable -because it's large. - - - - RLET - rlet - Name - RLET — - Macro - Synopsis - - - rlet (var typespec &rest initforms)* - &body body - - - Values - var - A symbol (a lisp variable) - typespec - A foreign type specifier or foreign record name. - initforms - As described above, for - - - Description - Executes body -in an environment in which each var is bound -to a MACPTR (see ) encapsulating the -address of a stack-allocated foreign memory block, allocated and -initialized from typespec and initforms as per -. -Returns whatever value(s) body -returns. - Record fields that aren't explicitly initialized have -unspecified contents. - - - - RLETZ - rletz - Name - RLETZ — - Macro - Synopsis - - - rletz (var typespec &rest initforms)* - &body body - - - Values - var - A symbol (a lisp variable) - typespec - A foreign type specifier or foreign record name. - initforms - As described above, for ccl:make-record - - - Description - Executes body in an environment in which each var is bound -to a MACPTR (see ) encapuslating the -address of a stack-allocated foreign memory block, allocated and -initialized from typespec and initforms as -ccl:make-record. - Returns whatever value(s) body returns. - Unlike rlet, record fields that aren't explicitly -initialized are set to binary 0. - - - - PREF - pref - Name - PREF — - Macro - Synopsis - - - pref ptr accessor-form - - - Values - ptr - a MACPTR (see ). - accessor-form - a keyword which names a foreign type or record, asdescribed in . - - - Description - References an instance of a foreign type (or a component of -a foreign type) accessible via ptr. - Expands into code which references the indicated scalar type -or component, or returns a pointer to a composite type. - PREF can be used with SETF. - RREF is a deprecated alternative to PREF. It accepts a -:STORAGE keyword and rather loudly ignores it. - - - - OPEN-SHARED-LIBRARY - open-shared-library - Name - OPEN-SHARED-LIBRARY — Asks the operating system to load a shared library -for OpenMCL to use. - Function - Synopsis - - - open-shared-library name => library - - - Values - name - A SIMPLE-STRING which is presumed to be the so-name ofor a filesystem path to the library. - library - An object of type SHLIB which describes thelibrary denoted by name. - - - Description - If the library denoted by name can -be loaded by the -operating system, returns an object of type SHLIB that describes -the library; if the library is already open, increments a -reference count. If the library can't be loaded, signals a -SIMPLE-ERROR which contains an often-cryptic message from the -operating system. - Examples - -;;; Try to do something simple. -? (open-shared-library "libgtk.so") -> Error: Error opening shared library "libgtk.so": /usr/lib/libgtk.so: undefined symbol: gdk_threads_mutex -> While executing: OPEN-SHARED-LIBRARY - -;;; Grovel around, curse, and try to find out where "gdk_threads_mutex" + + The Foreign-Function-Interface Dictionary + + + + def-foreign-type + + + + DEF-FOREIGN-TYPE + + Macro + + + + + def-foreign-type name foreign-type-spec + + + + + Values + + + + name + + + NIL or a keyword; the keyword may contain + escaping constructs. + + + + + foreign-type-spec + + + A foreign type specifier, whose syntax is (loosely) + defined above. + + + + + + + Description + + If name is non-NIL, defines name to be an alias for the + foreign type specified by foreign-type-spec. If foreign-type-spec + is a named structure or union type, additionally defines that + structure or union type. + + If name is NIL, foreign-type-spec must be a named foreign + struct or union definition, in which case the foreign structure + or + union definition is put in effect. + + Note that there are two separate namespaces for foreign + type names, one for the names of ordinary types and one for + the names of structs and unions. Which one + name refers to depends on + foreign-type-spec in the obvious manner. + + + + + + + make-record + + + + MAKE-RECORD + + Macro + + + + + make-record typespec + &rest; initforms => result + + + + + Values + + + + typespec + + + A foreign type specifier, or a keyword which is used + as the name of a foreign struct or union. + + + + + initforms + + + If the type denoted by typespec + is scalar, a single value appropriate for that type; + otherwise, a list of alternating field names and + values appropriate for the types of those fields. + + + + + result + + + + A macptr which encapsulates the address of a + newly-allocated record on the foreign heap. + + + + + + + + Description + + + Expands into code which allocates and initalizes + an instance of the type + denoted by typespec, on the foreign + heap. The record is allocated using the C function + malloc, and the user of + make-record must explicitly call + the C function free to deallocate the + record, when it is no longer needed. + + + + If initforms is provided, its value + or values are used in the initialization. When the type + is a scalar, initforms is either a single + value which can be coerced to that type, or no value, in which + case binary 0 is used. When the type is a struct, + initforms is a list, giving field names + and the values for each. Each field is treated in the same way + as a scalar is: If a value for it is given, it must be + coerceable to the field's type; if not, binary 0 is used. + + + + When the type is an array, initforms may + not be provided, because make-record + cannot initialize its values. make-record + is also unable to initialize fields of a struct + which are themselves + structs. The user of + make-record should set these values + by another means. + + + + A possibly-significant limitation is that it must be possible to + find the foreign type at the time the macro is expanded; + make-record signals an error if this is + not the case. + + + + + Notes + + + It is inconvenient that make-record is a + macro, because this means that typespec + cannot be a variable; it must be an immediate value. + + + + If it weren't for this requirement, + make-record could be a function. However, + that would mean that any stand-alone application using it would + have to include a copy of the interface database + (see ), which is undesireable + because it's large. + + + + + + + rlet + + + + RLET + + Macro + + + + + rlet (var typespec &rest; initforms)* + &body; body + + + + + Values + + + + var + + + A symbol (a lisp variable) + + + + + typespec + + + A foreign type specifier or foreign record name. + + + + + initforms + + + As described above, for + + + + + + + + Description + + Executes body + in an environment in which each var is bound + to a MACPTR encapsulating the + address of a stack-allocated foreign memory block, allocated and + initialized from typespec and initforms as per + . + Returns whatever value(s) body + returns. + + Record fields that aren't explicitly initialized have + unspecified contents. + + + + + + rletz + + + + RLETZ + + Macro + + + + + rletz (var typespec &rest; initforms)* + &body; body + + + + + Values + + + + var + + + A symbol (a lisp variable) + + + + + typespec + + + A foreign type specifier or foreign record name. + + + + + initforms + + + As described above, for ccl:make-record + + + + + + + Description + + Executes body in an environment in which each var is + bound to a + MACPTR encapuslating the address of a stack-allocated + foreign memory block, allocated and initialized from + typespec and initforms as ccl:make-record. + + Returns whatever value(s) body returns. + + Unlike rlet, record fields that aren't explicitly + initialized are set to binary 0. + + + + + + pref + + + + PREF + + Macro + + + + + pref ptr accessor-form + + + + + + Values + + + + ptr + + + a MACPTR. + + + + + accessor-form + + + a keyword which names a foreign type or record, as + described in . + + + + + + + + Description + + References an instance of a foreign type (or a component of + a foreign type) accessible via ptr. + + Expands into code which references the indicated scalar type + or component, or returns a pointer to a composite type. + + PREF can be used with SETF. + + RREF is a deprecated alternative to PREF. It accepts a + :STORAGE keyword and rather loudly ignores it. + + + + + + open-shared-library + + + + OPEN-SHARED-LIBRARY + Asks the operating system to load a shared library + for OpenMCL to use. + Function + + + + + open-shared-library name => library + + + + + Values + + + + name + + A SIMPLE-STRING which is presumed to be the so-name of + or a filesystem path to the library. + + + + + library + + An object of type SHLIB which describes the + library denoted by name. + + + + + + + Description + + If the library denoted by name can + be loaded by the + operating system, returns an object of type SHLIB that describes + the library; if the library is already open, increments a + reference count. If the library can't be loaded, signals a + SIMPLE-ERROR which contains an often-cryptic message from the + operating system. + + + + Examples + + ;;; Try to do something simple. +? (open-shared-library "libgtk.so") +> Error: Error opening shared library "libgtk.so": /usr/lib/libgtk.so: undefined symbol: gdk_threads_mutex +> While executing: OPEN-SHARED-LIBRARY + +;;; Grovel around, curse, and try to find out where "gdk_threads_mutex" ;;; might be defined. Then try again: -? (open-shared-library "libgdk.so") -# - -? (open-shared-library "libgtk.so") -# +? (open-shared-library "libgdk.so") +#<SHLIB libgdk.so #x3046DBB6> + +? (open-shared-library "libgtk.so") +#<SHLIB libgtk.so #x3046DC86> ;;; Reference an external symbol defined in one of those libraries. -? (external "gtk_main") -# +? (external "gtk_main") +#<EXTERNAL-ENTRY-POINT "gtk_main" (#x012C3004) libgtk.so #x3046FE46> ;;; Close those libraries. -? (close-shared-library "libgtk.so") +? (close-shared-library "libgtk.so") T -? (close-shared-library "libgdk.so") +? (close-shared-library "libgdk.so") T ;;; Reference the external symbol again. -? (external "gtk_main") -# - - Notes - It would be helpful to describe what an soname is and give -examples of one. - Does the SHLIB still get returned if the library is -already open? - - - - CLOSE-SHARED-LIBRARY - close-shared-library - Name - CLOSE-SHARED-LIBRARY — Stops using a shared library, informing the operating -system that it can be unloaded if appropriate. - Function - Synopsis - - - close-shared-library library &key - completely - - Values - library - either an object of type SHLIB, or a string whichdesignates one by its so-name. - completely - a boolean. The default is T. - - - Description - If completely is T, sets the -reference count of library to 0. Otherwise, -decrements it by 1. In either case, if the reference count -becomes 0, close-shared-library -frees all memory resources consumed library -and -causes any EXTERNAL-ENTRY-POINTs known to be defined by it to -become unresolved. - - - - EXTERNAL - external - Name - EXTERNAL — Resolves a reference to an external symbol which -is defined in a shared library. - Macro - Synopsis - - - external name => entry - - - Values - name - a simple-string which names an external symbol.Case-sensitive. - entry - an object of type EXTERNAL-ENTRY-POINT which maintainsthe address of the foreign symbol named byname. - - - Description - If there is already an EXTERNAL-ENTRY-POINT for -the symbol named by name, finds it and -returns it. If not, creates one and returns it. - Tries to resolve the entry point to a memory address, -and identify the containing library. - Be aware that under Darwin, external functions which -are callable from C have underscores prepended to their names, -as in "_fopen". - - - - %FF-CALL - %ff-call - Name - %FF-CALL — - Function - Synopsis - - - %ff-call entrypoint - {arg-type-keyword arg}* &optional result-type-keyword - - - Values - entrypoint - A fixnum or MACPTR - arg-type-keyword - One of the foreign argument-type keywords, describedabove - arg - A lisp value of type indicated by the correspondingarg-type-keyword - result-type-keyword - One of the foreign argument-type keywords, describedabove - - - Description - Calls the foreign function at address entrypoint passing the -values of each arg as a foreign argument of type indicated by the -corresponding arg-type-keyword. Returns the foreign function -result (coerced to a Lisp object of type indicated by -result-type-keyword), or NIL if result-type-keyword is :VOID or -NIL - - - - FF-CALL - ff-call - Name - FF-CALL — - Macro - Synopsis - - - ff-call entrypoint - {arg-type-specifier arg}* &optional result-type-specifier - - - Values - entrypoint - A fixnum or MACPTR - arg-type-specifer - One of the foreign argument-type keywords, describedabove, or an equivalent foreigntype specifier (see ). - arg - A lisp value of type indicated by the correspondingarg-type-specifier - result-type-specifier - One of the foreign argument-type keywords, describedabove, or an equivalent foreigntype specifier (see ). - - - Description - Calls the foreign function at address entrypoint passing the -values of each arg as a foreign argument of type indicated by the -corresponding arg-type-specifier. Returns the foreign function -result (coerced to a Lisp object of type indicated by -result-type-specifier), or NIL if result-type-specifer is :VOID or -NIL - - - - %REFERENCE-EXTERNAL-ENTRY-POINT - %reference-external-entry-point - Name - %REFERENCE-EXTERNAL-ENTRY-POINT — - Function - Synopsis - - - %reference-external-entry-point eep - - - Values - eep - An EXTERNAL-ENTRY-POINT, as obtained by the EXTERNALmacro. - - - Description - Tries to resolve the address of the EXTERNAL-ENTRY-POINT -eep; returns a fixnum representation of that address if -successful, else signals an error. - - - - EXTERNAL-CALL - external-call - Name - EXTERNAL-CALL — - Macro - Synopsis - - - external-call name - {arg-type-specifier arg}* &optional result-type-specifier - - - Values - name - A lisp string. See external, above. - arg-type-specifer - One of the foreign argument-type keywords, describedabove, or an equivalent foreigntype specifier (see ). - arg - A lisp value of type indicated by the correspondingarg-type-specifier - result-type-specifier - One of the foreign argument-type keywords, describedabove, or an equivalent foreigntype specifier (see ). - - - Description - Calls the foreign function at the address obtained by -resolving the external-entry-point associated with name, passing -the values of each arg as a foreign argument of type indicated by -the corresponding arg-type-specifier. Returns the foreign function -result (coerced to a Lisp object of type indicated by -result-type-specifier), or NIL if result-type-specifer is :VOID or -NIL - - - - FOREIGN-SYMBOL-ENTRY - foreign-symbol-entry - Name - FOREIGN-SYMBOL-ENTRY — - Function - Synopsis - - - foreign-symbol-entry name - - - Values - name - A lisp string. - - - Description - Tries to resolve the address of the foreign symbol name. If -successful, returns a fixnum representation of that address, else -returns NIL. - - - - FOREIGN-SYMBOL-ADDRESS - foreign-symbol-address - Name - FOREIGN-SYMBOL-ADDRESS — - Function - Synopsis - - - foreign-symbol-address name - - - Values - name - A lisp string. - - - Description - Tries to resolve the address of the foreign symbol name. If -successful, returns that address encapsulated -in a MACPTR (see ), else returns -NIL. - - - - DEFCALLBACK - defcallback - Name - DEFCALLBACK — - Macro - Synopsis - - - defcallback name - ({arg-type-specifier var}* &optional result-type-specifier) - &body body - - - Values - name - A symbol which can be made into a special variable - arg-type-specifer - One of the foreign argument-type keywords, describedabove, or an equivalent foreigntype specifier (see ).In addition, if the keyword:WITHOUT-INTERRUPTS is specified, the callback will beexecuted with lisp interrupts disabled if the correspondingvar is non-NIL. If :WITHOUT-INTERRUPTS is specified morethan once, the rightmost instance wins. - var - A symbol (lisp variable), which will be bound to avalue of the specified type. - body - A sequence of lisp forms, which should return a valuewhich can be coerced to the specified result-type. - - - Description - Proclaims name -to be a special variable; sets its value to a -MACPTR which, when called by foreign code, calls a lisp function -which expects foreign arguments of the specified types and which -returns a foreign value of the specified result type. Any argument -variables which correspond to foreign arguments of type :ADDRESS -are bound to stack-allocated MACPTRs. - If name -is already a callback function pointer, its value is -not changed; instead, it's arranged -that an -updated version of the lisp callback function will be called. -This feature allows for callback functions to be redefined -incrementally, just like Lisp functions are. - defcallback -returns the callback pointer, e.g., the -value of name. - - - - SHARP-AMPERSAND - #_ - Name - #_ — - Reader Macro - Description - Reads a symbol from the current input stream, with *PACKAGE* -bound to the "OS" package and with readtable-case preserved. - Does a lookup on that symbol -in the OpenMCL interface database (see ), -signalling -an error if no foreign function information can be found for the -symbol in any -active interface directory (see ). - Notes the foreign function information, including the foreign -function's return type, the number and type of the foreign -function's required arguments, and an indication of whether or -not the function accepts additional arguments (via e.g., the -"varargs" mechanism in C). - Defines a macroexpansion function on the symbol, which expand -macro calls involving the symbol into EXTERNAL-CALL forms where -foreign argument type specifiers for required arguments and the -return value specifer are provided from the information ind the -database. - Returns the symbol. - The effect of these steps is that it's possible to call -foreign functions that take fixed numbers of arguments by simply -providing argument values, as in: - -(#_isatty fd) -(#_read fd buf n) - - and to call foreign functions that take variable numbers of -arguments by specifying the types of non-required args, as in: - -(with-cstrs ((format-string "the answer is: %d")) - (#_printf format-string :int answer)) - - - - - SHARP-AMPERSAND - SHARP-AMPERSAND - Name - #& — - Reader Macro - Description - In OpenMCL 1.0 and later, the #& reader macro This -functionality was introduced as #? in 0.14.2, but ANSI CL requires -that #? be reserved for the user. #? is still supported as an alias -to #&, but that will change in the next release. can be used to -access foreign variables; this functionality depends on the presence -of "vars.cdb" files in the interface database. The current behavior of -the #& reader macro is to: - Read a symbol from the current input stream, with *PACKAGE* -bound to the "OS" package and with readtable-case preserved. - Use that symbol's pname to access the OpenMCL interface -database, signalling an error if no appropriate foreign variable -information can be found with that name in any active interface -directory. - Use type information recorded in the database to construct a -form which can be used to access the foreign variable, and return -that form. - Please note that the set of foreign variables declared in header files -may or may not match the set of foreign variables exported from -libraries (we're generally talking about C and Unix here ...). When -they do match, the form constructed by the #& reader macro manages the -details of resolving and tracking changes to the foreign variable's -address. - Future extensions (via prefix arguments to the reader macro) may -offer additional behavior; it might be convenient (for instance) to be -able to access the address of a foreign variable without dereferencing -that address. - Foreign variables in C code tend to be platform- and -package-specific (the canonical example - "errno" - is typically -not a variable when threads are involved. ) - In LinuxPPC, - -? #&stderr - - returns a pointer to the stdio error stream ("stderr" is a -macro under OSX/Darwin). - On both LinuxPPC and DarwinPPC, - -? #&sys_errlist - - returns a pointer to a C array of C error message strings. - - - - USE-INTERFACE-DIR - use-interface-dir - Name - USE-INTERFACE-DIR — - Function - Synopsis - - - use-interface-dir dir-id - - - Values - dir-id - A keyword whose pname, mapped to lower case, names asubdirectory of "ccl:headers;" (or"ccl:darwin-headers;") - - - Description - Tells OpenMCL to add the interface directory denoted by -dir-id to the list of interface directories which it consults for -foreign type and function information. Arranges that that -directory is searched before any others. - Note that use-interface-dir -merely adds an entry -to a search list. -If the named directory doesn't exist in the file system -or doesn't -contain a set of database files, a runtime error may occur -when OpenMCL -tries to open some database file in that directory, and it -will try to -open such a database file whenever it needs to find any -foreign type or -function information. -may come in -handy in that case. - Examples - One typically wants interface information to be -available at compile-time (or, in many cases, at read-time). -A typical idiom would be: - -(eval-when (:compile-toplevel :execute) - (use-interface-dir :GTK)) - - Using the :GTK interface directory makes available -information on -foreign types, functions, and constants. It's generally -necessary to -load foreign libraries before actually calling the -foreign code, which for GTK can be done like this: - -(load-gtk-libraries) - - It should now be possible to do things like: - -(#_gtk_widget_destroy w) - - - - - UNUSE-INTERFACE-DIR - unuse-interface-dir - Name - UNUSE-INTERFACE-DIR — - Function - Synopsis - - - unuse-interface-dir dir-id - - - Values - dir-id - A keyword whose pname, mapped to lower case, names asubdirectory of "ccl:headers;" (or"ccl:darwin-headers;") - - - Description - Tells OpenMCL to remove the interface directory denoted by -dir-id from the list of interface directories which are -consulted for -foreign type and function information. Returns T if the directory -was on the search list, NIL otherwise. - - - - TERMINATE-WHEN-UNREACHABLE - terminate-when-unreachable - Name - TERMINATE-WHEN-UNREACHABLE — - Function - Synopsis - - - terminate-when-unreachable object - - - Values - object - A CLOS object of a class for which there existsa method of the generic functionccl:terminate. - - - Description - The "termination" mechanism is a way to have the garbage -collector run a function right before an object is about to -become garbage. It is very similar to the "finalization" -mechanism which Java has. It is not standard Common Lisp, -although other Lisp implementations have similar features. -It is useful when there is some sort of special cleanup, -deallocation, or releasing of resources which needs to happen -when a certain object is no longer being used. - When the garbage collector discovers that an object is no -longer referred to anywhere in the program, it deallocates -that object, freeing its memory. However, if -ccl:terminate-when-unreachable has been -called on the object at any time, the garbage collector first -invokes the generic function ccl:terminate, -passing it the object as a parameter. - Therefore, to make termination do something useful, you need to -define a method on ccl:terminate. - Because calling -ccl:terminate-when-unreachable only -affects a single object, rather than all objects of its -class, you -may wish to put a call to it in the -initialize-instance method of a -class. Of course, this is only appropriate if you do in fact -want to use termination for all objects of a given class. - Example - +? (external "gtk_main") +#<EXTERNAL-ENTRY-POINT "gtk_main" {unresolved} libgtk.so #x3046FE46> + + + + Notes + + It would be helpful to describe what an soname is and give + examples of one. + + Does the SHLIB still get returned if the library is + already open? + + + + + + close-shared-library + + + + CLOSE-SHARED-LIBRARY + Stops using a shared library, informing the operating + system that it can be unloaded if appropriate. + Function + + + + + close-shared-library library &key; + completely + + + + Values + + + + library + + + either an object of type SHLIB, or a string which + designates one by its so-name. + + + + + completely + + + a boolean. The default is T. + + + + + + + Description + + If completely is T, sets the + reference count of library to 0. Otherwise, + decrements it by 1. In either case, if the reference count + becomes 0, close-shared-library + frees all memory resources consumed library + and + causes any EXTERNAL-ENTRY-POINTs known to be defined by it to + become unresolved. + + + + + + external + + + + EXTERNAL + Resolves a reference to an external symbol which + is defined in a shared library. + Macro + + + + + external name => entry + + + + + Values + + + + name + + + a simple-string which names an external symbol. + Case-sensitive. + + + + + entry + + + an object of type EXTERNAL-ENTRY-POINT which maintains + the address of the foreign symbol named by + name. + + + + + + + + Description + + If there is already an EXTERNAL-ENTRY-POINT for + the symbol named by name, finds it and + returns it. If not, creates one and returns it. + + Tries to resolve the entry point to a memory address, + and identify the containing library. + + Be aware that under Darwin, external functions which + are callable from C have underscores prepended to their names, + as in "_fopen". + + + + + + %ff-call + + + + %FF-CALL + + Function + + + + + %ff-call entrypoint + {arg-type-keyword arg}* &optional; result-type-keyword + + + + + Values + + + + entrypoint + + + A fixnum or MACPTR + + + + + arg-type-keyword + + + One of the foreign argument-type keywords, described + above + + + + + arg + + + A lisp value of type indicated by the corresponding + arg-type-keyword + + + + + result-type-keyword + + + One of the foreign argument-type keywords, described + above + + + + + + + Description + + Calls the foreign function at address entrypoint passing the + values of each arg as a foreign argument of type indicated by the + corresponding arg-type-keyword. Returns the foreign function + result (coerced to a Lisp object of type indicated by + result-type-keyword), or NIL if result-type-keyword is :VOID or + NIL + + + + + + ff-call + + + + FF-CALL + + Macro + + + + + ff-call entrypoint + {arg-type-specifier arg}* &optional; result-type-specifier + + + + + Values + + + + entrypoint + + + A fixnum or MACPTR + + + + + arg-type-specifer + + + One of the foreign argument-type keywords, described + above, or an equivalent foreign + type specifier. + + + + + arg + + + A lisp value of type indicated by the corresponding + arg-type-specifier + + + + + result-type-specifier + + + One of the foreign argument-type keywords, described + above, or an equivalent foreign + type specifier. + + + + + + + Description + + Calls the foreign function at address entrypoint passing the + values of each arg as a foreign argument of type indicated by the + corresponding arg-type-specifier. Returns the foreign function + result (coerced to a Lisp object of type indicated by + result-type-specifier), or NIL if result-type-specifer is :VOID or + NIL + + + + + + %reference-external-entry-point + + + + %REFERENCE-EXTERNAL-ENTRY-POINT + + Function + + + + + %reference-external-entry-point eep + + + + + Values + + + + eep + + + An EXTERNAL-ENTRY-POINT, as obtained by the EXTERNAL + macro. + + + + + + + Description + + Tries to resolve the address of the EXTERNAL-ENTRY-POINT + eep; returns a fixnum representation of that address if + successful, else signals an error. + + + + + + external-call + + + + EXTERNAL-CALL + + Macro + + + + + external-call name + {arg-type-specifier arg}* &optional; result-type-specifier + + + + + Values + + + + name + + + A lisp string. See external, above. + + + + + arg-type-specifer + + + One of the foreign argument-type keywords, described + above, or an equivalent foreign + type specifier. + + + + + arg + + + A lisp value of type indicated by the corresponding + arg-type-specifier + + + + + result-type-specifier + + + One of the foreign argument-type keywords, described + above, or an equivalent foreign + type specifier. + + + + + + + Description + + Calls the foreign function at the address obtained by + resolving the external-entry-point associated with name, passing + the values of each arg as a foreign argument of type indicated by + the corresponding arg-type-specifier. Returns the foreign function + result (coerced to a Lisp object of type indicated by + result-type-specifier), or NIL if result-type-specifer is :VOID or + NIL + + + + + + foreign-symbol-entry + + + + FOREIGN-SYMBOL-ENTRY + + Function + + + + + foreign-symbol-entry name + + + + + Values + + + + name + + + A lisp string. + + + + + + + Description + + Tries to resolve the address of the foreign symbol name. If + successful, returns a fixnum representation of that address, else + returns NIL. + + + + + + foreign-symbol-address + + + + FOREIGN-SYMBOL-ADDRESS + + Function + + + + + foreign-symbol-address name + + + + + Values + + + + name + + + A lisp string. + + + + + + + Description + + Tries to resolve the address of the foreign symbol + name. If successful, returns that address encapsulated in + a + MACPTR, else returns NIL. + + + + + + defcallback + + + + DEFCALLBACK + + Macro + + + + + defcallback name + ({arg-type-specifier var}* &optional; result-type-specifier) + &body; body + + + + + Values + + + + name + + + A symbol which can be made into a special variable + + + + + arg-type-specifer + + + One of the foreign argument-type keywords, described + above, or an equivalent foreign + type specifier. + In addition, if the keyword + :WITHOUT-INTERRUPTS is specified, the callback will be + executed with lisp interrupts disabled if the corresponding + var is non-NIL. If :WITHOUT-INTERRUPTS is specified more + than once, the rightmost instance wins. + + + + + var + + + A symbol (lisp variable), which will be bound to a + value of the specified type. + + + + + body + + + A sequence of lisp forms, which should return a value + which can be coerced to the specified result-type. + + + + + + + Description + + Proclaims name + to be a special variable; sets its value to a + MACPTR which, when called by foreign code, calls a lisp function + which expects foreign arguments of the specified types and which + returns a foreign value of the specified result type. Any argument + variables which correspond to foreign arguments of type :ADDRESS + are bound to stack-allocated MACPTRs. + + If name + is already a callback function pointer, its value is + not changed; instead, it's arranged + that an + updated version of the lisp callback function will be called. + This feature allows for callback functions to be redefined + incrementally, just like Lisp functions are. + + defcallback + returns the callback pointer, e.g., the + value of name. + + + + + + #_ + + + + #_ + + Reader Macro + + + + Description + + Reads a symbol from the current input stream, with *PACKAGE* + bound to the "OS" package and with readtable-case preserved. + + Does a lookup on that symbol + in the OpenMCL interface database, + signalling + an error if no foreign function information can be found for the + symbol in any + active interface directory. + + Notes the foreign function information, including the foreign + function's return type, the number and type of the foreign + function's required arguments, and an indication of whether or + not the function accepts additional arguments (via e.g., the + "varargs" mechanism in C). + + Defines a macroexpansion function on the symbol, which expand + macro calls involving the symbol into EXTERNAL-CALL forms where + foreign argument type specifiers for required arguments and the + return value specifer are provided from the information ind the + database. + + Returns the symbol. + + The effect of these steps is that it's possible to call + foreign functions that take fixed numbers of arguments by simply + providing argument values, as in: + + (#_isatty fd) +(#_read fd buf n) + + and to call foreign functions that take variable numbers of + arguments by specifying the types of non-required args, as in: + + (with-cstrs ((format-string "the answer is: %d")) + (#_printf format-string :int answer)) + + + + + + #? + + + + #? + + Reader Macro + + + + Description + + In OpenMCL 0.14.2 and later, the #? reader macro can be used to + access foreign variables; this functionality depends on the presence of + "vars.cdb" files in the interface database. The current behavior + of the #? reader macro is to: + + Read a symbol from the current input stream, with *PACKAGE* + bound to the "OS" package and with readtable-case preserved. + + Use that symbol's pname to access the OpenMCL interface + database, signalling an error if no appropriate foreign variable + information can be found with that name in any active interface + directory. + + Use type information recorded in the database to construct a + form which can be used to access the foreign variable, and return + that form. + + Please note that the set of foreign variables declared in header files + may or may not match the set of foreign variables exported from + libraries (we're generally talking about C and Unix here ...). When + they do match, the form constructed by the #? reader macro manages the + details of resolving and tracking changes to the foreign variable's + address. + + Future extensions (via prefix arguments to the reader macro) may + offer additional behavior; it might be convenient (for instance) to be + able to access the address of a foreign variable without dereferencing + that address. + + Foreign variables in C code tend to be platform- and + packge-specific (the canonical example - "errno" - is typically + not a variable when threads are involved. ) + + In LinuxPPC, + + ? #?stderr + + returns a pointer to the stdio error stream ("stderr" is a + macro under OSX/Darwin). + + On both LinuxPPC and DarwinPPC, + + ? #?sys_errlist + + returns a pointer to a C array of C error message strings. + + + + + + use-interface-dir + + + + USE-INTERFACE-DIR + + Function + + + + + use-interface-dir dir-id + + + + + Values + + + + dir-id + + + A keyword whose pname, mapped to lower case, names a + subdirectory of "ccl:headers;" (or + "ccl:darwin-headers;") + + + + + + + Description + + Tells OpenMCL to add the interface directory denoted by + dir-id to the list of interface directories which it consults for + foreign type and function information. Arranges that that + directory is searched before any others. + + Note that use-interface-dir + merely adds an entry + to a search list. + If the named directory doesn't exist in the file system + or doesn't + contain a set of database files, a runtime error may occur + when OpenMCL + tries to open some database file in that directory, and it + will try to + open such a database file whenever it needs to find any + foreign type or + function information. + may come in + handy in that case. + + + + Examples + + One typically wants interface information to be + available at compile-time (or, in many cases, at read-time). + A typical idiom would be: + + (eval-when (:compile-toplevel :execute) + (use-interface-dir :GTK)) + + Using the :GTK interface directory makes available + information on + foreign types, functions, and constants. It's generally + necessary to + load foreign libraries before actually calling the + foreign code, which for GTK can be done like this: + + (load-gtk-libraries) + + It should now be possible to do things like: + + (#_gtk_widget_destroy w) + + + + + + unuse-interface-dir + + + + UNUSE-INTERFACE-DIR + + Function + + + + + unuse-interface-dir dir-id + + + + + Values + + + + dir-id + + + A keyword whose pname, mapped to lower case, names a + subdirectory of "ccl:headers;" (or + "ccl:darwin-headers;") + + + + + + + Description + + Tells OpenMCL to remove the interface directory denoted by + dir-id from the list of interface directories which are + consulted for + foreign type and function information. Returns T if the directory + was on the search list, NIL otherwise. + + + + + + terminate-when-unreachable + + + + TERMINATE-WHEN-UNREACHABLE + + Function + + + + + terminate-when-unreachable object + + + + + Values + + + + object + + + A CLOS object of a class for which there exists + a method of the generic function + ccl:terminate. + + + + + + + + Description + + + The "termination" mechanism is a way to have the garbage + collector run a function right before an object is about to + become garbage. It is very similar to the "finalization" + mechanism which Java has. It is not standard Common Lisp, + although other Lisp implementations have similar features. + It is useful when there is some sort of special cleanup, + deallocation, or releasing of resources which needs to happen + when a certain object is no longer being used. + + + + When the garbage collector discovers that an object is no + longer referred to anywhere in the program, it deallocates + that object, freeing its memory. However, if + ccl:terminate-when-unreachable has been + called on the object at any time, the garbage collector first + invokes the generic function ccl:terminate, + passing it the object as a parameter. + + + + Therefore, to make termination do something useful, you need to + define a method on ccl:terminate. + + + + Because calling + ccl:terminate-when-unreachable only + affects a single object, rather than all objects of its + class, you + may wish to put a call to it in the + initialize-instance method of a + class. Of course, this is only appropriate if you do in fact + want to use termination for all objects of a given class. + + + + + Example + + (defclass resource-wrapper () ((resource :accessor resource))) @@ -7836,43 +11198,52 @@ (defmethod ccl:terminate ((x resource-wrapper)) (when (resource x) - (deallocate (resource x)))) - - See Also - Tutorial: Allocating Foreign Data on the Lisp Heap - + (deallocate (resource x)))) + + + + See Also + + + + + + + + - The Objective-C Bridge -OS X APIs use a language called "Objective C", which is built on C. -The Objective-C bridge makes it possible to work with ObjC -objects and classes from Lisp, and to define classes in Lisp which -can be used by ObjC. - The ultimate -purpose of the ObjC and Cocoa bridges is to make Cocoa as -easy as possible to use from OpenMCL, in order to support the -development of GUI applications and IDEs. The eventual goal, -which is much closer than it used to be, is complete integration of -Cocoa into CLOS (whatever that means). + The Objective-C Bridge + + OS X APIs use a language called "Objective C", which is + built on C. The Objective-C bridge makes it possible to work with + ObjC objects and classes from Lisp, and to define classes in Lisp + which can be used by ObjC. + The ultimate purpose of the ObjC and Cocoa bridges is to + make Cocoa as easy as possible to use from OpenMCL, in order to + support the development of GUI applications and IDEs. The + eventual goal, which is much closer than it used to be, is + complete integration of Cocoa into CLOS (whatever that + means). The current release provides Lisp-like syntax and naming -conventions for the basic ObjC operations, with automatic -type processing and messages checked for validity at -compile-time. It also provides some convenience facilities -for working with Cocoa. + conventions for the basic ObjC operations, with automatic type + processing and messages checked for validity at compile-time. It + also provides some convenience facilities for working with + Cocoa. - Using Objective-C Classes -The class of most "standard" CLOS classes is the -class named STANDARD-CLASS. In the Objective-C object model, each -class is an instance of a (usually unique) metaclass, which is -itself an instance of a "base" metaclass (often the -metaclass of the class named "NSObject".) So, the -Objective-C class named "NSWindow" and the ObjC class -"NSArray" are (sole) instances of their distinct -metaclasses whose names are also "NSWindow" and -"NSArray", respectively. (In the Objective-C world, -it's much more common and useful to specialize class behavior -such as instance allocation.) + Using Objective-C Classes + + The class of most "standard" CLOS classes is the class + named STANDARD-CLASS. In the Objective-C object model, each + class is an instance of a (usually unique) metaclass, which is + itself an instance of a "base" metaclass (often the metaclass of + the class named "NSObject".) So, the Objective-C class named + "NSWindow" and the ObjC class "NSArray" are (sole) instances of + their distinct metaclasses whose names are also "NSWindow" and + "NSArray", respectively. (In the Objective-C world, it's much + more common and useful to specialize class behavior such as + instance allocation.) When foreign libraries containing Objective-C classes are first loaded, the classes they contain are identified. The foreign class @@ -7914,21 +11285,21 @@ - Instantiating Objective-C Objects -Making an instance of an ObjC class (whether the class in question -is predefined or defined by the application) involves calling -MAKE-INSTANCE with the class and a set of initargs as arguments. -As with STANDARD-CLASS, making an instance involves initializing -(with INITIALIZE-INSTANCE) an object allocated with -ALLOCATE-INSTANCE. + Instantiating Objective-C Objects + Making an instance of an ObjC class (whether the class in + question is predefined or defined by the application) involves + calling MAKE-INSTANCE with the class and a set of initargs as + arguments. As with STANDARD-CLASS, making an instance involves + initializing (with INITIALIZE-INSTANCE) an object allocated with + ALLOCATE-INSTANCE. For example, you can create an ns:ns-number like this: ? (make-instance 'ns:ns-number :init-with-int 42) -# +#<NS-CF-NUMBER 42 (#x85962210)> It's worth looking at how this would be done if you were -writing in Objective C: + writing in Objective C: [[NSNumber alloc] initWithInt: 42] - + Allocating an instance of an ObjC class involves sending the class an "alloc" message, and then using those initargs that @@ -7941,5 +11312,5 @@ ? (setq *n* (ccl::send *n* :init-with-int 42)) -# +#<NS-CF-NUMBER 42 (#x16D340)> That setq is important; this is a case where init @@ -7967,5 +11338,5 @@ :init-with-window-nib-name #@"DataWindow" :owner *controller*)) -# (#x1FB520)> +#<NS-WINDOW-CONTROLLER <NSWindowController: 0x1fb520> (#x1FB520)> This example calls (make-instance) with no initargs. When you @@ -7975,18 +11346,18 @@ - Calling Objective-C Methods -In Objective-C, methods are called "messages", and there's -a special syntax to send a message to an object: + Calling Objective-C Methods + In Objective-C, methods are called "messages", and there's + a special syntax to send a message to an object: [w alphaValue] [w setAlphaValue: 0.5] [v mouse: p inRect: r] - + The first line sends the method "alphaValue" to the object -w, with no parameters. The second line -sends the method "setAlphaValue", with the parameter 0.5. -The third line sends the method "mouse:inRect:" - yes, all -one long word - with the -parameters p and r. + w, with no parameters. The second line sends + the method "setAlphaValue", with the parameter 0.5. The third + line sends the method "mouse:inRect:" - yes, all one long word - + with the parameters p and + r. In Lisp, these same three lines are: @@ -8014,89 +11385,63 @@ - Type Coercion for ObjC Method Calls -OpenMCL's FFI handles many common conversions between Lisp -and foreign data, such as unboxing floating-point args and -boxing floating-point results. The bridge adds a few more -automatic conversions: - NIL is equivalent to (%NULL-PTR) for any message argument that -requires a pointer. + Type Coercion for ObjC Method Calls + OpenMCL's FFI handles many common conversions between + Lisp and foreign data, such as unboxing floating-point args + and boxing floating-point results. The bridge adds a few more + automatic conversions: + NIL is equivalent to (%NULL-PTR) for any message + argument that requires a pointer. T/NIL are equivalent to #$YES/#$NO for any boolean argument. - A #$YES/#$NO returned by any method that returns BOOL will be -automatically converted to T/NIL. - To make this last conversion work, the bridge has to engage -in a bit of hackery. The bridge uses ObjC run-time type -info. Unfortunately, BOOL is typed as CHAR by ObjC. Thus, -a method that returns CHAR might actually return only BOOL, -or it might return any CHAR. - The bridge currently assumes -that any method that returns CHAR actually returns BOOL. -But it provides a facility for defining exceptions to this -assumption: (DEFINE-RETURNS-BOOLEAN-EXCEPTION "charValue"). -Eventually, the best way to handle issues like this is -probably to get our method type info directly from the -header files rather than using ObjC's runtime type system. - Note that no automatic conversion is currently performed between -Lisp strings and NSStrings. However, there is a convenient -reader macro for creating constant NSStrings: - -(SEND W :SET-TITLE #@"My Window") - - Note that #@"Hello" is a full ObjC object, so -messages can be sent to it: (SEND #@"Hello" 'LENGTH). - To go in the other direction, use the function -(ccl::lisp-string-from-nsstring). - To create variable NSStrings (which you will later -want to change the -values of), consider using CCL::NS-LISP-STRING. + A #$YES/#$NO returned by any method that returns BOOL + will be automatically converted to T/NIL. - Methods which Return Structures -Some Cocoa methods return small structures, such as those used -to represent points, rects, sizes and ranges. When writing in -Objective C, the compiler hides the implementation details. -Unfortunately, in Lisp we must be slightly more aware of them. - Methods which return structures are called in a special way; -the caller allocates space for the result, and passes a pointer -to it as an extra argument to the method. This is called a -Structure Return, or STRET. Don't look at me; I don't name -these things. + Methods which Return Structures + Some Cocoa methods return small structures, such as + those used to represent points, rects, sizes and ranges. When + writing in Objective C, the compiler hides the implementation + details. Unfortunately, in Lisp we must be slightly more + aware of them. + Methods which return structures are called in a special + way; the caller allocates space for the result, and passes a + pointer to it as an extra argument to the method. This is + called a Structure Return, or STRET. Don't look at me; I + don't name these things. Here's a simple use of this in Objective C. The first line -sends the "bounds" message to v1, which returns a rectangle. -The second line sends the "setBounds" message to v2, passing -that same rectangle as a parameter. + sends the "bounds" message to v1, which returns a rectangle. + The second line sends the "setBounds" message to v2, passing + that same rectangle as a parameter. NSRect r = [v1 bounds]; [v2 setBounds r]; - - In Lisp, we must explicitly allocate the memory, which is -done most easily and safely with . -We do it like this: + + In Lisp, we must explicitly allocate the memory, which + is done most easily and safely with . + We do it like this: -(rlet ((r :ect)) +(rlet ((r :<NSR>ect)) (send/stret r v1 'bounds) (send v2 :set-bounds r)) - The rlet allocates the storage (but doesn't initialize it), -and makes sure that it will be deallocated when we're done. -It binds the variable r to refer to it. -The call to send/stret is just like -an ordinary call to -send, except that r is passed as an -extra, first parameter. The third line, which calls -send, does not need to do anything -special, because there's nothing complicated about passing -a structure as a parameter. - In order to make STRETs easier to use, the bridge provides two -conveniences. + The rlet allocates the storage (but doesn't initialize + it), and makes sure that it will be deallocated when we're + done. It binds the variable r to refer to it. The call to + send/stret is just like an ordinary call to + send, except that r is passed as an extra, + first parameter. The third line, which calls + send, does not need to do anything special, + because there's nothing complicated about passing a structure + as a parameter. + In order to make STRETs easier to use, the bridge + provides two conveniences. First, you can use the macros slet -and slet* to allocate and -initialize local variables to foreign structures in one -step. The example above could have been written more tersely -as: + and slet* to allocate and initialize local + variables to foreign structures in one step. The example + above could have been written more tersely as: (slet ((r (send v1 'bounds))) (send v2 :set-bounds r)) - + Second, when one call to send is made inside another, the inner one has an implicit @@ -8135,5 +11480,6 @@ - Variable-Arity Messages + Variable-Arity Messages + There are a few messages in Cocoa which take variable numbers of arguments. Perhaps the most common examples involve @@ -8163,23 +11509,22 @@ - Optimization -The bridge works fairly hard to optimize message sends, when -it has enough information to do so. There are two cases when -it does. In either, a message send should be nearly as -efficient as when writing in Objective C. - The first case is when both the message -and the receiver's class are known at -compile-time. In general, the only way the receiver's class -is known is if you declare it, which you can do with -either a DECLARE or a THE form. For example: + Optimization + The bridge works fairly hard to optimize message sends, + when it has enough information to do so. There are two cases + when it does. In either, a message send should be nearly as + efficient as when writing in Objective C. + The first case is when both the message and the + receiver's class are known at compile-time. In general, the + only way the receiver's class is known is if you declare it, + which you can do with either a DECLARE or a THE form. For + example: (send (the ns:ns-window w) 'center) - + Note that there is no way in ObjC to name the class of a -class. Thus the bridge provides a declaration, @METACLASS. -The -type of an instance of "NSColor" is ns:ns-color. The type of -the class -"NSColor" is (@metaclass ns:ns-color): + class. Thus the bridge provides a declaration, @METACLASS. + The type of an instance of "NSColor" is ns:ns-color. The type + of the class "NSColor" is (@metaclass + ns:ns-color): (let ((c (find-class 'ns:ns-color))) @@ -8187,66 +11532,66 @@ (send c 'white-color)) - The other case that alllows optimization is when only the -message is known at compile-time, but its type signature -is unique. Of the more-than-6000 messages currently provided by -Cocoa, only about 50 of them have nonunique type signatures. - An example of a message with a type signature that is not -unique is SET. It returns VOID for NSColor, but ID for NSSet. -In order to optimize sends of messages with nonunique type -signatures, the class of the receiver must be declared at -compile-time. - If the type signature is nonunique or the message is unknown -at compile-time, then a slower runtime call must be used. - When the receiver's class is unknown, the bridge's ability to -optimize relies on a type-signature table which it maintains. -When first loaded, the bridge initializes this table -by scanning every method of every ObjC class. When new methods -are defined later, the table must be updated. This happens -automatically when you define methods in Lisp. After any -other major change, such as loading an external framework, -you should rebuild the table: + The other case that alllows optimization is when only + the message is known at compile-time, but its type signature + is unique. Of the more-than-6000 messages currently provided + by Cocoa, only about 50 of them have nonunique type + signatures. + An example of a message with a type signature that is + not unique is SET. It returns VOID for NSColor, but ID for + NSSet. In order to optimize sends of messages with nonunique + type signatures, the class of the receiver must be declared at + compile-time. + If the type signature is nonunique or the message is + unknown at compile-time, then a slower runtime call must be + used. + When the receiver's class is unknown, the bridge's + ability to optimize relies on a type-signature table which it + maintains. When first loaded, the bridge initializes this + table by scanning every method of every ObjC class. When new + methods are defined later, the table must be updated. This + happens automatically when you define methods in Lisp. After + any other major change, such as loading an external framework, + you should rebuild the table: ? (update-type-signatures) Because send and its relatives -send-super, -send/stret, and -send-super/stret are macros, they cannot -be funcalled, applyed, -or passed as arguments to functions. - To work around this, -there are function equivalents to them: -%send, -%send-super, -%send/stret, and -%send-super/stret. However, these -functions should be used only when the macros will not do, -because they are unable to optimize. + send-super, send/stret, + and send-super/stret are macros, they + cannot be funcalled, + applyed, or passed as arguments to + functions. + To work around this, there are function equivalents to + them: %send, + %send-super, + %send/stret, and + %send-super/stret. However, these + functions should be used only when the macros will not do, + because they are unable to optimize. - Defining Objective-C Classes -You can define your own foreign classes, which can then be -passed to foreign functions; the methods which you implement -in Lisp will be made available to the foreign code as -callbacks. - You can also define subclasses of existing classes, implementing -your subclass in Lisp even though the parent class was in -Objective C. One such subclass is -CCL::NS-LISP-STRING. -It is also particularly useful to make subclasses of -NS-WINDOW-CONTROLLER. - We can use the MOP to define new Objective-C classes, -but we have to do something a little -funny: the :METACLASS that we'd want to use in a DEFCLASS option -generally doesn't exist until we've created the class (recall -that ObjC classes have, for the sake of argument, unique and private -metaclasses.) We can sort of sleaze our way around this by specifying -a known ObjC metaclass object name as the value of -the DEFCLASS :METACLASS object; the metaclass of the root class -NS:NS-OBJECT, NS:+NS-OBJECT, makes a good choice. To make a subclass -of NS:NS-WINDOW (that, for simplicity's sake, doesn't define any -new slots), we could do: + Defining Objective-C Classes + You can define your own foreign classes, which can then be + passed to foreign functions; the methods which you implement in + Lisp will be made available to the foreign code as + callbacks. + You can also define subclasses of existing classes, + implementing your subclass in Lisp even though the parent class + was in Objective C. One such subclass is CCL::NS-LISP-STRING. + It is also particularly useful to make subclasses of + NS-WINDOW-CONTROLLER. + We can use the MOP to define new Objective-C classes, but + we have to do something a little funny: the :METACLASS that we'd + want to use in a DEFCLASS option generally doesn't exist until + we've created the class (recall that ObjC classes have, for the + sake of argument, unique and private metaclasses.) We can sort + of sleaze our way around this by specifying a known ObjC + metaclass object name as the value of the DEFCLASS :METACLASS + object; the metaclass of the root class NS:NS-OBJECT, + NS:+NS-OBJECT, makes a good choice. To make a subclass of + NS:NS-WINDOW (that, for simplicity's sake, doesn't define any + new slots), we could do: (defclass example-window (ns:ns-window) @@ -8255,22 +11600,23 @@ That'll create a new ObjC class named EXAMPLE-WINDOW whose -metaclass is the class named +EXAMPLE-WINDOW. The class will be an -object of type OBJC:OBJC-CLASS, and the metaclass will be of type -OBJC:OBJC-METACLASS. EXAMPLE-WINDOW will be a subclass of -NS-WINDOW. + metaclass is the class named +EXAMPLE-WINDOW. The class will be + an object of type OBJC:OBJC-CLASS, and the metaclass will be of + type OBJC:OBJC-METACLASS. EXAMPLE-WINDOW will be a subclass of + NS-WINDOW. - Defining classes with foreign slots -If a slot specification in an Objective-C class definition -contains the keyword :FOREIGN-TYPE, the slot will be a "foreign -slot" (i.e. an ObjC instance variable). Be aware that it is an -error to redefine an ObjC class so that its foreign slots -change in any way, and OpenMCL doesn't do anything consistent -when you try to. - The value of the :FOREIGN-TYPE initarg should be a foreign type -specifier. For example, if we wanted (for some reason) to define a -subclass of NS:NS-WINDOW that kept track of the number of key events -it had received (and needed an instance variable to keep that -information in), we could say: + Defining classes with foreign slots + If a slot specification in an Objective-C class + definition contains the keyword :FOREIGN-TYPE, the slot will + be a "foreign slot" (i.e. an ObjC instance variable). Be aware + that it is an error to redefine an ObjC class so that its + foreign slots change in any way, and OpenMCL doesn't do + anything consistent when you try to. + The value of the :FOREIGN-TYPE initarg should be a + foreign type specifier. For example, if we wanted (for some + reason) to define a subclass of NS:NS-WINDOW that kept track + of the number of key events it had received (and needed an + instance variable to keep that information in), we could + say: (defclass key-event-counting-window (ns:ns-window) @@ -8281,13 +11627,14 @@ Foreign slots are always SLOT-BOUNDP, and the initform -above is redundant: foreign slots are initialized to binary 0. + above is redundant: foreign slots are initialized to binary + 0. - Defining classes with Lisp slots -A slot specification in an ObjC class definition that doesn't -contain the :FOREIGN-TYPE initarg defines a pretty-much normal lisp -slot that'll happen to be associated with "an instance of a -foreign class". For instance: + Defining classes with Lisp slots + A slot specification in an ObjC class definition that + doesn't contain the :FOREIGN-TYPE initarg defines a + pretty-much normal lisp slot that'll happen to be associated + with "an instance of a foreign class". For instance: (defclass hemlock-buffer-string (ns:ns-string) @@ -8296,32 +11643,31 @@ :accessor string-hemlock-buffer)) (:metaclass ns:+ns-object)) - + As one might expect, this has memory-management -implications: we -have to maintain an association between a MACPTR and a set of lisp -objects (its slots) as long as the ObjC instance exists, and we have -to ensure that the ObjC instance exists (does not have its -dealloc -method called) while lisp is trying to think of it as a first-class -object that can't be "deallocated" while it's still -possible to reference it. Associating one or more lisp objects with a -foreign instance is something that's often very useful; if you -were to do this "by hand", you'd have to face many of the -same memory-management issues. + implications: we have to maintain an association between a + MACPTR and a set of lisp objects (its slots) as long as the + ObjC instance exists, and we have to ensure that the ObjC + instance exists (does not have its -dealloc method called) + while lisp is trying to think of it as a first-class object + that can't be "deallocated" while it's still possible to + reference it. Associating one or more lisp objects with a + foreign instance is something that's often very useful; if you + were to do this "by hand", you'd have to face many of the same + memory-management issues. - Defining Objective-C Methods -In ObjC, unlike in CLOS, every method belongs to some -particular class. This is probably not a strange -concept to you, because C++ and Java do the same thing. -When you use Lisp to define ObjC methods, it is only -possible to define methods belonging to ObjC classes which -have been defined in Lisp. + Defining Objective-C Methods + In ObjC, unlike in CLOS, every method belongs to some + particular class. This is probably not a strange concept to + you, because C++ and Java do the same thing. When you use Lisp + to define ObjC methods, it is only possible to define methods + belonging to ObjC classes which have been defined in + Lisp. The macro define-objc-method is used -for this. As described in , the names -of ObjC methods are broken into pieces, each piece followed -by a parameter. The types of all parameters must be explicitly -declared. + for this. As described in , the names of ObjC methods are + broken into pieces, each piece followed by a parameter. The + types of all parameters must be explicitly declared. Right now, I'm not sure how to formally describe the usage of define-objc-method, so I'm going to do it with some short @@ -8333,30 +11679,27 @@ (:metaclass ns:+ns-object)) - There's nothing special about this class. -It inherits from ns:ns-window-controller. -It has two slots: window is a -foreign slot, stored in the ObjC world; and -data is an ordinary slot, stored in -the Lisp world. + There's nothing special about this class. It inherits + from ns:ns-window-controller. It has two slots: + window is a foreign slot, stored in the ObjC + world; and data is an ordinary slot, stored + in the Lisp world. Here is an example of how to define a method which takes -no arguments. It happens to be an initialization method, -but that's not important: + no arguments. It happens to be an initialization method, but + that's not important: (define-objc-method ((:id get-window) data-window-controller) (window self)) - - The return type of this method is the foreign type :id, which -is used for all ObjC objects. -The name of the method is -get-window. The body of the method is -the single line (window self). The variable -self is bound, within the body, to the -instance which is receiving the message. The call to -window -uses the CLOS accessor to get the value of the window field. + + The return type of this method is the foreign type :id, + which is used for all ObjC objects. The name of the method is + get-window. The body of the method is the + single line (window self). The variable self + is bound, within the body, to the instance which is receiving + the message. The call to window uses the + CLOS accessor to get the value of the window field. Here's an example which takes a parameter. Notice that -the name of the method without a parameter was an ordinary -symbol, but with a parameter, it's a keyword: + the name of the method without a parameter was an ordinary + symbol, but with a parameter, it's a keyword: (define-objc-method ((:id :init-with-multiplier (:int multiplier)) @@ -8367,12 +11710,11 @@ (* i multiplier))) self) - + To Objective-C code which uses the class, the name of this -method is "initWithMultiplier:". The name of the parameter -is multiplier, and its type is :int. -The body of the method -does some meaningless things. Then it returns -self, because -this is an initialization method. + method is "initWithMultiplier:". The name of the parameter is + multiplier, and its type is :int. The body + of the method does some meaningless things. Then it returns + self, because this is an initialization + method. Here's an example with more than one parameter: @@ -8386,13 +11728,13 @@ addend))) self) - + To Objective-C, the name of this method is -"initWithMultiplier:andAddend:". Both parameters are -of type :int; the first is named multiplier, -and the second is addend. Again, the -method returns self. - Here is a method which does not return any value, a so-called -"void method". Where our other methods said :id, this one says -:void for the return type: + "initWithMultiplier:andAddend:". Both parameters are of type + :int; the first is named multiplier, and the + second is addend. Again, the method returns + self. + Here is a method which does not return any value, a + so-called "void method". Where our other methods said :id, this + one says :void for the return type: (define-objc-method ((:void :take-action (:id sender)) @@ -8403,20 +11745,21 @@ (- (aref (data self) i))))) - This method would be called "takeAction:" in ObjC. -The convention for methods that are going to be used as Cocoa -actions is that they take one parameter, which is the object -responsible for triggering the action. However, this method -doesn't actually need to use that parameter, so it explicitly -ignores it to avoid a compiler warning. As promised, the -method doesn't return any value. - There is also an alternate syntax, illustrated here. The following two method definitions are equivalent: + This method would be called "takeAction:" in ObjC. The + convention for methods that are going to be used as Cocoa + actions is that they take one parameter, which is the object + responsible for triggering the action. However, this method + doesn't actually need to use that parameter, so it explicitly + ignores it to avoid a compiler warning. As promised, the method + doesn't return any value. + There is also an alternate syntax, illustrated here. The + following two method definitions are equivalent: (define-objc-method ("applicationShouldTerminate:" "LispApplicationDelegate") - (:id sender :) + (:id sender :<BOOL>) (declare (ignore sender)) nil) -(define-objc-method ((: +(define-objc-method ((:<BOOL> :application-should-terminate sender) lisp-application-delegate) @@ -8425,82 +11768,63 @@ - Method Redefinition Constraints -Objective C was not designed, as Lisp was, with runtime -redefinition in mind. So, there are a few constraints about -how and when you can replace the definition of an Objective C -method. Currently, if you break these rules, nothing will -collapse, but the behaviour will be confusing; so don't. - Objective C methods can be redefined at runtime, but their -signatures shouldn't change. That is, the types of the arguments -and the return type have to stay the same. The reason for this -is that changing the signature changes the selector which is used -to call the method. - When a method has already been defined in one class, and you -define it in a subclass, shadowing the original method, they -must both have the same type signature. There is no such -constraint, though, if the two classes aren't related and the -methods just happen to have the same name. + Method Redefinition Constraints + Objective C was not designed, as Lisp was, with runtime + redefinition in mind. So, there are a few constraints about + how and when you can replace the definition of an Objective C + method. Currently, if you break these rules, nothing will + collapse, but the behaviour will be confusing; so + don't. + Objective C methods can be redefined at runtime, but + their signatures shouldn't change. That is, the types of the + arguments and the return type have to stay the same. The + reason for this is that changing the signature changes the + selector which is used to call the method. + When a method has already been defined in one class, and + you define it in a subclass, shadowing the original method, + they must both have the same type signature. There is no such + constraint, though, if the two classes aren't related and the + methods just happen to have the same name. - How Objective-C Names are Mapped to Lisp Symbols -There is a standard set of naming conventions for Cocoa classes, -messages, etc. As long as they are followed, the bridge is fairly -good at automaticallly translating between ObjC and Lisp names. + How Objective-C Names are Mapped to Lisp Symbols + There is a standard set of naming conventions for Cocoa + classes, messages, etc. As long as they are followed, the + bridge is fairly good at automaticallly translating between ObjC + and Lisp names. For example, "NSOpenGLView" becomes ns:ns-opengl-view; -"NSURLHandleClient" becomes ns:ns-url-handle-client; and -"nextEventMatchingMask:untilDate:inMode:dequeue:" becomes -(:next-event-matching-mask :until-date :in-mode :dequeue). -What a mouthful. - To see how a given ObjC or Lisp name will be translated by the -bridge, you can use the following functions: - - - - - - (ccl::objc-to-lisp-classname string) - - - - (ccl::lisp-to-objc-classname symbol) - - - - (ccl::objc-to-lisp-message string) - - - - (ccl::lisp-to-objc-message string) - - - - (ccl::objc-to-lisp-init string) - - - - (ccl::lisp-to-objc-init keyword-list) - - - - - - Of course, there will always be exceptions to any naming convention. -Please tell us on the mailing lists if you come across any -name translation problems -that seem to be bugs. Otherwise, the bridge provides two ways of -dealing with exceptions: + "NSURLHandleClient" becomes ns:ns-url-handle-client; and + "nextEventMatchingMask:untilDate:inMode:dequeue:" becomes + (:next-event-matching-mask :until-date :in-mode :dequeue). What + a mouthful. + To see how a given ObjC or Lisp name will be translated by + the bridge, you can use the following functions: + + (ccl::objc-to-lisp-classname string) + (ccl::lisp-to-objc-classname symbol) + (ccl::objc-to-lisp-message string) + (ccl::lisp-to-objc-message string) + (ccl::objc-to-lisp-init string) + (ccl::lisp-to-objc-init keyword-list) + + + Of course, there will always be exceptions to any naming + convention. Please tell us on the mailing lists if you come + across any name translation problems that seem to be bugs. + Otherwise, the bridge provides two ways of dealing with + exceptions: First, you can pass a string as the class name of -MAKE-OBJC-INSTANCE and as the message to SEND. These strings will -be directly interpreted as ObjC names, with no translation. This -is useful for a one-time exception. For example: + MAKE-OBJC-INSTANCE and as the message to SEND. These strings + will be directly interpreted as ObjC names, with no + translation. This is useful for a one-time exception. For + example: (ccl::make-objc-instance "WiErDclass") (ccl::send o "WiErDmEsSaGe:WithARG:" x y) - - Alternatively, you can define a special translation rule for your -exception. This is useful for an exceptional name that you need -to use througout your code. Some examples: + + Alternatively, you can define a special translation rule + for your exception. This is useful for an exceptional name that + you need to use througout your code. Some examples: (ccl::define-classname-translation "WiErDclass" wierd-class) @@ -8518,286 +11842,273 @@ (ccl::define-special-objc-word "QuickDraw") - Note that message keywords in a SEND such as -(SEND V :MOUSE P :IN-RECT R) may -look like the keyword arguments in a Lisp function call, but they -really aren't. All keywords must be -present and the order is significant. Neither (:IN-RECT :MOUSE) nor -(:MOUSE) translate to "mouse:inRect:" - Also, as a special exception, an "init" prefix is optional in the -initializer keywords, so -(MAKE-OBJC-INSTANCE 'NS-NUMBER :INIT-WITH-FLOAT 2.7) can also -be expressed as -(MAKE-OBJC-INSTANCE 'NS-NUMBER :WITH-FLOAT 2.7) + Note that message keywords in a SEND such as (SEND V + :MOUSE P :IN-RECT R) may look like the keyword arguments in a + Lisp function call, but they really aren't. All keywords must be + present and the order is significant. Neither (:IN-RECT :MOUSE) + nor (:MOUSE) translate to "mouse:inRect:" + Also, as a special exception, an "init" prefix is optional + in the initializer keywords, so (MAKE-OBJC-INSTANCE 'NS-NUMBER + :INIT-WITH-FLOAT 2.7) can also be expressed as + (MAKE-OBJC-INSTANCE 'NS-NUMBER :WITH-FLOAT 2.7) - - Using OpenMCL with Mac OS X, the Darwin Kernel, and the Cocoa GUI API - - - Differences in OpenMCL between Darwin and MacOS X -Most of the documentation and whatever experience you may have in using OpenMCL under LinuxPPC should apply to using it under Darwin/MacOS X. There are some differences between the platforms, and these differences are sometimes exposed in the implementation. - - - FASL file segregation. -99% of the compiled lisp code in the two implementations is -identical. Since both implementations target the PowerPC, this -shouldn't be too surprising. - The other 1% - code which deals with foreign functions or data or -which deals directly with the operating system - is very different but -superficially similar. The effects of running such code compiled for -the wrong platform range from harmless to catastrophic. - During bootstrapping, the 1% proved troublesome enough that it seemed -wise to strictly separate the two implementations. - FASL files have different file types (.dfsl for Darwin, .pfsl for -LinuxPPC) and contain different identifying information; it's not -possible to successfully load FASL files compiled for one -implementation into the other. - I think that it's wise to maintain this distinction; I spent a lot -less time recompiling the 99% of "non-troublesome" code than I'd been -spending tracking down bugs caused by the other 1%. - + + Platform-specific notes + + + + Overview + The documentation and whatever experience you may have in + using OpenMCL under Linux should also apply to using it under + Darwin/MacOS X and FreeBSD. There are some differences between + the platforms, and these differences are sometimes exposed in + the implementation. + - File-system case -Darwin and MacOS X use HFS+ file systems by default; HFS+ file systems -are case-insensitive. Most of OpenMCL's filesystem and pathname code -assumes that the underlying filesystem is case-sensitive; this -assumption extends to functions like EQUAL, which assumes that #p"FIXTHIS" -and #p"foo" denote different, un-EQUAL filenames. Since Darwin/MacOS X -can also use UFS and NFS filesystems, the opposite assumption would be -no more correct than the one that's currently made. + File-system case + + Darwin and MacOS X use HFS+ file systems by default; + HFS+ file systems are usually case-insensitive. Most of + OpenMCL's filesystem and pathname code assumes that the + underlying filesystem is case-sensitive; this assumption + extends to functions like EQUAL, which assumes that #p"FOO" + and #p"foo" denote different, un-EQUAL filenames. Since + Darwin/MacOS X can also use UFS and NFS filesystems, the + opposite assumption would be no more correct than the one + that's currently made. Whatever the best solution to this problem turns out to be, there are some practical considerations. Doing: ? (save-application "DPPCCL") - - has the unfortunate side-effect of trying to overwrite the Darwin -OpenMCL kernel, "dppccl", on a case-insensitive filesystem. - To work around this, the Darwin OpenMCL kernel expects the default -heap image file name to be the kernel's own filename with the string -".image" appended, so the idiom would be: - ? (save-application "dppccl.image") - + + on 32-bit DarwinPPC has the unfortunate side-effect of + trying to overwrite the Darwin OpenMCL kernel, "dppccl", on a + case-insensitive filesystem. + To work around this, the Darwin OpenMCL kernel expects + the default heap image file name to be the kernel's own + filename with the string ".image" appended, so the idiom would + be: + +? (save-application "dppccl.image") + - Line Termination Characters -MacOSX effectively supports two distinct line-termination -conventions. Programs in its Darwin substrate follow the Unix -convention of recognizing #\LineFeed as a line terminator; traditional -MacOS programs use #\Return for this purpose. - OpenMCL follows the Unix convention on both Darwin and LinuxPPC, -but offers (as of version 0.11) some support for reading and writing -files that use the MacOS convention as well. - This support (and anything like it) is by nature heuristic: it -can successfully hide the distinction between newline conventions -much of the time, but could mistakenly change the meaning of -otherwise correct programs (typically when files contain both -#\Return and #\Linefeed characters or when files contain mixtures of -text and binary data.) Because of this concern, the default settings -of some of the variables that control newline translation and -interpretation are somewhat conservative. - Although the issue of multiple newline conventions primarily -affects MacOSX users, the functionality described here is available -under LinuxPPC as well (and may occasionally be useful there.) - None of this addresses issues related -to the third newline convention ("CRLF") in widespread use -(since that convention isn't native to any platform on which -OpenMCL currently runs). If OpenMCL is ever ported to such a -platform, that issue might be revisited. - Note that some MacOS programs (including some versions of -commercial MCL) may use HFS file type information to recognize TEXT -and other file types and so may fail to recognize files created with -OpenMCL or other Darwin applications (regardless of line termination -issues.) - Unless otherwise noted, the symbols mentioned in this -documentation are exported from the CCL package. + Line Termination Characters + MacOSX effectively supports two distinct line-termination + conventions. Programs in its Darwin substrate follow the Unix + convention of recognizing #\LineFeed as a line terminator; traditional + MacOS programs use #\Return for this purpose. Many modern + GUI programs try to support several different line-termination + conventions (on the theory that the user shouldn't be too concerned + about what conventions are used an that it probably doesn't matter. + Sometimes this is true, other times ... not so much. + + OpenMCL follows the Unix convention on both Darwin and + LinuxPPC, but offers some support for reading and writing + files that use other conventions (including traditional MacOS + conventions) as well. + This support (and anything like it) is by nature + heuristic: it can successfully hide the distinction between + newline conventions much of the time, but could mistakenly + change the meaning of otherwise correct programs (typically + when files contain both #\Return and #\Linefeed characters or + when files contain mixtures of text and binary data.) Because + of this concern, the default settings of some of the variables + that control newline translation and interpretation are + somewhat conservative. + Although the issue of multiple newline conventions + primarily affects MacOSX users, the functionality described + here is available under LinuxPPC as well (and may occasionally + be useful there.) None of this addresses issues + related to the third newline convention ("CRLF") in widespread + use (since that convention isn't native to any platform on + which OpenMCL currently runs). If OpenMCL is ever ported to + such a platform, that issue might be revisited. + Note that some MacOS programs (including some versions + of commercial MCL) may use HFS file type information to + recognize TEXT and other file types and so may fail to + recognize files created with OpenMCL or other Darwin + applications (regardless of line termination issues.) + Unless otherwise noted, the symbols mentioned in this + documentation are exported from the CCL package. - Single-precision trig & transcendental functions -Despite what Darwin's man pages say, early versions of its math library -(up to and including OSX 10.2 (Jaguar) don't implement -single-precision variants of the transcendental and trig functions -(#_sinf, #_atanf, etc.) OpenMCL works around this by coercing -single-precision args to double-precision, calling the -double-precision version of the math library function, and coercing -the result back to a SINGLE-FLOAT. These steps can introduce rounding -errors (and potentially overflow conditions) that might not be present -or as severe if true 32-bit variants were available. + Single-precision trig & transcendental functions + + Despite what Darwin's man pages say, early versions of its math library + (up to and including at least OSX 10.2 (Jaguar) don't implement + single-precision variants of the transcendental and trig functions + (#_sinf, #_atanf, etc.) OpenMCL worked around this by coercing + single-precision args to double-precision, calling the + double-precision version of the math library function, and coercing + the result back to a SINGLE-FLOAT. These steps can introduce rounding + errors (and potentially overflow conditions) that might not be present + or as severe if true 32-bit variants were available. - Shared libraries -Darwin/MacOS X distinguishes between "shared libraries" and "bundles" -or "extensions"; Linux doesn't. In Darwin, "shared libraries" have the -file type "dylib" : the expectation is that this class of file is -linked against when executable files are created and loaded by the OS -when the executable is launched. The latter class - -"bundles/extensions" - are expected to be loaded into and unloaded -from a running application, via a mechanism like the one used by -OpenMCL's OPEN-SHARED-LIBRARY function. - OpenMCL wants to treat all shared libs as if they were "bundles"; this -is presumably possible, but none of it's implemented under OpenMCL -0.10. Ideally, whatever gets implemented will hide the distinction as -much as possible, but it may not be possible to hide it completely. + Shared libraries + Darwin/MacOS X distinguishes between "shared libraries" + and "bundles" or "extensions"; Linux and FreeBSD don't. In + Darwin, "shared libraries" have the file type "dylib" : the + expectation is that this class of file is linked against when + executable files are created and loaded by the OS when the + executable is launched. The latter class - + "bundles/extensions" - are expected to be loaded into and + unloaded from a running application, via a mechanism like the + one used by OpenMCL's OPEN-SHARED-LIBRARY function. - Unix/Posix/Darwin Features -OpenMCL has several convenience functions which allow you to -make Posix (portable Unix) calls without having to use the -foreign-function interface. Each of these corresponds directly -to a single Posix function call, as it might be made in C. There -is no attempt to make these calls correspond to Lisp idioms, such as -setf. This means that their behaviour is -simple and predictable. + Unix/Posix/Darwin Features + OpenMCL has several convenience functions which allow you + to make Posix (portable Unix) calls without having to use the + foreign-function interface. Each of these corresponds directly + to a single Posix function call, as it might be made in C. + There is no attempt to make these calls correspond to Lisp + idioms, such as setf. This means that their + behaviour is simple and predictable. For working with environment variables, there are -CCL::GETENV and CCL::SETENV. + CCL::GETENV and CCL::SETENV. For working with user and group IDs, there are -CCL::GETUID, CCL::SETUID, -and CCL::SETGID. To find the home directory -of an arbitrary user, as set in the user database (/etc/passwd), -there is CCL::GET-USER-HOME-DIR. - For process IDs, there -is CCL::GETPID. + CCL::GETUID, CCL::SETUID, and CCL::SETGID. To find the home + directory of an arbitrary user, as set in the user database + (/etc/passwd), there is CCL::GET-USER-HOME-DIR. + For process IDs, there is CCL::GETPID. For the system() function, there is -CCL::OS-COMMAND. Ordinarily, it is better - -both more efficient and more predictable - to use the features -described in . However, -sometimes you may want to specifically ask the shell to invoke a -command for you. + CCL::OS-COMMAND. Ordinarily, it is better - both more efficient + and more predictable - to use the features described in . However, + sometimes you may want to specifically ask the shell to invoke a + command for you. - Cocoa Programming in OpenMCL -Cocoa is one of Apple's APIs for GUI programming; for most purposes, -development is considerably faster with Cocoa than with the -alternatives. You should have a little familiarity with it, to -better understand this section. - A small sample Cocoa program can be invoked by evaluating (REQUIRE -'TINY) and then (CCL::TINY-SETUP). This program provides a simple -example of using several of the bridge's capabilities. - The Tiny demo creates Cocoa objects dynamically, at runtime, which is -always an option. However, for large applications, it is usually more -convenient to create your objects with Apple Interface Builder, and -store them in .nib files to be loaded when needed. Both approaches -can be freely mixed in a single program. + Cocoa Programming in OpenMCL + Cocoa is one of Apple's APIs for GUI programming; for most + purposes, development is considerably faster with Cocoa than + with the alternatives. You should have a little familiarity + with it, to better understand this section. + A small sample Cocoa program can be invoked by evaluating + (REQUIRE 'TINY) and then (CCL::TINY-SETUP). This program + provides a simple example of using several of the bridge's + capabilities. + The Tiny demo creates Cocoa objects dynamically, at + runtime, which is always an option. However, for large + applications, it is usually more convenient to create your + objects with Apple Interface Builder, and store them in .nib + files to be loaded when needed. Both approaches can be freely + mixed in a single program. - The Command Line and the Window System -OpenMCL is ordinarily a command-line application (it doesn't have a -connection to the OSX Window server, doesn't have its own menubar or -dock icon, etc.) By opening some libraries and jumping through some -hoops, it's able to sort of transform itself into a full-fledged GUI -application (while retaining its original TTY-based listener.) The -general idea is that this hybrid environment can be used to test and -protoype UI ideas and the resulting application can eventually be -fully transformed into a bundled, double-clickable application. This -is to some degree possible, but there needs to be a bit more -infrastructure in place before many people would find it easy. + The Command Line and the Window System + OpenMCL is ordinarily a command-line application (it + doesn't have a connection to the OSX Window server, doesn't + have its own menubar or dock icon, etc.) By opening some + libraries and jumping through some hoops, it's able to sort of + transform itself into a full-fledged GUI application (while + retaining its original TTY-based listener.) The general idea + is that this hybrid environment can be used to test and + protoype UI ideas and the resulting application can eventually + be fully transformed into a bundled, double-clickable + application. This is to some degree possible, but there needs + to be a bit more infrastructure in place before many people + would find it easy. Cocoa applications use the NSLog function to write -informational/warning/error messages to the application's standard -output stream. When launched by the Finder, a GUI application's -standard output is diverted to a logging facility that can be -monitored with the Console application (found in -/Applications/Utilities/Console.app). In the hybrid environment, the -application's standard output stream is usually the initial listener's -standard output stream. With two different buffered stream mechanisms -trying to write to the same underlying Unix file descriptor, it's not -uncommon to see NSLog output mixed with lisp output on the initial -listener. + informational/warning/error messages to the application's + standard output stream. When launched by the Finder, a GUI + application's standard output is diverted to a logging + facility that can be monitored with the Console application + (found in /Applications/Utilities/Console.app). In the hybrid + environment, the application's standard output stream is + usually the initial listener's standard output stream. With + two different buffered stream mechanisms trying to write to + the same underlying Unix file descriptor, it's not uncommon to + see NSLog output mixed with lisp output on the initial + listener. - Writing (and reading) Cocoa code -The syntax of the constructs used to define Cocoa classes and methods -has changed a bit (it was never documented outside of the source code -and never too well documented at all), largely as the result of -functionality offered by Randall Beer's bridge; the “standard -name-mapping conventions” referenced below are described in his -CocoaBridgeDoc.txt file, as are the constructs used to invoke (“send -messages to”) Cocoa methods. + Writing (and reading) Cocoa code The + syntax of the constructs used to define Cocoa classes and + methods has changed a bit (it was never documented outside of + the source code and never too well documented at all), largely + as the result of functionality offered by Randall Beer's + bridge; the “standard name-mapping conventions” + referenced below are described in his CocoaBridgeDoc.txt file, + as are the constructs used to invoke (“send messages + to”) Cocoa methods. All of the symbols described below are currently internal to the CCL package. - - - - - - CCL::@CLASS - - - - CCL::@SELECTOR - - - - CCL::DEFINE-OBJC-METHOD - - - - CCL::DEFINE-OBJC-CLASS-METHOD - - - - - + + + + + + - The Application Kit and Multiple Threads -The Cocoa API is broken into several pieces. The Application Kit, -affectionately called AppKit, is the one which deals with window -management, drawing, and handling events. AppKit really wants all -these things to be done by a "distinguished thread". creation, and -drawing to take place on a distinguished thread. - Apple has published some guidelines which discuss these issues in some -detail; see -the Apple Multithreading Documentation, and in particular the guidelines -on -Using the Application Kit from Multiple Threads. The upshot is that there -can sometimes be unexpected behavior when objects are created in -threads other than the distinguished event thread; eg, the event -thread sometimes starts performing operations on objects that haven't -been fully initialized. - It's certainly more convenient to do certain types of exploratory -programming by typing things into a listener or evaluating a “defun” -in an Emacs buffer; it may sometimes be necessary to be aware of this -issue while doing so. - Each thread in the Cocoa runtime system is expected to maintain a -current “autorelease pool” (an instance of the NSAutoreleasePool -class); newly created objects are often added to the current -autorelease pool (via the -autorelease method), and periodically the -current autorelease pool is sent a “-release” message, which causes -it to send “-release” messages to all of the objects that've been -added to it. - If the current thread doesn't have a current autorelease pool, the -attempt to autorelease any object will result in a severe-looking -warning being written via NSLog. The event thread maintains an -autorelease pool (it releases the current pool after each event is -processed and creates a new one for the next event), so code that only -runs in that thread should never provoke any of these severe-looking -NSLog messages. - To try to suppress these messages (and still participate in the Cocoa -memory management scheme), each listener thread (the initial listener -and any created via the “New Listener” command in the IDE) is given -a default autorelease pool; there are REPL colon-commands for -manipulating the current listener's “toplevel auturelease pool”. - In the current scheme, every time that Cocoa calls lisp code, a lisp -error handler is established which maps any lisp conditions to ObjC -exceptions and arranges that this exception is raised when the -callback to lisp returns. Whenever lisp code invokes a Cocoa method, -it does so with an ObjC exception handler in place; this handler maps -ObjC exceptions to lisp conditions and signals those conditions. - Any unhandled lisp error or ObjC exception that occurs during the -execution of the distinguished event thread's event loop causes a -message to be NSLog'ed and the event loop to (try to) continue -execution. Any error that occurs in other threads is handled at the -point of the outermost Cocoa method invocation. (Note that the error -is not necessarily “handled” in the dynamic context in which it -occurs.) + The Application Kit and Multiple Threads + The Cocoa API is broken into several pieces. The + Application Kit, affectionately called AppKit, is the one + which deals with window management, drawing, and handling + events. AppKit really wants all these things to be done by a + "distinguished thread". creation, and drawing to take place + on a distinguished thread. + Apple has published some guidelines which discuss these + issues in some detail; see the Apple Multithreading + Documentation, and in particular the guidelines on Using the + Application Kit from Multiple Threads. The upshot is that + there can sometimes be unexpected behavior when objects are + created in threads other than the distinguished event thread; + eg, the event thread sometimes starts performing operations on + objects that haven't been fully initialized. It's + certainly more convenient to do certain types of exploratory + programming by typing things into a listener or evaluating a + “defun” in an Emacs buffer; it may sometimes be + necessary to be aware of this issue while doing so. + Each thread in the Cocoa runtime system is expected to + maintain a current “autorelease pool” (an instance + of the NSAutoreleasePool class); newly created objects are + often added to the current autorelease pool (via the + -autorelease method), and periodically the current autorelease + pool is sent a “-release” message, which causes it + to send “-release” messages to all of the objects + that've been added to it. + If the current thread doesn't have a current autorelease + pool, the attempt to autorelease any object will result in a + severe-looking warning being written via NSLog. The event + thread maintains an autorelease pool (it releases the current + pool after each event is processed and creates a new one for + the next event), so code that only runs in that thread should + never provoke any of these severe-looking NSLog + messages. To try to suppress these messages (and + still participate in the Cocoa memory management scheme), each + listener thread (the initial listener and any created via the + “New Listener” command in the IDE) is given a + default autorelease pool; there are REPL colon-commands for + manipulating the current listener's “toplevel + auturelease pool”. + In the current scheme, every time that Cocoa calls lisp + code, a lisp error handler is established which maps any lisp + conditions to ObjC exceptions and arranges that this exception + is raised when the callback to lisp returns. Whenever lisp + code invokes a Cocoa method, it does so with an ObjC exception + handler in place; this handler maps ObjC exceptions to lisp + conditions and signals those conditions. Any + unhandled lisp error or ObjC exception that occurs during the + execution of the distinguished event thread's event loop + causes a message to be NSLog'ed and the event loop to (try to) + continue execution. Any error that occurs in other threads is + handled at the point of the outermost Cocoa method + invocation. (Note that the error is not necessarily + “handled” in the dynamic context in which it + occurs.) Both of these behaviors could possibly be improved; both of them seem to be substantial improvements over previous behaviors (where, @@ -8807,21 +12118,22 @@ - Acknowledgement -The Cocoa bridge was originally developed, and generously contributed -by, Randal Beer. + Acknowledgement + The Cocoa bridge was originally developed, and + generously contributed by, Randal Beer. - Building an Application Bundle -You may have noticed that (require "COCOA") takes a long time to load. -It is possible to avoid this by saving a Lisp heap image which has -everything already loaded. There is an example file which allows you -to do this, "ccl/examples/cocoa-application.lisp", by producing a -double-clickable application which runs your program. First, load -your own program. Then, do: + Building an Application Bundle + You may have noticed that (require "COCOA") takes a long + time to load. It is possible to avoid this by saving a Lisp + heap image which has everything already loaded. There is an + example file which allows you to do this, + "ccl/examples/cocoa-application.lisp", by producing a + double-clickable application which runs your program. First, + load your own program. Then, do: ? (require "COCOA-APPLICATION") - + When it finishes, you should be able to double-click the OpenMCL icon in the ccl directory, to quickly start your program. @@ -8856,458 +12168,915 @@ - Recommended Reading - Cocoa Documentation - This is the top page for all of Apple's documentation onCocoa. If you are unfamiliar with Cocoa, it is a goodplace to start. - Foundation Reference for Objective-C - This is one of the two most important Cocoa references; itcovers all of the basics, except for GUI programming. This isa reference, not a tutorial. - Application Kit Reference for Objective-C - This is the other; it covers GUI programming with Cocoain considerable depth. This is a reference, not a tutorial. - Apple Developer Documentation - This is the site which the above two documents are found on;go here to find the documentation on any other Apple API.Also go here if you need general guidance about OS X, Carbon,Cocoa, Core Foundation, or Objective C. - - + Recommended Reading> + + + + Cocoa Documentation + + + + + This is the top page for all of Apple's documentation on + Cocoa. If you are unfamiliar with Cocoa, it is a good + place to start. + + + + + + Foundation Reference for Objective-C + + + + + This is one of the two most important Cocoa references; it + covers all of the basics, except for GUI programming. This is + a reference, not a tutorial. + + + + + + + Application Kit Reference for Objective-C + + + + + This is the other; it covers GUI programming with Cocoa + in considerable depth. This is a reference, not a tutorial. + + + + + + + Apple Developer Documentation + + + + + This is the site which the above two documents are found on; + go here to find the documentation on any other Apple API. + Also go here if you need general guidance about OS X, Carbon, + Cocoa, Core Foundation, or Objective C. + + + + + - Operating-System Dictionary - - - CCL::GETENV - getenv - Name - CCL::GETENV — - Function - Synopsis - -getenv name => value - - Arguments and Values - name - a string which is the name of an existingenvironment variable;case-sensitive - value - if there is an environment variable namedname, its value, as a string; if thereis not, NIL - - - Description - Looks up the value of the environment variable named by -name, in the OS environment. - - - - CCL::SETENV - setenv - Name - CCL::SETENV — - Function - Synopsis - -setenv name value => errno - - Arguments and Values - name - a string which is the name of a new or existingenvironment variable;case-sensitive - value - a string, to be the new value of theenvironment variablenamed by name - errno - zero if the function call completes successfully;otherwise, a platform-dependent integer which describesthe problem - - - Description - Sets the value of the environment variable named by -name, in the OS environment. If there is -no such environment -variable, creates it. - - - - CCL::CURRENT-DIRECTORY-NAME - current-directory-name - Name - CCL::CURRENT-DIRECTORY-NAME — - Function - Synopsis - -current-directory-name - => path - - Values - path - a string, an absolute pathname in Posix format - withdirectory components separated by slashes - - - Description - Looks up the current working directory of the OpenMCL process; -unless it has been changed, this is the directory OpenMCL was -started in. - - - - CCL::GETUID - getuid - Name - CCL::GETUID — - Function - Synopsis - -getuid => uid - - Values - uid - a non-negative integer, identifying a specific useraccount as defined in the OS user database - - - Description - Returns the ("real") user ID of the current user. - - - - CCL::SETUID - setuid - Name - CCL::SETUID — - Function - Synopsis - -setuid uid => errno - - Arguments and Values - uid - a non-negative integer, identifying a specific useraccount as defined in the OS user database - errno - zero if the function call completes successfully;otherwise, a platform-dependent integer which describesthe problem - - - Description - Attempts to change the current user ID (both "real" and -"effective"); fails unless -the OpenMCL process has super-user privileges or the ID -given is that of the current user. - - - - CCL::SETGID - setgid - Name - CCL::SETGID — - Function - Synopsis - -setgid gid => errno - - Arguments and Values - gid - a non-negative integer, identifying a specificgroup as defined in the OS user database - errno - zero if the function call completes successfully;otherwise, a platform-dependent integer which describesthe problem - - - Description - Attempts to change the current group ID (both "real" and -"effective"); fails unless -the OpenMCL process has super-user privileges or the ID -given is that of a group to which the current user belongs. - - - - CCL::GETPID - getpid - Name - CCL::GETPID — - Function - Synopsis - -getpid => pid - - Values - pid - a non-negative integer, identifying an OS process - - - Description - Returns the ID of the OpenMCL OS process. - - - - CCL::GET-USER-HOME-DIR - get-user-home-dir - Name - CCL::GET-USER-HOME-DIR — - Function - Synopsis - -get-user-home-dir - uid => path - - Values - uid - a non-negative integer, identifying a specific useraccount as defined in the OS user database - path - a string, an absolute pathname in Posix format - withdirectory components separated by slashes; or NIL - - - Description - Looks up and returns the defined home directory of the user -identified by uid. This value comes from the -OS user database, not from the $HOME -environment variable. Returns NIL if there is no user with -the ID uid. - - - - CCL::OS-COMMAND - os-command - Name - CCL::OS-COMMAND — - Function - Synopsis - -os-command command-line - => exit-code - - Values - command-line - a string, obeying all the whitespace andescapingconventions required by the user's default system shell - - - exit-code - a non-negative integer, returned as the exitcode of a subprocess; zero indicates success - - - Description - Invokes the Posix function system(), which -invokes the user's default system shell (such as -sh or tcsh) as a new process, and has that shell execute -command-line. - If the shell was able to find the command specified in -command-line, then exit-code -is the exit code of that command. If not, it is the exit -code of the shell itself. - Notes - By convention, an exit code of 0 indicates success. There are -also other conventions; unfortunately, they are OS-specific, and -the portable macros to decode their meaning are implemented -by the system headers as C preprocessor macros. This means -that there is no good, automated way to make them available -to Lisp. - - - - CCL::@CLASS - @class - Name - CCL::@CLASS — - Macro - Synopsis - -@class class-name - - Arguments and Values - class-name - a string which denotes an existing class name, or asymbol which can be mapped to such a string via the standardname-mapping conventions for class names - - - Description - Used to refer to a known ObjC class by name. (Via the use -LOAD-TIME-VALUE, the results of a class-name -> class lookup -are cached.) - @class is obsolete as of late 2004, because -find-class now works on ObjC classes. It is described here -only because some old code still uses it. - - - - CCL::@SELECTOR - @selector - Name - CCL::@SELECTOR — - Macro - Synopsis - -@selector string - - Arguments and Values - string - a string constant, used to canonically refer to anObjC method selector - - - Description - Used to refer to an ObjC method selector (method name). Uses -LOAD-TIME-VALUE to cache the result of a string -> selector -lookup. - - - - CCL::DEFINE-OBJC-METHOD - define-objc-method - Name - CCL::DEFINE-OBJC-METHOD — - Macro - Synopsis - -define-objc-method - (selector class-name) &body body - - Arguments and Values - selector - either a string which represents the name of theselector or a list which describ+es the method's returntype, selector components, and argument types (see below.)If the first form is used, then the first form in the bodymust be a list which describes the selector's argumenttypes and return value type, as per DEFCALLBACK. - class-name - either a string which names an existing ObjC classname or a list symbol which can map to such a string via thestandard name-mapping conventions for class names. (Notethat the "canonical" lisp class name is such asymbol) - - - Description - Defines an ObjC-callable method which implements the -specified message selector for instances of the existing ObjC -class class-name. - - - - CCL::DEFINE-OBJC-CLASS-METHOD - define-objc-class-method - Name - CCL::DEFINE-OBJC-CLASS-METHOD — - Macro - Synopsis - -define-objc-class-method - (selector class-name) &body body - - Arguments and Values - As per DEFINE-OBJC-METHOD - Description - Like DEFINE-OBJC-METHOD, only used to define methods on the -class named by class-name and on its -subclasses. - For both DEFINE-OBJC-METHOD and DEFINE-OBJC-CLASS-METHOD, the -"selector" argument can be a list whose first element is a -foreign type specifier for the method's return value type and whose -subsequent elements are either: - - a non-keyword symbol, which can be mapped to a selector stringfor a parameterless method according to the standard name-mappingconventions for method selectors. - a list of alternating keywords and variable/type specifiers,where the set of keywords can be mapped to a selector string for aparameteriezed method according to the standard name-mappingconventions for method selectors and each variable/type-specifier iseither a variable name (denoting a value of type :ID) or a list whoseCAR is a variable name and whose CADR is the correspondingargument's foreign type specifier. - - - - - - CCL:*ALTERNATE-LINE-TERMINATOR* - *alternate-line-terminator* - Name - CCL:*ALTERNATE-LINE-TERMINATOR* — - Variable - Description - This variable is currently only used by the standard reader macro -function for #\; (single-line comments); that function reads successive -characters until EOF, a #\NewLine is read, or a character EQL to the -value of *alternate-line-terminator* is read. In OpenMCL for Darwin, the -value of this variable is initially #\Return ; in OpenMCL for LinuxPPC, -it's initially NIL. - Their default treatment by the #\; reader macro is the primary way -in which #\Return and #\Linefeed differ syntactally; by extending the -#\; reader macro to (conditionally) treat #\Return as a -comment-terminator, that distinction is eliminated. This seems to make -LOAD and COMPILE-FILE insensitive to line-termination issues in many -cases. It could fail in the (hopefully rare) case where a LF-terminated -(Unix) text file contains embedded #\Return characters, and this -mechanism isn't adequate to handle cases where newlines are embedded -in string constants or other tokens (and presumably should be translated -from an external convention to the external one) : it doesn't change -what READ-CHAR or READ-LINE "see", and that may be necessary to -handle some more complicated cases. - - - - :EXTERNAL-FORMAT - :external-format - Name - :EXTERNAL-FORMAT — - Keyword Argument - Description - Per ANSI CL, OpenMCL supports the :EXTERNAL-FORMAT keyword -argument to the functions OPEN, LOAD, and COMPILE-FILE. This argument is -intended to provide a standard way of providing implementation-dependent -information about the format of files opened with an element-type of -CHARACTER. This argument can meaningfully take on the values :DEFAULT -(the default), :MACOS, :UNIX, or :INFERRED in OpenMCL. - When defaulted to or specified as :DEFAULT, the format of the file -stream is determined by the value of the variable -CCL:*DEFAULT-EXTERNAL-FORMAT*. See below. - When specified as :UNIX, all characters are read from and written -to files verbatim. - When specified as :MACOS, all #\Return characters read from the -file are immediately translated to #\Linefeed (#\Newline); all #\Newline -(#\Linefeed) characters are written externally as #\Return characters. - When specified as :INFERRED and the file is open for input, the -first bufferful of input data is examined; if a #\Return character -appears in the buffer before the first #\Linefeed, the file stream's -external-format is set to :MACOS; otherwise, it is set to :UNIX. - All other values of :EXTERNAL-FORMAT - and any combinations that -don't make sense, such as trying to infer the format of a -newly-created output file stream - are treated as if :UNIX was -specified. As mentioned above, the :EXTERNAL-FORMAT argument doesn't -apply to binary file streams. - The translation performed when :MACOS is specified or inferred has -a somewhat greater chance of doing the right thing than the -*alternate-line-terminator* mechanism does; it probably has a somewhat -greater chance of doing the wrong thing, as well. - - - - CCL:*DEFAULT-EXTERNAL-FORMAT* - *default-external-format* - Name - CCL:*DEFAULT-EXTERNAL-FORMAT* — - Variable - Description - The value of this variable is used when :EXTERNAL-FORMAT is -unspecified or specified as :DEFAULT. It can meaningfully be given any -of the values :UNIX, :MACOS, or :INFERRED, each of which is interpreted -as described above. - Because there's some risk that unsolicited newline translation -could have undesirable consequences, the initial value of this variable -in OpenMCL is :UNIX. - - - - CCL::NS-LISP-STRING - ns-lisp-string - Name - CCL::NS-LISP-STRING — - Class - Superclasses - NS:NS-STRING - Initargs - :string - a Lisp string which is to be the content ofthe newly-created ns-lisp-string. - - - Description - This class -implements the interface of an NSString, which means that it can -be passed to any Cocoa or Core Foundation function which expects -one. - The string itself is stored on the Lisp heap, which -means that its memory management is automatic. However, the -ns-lisp-string object itself is a foreign -object (that is, it has an objc metaclass), and resides on the -foreign heap. Therefore, it is necessary to explicitly free -it, by sending a dealloc message. - Examples - You can create an ns-lisp-string with -make-instance, just like -any normal Lisp class: - -? (defvar *the-string* + Operating-System Dictionary + + + + getenv + + + + CCL::GETENV + + Function + + + + getenv name => value + + + + Arguments and Values + + + + name + + + a string which is the name of an existing + environment variable; + case-sensitive + + + + + value + + + if there is an environment variable named + name, its value, as a string; if there + is not, NIL + + + + + + + Description + + + Looks up the value of the environment variable named by + name, in the OS environment. + + + + + + + setenv + + + + CCL::SETENV + + Function + + + + setenv name value => errno + + + + Arguments and Values + + + + name + + + a string which is the name of a new or existing + environment variable; + case-sensitive + + + + + value + + + a string, to be the new value of the + environment variable + named by name + + + + + errno + + + zero if the function call completes successfully; + otherwise, a platform-dependent integer which describes + the problem + + + + + + + Description + + + Sets the value of the environment variable named by + name, in the OS environment. If there is + no such environment + variable, creates it. + + + + + + + current-directory-name + + + + CCL::CURRENT-DIRECTORY-NAME + + Function + + + + current-directory-name + => path + + + + Values + + + + path + + + a string, an absolute pathname in Posix format - with + directory components separated by slashes + + + + + + + Description + + + Looks up the current working directory of the OpenMCL process; + unless it has been changed, this is the directory OpenMCL was + started in. + + + + + + + getuid + + + + CCL::GETUID + + Function + + + + getuid => uid + + + + Values + + + + uid + + + a non-negative integer, identifying a specific user + account as defined in the OS user database + + + + + + + Description + + + Returns the ("real") user ID of the current user. + + + + + + + setuid + + + + CCL::SETUID + + Function + + + + setuid uid => errno + + + + Arguments and Values + + + + uid + + + a non-negative integer, identifying a specific user + account as defined in the OS user database + + + + + errno + + + zero if the function call completes successfully; + otherwise, a platform-dependent integer which describes + the problem + + + + + + + Description + + + Attempts to change the current user ID (both "real" and + "effective"); fails unless + the OpenMCL process has super-user privileges or the ID + given is that of the current user. + + + + + + + setgid + + + + CCL::SETGID + + Function + + + + setgid gid => errno + + + + Arguments and Values + + + + gid + + + a non-negative integer, identifying a specific + group as defined in the OS user database + + + + + errno + + + zero if the function call completes successfully; + otherwise, a platform-dependent integer which describes + the problem + + + + + + + Description + + + Attempts to change the current group ID (both "real" and + "effective"); fails unless + the OpenMCL process has super-user privileges or the ID + given is that of a group to which the current user belongs. + + + + + + + getpid + + + + CCL::GETPID + + Function + + + + getpid => pid + + + + Values + + + + pid + + + a non-negative integer, identifying an OS process + + + + + + + Description + + + Returns the ID of the OpenMCL OS process. + + + + + + + get-user-home-dir + + + + CCL::GET-USER-HOME-DIR + + Function + + + + get-user-home-dir + uid => path + + + + Values + + + + uid + + + a non-negative integer, identifying a specific user + account as defined in the OS user database + + + + path + + + a string, an absolute pathname in Posix format - with + directory components separated by slashes; or NIL + + + + + + + Description + + + Looks up and returns the defined home directory of the user + identified by uid. This value comes from the + OS user database, not from the $HOME + environment variable. Returns NIL if there is no user with + the ID uid. + + + + + + + os-command + + + + CCL::OS-COMMAND + + Function + + + + os-command command-line + => exit-code + + + + Values + + + + command-line + + a string, obeying all the whitespace and + escaping + conventions required by the user's default system shell + + + + + + exit-code + + a non-negative integer, returned as the exit + code of a subprocess; zero indicates success + + + + + + Description + + + Invokes the Posix function system(), which + invokes the user's default system shell (such as + sh or tcsh) as a new process, and has that shell execute + command-line. + + + + If the shell was able to find the command specified in + command-line, then exit-code + is the exit code of that command. If not, it is the exit + code of the shell itself. + + + + + Notes + + + By convention, an exit code of 0 indicates success. There are + also other conventions; unfortunately, they are OS-specific, and + the portable macros to decode their meaning are implemented + by the system headers as C preprocessor macros. This means + that there is no good, automated way to make them available + to Lisp. + + + + + + + @class + + + + CCL::@CLASS + + Macro + + + + @class class-name + + + + Arguments and Values + + + + class-name + + + a string which denotes an existing class name, or a + symbol which can be mapped to such a string via the standard + name-mapping conventions for class names + + + + + + + Description + + Used to refer to a known ObjC class by name. (Via the use + LOAD-TIME-VALUE, the results of a class-name -> class lookup + are cached.) + + + @class is obsolete as of late 2004, because + find-class now works on ObjC classes. It is described here + only because some old code still uses it. + + + + + + + @selector + + + + CCL::@SELECTOR + + Macro + + + + @selector string + + + + Arguments and Values + + + + string + + + a string constant, used to canonically refer to an + ObjC method selector + + + + + + + Description + + Used to refer to an ObjC method selector (method name). Uses + LOAD-TIME-VALUE to cache the result of a string -> selector + lookup. + + + + + + define-objc-method + + + + CCL::DEFINE-OBJC-METHOD + + Macro + + + + define-objc-method + (selector class-name) &body; body + + + + Arguments and Values + + + + selector + + + either a string which represents the name of the + selector or a list which describ+es the method's return + type, selector components, and argument types (see below.) + If the first form is used, then the first form in the body + must be a list which describes the selector's argument + types and return value type, as per DEFCALLBACK. + + + + + class-name + + + either a string which names an existing ObjC class + name or a list symbol which can map to such a string via the + standard name-mapping conventions for class names. (Note + that the "canonical" lisp class name is such a + symbol) + + + + + + + Description + + Defines an ObjC-callable method which implements the + specified message selector for instances of the existing ObjC + class class-name. + + + + + + define-objc-class-method + + + + CCL::DEFINE-OBJC-CLASS-METHOD + + Macro + + + + define-objc-class-method + (selector class-name) &body; body + + + + Arguments and Values + + As per DEFINE-OBJC-METHOD + + + + Description + + Like DEFINE-OBJC-METHOD, only used to define methods on the + class named by class-name and on its + subclasses. + + For both DEFINE-OBJC-METHOD and DEFINE-OBJC-CLASS-METHOD, the + "selector" argument can be a list whose first element is a + foreign type specifier for the method's return value type and whose + subsequent elements are either: + + + + a non-keyword symbol, which can be mapped to a selector string + for a parameterless method according to the standard name-mapping + conventions for method selectors. + + + + a list of alternating keywords and variable/type specifiers, + where the set of keywords can be mapped to a selector string for a + parameteriezed method according to the standard name-mapping + conventions for method selectors and each variable/type-specifier is + either a variable name (denoting a value of type :ID) or a list whose + CAR is a variable name and whose CADR is the corresponding + argument's foreign type specifier. + + + + + + + + *alternate-line-terminator* + + + + CCL:*ALTERNATE-LINE-TERMINATOR* + + Variable + + + + Description + + This variable is currently only used by the standard reader macro + function for #\; (single-line comments); that function reads successive + characters until EOF, a #\NewLine is read, or a character EQL to the + value of *alternate-line-terminator* is read. In OpenMCL for Darwin, the + value of this variable is initially #\Return ; in OpenMCL for LinuxPPC, + it's initially NIL. + + Their default treatment by the #\; reader macro is the primary way + in which #\Return and #\Linefeed differ syntactally; by extending the + #\; reader macro to (conditionally) treat #\Return as a + comment-terminator, that distinction is eliminated. This seems to make + LOAD and COMPILE-FILE insensitive to line-termination issues in many + cases. It could fail in the (hopefully rare) case where a LF-terminated + (Unix) text file contains embedded #\Return characters, and this + mechanism isn't adequate to handle cases where newlines are embedded + in string constants or other tokens (and presumably should be translated + from an external convention to the external one) : it doesn't change + what READ-CHAR or READ-LINE "see", and that may be necessary to + handle some more complicated cases. + + + + + + :external-format + + + + :EXTERNAL-FORMAT + + Keyword Argument + + + + Description + + Per ANSI CL, OpenMCL supports the :EXTERNAL-FORMAT keyword + argument to the functions OPEN, LOAD, and COMPILE-FILE. This argument is + intended to provide a standard way of providing implementation-dependent + information about the format of files opened with an element-type of + CHARACTER. This argument can meaningfully take on the values :DEFAULT + (the default), :MACOS, :UNIX, or :INFERRED in OpenMCL. + + When defaulted to or specified as :DEFAULT, the format of the file + stream is determined by the value of the variable + CCL:*DEFAULT-EXTERNAL-FORMAT*. See below. + + When specified as :UNIX, all characters are read from and written + to files verbatim. + + When specified as :MACOS, all #\Return characters read from the + file are immediately translated to #\Linefeed (#\Newline); all #\Newline + (#\Linefeed) characters are written externally as #\Return characters. + + When specified as :INFERRED and the file is open for input, the + first bufferful of input data is examined; if a #\Return character + appears in the buffer before the first #\Linefeed, the file stream's + external-format is set to :MACOS; otherwise, it is set to :UNIX. + + All other values of :EXTERNAL-FORMAT - and any combinations that + don't make sense, such as trying to infer the format of a + newly-created output file stream - are treated as if :UNIX was + specified. As mentioned above, the :EXTERNAL-FORMAT argument doesn't + apply to binary file streams. + + The translation performed when :MACOS is specified or inferred has + a somewhat greater chance of doing the right thing than the + *alternate-line-terminator* mechanism does; it probably has a somewhat + greater chance of doing the wrong thing, as well. + + + + + + *default-external-format* + + + + CCL:*DEFAULT-EXTERNAL-FORMAT* + + Variable + + + + Description + + The value of this variable is used when :EXTERNAL-FORMAT is + unspecified or specified as :DEFAULT. It can meaningfully be given any + of the values :UNIX, :MACOS, or :INFERRED, each of which is interpreted + as described above. + + Because there's some risk that unsolicited newline translation + could have undesirable consequences, the initial value of this variable + in OpenMCL is :UNIX. + + + + + + ns-lisp-string + + + + CCL::NS-LISP-STRING + + Class + + + + Superclasses + + NS:NS-STRING + + + + Initargs + + + + :string + + + + a Lisp string which is to be the content of + the newly-created ns-lisp-string. + + + + + + + + Description + + + This class + implements the interface of an NSString, which means that it can + be passed to any Cocoa or Core Foundation function which expects + one. + + + + The string itself is stored on the Lisp heap, which + means that its memory management is automatic. However, the + ns-lisp-string object itself is a foreign + object (that is, it has an objc metaclass), and resides on the + foreign heap. Therefore, it is necessary to explicitly free + it, by sending a dealloc message. + + + + + Examples + + + You can create an ns-lisp-string with + make-instance, just like + any normal Lisp class: + + + ? (defvar *the-string* (make-instance 'ccl::ns-lisp-string - :string "Hello, Cocoa.")) - - When you are done with the string, you must explicitly -deallocate it: - -? (ccl::send *the-string* 'dealloc) - - You may wish to use an unwind-protect -form to ensure that this happens: - -(let (*the-string*) + :string "Hello, Cocoa.")) + + + When you are done with the string, you must explicitly + deallocate it: + + + ? (ccl::send *the-string* 'dealloc) + + + You may wish to use an unwind-protect + form to ensure that this happens: + + + (let (*the-string*) (unwind-protect (progn (setq *the-string* (make-instance 'ccl::ns-lisp-string @@ -9316,50 +13085,61 @@ (ccl::send *the-string* 'length))) (when *the-string* - (ccl::send *the-string* 'dealloc)))) - - Notes - Currently, ns-lisp-string is defined in -the file ccl/examples/cocoa-backtrace.lisp, which is a -rather awkward place. It was probably not originally meant -as a public utility at all. It would be good if it were -moved someplace else. Use at your own risk. - + (ccl::send *the-string* 'dealloc)))) + + + + Notes + + + Currently, ns-lisp-string is defined in + the file ccl/examples/cocoa-backtrace.lisp, which is a + rather awkward place. It was probably not originally meant + as a public utility at all. It would be good if it were + moved someplace else. Use at your own risk. + + + - Understanding and Configuring the Garbage Collector + Understanding and Configuring the Garbage Collector - Heap space allocation -Release 0.10 or later of OpenMCL uses a different memory management -scheme than previous versions did. Those earlier versions would allocate a -block of memory (of specified size) at startup and would allocate lisp -objects within that block. When that block filled with live (non-GCed) -objects, the lisp would signal a "heap full" condition. The heap -size imposed a limit on the size of the largest object that could be -allocated. - The new strategy involves reserving a very large (2GB on DarwinPPC32, -1GB on LinuxPPC, "very large" on 64-bit implementations) block at -startup and consuming (and relinquishing) its contents as the size of -the live lisp heap data grows and shrinks. After the initial heap image -loads and after each full GC, the lisp kernel will try to ensure that a -specified amount (the "lisp-heap-gc-threshold") of free memory is -available. The inital value of this kernel variable is 16MB on 32-bit -implementations and 32MB on 64-bit implementations ; it can be -manipulated from Lisp (see below.) - The large reserved memory block consumes very little in the way of -system resources; memory that's actually committed to the lisp heap -(live data and the "threshold" area where allocation takes place) -consumes finite resources (physical memory and swap space). The lisp's -consumption of those resources is proportional to its actual memory usage, -which is generally a good thing. - This scheme is much more flexible than the old one, but it may also -increase the possibility that those resources can become exhausted. -Neither the new scheme nor the old handles that situation gracefully; -under the old scheme, a program that consumes lots of memory may have run -into an artificial limit on heap size before exhausting virtual memory. - The -R or –heap-reserve command-line option can be use to limit the -size of the reserved block and therefore bound heap expansion. Running + Heap space allocation + Release 0.10 or later of OpenMCL uses a different memory + management scheme than previous versions did. Those earlier + versions would allocate a block of memory (of specified size) at + startup and would allocate lisp objects within that block. When + that block filled with live (non-GCed) objects, the lisp would + signal a "heap full" condition. The heap size imposed a limit on + the size of the largest object that could be allocated. + The new strategy involves reserving a very large (2GB on + DarwinPPC32, 1GB on LinuxPPC, "very large" on 64-bit + implementations) block at startup and consuming (and + relinquishing) its contents as the size of the live lisp heap + data grows and shrinks. After the initial heap image loads and + after each full GC, the lisp kernel will try to ensure that a + specified amount (the "lisp-heap-gc-threshold") of free memory + is available. The inital value of this kernel variable is 16MB + on 32-bit implementations and 32MB on 64-bit implementations ; + it can be manipulated from Lisp (see below.) + The large reserved memory block consumes very little in + the way of system resources; memory that's actually committed to + the lisp heap (live data and the "threshold" area where + allocation takes place) consumes finite resources (physical + memory and swap space). The lisp's consumption of those + resources is proportional to its actual memory usage, which is + generally a good thing. + This scheme is much more flexible than the old one, but it + may also increase the possibility that those resources can + become exhausted. Neither the new scheme nor the old handles + that situation gracefully; under the old scheme, a program that + consumes lots of memory may have run into an artificial limit on + heap size before exhausting virtual memory. + + The -R or –heap-reserve command-line option can be + use to limit the size of the reserved block and therefore bound + heap expansion. Running > openmcl --heap-reserve 8M @@ -9370,476 +13150,684 @@ - The Ephemeral GC -For many programs, the following observations are true to a very -large degree: - - Most heap-allocated objects have very short lifetimes ("areephemeral"): they become inaccessible soon after they're created. - Most non-ephemeral objects have very long lifetimes: it'srarely productive for the GC to consider reclaiming them, sinceit's rarely able to do so. (An object that's survived a largenumber of GCs is likely to survive the next one. That's not alwaystrue of course, but it's a reasonable heuristic.) - It's relatively rare for an old object to be destructivelymodified (via SETF) so that it points to a new one, therefore mostreferences to newly-created objects can be found in the stacks andregisters of active threads. It's not generally necessary to scanthe entire heap to find references to new objects (or to prove thatsuch references don't exists), though it is necessary to keeptrack of the (hopefully exceptional) cases where old objects aremodified to point at new ones. - - "Ephemeral" (or "generational") garbage collectors try to exploit -these observations: by concentrating on frequently reclaiming -newly-created objects quickly, it's less often necessary to do more -expensive GCs of the entire heap in order to reclaim unreferenced memory. -In some environments, the pauses associated with such full GCs can be -noticable and disruptive, and minimizing the frequency (and sometimes the -duration) of these pauses is probably the EGC's primary goal (though -there may be other benefits, such as increased locality of reference and -better paging behavior.) The EGC generally leads to slightly longer -execution times (and slightly higher, amortized GC time), but there are -cases where it can improve overall performance as well; the nature and -degree of its impact on performance is highly application-dependant. - Most EGC strategies (including the one employed by OpenMCL) -logically or physically divide memory into one or more areas of relatively -young objects ("generations") and one or more areas of old objects. -Objects that have survived one or more GCs as members of a young -generation are promoted (or "tenured") into an older generation, where -they may or may not survive long enough to be promoted to the next -generation and eventually may become "old" objects that can only be -reclaimed if a full GC proves that there are no live references to them. -This filtering process isn't perfect - a certain amount of premature -tenuring may take place - but it usually works very well in practive. - It's important to note that a GC of the youngest generation is -typically very fast (perhaps a few milliseconds on a modern CPU, depending -on various factors), OpenMCL's EGC is not concurrent and doesn't -offer realtime guarantees. - OpenMCL's EGC maintains three ephemeral -generations; all newly created objects are created as members of the -youngest generation. Each generation has an associated -threshold, which indicates the number of bytes in it -and all younger generations that can be allocated before a GC is -triggered. These GCs will involve the target generation and all younger -ones (and may therefore cause some premature tenuring); since the older -generations have larger thresholds, they're GCed less frequently and -most short-lived objects that make it into an older generation tend not to -survive there very long. - The EGC can be enabled or disabled -under program control; under some circumstances, it may be enabled but -inactive (because a full GC is imminent.) Since it -may be hard to know or predict the consing behavior of other threads, the -distinction between the "active" and "inactive" state isn't very -meaningful, especially when native threads are involved. + The Ephemeral GC + For many programs, the following observations are true to + a very large degree: + + + + Most heap-allocated objects have very short lifetimes ("are + ephemeral"): they become inaccessible soon after they're created. + + + + Most non-ephemeral objects have very long lifetimes: it's + rarely productive for the GC to consider reclaiming them, since + it's rarely able to do so. (An object that's survived a large + number of GCs is likely to survive the next one. That's not always + true of course, but it's a reasonable heuristic.) + + + + It's relatively rare for an old object to be destructively + modified (via SETF) so that it points to a new one, therefore most + references to newly-created objects can be found in the stacks and + registers of active threads. It's not generally necessary to scan + the entire heap to find references to new objects (or to prove that + such references don't exists), though it is necessary to keep + track of the (hopefully exceptional) cases where old objects are + modified to point at new ones. + + + + + Most heap-allocated objects have very short lifetimes ("are + ephemeral"): they become inaccessible soon after they're created. + + + + Most non-ephemeral objects have very long lifetimes: it's + rarely productive for the GC to consider reclaiming them, since + it's rarely able to do so. (An object that's survived a large + number of GCs is likely to survive the next one. That's not always + true of course, but it's a reasonable heuristic.) + + + + It's relatively rare for an old object to be destructively + modified (via SETF) so that it points to a new one, therefore most + references to newly-created objects can be found in the stacks and + registers of active threads. It's not generally necessary to scan + the entire heap to find references to new objects (or to prove that + such references don't exists), though it is necessary to keep + track of the (hopefully exceptional) cases where old objects are + modified to point at new ones. + + + + "Ephemeral" (or "generational") garbage collectors try to + exploit these observations: by concentrating on frequently + reclaiming newly-created objects quickly, it's less often + necessary to do more expensive GCs of the entire heap in order + to reclaim unreferenced memory. In some environments, the + pauses associated with such full GCs can be noticable and + disruptive, and minimizing the frequency (and sometimes the + duration) of these pauses is probably the EGC's primary goal + (though there may be other benefits, such as increased locality + of reference and better paging behavior.) The EGC generally + leads to slightly longer execution times (and slightly higher, + amortized GC time), but there are cases where it can improve + overall performance as well; the nature and degree of its impact + on performance is highly application-dependant. + Most EGC strategies (including the one employed by + OpenMCL) logically or physically divide memory into one or more + areas of relatively young objects ("generations") and one or + more areas of old objects. Objects that have survived one or + more GCs as members of a young generation are promoted (or + "tenured") into an older generation, where they may or may not + survive long enough to be promoted to the next generation and + eventually may become "old" objects that can only be reclaimed + if a full GC proves that there are no live references to them. + This filtering process isn't perfect - a certain amount of + premature tenuring may take place - but it usually works very + well in practive. + It's important to note that a GC of the youngest + generation is typically very fast (perhaps a few milliseconds on + a modern CPU, depending on various factors), OpenMCL's EGC is + not concurrent and doesn't offer realtime guarantees. + OpenMCL's EGC maintains three ephemeral generations; all + newly created objects are created as members of the youngest + generation. Each generation has an associated + threshold, which indicates the number of + bytes in it and all younger generations that can be allocated + before a GC is triggered. These GCs will involve the target + generation and all younger ones (and may therefore cause some + premature tenuring); since the older generations have larger + thresholds, they're GCed less frequently and most short-lived + objects that make it into an older generation tend not to + survive there very long. + The EGC can be enabled or + disabled under program control; under some + circumstances, it may be enabled but + inactive (because a full GC is imminent.) + Since it may be hard to know or predict the consing behavior of + other threads, the distinction between the "active" and + "inactive" state isn't very meaningful, especially when native + threads are involved. - GC Page reclamation policy - After a full GC finishes, it'll try to ensure that at least -(LISP-HEAP-GC-THRESHOLD) of virtual memory are available; objects will be -allocated in this block of memory until it fills up, the GC is triggered, -and the process repeats itself. - Many programs reach near stasis in terms of the amount of logical -memory that's in use after full GC (or run for long periods of time in -a nearly static state), so the logical address range used for consing -after the Nth full GC is likely to be nearly or entirely identical to the -address range used by the N+1th full GC. - By default (and traditionally in OpenMCL), the GC's policy is to -"release" the pages in this address range: to advise the virtual memory -system that the pages contain garbage and any physical pages associated -with them don't need to be swapped out to disk before being reused and -to (re-)map the logical address range so that the pages will be -zero-filled by the virtual memory system when they're next accessed. -This policy is intended to reduce the load on the VM system and keep -OpenMCL's working set to a minimum. - For some programs (especially those that cons at a very high rate), -the default policy may be less than ideal: releasing pages that're -going to be needed almost immediately - and zero-fill-faulting them back -in, lazily - incurs unnecessary overhead. (There's a false economy -associated with minimizing the size of the working set if it's just -going to shoot back up again until the next GC.) A policy of "retaining" -pages between GCs might work better in such an environment. - Functions described below give the user some control over this -behavior. An adaptive, feedback-mediated approach might yield a better -solution. + GC Page reclamation policy + After a full GC finishes, it'll try to ensure that at + least (LISP-HEAP-GC-THRESHOLD) of virtual memory are available; + objects will be allocated in this block of memory until it fills + up, the GC is triggered, and the process repeats itself. + Many programs reach near stasis in terms of the amount of + logical memory that's in use after full GC (or run for long + periods of time in a nearly static state), so the logical + address range used for consing after the Nth full GC is likely + to be nearly or entirely identical to the address range used by + the N+1th full GC. + By default (and traditionally in OpenMCL), the GC's policy + is to "release" the pages in this address range: to advise the + virtual memory system that the pages contain garbage and any + physical pages associated with them don't need to be swapped out + to disk before being reused and to (re-)map the logical address + range so that the pages will be zero-filled by the virtual + memory system when they're next accessed. This policy is + intended to reduce the load on the VM system and keep OpenMCL's + working set to a minimum. + For some programs (especially those that cons at a very + high rate), the default policy may be less than ideal: releasing + pages that're going to be needed almost immediately - and + zero-fill-faulting them back in, lazily - incurs unnecessary + overhead. (There's a false economy associated with minimizing + the size of the working set if it's just going to shoot back up + again until the next GC.) A policy of "retaining" pages between + GCs might work better in such an environment. + Functions described below give the user some control over + this behavior. An adaptive, feedback-mediated approach might + yield a better solution. - "Pure" areas are read-only, paged from image file -SAVE-APPLICATION identifies code vectors and the pnames of interned -symbols and copies these objects to a "pure" area of the image -file it creates. (The "pure" area accounts for most of what the -ROOM function reports as "static" space.) - When the resulting image file is loaded, the pure area of the file -is now memory-mapped with read-only access. Code and pure data are paged -in from the image file as needed (and don't compete for global virtual -memory resources with other memory areas.) - Code-vectors and interned symbol pnames are immutable : it is an -error to try to change the contents of such an object. Previously, that -error would have manifested itself in some random way. In the new scheme, -it'll manifest itself as an "unhandled exception" error in the -Lisp kernel. The kernel could probably be made to detect a spurious, -accidental write to read-only space and signal a lisp error in that case, -but it doesn't yet do so. - The image file should be opened and/or mapped in some mode which -disallows writing to the memory-mapped regions of the file from other -processes. I'm not sure of how to do that; writing to the file when -it's mapped by OpenMCL can have unpredictable and unpleasant results. -SAVE-APPLICATION will delete its output file's directory entry and -create a new file; one may need to exercise care when using file system -utilities (like tar, for instance) that might overwrite an existing image -file. + "Pure" areas are read-only, paged from image file + SAVE-APPLICATION identifies code vectors and the pnames of + interned symbols and copies these objects to a "pure" area of + the image file it creates. (The "pure" area accounts for most of + what the ROOM function reports as "static" space.) + When the resulting image file is loaded, the pure area of + the file is now memory-mapped with read-only access. Code and + pure data are paged in from the image file as needed (and don't + compete for global virtual memory resources with other memory + areas.) + Code-vectors and interned symbol pnames are immutable : it + is an error to try to change the contents of such an + object. Previously, that error would have manifested itself in + some random way. In the new scheme, it'll manifest itself as an + "unhandled exception" error in the Lisp kernel. The kernel could + probably be made to detect a spurious, accidental write to + read-only space and signal a lisp error in that case, but it + doesn't yet do so. + The image file should be opened and/or mapped in some mode + which disallows writing to the memory-mapped regions of the file + from other processes. I'm not sure of how to do that; writing to + the file when it's mapped by OpenMCL can have unpredictable and + unpleasant results. SAVE-APPLICATION will delete its output + file's directory entry and create a new file; one may need to + exercise care when using file system utilities (like tar, for + instance) that might overwrite an existing image file. - Weak Hash Tables -In general, a "weak reference" is a reference to an object which -will not prevent the object from being garbage-collected. For -example, suppose that you want to keep a list of all the objects -of a certain type. If you don't take special steps, the fact that -you have a list of them will mean that the objects are always -"live", because you can always reference them through the list. -Therefore, they will never be garbage-collected, and their memory -will never be reclaimed, even if they are referenced nowhere else -in the program. You may want this behaviour. If you don't, you -need weak references. + Weak Hash Tables + In general, a "weak reference" is a reference to an object + which will not prevent the object from being garbage-collected. + For example, suppose that you want to keep a list of all the + objects of a certain type. If you don't take special steps, the + fact that you have a list of them will mean that the objects are + always "live", because you can always reference them through the + list. Therefore, they will never be garbage-collected, and + their memory will never be reclaimed, even if they are + referenced nowhere else in the program. You may want this + behaviour. If you don't, you need weak references. OpenMCL supports weak references with "weak hash tables". -Hash tables may be weak with respect to either their keys or -their values. To make a hash table with weak keys, invoke -make-hash-table with the option :weak t, -or, equivalently, :weak :key. To make one with weak values, -use :weak :value. When the key is weak, the equality test -must be #'eq (because it wouldn't make sense otherwise). - When garbage-collection occurs, key-value pairs are removed -from the hash table if there are no other references to the -weak element of the pair (key or value). - In general, weak-key hash tables are useful when you want to -use the hash to store some extra information about the objects -you look up in it, while weak-value hash tables are useful when you -want to use the hash as an index for looking up objects. - If you are experimenting with weak hash tables interactively, remember -that an object is not dead if it was returned by one of the last -three interactively-evaluated expressions, because of the variables -*, **, and -***. The easy workaround is to evaluate some -meaningless expression before invoking gc, -to get the object out of the repl variables. + Hash tables may be weak with respect to either their keys or + their values. To make a hash table with weak keys, invoke + make-hash-table with the option :weak t, or, + equivalently, :weak :key. To make one with weak values, use + :weak :value. When the key is weak, the equality test must be + #'eq (because it wouldn't make sense otherwise). + When garbage-collection occurs, key-value pairs are + removed from the hash table if there are no other references to + the weak element of the pair (key or value). + In general, weak-key hash tables are useful when you want + to use the hash to store some extra information about the + objects you look up in it, while weak-value hash tables are + useful when you want to use the hash as an index for looking up + objects. + If you are experimenting with weak hash tables + interactively, remember that an object is not dead if it was + returned by one of the last three interactively-evaluated + expressions, because of the variables *, + **, and ***. The easy + workaround is to evaluate some meaningless expression before + invoking gc, to get the object out of the + repl variables. - Garbage-Collection Dictionary - - - LISP-HEAP-GC-THRESHOLD - lisp-heap-gc-threshold - Name - LISP-HEAP-GC-THRESHOLD — - Function - Synopsis - -lisp-heap-gc-threshold - - Description - Returns the value of the kernel variable that specifies the -amount of free space to leave in the heap after full GC. - - - - SET-LISP-HEAP-GC-THRESHOLD - set-lisp-heap-gc-threshold - Name - SET-LISP-HEAP-GC-THRESHOLD — - Function - Synopsis - - - lisp-heap-gc-threshold new-threshold - - - Arguments and Values - new-value - The requested new lisp-heap-gc-threshold. - - - Description - Sets the value of the kernel variable that specifies the -amount of free space to leave in the heap after full GC to -new-value, which should be a non-negative fixnum. Returns the -value of that kernel variable (which may be somewhat larger than -what was specified). - - - - USE-LISP-HEAP-GC-THRESHOLD - use-lisp-heap-gc-threshold - Name - USE-LISP-HEAP-GC-THRESHOLD — - Function - Synopsis - - - use-lisp-heap-gc-threshold - - - Description - Tries to grow or shrink lisp's heap space, so that the -free space is (approximately) equal to the current heap threshold. -Returns NIL - - - - EGC - egc - Name - EGC — - Function - Synopsis - -egc arg - - Arguments and Values - arg - a generalized boolean - - - Description - Enables the EGC if arg is non-nil, disables the EGC -otherwise. Returns the previous enabled status. Although this -function is thread-safe (in the sense that calls to it are -serialized), it doesn't make a whole lot of sense to be -turning the EGC on and off from multiple threads ... - - - - EGC-ENABLED-P - egc-enabled-p - Name - EGC-ENABLED-P — - Function - Synopsis - -egc-enabled-p - - Description - Returns T if the EGC was enabled at the time of the call, -NIL otherwise. - - - - EGC-ACTIVE-P - egc-active-p - Name - EGC-ACTIVE-P — - Function - Synopsis - -egc-active-p - - Description - Returns T if the EGC was active at the time of the call, NIL -otherwise. Since this is generally a volatile piece of -information, it's not clear whether this function serves a -useful purpose when native threads are involved. - - - - EGC-CONFIGURATION - egc-configuration - Name - EGC-CONFIGURATION — - Function - Synopsis - -egc-configuration - - Description - Returns, as multiple values, the sizes in kilobytes of the -thresholds associated with the youngest ephemeral generation, the -middle ephemeral generation, and the oldest ephemeral generation - - - - CONFIGURE-EGC - configure-egc - Name - CONFIGURE-EGC — - Function - Synopsis - -configure-egc + Garbage-Collection Dictionary + + + + lisp-heap-gc-threshold + + + + LISP-HEAP-GC-THRESHOLD + + Function + + + + lisp-heap-gc-threshold + + + + Description + + Returns the value of the kernel variable that specifies the + amount of free space to leave in the heap after full GC. + + + + + + set-lisp-heap-gc-threshold + + + + SET-LISP-HEAP-GC-THRESHOLD + + Function + + + + + lisp-heap-gc-threshold new-threshold + + + + + Arguments and Values + + + + new-value + + + The requested new lisp-heap-gc-threshold. + + + + + + + Description + + Sets the value of the kernel variable that specifies the + amount of free space to leave in the heap after full GC to + new-value, which should be a non-negative fixnum. Returns the + value of that kernel variable (which may be somewhat larger than + what was specified). + + + + + + use-lisp-heap-gc-threshold + + + + USE-LISP-HEAP-GC-THRESHOLD + + Function + + + + + use-lisp-heap-gc-threshold + + + + + Description + + Tries to grow or shrink lisp's heap space, so that the + free space is (approximately) equal to the current heap threshold. + Returns NIL + + + + + + egc + + + + EGC + + Function + + + + egc arg + + + + Arguments and Values + + + + arg + + + a generalized boolean + + + + + + + Description + + Enables the EGC if arg is non-nil, disables the EGC + otherwise. Returns the previous enabled status. Although this + function is thread-safe (in the sense that calls to it are + serialized), it doesn't make a whole lot of sense to be + turning the EGC on and off from multiple threads ... + + + + + + egc-enabled-p + + + + EGC-ENABLED-P + + Function + + + + egc-enabled-p + + + + Description + + Returns T if the EGC was enabled at the time of the call, + NIL otherwise. + + + + + + egc-active-p + + + + EGC-ACTIVE-P + + Function + + + + egc-active-p + + + + Description + + Returns T if the EGC was active at the time of the call, NIL + otherwise. Since this is generally a volatile piece of + information, it's not clear whether this function serves a + useful purpose when native threads are involved. + + + + + + egc-configuration + + + + EGC-CONFIGURATION + + Function + + + + egc-configuration + + + + Description + + Returns, as multiple values, the sizes in kilobytes of the + thresholds associated with the youngest ephemeral generation, the + middle ephemeral generation, and the oldest ephemeral generation + + + + + + configure-gcc + + + + CONFIGURE-GCC + + Function + + + + configure-egc generation-0-size generation-1-size - generation-2-size - - Arguments and Values - generation-0-size - the requested threshold size of the youngestgeneration, in kilobytes - generation-1-size - the requested threshold size of the middle generation,in kilobytes - generation-2-size - the requested threshold size of the oldest generation,in kilobytes - - - Description - If the EGC is currently disabled, puts the indicated -threshold sizes in effect and returns T, otherwise, returns NIL. -(The provided threshold sizes are rounded up to a multiple of -64Kbytes in OpenMCL 0.14 and later, and to a multiple of 32KBytes in earlier -versions.) - - - - GC-RETAIN-PAGES - gc-retain-pages - Name - GC-RETAIN-PAGES — - Function - Synopsis - -gc-retain-pages arg - - Arguments and Values - arg - a generalized boolean - - - Description - Tries to influence the GC to retain/recycle the pages -allocated between GCs if arg is true, and to release them -otherwise. This is generally a tradeoff between paging and other -VM considerations. - - - - GC-RETAINING-PAGES - gc-retaining-pages - Name - GC-RETAINING-PAGES — - Function - Synopsis - -gc-retaining-pages - - Description - Returns T if the GC tries to retain pages between full GCs -and NIL if it's trying to release them to improve VM paging -performance. - + generation-2-size + + + + Arguments and Values + + + + generation-0-size + + + the requested threshold size of the youngest + generation, in kilobytes + + + + + generation-1-size + + + the requested threshold size of the middle generation, + in kilobytes + + + + + generation-2-size + + + the requested threshold size of the oldest generation, + in kilobytes + + + + + + + Description + + If the EGC is currently disabled, puts the indicated + threshold sizes in effect and returns T, otherwise, returns NIL. + (The provided threshold sizes are rounded up to a multiple of + 64Kbytes in OpenMCL 0.14 and to a multiple of 32KBytes in earlier + versions.) + + + + + + gc-retain-pages + + + + GC-RETAIN-PAGES + + Function + + + + gc-retain-pages arg + + + + Arguments and Values + + + + arg + + + a generalized boolean + + + + + + + Description + + Tries to influence the GC to retain/recycle the pages + allocated between GCs if arg is true, and to release them + otherwise. This is generally a tradeoff between paging and other + VM considerations. + + + + + + gc-retaining-pages + + + + GC-RETAINING-PAGES + + Function + + + + gc-retaining-pages + + + + Description + + Returns T if the GC tries to retain pages between full GCs + and NIL if it's trying to release them to improve VM paging + performance. + + - Implementation Details of OpenMCL -This chapter describes many aspects of OpenMCL's implementation as of -(roughly) version 1.1. Details vary a bit between the three archutectures -(PPC32, PPC64, and X86-64) currently supported and those details change -over time, so the definitive reference is the source code (especially -some files in the ccl/compiler/ directory whose names contain the string -"arch" and some files in the ccl/lisp-kernel/ directory whose namee -contain the string "constants".) Hopefully, this chapter will make -it easier for someone who's interested to read and understand the -contents of those files. + Implementation Details of OpenMCL + This chapter describes many aspects of OpenMCL's + implementation as of (roughly) version 1.1. Details vary a bit + between the three archutectures (PPC32, PPC64, and X86-64) + currently supported and those details change over time, so the + definitive reference is the source code (especially some files in + the ccl/compiler/ directory whose names contain the string "arch" + and some files in the ccl/lisp-kernel/ directory whose namee + contain the string "constants".) Hopefully, this chapter will + make it easier for someone who's interested to read and understand + the contents of those files. - Threads and exceptions -OpenMCL's threads are "native" (meaning that they're scheduled and -controlled by the operating system.) Most of the implications of -this are discussed elsewhere; this section tries to describe how -threads look from the lisp kernel's perspective (and especailly -from the GC's point of view.) - OpenMCL's runtime system tries to use machine-level exception -mechanisms (conditional traps when available, illegal instructions, -memory access protection in some cases) to detect and handle ... -exceptional situations. These situations include some TYPE-ERRORs -and PROGRAM-ERRORS (notably wrong-number-of-args errors), and also -include cases like "not being able to allocate memory without GCing -or obtaining more memory from the OS." The general idea is that -it's usually faster to pay (very occasional) exception-processing -overhead and figure out what's going on in an exception handler than -it is to maintain enough state and context to handle an exceptional -case via a lighter-weight mechanism when that exceptional case -(by definition) rarely occurs. - Some emulated execution environments (the Rosetta PPC emulator on -x86 versions of OSX) don't provide accurate exception information -to exception handling functions. OpenMCL can't run in such environments. + Threads and exceptions + OpenMCL's threads are "native" (meaning that they're + scheduled and controlled by the operating system.) Most of the + implications of this are discussed elsewhere; this section tries + to describe how threads look from the lisp kernel's perspective + (and especailly from the GC's point of view.) + OpenMCL's runtime system tries to use machine-level + exception mechanisms (conditional traps when available, illegal + instructions, memory access protection in some cases) to detect + and handle ... exceptional situations. These situations + include some TYPE-ERRORs and PROGRAM-ERRORS (notably + wrong-number-of-args errors), and also include cases like "not + being able to allocate memory without GCing or obtaining more + memory from the OS." The general idea is that it's usually + faster to pay (very occasional) exception-processing overhead + and figure out what's going on in an exception handler than it + is to maintain enough state and context to handle an exceptional + case via a lighter-weight mechanism when that exceptional case + (by definition) rarely occurs. + Some emulated execution environments (the Rosetta PPC + emulator on x86 versions of OSX) don't provide accurate + exception information to exception handling functions. OpenMCL + can't run in such environments. - The Thread Context Record -When a lisp thread is first created (or when a thread created by -foreign code first calls back to lisp), a data structure called -a Thread Context Record (or TCR) is allocated and initialized. On -modern versions of Linux and FreeBSD, the allocation actually happens -via a set of thread-local-storage ABI extensions, so a thread's -TCR is created when the thread is created and dies when the thread -dies. (The World's Most Advanced Operating System - as Apple's -marketing literature refers to Darwin - is not very advanced in this -regard, and I know of no reason to assume that advances will be made -in this area anytime soon.) - A TCR contains a few dozen fields (and is therefore a few hundred -bytes in size.) The fields are mostly thread-specific information -about the thread's stacks' locations and sizes, information about -the underlying (POSIX) thread, and information about the thread's -dynamic binding history and pending CATCH/UNWIND-PROTECTs. Some -of this information could be kept in individual machine registers -while the thread is running (and the PPC - which has more registers -available - keeps a few things in registers that the X86-64 has -to access via the TCR), but it's important to remember that the -information is thread-specific and can't (for instance) be kept -in a fixed global memory location. - When lisp code is running, the current thread's TCR is kept in -a register. On PPC platforms, a general purpose register is used; -on x86-64, an (otherwise nearly useless) segment register works -well (prevents the expenditure of a more generally useful general- -purpose register for this purpose.) - The address of a TCR is aligned in memory in such a way that a -FIXNUM can be used to represent it. The lisp function CCL::%CURRENT-TCR -returns the calling thread's TCR as a fixnum; actual value of the -TCR's address is 4 or 8 times the value of this fixnum. - When the lisp kernel initializes a new TCR, it's added to a global -list maintained by the kernel; when a thread exits, its TCR is removed -from this list. - When a thread calls foreign code, lisp stack pointers are saved in its -TCR, lisp registers (at least those whose value should be preserved -across the call) are saved on the thread's value stack, and (on x86-64) -RSP is switched to the control stack. A field in the TCR -(tcr.valence) is then set to indicate that the thread is running foreigm -code, foreign argument registers are loaded from a frame on the foreign -stack, and the foreign function is called. (That's a little oversimplified -and possibly inaccurate, but the important things to note are that the -thread "stops following lisp stack and register usage conventions" and -that it advertises the fact that it's done so. Similar transitions in -a thread's state ("valence") occur when it enters of exits an exception -handler (which is sort of an OS/hardware-mandated foreign function call -where the OS thoughtfully saves the thread's register state for it -beforehand.) + The Thread Context Record + + When a lisp thread is first created (or when a thread + created by foreign code first calls back to lisp), a data + structure called a Thread Context Record (or TCR) is allocated + and initialized. On modern versions of Linux and FreeBSD, the + allocation actually happens via a set of thread-local-storage + ABI extensions, so a thread's TCR is created when the thread + is created and dies when the thread dies. (The World's Most + Advanced Operating System - as Apple's marketing literature + refers to Darwin - is not very advanced in this regard, and I + know of no reason to assume that advances will be made in this + area anytime soon.) + A TCR contains a few dozen fields (and is therefore a + few hundred bytes in size.) The fields are mostly + thread-specific information about the thread's stacks' + locations and sizes, information about the underlying (POSIX) + thread, and information about the thread's dynamic binding + history and pending CATCH/UNWIND-PROTECTs. Some of this + information could be kept in individual machine registers + while the thread is running (and the PPC - which has more + registers available - keeps a few things in registers that the + X86-64 has to access via the TCR), but it's important to + remember that the information is thread-specific and can't + (for instance) be kept in a fixed global memory + location. + When lisp code is running, the current thread's TCR is + kept in a register. On PPC platforms, a general purpose + register is used; on x86-64, an (otherwise nearly useless) + segment register works well (prevents the expenditure of a + more generally useful general- purpose register for this + purpose.) + The address of a TCR is aligned in memory in such a way + that a FIXNUM can be used to represent it. The lisp function + CCL::%CURRENT-TCR returns the calling thread's TCR as a + fixnum; actual value of the TCR's address is 4 or 8 times the + value of this fixnum. + When the lisp kernel initializes a new TCR, it's added + to a global list maintained by the kernel; when a thread + exits, its TCR is removed from this list. + When a thread calls foreign code, lisp stack pointers + are saved in its TCR, lisp registers (at least those whose + value should be preserved across the call) are saved on the + thread's value stack, and (on x86-64) RSP is switched to the + control stack. A field in the TCR (tcr.valence) is then set + to indicate that the thread is running foreigm code, foreign + argument registers are loaded from a frame on the foreign + stack, and the foreign function is called. (That's a little + oversimplified and possibly inaccurate, but the important + things to note are that the thread "stops following lisp stack + and register usage conventions" and that it advertises the + fact that it's done so. Similar transitions in a thread's + state ("valence") occur when it enters of exits an exception + handler (which is sort of an OS/hardware-mandated foreign + function call where the OS thoughtfully saves the thread's + register state for it beforehand.) - Exception contexts, and exception-handling in general -Unix-like OSes tend to refer to exceptions as "signals"; the same -general mechanism ("signal handling") is used to process both -asynchronous OS-level events (such as the result of the keyboard driver -noticing that ^C or ^Z has been pressed) and synchronous hardware-level -events (like trying to execute and illegal instruction or access -protected memory.) It makes some sense to defer ("block") handling -of aysnchronous signals so that some critical code sequences complete -without interruption; since it's generally not possible for a thread -to proceed after a synchronous exception unless and until its state -is modified by an exception handler, it makes no sense to talk about -blocking synchronous signals (though some OSes will let you do so -and doing so can have mysterious effects.) - On OSX/Darwin, the POSIX signal handling facilities coexist with -lower-level Mach-based exception handling facilities. Unfortunately, -the way that this is implemented interacts poorly with debugging -tools: GDB will generally stop whenever the target program -encounters a Mach-level exception and offers no way to proceed from -that point (and let the program's POSIX signal handler try to handle -the exception); Apple's CrashReporter program has had a similar issue -and, depending on how it's configured, may bombard the user with -alert dialogs which falsely claim that an application has crashed -(when in fact the application in question has routinely handled -a routine exception.) On Darwin/OSX, OpenMCL uses Mach thread-level -exception handling facilities which run before GDB or CrashReporter -get a chance to confuse themeselves; OpenMCL's Mach exception handling -tries to force the thread which received a synchronous exception to -invoke a signal handling function ("as if" signal handling worked -more usefully under Darwin.) Mach exception handlers run in a dedicated -thread (which basically does nothing but wait for exception messages -from the lisp kernel, obtain and modify information about the state -of threads in which exceptions have occurred, and reply to the -exception messages with an indication that the exception has been -handled. The reply from a thread-level exception handler keeps the -exception from being reported to GDB or CrashReporter and avoids -the problems related to those programs. Since OpenMCL's Mach exception -handler doesn't claim to handle debugging-related exceptions (from -breakpoints or single-step operations), it's possible to use GDB to -debug OpenMCL. + Exception contexts, and exception-handling in general + Unix-like OSes tend to refer to exceptions as "signals"; + the same general mechanism ("signal handling") is used to + process both asynchronous OS-level events (such as the result + of the keyboard driver noticing that ^C or ^Z has been + pressed) and synchronous hardware-level events (like trying to + execute and illegal instruction or access protected memory.) + It makes some sense to defer ("block") handling of + aysnchronous signals so that some critical code sequences + complete without interruption; since it's generally not + possible for a thread to proceed after a synchronous exception + unless and until its state is modified by an exception + handler, it makes no sense to talk about blocking synchronous + signals (though some OSes will let you do so and doing so can + have mysterious effects.) + On OSX/Darwin, the POSIX signal handling facilities + coexist with lower-level Mach-based exception handling + facilities. Unfortunately, the way that this is implemented + interacts poorly with debugging tools: GDB will generally stop + whenever the target program encounters a Mach-level exception + and offers no way to proceed from that point (and let the + program's POSIX signal handler try to handle the exception); + Apple's CrashReporter program has had a similar issue and, + depending on how it's configured, may bombard the user with + alert dialogs which falsely claim that an application has + crashed (when in fact the application in question has + routinely handled a routine exception.) On Darwin/OSX, + OpenMCL uses Mach thread-level exception handling facilities + which run before GDB or CrashReporter get a chance to confuse + themeselves; OpenMCL's Mach exception handling tries to force + the thread which received a synchronous exception to invoke a + signal handling function ("as if" signal handling worked more + usefully under Darwin.) Mach exception handlers run in a + dedicated thread (which basically does nothing but wait for + exception messages from the lisp kernel, obtain and modify + information about the state of threads in which exceptions + have occurred, and reply to the exception messages with an + indication that the exception has been handled. The reply + from a thread-level exception handler keeps the exception from + being reported to GDB or CrashReporter and avoids the problems + related to those programs. Since OpenMCL's Mach exception + handler doesn't claim to handle debugging-related exceptions + (from breakpoints or single-step operations), it's possible to + use GDB to debug OpenMCL. On platforms where signal handling and debugging don't get in each other's way, a signal handler is entered with all signals blocked. @@ -9859,90 +13847,111 @@ exception, unblocks asynchronous signals, and waits for a global exception lock which serializes exception processing. - On Darwin, the Mach exception thread creates a signal context (and -maybe a siginfo_t structure), stores the signal context in the -thread's TCR, sets the TCR field wich describes the thread's state, -and arranges that the thread resume execution at its signal handling -function (with a signal handler, possibly NULL siginfo_t, and signal -context as arguments. When the thread resumes, it waits for the -global exception lock. - On x86-64 platforms where signal handing can be used to handle -synchronous exceptions, there's an additional complication: the -OS kernel ordinarily allocates the signal context and siginfo -structures on the stack of the thread which received the signal; -in practice, that means "wherever RSP is pointing." OpenMCL's - require that the -thread's value stack - where RSP is usually pointing while lisp -code is running - contain only "nodes" (properly tagged lisp -objects), and scribbling a signal context all over the value -stack would violate this requirement. To maintain consistency, -the sigaltstack() mechanism is used to cause the signal to be -delivered on (and the signal context and siginfo to be allocated -on) a special stack area (the last few pages of the thread's -cntrol stack, in practice. When the signal handler runs, it -(carefully) copies the signal context and siginfo to the thread's -control stack and makes RSP point into that stack before invoking -the "real" signal handler. (The effect of this hack is that the -"real" signal handler always runs on the thread's control stack.) - Once the exception handler has obtained the global exception lock, -it uses the values of the signal number, siginfo_t, and signal context -arguments to determine the (logical) cause of the exception. Some -exceptions may be caused by factors that should generate lisp errors -or other serious conditions (stack overflow); if this is the case, -the kernel code may release the global exception lock and call out -to lisp code. (The lisp code in question may need to repeat some -of the exception decoding process; in particular, it needs to be -able to interpret register values in the signal context that it -receives as an argument.) - In some cases, the lisp kernel exception handler may not be -able to recover from the exception (this is currently true of -some types of memory-access fault and is also true of traps -or illegal instructions that occur during foreign code execution. -In such cases, the kernel exception handler reports the -exception as "unhandled", and the kernel debugger is invoked. - If the kernel exception handler identifies the exception' cause -as being a transient out-of-memory condition (indicating that -the current thread needs more memory to cons in), it tries to -make that memory available. In some cases, doing so involves -invoking the GC. + On Darwin, the Mach exception thread creates a signal + context (and maybe a siginfo_t structure), stores the signal + context in the thread's TCR, sets the TCR field wich describes + the thread's state, and arranges that the thread resume + execution at its signal handling function (with a signal + handler, possibly NULL siginfo_t, and signal context as + arguments. When the thread resumes, it waits for the global + exception lock. + On x86-64 platforms where signal handing can be used to + handle synchronous exceptions, there's an additional + complication: the OS kernel ordinarily allocates the signal + context and siginfo structures on the stack of the thread + which received the signal; in practice, that means "wherever + RSP is pointing." OpenMCL's require that the thread's value + stack - where RSP is usually pointing while lisp code is + running - contain only "nodes" (properly tagged lisp objects), + and scribbling a signal context all over the value stack would + violate this requirement. To maintain consistency, the + sigaltstack() mechanism is used to cause the signal to be + delivered on (and the signal context and siginfo to be + allocated on) a special stack area (the last few pages of the + thread's cntrol stack, in practice. When the signal handler + runs, it (carefully) copies the signal context and siginfo to + the thread's control stack and makes RSP point into that stack + before invoking the "real" signal handler. (The effect of + this hack is that the "real" signal handler always runs on the + thread's control stack.) + Once the exception handler has obtained the global + exception lock, it uses the values of the signal number, + siginfo_t, and signal context arguments to determine the + (logical) cause of the exception. Some exceptions may be + caused by factors that should generate lisp errors or other + serious conditions (stack overflow); if this is the case, the + kernel code may release the global exception lock and call out + to lisp code. (The lisp code in question may need to repeat + some of the exception decoding process; in particular, it + needs to be able to interpret register values in the signal + context that it receives as an argument.) + In some cases, the lisp kernel exception handler may not + be able to recover from the exception (this is currently true + of some types of memory-access fault and is also true of traps + or illegal instructions that occur during foreign code + execution. In such cases, the kernel exception handler + reports the exception as "unhandled", and the kernel debugger + is invoked. + If the kernel exception handler identifies the + exception' cause as being a transient out-of-memory condition + (indicating that the current thread needs more memory to cons + in), it tries to make that memory available. In some cases, + doing so involves invoking the GC. - Threads, exceptions, and the GC -OpenMCL's GC is not concurrent: when the GC is invoked in response to an -exception in a particular thread, all other lisp threads must stop until -the GC's work is done. The thread that triggered the GC iterates over -the global TCR list, sending each other thread a distinguished "suspend" -signal, then iterates over the list again, waiting for a per-thread -semaphore that indicates that the thread has received the "suspend" -signal and responded appropriatedly. Once all other threads have -acknowledged the request to suspend themselves, the GC thread can -run the GC proper (after doing any necessary .) Once -the GC's completed its work, the thread that invoked the GC iterates -over the global TCR list, raising a per-thread "resume" semaphore -for each other thread. - The signal handler for the asynchronous "suspend" signal is entered -with all asynchronous signals blocked. It saves its signal-context -argument in a TCR slot, raises the tcr's "suspend" semaphore, then -waits on the TCR's "resume" semaphore. - The GC thread has access to the signal contexts of all TCRs (including -its own) at the time when the thread received an exception or -acknowledged a request to suspend itself. This information (and -information about stack areas in the TCR itself) allows the GC to -identify the "stack locations and register contents" that are elements -of the GC's root set. + Threads, exceptions, and the GC + OpenMCL's GC is not concurrent: when the GC is invoked + in response to an exception in a particular thread, all other + lisp threads must stop until the GC's work is done. The + thread that triggered the GC iterates over the global TCR + list, sending each other thread a distinguished "suspend" + signal, then iterates over the list again, waiting for a + per-thread semaphore that indicates that the thread has + received the "suspend" signal and responded appropriatedly. + Once all other threads have acknowledged the request to + suspend themselves, the GC thread can run the GC proper (after + doing any necessary .) Once the GC's completed its work, the + thread that invoked the GC iterates over the global TCR list, + raising a per-thread "resume" semaphore for each other + thread. + The signal handler for the asynchronous "suspend" signal + is entered with all asynchronous signals blocked. It saves + its signal-context argument in a TCR slot, raises the tcr's + "suspend" semaphore, then waits on the TCR's "resume" + semaphore. + The GC thread has access to the signal contexts of all + TCRs (including its own) at the time when the thread received + an exception or acknowledged a request to suspend itself. + This information (and information about stack areas in the TCR + itself) allows the GC to identify the "stack locations and + register contents" that are elements of the GC's root + set. - PC-luseringIt's not quite accurate to say that OpenMCL's compiler and runtime -follow precise stack and register usage conventions at all times; there -are a few exceptions: - - On both PPC and x86-64 platforms, consing isn't fully atomic.It takes at least a few instructions to allocate an object in memory(and slap a header on it if necesssary); if a thread is interrupted inthe middle of that instruction sequence, the new object may or may nothave been created or fully initialized at the point in time that theinterrupt occurred. (There are actually a few different states ofpartial initialization) - On the PPC, the common act of building a lisp control stack frameinvolves allocating a four-word frame and storing three register valuesinto that frame. (The fourth word - the back pointer to the previousframe - is automatically set when the frame is allocated.) The previouscontents of those three words are unknown (there might have been aforeign stack frame at the same address a few instructions earlier),so interrupting a thread that's in the process of initializing aPPC control stack frame isn't GC-safe. - There are similar problems with the initialization of temp stackframes on the PPC. (Allocation and initialization doesn't happenatomically, and the newly allocated stack memory may have undefinedcontents.) - 's write barrier has to be implemented atomically (i.e.,both an intergenerational store and the update of a correspondingreference bit has to happen without interruption, or neither of theseevents can happen.) - There are a few more similar cases. + PC-lusering + It's not quite accurate to say that OpenMCL's compiler + and runtime follow precise stack and register usage + conventions at all times; there are a few exceptions: + + + +On both PPC and x86-64 platforms, consing isn't fully atomic.It takes at least a few instructions to allocate an object in memory(and slap a header on it if necesssary); if a thread is interrupted inthe middle of that instruction sequence, the new object may or may nothave been created or fully initialized at the point in time that theinterrupt occurred. (There are actually a few different states ofpartial initialization) + + +On the PPC, the common act of building a lisp control stack frameinvolves allocating a four-word frame and storing three register valuesinto that frame. (The fourth word - the back pointer to the previousframe - is automatically set when the frame is allocated.) The previouscontents of those three words are unknown (there might have been aforeign stack frame at the same address a few instructions earlier),so interrupting a thread that's in the process of initializing aPPC control stack frame isn't GC-safe. + + +There are similar problems with the initialization of temp stackframes on the PPC. (Allocation and initialization doesn't happenatomically, and the newly allocated stack memory may have undefinedcontents.) + + +'s write barrier has to be implemented atomically (i.e.,both an intergenerational store and the update of a correspondingreference bit has to happen without interruption, or neither of theseevents can happen.) + + +There are a few more similar cases. + - + + Fortunately, the number of these non-atomic instruction sequences is small, and fortunately it's fairly easy for the interrupting thread @@ -9962,285 +13971,619 @@ - Register usage and tagging + Register usage and tagging - Register usage and tagging overview -Regardless of other details of its implementation, a garbage -collector's job is to partition the set of all heap-allocated -lisp objects (CONSes, STRINGs, INSTANCEs, etc.) into two subsets. -The first subset contains all objects that are transitively referenced -from a small set of "root" objects (the contents of the stacks -and registers of all active threads at the time the GC occurs -and the values of some global variables.) The second subset contains -everything else: those lisp objects that are not transitively -reachable from the roots are garbage, and the memory occupied -by garbage objects can be reclaimed (since the GC has just proven -that it's impossible to reference them.) - The set of live, reachable lisp objects basically form the nodes -of a (usually large) graph, with edges from each node A to any other -objects (nodes) that object A references. - Some nodes in this graph can never have outgoing edges: an array with a -specialized numeric or character type usually represents its elements in -some (possibly more compact) specialized way. Some nodes may refer -to lisp objects that are never allocated in memory (FIXNUMs, CHARACTERs, -SINGLE-FLOATs on 64-bit platforms ..) This latter class of objects -are sometimes called "immediates", but that's a little confusing -because the term "immediate" is sometimes used to refer to things -that can never be part of the big connectivity graph (e.g., the "raw" -bits that make up a floating-point value, foreign address, or -numeric value that needs to be used - at least fleetingly - in -compiled code.) - For the GC to be able to build the connectivity graph reliably, it's -necessary for it to be able to reliably tell (a) whether or not a -"potential root" - the contents of a machine register or stack location -- is in fact a node and (b) for any node, whether it may have components -that refer to other nodes. - There's no reliable way to answer the first question on stock hardware. -(If everything was a node, as might be the case on specially microcoded -"lisp machine" hardware, it wouldn't even need to be asked.) Since -there's no way to just look at a machine word (the contents of a machine -register or stack location) and tell whether or not it's a node or just -some random non-node value, we have to either adopt and enforce strict -conventions on register and stack usage or tolerate ambiguity. - "Tolerating ambiguity" is an approach taken by some ("conservative") GC -schemes; by contrast, OpenMCL's GC is "precise", which in this case -means that it believes that the contents of certain machine registers -and stack locations are always nodes and that other registers and stack -locations are never nodes and that these conventions are never violated -by the compiler or runtime system. The fact that threads are -preemptively scheduled means that a GC could occur (because of activity -in some other thread) on any instruction boundary, which in turn means -that the compiler and runtime system must follow precise at all times. - Once we've decided that a given machine word is a node, a describes how the node's value and type are encoded in that -machine word. - Most of this - so far - has discussed thigs from the GC's very low-level -perspective. From a much higher point of view, lisp functions accept -nodes as arguments, return nodes as values, and (usually) perform -some operations on those arguments in order to produce those results. -(In many cases, the operations in question involve raw non-node values.) -Higher-level parts of the lisp type system (functions like TYPE-OF -and CLASS-OF, etc.) depend on the . + Overview + Regardless of other details of its implementation, a + garbage collector's job is to partition the set of all + heap-allocated lisp objects (CONSes, STRINGs, INSTANCEs, etc.) + into two subsets. The first subset contains all objects that + are transitively referenced from a small set of "root" objects + (the contents of the stacks and registers of all active + threads at the time the GC occurs and the values of some + global variables.) The second subset contains everything + else: those lisp objects that are not transitively reachable + from the roots are garbage, and the memory occupied by garbage + objects can be reclaimed (since the GC has just proven that + it's impossible to reference them.) + The set of live, reachable lisp objects basically form + the nodes of a (usually large) graph, with edges from each + node A to any other objects (nodes) that object A + references. + Some nodes in this graph can never have outgoing edges: + an array with a specialized numeric or character type usually + represents its elements in some (possibly more compact) + specialized way. Some nodes may refer to lisp objects that + are never allocated in memory (FIXNUMs, CHARACTERs, + SINGLE-FLOATs on 64-bit platforms ..) This latter class of + objects are sometimes called "immediates", but that's a little + confusing because the term "immediate" is sometimes used to + refer to things that can never be part of the big connectivity + graph (e.g., the "raw" bits that make up a floating-point + value, foreign address, or numeric value that needs to be used + - at least fleetingly - in compiled code.) + For the GC to be able to build the connectivity graph + reliably, it's necessary for it to be able to reliably tell + (a) whether or not a "potential root" - the contents of a + machine register or stack location - is in fact a node and (b) + for any node, whether it may have components that refer to + other nodes. + There's no reliable way to answer the first question on + stock hardware. (If everything was a node, as might be the + case on specially microcoded "lisp machine" hardware, it + wouldn't even need to be asked.) Since there's no way to just + look at a machine word (the contents of a machine register or + stack location) and tell whether or not it's a node or just + some random non-node value, we have to either adopt and + enforce strict conventions on register and stack usage or + tolerate ambiguity. + "Tolerating ambiguity" is an approach taken by some + ("conservative") GC schemes; by contrast, OpenMCL's GC is + "precise", which in this case means that it believes that the + contents of certain machine registers and stack locations are + always nodes and that other registers and stack locations are + never nodes and that these conventions are never violated by + the compiler or runtime system. The fact that threads are + preemptively scheduled means that a GC could occur (because of + activity in some other thread) on any instruction boundary, + which in turn means that the compiler and runtime system must + follow precise at all times. + Once we've decided that a given machine word is a node, + a describes how the node's value and type are encoded in that + machine word. + Most of this - so far - has discussed thigs from the + GC's very low-level perspective. From a much higher point of + view, lisp functions accept nodes as arguments, return nodes + as values, and (usually) perform some operations on those + arguments in order to produce those results. (In many cases, + the operations in question involve raw non-node values.) + Higher-level parts of the lisp type system (functions like + TYPE-OF and CLASS-OF, etc.) depend on the . - pc-locatives on the PPC -On the PPC, there's a third case (besides "node" and "immediate" values). -As discussed below, a node that denotes a memory-allocated lisp object -is a biased (tagged) pointer -to- that object; it's not generally possible -to point -into- some composite (multi-element) object (such a pointer -would not be a node, and the GC would have no way to update the pointer -if it were to move the underlying object.) - Such a pointer ("into" the interior of a heap-allocated object) is -often called a locative; the cases where locatives are allowed -in OpenMCL mostly involve the behavior of function call and return -instructions. (To be technicaly accurate, the other case also arises -on x86-64, but that case isn't as user-visible.) - On the PowerPC (both PPC32 and PPC64), all machine instructions are -32 bits wide and all instruction words are allocated on 32-bit boundaries. -In PPC OpenMCL, a CODE-VECTOR is a specialized type of vector-like object; -its elements are 32-bit PPC machine instructions. A CODE-VECTOR is -an attribute of FUNCTION object; a function call involves accessing the -function's code-vector and jumping to the address of its first instruction. - As each instruction in the code vector sequentially executes, the -hardware program counter (PC) register advances to the address of the -next instruction (a locative into the code vector); since PPC -instructions are always 32 bits wide and aligned on 32-bit boundaries, -the low two bits of the PC are always 0. If the function executes a -call (simple call instrucions have the mnemonic "bl" on the PPC, which -stands for "branch and link"), the address of the next instruction (also -a word-aligned locative into a code-vector) is copied into the special- -purpose PPC "link register" (lr); a function returns to its caller -via a "branch to link register" (blr) instruction. Some cases of -function call and return might also use the PPC's "count register" -(ctr), and if either the lr or ctr needs to be stored in memory it -needs to first be copied to a general-purpose registers. - OpenMCL's GC understands that certain registers contain these special -"pc-locatives" (locatives that point into CODE-VECTOR objects); it -contains specal support for finding the containing CODE-VECTOR object -and for adjusting all of these "pc-locatives" if the containing object -is moved in memory. The first part of that - finding the containing -object - is possible and practical on the PPC because of architectural -artifcacts (fixed-width instructions and arcana of instruction encoding.) -It's not possible on x86-64, but fortunately not necessary either (though -the second part - adjusting the PC/RIP when the containing object moves) -is both necessary and simple. + pc-locatives on the PPC + On the PPC, there's a third case (besides "node" and + "immediate" values). As discussed below, a node that denotes + a memory-allocated lisp object is a biased (tagged) pointer + -to- that object; it's not generally possible to point -into- + some composite (multi-element) object (such a pointer would + not be a node, and the GC would have no way to update the + pointer if it were to move the underlying object.) + Such a pointer ("into" the interior of a heap-allocated + object) is often called a locative; the + cases where locatives are allowed in OpenMCL mostly involve + the behavior of function call and return instructions. (To be + technicaly accurate, the other case also arises on x86-64, but + that case isn't as user-visible.) + On the PowerPC (both PPC32 and PPC64), all machine + instructions are 32 bits wide and all in1struction words are + allocated on 32-bit boundaries. In PPC OpenMCL, a CODE-VECTOR + is a specialized type of vector-like object; its elements are + 32-bit PPC machine instructions. A CODE-VECTOR is an + attribute of FUNCTION object; a function call involves + accessing the function's code-vector and jumping to the + address of its first instruction. + As each instruction in the code vector sequentially + executes, the hardware program counter (PC) register advances + to the address of the next instruction (a locative into the + code vector); since PPC instructions are always 32 bits wide + and aligned on 32-bit boundaries, the low two bits of the PC + are always 0. If the function executes a call (simple call + instrucions have the mnemonic "bl" on the PPC, which stands + for "branch and link"), the address of the next instruction + (also a word-aligned locative into a code-vector) is copied + into the special- purpose PPC "link register" (lr); a function + returns to its caller via a "branch to link register" (blr) + instruction. Some cases of function call and return might + also use the PPC's "count register" (ctr), and if either the + lr or ctr needs to be stored in memory it needs to first be + copied to a general-purpose registers. + OpenMCL's GC understands that certain registers contain + these special "pc-locatives" (locatives that point into + CODE-VECTOR objects); it contains specal support for finding + the containing CODE-VECTOR object and for adjusting all of + these "pc-locatives" if the containing object is moved in + memory. The first part of that - finding the containing + object - is possible and practical on the PPC because of + architectural artifcacts (fixed-width instructions and arcana + of instruction encoding.) It's not possible on x86-64, but + fortunately not necessary either (though the second part - + adjusting the PC/RIP when the containing object moves) is both + necessary and simple. - Register and stack usage conventions + Register and stack usage conventions - Stack conventions -On both PPC and X86 platforms, each lisp thread uses 3 stacks; the ways -in which these stacks are used differs between the PPC and X86. + Stack conventions + On both PPC and X86 platforms, each lisp thread uses 3 + stacks; the ways in which these stacks are used differs + between the PPC and X86. Each thread has: - - A "control stack".On both platforms, this is "the stack" used by foreign code.On the PPC, it consists of a linked list of frameswhere the first word in each frame points to the first word in theprevious frame (and the outermost frame points to 0.) Some frameson a PPC control stack are lisp frames; lisp frames are always 4 wordsin size and contain (in addition to the back pointer to the previousframe) the calling function (a node), the return address (a "locative"into the calling function's code-vector), and the value to which thevalue-stack pointer (see below) should be restored on function exit.On the PPC, the GC has to look at control-stack frames, identifywhich of those frames are lisp frames, and treat the contents ofthe saved function slot as a node (and handle the return addresslocative specially.)On x86-64, the control stack is used for dynamic-extent allocationof immediate objects. Since the control stack never contains nodeson x86-64, the GC ignores it on that platform.Alignment of the control stack follows the ABI conventions of theplatform (at least at any point in time where foreign code could run.)On PPC, the r1 register always points to the top of the current thread'scontrol stack; on x86-64, the RSP register points to the top of thecurrent thread's control stack when the thread is running foreigncode and the address of the top of the control stack is kept in thethread's TCR see when not running foreigncode.The control stack "grows down." - A "value stack".On both platforms, all values on the value stack are nodes (including"tagged return addresses" on x86-64.) The value stack is always alignedto the native word size; objects are always pushed on the value stackusing atomic instructions ("stwu"/"stdu" on PPC, "push" on x86-64), sothe contents of the value stack between its bottom and top are alwaysunambiguously nodes; the compiler usually tries to pop or discardnodes from the value stack as soon as possible after their last use(as soon as they may have become garbage.)On x86-64, the RSP register addresses the top of the value stackwhen running lisp code; that address is saved in the TCR whenrunning foreign code.On the PPC, a dedicated regiter (VSP, currently r15) is used toaddress the top of the value stack when running lisp code, and theVSP value is saved in the TCR when running foreign code.The value stack grows down. - A "temp stack".The temp stack consists of a linked list of frames, each of which pointsto the previous temp stack frame. The number of native machine wordsin each temp stack frame is always even, so the temp stack is alignedon a two-word (64- or 128-bit) boundary.The temp stack is used for dynamic-extent objects on both platforms;on the PPC, it's used for essentially all such objects (regardlessof whether or not the objects contain nodes); on the x86-64, immediatedynamic-extent objects (strings, foreign pointers, etc.) are allocatedon the control stack and only node-containing dynamic-extent objectsare allocated on the temp stack.Data structures used to implement CATCH and UNWIND-PROTECT are stored onthe temp stack on both ppc and x86-64.Temp stack frames are always doublenode aligned and objects withina temp stack frame are aligned on doublenode boundaries. The firstword in each frame contains a back pointer to the previous frame; onthe PPC, the second word is used to indicate to the GC whethe theremaining objects are nodes (if the second word is 0) or immediate(otherwise.) On x86-64, where temp stack frames always contain nodes,the second word is always 0.The temp stack grows down. It usually takes several instuctions toallocate and safely initialize a temp stack frame that's intended tocontain nodes, and the GC has to recognize the case where a threadis in the process of allocating and initializing a temp stack frameand take care not to interpret any uninitialized words in the frameas nodes. See .The PPC keeps the current top of the temp stack in a dedicated register(TSP, currently r12) when running lisp code and saves this register'svalue in the TCR when running foreign code. The x86-64 keeps theaddress of the top of each thread's temp stack in the thread's TCR. - - + + + A "control stack".On both platforms, this is "the + stack" used by foreign code.On the PPC, it consists of a + linked list of frameswhere the first word in each frame + points to the first word in theprevious frame (and the + outermost frame points to 0.) Some frameson a PPC + control stack are lisp frames; lisp frames are always 4 + wordsin size and contain (in addition to the back + pointer to the previousframe) the calling function (a + node), the return address (a "locative"into the calling + function's code-vector), and the value to which + thevalue-stack pointer (see below) should be restored on + function exit.On the PPC, the GC has to look at + control-stack frames, identifywhich of those frames are + lisp frames, and treat the contents ofthe saved function + slot as a node (and handle the return addresslocative + specially.)On x86-64, the control stack is used for + dynamic-extent allocationof immediate objects. Since + the control stack never contains nodeson x86-64, the GC + ignores it on that platform.Alignment of the control + stack follows the ABI conventions of theplatform (at + least at any point in time where foreign code could + run.)On PPC, the r1 register always points to the top of + the current thread'scontrol stack; on x86-64, the RSP + register points to the top of thecurrent thread's + control stack when the thread is running foreigncode and + the address of the top of the control stack is kept in + thethread's TCR see when not running foreigncode.The + control stack "grows down." + + + A "value stack".On both platforms, all values on + the value stack are nodes (including"tagged return + addresses" on x86-64.) The value stack is always + alignedto the native word size; objects are always + pushed on the value stackusing atomic instructions + ("stwu"/"stdu" on PPC, "push" on x86-64), sothe contents + of the value stack between its bottom and top are + alwaysunambiguously nodes; the compiler usually tries to + pop or discardnodes from the value stack as soon as + possible after their last use(as soon as they may have + become garbage.)On x86-64, the RSP register addresses + the top of the value stackwhen running lisp code; that + address is saved in the TCR whenrunning foreign code.On + the PPC, a dedicated regiter (VSP, currently r15) is + used toaddress the top of the value stack when running + lisp code, and theVSP value is saved in the TCR when + running foreign code.The value stack grows down. + + + A "temp stack".The temp stack consists of a linked + list of frames, each of which pointsto the previous temp + stack frame. The number of native machine wordsin each + temp stack frame is always even, so the temp stack is + alignedon a two-word (64- or 128-bit) boundary.The temp + stack is used for dynamic-extent objects on both + platforms;on the PPC, it's used for essentially all such + objects (regardlessof whether or not the objects contain + nodes); on the x86-64, immediatedynamic-extent objects + (strings, foreign pointers, etc.) are allocatedon the + control stack and only node-containing dynamic-extent + objectsare allocated on the temp stack.Data structures + used to implement CATCH and UNWIND-PROTECT are stored + onthe temp stack on both ppc and x86-64.Temp stack + frames are always doublenode aligned and objects withina + temp stack frame are aligned on doublenode boundaries. + The firstword in each frame contains a back pointer to + the previous frame; onthe PPC, the second word is used + to indicate to the GC whethe theremaining objects are + nodes (if the second word is 0) or immediate(otherwise.) + On x86-64, where temp stack frames always contain + nodes,the second word is always 0.The temp stack grows + down. It usually takes several instuctions toallocate + and safely initialize a temp stack frame that's intended + tocontain nodes, and the GC has to recognize the case + where a threadis in the process of allocating and + initializing a temp stack frameand take care not to + interpret any uninitialized words in the frameas nodes. + See .The PPC keeps the current top of the temp stack in + a dedicated register(TSP, currently r12) when running + lisp code and saves this register'svalue in the TCR when + running foreign code. The x86-64 keeps theaddress of + the top of each thread's temp stack in the thread's + TCR. + + - Register conventions -If there are a "reasonable" (for some value of "reasonable") number or -general-purpose registers and the instruction set is "reasonably" -orthogonal (most instructions that operate on GPRs can operate on any -GPR), then it's possible to statically partition the GPRs into at least -two sets: "immediate registers" never contain nodes, and "node registers" -always contain nodes. (On the PPC, a few registers are members of a -third set of "PC locatives", and on both platforms some registers may -have dedicated roles as stack or heap pointers; the latter class is -treated as immediates by the GC proper but may be used to help determine -the bounds of stack and heap memory areas.) - The ultimate definition of register partitioning is hardwired -into the GC in functions like "mark_xp()" and "forward_xp()", which -process the values of some of the registers in an exception frame -as nodes and may give some sort of special treatment to other register -values they encounter there.) + Register conventions + If there are a "reasonable" (for some value of + "reasonable") number or general-purpose registers and the + instruction set is "reasonably" orthogonal (most + instructions that operate on GPRs can operate on any GPR), + then it's possible to statically partition the GPRs into at + least two sets: "immediate registers" never contain nodes, + and "node registers" always contain nodes. (On the PPC, a + few registers are members of a third set of "PC locatives", + and on both platforms some registers may have dedicated + roles as stack or heap pointers; the latter class is treated + as immediates by the GC proper but may be used to help + determine the bounds of stack and heap memory areas.) + The ultimate definition of register partitioning is + hardwired into the GC in functions like "mark_xp()" and + "forward_xp()", which process the values of some of the + registers in an exception frame as nodes and may give some + sort of special treatment to other register values they + encounter there.) On x86-64, the static register partitioning scheme involves: - - (only) two "immediate" registers.The RAX and RDX registers are used as the implicit operands andresults of some extended-precision multiply and divide instructionswhich generally involve non-node values; since their use in theseinstructions means that they can't be guaranteed to contain nodevalues at all times, it's natural to put these registers in the"immediate" set. RAX is generally given the symbolic name "imm0",and RDX is given the symbolic name "imm1"; you may see these namesin disassembled code, usually in operations involving type checking,array indexing, and foreign memory and function access. - (only) two "dedicated" registers.RSP and RBP have dedicated functionality dictated by the hardwareand calling conventions. (There are a few places where RBP istemporarily used as an extra immediate register.) - 12 "node" registers.All other registers (RBX, RCX, RSI, RDI, and R8-R15) are asserted tocontain node values at (almost) all times; legacy "string" operationsthat implicitly use RSI and/or RDI are not used. Shift and rotateintructions which shift/rotate by a variable number of bits arerequired by the architecture to use the low byte of RCX (the traditionalCL register) as the implicit shift count; when it's necessary to keepa non-node shift count in the low byte of RCX, the upper 7 bytes ofthe register are zeroed (so that misinterpetation of the immediatevalue in RCX as a node will not have negative GC affects. (The GCmight briefly treate it as a node, but since it's not pointing anywherenear the lisp heap it'll soon lose interest in it.)Legacy instructions that use RCX (or some portions of it) as a loopcounter can not be used (since such instructions might introducenon-node values into RCX.) - - + + + (only) two "immediate" registers.The RAX and RDX + registers are used as the implicit operands andresults + of some extended-precision multiply and divide + instructionswhich generally involve non-node values; + since their use in theseinstructions means that they + can't be guaranteed to contain nodevalues at all times, + it's natural to put these registers in the"immediate" + set. RAX is generally given the symbolic name + "imm0",and RDX is given the symbolic name "imm1"; you + may see these namesin disassembled code, usually in + operations involving type checking,array indexing, and + foreign memory and function access. + + + (only) two "dedicated" registers.RSP and RBP have + dedicated functionality dictated by the hardwareand + calling conventions. (There are a few places where RBP + istemporarily used as an extra immediate + register.) + + + 12 "node" registers.All other registers (RBX, RCX, + RSI, RDI, and R8-R15) are asserted tocontain node values + at (almost) all times; legacy "string" operationsthat + implicitly use RSI and/or RDI are not used. Shift and + rotateintructions which shift/rotate by a variable + number of bits arerequired by the architecture to use + the low byte of RCX (the traditionalCL register) as the + implicit shift count; when it's necessary to keepa + non-node shift count in the low byte of RCX, the upper 7 + bytes ofthe register are zeroed (so that + misinterpetation of the immediatevalue in RCX as a node + will not have negative GC affects. (The GCmight briefly + treate it as a node, but since it's not pointing + anywherenear the lisp heap it'll soon lose interest in + it.)Legacy instructions that use RCX (or some portions + of it) as a loopcounter can not be used (since such + instructions might introducenon-node values into + RCX.) + + On the PPC, the static register partitioning scheme involves: - - 6 "immediate" registersRegisters r3-r8 are given the symbolic names imm0-imm5. As a RISCarchitecture with simpler addressing modes, the PPC probably usesimmediate registers a bit more often than the CISC x86-64 does, butthey're generally used for the same sort of things (type checking,array indexing, FFI, etc.) - 9 dedicated registers - - r0 (symbolic name rzero) always contains the value 0 when runninglisp code. Its value is sometimes read as 0 when it's used as thebase register in a memory address; keeping the value 0 there issometimes convenient and avoids asymmetry. - r1 (symbolic name sp) is the control stack pointer, by PPC convention. - r2 is used to hold the current thread's TCR on ppc64 systems; it'snot used on ppc32. - r9 and r10 (symbolic names allocptr and allocbase) are used to do per-thread memory allocation - r11 (symbolic name nargs) contains the number of function arguments on entry and the number of return values in multiple-value returning constructs. It's not used more generally as either a node or immediate register because of the way that certain trap instruction encodings are interpreted. - r12 (symbolic name tsp) holds the top of the current thread's temp stack. - r13 is used to hold the TCR on PPC32 sytems; it's not used on PPC64. - r14 (symbolic name loc-pc) is used to copy "pc-locative" values between main memory and special-purpose PPC registers (LR and CTR) used in function-call and return instructions. - r15 (symbolic name vsp) addresses the top of the current thread's value stack. - lr and ctr are PPC branch-unit registers used in function call and return instructions; they're always treated as "pc-locatives", which precludes the use of the ctr in some PPC looping constructs. + + + + 6 "immediate" registersRegisters r3-r8 are given + the symbolic names imm0-imm5. As a RISCarchitecture + with simpler addressing modes, the PPC probably + usesimmediate registers a bit more often than the CISC + x86-64 does, butthey're generally used for the same sort + of things (type checking,array indexing, FFI, + etc.) + + + 9 dedicated registers + + + r0 (symbolic name rzero) always contains the + value 0 when runninglisp code. Its value is + sometimes read as 0 when it's used as thebase + register in a memory address; keeping the value 0 + there issometimes convenient and avoids + asymmetry. + + + r1 (symbolic name sp) is the control stack + pointer, by PPC convention. + + + r2 is used to hold the current thread's TCR on + ppc64 systems; it'snot used on ppc32. + + + r9 and r10 (symbolic names allocptr and + allocbase) are used to do per-thread memory + allocation + + + r11 (symbolic name nargs) contains the number + of function arguments on entry and the number of + return values in multiple-value returning + constructs. It's not used more generally as either + a node or immediate register because of the way that + certain trap instruction encodings are + interpreted. + + + r12 (symbolic name tsp) holds the top of the current thread's temp stack. + + + r13 is used to hold the TCR on PPC32 sytems; it's not used on PPC64. + + + r14 (symbolic name loc-pc) is used to copy + "pc-locative" values between main memory and + special-purpose PPC registers (LR and CTR) used in + function-call and return instructions. + + + r15 (symbolic name vsp) addresses the top of + the current thread's value stack. + + + lr and ctr are PPC branch-unit registers used + in function call and return instructions; they're + always treated as "pc-locatives", which precludes + the use of the ctr in some PPC looping + constructs. + - - - 17 "node" registersr15-r31 are always treated as node registers - - + + + + + 17 "node" registersr15-r31 are always treated as + node registers + + + - Tagging schemeOpenMCL always allocates lisp objects on double-node (64-bit for 32-bit -platforms, 128-bit for 64-bit platforms) boundaries; this mean that the -low 3 bits (32-bit lisp) or 4 bits (64-bit lisp) are always 0 and are -therefore redundant (we only really need to know the upper 29 or 60 bits -in order to identify the aligned object address.) The extra bits in a -lisp node can be used to encode at least some information about the -node's type, and the other 29/60 bits represent either an immediate -value or a doublenode-aligned memory address. The low 3 or 4 bits of a -node are called the node's "tag bits", and the conventions used to -encode type information in those tag bits are called a "tagging scheme." - It might be possible to use the same tagging scheme on all platforms (at -least on all platforms with the same word size and/or the same number of -available tag bits), but there are often some strong reasons for not -doing so. These arguments tend to be very machine-specific: sometimes, -there are fairly obvious machine-dependent tricks that can be exploited -to make common operations on some types of tagged objects faster; other -times, there are architectural restrictions that make it impractical to -use certain tags for certain types. (On PPC64, the "ld" (load -doubleword) and "std" (store doubleword) instructions - which load and -store a GPR operand at the effective address formed by adding the value -of another GPR operand and a 16-bit constant operand - require that the -low two bits of that constant operand be 0. Since such instructions -would typically be used to access the fields of things like CONS cells -and structures, it's desirable that that the tags chosen for CONS cells -and structures allow the use of these intructions as opposed to more -expensive alternatives.) - One architecture-dependent tagging trick that works well on all -architectures is to use a tag of 0 for FIXNUMs: a fixnum basically -encodes its value shifted left a few bits and keeps those low bits -clear. FIXNUM addition, subtraction, and binary logical operations can -operate directly on the node operands, addition and subtraction can -exploit hardware-based overflow detection, and (in the absence of -overflow) the hardware result of those operations is a node (fixnum). -Some other slightly-less-common operations may require a few extra -instructions, but arithmetic operations on FIXNUMs should be as -cheap as possible and using a tag of zero for FIXNUMs helps to -ensure that it will be. - If we have N available tag bits (N = 3 for 32-bit OpenMCL and N = 4 -for 64-bit OpenMCL), this way of representing fixnums with the low -M bits forced to 0 works as long as M <= N. The smaller we make -M, the larger the values of MOST-POSITIVE-FIXNUM and MOST-NEGATIVE -become; the larger we make N, the more distinct non-FIXNUM tags -become available. A reasonable compromise is to choose M = N-1; -this basically yields two distinct FIXNUM tags (one for even -fixnums, one for odd fixnums), gives 30-bit fixnums on 32-bit -platforms and 61-bit fixnums on 64-bit platforms, and leaves us -with 6 or 14 tags to encoded other types. - Once we get past the assignment of FIXNUM tags, things quickly devolve -into machine-dependencies. We can fairly easily see that we can't -directly all other primitive lisp object types with only 6 or 14 -available tag values; the details of how types are encoded vary between -the ppc32, ppc64, and x86-64 implementations, but there are some general -common principles: - - CONS cells always contain exactly 2 elements and are usually fairly common.It therefore makes sense to give CONS cells their own tag. Unlike thefixnum case - where a tag value of 0 had positive implications - theredoesn't seem to be any advantage to using any particular value. (A longtime ago - in the case of 68K MCL - the CONS tag and the order of CARand CDR in memory were chosen to allow smaller, cheaper addressing modesto be used to "cdr down a list." That's not a factor on ppc or x86-64,but all versions of OpenMCL still store the CDR of a CONS cell first inmemory. It doesn't matter, but doing it the way that the host systemdid made boostrapping to a new target system a little easier.) - Any way you look at it, NIL is a bit ... unusual.NIL is both a SYMBOL and a LIST (as well as being a canonical truthvalue and probably a few other things.) Its role as a LIST is probablymuch more important to most programs than its role as a SYMBOL is:LISTP has to be true of NIL and primitives like CAR and CDR do LISTPimplicitly when safe and want that operation to be fast.There are several possible approaches to this; OpenMCL uses two of them.On PPC32 and X86-64, NIL is basically a weird CONS cell that straddlestwo doublenodes; the tag of NIL is unique and congruent modulo 4 (modulo8 on 64-bit) with the tag used for CONS cells. LISTP is thereforetrue of any node whose low 2 (or 3) bits contain the appropriate tagvalue (it's not otherwise necessary to special-case NIL.) SYMBOLaccessors (SYMBOL-NAME, SYMBOL-VALUE, SYMBOL-PLIST ..) -do- haveto special-case NIL (and access the components of an internal proxysymbol.)On PPC64 (where architectural restrictions dictate the set of tagsthat can be used to access fixed components of an object), thatapproach wasn't practical. NIL is just a distinguished SYMBOL,and it just happens to be the case that its pname slot and valueslots are at the same offsets from a tagged pointer as a CONS cellsCDR and CAR would be. NIL's pname is set to NIL (SYMBOL-NAMEchecks for this and returns the string "NIL"), and LISTP (andtherefore safe CAR and CDR) have to check for (OR NULL CONSP).At least in the case of CAR and CDR, the fact that the PPC hasmultiple condition-code fields keeps that extra test from beingprohibitively expensive. - Some objects are immediate.(but not FIXNUMs).This is true of CHARACTERs and, on 64-bit platforms, SINGLE-FLOATs.It's also true of some nodes used in the runtime system (specialvalues used to indicate unbound variables and slots, for instance.)On 64-bit platforms, SINGLE-FLOATs have their own unique tag (makingthem a little easier to recognize; on all platforms, CHARACTERs sharea tag with other immediate objects (unbound markers) but are easyto recognize (by looking at several of their low bits.) The GCtreats any node with an immediate tag (and any node with a fixnumtag) as a leaf. - There are some advantages to treating everything else - memory-allocated objects that aren't CONS cells - uniformly.There are some disadvantages to that uniform treatment as well, and thetreatment of "memory-allocated non-CONS objects" isn't entirely uniformaccross all OpenMCL implementations. Let's first pretend that thetreatment is uniform, then discuss the ways in which it isn't.The "uniform approach" is to treat all memory-allocated non-CONS objectsas if they were vectors; this use of the term is a little looser thanwhat's implied by the CL VECTOR type. OpenMCL actually uses the term"uvector" to mean "a memory-allocated lisp object other than a CONS cell,whose first word is a header which describes the object's type andthe number of elements that it contains." In this view, a SYMBOL isa UVECTOR, as is a STRING, a STANDARD-INSTANCE, a CL array or vector,a FUNCTION, and even a DOUBLE-FLOAT.In the PPC implementations (where things are a little more ... uniform),a single tag value is used to denote any uvector; in order to determinesomething more specific about the type of the object in question, it'snecessary to fetch the low byte of the header word from memory. On thex86-64 platform, certain types of uvectors - SYMBOLs and FUNCTIONs -are given their own unique tags. The good news about the x86-64 approachis that SYMBOLs and FUNCTIONs can be recognized without referencingmemory; the slightly bad news is that primitive operations that workon UVECTOR-tagged objects - like the function CCL:UVREF - don't workon SYMBOLs or FUNCTIONs on x86-64 (but -do- work on those types of objectsin the PPC ports.)The header word which precedes a UVECTOR's data in memory contains 8bits of type information in the low byte and either 24 or 56 bits of"element-count" information in the rest of the word. (This is wherethe sometimes-limiting value of 2^24 for ARRAY-TOTAL-SIZE-LIMIT onPPC32 platforms comes from.) The low byte of the header - sometimescalled the uvector's subtag - is itself tagged (which means thatthe header is tagged.) The (3 or 4) tag bits in the subtag are usedto determine whether the uvector's elements are nodes or immediates.(A UVECTOR whose elements are nodes is called a GVECTOR; a UVECTORwhose elements are immediates is called an IVECTOR. This terminologycame from Spice Lisp, which was a predecessor of CMUCL.)Even though a uvector header is tagged, a header is not a node. There'sno (supported) way to get your hands on one in lisp and doing so couldbe dangerous. (If the value of a header wound up in a lisp noderegister and that register wound up getting pushed on a thread's valuestack, the GC might misinterpret that situation to mean that therewas a stack-allocated UVECTOR on the value stack.) + Tagging scheme + OpenMCL always allocates lisp objects on double-node + (64-bit for 32-bit platforms, 128-bit for 64-bit platforms) + boundaries; this mean that the low 3 bits (32-bit lisp) or 4 + bits (64-bit lisp) are always 0 and are therefore redundant + (we only really need to know the upper 29 or 60 bits in order + to identify the aligned object address.) The extra bits in a + lisp node can be used to encode at least some information + about the node's type, and the other 29/60 bits represent + either an immediate value or a doublenode-aligned memory + address. The low 3 or 4 bits of a node are called the node's + "tag bits", and the conventions used to encode type + information in those tag bits are called a "tagging + scheme." + It might be possible to use the same tagging scheme on + all platforms (at least on all platforms with the same word + size and/or the same number of available tag bits), but there + are often some strong reasons for not doing so. These + arguments tend to be very machine-specific: sometimes, there + are fairly obvious machine-dependent tricks that can be + exploited to make common operations on some types of tagged + objects faster; other times, there are architectural + restrictions that make it impractical to use certain tags for + certain types. (On PPC64, the "ld" (load doubleword) and + "std" (store doubleword) instructions - which load and store a + GPR operand at the effective address formed by adding the + value of another GPR operand and a 16-bit constant operand - + require that the low two bits of that constant operand be 0. + Since such instructions would typically be used to access the + fields of things like CONS cells and structures, it's + desirable that that the tags chosen for CONS cells and + structures allow the use of these intructions as opposed to + more expensive alternatives.) + One architecture-dependent tagging trick that works well + on all architectures is to use a tag of 0 for FIXNUMs: a + fixnum basically encodes its value shifted left a few bits and + keeps those low bits clear. FIXNUM addition, subtraction, and + binary logical operations can operate directly on the node + operands, addition and subtraction can exploit hardware-based + overflow detection, and (in the absence of overflow) the + hardware result of those operations is a node (fixnum). Some + other slightly-less-common operations may require a few extra + instructions, but arithmetic operations on FIXNUMs should be + as cheap as possible and using a tag of zero for FIXNUMs helps + to ensure that it will be. + If we have N available tag bits (N = 3 for 32-bit + OpenMCL and N = 4 for 64-bit OpenMCL), this way of + representing fixnums with the low M bits forced to 0 works as + long as M <= N. The smaller we make M, the larger the + values of MOST-POSITIVE-FIXNUM and MOST-NEGATIVE become; the + larger we make N, the more distinct non-FIXNUM tags become + available. A reasonable compromise is to choose M = N-1; this + basically yields two distinct FIXNUM tags (one for even + fixnums, one for odd fixnums), gives 30-bit fixnums on 32-bit + platforms and 61-bit fixnums on 64-bit platforms, and leaves + us with 6 or 14 tags to encoded other types. + Once we get past the assignment of FIXNUM tags, things + quickly devolve into machine-dependencies. We can fairly + easily see that we can't directly all other primitive lisp + object types with only 6 or 14 available tag values; the + details of how types are encoded vary between the ppc32, + ppc64, and x86-64 implementations, but there are some general + common principles: + + + + CONS cells always contain exactly 2 elements and are + usually fairly common.It therefore makes sense to give + CONS cells their own tag. Unlike thefixnum case - where a + tag value of 0 had positive implications - theredoesn't + seem to be any advantage to using any particular value. + (A longtime ago - in the case of 68K MCL - the CONS tag + and the order of CAR and CDR in memory were chosen to allow + smaller, cheaper addressing modes to be used to "cdr down a + list." That's not a factor on ppc or x86-64,but all + versions of OpenMCL still store the CDR of a CONS cell + first in memory. It doesn't matter, but doing it the way + that the host system did made boostrapping to a new target + system a little easier.) + + + + Any way you look at it, NIL is a bit ... unusual.NIL + is both a SYMBOL and a LIST (as well as being a canonical + truth value and probably a few other things.) Its role as + a LIST is probably much more important to most programs + than its role as a SYMBOL is:LISTP has to be true of NIL + and primitives like CAR and CDR do LISTP implicitly when + safe and want that operation to be fast.There are several + possible approaches to this; OpenMCL uses two of them. On + PPC32 and X86-64, NIL is basically a weird CONS cell that + straddles two doublenodes; the tag of NIL is unique and + congruent modulo 4 (modulo 8 on 64-bit) with the tag used + for CONS cells. LISTP is therefore true of any node whose + low 2 (or 3) bits contain the appropriate tag value (it's + not otherwise necessary to special-case NIL.) + SYMBOL accessors (SYMBOL-NAME, SYMBOL-VALUE, SYMBOL-PLIST + ..) -do- have to special-case NIL (and access the + components of an internal proxy symbol.) On PPC64 (where + architectural restrictions dictate the set of tags that can + be used to access fixed components of an object), + that approach wasn't practical. NIL is just a + distinguished SYMBOL,and it just happens to be the case + that its pname slot and values lots are at the same offsets + from a tagged pointer as a CONS cell's CDR and CAR would be. + NIL's pname is set to NIL (SYMBOL-NAME checks for this and + returns the string "NIL"), and LISTP (and therefore safe + CAR and CDR) have to check for (OR NULL CONSP).At least in + the case of CAR and CDR, the fact that the PPC has multiple + condition-code fields keeps that extra test from + being prohibitively expensive. + + + Some objects are immediate.(but not FIXNUMs).This is + true of CHARACTERs and, on 64-bit platforms, + SINGLE-FLOATs.It's also true of some nodes used in the + runtime system (specialvalues used to indicate unbound + variables and slots, for instance.)On 64-bit platforms, + SINGLE-FLOATs have their own unique tag (makingthem a + little easier to recognize; on all platforms, CHARACTERs + sharea tag with other immediate objects (unbound markers) + but are easyto recognize (by looking at several of their + low bits.) The GCtreats any node with an immediate tag + (and any node with a fixnumtag) as a leaf. + + + There are some advantages to treating everything + else - memory-allocated objects that aren't CONS cells - + uniformly.There are some disadvantages to that uniform + treatment as well, and the treatment of "memory-allocated + non-CONS objects" isn't entirely uniformaccross all + OpenMCL implementations. Let's first pretend that + the treatment is uniform, then discuss the ways in which it + isn't.The "uniform approach" is to treat all + memory-allocated non-CONS objectsas if they were vectors; + this use of the term is a little looser thanwhat's implied + by the CL VECTOR type. OpenMCL actually uses the + term"uvector" to mean "a memory-allocated lisp object + other than a CONS cell,whose first word is a header which + describes the object's type andthe number of elements that + it contains." In this view, a SYMBOL isa UVECTOR, as is a + STRING, a STANDARD-INSTANCE, a CL array or vector,a + FUNCTION, and even a DOUBLE-FLOAT.In the PPC + implementations (where things are a little more + ... uniform),a single tag value is used to denote any + uvector; in order to determinesomething more specific + about the type of the object in question, it'snecessary to + fetch the low byte of the header word from memory. On + thex86-64 platform, certain types of uvectors - SYMBOLs + and FUNCTIONs -are given their own unique tags. The good + news about the x86-64 approachis that SYMBOLs and + FUNCTIONs can be recognized without referencingmemory; the + slightly bad news is that primitive operations that workon + UVECTOR-tagged objects - like the function CCL:UVREF - + don't workon SYMBOLs or FUNCTIONs on x86-64 (but -do- work + on those types of objectsin the PPC ports.)The header word + which precedes a UVECTOR's data in memory contains 8bits + of type information in the low byte and either 24 or 56 + bits of"element-count" information in the rest of the + word. (This is wherethe sometimes-limiting value of 2^24 + for ARRAY-TOTAL-SIZE-LIMIT onPPC32 platforms comes from.) + The low byte of the header - sometimescalled the uvector's + subtag - is itself tagged (which means thatthe header is + tagged.) The (3 or 4) tag bits in the subtag are usedto + determine whether the uvector's elements are nodes or + immediates.(A UVECTOR whose elements are nodes is called a + GVECTOR; a UVECTORwhose elements are immediates is called + an IVECTOR. This terminologycame from Spice Lisp, which + was a predecessor of CMUCL.)Even though a uvector header + is tagged, a header is not a node. There'sno (supported) + way to get your hands on one in lisp and doing so couldbe + dangerous. (If the value of a header wound up in a lisp + noderegister and that register wound up getting pushed on + a thread's valuestack, the GC might misinterpret that + situation to mean that therewas a stack-allocated UVECTOR + on the value stack.) + - + - Heap Allocation -When the OpenMCL kernel first starts up, a large contiguous chunk of the -process's address space is mapped as "anonymous, no access" memory. ("Large" -means different things in different contexts; on LinuxPPC32, it means "about -1 gigabyte", on DarwinPPC32, it means "about 2 gigabytes", and on current -64-bit platforms it ranges from 128 to 512 gigabytes, depending on OS. These -values are both defaults and upper limits; the –heap-reserve argument can -be used to try to reserve less than the default.) - Reserving address space that can't (yet) be read or written to doesn't -cost much; in particular, it doesn't require that correspinding swap -space or physical memory be available. Marking the address range as being -"mapped" helps to ensure that other things (random calls to malloc(), -dynamically loaded shared libraries) won't be allocated in this region -that lisp has reserved for its own heap growth. - A small portion (around 1/32 on 32-bit platforms and 1/64 on 64-bit -platforms) of that large chunk of address space is reserved for GC -data structures. Memory pages reserved for these data structures are -mapped read-write as pages made writable in the main portion of the -heap. - The initial heap image is mapped into this reserved address space and -an additional (LISP-HEAP-GC-THRESHOLD) bytes are mapped read-write. -GC data structures grow to match the amount of GC-able memory in the -initial image + the gc threshold, and control is transferred to lisp -code. Inevitably, that code spoils everything and starts consing; -there are basically three layers of memory allocation that can go on. + Heap Allocation When the OpenMCL kernel + first starts up, a large contiguous chunk of the process's + address space is mapped as "anonymous, no access" + memory. ("Large" means different things in different contexts; + on LinuxPPC32, it means "about 1 gigabyte", on DarwinPPC32, it + means "about 2 gigabytes", and on current 64-bit platforms it + ranges from 128 to 512 gigabytes, depending on OS. These values + are both defaults and upper limits; the --heap-reserve + argument can be used to try to reserve less than the + default.) + Reserving address space that can't (yet) be read or + written to doesn't cost much; in particular, it doesn't require + that correspinding swap space or physical memory be available. + Marking the address range as being "mapped" helps to ensure that + other things (result from random calls to malloc(), dynamically + loaded shared libraries) won't be allocated in this region that + lisp has reserved for its own heap growth. + A small portion (around 1/32 on 32-bit platforms and 1/64 + on 64-bit platforms) of that large chunk of address space is + reserved for GC data structures. Memory pages reserved for + these data structures are mapped read-write as pages made + writable in the main portion of the heap. + The initial heap image is mapped into this reserved + address space and an additional (LISP-HEAP-GC-THRESHOLD) bytes + are mapped read-write. GC data structures grow to match the + amount of GC-able memory in the initial image + the gc + threshold, and control is transferred to lisp code. Inevitably, + that code spoils everything and starts consing; there are + basically three layers of memory allocation that can go + on. - Per-thread object allocationEach lisp thread has a private "reserved memory segment"; when a thread -starts up, its reserved memory segment is empty. PPC ports maintain the -highest unallocated addres and he lowest allocated address in the -current segment in registers when running lisp code; on x86-664, these -values are maintained in the current threads's TCR. (An "empty" heap -segment is one whose high pointer and low pointer are equal.) When -a thread is not in the midde of allocating something, the low 3 or -4 bits of the high and low pointers are clear (the pointers are -doublenode-aligned.) - A thread tries to allocate an object whose physical size in bytes is X -and whose tag is Y by: - - decrementing the "high" pointer by (- X Y) - trapping if the high pointer is less than the low pointer - using the (tagged) high pointer to initialize the object, if necessary - clearing the low bits of the high pointer - - On PPC32, where the size of a CONS cell is 8 bytes and the tag of a CONS -cell is 1, machine code which sets the arg_z register to the result of -doing (CONS arg_y arg_z) looks like: + Per-thread object allocation + Each lisp thread has a private "reserved memory + segment"; when a thread starts up, its reserved memory segment + is empty. PPC ports maintain the highest unallocated addres + and he lowest allocated address in the current segment in + registers when running lisp code; on x86-664, these values are + maintained in the current threads's TCR. (An "empty" heap + segment is one whose high pointer and low pointer are equal.) + When a thread is not in the midde of allocating something, the + low 3 or 4 bits of the high and low pointers are clear (the + pointers are doublenode-aligned.) + A thread tries to allocate an object whose physical size + in bytes is X and whose tag is Y by: + + + decrementing the "high" pointer by (- X Y) + + + trapping if the high pointer is less than the low + pointer + + + using the (tagged) high pointer to initialize the + object, if necessary + + + clearing the low bits of the high pointer + + + On PPC32, where the size of a CONS cell is 8 bytes and + the tag of a CONS cell is 1, machine code which sets the arg_z + register to the result of doing (CONS arg_y arg_z) looks + like: (SUBI ALLOCPTR ALLOCPTR 7) ; decrement the high pointer by (- 8 1) @@ -10251,9 +14594,10 @@ (RLWINM ALLOCPTR ALLOCPTR 0 0 28) ; clear tag bits - On x86-64, the idea's similar but the implementation is different. The -high and low pointers to the current thread's reserved segment are kept -in the TCR, which is addressed by the gs segment register. An x86-64 -CONS cell is 16 bytes wide and has a tag of 3; we canonically use the -temp0 register to initialize the object + On x86-64, the idea's similar but the implementation is + different. The high and low pointers to the current thread's + reserved segment are kept in the TCR, which is addressed by + the gs segment register. An x86-64 CONS cell is 16 bytes wide + and has a tag of 3; we canonically use the temp0 register to + initialize the object (subq ($ 13) ( (% gs) 216)) ; decrement allocptr @@ -10267,51 +14611,61 @@ (movq (% arg_z) ( -3 (% temp0))); set the cdr (movq (% temp0) (% arg_z)) ; return the cons - - If we don't take the trap (if allocating 8-16 bytes doesn't exhaust -the thread's reserved memory segment), that's a fairly short and -simple instruction sequence. If we do take the trap, we'll have to -do some additional work in order to get a new segment for the current -thread. + + If we don't take the trap (if allocating 8-16 bytes + doesn't exhaust the thread's reserved memory segment), that's + a fairly short and simple instruction sequence. If we do take + the trap, we'll have to do some additional work in order to + get a new segment for the current thread. - Allocation of reserved heap segmentsAfter lisp image is first mapped into memory - and after each full GC - the -lisp kernel ensures that (LISP-HEAP-GC-TRESHOLD) additional bytes beyond -the current end of the heap are mapped read-write. - If a thread traps while trying to allocate memory, the thread goes -through the usual exception-handling protocol (to ensure that any oher -thread that GCs "sees" the state of the trapping thread and to serialize -exception handling.) When the exception handler runs, it determines -the nature and size of the failed allocation and tries to complete the -allocation on the thread's behalf (and leave it with a reasonably large -thread-specific memory segment so that the next small allocation is -unlikely to trap. - Depending on the size of the requested segment allocation, the number of -segment allocations that have occurred since the last GC, and the EGC -and GC thresholds, the segment allocation trap handler may invoke a full -or ephemeral GC before returning a new segment. It's worth noting that -the [E]GC is triggered based on the number of and size of these segments -that've been allocated since the last GC; it doesn't have much to do -with how "full" each of those per-thread segments are. It's possible -for a large number of threads to do fairly incidental memory allocation -and trigger the GC as a result; avoiding this involves tuning the -per-thread allocation quantum and the GC/EGC thresholds appropriately. + Allocation of reserved heap segments + After lisp image is first mapped into memory - and after + each full GC - the lisp kernel ensures that + (LISP-HEAP-GC-TRESHOLD) additional bytes beyond the current + end of the heap are mapped read-write. + If a thread traps while trying to allocate memory, the + thread goes through the usual exception-handling protocol (to + ensure that any oher thread that GCs "sees" the state of the + trapping thread and to serialize exception handling.) When + the exception handler runs, it determines the nature and size + of the failed allocation and tries to complete the allocation + on the thread's behalf (and leave it with a reasonably large + thread-specific memory segment so that the next small + allocation is unlikely to trap. + Depending on the size of the requested segment + allocation, the number of segment allocations that have + occurred since the last GC, and the EGC and GC thresholds, the + segment allocation trap handler may invoke a full or ephemeral + GC before returning a new segment. It's worth noting that the + [E]GC is triggered based on the number of and size of these + segments that've been allocated since the last GC; it doesn't + have much to do with how "full" each of those per-thread + segments are. It's possible for a large number of threads to + do fairly incidental memory allocation and trigger the GC as a + result; avoiding this involves tuning the per-thread + allocation quantum and the GC/EGC thresholds + appropriately. - Heap growthAll OSes on which OpenMCL currently runs use an "overcommit" memory -allocation strategy by default (though some of them provide ways of -overriding that default.) What this means in general is that the OS -doesn't necessarily ensure that backing store is available when asked -to map pages as read-write; it'll often return a success indicator from -the mapping attempt (mapping the pages as "zero-fill, copy-on-write"), -and only try to allocate the backing store (swap space and/or physical -memory) when non-zero contents are written to the pages. - It -sounds- like it'd be better to have the mmap() call fail -immediately, but it's actually a complicated issue. (It's possible that -other applications will stop using some backing store before lisp code -actually touches the pages that need it, for instance.) It's also -not guaranteed that lisp code would be able to "cleanly" signal an -out-of-memory condition if lisp is ... out of memory + Heap growth + All OSes on which OpenMCL currently runs use an + "overcommit" memory allocation strategy by default (though + some of them provide ways of overriding that default.) What + this means in general is that the OS doesn't necessarily + ensure that backing store is available when asked to map pages + as read-write; it'll often return a success indicator from the + mapping attempt (mapping the pages as "zero-fill, + copy-on-write"), and only try to allocate the backing store + (swap space and/or physical memory) when non-zero contents are + written to the pages. + It -sounds- like it'd be better to have the mmap() call + fail immediately, but it's actually a complicated issue. + (It's possible that other applications will stop using some + backing store before lisp code actually touches the pages that + need it, for instance.) It's also not guaranteed that lisp + code would be able to "cleanly" signal an out-of-memory + condition if lisp is ... out of memory I don't know that I've ever seen an abrupt out-of-memory failure that wasn't preceeded by several minutes of excessive paging activity. The @@ -10323,408 +14677,537 @@ - GC details -The GC uses a Mark/Compact algorithm; its execution time is -essentially a factor of the amount of live data in the heap. (The somewhat -better-known Mark/Sweep algorithms don't compact the live data but instead -traverse the garbage to rebuild free-lists; their execution time is -therefore a factor of the total heap size.) + GC details + The GC uses a Mark/Compact algorithm; its + execution time is essentially a factor of the amount of live + data in the heap. (The somewhat better-known Mark/Sweep + algorithms don't compact the live data but instead traverse the + garbage to rebuild free-lists; their execution time is therefore + a factor of the total heap size.) As mentioned in , two auxiliary data structures -(proportional to the size of the lisp heap) are maintained. These are - - the markbits bitvector, which contains a bit for everydoublenode in the dynamic heap (plus a few extra words for alignmentand so that sub-bitvectors can start on word boundaries.) - the relocation table, which contains a native word for every 32 or 64doublenodes in the dynamic heap, plus an extra word used to keep trackof the end of the heap. - - The total GC space overhead is therefore on the order of 3% (2/64 or -1/32). + (proportional to the size of the lisp heap) are maintained. These are + + + the markbits bitvector, which contains a bit for + everydoublenode in the dynamic heap (plus a few extra words + for alignmentand so that sub-bitvectors can start on word + boundaries.) + + + the relocation table, which contains a native word for + every 32 or 64 doublenodes in the dynamic heap, plus an + extra word used to keep trackof the end of the heap. + + + The total GC space overhead is therefore on the order of + 3% (2/64 or 1/32). The general algorithm proceeds as follows: - Mark phase -Each doublenode in the dynamic heap has a corresponding bit in the -markbits vector. (For any doublenode in the heap, the index of its mark -bit is determined by subtracing the address of the start of the heap -from the address of the object and dividing the result by 8 or 16.) The GC -knows the markbit index of the free pointer, so determining that the -markbit index of a doubleword address is between the start of the heap -and the free pointer can be done with a single unsigned -comparison. - The markbits of all doublenodes in the dynamic heap are zeroed -before the mark phase begins. An object is marked -if the markbits of all of its constituent doublewords are set and -unmarked otherwise; setting an object's markbits involves setting the -corrsponding markbits of all constituent doublenodes in the -object. - The mark phase traverses each root. If the tag of the value of -the root indicates that it's a non-immediate node whose address lies -in the lisp heap, then: - - If the object is already marked, do nothing. - Set the object's markbit(s). - If the object is an ivector, do nothing further. - If the object is a cons cell, recursively mark its car andcdr. - Otherwise, the object is a gvector. Recursively mark itselements. - - Marking an object thus involves ensuring that its mark bits are set and -then recursively marking any pointers contained within the object if the -object was originally unmarked. If this recursive step was implemented -in the obvious manner, marking an object would take stack space -proportional to the length of the pointer chain from some root to that -object. Rather than storing that pointer chain implicitly on the stack -(in a series of recursive calls to the mark subroutine), the OpenMCL -marker uses mixture of recursion and a technique called link -inversion to store the pointer chain in the objects themselves. -(Recursion tends to be simpler and faster; if a recursive step notes -that stack space is becoming limited, the link-inversion technique is -used.) + Mark phase + Each doublenode in the dynamic heap has a corresponding + bit in the markbits vector. (For any doublenode in the heap, + the index of its mark bit is determined by subtracing the + address of the start of the heap from the address of the + object and dividing the result by 8 or 16.) The GC knows the + markbit index of the free pointer, so determining that the + markbit index of a doubleword address is between the start of + the heap and the free pointer can be done with a single + unsigned comparison. + The markbits of all doublenodes in the dynamic heap are + zeroed before the mark phase begins. An object is + marked if the markbits of all of its + constituent doublewords are set and unmarked otherwise; + setting an object's markbits involves setting the corrsponding + markbits of all constituent doublenodes in the object. + The mark phase traverses each root. If the tag of the + value of the root indicates that it's a non-immediate node + whose address lies in the lisp heap, then: + + + If the object is already marked, do nothing. + + + Set the object's markbit(s). + + + If the object is an ivector, do nothing further. + + + If the object is a cons cell, recursively mark its + car and cdr. + + + Otherwise, the object is a gvector. Recursively mark + itselements. + + + Marking an object thus involves ensuring that its mark + bits are set and then recursively marking any pointers + contained within the object if the object was originally + unmarked. If this recursive step was implemented in the + obvious manner, marking an object would take stack space + proportional to the length of the pointer chain from some root + to that object. Rather than storing that pointer chain + implicitly on the stack (in a series of recursive calls to the + mark subroutine), the OpenMCL marker uses mixture of recursion + and a technique called link inversion to + store the pointer chain in the objects themselves. (Recursion + tends to be simpler and faster; if a recursive step notes that + stack space is becoming limited, the link-inversion technique + is used.) Certain types of objects are treated a little specially: - - To support a feature called GCTWA - -I believe that theacronym comes from MACLISP, where it stood for "Garbage Collection ofTruly Worthless Atoms". - -, the vector which contains the internalsymbols of the current package is marked on entry to the mark phasebut the symbols themselves are not marked at this time. Near the endof the mark phase, symbols referenced from this vector which are nototherwise marked are marked if and only if they're somehowdistinguishable from newly created symbols (by virtue of their havingfunction bindings, value bindings, plists, or other attributes.) - Pools have their first element set to NIL before any otherelements are marked. - All hash tables have certain fields (used to cache previousresults) invalidated. - Weak Hash Tables and other weak objects are put on a linkedlist as they're encountered; their contents are only retained ifthere are other (non-weak) references to them. - + + + To support a feature called GCTWA + + I believe that theacronym comes from MACLISP, + where it stood for "Garbage Collection ofTruly + Worthless Atoms". + + , the vector which contains the + internalsymbols of the current package is marked on + entry to the mark phasebut the symbols themselves are + not marked at this time. Near the endof the mark phase, + symbols referenced from this vector which are + nototherwise marked are marked if and only if they're + somehowdistinguishable from newly created symbols (by + virtue of their havingfunction bindings, value bindings, + plists, or other attributes.) + + + Pools have their first element set to NIL before any + otherelements are marked. + + + All hash tables have certain fields (used to cache + previous results) invalidated. + + + Weak Hash Tables and other weak objects are put on a + linkedlist as they're encountered; their contents are only + retained if there are other (non-weak) references to + them. + + At the end of the mark phase, the markbits of all objects which -are transitively reachable from the roots are set and all other markbits -are clear. + are transitively reachable from the roots are set and all other markbits + are clear. - Relocation phase -The forwarding address of a doublenode in the -dynamic heap is ( - (size_of_doublenode * )) or alternately ( + (size_of_doublenode * )). Rather than count the number of preceding markbits each time, -the relocation table is used to precompute an approximation of the -forwarding addresses for all doublewords. Given this approximate address -and a pointer into the markbits vector, it's relatively easy to compute -the exact forwarding address. - The relocation table contains the forwarding addresses of each -pagelet, where a pagelet is 256 bytes (or 32 -doublenodes). The forwarding address of the first pagelet is the base of -the heap. The forwarding address of the second pagelet is the sum of the -forwarding address of the first and 8 bytes for each mark bit set in the -first 32-bit word in the markbits table. The last entry in the -relocation table contains the forwarding address that the freepointer -would have, e.g., the new value of the freepointer after -compaction. - In many programs, old objects rarely become garbage and new -objects often do. When building the relocation table, the relocation -phase notes the address of the first unmarked object in the dynamic -heap. Only the area of the heap between the first unmarked object and -the freepointer needs to be compacted; only pointers to this area will -need to be forwarded (the forwarding address of all other pointers to -the dynamic heap is the address of that pointer.) Often, the first -unmarked object is much nearer the free pointer than it is to the base -of the heap. + Relocation phase + The forwarding address of a + doublenode in the dynamic heap is (<its current address> - + (size_of_doublenode * <the number of unmarked markbits that + precede it>)) or alternately (<the base of the heap> + + (size_of_doublenode * <the number of marked markbits that + preced it >)). Rather than count the number of preceding + markbits each time, the relocation table is used to precompute + an approximation of the forwarding addresses for all + doublewords. Given this approximate address and a pointer into + the markbits vector, it's relatively easy to compute the exact + forwarding address. + The relocation table contains the forwarding addresses + of each pagelet, where a pagelet is 256 + bytes (or 32 doublenodes). The forwarding address of the first + pagelet is the base of the heap. The forwarding address of the + second pagelet is the sum of the forwarding address of the + first and 8 bytes for each mark bit set in the first 32-bit + word in the markbits table. The last entry in the relocation + table contains the forwarding address that the freepointer + would have, e.g., the new value of the freepointer after + compaction. + In many programs, old objects rarely become garbage and + new objects often do. When building the relocation table, the + relocation phase notes the address of the first unmarked + object in the dynamic heap. Only the area of the heap between + the first unmarked object and the freepointer needs to be + compacted; only pointers to this area will need to be + forwarded (the forwarding address of all other pointers to the + dynamic heap is the address of that pointer.) Often, the + first unmarked object is much nearer the free pointer than it + is to the base of the heap. - Forwarding phase -The forwarding phase traverses all roots and the -"old" part of the dynamic heap (the part between the base -of the heap and the first unmarked object.) All references to objects -whose address is between the first unmarked object and the free pointer -are updated to point to the address the object will have after -compaction by using the relocation table and the markbits vector and -interpolating. - The relocation table entry for the pagelet nearest the object is -found. If the pagelet's address is less than the object's address, the -number of set markbits that precede the object on the pagelet is used to -determine the object's address; otherwise, the number of set markbits -the follow the object on the pagelet is used. - Since forwarding views the heap as a set of doublewords, locatives -are (mostly) treated like any other pointers. (The basic difference is -that locatives may appear to be tagged as fixnums, in which case they're -treated as word-aligned pointers into the object.) - If the forward phase changes the address of any hash table key in -a hash table that hashes by address (e.g., an EQ hash table), it sets a -bit in the hash table's header. The hash table code will rehash the hash -table's contents if it tries to do a lookup on a key in such a -table. - Profiling reveals that about half of the total time spent in the -GC is spent in the subroutine which determines a pointer's forwarding -address. Exploiting GCC-specific idioms, hand-coding the routine, and -inlining calls to it could all be expected to improve GC -performance. + Forwarding phase + The forwarding phase traverses all roots and the "old" + part of the dynamic heap (the part between the base of the + heap and the first unmarked object.) All references to objects + whose address is between the first unmarked object and the + free pointer are updated to point to the address the object + will have after compaction by using the relocation table and + the markbits vector and interpolating. + The relocation table entry for the pagelet nearest the + object is found. If the pagelet's address is less than the + object's address, the number of set markbits that precede the + object on the pagelet is used to determine the object's + address; otherwise, the number of set markbits the follow the + object on the pagelet is used. + Since forwarding views the heap as a set of doublewords, + locatives are (mostly) treated like any other pointers. (The + basic difference is that locatives may appear to be tagged as + fixnums, in which case they're treated as word-aligned + pointers into the object.) + If the forward phase changes the address of any hash + table key in a hash table that hashes by address (e.g., an EQ + hash table), it sets a bit in the hash table's header. The + hash table code will rehash the hash table's contents if it + tries to do a lookup on a key in such a table. + Profiling reveals that about half of the total time + spent in the GC is spent in the subroutine which determines a + pointer's forwarding address. Exploiting GCC-specific idioms, + hand-coding the routine, and inlining calls to it could all be + expected to improve GC performance. - Compact phase -The compact phase compacts the area between the first unmarked -object and the freepointer so that it contains only marked objects. -While doing so, it forwards any pointers it finds in the objects it -copies. - When the compact phase is finished, so is the GC (more or less): -the free pointer and some other data structures are updated and control -returns to the exception handler that invoked the GC. If sufficient -memory has been freed to satisfy any allocation request that may have -triggered the GC, the exception handler returns; otherwise, a -"seriously low on memory" condition is signalled, possibly -after releasing a small emergency pool of memory. + Compact phase + The compact phase compacts the area between the first + unmarked object and the freepointer so that it contains only + marked objects. While doing so, it forwards any pointers it + finds in the objects it copies. + When the compact phase is finished, so is the GC (more + or less): the free pointer and some other data structures are + updated and control returns to the exception handler that + invoked the GC. If sufficient memory has been freed to satisfy + any allocation request that may have triggered the GC, the + exception handler returns; otherwise, a "seriously low on + memory" condition is signalled, possibly after releasing a + small emergency pool of memory. - The ephemeral GC -In the OpenMCL memory management scheme, the relative age of two -objects in the dynamic heap can be determined by their addresses: if -addresses X and Y are both addresses in the dynamic heap, X is younger -than Y (X was created more recently than Y) if it is nearer to the free -pointer (and farther from the base of the heap) than Y. - Ephemeral (or generational) garbage collectors attempt to exploit -the following assumptions: - - most newly created objects become garbage soon after they'recreated. - most objects that have already survived several GCs are unlikelyto ever become garbage. - old objects can only point to newer objects as the result of adestructive modification (e.g., via SETF.) + The ephemeral GC + In the OpenMCL memory management scheme, the relative age + of two objects in the dynamic heap can be determined by their + addresses: if addresses X and Y are both addresses in the + dynamic heap, X is younger than Y (X was created more recently + than Y) if it is nearer to the free pointer (and farther from + the base of the heap) than Y. + Ephemeral (or generational) garbage collectors attempt to + exploit the following assumptions: + + + most newly created objects become garbage soon after + they'recreated. + + + most objects that have already survived several GCs + are unlikely to ever become garbage. + + + old objects can only point to newer objects as the + result of adestructive modification (e.g., via + SETF.) + + + + By concentrating its efforts on (frequently and quickly) + reclaiming newly created garbage, an ephemeral collector hopes + to postpone the more costly full GC as long as possible. It's + important to note that most programs create some long-lived + garbage, so an EGC can't typically eliminate the need for full + GC. + An EGC views each object in the heap as belonging to + exactly one generation; generations are + sets of objects that are related to each other by age: some + generation is the youngest, some the oldest, and there's an age + relationship between any intervening generations. Objects are + typically assigned to the youngest generation when first + allocated; any object that has survived some number of GCs in + its current generation is promoted (or + tenured) into an older generation. + When a generation is GCed, the roots consist of the + stacks, registers, and global variables as always and also of + any pointers to objects in that generation from other + generations. To avoid the need to scan those (often large) other + generations looking for such intergenerational references, the + runtime system must note all such intergenerational references + at the point where they're created (via Setf).This is + sometimes called "The Write Barrier": all assignments which + might result in intergenerational references must be noted, as + if the other generations were write-protected. The + set of pointers that may contain intergenerational references is + sometimes called the remembered set. + In OpenMCL's EGC, the heap is organized exactly the same + as otherwise; "generations" are merely structures which contain + pointers to regions of the heap (which is already ordered by + age.) When a generation needs to be GCed, any younger generation + is incorporated into it; all objects which survive a GC of a + given generation are promoted into the next older + generation. The only intergenerational references that can exist + are therefore those where an old object is modified to contain a + pointer to a new object. + The EGC uses exactly the same code as the full GC. When a + given GC is "ephemeral", + + + the "base of the heap" used to determine anobject's + markbit address is the base of the generation + being collected; + + + the markbits vector is actually a pointer into the + middle of the global markbits table; preceding entries in + this table are used to note doubleword addresses in older + generations that (may) contain intergenerational + references; + + + some steps (notably GCTWA and the handling of weak + objects) are not performed; + + + the intergenerational references table is used to + findadditional roots for the mark and forward phases. If a + bit is set inthe intergenerational references table, that + means that thecorresponding doubleword (in some "old" + generation, insome "earlier" part of the heap) may have had + a pointerto an object in a younger generation stored into + it. + - - By concentrating its efforts on (frequently and quickly) reclaiming -newly created garbage, an ephemeral collector hopes to postpone the more -costly full GC as long as possible. It's important to note that most -programs create some long-lived garbage, so an EGC can't typically -eliminate the need for full GC. - An EGC views each object in the heap as belonging to exactly one -generation; generations are sets of objects that are -related to each other by age: some generation is the youngest, some the -oldest, and there's an age relationship between any intervening -generations. Objects are typically assigned to the youngest generation -when first allocated; any object that has survived some number of GCs in -its current generation is promoted (or tenured) into -an older generation. - When a generation is GCed, the roots consist of the stacks, -registers, and global variables as always and also of any pointers to -objects in that generation from other generations. To avoid the need to -scan those (often large) other generations looking for such -intergenerational references, the runtime system must note all such -intergenerational references at the point where they're created (via -Setf).This is sometimes called "The Write Barrier": all -assignments which might result in intergenerational references must be -noted, as if the other generations were write-protected. The set of pointers that may contain intergenerational -references is sometimes called the remembered -set. - In OpenMCL's EGC, the heap is organized exactly the same as otherwise; -"generations" are merely structures which contain pointers to -regions of the heap (which is already ordered by age.) When a generation -needs to be GCed, any younger generation is incorporated into it; all -objects which survive a GC of a given generation are promoted into the -next older generation. The only intergenerational references that can -exist are therefore those where an old object is modified to contain a -pointer to a new object. - The EGC uses exactly the same code as the full GC. When a given GC -is "ephemeral", - - the "base of the heap" used to determine anobject's markbit address is the base of the generation beingcollected; - the markbits vector is actually a pointer into the middle of theglobal markbits table; preceding entries in this table are used tonote doubleword addresses in older generations that (may) containintergenerational references; - some steps (notably GCTWA and the handling of weak objects) arenot performed; - the intergenerational references table is used to findadditional roots for the mark and forward phases. If a bit is set inthe intergenerational references table, that means that thecorresponding doubleword (in some "old" generation, insome "earlier" part of the heap) may have had a pointerto an object in a younger generation stored into it. - - - The intergenerational references table is maintained indirectly: -whenever a setf operation that may introduce an intergenerational -reference occurs, a pointer to the doubleword being stored into is pushed -onto the memo buffer, which is a stack whos top is -addressed by the memo register. Whenever the memo buffer -overflowsA guard page at the end of the memo buffer simplifies overflow -detection. when the EGC is active, the handler scans the buffer and -sets bits in the intergenerational references table for each doubleword -address it finds in the buffer that belongs to some generation other than -the youngest; the same scan is performed on entry to any ephemeral GC. -After (possibly) performing this scan, the handler resets the memo -register to point to the bottom of the memo stack; this means that when -the EGC is inactive, the memo buffer is constantly being filled and -emptied for no apparent reason. - With one exception (the implicit setfs that occur on entry to and -exit from the binding of a special variable), all setfs that might -introduce an intergenerational reference must be memoized.Note that the implicit setfs that occur when initializing an -object - as in the case of a call to cons or vector - can't introduce -intergenerational references, since the newly created object is always -younger than the objects used to initialize it. It's always safe to push any cons cell or gvector locative -onto the memo stack; it's never safe to push anything else. - Typically, the intergenerational references bitvector is sparse: a -relatively small number of old locations are stored into, although some of -them may have been stored into many times. The routine that scans the -memoization buffer does a lot of work and usually does it fairly often; it -uses a simple, brute-force method but might run faster if it was smarter -about recognizing addresses that it'd already seen. - When the EGC mark and forward phases scan the intergenerational -reference bits, they can clear any bits that denote doublewords that -definitely do not contain intergenerational references. + + The intergenerational references table is maintained + indirectly: whenever a setf operation that may introduce an + intergenerational reference occurs, a pointer to the doubleword + being stored into is pushed onto the memo + buffer, which is a stack whos top is addressed by the + memo register. Whenever the memo buffer overflowsA + guard page at the end of the memo buffer simplifies overflow + detection. when the EGC is active, the handler + scans the buffer and sets bits in the intergenerational + references table for each doubleword address it finds in the + buffer that belongs to some generation other than the youngest; + the same scan is performed on entry to any ephemeral GC. After + (possibly) performing this scan, the handler resets the memo + register to point to the bottom of the memo stack; this means + that when the EGC is inactive, the memo buffer is constantly + being filled and emptied for no apparent reason. + With one exception (the implicit setfs that occur on entry + to and exit from the binding of a special variable), all setfs + that might introduce an intergenerational reference must be + memoized.Note that the implicit setfs that occur when + initializing an object - as in the case of a call to cons or + vector - can't introduce intergenerational references, since the + newly created object is always younger than the objects used to + initialize it. It's always safe to push any cons + cell or gvector locative onto the memo stack; it's never safe to + push anything else. + Typically, the intergenerational references bitvector is + sparse: a relatively small number of old locations are stored + into, although some of them may have been stored into many + times. The routine that scans the memoization buffer does a lot + of work and usually does it fairly often; it uses a simple, + brute-force method but might run faster if it was smarter about + recognizing addresses that it'd already seen. + When the EGC mark and forward phases scan the + intergenerational reference bits, they can clear any bits that + denote doublewords that definitely do not contain + intergenerational references. - Fasl files -The information in this section was current in November 2004. -Saving and loading of Fasl files is implemented in -xdump/faslenv.lisp, level-0/nfasload.lisp, and lib/nfcomp.lisp. -The information here is only an overview, which might help when -reading the source. - The OpenMCL Fasl format is forked from the old MCL Fasl format; there -are a few differences, but they are minor. The name "nfasload" -comes from the fact that this is the so-called "new" Fasl system, -which was true in 1986 or so. The format has held up well, although -it would certainly need extensions to deal with 64-bit data, and -some other modernization might be possible. - A Fasl file begins with a "file header", which contains version -information and a count of the following "blocks". There's typically -only one "block" per Fasl file. The blocks are part of a mechanism -for -combining multiple logical files into a single physical file, in order -to simplify the distribution of precompiled programs. (Nobody seems -to be doing anything interesting with this feature, at the moment, -probably because it isn't documented.) - Each block begins with a header for itself, which just describes the -size of the data that follows. - The data in each block is treated as a simple stream of bytes, which -define a bytecode program. The actual bytecodes, "fasl operators", -are defined in xdump/faslenv.lisp. The descriptions in the source -file are terse, but, according to Gary, "probably accurate". - Some of the operators are used to create a per-block "object table", -which -is a vector used to keep track of previously-loaded objects and -simplify references to them. When the table is created, an index -associated with it is set to zero; this is analogous to an array -fill-pointer, and allows the table to be treated like a stack. - The low seven bits of each bytecode are used to specify the -fasl operator; -currently, about fifty operators are defined. The high byte, when -set, indicates that the result of the operation should be pushed -onto the object table. - Most bytecodes are followed by operands; the operand data is -byte-aligned. -How many operands there are, and their type, depend on the bytecode. -Operands can be indices into the object table, immediate values, or -some combination of these. - An exception is the bytecode #xFF, which has the symbolic name -ccl::$faslend; it is used to mark the end of the block. + Fasl files + The information in this section was current in November + 2004. Saving and loading of Fasl files is implemented in + xdump/faslenv.lisp, level-0/nfasload.lisp, and lib/nfcomp.lisp. + The information here is only an overview, which might help when + reading the source. + The OpenMCL Fasl format is forked from the old MCL Fasl + format; there are a few differences, but they are minor. The + name "nfasload" comes from the fact that this is the so-called + "new" Fasl system, which was true in 1986 or so. The format has + held up well, although it would certainly need extensions to + deal with 64-bit data, and some other modernization might be + possible. + A Fasl file begins with a "file header", which contains + version information and a count of the following "blocks". + There's typically only one "block" per Fasl file. The blocks + are part of a mechanism for combining multiple logical files + into a single physical file, in order to simplify the + distribution of precompiled programs. (Nobody seems to be doing + anything interesting with this feature, at the moment, probably + because it isn't documented.) + Each block begins with a header for itself, which just + describes the size of the data that follows. + The data in each block is treated as a simple stream of + bytes, which define a bytecode program. The actual bytecodes, + "fasl operators", are defined in xdump/faslenv.lisp. The + descriptions in the source file are terse, but, according to + Gary, "probably accurate". + Some of the operators are used to create a per-block + "object table", which is a vector used to keep track of + previously-loaded objects and simplify references to them. When + the table is created, an index associated with it is set to + zero; this is analogous to an array fill-pointer, and allows the + table to be treated like a stack. + The low seven bits of each bytecode are used to specify + the fasl operator; currently, about fifty operators are defined. + The high byte, when set, indicates that the result of the + operation should be pushed onto the object table. + Most bytecodes are followed by operands; the operand data + is byte-aligned. How many operands there are, and their type, + depend on the bytecode. Operands can be indices into the object + table, immediate values, or some combination of these. + An exception is the bytecode #xFF, which has the symbolic + name ccl::$faslend; it is used to mark the end of the + block. - - Heap images -Needs complete rewrite. - - - - Operating system dependencies - + - The Objective-C Bridge -Unlike the rest of this chapter, this section was written in late -2004. + The Objective-C Bridge - How OpenMCL Recognizes Objective-C Objects -In most cases, pointers to instances of Objective-C classes are -recognized as such; the recognition is (and probably always will be) -slightly heuristic. Basically, any pointer that passes basic sanity -checks and whose first word is a pointer to a known ObjC class is -considered to be an instance of that class; the Objective-C runtime -system would reach the same conclusion. - It's certainly possible that a random pointer to an arbitrary -memory address could look enough like an ObjC instance to fool the lisp -runtime system, and it's possible that pointers could have their -contents change so that something that had either been a true ObjC -instance (or had looked a lot like one) is changed (possibly by virtue -of having been deallocated.) - In the first case, we can improve the heuristics substantially: we -can make stronger assertions that a particular pointer is really "of -type :ID" when it's a parameter to a function declared to take -such a pointer as an argument or a similarly declared function result; we -can be more confident of something we obtained via SLOT-VALUE of a slot -defined to be of type :ID than if we just dug a pointer out of memory -somewhere. - The second case is a little more subtle: ObjC memory management is -based on a reference-counting scheme, and it's possible for an -object to ... cease to be an object while lisp is still referencing it. -If we don't want to deal with this possibility (and we don't), -we'll basically have to ensure that the object is not deallocated -while lisp is still thinking of it as a first-class object. There's -some support for this in the case of objects created with MAKE-INSTANCE, -but we may need to give similar treatment to foreign objects that are -introduced to the lisp runtime in other ways (as function arguments, -return values, SLOT-VALUE results, etc. as well as those instances -that're created under lisp control.) - This doesn't all work yet (in fact, not much of it works yet); -in practice, this has not yet been as much of a problem as anticipated, -but that may be because existing Cocoa code deals primarily with -relatively long-lived objects such as windows, views, menus, etc. + How OpenMCL Recognizes Objective-C Objects + In most cases, pointers to instances of Objective-C + classes are recognized as such; the recognition is (and + probably always will be) slightly heuristic. Basically, any + pointer that passes basic sanity checks and whose first word + is a pointer to a known ObjC class is considered to be an + instance of that class; the Objective-C runtime system would + reach the same conclusion. + It's certainly possible that a random pointer to an + arbitrary memory address could look enough like an ObjC + instance to fool the lisp runtime system, and it's possible + that pointers could have their contents change so that + something that had either been a true ObjC instance (or had + looked a lot like one) is changed (possibly by virtue of + having been deallocated.) + In the first case, we can improve the heuristics + substantially: we can make stronger assertions that a + particular pointer is really "of type :ID" when it's a + parameter to a function declared to take such a pointer as an + argument or a similarly declared function result; we can be + more confident of something we obtained via SLOT-VALUE of a + slot defined to be of type :ID than if we just dug a pointer + out of memory somewhere. + The second case is a little more subtle: ObjC memory + management is based on a reference-counting scheme, and it's + possible for an object to ... cease to be an object while lisp + is still referencing it. If we don't want to deal with this + possibility (and we don't), we'll basically have to ensure + that the object is not deallocated while lisp is still + thinking of it as a first-class object. There's some support + for this in the case of objects created with MAKE-INSTANCE, + but we may need to give similar treatment to foreign objects + that are introduced to the lisp runtime in other ways (as + function arguments, return values, SLOT-VALUE results, etc. as + well as those instances that're created under lisp + control.) + This doesn't all work yet (in fact, not much of it works + yet); in practice, this has not yet been as much of a problem + as anticipated, but that may be because existing Cocoa code + deals primarily with relatively long-lived objects such as + windows, views, menus, etc. - - Recommended Reading - The Apple Objective-C Runtime Reference - This describes the internal data structures and programminginterface of Objective C as it is implemented on OS X. - - + + Recommended Reading + + + + + Cocoa Documentation + + + + + This is the top page for all of Apple's documentation on + Cocoa. If you are unfamiliar with Cocoa, it is a good + place to start. + + + + + + Foundation Reference for Objective-C + + + + + This is one of the two most important Cocoa references; it + covers all of the basics, except for GUI programming. This is + a reference, not a tutorial. + + + + + - Modifying OpenMCL + Modifying OpenMCL - Contributing Code Back to the OpenMCL Project -This section is a placeholder, added as of August 2004. The -full text is being written, and will be added as soon as it is -available. + Contributing Code Back to the OpenMCL Project + This section is a placeholder, added as of August 2004. The + full text is being written, and will be added as soon as it is + available. - Using OpenMCL in "development" and in "user" mode -As it's distributed, OpenMCL starts up with *PACKAGE* set to the CL-USER -package and with most predefined functions and methods protected against -accidental redefinition. The package setting is of course a requirement -of ANSI CL, while the protection protection is intended to catch certain -types of programming errors (accidentally redefining a CL or CCL function) -before those errors have a chance to do much damage. - These settings may make using OpenMCL to develop OpenMCL a bit more -awkward, since much of that process assumes that the CCL package is -current (and a primary purpose of that process is to redefine some -"predefined, builtin functions".) The standard, "routine" -ways of building OpenMCL from sources (see ) -- COMPILE-CCL, XCOMPILE-CCL, and XLOAD-LEVEL-0 - bind *PACKAGE* to the -"CCL" package and enable the redefinition of predefined functions; -the symbols COMPILE-CCL, XCOMPILE-CCL, and XLOAD-LEVEL-0 are additionally -now exported from the "CCL" package. - Some other (more ad-hoc) ways of doing development on OpenMCL - -compiling and/or loading individual files, incrementally redefining -individual functions - may be awkward unless one reverts to the mode of -operation which was traditionally offered in OpenMCL. (Some OpenMCL source -files - especially those that comprise the bootstrapping image sources and -the first few files in the "cold load" sequence - are compiled and -loaded in the "CCL" package but don't contain (IN-PACKAGE -"CCL") forms, since IN-PACKAGE doesn't work until later in the -cold load sequence.) - The somewhat bizarre behavior of both SET-USER-ENVIRONMENT and -SET-DEVELOPMENT-ENVIRONMENT with respect to the special variables they -affect is intended to allow those constructs to take effect when the -read-eval-print loop next returns to a top-level '? ' prompt; -the constructs can meaningfully be used inside LOAD, for instance -(recall that LOAD binds *PACKAGE*), though using both constructs within -the same LOAD call would likely be pretty confusing. - "user" and "development" are otherwise very -generic terms; here they're intended to enforce the distinction -between "using" OpenMCL and "developing" it. - The initial environment from which OpenMCL images are saved is one -where (SET-USER-ENVIRONMENT T) has just been called; in previous -versions, it was effectively as if (SET-DEVELOPMENT-ENVIRONMENT T) had -just been called. - Hopefully, most users of OpenMCL can safely ignore these issues -most of the time. Note that doing (SET-USER-ENVIRONMENT T) after loading -one's own code (or 3rd-party code) into OpenMCL would protect that -code (as well as OpenMCL's) from accidental redefinition; that may -be useful in some cases. + Using OpenMCL in "development" and in "user" mode + + As it's distributed, OpenMCL starts up with *PACKAGE* set + to the CL-USER package and with most predefined functions and + methods protected against accidental redefinition. The package + setting is of course a requirement of ANSI CL, while the + protection protection is intended to catch certain types of + programming errors (accidentally redefining a CL or CCL + function) before those errors have a chance to do much + damage. + These settings may make using OpenMCL to develop OpenMCL a + bit more awkward, since much of that process assumes that the + CCL package is current (and a primary purpose of that process is + to redefine some "predefined, builtin functions".) The standard, + "routine" ways of building OpenMCL from sources (see ) - + COMPILE-CCL, XCOMPILE-CCL, and XLOAD-LEVEL-0 - bind *PACKAGE* to + the "CCL" package and enable the redefinition of predefined + functions; the symbols COMPILE-CCL, XCOMPILE-CCL, and + XLOAD-LEVEL-0 are additionally now exported from the "CCL" + package. + Some other (more ad-hoc) ways of doing development on + OpenMCL - compiling and/or loading individual files, + incrementally redefining individual functions - may be awkward + unless one reverts to the mode of operation which was + traditionally offered in OpenMCL. (Some OpenMCL source files - + especially those that comprise the bootstrapping image sources + and the first few files in the "cold load" sequence - are + compiled and loaded in the "CCL" package but don't contain + (IN-PACKAGE "CCL") forms, since IN-PACKAGE doesn't work until + later in the cold load sequence.) + The somewhat bizarre behavior of both SET-USER-ENVIRONMENT + and SET-DEVELOPMENT-ENVIRONMENT with respect to the special + variables they affect is intended to allow those constructs to + take effect when the read-eval-print loop next returns to a + top-level '? ' prompt; the constructs can meaningfully be used + inside LOAD, for instance (recall that LOAD binds *PACKAGE*), + though using both constructs within the same LOAD call would + likely be pretty confusing. + "user" and "development" are otherwise very generic terms; + here they're intended to enforce the distinction between "using" + OpenMCL and "developing" it. + The initial environment from which OpenMCL images are + saved is one where (SET-USER-ENVIRONMENT T) has just been + called; in previous versions, it was effectively as if + (SET-DEVELOPMENT-ENVIRONMENT T) had just been called. + Hopefully, most users of OpenMCL can safely ignore these + issues most of the time. Note that doing (SET-USER-ENVIRONMENT + T) after loading one's own code (or 3rd-party code) into OpenMCL + would protect that code (as well as OpenMCL's) from accidental + redefinition; that may be useful in some cases. - Debugging facilities in the lisp kernel -In a perfect world, something like this couldn't happen: + Debugging facilities in the lisp kernel + In a perfect world, something like this couldn't + happen: Welcome to OpenMCL Version x.y! @@ -10736,5 +15219,5 @@ ? (foo -1) ;Oops. Too late ... Unhandled exception 11 at 0x300e90c8, context->regs at #x7ffff6b8 -Continue/Debugger/eXit ? +Continue/Debugger/eXit <enter>? As you may have noticed, it's not a perfect world; it's rare @@ -10750,6 +15233,6 @@ ? (defun classify (n) - (cond ((> n 0) "Greater") - ((< n 0) "Less") + (cond ((> n 0) "Greater") + ((< n 0) "Less") (t ;;; Sheesh ! What else could it be ? @@ -10762,16 +15245,18 @@ ? for help [12345] OpenMCL kernel debugger: - - CCL::BUG isn't quite the right tool for this example (a call to -BREAK or PRINT might do a better job of clearing up the mystery), but -it's sometimes helpful when those other tools can't be used. -The -lisp error system notices, for instance, if attempts to signal errors -themselves cause errors to be signaled; this sort of thing can happen if -CLOS or the I/O system are broken or missing. After some small number of -recursive errors, the error system gives up and calls CCL::BUG. - If one enters a '?' at the kernel debugger prompt, one will see output -like: - (S) Find and describe symbol matching specified name + + CCL::BUG isn't quite the right tool for this example (a + call to BREAK or PRINT might do a better job of clearing up the + mystery), but it's sometimes helpful when those other tools + can't be used. The lisp error system notices, for instance, if + attempts to signal errors themselves cause errors to be + signaled; this sort of thing can happen if CLOS or the I/O + system are broken or missing. After some small number of + recursive errors, the error system gives up and calls + CCL::BUG. + If one enters a '?' at the kernel debugger prompt, one + will see output like: + +(S) Find and describe symbol matching specified name (B) Show backtrace (X) Exit from this debugger, asserting that any exception was handled @@ -10779,15 +15264,17 @@ (?) Show this help - CCL::BUG just does an FF-CALL into the lisp kernel. If the kernel debugger -was invoked because of an unhandled exception (such as an illegal memory -reference) the OS kernel saves the machine state -("context") in a data structure for us, and in that case some additional -options can be used to display the contents of the registers at the point -of the exception. Another function - CCL::DBG - causes a special exception -to be generated and enters the lisp kernel debugger with a non-null -"context": - ? (defun classify2 (n) - (cond ((> n 0) "Greater") - ((< n 0) "Less") + CCL::BUG just does an FF-CALL into the lisp kernel. If + the kernel debugger was invoked because of an unhandled + exception (such as an illegal memory reference) the OS kernel + saves the machine state ("context") in a data structure for us, + and in that case some additional options can be used to display + the contents of the registers at the point of the + exception. Another function - CCL::DBG - causes a special + exception to be generated and enters the lisp kernel debugger + with a non-null "context": + +? (defun classify2 (n) + (cond ((> n 0) "Greater") + ((< n 0) "Less") (t (dbg n)))) CLASSIFY2 @@ -10795,5 +15282,5 @@ ? (classify2 0) Lisp Breakpoint - While executing: # + While executing: #<Function CLASSIFY2 #x08476cfe> ? for help [12345] OpenMCL kernel debugger: ? @@ -10817,16 +15304,16 @@ rnil = 0x01836015 nargs = 0 -r16 (fn) = # +r16 (fn) = #<Function CLASSIFY2 #x30379386> r23 (arg_z) = 0 r22 (arg_y) = 0 r21 (arg_x) = 0 -r20 (temp0) = #<26-element vector subtag = 2F @#x303793ee> +r20 (temp0) = #<26-element vector subtag = 2F @#x303793ee> r19 (temp1/next_method_context) = 6393788 -r18 (temp2/nfn) = # +r18 (temp2/nfn) = #<Function CLASSIFY2 #x30379386> r17 (temp3/fname) = CLASSIFY2 r31 (save0) = 0 r30 (save1) = *TERMINAL-IO* r29 (save2) = 0 -r28 (save3) = (# #) +r28 (save3) = (#<RESTART @#x01867f2e> #<RESTART @#x01867f56>) r27 (save4) = () r26 (save5) = () @@ -10847,44 +15334,47 @@ - Using AltiVec in OpenMCL LAP functions + Using AltiVec in OpenMCL LAP functions - Overview -It's now possible to use AltiVec instructions in PPC LAP -(assembler) functions. - The lisp kernel detects the presence or absence of AltiVec and -preserves AltiVec state on lisp thread switch and in response to -exceptions, but the implementation doesn't otherwise use vector -operations. - This document doesn't document PPC LAP programming in general. -Ideally, there would be some document that did. - This document does explain AltiVec register-usage conventions in -OpenMCL and explains the use of some lap macros that help to enforce those -conventions. - All of the global symbols described below are exported from the CCL -package. Note that lap macro names, ppc instruction names, and (in most -cases) register names are treated as strings, so this only applies to -functions and global variable names. - Much of the OpenMCL support for AltiVec LAP programming is based on -work contributed to MCL by Shannon Spires. + Overview + It's now possible to use AltiVec instructions in PPC LAP + (assembler) functions. + The lisp kernel detects the presence or absence of + AltiVec and preserves AltiVec state on lisp thread switch and + in response to exceptions, but the implementation doesn't + otherwise use vector operations. + This document doesn't document PPC LAP programming in + general. Ideally, there would be some document that + did. + This document does explain AltiVec register-usage + conventions in OpenMCL and explains the use of some lap macros + that help to enforce those conventions. + All of the global symbols described below are exported + from the CCL package. Note that lap macro names, ppc + instruction names, and (in most cases) register names are + treated as strings, so this only applies to functions and + global variable names. + Much of the OpenMCL support for AltiVec LAP programming + is based on work contributed to MCL by Shannon Spires. - Register usage conventions -OpenMCL LAP functions that use AltiVec instructions must -interoperate with each other and with C functions; that suggests that they -follow C AltiVec register usage conventions. (vr0-vr1 scratch, vr2-vr13 -parameters/return value, vr14-vr19 temporaries, vr20-vr31 callee-save -non-volatile registers.) - The EABI (Embedded Application Binary Interface) used in LinuxPPC -doesn't ascribe particular significance to the vrsave special-purpose -register; on other platforms (notably MacOS), it's used as a bitmap -which indicates to system-level code which vector registers contain -meaningful values. - The WITH-ALTIVEC-REGISTERS lapmacro generates code which which -saves, updates, and restores VRSAVE on platforms where this is required -(as indicated by the value of the special variable which controls this) -and ignores VRSAVE on platforms that don't require it to be -maintained. + Register usage conventions + OpenMCL LAP functions that use AltiVec instructions must + interoperate with each other and with C functions; that + suggests that they follow C AltiVec register usage + conventions. (vr0-vr1 scratch, vr2-vr13 parameters/return + value, vr14-vr19 temporaries, vr20-vr31 callee-save + non-volatile registers.) + The EABI (Embedded Application Binary Interface) used in + LinuxPPC doesn't ascribe particular significance to the vrsave + special-purpose register; on other platforms (notably MacOS), + it's used as a bitmap which indicates to system-level code + which vector registers contain meaningful values. + The WITH-ALTIVEC-REGISTERS lapmacro generates code which + which saves, updates, and restores VRSAVE on platforms where + this is required (as indicated by the value of the special + variable which controls this) and ignores VRSAVE on platforms + that don't require it to be maintained. On all PPC platforms, it's necessary to save any non-volatile vector registers (vr20 .. vr31) before assigning to them and to restore @@ -10932,440 +15422,262 @@ - Development-Mode Dictionary - - - *WARN-IF-REDEFINE-KERNEL* - *warn-if-redefine-kernel - Name - *WARN-IF-REDEFINE-KERNEL* — - Variable - Description - When true, attempts to redefine (via DEFUN or DEFMETHOD) -functions and methods that are marked as being -"predefined" signal continuable errors. - Note that these are CERRORs, not warnings, and that -no lisp functions or methods have been defined in the kernel -in MCL or OpenMCL since 1987 or so. - - - - SET-DEVELOPMENT-ENVIRONMENT - set-development-environment - Name - SET-DEVELOPMENT-ENVIRONMENT — - Function - Synopsis - -set-development-environment - &optional - unmark-builtin-functions - - Description - Arranges that the outermost special bindings of *PACKAGE* -and *WARN-IF-REDEFINE-KERNEL* restore values of the "CCL" -package and NIL to these variables, respectively. If the optional -argument is true, marks all globally defined functions and methods -as being "not predefined" (this is a fairly expensive -operation.) - - - - SET-USER-ENVIRONMENT - set-user-environment - Name - SET-USER-ENVIRONMENT — - Function - Synopsis - -set-user-environment - &optional mark-builtin-functions - - Description - Arranges that the outermost special bindings of *PACKAGE* -and *WARN-IF-REDEFINE-KERNEL* restore values of the -"CL-USER" package and T to these variables, respectively. -If the optional argument is true, marks all globally defined -functions and methods as being "predefined" (this is a -fairly expensive operation.) - - - - *ALTIVEC-AVAILABLE* - *altivec-available* - Name - *ALTIVEC-AVAILABLE* — - Variable - Description - This variable is intitialized each time an OpenMCL session -starts based on information provided by the lisp kernel. Its value -is true if AltiVec is present and false otherwise. This variable -shouldn't be set by user code. - - - - ALTIVEC-AVAILABLE-P - altivec-available-p - Name - ALTIVEC-AVAILABLE-P — - Function - Synopsis - -altivec-available-p - - Description - Returns non-NIL if AltiVec is available. - - - - *ALTIVEC-LAPMACROS-MAINTAIN-VRSAVE-P* - *altivec-lapmacros-maintain-vrsave-p* - Name - *ALTIVEC-LAPMACROS-MAINTAIN-VRSAVE-P* — - Variable - Description - Intended to control the expansion of certain lap macros. -Initialized to NIL on LinuxPPC; initialized to T on platforms -(such as MacOS X/Darwin) that require that the VRSAVE SPR contain -a bitmask of active vector registers at all times. - - - - WITH-ALTIVEC-REGISTERS - with-altivec-registers - Name - WITH-ALTIVEC-REGISTERS — - LAP Macro - Synopsis - -with-altivec-registers - reglist &body body - - Arguments and Values - reglist - A list of vector register names (vr0 .. vr31). - body - A sequence of PPC LAP instructions. - - - Description - Specifies the set of AltiVec registers used in body. If -*altivec-lapmacros-maintain-vrsave-p* is true when the macro is -expanded, generates code to save the VRSAVE SPR and updates VRSAVE -to incude a bitmask generated from the specified register list. -Generates code which saves any non-volatile vector registers which -appear in the register list, executes body, and restores the saved -non-volatile vector registers (and, if -*altivec-lapmacros-maintain-vrsave-p* is true, restores VRSAVE as -well. Uses the IMM0 register (r3) as a temporary. - - - - WITH-VECTOR-BUFFER - with-vector-buffer - Name - WITH-VECTOR-BUFFER — - LAP Macro - Synopsis - -with-vector-buffer base n &body body - - Arguments and Values - base - Any available general-purpose register. - n - An integer between 1 and 254, inclusive. (Shouldtypically be much, much closer to 1.) Specifies the size ofthe buffer, in 16-byte units. - body - A sequence of PPC LAP instructions. - - - Description - Generates code which allocates a 16-byte aligned buffer -large enough to contain N vector registers; the GPR base points to -the lowest address of this buffer. After processing body, the -buffer will be deallocated. The body should preserve the value of -base as long as it needs to reference the buffer. It's -intended that base be used as a base register in stvx and lvx -instructions within the body. - + Development-Mode Dictionary + + + + *warn-if-redefine-kernel + + + + *WARN-IF-REDEFINE-KERNEL* + + Variable + + + + Description + + When true, attempts to redefine (via DEFUN or DEFMETHOD) + functions and methods that are marked as being + "predefined" signal continuable errors. + + Note that these are CERRORs, not warnings, and that + no lisp functions or methods have been defined in the kernel + in MCL or OpenMCL since 1987 or so. + + + + + + set-development-environment + + + + SET-DEVELOPMENT-ENVIRONMENT + + Function + + + + set-development-environment + &optional; + unmark-builtin-functions + + + + Description + + Arranges that the outermost special bindings of *PACKAGE* + and *WARN-IF-REDEFINE-KERNEL* restore values of the "CCL" + package and NIL to these variables, respectively. If the optional + argument is true, marks all globally defined functions and methods + as being "not predefined" (this is a fairly expensive + operation.) + + + + + + set-user-environment + + + + SET-USER-ENVIRONMENT + + Function + + + + set-user-environment + &optional; mark-builtin-functions + + + + Description + + Arranges that the outermost special bindings of *PACKAGE* + and *WARN-IF-REDEFINE-KERNEL* restore values of the + "CL-USER" package and T to these variables, respectively. + If the optional argument is true, marks all globally defined + functions and methods as being "predefined" (this is a + fairly expensive operation.) + + + + + *altivec-available* + + + + *ALTIVEC-AVAILABLE* + + Variable + + + + Description + This variable is intitialized each time an OpenMCL session + starts based on information provided by the lisp kernel. Its value + is true if AltiVec is present and false otherwise. This variable + shouldn't be set by user code. + + + + + + altivec-available-p + + + + ALTIVEC-AVAILABLE-P + + Function + + + + altivec-available-p + + + + Description + + Returns non-NIL if AltiVec is available. + + + + + + *altivec-lapmacros-maintain-vrsave-p* + + + + *ALTIVEC-LAPMACROS-MAINTAIN-VRSAVE-P* + + Variable + + + + Description + + Intended to control the expansion of certain lap macros. + Initialized to NIL on LinuxPPC; initialized to T on platforms + (such as MacOS X/Darwin) that require that the VRSAVE SPR contain + a bitmask of active vector registers at all times. + + + + + + with-altivec-registers + + + + WITH-ALTIVEC-REGISTERS + + LAP Macro + + + + with-altivec-registers + reglist &body; body + + + + Arguments and Values + + + + reglist + + + A list of vector register names (vr0 .. vr31). + + + + + body + + + A sequence of PPC LAP instructions. + + + + + + + Description + + Specifies the set of AltiVec registers used in body. If + *altivec-lapmacros-maintain-vrsave-p* is true when the macro is + expanded, generates code to save the VRSAVE SPR and updates VRSAVE + to incude a bitmask generated from the specified register list. + Generates code which saves any non-volatile vector registers which + appear in the register list, executes body, and restores the saved + non-volatile vector registers (and, if + *altivec-lapmacros-maintain-vrsave-p* is true, restores VRSAVE as + well. Uses the IMM0 register (r3) as a temporary. + + + + + + with-vector-buffer + + + + WITH-VECTOR-BUFFER + + LAP Macro + + + + with-vector-buffer base n &body; body + + + + Arguments and Values + + + + base + + + Any available general-purpose register. + + + + + n + + + An integer between 1 and 254, inclusive. (Should + typically be much, much closer to 1.) Specifies the size of + the buffer, in 16-byte units. + + + + + body + + + A sequence of PPC LAP instructions. + + + + + + + Description + Generates code which allocates a 16-byte aligned buffer + large enough to contain N vector registers; the GPR base points to + the lowest address of this buffer. After processing body, the + buffer will be deallocated. The body should preserve the value of + base as long as it needs to reference the buffer. It's + intended that base be used as a base register in stvx and lvx + instructions within the body. + + - Symbol Index - - - #_, see - - %ff-call, see - - %reference-external-entry-point, see - - *alternate-line-terminator*, see - - *altivec-available*, see - - *altivec-lapmacros-maintain-vrsave-p*, see - - *current-process*, see - - *default-external-format*, see - - *ticks-per-second*, see - - *warn-if-redefine-kernel, see - - :external-format, see - - :y, see - - @class, see - - @selector, see -A - - accept-connection, see - - all-processes, see - - altivec-available-p, see -C - - close, see - - close-shared-library, see - - configure-egc, see - - current-directory-name, see -D - - def-foreign-type, see - - defcallback, see - - define-objc-class-method, see - - define-objc-method, see - - dotted-to-ipaddr, see -E - - egc, see - - egc-active-p, see - - egc-configuration, see - - egc-enabled-p, see - - external, see - - external-call, see - - external-process-error-stream, see - - external-process-id, see - - external-process-input-stream, see - - external-process-output-stream, see - - external-process-status, see -F - - ff-call, see - - foreign-symbol-address, see - - foreign-symbol-entry, see -G - - gc-retain-pages, see - - gc-retaining-pages, see - - get-user-home-dir, see - - getenv, see - - getpid, see - - getuid, see - - grab-lock, see -I - - ipaddr-to-dotted, see - - ipaddr-to-hostname, see -L - - lisp-heap-gc-threshold, see - - local-host, see - - local-port, see - - lookup-hostname, see - - lookup-port, see -M - - make-lock, see - - make-process, see - - make-read-write-lock, see - - make-record, see - - make-semaphore, see - - make-socket, see -N - - ns-lisp-string, see -O - - open-shared-library, see - - os-command, see -P - - pref, see - - process-abort, see - - process-allow-schedule, see - - process-enable, see - - process-input-wait, see - - process-interrupt, see - - process-kill, see - - process-output-wait, see - - process-preset, see - - process-reset, see - - process-resume, see - - process-run-function, see - - process-suspend, see - - process-suspend-count, see - - process-wait, see - - process-wait-with-timeout, see - - process-whostate, see -R - - receive-from, see - - release-lock, see - - remote-host, see - - remote-port, see - - request-terminal-input-via-break, see - - rlet, see - - rletz, see - - run-program, see -S - - send-to, see - - set-development-environment, see - - set-lisp-heap-gc-threshold, see - - set-user-environment, see - - setenv, see - - setgid, see - - setuid, see - - SHARP-AMPERSAND, see - - shutdown, see - - signal-external-process, see - - signal-semaphore, see - - socket-address-family, see - - socket-connect, see - - socket-error, see - - socket-error-code, see - - socket-error-identifier, see - - socket-error-situation, see - - socket-format, see - - socket-os-fd, see - - socket-type, see - - stream-device, see - - stream-read-ivector, see - - stream-read-list, see - - stream-read-vector, see - - stream-write-ivector, see - - stream-write-list, see - - stream-write-vector, see -T - - terminate-when-unreachable, see - - timed-wait-on-semaphore, see - - try-lock, see -U - - unuse-interface-dir, see - - use-interface-dir, see - - use-lisp-heap-gc-threshold, see -W - - wait-on-semaphore, see - - with-altivec-registers, see - - with-lock-grabbed, see - - with-open-socket, see - - with-read-lock, see - - with-terminal-input, see - - with-vector-buffer, see - - with-write-lock, see - - without-interrupts, see + + Symbol Index -