Cocoa IDE Implementation Notes

The information below is obsolete as of rev 8428. New documentation should be coming soon.

This is an attempt to describe the overall architecture of OpenMCL's Cocoa IDE.

AppKit and threads

There are differences of opinion as to whether or not AppKit (the GUI parts of Cocoa) is thread-safe.  Apple's take on the issue may strike many people as being a bit inconclusive. A few things that I think can be safely said include:

  • Low-level communication with the window server - the basis for event-processing and redisplay - works best if the event loop is run on the initial thread (the thread that was created by the OS when the process was run.) It's not clear that this communication works at all from other threads (that may depend on AppKit version), and it's clear that a scheme which allowed multiple threads to obtain events from the window server would introduce serialization issues.
  • Many AppKit methods only work if they're run on the main thread. (They may or may not check for this.)
  • The NSObject class provides methods - including #/performSelectorOnMainThread:withObject:waitUntilDone: - which can be used to ensure that a method invocation runs on the main thread.

Accordingly, OpenMCL's Cocoa IDE runs an event loop (-[NSApplication run]) on its initial thread; the event loop repeatedly fetches events and window update notifications from the window server, determines the objects to which these events and notifications apply, and calls methods on those objects. There are a few differences (the IDE actually subclasses NSApplication and overrides a few NSApplication methods; the reasons for this have to do with exception-handling arcana and with the way that PROCESS-INTERRUPT interrupts the event thread), but this {event loop running on the main thread) is typical of most AppKit applications. The atypical things about the IDE mostly have to do with how Hemlock is integrated into it.

As of r16524, if the GUI thread attempts to print to an IDE listener stream, that output will be dropped on the floor. This was a simplification to avoid having to deal with deadlock issues. Since the default is for the GUI thread to output to the AltConsole, or, if started from a command-line shell, to that shell, this shouldn't be a practical problem.


Hemlock was originally designed to run in a "single window" (or, in Emacs terminology, a "single frame"). At some points in its history, Hemlock has run in a cursor-addressible TTY window; at most points in its history, it's run in a single X window (under CLX.) Like most Emacs-like editors, Hemlock runs (user-definable) "command" functions in response to key events. Some of those commands (things like incremental search) may need to process subsequent key events, and the way that this was implemented in Hemlock involved blocking while waiting for a key event from the window system. This sort of modal behavior doesn't scale well to multi-window environments, and it's difficult to make it "work right" and keep the application as a whole responsive.

The Hemlock code base and event model can be adapted to a Cocoa-like environment if each window (Emacs "frame") is effectively given its own Hemlock command interpreter running in a separate thread. The main Cocoa event thread processes key events by posting them to a per-window queue; the Hemlock command interpreter waits for key events to appear on that queue and runs the associated command. If that command needs to recursively process key events, it can do so (by blocking on its window's queue) without interfering with application-level event processing.

This scheme simplifies some things (notably recursive event processing), but introduces a good deal of complexity in other areas.

The Cocoa Text System

AppKit provides a set of interrelated classes (NSTextView, NSTextStorage, NSTextContainer, NSLayoutManager, ...) which are often collectively referred to as "the text system". The text system provides a rich set of features for displaying and manipulating text; it even provides basic Emacs-like command processing (so basic Emacs editing features often work in text fields in Cocoa-based applications.) The built-in command processing doesn't provide for lisp-syntax-aware navigation, and some early attempts to use the built-in stuff for lisp editing weren't too successful: the object that represents the "buffer" being edited - an instance of NSTextStorage - is presented as a sequence of characters, and it's difficult to find ways to annotate that sequence (with marks and other data structures) in ways that make richer lisp-aware functionality available.

A few years ago, Mikel Evins pointed out that NSTextStorage is an abstract class (and itself a subclass of NSAttributedString), and that a concrete subclass of NSTextStorage need only implement a few methods (to provide the length of the abstract string and access to specific characters and their attributes) and that it was therefore possible to represent a Hemlock buffer as a subclass of NSTextStorage.

(For a variety of reasons - some of them performance-related - the current scheme maintains a "real" NSMutableAttributedString in parallel with the Hemlock-buffer-as-string representation; changes to the Hemlock buffer need to be reflected in the NSMutableAttributedString and vice versa.)

This scheme generally worked, and it allows the IDE to use many aspects of the Cocoa text system (layout and redisplay, mouse and selection handling, interaction with a containing scroll view) "for free." In this scheme, the Hemlock command thread associated with a text view's window processes key events and executes commands which modify a Hemlock buffer and the main event thread handles text layout and display issues. Of course, that scheme requires a certain amount of synchronization and coordination between threads, to insure that the display code recognizes changes to the buffer content and selection and to ensure that the display code never accesses the buffer while it's in the process of being modified.

The division of labor

The text system provides a mechanism by which redisplay and layout in a text view can be inhibited while modifications to the underlying NSTextStorage are being performed under program control; that mechanism involves bracketing those changes with calls to #/beginEditing and #/endEditing on the textstorage object. In the IDE, Hemlock encloses the code which processes an editor command with #/beginEditing/#/endEditing, and ends/begins editing in places where a key event is fetched recursively. (This means that ordinarily redisplay is inhibited during Hemlock command processing.) It's necessary - for things to be serialized reasonably - to ensure that the #/beginEditing/#/endEditing calls happen on the main thread.) While processing changes to the buffer that occur in this context, it's necessary to notify the text system of those changes and to ensure that they're reflected in the parallel NSMutableAttributedString. These notifications also have to occur on the main thread. Changes to the selection (movement of the Hemlock buffer's point, with or without an active mark that delineates a region) are processed (on the main thread) when the Hemlock command thread notices that no events are pending.

A Hemlock buffer is a doubly-linked list of lines, where each line is conceptually just a simple string and newline characters are implicitly present between lines. Like many editors, Hemlock tries to reduce the cost of a series of changes at a given position by maintaing a "gap" : a line being modified might be temporarily represented as a sequence of strings (constant data to the left and right of the change, a string used to process insertions) and indices into these strings. A line that was represented this way was said to be "open"; in the single-threaded single-window Hemlock world, a set of global variables identified that line and the related gap data structures. (The open line in Hemlock wasn't always in the "current buffer" and might not have been in any buffer.)

When processing the lines in a Hemlock buffer, it's necessary to know whether that line is "the open line" or not. If it were only necessary to know this in the Hemlock command thread associated with that buffer, it would be adequate to simply maintain that information ("the gap context") in a handful of per-thread special variables; however, that gap context information is also needed when the buffer is accessed from the event thread (in order to access character data for display purposes, in order to highlight matching parens, and in order to process mouse events that manage the selection.) Accordingly, each Hemlock buffer has an associated "gap context" structure, each Hemlock command thread has an associated special binding of the special variable HI::*BUFFER-GAP-CONTEXT*, and the half-dozen or so special variables which ordinarily maintained information about the open line are defined as symbol-macros that access the slots in the structure which is the current value of HI::*BUFFER-GAP-CONTEXT*. When accessing a buffer from some thread other than its Hemlock command thread (from the event thread, for instance), it's necessary to bind HI::*BUFFER-GAP-CONTEXT* to the value of the buffer's gap-context slot. (Yes, this is very ugly. It does seem to work.)


The division of labor described above tries to ensure that the buffer's contents can't be accessed (by layout or display code) while they're being modified. There may be some other cases (e.g., trying to change the selection via the mouse while an editor command is modifying the buffer) where race conditions are possible. It is probably viable to enforce locking protocols in such cases; it's not desirable to make the event thread block, but there may be other solutions (e.g., if the event thread can't set the selection because the buffer's locked, it could post something to the command thread that'll set the selection from the command thread.)