|Version 8 (modified by gb, 8 years ago) (diff)|
Notes on installing and using Clozure CL on Windows
The files that make up a Clozure CL distribution for a "platform" (an OS/CPU architecture combination) are contained in a directory that's canonically called "ccl". (You can rename this if you want to, but it's more concise and convenient to refer to this as "the CCL directory" than to use a longer term.) The ccl directory for Windows contains:
- wx86cl.exe - the kernel for the 32-bit implementation
- wx86cl.image - a heap image for the 32-bit implementation
- wx86cl64.exe, wx86cl64.image - the 64-bit versions of the above
- a directory named "win32-headers", which contains database files that describe win32 foreign functions, constants, and data structures
- a directory named "win64-headers", which contain 64-bit FFI info
- several other directories containing sources, examples, and auxiliary files.
The canonical versions of these files are maintained in a "subversion repository"; subversion (aka "svn") is a revision control system (also known as a "source code control system.") We use svn both as a way of maintaining and tracking changes to the CCL sources and as a way of distributing CCL sources and binaries.
An svn repository can be accessed via an "http" URL and it's possible to access individual files by downloading them in a web browser, but it's generally not desirable to do so. It's preferable to use an svn client program to obtain a "working copy" of a directory from the repository. This will ensure that you get all of the distribution (sources, binaries, interfaces, and documentation) and will allow you to periodically update your working copy to obtain bug fixes and other changes.
1) get a subversion (svn) client for Windows. There are several choices available; the "certified binaries" available at
should work well.
If you're not comfortable with the Windows command-line, you may prefer TortoiseSVN (< http://tortoisesvn.tigris.org>), which adds svn-related entries to the context menu (right-click) in Windows Explorer. There are some things in the lisp build process that work better if a command-line version of svn is installed, but if you're shell-averse you might find it easiest to get started with something like TortoiseSVN and switch to a command-line client after you're more comfortable with the general idea of using svn.)
If you use Cygwin ( http://www.cygwin.com), you'll find that the Cygwin svn client generally works fine.
There are several other options available. Some GUI svn clients don't understand an svn feature called "svn:externals properties", and the CCL repository makes heavy use of this feature. The clients listed above do understand this.
3) check out the windows version of CCL by following the instructions on that page. The "PLATFORM" in question is called "windows", so the command to check out a fresh copy would be:
shell> svn co http://svn.clozure.com/publicsvn/openmcl/trunk/windows/ccl
which would create and populate a directory called "ccl".
In tortoisesvn, right-clicking on the desktop will activate a popup menu with an "SVN Checkout ..." entry. Choose this entry; in the resulting dialog, enter the URL above as "URL of repository", select a directory where you want to keep the "ccl" directory (it'll be created if it doesn't exist when you start the checkout), leave other options with default values (checkout depth = fully recursive, "omit externals" unchecked, HEAD revision.) Click the "OK" button when ready; that'll confirm the creation of the CCL directory and then start downloading the distribution.)
If you later right-click on the CCL directory that was created above, the popup menu will contain an additional entry named "SVN Update" and a "TortoiseSVN" submenu which contains entries for many less-frequently-used options. "SVN Update" corresponds (not suprisingly) to the shell
shell> svn update
command described in the wiki page. However it's invoked, "svn update" checks to see if newer versions of files are available in the repository and downloads them into your working copy if so. The wiki page describes some of the issues related to "svn update" when local copies of files have been modified since they were last obtained from the repository.
As the wiki page also notes: we don't usually update the binaries every time a small change is made to the source; it's assumed that you can (usually) rebuild the lisp image yourself. The wiki page recommends doing that via:
? (rebuild-ccl :full t)
which would rebuild the lisp kernel, delete any FASL files, recompile all lisp sources, and build a new heap image. Unfortunately - because of Windows file-sharing issues - it's not possible to overwrite an executable file (.exe) while it's running, so the :FULL T option to REBULD-CCL will fail. You can do everything that the :FULL T option does besides rebuilding the kernel via:
? (rebuild-ccl :clean t)
Since it's likely that relatively few users will have the GNU toolchain for Windows installed, we're probably a little better about keeping the kernel up-to-date (e.g., we try to check in an updated wx86cl(64).exe soon after a significant change is made to its sources). As is noted on the wiki page above, rebuilding the lisp image from its sources should usually work, but there may be some warnings and continuable errors (when the value of a constant changes) during the build process.
REBUILD-CCL tries to obtain the current svn version (by using RUN-PROGRAM to run "svnversion" and capturing its output) and stores this version number in the value returned by LISP-IMPLEMENTATION-VERSION (it's also printed in the Welcome! banner.) If the "svnversion" program doesn't exist (or can't be found on the shell search path), the version string will just be the emtpy string (and there won't be anything too useful in LISP-IMPLEMENTATION-VERSION.)
The 64-bit Windows port is probably more mature than the 32-bit Windows port. It only runs on 64-bit versions of Windows; it's been tested/used some on XP64 and quite a bit more extensively on 64-bit Vista. (Don't laugh: as of this writing, a large percentage of new Windows boxes sold at retail outlets - like BestBuy in the US - come with 64-bit versions of Vista installed. It's not clear if "a large percentage" is "a majority" yet, but it's much more common than it was even 6 months ago.)
The 32-bit port should run on 32-bit versions of XP and Vista. (For obscure reasons, the 32-bit port doesn't run on 64-bit OS versions at this point. We intend to address that, if only to avoid having to explain that it doesn't and why it doesn't.)
Windows pathnames can use either forward-slash or backward-slash characters as directory separators. CCL translates backslashes to forward slashes on namestrings returned by the OS and generally doesn't interpret backslashes as directory separators in other contexts.
Windows namestrings generally contain a "drive letter" preceded by a colon; the drive letter of a pathname derived from such a namestring is accessible via PATHNAME-DEVICE.
The value returned by USER-HOMEDIR-PATHNAME depends on OS version. The "home" logical host has its translations set up so that "home:" references this directory, and the lisp will try to load "home:ccl-init" (as a lisp or fasl file) on startup, just as on other platforms.
As of this writing, there are still general stability issues that I think are more severe on win32. At least one of them has to do with cases where thread "A" stops or suspends thread "B", accesses and/or modifies thread "B"'s state in some way, and then allows thread "B" to resume execution. (There are cases where B is in the middle of modifying its own state when it's stopped, and issues with recognizing and safely handling those cases.)
The good news is that the troublesome cases occur relatively rarely, ordinarily.
The bad news is that a fairly common function - PROCESS-WHOSTATE, which obtains a string which describes the state of a thread - needs to stop/start the target thread in order to access this state reliably.
The good news is that PROCESS-WHOSTATE is ordinarily only used for debugging - the :PROC toplevel command and when printing a PROCESS object.
The bad news is that SLIME does some sort of transaction logging by default, and the information that it logs includes (the printed representation of) all lisp processes involved in transactions between Emacs and the lisp. (In other words, running under SLIME tends to exacerbate the thread suspend/resume bug[s].)
The good news is that there's a temporary workaround: when a PROCESS object is printed, its PROCESS-WHOSTATE is only included in its printed representation if the value of CCL::*PRINT-PROCESS-WHOSTATE* is non-NIL. (The bad news is that this variable defaults to T for backward compatibility; the good news is that you can set it to NIL in "home:ccl-init" and therefore decrease your chances of running into the suspend/resume bug when running under SLIME.)
There's a very simple (but heavily commented) example program in "ccl:examples;mswin.lisp". Doing:
? (require "MSWIN")
and then doing
will create a (very) simple window and process events related to that window until its close box is pressed. You might prefer to have the window created (and the event loop run in) a separate thread:
? (process-run-function "window" #'cl-user::make-simple-ms-window)
The thrill of seeing a window appear onscreen will likely wear off pretty quickly; a window that contains interesting content or which has interesting behavior might be less boring.
Building the lisp kernel on Windows
The Windows kernel Makefiles (in ccl/lisp-kernel/win32 and ccl/lisp-kernel/win64) assume that the Cygwin tools ( http://www.cygwin.com) are installed. (It's likely that small changes to the Makefiles would allow the use of the MinGW ( http://www.mingw.org) tools instead.)
A full discussion of Cygwin installation and configuration issues is *way* beyond the scope of this document. Fortunately, the net is full of helpful and accurate documents that discuss these issues in great detail. For the sake of argument, let's assume that Cygwin is installed in C:\cygwin (which is the default) and that it's configured to use Unix-style line-termination.
For both the win32 and win64 platforms, it's necessary to install the following Cygwin packages (if you notice any dependencies not listed here, please edit this page or let me (gb) know.)
- make (Unix "make" program)
- m4 (m4 macro processor)
You'll probably find life easier if you also install standard utilities ('coreutils', 'fileutils', 'tar', ...) but these things aren't used by the build process.
To build the win32 kernel, you'd additionally need:
- gcc-core (32-bit C compiler)
- gcc-mingw (mingw integration)
- mingw-runtime (static C runtime library)
- w32-api (win32 headers)
- binutils (assembler/linker/object file utilities.)
64-bit toolchains for the Cygwin/MinGW environments are available from the Mingw-w64 project ( http://sourceforge.net/projects/mingw-w64/). In the "Downloads" section of that site, find a "complete toolchain snapshot" for Cygwin and download the compressed tar archive (it's largish - probably > 70MB). In a Cygwin shell, extract the the tar archive into /usr :
shell> tar xvf mingw-w64-bin-i686-cygwin-VERSION-YYYYMMDD.tar.bz2 -C /usr
This archive contains everything that you need (64-bit toolchain, libraries, headers) to build the lisp kernel for win64. (You still need the 'make' and 'm4' packages.)
Actually building a Windows kernel
Ensure that no copies of the lisp are running. (Windows doesn't like to overwrite an executable file while it's in use.)
From a Cygwin shell, cd to .../ccl/lisp-kernel/win32 (or win64) and do:
If all of the tools/headers/libraries are installed, that should create a working kernel executable in the ccl directory. There are still (as of November 2008) some warnings in the win32 build that are intended to serve as reminders that some cases of thread interrupt/suspend/resume still aren't handled correctly; the win64 kernel should generally build without warnings.