wiki:WindowsNotes

Version 23 (modified by dcooper8, 11 months ago) (diff)

--

Notes on installing and using Clozure CL on Windows

Installation

The files that make up a Clozure CL distribution for a "platform" (an OS/CPU architecture combination) are contained in a directory that's canonically called "ccl". (You can rename this if you want to, but it's more concise and convenient to refer to this as "the CCL directory" than to use a longer term.) The ccl directory for Windows contains:

  • wx86cl.exe - the kernel for the 32-bit implementation
  • wx86cl.image - a heap image for the 32-bit implementation
  • wx86cl64.exe, wx86cl64.image - the 64-bit versions of the above
  • a directory named "win32-headers", which contains database files that describe win32 foreign functions, constants, and data structures
  • a directory named "win64-headers", which contain 64-bit FFI info
  • several other directories containing sources, examples, and auxiliary files.

The canonical versions of these files are maintained in a "subversion repository"; subversion (aka "svn") is a revision control system (also known as a "source code control system.") We use svn both as a way of maintaining and tracking changes to the CCL sources and as a way of distributing CCL sources and binaries.

An svn repository can be accessed via an "http" URL and it's possible to access individual files by downloading them in a web browser, but it's generally not desirable to do so. It's preferable to use an svn client program to obtain a "working copy" of a directory from the repository. This will ensure that you get all of the distribution (sources, binaries, interfaces, and documentation) and will allow you to periodically update your working copy to obtain bug fixes and other changes.

So:

1) get a subversion (svn) client for Windows. There are several choices available;  http://subversion.apache.org/packages.html#windows lists a number of them. SlikSVN has worked well for me (rme).

If you're not comfortable with the Windows command-line, you may prefer TortoiseSVN (< http://tortoisesvn.tigris.org>), which adds svn-related entries to the context menu (right-click) in Windows Explorer. There are some things in the lisp build process that work better if a command-line version of svn is installed, but if you're shell-averse you might find it easiest to get started with something like TortoiseSVN and switch to a command-line client after you're more comfortable with the general idea of using svn.)

If you use Cygwin ( http://www.cygwin.com), you'll find that the Cygwin svn client generally works fine.

There are several other options available. Some GUI svn clients don't understand an svn feature called "svn:externals properties", and the CCL repository makes heavy use of this feature. The clients listed above do understand this.

2) read

 http://trac.clozure.com/openmcl/wiki/TrackingTheTrunk

3) check out the windows version of CCL by following the instructions on that page. The "PLATFORM" in question is called "windows", so the command to check out a fresh copy of the current release would be:

shell> svn co http://svn.clozure.com/publicsvn/openmcl/release/1.7/windows/ccl 

and the trunk (day-to-day development version) can be obtained from a slightly different URL:

shell> svn co http://svn.clozure.com/publicsvn/openmcl/trunk/windows/ccl

either of which would create and populate a directory called "ccl".

In tortoisesvn, right-clicking on the desktop will activate a popup menu with an "SVN Checkout ..." entry. Choose this entry; in the resulting dialog, enter the URL above as "URL of repository", select a directory where you want to keep the "ccl" directory (it'll be created if it doesn't exist when you start the checkout), leave other options with default values (checkout depth = fully recursive, "omit externals" unchecked, HEAD revision.) Click the "OK" button when ready; that'll confirm the creation of the CCL directory and then start downloading the distribution.)

If you later right-click on the CCL directory that was created above, the popup menu will contain an additional entry named "SVN Update" and a "TortoiseSVN" submenu which contains entries for many less-frequently-used options. "SVN Update" corresponds (not suprisingly) to the shell

shell> svn update

command described in the wiki page. However it's invoked, "svn update" checks to see if newer versions of files are available in the repository and downloads them into your working copy if so. The wiki page describes some of the issues related to "svn update" when local copies of files have been modified since they were last obtained from the repository.

As the wiki page also notes: we don't usually update the binaries every time a small change is made to the source; it's assumed that you can (usually) rebuild the lisp image yourself. The wiki page recommends doing that via:

? (rebuild-ccl :full t)

which would rebuild the lisp kernel, delete any FASL files, recompile all lisp sources, and build a new heap image. This is not so straightforward on Windows. In order to rebuild the kernel (wx86cl.exe or wx86cl64.exe) the GNU toolchain (Cygwin or MinGW), must be present on the path (see below.)

Additionally, because of Windows file-sharing issues it's not possible to overwrite an executable file (.exe) while it's running, so on Windows, trying to use the :FULL T option to REBUILD-CCL will fail (even though it may appear to have worked). You can do everything that the :FULL T option does besides rebuilding the kernel via:

? (rebuild-ccl :clean t)

You can rebuild the lisp kernel by hand by doing:

$ cd lisp-kernel/win32 # or win64
$ make clean && make

Since it's likely that relatively few users will have the GNU toolchain for Windows installed, we're probably a little better about keeping the kernel up-to-date (e.g., we try to check in an updated wx86cl(64).exe soon after a significant change is made to its sources). As is noted on the wiki page above, rebuilding the lisp image from its sources should usually work, but there may be some warnings and continuable errors (when the value of a constant changes) during the build process.

REBUILD-CCL tries to obtain the current svn version (by using RUN-PROGRAM to run "svnversion" and capturing its output) and stores this version number in the value returned by LISP-IMPLEMENTATION-VERSION (it's also printed in the Welcome! banner.) If the "svnversion" program doesn't exist (or can't be found on the shell search path), the version string will just be the emtpy string (and there won't be anything too useful in LISP-IMPLEMENTATION-VERSION.)

Windows-specific issues

The 64-bit Windows port is probably more mature than the 32-bit Windows port. It only runs on 64-bit versions of Windows; it's been tested/used some on XP64 and quite a bit more extensively on 64-bit Vista and 64-bit Win7,

The 32-bit port should run on 32-bit versions of XP and Vista. (For obscure reasons, the 32-bit port didn't run on 64-bit OS versions prior to CCL version 1.7)

Windows pathnames can use either forward-slash or backward-slash characters as directory separators. As of the 1.3 release, CCL should handle namestrings which use either forward- or backward-slashes; some prereleases and release-candidates generally had difficulty with backslashes.

Windows namestrings generally contain a "drive letter" preceded by a colon; the drive letter of a pathname derived from such a namestring is accessible via PATHNAME-DEVICE.

The value returned by USER-HOMEDIR-PATHNAME depends on OS version. The "home" logical host has its translations set up so that "home:" references this directory, and the lisp will try to load "home:ccl-init" (as a lisp or fasl file) on startup, just as on other platforms.

Windows example

There's a very simple (but heavily commented) example program in "ccl:examples;mswin.lisp". Doing:

? (require "MSWIN")

and then doing

? (cl-user::make-simple-ms-window)

will create a (very) simple window and process events related to that window until its close box is pressed. You might prefer to have the window created (and the event loop run in) a separate thread:

? (process-run-function "window" #'cl-user::make-simple-ms-window)

The thrill of seeing a window appear onscreen will likely wear off pretty quickly; a window that contains interesting content or which has interesting behavior might be less boring.

Building the lisp kernel on Windows

The Windows kernel Makefiles (in ccl/lisp-kernel/win32 and ccl/lisp-kernel/win64) assume that the Cygwin tools ( http://www.cygwin.com) are installed. (It's likely that small changes to the Makefiles would allow the use of the MinGW ( http://www.mingw.org) tools instead.)

A full discussion of Cygwin installation and configuration issues is *way* beyond the scope of this document. Fortunately, the net is full of helpful and accurate documents that discuss these issues in great detail. For the sake of argument, let's assume that Cygwin is installed in C:\cygwin (which is the default) and that it's configured to use Unix-style line-termination.

Unfortunately, the compiler situation on Cygwin has changed around a bit, so this makes matters somewhat complicated.

Trunk and Current Cygwin

Go to  http://cygwin.com and download the setup.exe program. Install the following packages:

  • make
  • m4

If you want to build the 32-bit Windows lisp kernel, also install:

  • mingw64-i686-binutils
  • mingw64-i686-gcc-core
  • mingw64-i686-headers
  • mingw64-i686-pthreads
  • mingw64-i686-runtime

If you want to build the 64-bit Windows lisp kernel, also install:

  • mingw64-x86_64-binutils
  • mingw64-x86_64-gcc-core
  • mingw64-x86_64-headers
  • mingw64-x86_64-pthreads
  • mingw64-x86_64-runtime

Accept the dependencies (a handful of libraries) that the installation tool adds. Note that you can install all of these packages if you want to build both 32- and 64-bit lisp kernels.

Older Cygwin (used in ccl 1.7 and earlier)

For both the win32 and win64 platforms, it's necessary to install the following Cygwin packages (if you notice any dependencies not listed here, please edit this page or let me (gb) know.)

  • make (Unix "make" program)
  • m4 (m4 macro processor)

You'll probably find life easier if you also install standard utilities ('coreutils', 'fileutils', 'tar', ...) but these things aren't used by the build process.

To build the win32 kernel, you'd additionally need:

  • gcc-core (32-bit C compiler)
  • gcc-mingw (mingw integration)
  • gcc-mingw-core
  • mingw-runtime (static C runtime library)
  • w32-api (win32 headers)
  • binutils (assembler/linker/object file utilities.)

64-bit toolchains for the Cygwin/MinGW environments are available from the Mingw-w64 project ( http://sourceforge.net/projects/mingw-w64/). In the "Downloads" section of that site, find a "complete toolchain snapshot" for Cygwin and download the compressed tar archive (it's largish - probably > 70MB). In a Cygwin shell, extract the the tar archive into /usr :

shell> tar xvf mingw-w64-bin-i686-cygwin-VERSION-YYYYMMDD.tar.bz2 -C /usr

This archive contains everything that you need (64-bit toolchain, libraries, headers) to build the lisp kernel for win64. (You still need the 'make' and 'm4' packages.)

Important note: If you're running the 1.6 release or earlier, newer versions of this toolchain won't work to compile the lisp kernel. In this case, please use  ftp://ftp.clozure.com/pub/mingw-w64-bin_i686-cygwin-1.5.25-15_4.4.1a.tar.bz2 which is known to work. (It will still work on cygwin 1.7.)

Note that recent versions of Cygwin come with gcc-4 as well. If you install both, gcc-3 and gcc-4, compilation of the kernel will fail with gcc complaining that "the -mno-cygwin flag has been removed". Switch to gcc-3 by adjusting /etc/alternatives:

shell> ln -fs `which gcc-3` /etc/alternatives/gcc

Actually building a Windows kernel

Ensure that no copies of the lisp are running. (Windows doesn't like to overwrite an executable file while it's in use.)

From a Cygwin shell, cd to .../ccl/lisp-kernel/win32 (or win64) and do:

shell> make

If all of the tools/headers/libraries are installed, that should create a working kernel executable in the ccl directory.