wiki:UpdatingFromSource

Version 10 (modified by rme, 6 years ago) (diff)

--

Updating from Subversion and rebuilding Clozure CL from source

This page describes how to update and rebuild Clozure CL from subversion ("svn"), starting with the most recent "release" tarball. ("Release tarballs" are distributed periodically in  ftp://clozure.com/pub/release/ ) Extracting the contents of a release tarball into a local "ccl" directory creates a "working copy" of the contents of the svn repository at the time that the release was made. (This "working copy" contains metainformation - in ".svn" directories - that enable svn to track changes.)

  1. In a shell, enter the "ccl" directory created when the tarball was extracted.
    shell> cd ccl
    
  1. Tell svn to update the sources, creating new directories and removing empty ones as needed.
    shell> svn update
    

This will try to reconcile changes made in the svn repository with the local, working copy. If you've made changes to files in the working copy and changes have been made in the repository, svn will attempt to merge those changes into an updated copy of the local file. It's often able to do so reasonably well (for text files); sometimes, the repository changes and local changes can't be resolved automatically and the affected file is said to be "in conflict." svn has no good way to even try to merge changes to binary files (like the CCL kernel and heap image).

As "svn update" reconciles changes to the repository with any changes made to the working copy, it prints a line consisting of a one-letter code, some whitespace, and the pathname of the affected item. These codes include:

A path The file or directory at path was added to the repository and has been added to the working copy
D path The file or directory at path was deleted from the repository and has been deleted from the working copy
U file The file at file was changed in the repository and the local copy has been updated
G file The file at file was changed in both the repository and working copy, and the local copy contains mixed content
C file The file at file was changed in both the repository and working copy, and svn was unable to resolve conflicts

Binary files that have been locally modified (the act of recompiling the kernel or rebuilding the image generally count as being "modified") generally can't be merged with newer versions from the repository. The lisp kernel and heap image generally don't change often in a nominally stable "release", but when they do the changes are often significant. Doing:

shell> svn revert ''file''

will effectively discard any local changes to the named file and replace it with the repository version.

  1. Once svn has finished creating local copies of files that have changed on the svn server since the last time you did this (or since the tarballs were created), use the currently installed Clozure CL to compile the updated sources and build a new kernel and heap image:
shell> ccl  # or "ccl64", or m-x slime, or whatever you do to start Clozure CL
Welcome to whatever version this is!
? (ccl:rebuild-ccl :full t)

That'll recompile the lisp kernel and all lisp sources used to build a heap image and build that image. Once that completes (usually takes 1 minute +/- 30 seconds) there should be a new heap image and new lisp kernel. There may be compiler warnings during compilation, usually related to calls to functions that will be defined when the process finishes but which are not yet defined during compilation (or which take a different number of arguments at compile time than they will when the image is built, etc.) Once in a while, some constants get redefined and a continuable error is signaled; continue from the CERROR to use the new value.

After a while, this (simple and simple-minded) bootstrapping process will cease to work: the newer source code will diverge from the code used to build the lisp image enough that it'll have to be bootstrapped in a more complicated (and usually totally ad-hoc) way. When this happens, new heap images and/or lisp kernels are checked in to svn (as soon as possible after the affected sources are checked in.) As noted above, this shouldn't happen too often in the "release" tree (where most changes are simple, relatively isolated bug fixes.) If it ever does happen, it's worth being aware of the fact that this may result in those files being marked as being in conflict and that "svn revert" can be used to resolve that conflict.