Changes between Version 8 and Version 9 of UpdatingFromSource


Ignore:
Timestamp:
Aug 8, 2008, 10:47:58 PM (11 years ago)
Author:
gb
Comment:

Update: cvs->svn, etc.

Legend:

Unmodified
Added
Removed
Modified
  • UpdatingFromSource

    v8 v9  
    1 = Updating from CVS and rebuilding OpenMCL from source =
     1= Updating from Subversion and rebuilding Clozure CL from source =
    22
    3 This page describes how to update and rebuild OpenMCL from CVS, starting with the most recent "snapshot" tarball.
    4 ("Snapshot tarballs" are distributed periodically in ftp://clozure.com/pub/testing )  Feel free to mentally substitute "Clozure CL" for "OpenMCL".
     3This page describes how to update and rebuild Clozure CL from subversion ("svn"), starting with the most recent "release" tarball.
     4("Release tarballs" are distributed periodically in ftp://clozure.com/pub/release/ )  Extracting the contents of a release tarball into a local "ccl" directory creates a "working copy" of the contents of the svn repository at the time that the release was made.  (This "working copy" contains metainformation - in ".svn" directories - that enable svn to track changes.)
    55
    661. In a shell, enter the "ccl" directory created when the tarball was extracted.
     
    99}}}
    1010
    11 2. If you've never done so before on the current account on the current machine, "login" to the CVS server
     112.  Tell svn to update the sources, creating new directories and removing empty ones as needed.
    1212{{{
    13 shell>cvs login
    14 Password:
    15 }}}
    16 The password is "cvs".  You may see some ominous looking warnings about "~/.cvspass" not existing and having to be created; these are actually benign and expected (but phrased in a way that makes it sound like something bad is happening.  It isn't; we want CVS to create a ~/.cvspass file, and store the trivially-encrypted password there.)
    17 
    18 3.  Tell CVS to update the sources, creating new directories and removing empty ones as needed.
    19 {{{
    20 shell> cvs update -d -P
     13shell> svn update
    2114}}}
    2215
    23 The "d" above is lowercase, the "P" is uppercase, and case matters.
     16This will try to reconcile changes made in the svn repository with the local, working copy.  If you've made changes to files in
     17the working copy and changes have been made in the repository, svn will attempt to merge those changes into an updated copy of
     18the local file.  It's often able to do so reasonably well (for text files); sometimes, the repository changes and local changes can't be resolved automatically and the affected file is said to be "in conflict."  svn has no good way to even try to merge changes to binary files (like the CCL kernel and heap image).
    2419
    25 4.  Once CVS has finished creating local copies of files that have changed on the CVS server since the last time you did this (or since the tarballs were created), use the currently installed OpenMCL to compile the updated sources and build a new kernel and heap image:
     20As "svn update" reconciles changes to the repository with any changes made to the working copy, it prints a line consisting of
     21a one-letter code, some whitespace, and the pathname of the affected item.   These codes include:
     22
     23||A|| ''path''|| The file or directory at ''path'' was added to the repository and has been added to the working copy ||
     24||D|| ''path''|| The file or directory at ''path'' was deleted from the repository and has been deleted from the working copy ||
     25||U|| ''file''|| The file at ''file'' was changed in the repository and the local copy has been updated ||
     26||G|| ''file''|| The file at ''file'' was changed in both the repository and working copy, and the local copy contains mixed content ||
     27||C|| ''file''|| The file at ''file'' was changed in both the repository and working copy, and svn was unable to resolve conflicts ||
     28
     29Binary files that have been locally modified (the act of recompiling the kernel or rebuilding the image generally count as being "modified") generally can't be merged with newer versions from the repository.  The lisp kernel and heap image generally don't change often in a nominally stable "release", but when they do the changes are often significant.  Doing:
    2630
    2731{{{
    28 shell> openmcl  # or "openmcl64", or m-x slime, or whatever you do to start OpenMCL
     32shell> svn revert ''file''
     33}}}
     34
     35will effectively discard any local changes to the named file and replace it with the repository version.
     36
     373.  Once svn has finished creating local copies of files that have changed on the svn server since the last time you did this (or since the tarballs were created), use the currently installed Clozure CL to compile the updated sources and build a new kernel and heap image:
     38
     39{{{
     40shell> ccl  # or "ccl64", or m-x slime, or whatever you do to start Clozure CL
    2941Welcome to whatever version this is!
    3042? (ccl:rebuild-ccl :force t)
     
    3648when the process finishes but which are not yet defined during compilation (or which take a different number of arguments at compile time than they will when the image is built, etc.)  Once in a while, some constants get redefined and a continuable error is signaled; continue from the CERROR to use the new value.
    3749
    38 After a while, this (simple and simple-minded) bootstrapping process will cease to work: the newer source code will diverge from the code used to build the lisp image enough that it'll have to be bootstrapped in a more complicated (and usually totally ad-hoc) way.  When this happens, new tarballs are produced.
    39 
    40 = Tracking changes =
    41 A !ChangeLog (basically, svn commit history that affects the ''trunk'') file is maintained as part of the source hierarchy and regenerated automatically every hour or so; when it changes, it's also copied to the ftp testing directory on clozure.com.  Glancing at this file every now and then enables you to see whether recent changes are of interest to you.
    42 
    43 It's also possible to use the "Timeline" feature on this Trac site for that purpose; the Timeline also keeps track of svn commits to development branches, bug report activity, and changes to Wiki pages.  Note that the Trac site makes the Timeline available via RSS;  use the orange XML button at the bottom of the [http://trac.clozure.com/openmcl/timeline Timeline] to subscribe.
    44 
    45 = Notes: =
    46 
    47 1) OpenMCL development actually takes place using subversion (as the "Browse Source" and "Timeline" and other links on this Trac site indicate); the CVS repository is synchronized with the SVN trunk about once an hour (currently at around 20 minutes past the hour and usually takes a few minutes.)  At some not-too-distant point, it'd probably make sense to switch to SVN completely and to stop having to worry about mirroring things in CVS; CVS is still a little more widely available than SVN (comes with XCode tools on OSX Tiger and earlier, but SVN isn't bundled prior to Leopard.)
    48 
    49 2) Since sometime in September 2007, the version number (in the greeting message and in LISP-IMPLEMENTATION-VERSION) will try to incorporate the SVN revision.
    50 
    51 3) These instructions assume that it's possible to connect to TCP port 2401 on clozure.com; some firewalls may block traffic to that port.  Subversion can generally use more standard protocols (http, ssh, https) that run on ports that firewalls don't block as often; it's possible (and not '''too''' difficult) to use SVN instead of CVS.  See UpdateFromSubversion for instructions on how to do that.
    52 
    53 = The Cocoa IDE =
    54 ----
    55 '''Note that building the IDE on x86-64 Leopard requires that interface files describing Cocoa classes, methods, and data structures be installed in ccl/darwin-x86-headers64/cocoa/*.cdb .  These files are not included in OpenMCL snapshots released prior to Leopard's release (in late October 2007); an archive containing just those .cdb files is availale at ftp://clozure.com/pub/testing/darwinx8664-leopard-cocoa-interfaces.tar.gz .'''
    56 ----
    57 
    58 People interested in using the Cocoa IDE ("Clozure CL.app") can rebuild it by simply doing:
    59 
    60 ? (require "COCOA-APPLICATION")
    61 
    62 in a freshly-updated OpenMCL.  That'll initialize the ObjC bridge, load the lisp sources that comprise the IDE, compile and load a few dozen Hemlock source files, start the GUI, and then save an executable heap image in ccl/Clozure\ Cl.app.  The standalone IDE may need to know where the "ccl" directory is (in order to find interface definitions used by the bridge and FFI and in order for things like REQUIRE and Meta-. to find lisp files;
    63 this can be set via a pane in the preferences panel.  (Some of the pre-build .dmg files announced on info-mcl and on openmcl-devel in October 2007 contained their interface files inside the application bundle; this simplified distribution a bit but that may not be the "final" approach to this issue.)
    64 
    65 It's also possible to do:
    66 
    67 ? (require "COCOA")
    68 
    69 which does everything that requiring COCOA-APPLICATION does except for the last (save-application) step.
    70 In this case, the original listener thread stays active (as does the SLIME REPL thread) and the original
    71 listener stays connected to the process's standard I/O streams.   The tradeoffs include:
    72 
    73 * A few more things (hard to remember what) work in the standalone IDE than work in the hybrid environment.
    74 (The Help menu item that may invoke Apple's Help Viewer only works in the standalone IDE; there may be other
    75 things.)
    76 
    77 * Diagnostic output generally gets written to the process's standard output/error streams, meaning that it's
    78 visible in the hybrid environment but hidden a bit (written to a logging device) in the standalone .app.  (Apple's
    79 "Console" application - in /Applications/Utilities - can be used to view this output, which may be useful to include in bug reports.)
     50After a while, this (simple and simple-minded) bootstrapping process will cease to work: the newer source code will diverge from the code used to build the lisp image enough that it'll have to be bootstrapped in a more complicated (and usually totally ad-hoc) way.  When this happens, new heap images and/or lisp kernels are checked in to svn (as soon as possible after the affected sources are checked in.)  As noted above, this shouldn't happen too often in the "release" tree (where most changes are simple, relatively isolated bug fixes.)  If it ever does happen, it's worth being aware of the fact that this may result in those files being marked as being in conflict and that "svn revert" can be used to resolve that conflict.
    8051
    8152