Version 25 (modified by rme, 14 years ago) (diff)


Notes on an IA-32 port

A port of OpenMCL to IA-32 (Intel's name for the 32 bit x86 architecture) would be a win for several reasons.

The most obvious reason would be to support Apple hardware that isn't 64-bit capable. This includes the first-generation Intel-based iMac, MacBook, MacBook Pro, and all Intel-based Mac Minis. Of course, support for non-Apple IA-32 hardware would also be nice.

Another reason would be to support access to Cocoa and Carbon (and other frameworks) on Intel-based Macintoshes running Mac OS X Tiger.

Although Apple has announced that Cocoa will be 64 bit in Leopard, they have publically confirmed that Carbon won't be. Therefore, a 32 bit lisp would still be needed to use Carbon, even on Leopard running on 64 bit hardware.

It would be interesting to support the AMD Geode LX (as used in the OLPC laptop) as the minimum processor. This processor supports the P6 family instructions, including MMX instructions. We can therefore use the conditional move instructions, and maybe some MMX instructions to help out with bignums.

On the other hand, the AMD Geode LX processor doesn't support any of the SSE/SSE2/SSE3 instructions; this means that we'd have to use the x87 FPU (which is sort of funky). This would require modifications to the compiler, which believes that every floating point register can be accessed independently.

It might be reasonable to target the Core Solo/Duo? processor, at least to begin with. This would cover all Intel-based Macintosh systems ever shipped, and would allow us to avoid adding x87 FPU support for now.

Register usage and tagging

(See also

We want to keep the precise GC, but the limited number of registers that we have will probably make it impossible to statically partition the register file into immediate and tagged sets.

Anyway, we're looking at something like this:

eax 	 imm0
ecx 	 nargs, temp0
edx 	 imm1
ebx 	 fn
esp 	 stack pointer
ebp 	 frame pointer
esi 	 arg_z
edi 	 arg_y

We will augment this with a dynamic scheme: we will set or clear a bit in thread-private memory whenever a register transitions from one class to another. The GC will then look at these flag bits to decide how to treat the registers. (This may make the lispy register names confusing, since at times imm0 might actually contain a node, or arg_y an immediate.)

Callee-saved "non-volatile" registers are probably a non-starter.

The tagging scheme can basically follow the PPC32 port. An important difference is that the three-bit tag #b101, which is for NIL on PPC32, would be used for a thing called a tagged return addresses on IA-32. (More on this later.)

Comment by gb on Wed Aug 1 22:16:05 2007

It's probably sanest to think of the dynamic register partitioning as being a set of (local, temporary) changes relative to a baseline scheme, where the baseline scheme is in effect any time a function is entered (and therefore at the time of a function call). At that time, we probably need more node regs and fewer imm regs than the scheme suggested above provides, and we can probably overload nargs and imm0.

If we pass two arguments in registers, then we probably need a node register to address the callee on a function call (something like:

(movl (@ 'foo (% fn)) (% temp0))
(:talign ia32::fulltag-tra)
(call (@ ia32::symbol.fcell (% temp0))
(movl ($ :self) (% fn))

The CLOS implementation will sometimes funcall a method-function with an invisible argument (not counted against nargs) in a node register. (That's context information for CALL-NEXT-METHOD and it's done in a way that's not MOP-compliant.)

I think that in general if we err on the side of "too many node regs" in the baseline partitioning, we always (cheaply) save the values in those node regs if we need to temporarily make the register immediate for consing, shifts, multiply/divide, memory assignment, whatever else needs more than one imm reg.)

I think that having imm0=nargs would work fairly well, since we're usually either validating/defaulting based on nargs or doing tag/bounds checking, but rarely if ever need to do both at the same time.


We don't have the __thread storage class on Darwin, so we will need to use i386_set_ldt to install a segment descriptor for each thread into the LDT. When a thread's segment descriptor is loaded into the %fs segment register, %fs can be used to refer to thread-local storage.

(This implies an 8K limit to the number of threads, by the way. Probably not a big deal for a 32 bit lisp.)

LAP notes

quoted numbers are fixnums, e.g., for x8664, (add ($ '2) (% rsp)) adds 16 to %rsp.

memory operand form: ([seg] [disp] [base] [index] [scale])

Bootstrapping notes

Follow these instructions to use a current Darwin/x86-64 lisp to build the ia32 branch. (I think that the 070722 snapshot should also work.)

Check out the ia32 branch from svn.

$ svn co svn+ssh://


$ svn co

if you don't have write access to the repository.

Next, copy a kernel, image, and interfaces into the ia32 tree.

$ p=/path/to/trunk/ccl/
$ cp $p/{dx86cl64,dx86cl64.image} .
$ cd darwin-x86-headers64/libc
$ cp $p/darwin-x86-headers64/libc/*.cdb .

Build the ia32 branch sources. Follow the directions below.

;;; pick up operators i386-ff-call and i386-syscall
(load "compiler/nxenv.lisp")
(load "compiler/nx1.lisp")

;;; define some stuff from x86-asm.lisp
(in-package "X86")
(defparameter *opcode-flags*
  `((:jump . ,(ash 1 0))		;special case for jump insns
    (:CpuNo64 . ,(ash 1 16))		;not supported in 64 bit mode
    (:Cpu64 . ,(ash 1 17))		;64 bit mode required
    (:CpuSSE . ,(ash 1 18))		;SSE extensions required
    (:CpuSSE2 . ,(ash 1 19))		;SSE2 extensions required
    (:CpuSSE3 . ,(ash 1 20))		;SSE3 extensions required

(defun %encode-opcode-flags (flags &optional errorp)
  (flet ((encode-atomic-flag (f)
	   (if f
	     (cdr (assoc f *opcode-flags*))
     (if (atom flags)
       (encode-atomic-flag flags)
       (let* ((k 0))
	 (dolist (f flags k)
	   (let* ((k0 (encode-atomic-flag f)))
	     (if k0
	       (setq k (logior k0 k))
     (if errorp (error "Unknown x86 opcode flags: ~s" flags)))))

(defmacro encode-opcode-flags (&rest flags)
  (%encode-opcode-flags flags t))

(in-package "CCL")

(load "compiler/X86/X8632/x8632-arch.lisp")
(load "lib/x8632env.lisp")
(compile-file "compiler/X86/X8664/x8664-backend.lisp")
(compile-file "compiler/X86/x86-asm.lisp")
(compile-file "compiler/X86/x86-lap.lisp")
(compile-file "compiler/X86/x862.lisp")

;;; Ignore the warnings about the undeclared x86::*x86...* free variables.

(load "compiler/X86/X8664/x8664-backend")
(load "compiler/X86/x86-asm")
(load "compiler/X86/x86-lap")
(load "compiler/X86/x862")

(compile-ccl t)

Exit, start up with the bootstrap image, and then save a new full image.

Darwin/IA-32 interface databases

To build interface files, get an ffigen binary from and install it.

$ cd darwin-x86-headers/libc/C
$ ./
+++ /Developer/SDKs/MacOSX10.4u.sdk/usr/include/ar.h
+++ /Developer/SDKs/MacOSX10.4u.sdk/usr/include/arpa/ftp.h
+++ /Developer/SDKs/MacOSX10.4u.sdk/usr/include/arpa/inet.h

There's one more step to making these interfaces available to the lisp, but we'll get to that in a minute.

Cross compiling

Given an image made as above, to set up for cross-compiling:

(in-package "CCL")
;;; may not actually need all this stuff
(load "compiler/X86/X8632/x8632-arch.lisp")
(require "X8632ENV")
(require "X8632-ARCH")
(defpackage "X86-DARWIN32")
(compile-file "compiler/X86/x86-lap.lisp")
(load "compiler/X86/x86-lap")
(load "compiler/X86/x86-backend.lisp")
(load "compiler/X86/x86-disassemble.lisp")
(load "compiler/X86/X8632/x8632-backend.lisp")

(let ((*target-backend* *x8632-backend*))
  (load "ccl:compiler;X86;X8632;x8632-vinsns.lisp")
  (load "ccl:compiler;X86;x86-lapmacros.lisp"))

(load "lib/ffi-darwinx8632.lisp")

(require-update-modules *x8632-xload-modules* t)

The first time you do this, you need to finish processing the interface databases.

(require "PARSE-FFI")
(parse-standard-ffi-files :libc :darwinx8632)
;;; lots of output...
;;; Do it once more...
(parse-standard-ffi-files :libc :darwinx8632)


(cross-xload-level-0 :darwinx8632 t)

will create a boot image in x86-boot32.image.

To build an IA-32 lisp kernel:

$ cd lisp-kernel/darwinx8632
$ make

The current state of the port is that it will map in the boot image and run %toplevel-function% before dying in the %fasload form (because syscalls don't work yet).

To run the IA-32 lisp (from the lisp-kernel/darwinx8632 directory):

$ gdb ../../dx86cl
GNU gdb 6.3.50-20050815 (Apple version gdb-768) (Tue Oct  2 04:07:49 UTC 2007)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .. done

Breakpoint 1 at 0xac79: file ../lisp-debug.c, line 892.
(gdb) b start_lisp
Breakpoint 2 at 0x5c61: file {standard input}, line 61.
(gdb) run ../../x86-boot32.image 
Starting program: /Users/rme/openmcl/ccl/dx86cl ../../x86-boot32.image
Reading symbols for shared libraries +. done

Breakpoint 2, start_lisp () at {standard input}:66
66	{standard input}: No such file or directory.
	in {standard input}
1: x/i $pc  0x5c61 <start_lisp+6>:	mov    0x8(%ebp),%ebx

If you take a look and have questions or comments, please send mail to openmcl-devel@…, or to me directly (rme@…).