wiki:PortToIA-32

Version 20 (modified by rme, 6 years ago) (diff)

--

Notes on an IA-32 port

A port of OpenMCL to IA-32 (Intel's name for the 32 bit x86 architecture) would be a win for several reasons.

The most obvious reason would be to support Apple hardware that isn't 64-bit capable. This includes the first-generation Intel-based iMac, MacBook, MacBook Pro, and all Intel-based Mac Minis. Of course, support for non-Apple IA-32 hardware would also be nice.

Another reason would be to support access to Cocoa and Carbon (and other frameworks) on Intel-based Macintoshes running Mac OS X Tiger.

Although Apple has announced that Cocoa will be 64 bit in Leopard, they have publically confirmed that Carbon won't be. Therefore, a 32 bit lisp would still be needed to use Carbon, even on Leopard running on 64 bit hardware.

It would be interesting to support the AMD Geode LX (as used in the  OLPC laptop) as the minimum processor. This processor supports the P6 family instructions, including MMX instructions. We can therefore use the conditional move instructions, and maybe some MMX instructions to help out with bignums.

On the other hand, the AMD Geode LX processor doesn't support any of the SSE/SSE2/SSE3 instructions; this means that we'd have to use the x87 FPU (which is sort of funky). This would require modifications to the compiler, which believes that every floating point register can be accessed independently.

It might be reasonable to target the Core Solo/Duo? processor, at least to begin with. This would cover all Intel-based Macintosh systems ever shipped, and would allow us to avoid adding x87 FPU support for now.

Register usage and tagging

(See also  http://openmcl.clozure.com/Doc/index.html#Register-usage-and-tagging)

We want to keep the precise GC, but the limited number of registers that we have will probably make it impossible to statically partition the register file into immediate and tagged sets.

Anyway, we're looking at something like this:

eax 	 imm0
ecx 	 nargs, temp0
edx 	 imm1
ebx 	 fn
esp 	 stack pointer
ebp 	 frame pointer
esi 	 arg_z
edi 	 arg_y

We will augment this with a dynamic scheme: we will set or clear a bit in thread-private memory whenever a register transitions from one class to another. The GC will then look at these flag bits to decide how to treat the registers. (This may make the lispy register names confusing, since at times imm0 might actually contain a node, or arg_y an immediate.)

Callee-saved "non-volatile" registers are probably a non-starter.

The tagging scheme can basically follow the PPC32 port. An important difference is that the three-bit tag #b101, which is for NIL on PPC32, would be used for a thing called a tagged return addresses on IA-32. (More on this later.)

Comment by gb on Wed Aug 1 22:16:05 2007

It's probably sanest to think of the dynamic register partitioning as being a set of (local, temporary) changes relative to a baseline scheme, where the baseline scheme is in effect any time a function is entered (and therefore at the time of a function call). At that time, we probably need more node regs and fewer imm regs than the scheme suggested above provides, and we can probably overload nargs and imm0.

If we pass two arguments in registers, then we probably need a node register to address the callee on a function call (something like:

(movl (@ 'foo (% fn)) (% temp0))
(:talign ia32::fulltag-tra)
(call (@ ia32::symbol.fcell (% temp0))
(movl ($ :self) (% fn))

The CLOS implementation will sometimes funcall a method-function with an invisible argument (not counted against nargs) in a node register. (That's context information for CALL-NEXT-METHOD and it's done in a way that's not MOP-compliant.)

I think that in general if we err on the side of "too many node regs" in the baseline partitioning, we always (cheaply) save the values in those node regs if we need to temporarily make the register immediate for consing, shifts, multiply/divide, memory assignment, whatever else needs more than one imm reg.)

I think that having imm0=nargs would work fairly well, since we're usually either validating/defaulting based on nargs or doing tag/bounds checking, but rarely if ever need to do both at the same time.

AddComment?

Threads

We don't have the __thread storage class on Darwin, so we will need to use i386_set_ldt to install a segment descriptor for each thread into the LDT. When a thread's segment descriptor is loaded into the %fs segment register, %fs can be used to refer to thread-local storage.

(This implies an 8K limit to the number of threads, by the way. Probably not a big deal for a 32 bit lisp.)

LAP notes

quoted numbers are fixnums, e.g., for x8664, (add ($ '2) (% rsp)) adds 16 to %rsp.

memory operand form: ([seg] [disp] [base] [index] [scale])

Bootstrapping notes

Using a current x86-64 OpenMCL to build the ia32 branch:

;;; pick up operators i386-ff-call and i386-syscall
(load "compiler/nxenv.lisp")
(load "compiler/nx1.lisp")

;;; define some stuff from x86-asm.lisp
(in-package "X86")
(defparameter *opcode-flags*
  `((:jump . ,(ash 1 0))		;special case for jump insns
    (:CpuNo64 . ,(ash 1 16))		;not supported in 64 bit mode
    (:Cpu64 . ,(ash 1 17))		;64 bit mode required
    (:CpuSSE . ,(ash 1 18))		;SSE extensions required
    (:CpuSSE2 . ,(ash 1 19))		;SSE2 extensions required
    (:CpuSSE3 . ,(ash 1 20))		;SSE3 extensions required
))

(defun %encode-opcode-flags (flags &optional errorp)
  (flet ((encode-atomic-flag (f)
	   (if f
	     (cdr (assoc f *opcode-flags*))
	     0)))
    (or
     (if (atom flags)
       (encode-atomic-flag flags)
       (let* ((k 0))
	 (dolist (f flags k)
	   (let* ((k0 (encode-atomic-flag f)))
	     (if k0
	       (setq k (logior k0 k))
	       (return))))))
     (if errorp (error "Unknown x86 opcode flags: ~s" flags)))))

(defmacro encode-opcode-flags (&rest flags)
  (%encode-opcode-flags flags t))

(in-package "CCL")

(load "compiler/X86/X8632/x8632-arch.lisp")
(load "lib/x8632env.lisp")
(compile-file "compiler/X86/X8664/x8664-backend.lisp")
(compile-file "compiler/X86/x86-asm.lisp")
(compile-file "compiler/X86/x86-lap.lisp")
(compile-file "compiler/X86/x862.lisp")

;;; Ignore the warnings about the undeclared x86::*x86...* free variables.

(load "compiler/X86/X8664/x8664-backend")
(load "compiler/X86/x86-asm")
(load "compiler/X86/x86-lap")
(load "compiler/X86/x862")

(compile-ccl t)
(xload-level-0)

Exit, start up with the bootstrap image, and then save a new full image.

Cross compiling

Given an image made as above, to set up for cross-compiling:

(in-package "CCL")
(set-development-environment)
;;; may not actually need all this stuff
(load "compiler/X86/X8632/x8632-arch.lisp")
(require "X8632ENV")
(require "X8632-ARCH")
(defpackage "X86-DARWIN32")
(compile-file "compiler/X86/x86-lap.lisp")
(load "compiler/X86/x86-lap")
(load "compiler/X86/x86-backend.lisp")
(load "compiler/X86/x86-disassemble.lisp")
(load "compiler/X86/X8632/x8632-backend.lisp")
(let ((*target-backend* *x8632-backend*))
  (load "ccl:compiler;X86;X8632;x8632-vinsns.lisp"))
(load "lib/ffi-darwinx8632.lisp")

Now, you can run

(cross-compile-ccl :darwinx8632 t)

and see how far that gets you. (It should get to l1-boot-2.lisp then complain about missing interface files.)

To build interface files, get an ffigen binary from clozure.com:/pub/testing and install it.

$ cd darwin-x86-headers/libc/C
$ ./populate.sh
+++ /Developer/SDKs/MacOSX10.4u.sdk/usr/include/ar.h
+++ /Developer/SDKs/MacOSX10.4u.sdk/usr/include/arpa/ftp.h
+++ /Developer/SDKs/MacOSX10.4u.sdk/usr/include/arpa/inet.h
[...]
$

Start up a lisp, and do the following:

(require "PARSE-FFI")
(parse-standard-ffi-files :libc :darwinx8632)
;;; lots of output...

Now, you'll get farther with the cross-compile, but you'll run into an error (value 8 is not of the expected type (member ...)) whil compiling linux-files.lisp.