wiki:CrossCompileWin64

Version 2 (modified by gb, 6 years ago) (diff)

--

Cross-compiling for a Win64 target

These notes briefly describe how to set up a Linux x8664 host to cross-compile for a win64 target. As of early/mid March 2008, one would likely only do this to get a handle on changes needed to get a fully working CCL running on Win64. (Generally, vanilla Lisp code should be OK; code that deals with the FFI is a bit more suspect, and pathname and other code may need to be rewritten to be usable in a Windows environment.)

In theory, it should be possible to do the cross-compilation as well from Darwin or FreeBSD; it's a little easier to do it on a machine with the same endianness and word size as the target.

Setup.

Get sources from the win64 branch. (This branch was forked from the trunk around the beginning of March 2008; the code's in a branch so that it's less likely to break the trunk.) The incantation is:

shell> svn co svn+ssh://svn.clozure.com/usr/local/publicsvn/openmcl/trunk/win64 <optional-working-dir>

or

shell> svn co http://svn.clozure.com/publicsvn/openmcl/trunk/win64 <optional-working-dir>

Let's assume that the <optional-working-dir> is "win64". The directory should contain a Linux x8664 heap image named "CROSS-LX8664"; it was built from the sources you just checked out (with a few things to allow cross-compilation added.) I usually run with *PACKAGE* set to "CCL" and with redefinition warnings disabled; this image was built in such an environment.

Build a linuxx8664 kernel; verify that the image and kernel work.

If you're going to be recompiling the linux (host) image - as may be necessary from time to time - you'll likely need the linux-x8664 headers in the win64 directory. You can get them via svn:

shell> cd win64
shell> svn co http://svn.clozure.com/publicsvn/openmcl/trunk/x86-headers64

To cross-compile win64 code, you'll also need win64 interfaces. The cross-compilation environment assumes that they'll be installed in "ccl:cross-win64-headers;", so let's put them there. (The idea of keeping headers in different locations when cross-compiling than when running native helps when using shared directories mounted via NFS or SMB or the like; the .cdb files are implicitly native-endian, and keeping the host and target's .cdb files in separate directories helps to avoid clobbering .cdb files of different endianness.)

Cross-compiling =

Assuming that you have the host system ready to cross-compile, you can do so by doing:

;;; Assume that we're in the CCL package here.
? (require-modules *x8664-xload-modules* t)
;;; whatever output that produces ...
? (cross-xload-level-0 :win64 :force)
;;; This should compile the level-0 sources to fasls, "load" the fasls into a simulated memory image, and write the image to "ccl:wx86-boot64.image".
;;; As of this writing, I think that it all compiles cleanly, but there's likely missing functionality (code in l0-cfm-support to deal with shared
;;; libraries, etc.
;;;
;;; To compile the rest of the lisp:
? (cross-compile-ccl :win64 t)

Around half a dozen of the main lisp sources don't currently compile cleanly (mostly because of FFI issues), and some files that do compile will likely need some work.

Rebuilding the cross-compilation environment

If you need to update the host as changes are made, the general idea is to define a "win64" compiler backend (the terms "backend" and "target" are sometimes used interchangeably, but a "target" is more often a keyword that names a "backend", where a backend is a structure that defines a particular platform) and that backend's "foreign type data" (FTD) and initialize the backend and FTD. The win64 backend is defined in "ccl:compiler;X86;X8664;x8664-backend.lisp", but it's ordinarily conditionalized on #+win64-target. In the transcript below, I moused on the backend definition and the form that pushes *WIN64-BACKEND* on *KNOWN-X8664-BACKENDS*, then typed the subsequent forms into the listener.

? Welcome to Clozure Common Lisp Version 1.2-r8710:8736MS  (LinuxX8664)!
? 
? ;;; Evaluating defvar *win64-backend*
*WIN64-BACKEND*
? ;;; Evaluating pushnew
(#<BACKEND WIN64 #x300040E373ED> #<BACKEND LINUXX8664 #x3000406FD1ED>)
? (pushnew *WIN64-BACKEND* *known-backends*)
(#<BACKEND WIN64 #x300040E373ED> #<BACKEND LINUXX8664 #x3000406FD1ED>)
? (fixup-x8664-backend)
NIL
? (defpackage "WIN64" (:use))
#<Package "WIN64">
? (setup-x8664-ftd *win64-backend*)
#S(FOREIGN-TYPE-DATA :TRANSLATORS #<HASH-TABLE :TEST EQ size 16/60 #x300040E2F37D> :KIND-INFO #<HASH-TABLE :TEST EQ size 25/60 #x300040E2ED6D> :DEFINITIONS #<HASH-TABLE :TEST EQ size 25/60 #x300040E2E75D> :STRUCT-DEFINITIONS #<HASH-TABLE :TEST EQ size 3/60 #x300040E2E14D> :UNION-DEFINITIONS #<HASH-TABLE :TEST EQ size 0/60 #x300040E2DB3D> :ENUM-DEFINITIONS #<HASH-TABLE :TEST EQ size 0/60 #x300040E2D52D> :INTERFACE-DB-DIRECTORY "ccl:cross-win64-headers;" :INTERFACE-PACKAGE-NAME "WIN64" :EXTERNAL-FUNCTION-DEFINITIONS #<HASH-TABLE :TEST EQ size 0/60 #x300040E2CF1D> :DIRLIST #<DLL-HEADER  #x300040E2CEFD> :ATTRIBUTES (:BITS-PER-WORD 64 :STRUCT-BY-VALUE T :BITS-PER-LONG 32) :FF-CALL-EXPAND-FUNCTION WIN64::EXPAND-FF-CALL :FF-CALL-STRUCT-RETURN-BY-IMPLICIT-ARG-FUNCTION WIN64::RECORD-TYPE-RETURNS-STRUCTURE-AS-FIRST-ARG :CALLBACK-BINDINGS-FUNCTION WIN64::GENERATE-CALLBACK-BINDINGS :CALLBACK-RETURN-VALUE-FUNCTION WIN64::GENERATE-CALLBACK-RETURN-VALUE :ORDINAL 106 :ORDINAL-LOCK #<RECURSIVE-LOCK [ptr @ #x655740] #x300040E2CE7D> :ORDINAL-TYPES #<HASH-TABLE :TEST EQ size 15/60 #x300040E2C86D> :POINTER-TYPES #<HASH-TABLE :TEST EQ size 2/60 #x300040E2C25D> :ARRAY-TYPES #<HASH-TABLE :TEST EQUAL size 0/60 #x300040E2BC4D>)
? (canonicalize-foreign-type-ordinals *)
22
? (require "FFI-WIN64")
"FFI-WIN64"
("FFI-WIN64")
? (save-application "CROSS-LX86CL64")

A FOREIGN-TYPE is an object (structure instance) used to describe a foreign type (of all things.) When instantiating a foreign type (creating a pointer to an instance of that type), it's desirable to be able to assert the type of the object that the pointer's allegedly pointing to. Each foriegn type object is assigned a unique small integer, and the code that allocates a typed pointer ordinarily uses LOAD-TIME-VALUE to reference the type object and extract its "ordinal", and sticks that small integer in the pointer object. This works reasonably well ... as long as the foreign type system exists and LOAD-TIME-VALUE works; neither of these things are true early in the loading sequence. The workaround is to pre-assign foreign type ordinals to foreign types that are used "early"; if a foreign type has an ordinal number below some threshold (around 100), it's assumed to be "canonical" and the macro that allocates the foreign object uses the ordinal (and avoids the use of LOAD-TIME-VALUE and the dependence on the foreign type system.)

The predefined foreign-type ordinals are probably a bit POSIX-specific; we may find that we need to conditionalize CANONICALIZE-FOREIGN-TYPE-ORDINALS to add Windows-specific types and avoid POSIX-specific ones.