Opened 11 years ago

Closed 11 years ago

#366 closed defect (fixed)

darwinx8632 FFI issues: small structure return.

Reported by: gb Owned by: rme
Priority: normal Milestone: IA-32 port
Component: Foreign Function Interface Version: trunk
Keywords: Cc: rme

Description

(None of this is unexpected; I'm just trying to document things as I find them so that we hopefully know what to to fix.)

http://developer.apple.com/documentation/DeveloperTools/Conceptual/LowLevelABI/Articles/IA32.html is nominally the authoritative ABI reference, but "whatever GCC does" is probably more authoritative ...

The reference claims that "structures whose aligned size is one or two bytes are returned in %eax, and those whose size is 4 or 8 bytes are returned in %eax and %edx". Structures of other sizes are returned via an invisible first argument.

This is pretty obviously bogus: 4-byte structures are (as one would expect) returned in %eax; %edx is undefined on return. The gcc implementers seem to be as confused by this as anyone else would be: a function that returns a 3-byte structure:

struct foo {
  short x;
  char y;
};

struct foo
getfoo()
{
  struct foo f;
  f.x = 17;
  f.y = 0;
  return f;
}

generates the following code (when compiled with -O2):

	.text
	.align 4,0x90
.globl _getfoo
_getfoo:
	pushl	%ebp
	andl	$-16711681, %eax
	movl	%esp, %ebp
	movw	$17, %ax
	leave
	ret

Ahem. Let's ignore small structures whose size isn't 1,2,4, or 8, and hope that neither CCL nor GCC ever encounter them ...

Different platforms have (wildly) different structure return conventions; AFAIK, they all support the "pointer to structure as first argument" convention in at least some cases. CCL's FFI tries to hide the differences/exceptions in different ABIs by making "high-level" foreign function calls (EXTERNAL-CALL, FF-CALL) follow the pointer-to-structure-as-first arg convention and expand into a lower-level %FF-CALL that either follows that convention or accepts the structure return value in register(s) and stores those registers into the structure in the first arg.

For instance, the x8664 ABI has a fairly bizarre convention where small structures are returned in some combination of %rax, %rdx, %xmm0, and %xmm1, according to nearly incomprehensible rules about whether the structures's halves are or are not entirely of some floating-point type.

Given:

(def-foreign-type :example
    (:struct :example
             (:x :float)
             (:y :float)))

a call to a function "foo" that returns an :example struct and assigns it to P:

  (external-call "foo" p :example)

macroexpands into:

(LET* ((#:RESULT (%NULL-PTR)))
  (DECLARE (DYNAMIC-EXTENT #:RESULT) (TYPE MACPTR #:RESULT))
  (%SETF-MACPTR #:RESULT P)
  (%STACK-BLOCK ((#:REGISTERS (+ (* 2 8) (* 2 8))))
                (%FF-CALL (%REFERENCE-EXTERNAL-ENTRY-POINT
                            (LOAD-TIME-VALUE (EXTERNAL "foo")))
                          :REGISTERS
                          #:REGISTERS
                          :VOID)
                (PROGN (SETF (%GET-UNSIGNED-LONG #:RESULT 0)
                             (%GET-UNSIGNED-LONG #:REGISTERS 16))
                       (SETF (%GET-UNSIGNED-LONG #:RESULT 4)
                             (%GET-UNSIGNED-LONG #:REGISTERS 20)))))

and there's special runtime and compiler support for obtaining all of the register results in this case. (Some other platforms return structure results in multiple registers, so the notion of a platform-dependent multiple-register return buffer is generalized in the FFI, perhaps a bit too much.) The temporary #:RESULT pointer is only there to ensure left-to-right evaluation order.

Given the same definition of the :example structure, the Darwin X8632 port compiles the same call into:

(%FF-CALL (%REFERENCE-EXTERNAL-ENTRY-POINT (LOAD-TIME-VALUE (EXTERNAL "foo"))) :ADDRESS P :VOID)

We maybe don't need the generality of returning multiple registers, since we're either going to get a single 32-bit value in %eax or a 64-bit value in %eax:%edx, so I think that this should expand into something like:

(LET* ((#:RESULT (%NULL-PTR)))
  (DECLARE (DYNAMIC-EXTENT #:RESULT) (TYPE MACPTR #:RESULT))
  (%SETF-MACPTR #:RESULT P)
  (SETF 
    (%%GET-SIGNED-LONGLONG #:RESULT 0)
    (%FF-CALL (%REFERENCE-EXTERNAL-ENTRY-POINT (LOAD-TIME-VALUE (EXTERNAL "foo")))
              :SIGNED-DOUBLEWORD)))

(assuming that I've indented and counted parens sanely and assuming that I'm thinking about endianness correctly.)

I've cut-and-pasted code from ffi-darwinx8632.lisp into the Linux and Windows x8632 files; I think that those other platforms always use the "pass as first arg" convention, so AFAIK it's just Darwin that needs to worry about this, so we should probably make sure that the other platforms just return T from the hook function that says whether or not to use that convention on those other platforms.

Change History (4)

comment:1 Changed 11 years ago by rme

r11346 tries to address this a bit for Darwin.

Test case:

Compile following code with cc -dynamiclib -o libjunk.dylib x.c

struct junk {
  int i;
  char c;
};

struct junk moof(void)
{
  struct junk jj = {2, 'b'};
  return jj;
}

Try it out:

$ ccl
Welcome to Clozure Common Lisp Version 1.3-dev-r11320M-trunk  (DarwinX8632)!
? (open-shared-library "libjunk.dylib")
#<SHLIB libjunk.dylib #x893C776>
? (def-foreign-type :junk
    (:struct :junk
             (:x :int)
             (:y :char)))
:JUNK
? (rlet ((j :junk))
       (external-call "_moof" j :junk)
       (print (pref j :junk.x))
       (print (pref j :junk.y)))

2 
98 
98
?

comment:2 Changed 11 years ago by rme

  • Owner changed from gb to rme
  • Status changed from new to assigned

We're getting closer to figuring all this out. r11552 through r11555 try to unify most of the x8632 FFI stuff.

Still to do: in cases where structs are returned via a "hidden" first argument, we need to be sure to discard that argument at the end of .SPcallback.

(.SPffcall doesn't care about what's left on the foreign stack frame on return, since we just discard the whole thing. .SPcallback needs to care: one not-too-far-fetched example would be a user-defined ObjC method that returns a large structure, such as an NSRect. Not popping the hidden first arg would leave the stack in an incorrect state, and would probably confuse the caller.)

comment:3 Changed 11 years ago by rme

r11590 addresses discarding the "hidden" first argument in .SPcallback.

I may have overlooked something, of course, but I think this should about do it.

comment:4 Changed 11 years ago by rme

  • Resolution set to fixed
  • Status changed from assigned to closed
Note: See TracTickets for help on using tickets.