Changes between Initial Version and Version 1 of ForeignFunctionInterface

Timestamp:

05/09/11 15:27:32 (2 years ago)

Author:

rme

Comment:

partly-done stuff

ForeignFunctionInterface

@@ +1 @@
+= Using Clozure CL's FFI = 
+
+== BLAS and LAPACK libraries ==
+
+The BLAS and LAPACK libraries were originally written in Fortran.
+It's certainly possible to call the Fortran-style routines directly
+from CCL, but some care is needed, because Fortran conventions differ
+from C conventions.  (CCL's FFI is geared towards calling C
+functions.)
+
+In Fortran, all arguments to a function, even scalar values, are passed
+by reference (that is, as pointers).  Arrays are also different.  In
+Common Lisp, arrays are indexed from 0 and are stored in row-major
+order.  In Fortran, arrays are indexed from 1, and are stored in
+column-major order.
+
+To show how this works in practice, we'll call the BLAS routine
+idamax, which finds the index of the element of a double-float vector
+with the maximum absolute value.
+
+{{{
+Welcome to Clozure Common Lisp Version 1.7-dev-r14780  (LinuxARM32)!
+? (open-shared-library "libblas.so")
+#<SHLIB libblas.so.3gf #x5454AF36>
+? (rletz ((n :int 10)
+          (dx (:array :double 10))
+          (incx :int 1))
+     (setf (paref dx (:array :double) 5) 100d0
+           (paref dx (:array :double) 8) -1000d0)
+     (external-call "idamax_" (:* int) n (:* double) dx (:* int) incx :int))
+9
+?
+}}}
+
+Notice that the function parameters are passed by reference (that is,
+as pointers).  Second, note that the returned answer is 9 and not 8 as
+we might expect.  This is because Fortran array indexes start at 1,
+not at 0 as they do in Common Lisp.  Finally, note the "_" suffix on
+the name of the Fortran function---that's simply a common Unix
+convention.
+
+There is also a C interface to the BLAS.  This is somewhat more
+convenient to use, as the following example shows:
+{{{
+? (rletz ((x (:array :double 10)))
+   (setf (paref x (:array :double) 5) 100d0
+         (paref x (:array :double) 8) -1000d0)
+   (external-call "cblas_idamax" :int 10 (:* double) x :int 1 :int))
+8
+}}}
+With the C interface, we don't have to pass scalar parameters by reference,
+and the indexes are 0-based, as a Lisp programmer would expect.  (The
+libblas.so that we loaded earlier also contains the C interface functions
+on this system.)
+
+We can make this more convenient still by using the interface
+translator to parse the cblas.h header file and generate an interface
+database for it.
+
+The directions at http://trac.clozure.com/ccl/wiki/CustomFramework
+explain how to do this.  With the interface database present, calling
+the BLAS function becomes even easier:
+{{{
+? (use-interface-dir :cblas)
+#<INTERFACE-DIR :CBLAS #P"cblas/" #x54547E8E>
+? (rletz ((x (:array :double 10)))
+    (setf (paref x (:array :double) 5) 100d0
+          (paref x (:array :double) 8) -1000d0)
+    (#_cblas_idamax 10 x 1))
+8
+}}}
+Now we can use the handy #_ reader macro, and we no longer have to
+provide the return type of the function, or the types of its
+arguments.  The #_ reader macro looks them up in the interface
+database and generates the correct external-call form automatically.
+
+== Lisp Arrays and Foreign Arrays ==
+
+You may be wondering if there is some way to pass lisp arrays to
+foreign functions.  In CCL, this is generally not possible.  Even if
+it were possible to obtain a pointer to the array data, and assuming
+that the lisp array data is stored in the format that the foreign
+function expects, the GC might run at any time and move the underlying
+object.  (Keep in mind that CCL uses multiple threads, and any thread
+might initiate a GC at arbitrary times.)
+
+Data in a lisp array must therefore be copied to memory either on the
+stack or on the foreign heap before being passed to a foreign
+function.
+
+There are, however, a few features in CCL that may be useful for
+avoiding the need to copy.
+
+The first is an object called a heap ivector.
+http://ccl.clozure.com/ccl-documentation.html#f_make-heap-ivector
+
+Here's an example.  (Presumably you would keep the heap ivector object
+around somewhere, and free it when you were completely done with it.)
+{{{
+(multiple-value-bind (vector ptr)
+    (ccl:make-heap-ivector 10 'double-float)
+  (setf (aref vector 5) 100d0
+        (aref vector 8) -1000d0)
+  (prog1
+      (#_cblas_idamax 10 ptr 1)
+    (ccl:dispose-heap-ivector vector)))
+}}}
+
+The second is a macro called ccl:with-pointer-to-ivector.  See the
+docstring.  This must be used with some care, because it inhibits
+the GC within its body.
+{{{
+(let ((vector (make-array 10 :element-type 'double-float)))
+  (setf (aref vector 5) 100d0
+        (aref vector 8) -1000d0)
+  (ccl:with-pointer-to-ivector (ptr vector)
+    (#_cblas_idamax 10 ptr 1)))
+}}}
+
+The nice thing about the above two examples is that you can still
+access the vector in the normal way.
+
+In some earlier examples, we used ccl:rlet to allocate a vector on the
+stack.  We can also allocate memory on the foreign heap either with
+#_malloc or with the slightly higher-level ccl:make-record.  In either
+case, you must explicitly call ccl:free.
+{{{
+(let ((x (ccl:make-record (:array :double 10))))
+  (setf (ccl:paref x (:array :double) 5) 100d0
+        (ccl:paref x (:array :double) 8) -1000d0)
+  (prog1
+      (#_cblas_idamax 10 x 1)
+    (ccl:free x)))
+}}}
+
+It's also possible to use ccl:make-gcable-record to allocate foreign
+memory that will be freed when it is unreachable from lisp.
+{{{
+(let ((x (ccl:make-gcable-record (:array :double 10))))
+  (setf (paref x (:array :double) 5) 100d0
+        (paref x (:array :double) 8) -1000d0)
+  (#_cblas_idamax 10 x 1))
+}}}
+
+This works well as long as the gcable pointer is the only thing
+(either in the lisp world or foreign world) that refers to the memory.
+See the manual for a more detailed explanation:
+http://ccl.clozure.com/ccl-documentation.html#f_make-gcable-record
+