Changes between Initial Version and Version 1 of ForeignFunctionInterface


Ignore:
Timestamp:
05/09/11 20:27:32 (3 years ago)
Author:
rme
Comment:

partly-done stuff

Legend:

Unmodified
Added
Removed
Modified
  • ForeignFunctionInterface

    v1 v1  
     1= Using Clozure CL's FFI =  
     2 
     3== BLAS and LAPACK libraries == 
     4 
     5The BLAS and LAPACK libraries were originally written in Fortran. 
     6It's certainly possible to call the Fortran-style routines directly 
     7from CCL, but some care is needed, because Fortran conventions differ 
     8from C conventions.  (CCL's FFI is geared towards calling C 
     9functions.) 
     10 
     11In Fortran, all arguments to a function, even scalar values, are passed 
     12by reference (that is, as pointers).  Arrays are also different.  In 
     13Common Lisp, arrays are indexed from 0 and are stored in row-major 
     14order.  In Fortran, arrays are indexed from 1, and are stored in 
     15column-major order. 
     16 
     17To show how this works in practice, we'll call the BLAS routine 
     18idamax, which finds the index of the element of a double-float vector 
     19with the maximum absolute value. 
     20 
     21{{{ 
     22Welcome to Clozure Common Lisp Version 1.7-dev-r14780  (LinuxARM32)! 
     23? (open-shared-library "libblas.so") 
     24#<SHLIB libblas.so.3gf #x5454AF36> 
     25? (rletz ((n :int 10) 
     26          (dx (:array :double 10)) 
     27          (incx :int 1)) 
     28     (setf (paref dx (:array :double) 5) 100d0 
     29           (paref dx (:array :double) 8) -1000d0) 
     30     (external-call "idamax_" (:* int) n (:* double) dx (:* int) incx :int)) 
     319 
     32? 
     33}}} 
     34 
     35Notice that the function parameters are passed by reference (that is, 
     36as pointers).  Second, note that the returned answer is 9 and not 8 as 
     37we might expect.  This is because Fortran array indexes start at 1, 
     38not at 0 as they do in Common Lisp.  Finally, note the "_" suffix on 
     39the name of the Fortran function---that's simply a common Unix 
     40convention. 
     41 
     42There is also a C interface to the BLAS.  This is somewhat more 
     43convenient to use, as the following example shows: 
     44{{{ 
     45? (rletz ((x (:array :double 10))) 
     46   (setf (paref x (:array :double) 5) 100d0 
     47         (paref x (:array :double) 8) -1000d0) 
     48   (external-call "cblas_idamax" :int 10 (:* double) x :int 1 :int)) 
     498 
     50}}} 
     51With the C interface, we don't have to pass scalar parameters by reference, 
     52and the indexes are 0-based, as a Lisp programmer would expect.  (The 
     53libblas.so that we loaded earlier also contains the C interface functions 
     54on this system.) 
     55 
     56We can make this more convenient still by using the interface 
     57translator to parse the cblas.h header file and generate an interface 
     58database for it. 
     59 
     60The directions at http://trac.clozure.com/ccl/wiki/CustomFramework 
     61explain how to do this.  With the interface database present, calling 
     62the BLAS function becomes even easier: 
     63{{{ 
     64? (use-interface-dir :cblas) 
     65#<INTERFACE-DIR :CBLAS #P"cblas/" #x54547E8E> 
     66? (rletz ((x (:array :double 10))) 
     67    (setf (paref x (:array :double) 5) 100d0 
     68          (paref x (:array :double) 8) -1000d0) 
     69    (#_cblas_idamax 10 x 1)) 
     708 
     71}}} 
     72Now we can use the handy #_ reader macro, and we no longer have to 
     73provide the return type of the function, or the types of its 
     74arguments.  The #_ reader macro looks them up in the interface 
     75database and generates the correct external-call form automatically. 
     76 
     77== Lisp Arrays and Foreign Arrays == 
     78 
     79You may be wondering if there is some way to pass lisp arrays to 
     80foreign functions.  In CCL, this is generally not possible.  Even if 
     81it were possible to obtain a pointer to the array data, and assuming 
     82that the lisp array data is stored in the format that the foreign 
     83function expects, the GC might run at any time and move the underlying 
     84object.  (Keep in mind that CCL uses multiple threads, and any thread 
     85might initiate a GC at arbitrary times.) 
     86 
     87Data in a lisp array must therefore be copied to memory either on the 
     88stack or on the foreign heap before being passed to a foreign 
     89function. 
     90 
     91There are, however, a few features in CCL that may be useful for 
     92avoiding the need to copy. 
     93 
     94The first is an object called a heap ivector. 
     95http://ccl.clozure.com/ccl-documentation.html#f_make-heap-ivector 
     96 
     97Here's an example.  (Presumably you would keep the heap ivector object 
     98around somewhere, and free it when you were completely done with it.) 
     99{{{ 
     100(multiple-value-bind (vector ptr) 
     101    (ccl:make-heap-ivector 10 'double-float) 
     102  (setf (aref vector 5) 100d0 
     103        (aref vector 8) -1000d0) 
     104  (prog1 
     105      (#_cblas_idamax 10 ptr 1) 
     106    (ccl:dispose-heap-ivector vector))) 
     107}}} 
     108 
     109The second is a macro called ccl:with-pointer-to-ivector.  See the 
     110docstring.  This must be used with some care, because it inhibits 
     111the GC within its body. 
     112{{{ 
     113(let ((vector (make-array 10 :element-type 'double-float))) 
     114  (setf (aref vector 5) 100d0 
     115        (aref vector 8) -1000d0) 
     116  (ccl:with-pointer-to-ivector (ptr vector) 
     117    (#_cblas_idamax 10 ptr 1))) 
     118}}} 
     119 
     120The nice thing about the above two examples is that you can still 
     121access the vector in the normal way. 
     122 
     123In some earlier examples, we used ccl:rlet to allocate a vector on the 
     124stack.  We can also allocate memory on the foreign heap either with 
     125#_malloc or with the slightly higher-level ccl:make-record.  In either 
     126case, you must explicitly call ccl:free. 
     127{{{ 
     128(let ((x (ccl:make-record (:array :double 10)))) 
     129  (setf (ccl:paref x (:array :double) 5) 100d0 
     130        (ccl:paref x (:array :double) 8) -1000d0) 
     131  (prog1 
     132      (#_cblas_idamax 10 x 1) 
     133    (ccl:free x))) 
     134}}} 
     135 
     136It's also possible to use ccl:make-gcable-record to allocate foreign 
     137memory that will be freed when it is unreachable from lisp. 
     138{{{ 
     139(let ((x (ccl:make-gcable-record (:array :double 10)))) 
     140  (setf (paref x (:array :double) 5) 100d0 
     141        (paref x (:array :double) 8) -1000d0) 
     142  (#_cblas_idamax 10 x 1)) 
     143}}} 
     144 
     145This works well as long as the gcable pointer is the only thing 
     146(either in the lisp world or foreign world) that refers to the memory. 
     147See the manual for a more detailed explanation: 
     148http://ccl.clozure.com/ccl-documentation.html#f_make-gcable-record 
     149