source: trunk/source/examples/jfli/docs/jfli.html @ 11435

Last change on this file since 11435 was 11435, checked in by gb, 12 years ago

Rich Hickey's 'jfli' (Lisp<->Java) bridge, as of jfli091104.zip.
The file 'jni.lisp' is LispWorks?-specific FFI code; there'll need
to be something CCL-specific that exports the same interface.

Text files in the .zip were CRLF-terminated; hopefully, svn will
fix that ...
File size: 21.8 KB
Line 
1<html>
2<head>
3        <link rel="stylesheet" type="text/css" media="screen" href="jfli.css">
4</head>
5        <body>
6        <h1><a href="/" title="Rich Hickey's jfli - a Java Foreign Language Interface for Common Lisp">jfli
7        - a Java Foreign Language Interface for Common Lisp</a></h1>
8
9        <h5>Copyright (c) Rich Hickey. All rights reserved.</h5>
10
11        <p>
12        The use and distribution terms for this software are covered by the <a href="http://opensource.org/licenses/cpl.php">Common
13        Public License 1.0</a>, which can be found in the file CPL.TXT at the root of
14        this distribution. By using this software in any fashion, you are agreeing to be
15        bound by the terms of this license. You must not remove this notice, or any
16        other, from this software.
17        </p>
18
19        <h2>Contents</h2>
20
21        <ul>
22                <li>
23                        <a href="#intro">Introduction</a>
24                </li>
25                <li>
26                        <a href="#download">Download</a>
27                </li>
28                <li>
29                        <a href="#setup">Setup and Configuration</a>
30                </li>
31                <li>
32                        <a href="#quickstart">Quick Start</a>
33                </li>
34                <li>
35                        <a href="#api">API Reference</a>
36                        <ul>
37                                <li>
38                                        <a href="#jvmcreation">JVM Creation and Initialization</a>
39                                </li>
40                                <li>
41                                        <a href="#wrappergen">Wrapper Generation</a>
42                                </li>
43                                <li>
44                                        <a href="#objects">Object Creation</a>
45                                </li>
46                                <li>
47                                        <a href="#arrays">Arrays</a>
48                                </li>
49                                <li>
50                                        <a href="#proxies">Proxies - Java calling back to Lisp</a>
51                                </li>
52                                <li>
53                                        <a href="#utilities">Utilities</a>
54                                </li>
55                        </ul>
56                </li>
57                <li>
58                        <a href="#summary">Summary</a>
59                </li>
60        </ul>
61
62
63        <a name="intro"></a> <h3>Introduction</h3>
64
65        <p>
66        My objective was to provide comprehensive, safe, dynamic and Lisp-y access to
67        Java and Java libraries as if they were Lisp libraries, for use in Lisp programs,
68        i.e.  with an emphasis on working in Lisp rather than in Java.
69        </p>
70
71        <p>
72        The approach I took was to embed a JVM instance in the Lisp process using JNI. I
73        was able to do this using LispWorks' own FLI and no C (or Java! *) code, which
74        is a tribute to the LW FLI.  On top of the JNI layer (essentially a wrapper
75        around the entire JNI API), I built this user-level API using Java Reflection.
76        This first version was built with, and contains code specific to, Xanalys <a
77                                                                 href="http://www.lispworks.com/">LispWorks</a>.
78        </p>
79
80
81        <p>
82        <em>jfli</em> ("jay fly") provides:
83        </p>
84
85        <ul>
86
87                <li>
88                        Automatic function generation for constructors, fields and methods, either by
89                        named class, or entire package (sub)trees given a jar file.
90                </li>
91
92                <li>
93                        Java -> Lisp package and name mapping with an eye towards lack of surprise, lack
94                        of conflict, and useful editor completion.
95                </li>
96
97                <li>
98                        setf-able setter generation for fields as well as for methods that follow the
99                        JavaBeans property protocol.
100                </li>
101
102                <li>
103                        Java array creation and aref-like access to Java arrays.
104                </li>
105
106                <li>
107                        A 'new' macro that allows for keyword-style field and property initialization.
108                </li>
109
110                <li>
111                        Typed references to Java objects with an inheritance hierarchy on the Lisp side
112                        mirroring that on the Java side - allowing for Lisp methods specialized on Java
113                        class and interface types.
114                </li>
115
116                <li>
117                        Implementation of arbitrary Java interfaces in Lisp, and callbacks from Java to
118                        Lisp via those interfaces.  (* this required a single 5-line dummy Java proxy
119                        stub, provided with jfli)
120                </li>
121
122                <li>
123                        Automatic lifetime maintenance of Lisp-referenced Java objects, boxing/unboxing
124                        of primitive args/returns, string conversions, Java exception handling, overload
125                        resolution etc.
126                </li>
127        </ul>
128
129        <p>
130        I built jfli using LWM and LWW (using Apple's and Sun's JVMs respectively), and
131        it works fine on both.  Should be a trivial port to other LispWorks, and a
132        possible port to any Common Lisp with a robust FLI. Should also work with any
133        JVM with a conformant JNI implementation.
134        </p>
135
136        <a name="download"></a> <h3>Download</h3>
137
138        <p>
139        jfli is hosted on <a href="http://sourceforge.net/projects/jfli/">SourceForge</a>
140        </p>
141
142        <a name="setup"></a> <h3>Setup and Configuration</h3>
143
144        <p>
145        jfli is supplied in 2 Lisp files and an optional Java .jar file.  The first Lisp
146        file, jni.lisp, defines a low-level API to the Java Native Interface, and is not
147        documented here. The second, jfli.lisp, depends upon jni.lisp, and provides the
148        user API documented here. Simply compile and load jni.lisp, then compile and
149        load jfli.lisp. <code>(use-package :jfli)</code> and you are ready to use the
150        API. Note that prior to creating the JVM you must tell the library how to find
151        the Java JNI library by setting <a
152                                                                  href="#jnilibpath"><code>*jni-lib-path*</code></a>.
153        </p>
154        <p>
155        If you wish to allow for callbacks from Java to Lisp, you must place jfli.jar in
156        your classpath when <a href="#jvmcreation">creating the JVM</a>.
157        </p>
158
159
160        <a name="quickstart"></a> <h3>Quick Start</h3>
161        <p>
162        This sample session presumes you have already compiled jni.lisp and jfli.lisp into fasl files.
163        </p>
164        <pre>
165CL-USER 4 > (load "/lisp/jni")
166; Loading fasl file C:\lisp\jni.fsl
167#P"C:/lisp/jni.fsl"
168
169CL-USER 5 > (load "/lisp/jfli")
170; Loading fasl file C:\lisp\jfli.fsl
171#P"C:/lisp/jfli.fsl"
172
173;The user API is entirely in the jfli package
174CL-USER 6 > (use-package :jfli)
175T
176
177;tell the library where Java is located
178CL-USER 7 > (setf *jni-lib-path* "/j2sdk1.4.2_01/jre/bin/client/jvm.dll")
179"/j2sdk1.4.2_01/jre/bin/client/jvm.dll"
180
181;this starts the VM - note how you can set the classpath
182CL-USER 8 > (create-jvm "-Djava.class.path=/lisp/jfli.jar")
1830
184#&lt;Pointer: JNI:PVM = #x081022A0>
185#&lt;Pointer: JNI:PENV = #x0086A858>
186
187;define wrappers for the members of Object
188CL-USER 9 > (def-java-class "java.lang.Object")
189NIL
190
191;and of Properties, a Hashtable-like class
192CL-USER 10 > (def-java-class "java.util.Properties")
193#&lt;STANDARD-CLASS |java.util|:PROPERTIES. 2066B964>
194
195;the above will create these packages if they do not already exist
196;use the packages for easy name access
197
198CL-USER 11 > (use-package "java.lang")
199T
200
201CL-USER 12 > (use-package "java.util")
202T
203
204;create a Properties instance, note keyword-style member inits, string conversion etc
205;also note typed return value
206CL-USER 13 > (setf p (new properties. :getproperty "fred" "ethel"))
207#&lt;PROPERTIES. 20664A94>
208
209;virtual functions work as normal
210CL-USER 14 > (object.tostring p)
211"{fred=ethel}"
212
213;setter was generated for member function because it follows the JavaBeans property protocol
214CL-USER 15 > (setf (properties.getproperty p "ricky") "lucy")
215"lucy"
216
217CL-USER 16 > (object.tostring p)
218"{ricky=lucy, fred=ethel}"
219
220CL-USER 17 > (properties.size p)
2212
222
223;totally dynamic access, create wrappers as you need
224CL-USER 18 > (def-java-class "java.lang.Class")
225#&lt;STANDARD-CLASS CLASS. 20680EC4>
226
227CL-USER 19 > (class.getname (object.getclass p))
228"java.util.Properties"
229
230CL-USER 20 > (def-java-class "java.util.Enumeration")
231#&lt;STANDARD-CLASS ENUMERATION. 20669274>
232
233;no need to wait for the vendor to enhance the language - you use Lisp!
234CL-USER 21 > (defmacro doenum ((e enum) &body body)
235               (let ((genum (gensym)))
236                 `(let ((,genum ,enum))
237                    (do ()
238                        ((not (enumeration.hasmoreelements ,genum)))
239                      (let ((,e (enumeration.nextelement ,genum)))
240                        ,@body)))))
241DOENUM
242
243;can't do this in Java yet
244CL-USER 22 > (doenum (prop (properties.elements p)) (print (object.tostring prop)))
245
246"lucy"
247"ethel"
248NIL
249
250;doc strings are created giving original Java signatures and indicating
251overloads
252CL-USER 23 > (documentation 'properties.getproperty 'function)
253"java.lang.String getProperty(java.lang.String,java.lang.String)
254java.lang.String getProperty(java.lang.String)
255"
256
257CL-USER 24 > (documentation 'properties.new 'function)
258"java.util.Properties()
259java.util.Properties(java.util.Properties)
260"
261        </pre>
262
263        <a name="api"></a> <h2>API Reference</h2>
264
265        <a name="jvmcreation"></a> <h3>JVM Creation and Initialization</h3>
266        <ul>
267
268                <li>
269                        <a name="jnilibpath"></a><code>*jni-lib-path*</code>
270                        <p>
271                        Set this to point to your jvm dll prior to calling create-jvm.
272                        </p>
273                </li>
274
275                <li>
276                        <strong>Function</strong> <code>(create-jvm &rest option-strings) -> unspecified</code>
277                        <p>
278                        Creates/starts the JVM. This can only be done once (a Java limitation). <em>You
279                        must call this prior to calling any other jfli function.</em> The option strings
280                        can be used to control the JVM, esp. the classpath:
281                        </p>
282                        <p>
283                        <pre>(create-jvm "-Djava.class.path=/Lisp/jfli.jar")</pre>
284                        </p>
285                        <p>
286                        See the JNI documentation for other initialization options.
287                        </p>
288                </li>
289
290                <li>
291                        <a name="enableproxies"></a> <strong>Function</strong> <code>(enable-java-proxies)
292                        -> unspecified</code>
293                        <p>
294                        Sets up the Java->Lisp callback support. Must be called (once) before any calls
295                        to new-proxy, and requires jfli.jar be in the classpath.
296                        </p>
297                </li>
298        </ul>
299
300        <a name="wrappergen"></a> <h3>Wrapper Generation</h3>
301
302        <ul>
303                <li>
304                        <strong>Macro</strong> <code>(def-java-class full-class-name) -> unspecified</code>
305                        <p>
306                        Given the package-qualified, case-correct name of a Java class as a string, will
307                        generate wrapper functions for its public constructors, fields and methods.
308                        </p>
309                        <p>
310                        The core API for generation interfaces to Java is the def-java-class macro. This
311                        macro will, at expansion time, use Java reflection to find all of the public
312                        constructors, fields and methods of the given class and generate functions to
313                        access them.
314                        </p>
315                        <h4>The Generated API</h4> When you e.g. <code>(def-java-class "java.lang.ClassName
316                         ")</code> you get several symbols/functions:
317                        <ul>
318                                <li>
319                                        A package named <code>|java.lang|</code> (note case)<br>
320                                        from which the following are exported:
321                                </li>
322                                <li>
323                                        A class-symbol: <code>classname.</code> (note the dot is part of the name)<br>
324                                        which can usually be used where a typename is required. It also serves as the
325                                        name of the Lisp typed reference class.
326                                </li>
327                                <li>
328                                        Every non-interface class with a public constructor will get;
329                                        <ul>
330                                                <li>
331                                                        A constructor, <code>(classname.new &rest args) -> typed-reference</code>, which
332                                                        returns a typed reference to the newly created object
333                                                </li>
334                                                <li>
335                                                        A method defined on <a href=#makenew><code>make-new</code></a>, ultimately
336                                                        calling <code>classname.new</code>, specialized on (the value of) the class-symbol
337                                                </li>
338                                        </ul>
339                                        Note that if the constructor is overloaded, there is just one function generated,
340                                        which handles overload resolution. The function documentation string describes
341                                        the constructor signature(s) from the Java perspective. The same argument
342                                        conversions are performed as are for fields (see below).
343                                </li>
344                                <li>
345                                        All public fields will get a getter function:<br>
346                                        <code>(classname.fieldname [instance]) -> field value</code><br>
347                                        and a setter:<br>
348                                        <code>(setf classname.fieldname [instance])</code><br>
349                                        Instance field wrappers take a first arg which is the instance. Static fields
350                                        get a symbol-macro <code>*classname.fieldname*</code>
351                                        <p>
352                                        If the type of the field is primitive, the field value will be converted to a
353                                        native Lisp value. If it is a Java String, it will be converted to a Lisp string.
354                                        Otherwise, a generic reference to the Java object is returned. Similarly, when
355                                        setting, Lisp values will be accepted for primitives, Lisp strings for Strings,
356                                        or (generic or typed) references for reference types.
357                                        </p>
358                                </li>
359                                <li>
360                                        Every public method will get a wrapper function:<br>
361                                        <code>(classname.methodname &rest args) -> return-value</code><br>
362                                        As with constructors, if a method is overloaded a single wrapper is created that
363                                        handles overload resolution. If a method follows the JavaBeans property protocol
364                                        (i.e. it is called <code>getSomething</code> or <code>isSomething</code> and
365                                        there is a corresponding <code>setSomething</code>), then a <code>(setf
366                                        classname.methodname)</code> will be defined that calls the latter.
367                                        <p>
368                                        The same argument and return value conversions are performed as are for fields.
369                                        The function documentation string describes the method signature(s) from the
370                                        Java perspective.
371                                        </p>
372
373                                </li>
374                                <li>
375                                        A Lisp class with the class-symbol as its name. It will have as its superclasses
376                                        other Lisp classes corresponding to the Java superclass/superinterfaces, some of
377                                        which may be forward-referenced-classes.  An instance of this class will be
378                                        returned by classname.new/make-new/new, at which point the entire hierarchy will
379                                        consist of finalized standard-classes.
380                                </li>
381                                <li>
382                                        Note that, due to the need to reference other Java types during the definition
383                                        of a class wrapper, symbols, classes, and packages relating to those other types
384                                        may also be created. In all cases they will be created with names and
385                                        packages as described above.
386                                </li>
387                        </ul>
388                </li>
389
390                <li>
391                        <strong>Function</strong> <code>(get-jar-classnames jar-file-name &rest packages)
392                        -> list-of-strings</code>
393                        <p>
394                        Returns a list of class name strings. Packages should be strings of the form "java/lang
395                         " for recursive lookup and "java/util/" (note trailing slash) for non-recursive.
396                        </p>
397                </li>
398                <li>
399                        <strong>Function</strong> <code>(dump-wrapper-defs-to-file filename classnames)  ->
400                        unspecified</code>
401                        <p>
402                        Given a list of classnames (say from <code>get-jar-classnames</code>), writes
403                        calls to <code>def-java-class</code> to a file:
404                        </p>
405                        <pre>
406(dump-wrapper-defs-to-file "/lisp/java-lang.lisp"
407  (get-jar-classnames "/j2sdk1.4.2_01/jre/lib/rt.jar " "java/lang/"))
408(compile-file "/lisp/java-lang")
409(load "/lisp/java-lang")
410(use-package "java.lang")
411;Wrappers for all of java.lang are now available
412</pre>
413                </li>
414        </ul>
415
416        <a name="objects"></a> <h3>Object Creation</h3>
417        <ul>
418                <li>
419                        <strong><a name="makenew"></a>Generic Function</strong> <code>(make-new class-symbol
420                        &rest args) -> typed-reference</code>
421                        <p>
422                        Allows for definition of before/after methods on constructors. Calls <code>classname.new</code>.
423                        The new macro expands into a call to this.
424                        </p>
425                </li>
426                <li>
427                        <strong>Macro</strong> <code>(new class-spec &rest args) -> typed-reference</code>
428                        <br>
429                        <p>
430                        class-spec -> class-name | (class-name this-name)<br>
431                        class-name -> "package.qualified.ClassName" | classname.<br>
432                        args -> [actual-arg]* [init-arg-spec]*<br>
433                        init-arg-spec -> init-arg | (init-arg)<br>
434                        init-arg -> :settable-field-or-method [params]* value (note keyword)<br>
435                        | .method-name [args]* (note leading dot)<br>
436                        </p>
437
438                        <p>
439                        Creates a new instance of class-name, by expanding into a call to the make-new
440                        generic function, then initializes it by setting fields or accessors and/or
441                        calling member functions. If this-name is supplied, it will be bound to the
442                        newly-allocated object and available to the init-args:
443                        </p>
444                        <pre>
445(new (button. this) shell *SWT.CENTER*   ;the actual args
446       :gettext "Call Lisp"               ;a javabean property
447       (.addlistener *swt.selection*      ;a method call
448         (new-proxy (listener.
449                     (handleevent (event)
450                       (declare (ignore event))
451                       (setf (button.gettext this)   ;this is bound to new instance
452                             (format nil "~A ~A"
453                                   (lisp-implementation-type)
454                                   (lisp-implementation-version)))
455                       nil))))
456      .setsize 200 100                    ;can omit parens
457      (.setlocation 40 40))
458                        </pre>
459                        Expands into:
460                        <pre>
461(LET* ((#:G598 (MAKE-NEW BUTTON. SHELL *SWT.CENTER*)) (THIS #:G598))
462  (SETF (BUTTON.GETTEXT #:G598) "Call Lisp")
463  (BUTTON.ADDLISTENER #:G598
464                      *SWT.SELECTION*
465                      (NEW-PROXY (LISTENER.
466                                  (HANDLEEVENT
467                                   (EVENT)
468                                   (DECLARE (IGNORE EVENT))
469                                   (SETF (BUTTON.GETTEXT THIS)
470                                         (FORMAT NIL
471                                                 "~A ~A"
472                                                 (LISP-IMPLEMENTATION-TYPE)
473                                                 (LISP-IMPLEMENTATION-VERSION)))
474                                   NIL))))
475  (BUTTON.SETSIZE #:G598 200 100)
476  (BUTTON.SETLOCATION #:G598 40 40)
477  #:G598)
478                        </pre>
479
480                </li>
481        </ul>
482        <a name="arrays"></a> <h3>Array Support</h3>
483        <ul>
484                <li>
485                        <strong>Generic Function</strong> <code>(make-new-array type &rest dimensions) ->
486                        reference to new array</code>
487                        <p>
488                        Generic function with methods defined for all Java class designators:
489                        <ul>
490                                <li>
491                                        A "package.qualified.ClassName" string
492                                </li>
493                                <li>
494                                        (the value of) A class-symbol - classname.
495                                </li>
496                                <li>
497                                        A primitive designator keyword - :boolean|:byte|:char|:double|:float|:int|:long|:short
498                                </li>
499                        </ul>
500                        </p>
501                        <p>
502                        Creates a Java array of the requested type with the requested dimensions.
503                        </p>
504                </li>
505                <li>
506                        <strong>Function</strong> <code>(jlength array) -> integer</code>
507                        <p>
508                        Like length, for Java arrays
509                        </p>
510                </li>
511
512                <li>
513                        <strong>Function</strong> <code>(jref array &rest subscripts) -> reference</code>
514                        <p>
515                        Like aref, for Java arrays of non-primitive (reference) types, settable.
516                        </p>
517                </li>
518                <li>
519                        <strong>Function</strong> <code>(jref-xxx array &rest subscripts) -> value</code>
520                        <p>
521                        Where xxx = boolean|byte|char|double|float|int|long|short. Like jref, for Java
522                        arrays of primitive types, settable.
523                        </p>
524                </li>
525
526        </ul>
527
528        <a name="proxies"></a> <h3>Proxies - Java calling back to Lisp</h3>
529        <ul>
530                <p>
531                Proxies allow the creation of Java objects that implement one or more interfaces
532                in Lisp, and thus callbacks from Java to Lisp. You must call <a href=#enableproxies>
533                <code>enable-java-proxies</code></a> before using this proxy API. A significant
534                limitation is that LispWorks appears to not support calls back into Lisp other
535                than from threads initiated by Lisp, so you must ensure that the proxy will not
536                be called from an arbitrary Java thread!
537                </p>
538                <li>
539                        <strong>Macro</strong> <code>(new-proxy &rest interface-defs) -> reference</code>
540                        <p>
541                        interface-def -> (interface-name method-defs+)<br>
542                        interface-name -> "package.qualified.ClassName" | classname. (must name a Java
543                        interface type)<br>
544                        method-def -> (method-name arg-defs* body) <br>
545                        arg-def -> arg-name | (arg-name arg-type) arg-type -> "package.qualified.ClassName
546                         " | classname. | :primitive <br>
547                        method-name -> symbol | string (matched case-insensitively)
548                        </p>
549
550                        <p>
551                        Creates, registers and returns a Java object that implements the supplied
552                        interfaces
553                        </p>
554                </li>
555                <li>
556                        <strong>Function</strong> <code>(unregister-proxy proxy) -> unspecified</code>
557                        <p>
558                        Stops handling for the proxy (which must have been created by <code>new-proxy</code>)
559                        and removes references from the Lisp side. Make sure it is no longer referenced
560                        from Java first!
561                        </p>
562                </li>
563        </ul>
564
565        <a name="utilities"></a> <h3>Utilities</h3>
566        <ul>
567
568                <li>
569                        <strong>Function</strong> <code>(jeq obj1 obj2) -> boolean</code>
570                        <p>
571                        Are the 2 java objects the same object? Note that this is not the same as Object.equals()
572                        <p>
573                </li>
574                <li>
575                        <strong>Function</strong> <code>(find-java-class class-sym-or-string) ->
576                        reference to Java Class object</code>
577                        <p>
578                        Given a Java class designator, returns the Java Class object. Use this in
579                        preference to Class.forName() when using jfli.
580                        </p>
581                </li>
582
583                <li>
584                        <strong>Function</strong> <code>(make-typed-ref java-ref) -> typed-reference</code>
585                        <p>
586                        Given a generic Java reference, determines the full type of the object and
587                        returns an instance of a typed reference wrapper. classname.new/make-new/new always return typed
588                        references, but since Java methods might return Object or some interface type,
589                        and we don't want to always incur the cost of type determination, field and
590                        method wrapper functions return generic references. Use this function to create
591                        a typed reference corresponding to the full actual type of the object when
592                        desired.
593                        </p>
594                 </li>
595                <li>
596                        <strong>Function</strong> <code>(box-xxx value) -> reference to Java primitive wrapper class</code>
597                        <p>Where xxx = boolean|byte|char|double|float|int|long|short|string.
598                        Given a compatible Lisp value, creates an instance of the corresponding Java primitive wrapper class,
599                        e.g. Integer. This should rarely be needed, but can be used to force overloading resolution</p>
600                </li>
601                <li>
602                        <strong>Function</strong> <code>(unbox-xxx ref) -> Lisp value</code>
603                        <p>Where xxx = boolean|byte|char|double|float|int|long|short|string.
604                        Given an instance of a Java primitive wrapper class, creates an instance of the corresponding
605                        compatible Lisp value. This should rarely be needed, but can be used to unbox values returned by Java
606                        Object-based APIs.</p>
607                </li>
608        </ul>
609       
610        <a name="summary"></a> <h3>Summary</h3>
611<p>
612I hope you find jfli useful. It is my sincere intent that it enhance the utility and interoperability of Common Lisp,
613a language with which I am still becoming familiar, and grow to appreciate more every day. I welcome comments
614and code contributions.
615</p>
616<p>
617Rich Hickey, July 2004
618</p>
619        </body>
620</html>
Note: See TracBrowser for help on using the repository browser.