Ignore:
Timestamp:
May 14, 2011, 3:39:10 PM (8 years ago)
Author:
gz
Message:

Initial support for incremental code coverage info.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/source/doc/src/using.xml

    r14794 r14798  
    29332933(progn
    29342934  (do-this)
    2935   (setq that ...) ...))
    2936   </programlisting>
     2935  (setq that ...) ...))</programlisting>
    29372936  do:
    29382937  <programlisting>
     
    29402939  (do-this)
    29412940  (setq that ...)  ...)
    2942 (init-this-and-that)
    2943   </programlisting>
     2941(init-this-and-that)</programlisting>
     2942
    29442943Then you can see the coverage information in the definition of
    29452944<literal>init-this-and-that</literal>.
     
    29562955  <programlisting>
    29572956(setq ccl:*compile-code-coverage* t)
    2958 (recompile-all-your-files)
    2959   </programlisting>
     2957(recompile-all-your-files)</programlisting>
    29602958</para>
    29612959<para>
     
    29642962</para>
    29652963<para>
    2966   When you execute function loaded from instrumented fasl files, they
    2967   will record coverage information every time they are executed. The
    2968   system keeps track of which instrumented files have been loaded.
     2964  When you execute functions loaded from instrumented fasl files, they
     2965  will record coverage information every time they are executed.
     2966  You can examine that information by calling <literal>ccl:report-coverage</literal>
     2967  or <literal>ccl:coverage-statistics</literal>.
    29692968</para>
     2969 <para>
     2970   While recording coverage, you can collect incremental coverage deltas between any two points
     2971   in time.  You might do this while running a test suite, to record the coverage for
     2972   each test, for example:
     2973  <programlisting>
     2974(ccl:reset-incremental-coverage)
     2975(loop with coverage = (make-hash-table)
     2976      for test in (tests-to-run)
     2977      do (run-test test)
     2978      do (setf (gethash test coverage) (ccl:get-incremental-coverage))
     2979      finally (return coverage))</programlisting>creates a hash table mapping a test to a representation of all coverage recorded while running the
     2980   test.  This hash table can then be passed to <literal>ccl:incremental-coverage-svn-matches</literal>
     2981   or <literal>ccl:incremental-coverage-source-matches</literal>.
     2982</para>
     2983</sect2>
     2984
     2985<sect2 id="code-coverage-functions-and-variables"><title>Functions and Variables</title>
    29702986<para>
    29712987  The following functions can be used to manage the coverage data:
     
    30873103    <para>
    30883104      Gets rid of the information about which instrumented files have
    3089       been loaded, so ccl:report-coverage will not report any files,
    3090       and ccl:save-coverage-in-file will not save any info, until more
     3105      been loaded, so <literal>ccl:report-coverage</literal> will not report any files,
     3106      and <literal>ccl:save-coverage-in-file</literal> will not save any info, until more
    30913107      instrumented files are loaded.
    30923108    </para>
     
    31173133      coverage state later. This allows you to combine multiple runs
    31183134      or continue in a later session. Equivalent to
    3119       (ccl:write-coverage-to-file (ccl:save-coverage) pathname).
     3135      <literal>(ccl:write-coverage-to-file (ccl:get-coverage) pathname)</literal>.
    31203136    </para>
    31213137  </refsect1>
     
    31473163      info is only restored for files that have been loaded in this
    31483164      session. For example if in a previous session you had loaded
    3149       "foo.lx86fsl" and then saved the coverage info, in this session
    3150       you must load the same "foo.lx86fsl" before calling
    3151       restore-coverage-from-file in order to retrieve the stored
    3152       coverage info for "foo".  Equivalent to (ccl:restore-coverage
    3153       (ccl:read-coverage-from-file pathname)).
     3165      <filename>"foo.lx86fsl"</filename> and then saved the coverage info, in this session
     3166      you must load the same <filename>"foo.lx86fsl"</filename> before calling
     3167      <literal>restore-coverage-from-file</literal> in order to retrieve the stored
     3168      coverage info for "foo".  Equivalent to <literal>(ccl:restore-coverage
     3169      (ccl:read-coverage-from-file pathname))</literal>.
    31543170    </para>
    31553171  </refsect1>
    31563172</refentry>
    31573173
    3158 <refentry id="f_save-coverage">
    3159   <indexterm zone="f_save-coverage">
    3160     <primary>save-coverage</primary>
     3174<refentry id="f_get-coverage">
     3175  <indexterm zone="f_get-coverage">
     3176    <primary>get-coverage</primary>
    31613177  </indexterm>
    31623178 
    31633179  <refnamediv>
    3164     <refname>SAVE-COVERAGE</refname>
     3180    <refname>GET-COVERAGE</refname>
    31653181    <refpurpose>
    31663182      Returns a snapshot of the current coverage data.
     
    31733189      Returns a snapshot of the current coverage data. A snapshot is a
    31743190      copy of the current coverage state. It can be saved in a file
    3175       with ccl:write-coverage-to-file, reinstated back as the current
    3176       state with ccl:restore-coverage, or combined with other
    3177       snapshots with ccl:combine-coverage.
     3191      with <literal>ccl:write-coverage-to-file</literal>, reinstated back as the current
     3192      state with <literal>ccl:restore-coverage</literal>, or combined with other
     3193      snapshots with <literal>ccl:combine-coverage</literal>.
    31783194    </para>
    31793195  </refsect1>
     
    32533269    <para>
    32543270      Saves the coverage snapshot in a file. The snapshot can be
    3255       loaded back with ccl:read-coverage-from-file or loaded and
    3256       restored with ccl:restore-coverage-from-file. Note that the file
     3271      loaded back with <literal>ccl:read-coverage-from-file</literal> or loaded and
     3272      restored with <literal>ccl:restore-coverage-from-file</literal>. Note that the file
    32573273      created is actually a lisp source file and can be compiled for
    32583274      faster loading.
     
    32833299      Returns the snapshot saved in pathname. Doesn't affect the
    32843300      current coverage state. pathname can be the file previously
    3285       created with ccl:write-coverage-to-file or
    3286       ccl:save-coverage-in-file, or it can be the name of the fasl
     3301      created with <literal>ccl:write-coverage-to-file</literal> or
     3302      <literal>ccl:save-coverage-in-file</literal>, or it can be the name of the fasl
    32873303      created from compiling such a file.
    32883304    </para>
     
    32983314    <refname>COVERAGE-STATISTICS</refname>
    32993315    <refpurpose>
    3300       Returns a sequence of ccl:coverage-statistics objects, one per source file.
     3316      Returns a sequence of <literal>ccl:coverage-statistics</literal> objects, one per source file.
    33013317    </refpurpose>
    33023318    <refclass>Function</refclass>
     
    33103326  <refsect1><title>Description</title>
    33113327    <para>
    3312       Returns a sequence ccl:coverage-statistics objects, one for each
     3328      Returns a sequence of <literal>ccl:coverage-statistics</literal> objects, one for each
    33133329      source file, containing the same information as that written to
    3314       the statistics file by ccl:report-coverage. The following
    3315       accessors are defined for ccl:coverage-statistics objects:
     3330      the statistics file by <literal>ccl:report-coverage</literal>. The following
     3331      accessors are defined for <literal>ccl:coverage-statistics</literal> objects:
    33163332      <variablelist>
    33173333      <varlistentry>
     
    34103426  </refsect1>
    34113427</refentry>
     3428
     3429<refentry id="f_reset-incremental-coverage">
     3430  <indexterm zone="f_reset-incremental-coverage">
     3431    <primary>reset-incremental-coverage</primary>
     3432  </indexterm>
     3433 
     3434  <refnamediv>
     3435    <refname>RESET-INCREMENTAL-COVERAGE</refname>
     3436    <refpurpose>
     3437      Reset incremental coverage.
     3438    </refpurpose>
     3439    <refclass>Function</refclass>
     3440  </refnamediv>
     3441
     3442  <refsynopsisdiv>
     3443    <synopsis><function>reset-incremental-coverage</function>
     3444    </synopsis>
     3445  </refsynopsisdiv>
     3446
     3447  <refsect1><title>Description</title>
     3448    <para>
     3449      Marks a starting point for recording incremental coverage.
     3450      Note that calling
     3451      this function does not affect regular coverage data (whereas calling
     3452      <literal>ccl:reset-coverage</literal> resets incremental coverage as well).
     3453    </para>
     3454  </refsect1>
     3455</refentry>
     3456
     3457
     3458<refentry id="f_get-incremental-coverage">
     3459  <indexterm zone="f_get-incremental-coverage">
     3460    <primary>get-incremental-coverage</primary>
     3461  </indexterm>
     3462 
     3463  <refnamediv>
     3464    <refname>GET-INCREMENTAL-COVERAGE</refname>
     3465    <refpurpose>
     3466      Returns the delta of coverage since the last incremental reset.
     3467    </refpurpose>
     3468    <refclass>Function</refclass>
     3469  </refnamediv>
     3470
     3471  <refsynopsisdiv>
     3472    <synopsis><function>get-incremental-coverage</function> &key; (reset t)
     3473    </synopsis>
     3474  </refsynopsisdiv>
     3475
     3476  <refsect1><title>Description</title>
     3477    <para>
     3478      Returns the delta of coverage since the last reset of incremental coverage.
     3479     If <literal>reset</literal> is true (the default), it also resets incremental coverage
     3480     now, so that the next call to <literal>get-incremental-coverage</literal> will return
     3481     the delta from this point.
     3482     </para>
     3483     <para>
     3484      Incremental coverage deltas are represented differently than the full coverage snapshots
     3485      returned by functions such as <literal>ccl:get-coverage</literal>.  Incremental
     3486      coverage uses an abbreviated format
     3487      and is missing some of the information in a full snapshot, and therefore cannot be passed to
     3488      functions documented to accept a <variable>snapshot</variable>, only to functions
     3489      specifically documented to accept incremental coverage deltas.
     3490     </para>
     3491  </refsect1>
     3492</refentry>
     3493
     3494<refentry id="f_incremental-coverage-source-matches">
     3495  <indexterm zone="f_incremental-coverage-source-matches">
     3496    <primary>incremental-coverage-source-matches</primary>
     3497  </indexterm>
     3498 
     3499  <refnamediv>
     3500    <refname>INCREMENTAL-COVERAGE-SOURCE-MATCHES</refname>
     3501    <refpurpose>
     3502      Find incremental coverage deltas intersecting source regions.
     3503    </refpurpose>
     3504    <refclass>Function</refclass>
     3505  </refnamediv>
     3506
     3507  <refsynopsisdiv>
     3508    <synopsis><function>incremental-coverage-source-matches</function> collection sources
     3509    </synopsis>
     3510  </refsynopsisdiv>
     3511
     3512  <refsect1><title>Arguments and Values</title>
     3513  <variablelist>
     3514    <varlistentry>
     3515      <term>collection</term>
     3516       <listitem>
     3517         <para>
     3518          A hash table mapping arbitrary keys to incremental coverage deltas, or a sequence of
     3519          incremental coverage deltas.
     3520         </para>
     3521        </listitem>
     3522    </varlistentry>
     3523    <varlistentry>
     3524      <term>sources</term>
     3525       <listitem>
     3526         <para>
     3527          A list of pathnames and/or source-notes, the latter representing a range within a file.
     3528         </para>
     3529        </listitem>
     3530    </varlistentry>
     3531   </variablelist>
     3532   </refsect1>
     3533
     3534  <refsect1><title>Description</title>
     3535    <para>
     3536     Given a hash table <literal>collection</literal> whose values are incremental coverage
     3537     deltas, return a list of all keys corresponding to those deltas that intersect any region
     3538     in <literal>sources</literal>.
     3539     </para>
     3540     <para>
     3541     For example if the deltas represent tests, then the returned value is a list of all tests
     3542     that cover some part of the source regions.
     3543     </para>
     3544     <para>
     3545     <literal>collection</literal> can also be a sequence of deltas, in which case a subsequence
     3546      of matching deltas is returned.  In particular you can test whether any particular delta
     3547      intersects the sources by passing it in as a single-element list.
     3548     </para>
     3549  </refsect1>
     3550</refentry>
     3551
     3552<refentry id="f_incremental-coverage-svn-matches">
     3553  <indexterm zone="f_incremental-coverage-svn-matches">
     3554    <primary>incremental-coverage-svn-matches</primary>
     3555  </indexterm>
     3556 
     3557  <refnamediv>
     3558    <refname>INCREMENTAL-COVERAGE-SVN-MATCHES</refname>
     3559    <refpurpose>
     3560      Find incremental coverage deltas matching changes from a particular subversion revision.
     3561    </refpurpose>
     3562    <refclass>Function</refclass>
     3563  </refnamediv>
     3564
     3565  <refsynopsisdiv>
     3566    <synopsis><function>incremental-coverage-svn-matches</function> collection &key; (directory (current-directory)) (revision :base)
     3567    </synopsis>
     3568  </refsynopsisdiv>
     3569
     3570  <refsect1><title>Arguments and Values</title>
     3571  <variablelist>
     3572    <varlistentry>
     3573      <term>collection</term>
     3574       <listitem>
     3575         <para>
     3576          A hash table mapping arbitrary keys to incremental coverage deltas, or a sequence of
     3577          incremental coverage deltas.
     3578         </para>
     3579        </listitem>
     3580    </varlistentry>
     3581    <varlistentry>
     3582      <term>directory</term>
     3583       <listitem>
     3584         <para>
     3585          The pathname of a subversion working directory.
     3586         </para>
     3587        </listitem>
     3588    </varlistentry>
     3589    <varlistentry>
     3590      <term>revision</term>
     3591       <listitem>
     3592         <para>
     3593          The revision to compare to the working directory, an integer or another
     3594          value whose printed representation is suitable for passing as the
     3595          <literal>--revision</literal> argument
     3596          to <filename>svn</filename>.
     3597         </para>
     3598        </listitem>
     3599    </varlistentry>
     3600   </variablelist>
     3601   </refsect1>
     3602
     3603  <refsect1><title>Description</title>
     3604    <para>
     3605     Given a hash table <literal>collection</literal> whose values are incremental coverage
     3606     deltas, return a list of all keys corresponding to those deltas that intersect any changed
     3607     source in <literal>directory</literal> since revision <literal>revision</literal> in subversion.
     3608     </para>
     3609     <para>
     3610     For example if the deltas represent tests, then the returned value is a list of all tests
     3611     that might be affected by the changes.
     3612     </para>
     3613     <para>
     3614     <literal>collection</literal> can also be a sequence of deltas, in which case a subsequence
     3615      of matching deltas is returned.  In particular you can test whether any particular delta
     3616      is affected by the changes by passing it in as a single-element list.
     3617     </para>
     3618  </refsect1>
     3619</refentry>
     3620
     3621
    34123622
    34133623<refentry id="v_compile-code-coverage">
     
    34603670      This macro arranges so that body doesn't record internal details
    34613671      of code coverage. It will be considered totally covered if it's
    3462       entered at all. The Common Lisp macros ASSERT and CHECK-TYPE use
     3672      entered at all. The Common Lisp macros <literal>ASSERT</literal> and <literal>CHECK-TYPE</literal> use
    34633673      this macro.
    34643674    </para>
     
    34903700 form will have the outer form's coverage color. If the syntax of outer form involves some non-executable
    34913701 forms, or forms that do not have coverage info of their own for whatever reason, then they will just
    3492  inherit the color of the outer form, because it doesn't get replaced with a color of its own.
     3702 inherit the color of the outer form, because they don't get repainted with a color of their own.
    34933703</para>
    34943704
Note: See TracChangeset for help on using the changeset viewer.