Changeset 14744


Ignore:
Timestamp:
Apr 28, 2011, 12:52:41 PM (14 years ago)
Author:
gz
Message:

Clean up coverage doc; add a section about interpreting the coloring results

File:
1 edited

Legend:

Unmodified
Added
Removed

  • trunk/source/doc/src/using.xml

    r14694 r14744  
    28112811<para>
    28122812  Because the code coverage information is associated with compiled
    2813   functions, load-time toplevel expressions do not get reported
    2814   on. You can work around this by creating a function and calling
     2813  functions, code coverage information is not available for load-time toplevel
     2814  expressions. You can work around this by creating a function and calling
    28152815  it. I.e. instead of
    28162816  <programlisting>
    … …  
    28272827  </programlisting>
    28282828Then you can see the coverage information in the definition of
    2829 init-this-and-that.
     2829<literal>init-this-and-that</literal>.
    28302830</para>
    28312831</sect2>
    … …  
    28632863  <refnamediv>
    28642864    <refname>REPORT-COVERAGE</refname>
    2865     <refpurpose>Generate code coverage report</refpurpose>
     2865    <refpurpose>Generate a code coverage report</refpurpose>
    28662866    <refclass>Function</refclass>
    28672867  </refnamediv>
    28682868
    28692869  <refsynopsisdiv>
    2870     <synopsis><function>report-coverage</function> &key;
     2870    <synopsis><function>report-coverage</function> output-file &key;
    28712871    (external-format :default) (statistics t) (html t)
    28722872    </synopsis>
    … …  
    28752875  <refsect1><title>Arguments and Values</title>
    28762876  <variablelist>
     2877    <varlistentry>
     2878      <term>output-file</term>
     2879      <listitem>
     2880        <para>
     2881          Pathname for the output index file.
     2882        </para>
     2883      </listitem>
     2884    </varlistentry>
     2885   
    28772886    <varlistentry>
    28782887      <term>html</term>
    28792888      <listitem>
    28802889        <para>
    2881           If non-nil, this will generate an HTML report, consisting of
    2882           an index file and one html file for each instrumented source
    2883           file that has been loaded in the current session. The
    2884           individual source file reports are stored in the same
    2885           directory as the index file.
     2890          If non-nil (the default), this will generate an HTML report, consisting of
     2891          an index file in <varname>output-file</varname> and, in the same directory,
     2892          one html file for each instrumented source file that has been loaded in the
     2893          current session.
    28862894        </para>
    28872895      </listitem>
    … …  
    28992907      <listitem>
    29002908        <para>
    2901           If :statistics is non-nil, a comma-separated file is also
     2909          If non-nil (the default), a comma-separated file is also
    29022910          generated with the summary of statistics. You can specify a
    29032911          filename for the statistics argument, otherwise
    2904           "statistics.csv" is created in the output directory. See
    2905           documentation of ccl:coverage-statistics below for a
     2912          "statistics.csv" is created in the directory of <varname>output-file</varname>.
     2913          See documentation of coverage-statistics below for a
    29062914          description of the values in the statistics file.
    29072915        </para>
    … …  
    29162924      do
    29172925    <programlisting>
    2918 (CCL:REPORT-COVERAGE "/my/dir/coverage/report.html")
     2926(REPORT-COVERAGE "/my/dir/coverage/report.html")
    29192927    </programlisting>
    29202928    and this would generate <filename>report.html</filename>,
    … …  
    29332941 
    29342942  <refnamediv>
    2935     <refname>reset-coverage</refname>
     2943    <refname>RESET-COVERAGE</refname>
    29362944    <refpurpose>
    29372945      Resets all coverage data back to the "Not Executed" state
    … …  
    29402948  </refnamediv>
    29412949
    2942   <refsect1><title>Summary</title>
     2950  <refsect1><title>Description</title>
    29432951    <para>
    29442952      Resets all coverage data back to the "Not Executed" state
    … …  
    29532961 
    29542962  <refnamediv>
    2955     <refname>clear-coverage</refname>
     2963    <refname>CLEAR-COVERAGE</refname>
    29562964    <refpurpose>
    29572965      Forget about all instrumented files that have been loaded.
    … …  
    29602968  </refnamediv>
    29612969
    2962   <refsect1><title>Summary</title>
     2970  <refsect1><title>Description</title>
    29632971    <para>
    29642972      Gets rid of the information about which instrumented files have
    … …  
    29762984 
    29772985  <refnamediv>
    2978     <refname>save-coverage-in-file</refname>
     2986    <refname>SAVE-COVERAGE-IN-FILE</refname>
    29792987    <refpurpose>
    29802988      Save all coverage into to a file so you can restore it later.
    … …  
    29882996  </refsynopsisdiv>
    29892997
    2990   <refsect1><title>Summary</title>
     2998  <refsect1><title>Description</title>
    29912999    <para>
    29923000      Saves all coverage info in a file, so you can restore the
    … …  
    30043012 
    30053013  <refnamediv>
    3006     <refname>restore-coverage-from-file</refname>
     3014    <refname>RESTORE-COVERAGE-FROM-FILE</refname>
    30073015    <refpurpose>
    30083016      Load coverage state from a file.
    … …  
    30163024  </refsynopsisdiv>
    30173025
    3018   <refsect1><title>Summary</title>
     3026  <refsect1><title>Description</title>
    30193027    <para>
    30203028      Restores the coverage data previously saved with
    3021       CCL:SAVE-COVERAGE-IN-FILE, for the set of instrumented fasls
     3029      ccl:save-coverage-in-file, for the set of instrumented fasls
    30223030      that were loaded both at save and restore time. I.e. coverage
    30233031      info is only restored for files that have been loaded in this
    … …  
    30253033      "foo.lx86fsl" and then saved the coverage info, in this session
    30263034      you must load the same "foo.lx86fsl" before calling
    3027       ccl:restore-coverage-from-file in order to retrieve the stored
     3035      restore-coverage-from-file in order to retrieve the stored
    30283036      coverage info for "foo".  Equivalent to (ccl:restore-coverage
    30293037      (ccl:read-coverage-from-file pathname)).
    … …  
    30383046 
    30393047  <refnamediv>
    3040     <refname>save-coverage</refname>
     3048    <refname>SAVE-COVERAGE</refname>
    30413049    <refpurpose>
    30423050      Returns a snapshot of the current coverage data.
    … …  
    30453053  </refnamediv>
    30463054
    3047   <refsect1><title>Summary</title>
     3055  <refsect1><title>Description</title>
    30483056    <para>
    30493057      Returns a snapshot of the current coverage data. A snapshot is a
    … …  
    30743082  </refsynopsisdiv>
    30753083
    3076   <refsect1><title>Summary</title>
     3084  <refsect1><title>Description</title>
    30773085    <para>
    30783086      Reinstalls a coverage snapshot as the current coverage state.
    … …  
    30803088  </refsect1>
    30813089</refentry>
     3090
     3091<refentry id="f_combine-coverage">
     3092  <indexterm zone="f_combine-coverage">
     3093    <primary>combine-coverage</primary>
     3094  </indexterm>
     3095 
     3096  <refnamediv>
     3097    <refname>COMBINE-COVERAGE</refname>
     3098    <refpurpose>
     3099      Combines multiple coverage snapshots into one.
     3100    </refpurpose>
     3101    <refclass>Function</refclass>
     3102  </refnamediv>
     3103
     3104   <refsynopsisdiv>
     3105     <synopsis><function>combine-coverage</function> snapshots
     3106     </synopsis>
     3107   </refsynopsisdiv>
     3108
     3109  <refsect1><title>Description</title>
     3110    <para>
     3111      Takes a list of coverage snapshots and returns a new coverage snapshot
     3112      representing a union of all the coverage data.
     3113    </para>
     3114  </refsect1>
     3115</refentry>
     3116
    30823117
    30833118<refentry id="f_write-coverage-to-file">
    … …  
    30993134  </refsynopsisdiv>
    31003135
    3101   <refsect1><title>Summary</title>
     3136  <refsect1><title>Description</title>
    31023137    <para>
    31033138      Saves the coverage snapshot in a file. The snapshot can be
    … …  
    31283163  </refsynopsisdiv>
    31293164
    3130   <refsect1><title>Summary</title>
     3165  <refsect1><title>Description</title>
    31313166    <para>
    31323167      Returns the snapshot saved in pathname. Doesn't affect the
    … …  
    31473182    <refname>COVERAGE-STATISTICS</refname>
    31483183    <refpurpose>
    3149       Returns a sequence of coverage-statistics objects, one per source file.
     3184      Returns a sequence of ccl:coverage-statistics objects, one per source file.
    31503185    </refpurpose>
    31513186    <refclass>Function</refclass>
    … …  
    31573192  </refsynopsisdiv>
    31583193
    3159   <refsect1><title>Summary</title>
     3194  <refsect1><title>Description</title>
    31603195    <para>
    31613196      Returns a sequence ccl:coverage-statistics objects, one for each
    … …  
    31653200      <variablelist>
    31663201      <varlistentry>
    3167         <term><function>ccl:coverage-source-file</function></term>
     3202        <term><function>coverage-source-file</function></term>
    31683203        <listitem>
    31693204          <para>
    … …  
    31733208      </varlistentry>
    31743209      <varlistentry>
    3175         <term><function>ccl:coverage-expressions-total</function></term>
     3210        <term><function>coverage-expressions-total</function></term>
    31763211        <listitem>
    31773212          <para>
    … …  
    31813216      </varlistentry>
    31823217      <varlistentry>
    3183         <term><function>ccl:coverage-expressions-entered</function></term>
     3218        <term><function>coverage-expressions-entered</function></term>
    31843219        <listitem>
    31853220          <para>
    … …  
    31903225      </varlistentry>
    31913226      <varlistentry>
    3192         <term><function>ccl:coverage-expressions-covered</function></term>
     3227        <term><function>coverage-expressions-covered</function></term>
    31933228        <listitem>
    31943229          <para>
    … …  
    31983233      </varlistentry>
    31993234      <varlistentry>
    3200         <term><function>ccl:coverage-unreached-branches</function></term>
     3235        <term><function>coverage-unreached-branches</function></term>
    32013236        <listitem>
    32023237          <para>
    … …  
    32063241      </varlistentry>
    32073242      <varlistentry>
    3208         <term><function>ccl:coverage-code-forms-total</function></term>
     3243        <term><function>coverage-code-forms-total</function></term>
    32093244        <listitem>
    32103245          <para>
    … …  
    32163251      </varlistentry>
    32173252      <varlistentry>
    3218         <term><function>ccl:coverage-code-forms-covered</function></term>
     3253        <term><function>coverage-code-forms-covered</function></term>
    32193254        <listitem>
    32203255          <para>
    … …  
    32243259      </varlistentry>
    32253260      <varlistentry>
    3226         <term><function>ccl:coverage-functions-total</function></term>
     3261        <term><function>coverage-functions-total</function></term>
    32273262        <listitem>
    32283263          <para>
    … …  
    32323267      </varlistentry>
    32333268      <varlistentry>
    3234         <term><function>ccl:coverage-functions-fully-covered</function></term>
     3269        <term><function>coverage-functions-fully-covered</function></term>
    32353270        <listitem>
    32363271          <para>
    … …  
    32403275      </varlistentry>
    32413276      <varlistentry>
    3242         <term><function>ccl:coverage-functions-partly-covered</function></term>
     3277        <term><function>coverage-functions-partly-covered</function></term>
    32433278        <listitem>
    32443279          <para>
    … …  
    32483283      </varlistentry>
    32493284      <varlistentry>
    3250         <term><function>ccl:coverage-functions-not-entered</function></term>
     3285        <term><function>coverage-functions-not-entered</function></term>
    32513286        <listitem>
    32523287          <para>
    … …  
    32683303    <refname>*COMPILE-CODE-COVERAGE*</refname>
    32693304    <refpurpose>
    3270       When true, instrument functions for code coverage.
     3305      When true, instrument functions being compiled to collect code coverage information.
    32713306    </refpurpose>
    32723307    <refclass>Variable</refclass>
    … …  
    32783313  </refsynopsisdiv>
    32793314
    3280   <refsect1><title>Summary</title>
     3315  <refsect1><title>Description</title>
    32813316    <para>
    32823317      This variable controls whether functions are instrumented for
    … …  
    33053340  </refsynopsisdiv>
    33063341
    3307   <refsect1><title>Summary</title>
     3342  <refsect1><title>Description</title>
    33083343    <para>
    33093344      This macro arranges so that body doesn't record internal details
    … …  
    33163351
    33173352</sect2>
     3353
     3354<sect2 id="code-coverage-interpreting-code-coloring"><title>Interpreting Code Coloring</title>
     3355<para>
     3356
     3357 The output of ccl:report-coverage consists of formatted source code, with coverage indicated by
     3358 coloring.  Four colors are used: dark green for forms that compiled to code in which every single
     3359 instruction was executed, light green for forms that have been entered but weren't totally covered, red
     3360 for forms that were never entered, and the page background color for toplevel forms that weren't
     3361 instrumented.
     3362
     3363</para>
     3364<para>
     3365 The source coloring is applied from outside in.  So for example if you have
     3366
     3367  <programlisting>
     3368(outer-form ... (inner-form ...) ...)
     3369  </programlisting>
     3370
     3371 first the whole outer form is painted with whatever color expresses the outer form coverage, and then the
     3372 inner form color is replaced with whatever color expresses the inner form coverage.  One consequence of
     3373 this approach is that every part of the outer form that is not specifically inside some executable inner
     3374 form will have the outer form's coverage color. If the syntax of outer form involves some non-executable
     3375 forms, or forms that do not have coverage info of their own for whatever reason, then they will just
     3376 inherit the color of the outer form, because it doesn't get replaced with a color of its own.
     3377</para>
     3378
     3379<para>
     3380 One case in which this approach can be confusing is in the case of symbols.  As noted in the Limitations
     3381 section, coverage information is not recorded for variables; hence the coloring of a variable does not
     3382 convey information about whether the variable was evaluated or not -- that information is not available,
     3383 and the variable just inherits the color of the form that contains it.
     3384</para>
     3385
     3386
     3387</sect2>
    33183388</sect1>
     3389
    33193390<sect1 id="other-extensions"><title>Other Extensions</title>
    33203391<refentry id="v_quit">
Note: See TracChangeset for help on using the changeset viewer.