Changeset 14634


Ignore:
Timestamp:
Feb 3, 2011, 7:43:20 AM (8 years ago)
Author:
gb
Message:

Document new CHUD stuff, :NATIVE arg to SAVE-APPLICATION.

Location:
trunk/source
Files:
3 edited

Legend:

Unmodified
Added
Removed
  • trunk/source/doc/src/profile.xml

    r13573 r14634  
    11<?xml version="1.0" encoding="utf-8"?>
    2 <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
     2<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
     3"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
    34<!ENTITY rest "<varname>&amp;rest</varname>">
    45<!ENTITY key "<varname>&amp;key</varname>">
     
    910<!ENTITY CCL "Clozure CL">
    1011]>
    11 
    12   <chapter id="Profiling">
    13     <title>Profiling</title>
    14 
    15     <sect1 id="Profiling-Using-the-Linux-oprofile-system-level-profiler">
    16       <title>Using the Linux oprofile system-level profiler</title>
    17 
    18       <para><ulink url="http://oprofile.sourceforge.net"><code>oprofile</code></ulink> is
    19       a system-level profiler that's available for most modern Linux distributions.</para>
    20 
    21       <para>Use of oprofile and its companion programs isn't really documented here; what
    22       is described is a way of generating symbolic information that enables profiling
    23       summaries generated by the <code>opreport</code> program to identify lisp functions
    24       meaningfully.</para>
    25 
    26       <sect2 id="Profiling-oprofile-generating-a-lisp-image-for-use-with-oprofile">
    27         <title>Generating a lisp image for use with oprofile</title>
    28 
    29         <para>Modern Linux uses the 'ELF" (Executable and Linking Format) object file
    30         format; the oprofile tools can associate symbolic names with addresses in a
    31         memory-mapped file if that file appears to be an ELF object file and if it
    32         contains ELF symbol information that describes those memory regions.  So, the
    33         general idea is to make a lisp heap image that looks enough like an ELF shared
    34         library to fool the <code>oprofile</code> tools (we don't actually load heap
    35         images via ELF dynamic linking technology, but we can make it look like we
    36         did.)</para>
    37 
    38       </sect2>
    39       <sect2 id="Profiling-oprofile-prerequisites">
    40         <title>Prerequisites</title>
    41 
    42         <itemizedlist>
    43           <listitem>
    44             <para><code>oprofile</code> itself, which is almost certainly available via
    45             your distribution's package management system if not already
    46             preinstalled.</para>
    47           </listitem>
    48           <listitem>
    49             <para><code>libelf</code>, which provides utilities for reading and writing
    50             ELF files (and is likewise likely preinstalled or readily installable.)</para>
    51           </listitem>
    52         </itemizedlist>
    53       </sect2>
    54       <sect2 id="Profiling-oprofile-generating-elf-symbols-for-lisp-functions">
    55         <title>Generating ELF symbols for Lisp functions</title>
    56 
    57         <para>In order to create a lisp heap image which can be used for
    58         <code>oprofile</code>- based profiling, we need to:</para>
    59         <orderedlist>
    60           <listitem>
    61             <para>load any code that we want to profile</para>
    62           </listitem>
    63           <listitem>
    64             <para>generate a file that contains ELF symbol information describing the
    65             names and addresses of all lisp functions.</para>
    66             <para>This step involves doing (from within &CCL;)</para>
    67             <programlisting>
     12<chapter id="Profiling">
     13  <title>Profiling</title>
     14
     15  <sect1 id="Profiling-Using-the-Linux-oprofile-system-level-profiler">
     16    <title>Using the Linux oprofile system-level profiler</title>
     17
     18    <para><ulink
     19    url="http://oprofile.sourceforge.net"><code>oprofile</code></ulink> is a
     20    system-level profiler that's available for most modern Linux
     21    distributions.</para>
     22
     23    <para>Use of oprofile and its companion programs isn't really documented
     24    here; what is described is a way of generating symbolic information that
     25    enables profiling summaries generated by the <code>opreport</code> program
     26    to identify lisp functions meaningfully.</para>
     27
     28    <sect2 id="Profiling-oprofile-generating-a-lisp-image-for-use-with-oprofile">
     29      <title>Generating a lisp image for use with oprofile</title>
     30
     31      <para>Modern Linux uses the 'ELF" (Executable and Linking Format) object
     32      file format; the oprofile tools can associate symbolic names with
     33      addresses in a memory-mapped file if that file appears to be an ELF
     34      object file and if it contains ELF symbol information that describes
     35      those memory regions. So, the general idea is to make a lisp heap image
     36      that looks enough like an ELF shared library to fool the
     37      <code>oprofile</code> tools (we don't actually load heap images via ELF
     38      dynamic linking technology, but we can make it look like we did.)</para>
     39    </sect2>
     40
     41    <sect2 id="Profiling-oprofile-prerequisites">
     42      <title>Prerequisites</title>
     43
     44      <itemizedlist>
     45        <listitem>
     46          <para><code>oprofile</code> itself, which is almost certainly
     47          available via your distribution's package management system if not
     48          already preinstalled.</para>
     49        </listitem>
     50
     51        <listitem>
     52          <para><code>libelf</code>, which provides utilities for reading and
     53          writing ELF files (and is likewise likely preinstalled or readily
     54          installable.) Somewhat confusingly, there are two libelf
     55          implementations in widespread use on Linux, and different
     56          distributions refer to them by different names (they may be
     57          available as part of an 'elfutils' package.) The oprofile insterface
     58          was designed to work with a libelf implementation whose version
     59          number is currently around 147; the other (incompatible) libelf
     60          implementation has a version number around 0.8. It may be necessary
     61          to install the corresponding development package (-dev or -devel,
     62          usuallly) in order to actually be able to use the libelf shared
     63          library.</para>
     64        </listitem>
     65      </itemizedlist>
     66    </sect2>
     67
     68    <sect2 id="Profiling-oprofile-generating-elf-symbols-for-lisp-functions">
     69      <title>Generating ELF symbols for Lisp functions</title>
     70
     71      <para>In order to create a lisp heap image which can be used for
     72      <code>oprofile</code>- based profiling, we need to:</para>
     73
     74      <orderedlist>
     75        <listitem>
     76          <para>load any code that we want to profile</para>
     77        </listitem>
     78
     79        <listitem>
     80          <para>generate a file that contains ELF symbol information
     81          describing the names and addresses of all lisp functions.</para>
     82
     83          <para>This step involves doing (from within &CCL;)</para>
     84
     85          <programlisting>
    6886? (require "ELF")
    6987"ELF"
     
    7290? (ccl::write-elf-symbols-to-file "home:elf-symbols")
    7391            </programlisting>
    74             <para>The argument to CCL::WRITE-ELF-SYMBOLS-TO-FILE can be any writable
    75             pathname.  The function will do whatever's necessary to nail lisp functions
    76             down in memory (so that they aren't moved by GC), then write an ELF object
    77             file to the indicated pathname.  This typically takes a few seconds.</para>
    78           </listitem>
    79           <listitem>
    80             <para>Generate a lisp heap image in which the ELF symbols generated in the
    81             previous step are prepended.</para>
    82             <para>The function CCL:SAVE-APPLICATION provides a :PREPEND-KERNEL argument,
    83             which is ordinarily used to save a standalone application in which the kernel
    84             and heap image occupy a single file.  :PREPEND-KERNEL doesn't really care what
    85             it's prepending to the image, and we can just as easily ask it to prepend the
    86             ELF symbol file generated in the previous step.</para>
    87             <programlisting>
     92
     93          <para>The argument to CCL::WRITE-ELF-SYMBOLS-TO-FILE can be any
     94          writable pathname. The function will do whatever's necessary to nail
     95          lisp functions down in memory (so that they aren't moved by GC),
     96          then write an ELF object file to the indicated pathname. This
     97          typically takes a few seconds.</para>
     98        </listitem>
     99
     100        <listitem>
     101          <para>Generate a lisp heap image in which the ELF symbols generated
     102          in the previous step are prepended.</para>
     103
     104          <para>The function CCL:SAVE-APPLICATION provides a :PREPEND-KERNEL
     105          argument, which is ordinarily used to save a standalone application
     106          in which the kernel and heap image occupy a single file.
     107          :PREPEND-KERNEL doesn't really care what it's prepending to the
     108          image, and we can just as easily ask it to prepend the ELF symbol
     109          file generated in the previous step.</para>
     110
     111          <programlisting>
    88112? (save-application "somewhere/image-for-profiling"
    89113    :prepend-kernel "home:elf-symbols")
    90114            </programlisting>
    91             <para>If you then run</para>
    92             <programlisting>
    93 shell> ccl64 somewhare/image-for-profiling
     115
     116          <para>If you then run</para>
     117
     118          <programlisting>
     119shell&gt; ccl64 somewhare/image-for-profiling
    94120            </programlisting>
    95             <para>any lisp code sampled by oprofile in that image will be identified
    96             "symbolically" by <code>opreport</code>.</para>
    97           </listitem>
    98         </orderedlist>
    99       </sect2>
    100       <sect2 id="Profiling-oprofile-example">
    101         <title>Example</title>
    102         <programlisting>
     121
     122          <para>any lisp code sampled by oprofile in that image will be
     123          identified "symbolically" by <code>opreport</code>.</para>
     124        </listitem>
     125      </orderedlist>
     126    </sect2>
     127
     128    <sect2 id="Profiling-oprofile-example">
     129      <title>Example</title>
     130
     131      <programlisting>
    103132;;; Define some lisp functions that we want to profile and save
    104133;;; a profiling-enabled image.  In this case, we just want to
     
    120149;;; 'sudo' or equivalent:
    121150
    122 [~] gb@rinpoche> sudo opcontrol --no-vmlinux --setup
     151[~] gb@rinpoche&gt; sudo opcontrol --no-vmlinux --setup
    123152
    124153;;; Start the profiler
    125154
    126 [~] gb@rinpoche> sudo opcontrol --start
     155[~] gb@rinpoche&gt; sudo opcontrol --start
    127156Using 2.6+ OProfile kernel interface.
    128157Using log file /var/lib/oprofile/samples/oprofiled.log
     
    133162;;; Invoke "(FACT 10000)"
    134163
    135 [~] gb@rinpoche> ccl64 profiled-ccl
     164[~] gb@rinpoche&gt; ccl64 profiled-ccl
    136165Welcome to Clozure Common Lisp Version 1.2-r9198M-trunk  (LinuxX8664)!
    137166? (null (fact 10000))
     
    142171;;; we simply flush profiling data to disk, where 'opreport' can
    143172;;; find it.
    144 
    145 [~] gb@rinpoche> sudo opcontrol --dump
     173<token></token>
     174[~] gb@rinpoche&gt; sudo opcontrol --dump
    146175
    147176;;; Ask opreport to show us where we were spending time in the
    148177;;; 'profiled-ccl' image.
    149178
    150 [~] gb@rinpoche> opreport -l profiled-ccl | head
     179[~] gb@rinpoche&gt; opreport -l profiled-ccl | head
    151180CPU: Core 2, speed 1596 MHz (estimated)
    152181Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 100000
     
    1601897         0.0712  &lt;Compiled-function.%ZERO-TRAILING-SIGN-DIGITS.0x300040030F3F&gt;
    161190        </programlisting>
    162       </sect2>
    163       <sect2 id="Profiling-oprofile-Issues">
    164         <title>Issues</title>
    165         <para>CCL::WRITE-ELF-SYMBOLS-TO-FILE currently only works on x86-64; it certainly
    166         -could- be made to work on ppc32/ppc64 as well.</para>
    167 
    168         <para>So far, no one has been able to make oprofile/opreport options that're
    169         supposed to generate call-stack info generate meaningful call-stack info.</para>
    170 
    171         <para>As of a few months ago, there was an attempt to provide symbol info for
    172         oprofile/opreport "on the fly", e.g., for use in JIT compilation or other
    173         incremental compilations scenarios.  That's obviously more nearly The Right Thing,
    174         but it might be awhile before that experimental code makes it into widespread
    175         use.</para>
    176       </sect2>
    177     </sect1>
    178 
    179     <sect1 id="Profiling-Using-Apples-CHUD-metering-tools">
    180       <title>Using Apple's CHUD metering tools</title>
    181      
    182       <sect2 id="Profiling-CHUD-prerequisites">
    183         <title>Prerequisites</title>
    184 
    185         <para>Apple's CHUD metering tools are available (as of this writing) from:</para>
    186 
    187         <para><ulink url="ftp://ftp.apple.com/developer/Tool_Chest/Testing_-_Debugging/Performance_tools/">
    188           ftp://ftp.apple.com/developer/Tool_Chest/Testing_-_Debugging/Performance_tools/</ulink>.</para>
    189 
    190         <para>The CHUD tools are also generally bundled with Apple's XCode tools.  CHUD
    191         4.5.0 (which seems to be bundled with XCode 3.0) seems to work well with this
    192         interface; later versions may have problems.  Versions of CHUD as old as 4.1.1 may
    193         work with 32-bit PPC versions of CCL; later versions (not sure exactly -what-
    194         versions) added x86, ppc64, and x86-64 support.</para>
    195 
    196         <para>One way to tell whether any version of the CHUD tools is installed is to try
    197         to invoke the "shark" command-line program (/usr/bin/shark) from the shell:</para>
    198         <programlisting>
    199 shell> shark --help
    200         </programlisting>
    201         <para>and verifying that that prints a usage summary.</para>
    202 
    203         <para>CHUD consists of several components, including command-line programs, GUI
    204         applications, kernel extensions, and "frameworks" (collections of libraries,
    205         headers, and other resources which applications can use to access functionality
    206         provided by the other components.)  Past versions of &CCL;/OpenMCL have used the
    207         CHUD framework libraries to control the CHUD profiler.  Even though the rest of
    208         CHUD is currently 64-bit aware, the frameworks are unfortunately still only
    209         available as 32-bit libraries, so the traditional way of controlling the profiling
    210         facility from &CCL; has only worked from DarwinPPC32 versions.</para>
    211 
    212         <para>Two of the CHUD component programs are of particular interest:</para>
    213 
    214         <orderedlist>
    215           <listitem>
    216             <para>The "Shark" application (often installed in
    217             "/Developer/Applications/Performance Tools/Shark.app"), which provides a
    218             graphical user interface for exploring and analyzing profiling results and
    219             provides tools for creating "sampling configurations" (see below), among other
    220             things.</para>
    221           </listitem>
    222           <listitem>
    223             <para>The "shark" program ("/usr/bin/shark"), which can be used to control the
    224             CHUD profiling facility and to collect sampling data, which can then be
    225             displayed and analyzed in Shark.app.</para>
    226           </listitem>
    227         </orderedlist>
    228 
    229         <para>The fact that these two (substantially different) programs have names that
    230         differ only in alphabetic case may be confusing.  The discussion below tries to
    231         consistently distinguish between "the shark program" and "the Shark
    232         application".</para>
    233 
    234       </sect2>
    235       <sect2 id="Profiling-CHUD-usage-synopsis">
    236           <title>Usage synopsis</title>
    237           <programlisting>
    238 ? (defun fact (n) (if (zerop n) 1 (* n (fact (1- n)))))
     191    </sect2>
     192
     193    <sect2 id="Profiling-oprofile-Issues">
     194      <title>Issues</title>
     195
     196      <para>CCL::WRITE-ELF-SYMBOLS-TO-FILE currently only works on x86-64; it
     197      certainly -could- be made to work on ppc32/ppc64 as well.</para>
     198
     199      <para>So far, no one has been able to make oprofile/opreport options
     200      that're supposed to generate call-stack info generate meaningful
     201      call-stack info.</para>
     202
     203      <para>As of a few months ago, there was an attempt to provide symbol
     204      info for oprofile/opreport "on the fly", e.g., for use in JIT
     205      compilation or other incremental compilation scenarios. That's obviously
     206      more nearly The Right Thing, but it might be awhile before that
     207      experimental code makes it into widespread use.</para>
     208    </sect2>
     209  </sect1>
     210
     211  <sect1 id="Profiling-Using-Apples-CHUD-metering-tools">
     212    <title>Using Apple's CHUD metering tools</title>
     213
     214    <para>Apple's CHUD package provides libraries, kernel extensions, and a
     215    set of graphical and command-line programs that can be used to measure
     216    many aspects of application and system performance.</para>
     217
     218    <para>One of these programs is the Shark application (often installed in
     219    "/Developer/Applications/Performance Tools/Shark.app"), which provides a
     220    graphical user interface for exploring and analyzing profiling results and
     221    provides tools for creating "sampling configurations" (see below), among
     222    other things. Use of Shark isn't really documented here (a Shark manual is
     223    available at "Developer/Documentation/CHUD/Shark/ SharkUserGuide.pdf");
     224    what is described is a way of providing information about Lisp function
     225    names and addresses so that Shark can meaningly identify those functions
     226    in its output.</para>
     227
     228    <sect2 id="Profiling-CHUD-prerequisites">
     229      <title>Prerequisites</title>
     230
     231      <para>Apple's CHUD tools have been distributed with the last several
     232      XCode releases. One way to determine whether or not the tools are
     233      installed is to run:</para>
     234
     235      <programlisting>$ /usr/bin/shark -v</programlisting>
     236
     237      <para>in a terminal or Emacs shell buffer. If that returns output
     238      like</para>
     239
     240      <programlisting>shark 4.7.3 (365)</programlisting>
     241
     242      <para>then the CHUD package is installed. Output like</para>
     243
     244      <programlisting>shark: Command not found.</programlisting>
     245
     246      <para>strongly suggests that it isn't ...</para>
     247    </sect2>
     248
     249    <sect2>
     250      <title>Generating a lisp image for use with Shark</title>
     251
     252      <para>Shark can only properly identify functions that're defined in a
     253      shared library that's loaded by the target application. (Any other
     254      functions will be identified by a hex address described as being in an
     255      "Unknown Library"; the hex address is generally somewhat near the actual
     256      function, but it's determined heuristically and isn't always
     257      accurate.)</para>
     258
     259      <para>For those reasons, it's desirable to load the code that you wish
     260      to profile in one lisp session, save a native (Mach-O library) image,
     261      and invoke Shark in a new session which uses that native image. (It may
     262      also be useful to load the CHUD-METERING module, which defines
     263      CHUD:METER and friends.</para>
     264
     265      <para></para>
     266    </sect2>
     267
     268    <sect2 id="Profiling-CHUD-usage-synopsis">
     269      <title>Usage synopsis</title>
     270
     271      <programlisting>[src/ccl-dev] gb@antinomial&gt; ccl64
     272Welcome to Clozure Common Lisp Version 1.7-dev-r14624M-trunk  (DarwinX8664)!
     273? (defun fact(n) (if (zerop n) 1 (* n (fact (1- n)))))
    239274FACT
    240275? (require "CHUD-METERING")
    241276"CHUD-METERING"
    242277("CHUD-METERING")
    243 ? (chud:meter (null (fact 10000)))
    244 NIL           ; since that large number is not NULL
    245           </programlisting>
    246 
    247           <para>and, a few seconds after the result is returned, a file whose name is of
    248           the form "session_nnn.mshark" will open in Shark.app.</para>
    249 
    250           <para>The fist time that CHUD:METER is used in a lisp session, it'll do a few
    251           things to prepare subsequent profiling sessions.  Those things include:</para>
    252 
    253           <itemizedlist>
    254             <listitem>
    255               <para>creating a directory to store files that are related to using the CHUD
    256               tools in this lisp session.  This directory is created in the user's home
    257               directory and has a name of the form:</para>
    258 
    259               <programlisting>
     278? (save-application "ccl:dx86cl64.dylib" :native t)
     279[src/ccl-dev] gb@antinomial&gt; ccl64 -I dx86cl64.dylib
     280Welcome to Clozure Common Lisp Version 1.7-dev-r14624M-trunk  (DarwinX8664)!
     281? (chud:meter (dotimes (i 1000) (fact 1000)))
     282;;; Waiting for shark to process samples ...done.
     283NIL</programlisting>
     284
     285      <para>and, a few seconds after the result is returned, a file whose name
     286      is of the form "session_nnn.mshark" will open in Shark.app.</para>
     287
     288      <para></para>
     289
     290      <para>The fist time that CHUD:METER is used in a lisp session, it'll do
     291      a few things to prepare subsequent profiling sessions. Those things
     292      include:</para>
     293
     294      <itemizedlist>
     295        <listitem>
     296          <para>creating a directory to store files that are related to using
     297          the CHUD tools in this lisp session. This directory is created in
     298          the user's home directory and has a name of the form:</para>
     299
     300          <programlisting>
    260301profiling-session-&lt;lisp-kernel&gt;-&lt;pid&gt;_&lt;mm&gt;-&lt;dd&gt;-&lt;yyyy&gt;_&lt;h&gt;.&lt;m&gt;.&lt;s&gt;
    261302              </programlisting>
    262 
    263               <para>where &lt;pid&gt; is the lisp's process id, &lt;lisp-kernel&gt; is the
    264               name of the lisp kernel (of all things ...), and the other values provide a
    265               timestamp.</para>
    266             </listitem>
    267             <listitem>
    268               <para>does whatever needs to be done to ensure that currently-defined lisp
    269               functions don't move around as the result of GC activity, then writes a text
    270               file describing the names and addresses of those functions to the
    271               profiling-session directory created above.  (The naming conventions for and
    272               format of that file are described in</para>
    273               <para>
    274               <ulink url="http://developer.apple.com/documentation/DeveloperTools/Conceptual/SharkUserGuide/MiscellaneousTopics/chapter_951_section_4.html#//apple_ref/doc/uid/TP40005233-CH14-DontLinkElementID_42">http://developer.apple.com/documentation/DeveloperTools/Conceptual/SharkUserGuide/MiscellaneousTopics/chapter_951_section_4.html#//apple_ref/doc/uid/TP40005233-CH14-DontLinkElementID_42</ulink></para>
    275             </listitem>
    276             <listitem>
    277               <para>run the shark program ("/usr/bin/shark") and wait until it's ready to
    278               receive signals that control its operation.</para>
    279             </listitem>
    280           </itemizedlist>
    281 
    282           <para>This startup activity typically takes a few seconds; after it's been
    283           completed, subsequent use of CHUD:METER doesn't involve that overhead.  (See the
    284           discussion of :RESET below.)</para>
    285 
    286           <para>After any startup activity is complete, CHUD:METER arranges to send a
    287           "start profiling" signal to the running shark program, executes the form, sends
    288           a "stop profiling" signal to the shark program, and reads its diagnostic output,
    289           looking for the name of the ".mshark" file it produces.  If it's able to find
    290           this filename, it arranges for "Shark.app" to open it.</para>
    291         </sect2>
    292 
    293         <sect2 id="Profiling-CHUD-profiling-configurations">
    294           <title>Profiling "configurations"</title>
    295 
    296           <para>By default, a shark profiling session will:</para>
    297           <itemizedlist>
    298             <listitem>
    299               <para>use "time based" sampling, to periodically interrupt the lisp
    300             process and note the value of the program counter and at least a few levels of
    301             call history.</para>
    302             </listitem>
    303             <listitem>
    304               <para>do this sampling once every millisecond</para>
    305             </listitem>
    306             <listitem>
    307               <para>run for up to 30 seconds, unless told to stop earlier.</para>
    308             </listitem>
    309           </itemizedlist>
    310 
    311           <para>This is known as "the default configuration"; it's possible to use items
    312           on the "Config" menu in the Shark application to create alternate configurations
    313           which provide different kinds of profiling parameters and to save these
    314           configurations in files for subsequent reuse.  (The set of things that CHUD
    315           knows how to monitor is large and interesting.)</para>
    316 
    317           <para>You use alternate profiling configurations (created and "exported" via
    318           Shark.app) with CHUD:METER, but the interface is a little awkward.</para>
    319         </sect2>
    320 
    321         <sect2 id="Profiling-CHUD-Reference">
    322           <title>Reference</title>
    323 
    324           <para>
    325             <indexterm zone="chud_shark-config-file"/>
    326             <command><varname id="chud_shark-config-file">CHUD:*SHARK-CONFIG-FILE*</varname> [Variable]</command>
    327           </para>
    328 
    329           <para>When non-null, this should be the pathname of an alternate profiling
    330           configuration file created by the "Config Editor" in Shark.app.</para>
    331 
    332           <para>
    333             <indexterm zone="chud_meter"/>
    334             <command><varname id="chud_meter">CHUD:METER</varname> form &key; (reset nil) (debug-output nil) [Macro]</command>
    335           </para>
    336              
    337           <para>Executes FORM (an arbitrary lisp form) and returns whatever result(s) it
    338           returns, with CHUD profiling enabled during the form's execution.  Tries to
    339           determine the name of the session file (*.mshark) to which the shark program
    340           wrote profiling data and opens this file in the Shark application.</para>
    341          
    342           <para>Arguments:</para>
    343 
    344           <variablelist>
    345             <varlistentry>
    346               <term><varname>debug-output</varname></term>
    347               <listitem>
    348                 <para>when non-nil, causes output generated by the shark program to be
    349                 echoed to *TERMINAL-IO*.  For debugging.</para>
    350               </listitem>
    351             </varlistentry>
    352             <varlistentry>
    353               <term><varname>reset</varname></term>
    354               <listitem>
    355                 <para>when non-nil, terminates any running instance of the shark program
    356                 created by previous invocations of CHUD:METER in this lisp session,
    357                 generates a new .spatch file (describing the names and addresses of lisp
    358                 functions), and starts a new instance of the shark program; if
    359                 CHUD:*SHARK-CONFIG-FILE* is non-NIL when this new instance is started,
    360                 that instance is told to use the specified config file for profiling (in
    361                 lieu of the default profiling configuration.)</para>
    362               </listitem>
    363             </varlistentry>
    364           </variablelist>
    365         </sect2>
    366 
    367         <sect2 id="Profiling-CHUD-Acknowledgments">
    368           <title>Acknowledgement</title>
    369           <para>Both Dan Knapp and Hamilton Link have posted similar CHUD interfaces to
    370           openmcl-devel in the past; Hamilton's also reported bugs in the spatch mechanism
    371           to CHUD developers (and gotten those bugs fixed.)</para>
    372         </sect2>
    373 
    374       </sect1>
    375 
    376   </chapter>
     303        </listitem>
     304
     305        <listitem>
     306          <para>run the shark program ("/usr/bin/shark") and wait until it's
     307          ready to receive signals that control its operation.</para>
     308        </listitem>
     309      </itemizedlist>
     310
     311      <para>This startup activity typically takes a few seconds; after it's
     312      been completed, subsequent use of CHUD:METER doesn't involve that
     313      overhead. (See the discussion of :RESET below.)</para>
     314
     315      <para>After any startup activity is complete, CHUD:METER arranges to
     316      send a "start profiling" signal to the running shark program, executes
     317      the form, sends a "stop profiling" signal to the shark program, and
     318      reads its diagnostic output, looking for the name of the ".mshark" file
     319      it produces. If it's able to find this filename, it arranges for
     320      "Shark.app" to open it.</para>
     321    </sect2>
     322
     323    <sect2 id="Profiling-CHUD-profiling-configurations">
     324      <title>Profiling "configurations"</title>
     325
     326      <para>By default, a shark profiling session will:</para>
     327
     328      <itemizedlist>
     329        <listitem>
     330          <para>use "time based" sampling, to periodically interrupt the lisp
     331          process and note the value of the program counter and at least a few
     332          levels of call history.</para>
     333        </listitem>
     334
     335        <listitem>
     336          <para>do this sampling once every millisecond</para>
     337        </listitem>
     338
     339        <listitem>
     340          <para>run for up to 30 seconds, unless told to stop earlier.</para>
     341        </listitem>
     342      </itemizedlist>
     343
     344      <para>This is known as "the default configuration"; it's possible to use
     345      items on the "Config" menu in the Shark application to create alternate
     346      configurations which provide different kinds of profiling parameters and
     347      to save these configurations in files for subsequent reuse. (The set of
     348      things that CHUD knows how to monitor is large and interesting.)</para>
     349
     350      <para>You use alternate profiling configurations (created and "exported"
     351      via Shark.app) with CHUD:METER, but the interface is a little
     352      awkward.</para>
     353    </sect2>
     354
     355    <sect2 id="Profiling-CHUD-Reference">
     356      <title>Reference</title>
     357
     358      <para><indexterm zone="chud_shark-config-file"></indexterm>
     359      <command><varname
     360      id="chud_shark-config-file">CHUD:*SHARK-CONFIG-FILE*</varname>
     361      [Variable]</command></para>
     362
     363      <para>When non-null, this should be the pathname of an alternate
     364      profiling configuration file created by the "Config Editor" in
     365      Shark.app.</para>
     366
     367      <para><indexterm zone="chud_meter"></indexterm> <command><varname
     368      id="chud_meter">CHUD:METER</varname> form &key; (reset nil)
     369      (debug-output nil) [Macro]</command></para>
     370
     371      <para>Executes FORM (an arbitrary lisp form) and returns whatever
     372      result(s) it returns, with CHUD profiling enabled during the form's
     373      execution. Tries to determine the name of the session file (*.mshark) to
     374      which the shark program wrote profiling data and opens this file in the
     375      Shark application.</para>
     376
     377      <para>Arguments:</para>
     378
     379      <variablelist>
     380        <varlistentry>
     381          <term><varname>debug-output</varname></term>
     382
     383          <listitem>
     384            <para>when non-nil, causes output generated by the shark program
     385            to be echoed to *TERMINAL-IO*. For debugging.</para>
     386          </listitem>
     387        </varlistentry>
     388
     389        <varlistentry>
     390          <term><varname>reset</varname></term>
     391
     392          <listitem>
     393            <para>when non-nil, terminates any running instance of the shark
     394            program created by previous invocations of CHUD:METER in this lisp
     395            session, generates a new .spatch file (describing the names and
     396            addresses of lisp functions), and starts a new instance of the
     397            shark program; if CHUD:*SHARK-CONFIG-FILE* is non-NIL when this
     398            new instance is started, that instance is told to use the
     399            specified config file for profiling (in lieu of the default
     400            profiling configuration.)</para>
     401          </listitem>
     402        </varlistentry>
     403      </variablelist>
     404    </sect2>
     405
     406    <sect2 id="Profiling-CHUD-Acknowledgments">
     407      <title>Acknowledgement</title>
     408
     409      <para>Both Dan Knapp and Hamilton Link have posted similar CHUD
     410      interfaces to openmcl-devel in the past; Hamilton's also reported bugs
     411      in the spatch mechanism to CHUD developers (and gotten those bugs
     412      fixed.)</para>
     413    </sect2>
     414  </sect1>
     415</chapter>
  • trunk/source/doc/src/using.xml

    r14528 r14634  
    19181918      does. Alternatively, if you choose not to prepend the kernel, you
    19191919      can save an image and then later run it by passing it as a
    1920       command-line parameter to the <literal>opencml</literal>
    1921       or <literal>opencml64</literal> script.</para>
     1920      command-line parameter to the <literal>ccl</literal>
     1921      or <literal>ccl64</literal> script.</para>
    19221922
    19231923    <para>
     
    19351935        <parameter>(mode #o644)</parameter>
    19361936        <parameter>prepend-kernel</parameter>
     1937        <parameter>native</parameter>
    19371938        [Function]</command>
    19381939    </para>
     
    20092010            any true value.</para>
    20102011        </listitem>
     2012     </varlistentry>
     2013     <varlistentry>
     2014        <term><varname>native</varname></term>
     2015        <listitem>
     2016          <para>If true, saves the image as a native (ELF, Mach-O, PE)
     2017          shared library.  (On platforms where this isn't yet supported,
     2018          a warning is issued and the option is ignored.)
     2019         </para>
     2020      </listitem>
    20112021      </varlistentry>
    20122022     
  • trunk/source/library/chud-metering.txt

    r11462 r14634  
    1 Using Apple's CHUD metering tools from CCL
    2 ==========================================
    3 
    4 Prerequisites
    5 -------------
    6 
    7 Apple's CHUD metering tools are available (as of this writing) from:
    8 
    9 <ftp://ftp.apple.com/developer/Tool_Chest/Testing_-_Debugging/Performance_tools/>.
    10 
    11 The CHUD tools are also generally bundled with Apple's XCode tools.
    12 CHUD 4.5.0 (which seems to be bundled with XCode 3.0) seems to work
    13 well with this interface; later versions may have problems.
    14 Versions of CHUD as old as 4.1.1 may work with 32-bit PPC versions
    15 of CCL; later versions (not sure exactly -what- versions) added
    16 x86, ppc64, and x86-64 support.
    17 
    18 One way to tell whether any version of the CHUD tools is installed
    19 is to try to invoke the "shark" command-line program (/usr/bin/shark)
    20 from the shell:
    21 
    22 shell> shark --help
    23 
    24 and verifying that that prints a usage summary.
    25 
    26 CHUD consists of several components, including command-line programs,
    27 GUI applications, kernel extensions, and "frameworks" (collections of
    28 libraries, headers, and other resources which applications can use to
    29 access functionality provided by the other components.)  Past versions
    30 of CCL/OpenMCL have used the CHUD framework libraries to control the
    31 CHUD profiler.  Even though the rest of CHUD is currently 64-bit aware,
    32 the frameworks are unfortunately still only available as 32-bit libraries,
    33 so the traditional way of controlling the profiling facility from OpenMCL
    34 has only worked from DarwinPPC32 versions.
    35 
    36 Two of the CHUD component programs are of particular interest:
    37 
    38 1) The "Shark" application (often installed in
    39 "/Developer/Applications/Performance Tools/Shark.app"), which provides
    40 a graphical user interface for exploring and analyzing profiling results
    41 and provides tools for creating "sampling configurations" (see below),
    42 among other things.
    43 
    44 2) the "shark" program ("/usr/bin/shark"), which can be used to control
    45 the CHUD profiling facility and to collect sampling data, which can then
    46 be displayed and analyzed in Shark.app.
    47 
    48 The fact that these two (substantially different) programs have names that
    49 differ only in alphabetic case may be confusing.  The discussion below
    50 tries to consistently distinguish between "the shark program" and "the
    51 Shark application".
    52 
    53 Usage synopsis
    54 --------------
    55 
    56 ? (defun fact (n) (if (zerop n) 1 (* n (fact (1- n)))))
    57 FACT
    58 ? (require "CHUD-METERING")
    59 "CHUD-METERING"
    60 ("CHUD-METERING")
    61 ? (chud:meter (null (fact 10000)))
    62 NIL           ; since that large number is not NULL
    63 
    64 and, a few seconds after the result is returned, a file whose
    65 name is of the form "session_nnn.mshark" will open in Shark.app.
    66 
    67 The fist time that CHUD:METER is used in a lisp session, it'll do a
    68 few things to prepare subsequent profiling sessions.  Those things
    69 include:
    70 
    71 1) creating a directory to store files that are related to using
    72 the CHUD tools in this lisp session.  This directory is created in
    73 the user's home directory and has a name of the form:
    74 
    75 profiling-session-<lisp-kernel>-<pid>_<mm>-<dd>-<yyyy>_<h>.<m>.<s>
    76 
    77 where <pid> is the lisp's process id, <lisp-kernel> is the name of
    78 the lisp kernel (of all things ...), and the other values provide
    79 a timestamp.
    80 
    81 2) does whatever needs to be done to ensure that currently-defined
    82 lisp functions don't move around as the result of GC activity, then
    83 writes a text file describing the names and addresses of those functions
    84 to the profiling-session directory created above.  (The naming conventions
    85 for and format of that file are described in
    86 
    87 <http://developer.apple.com/documentation/DeveloperTools/Conceptual/SharkUserGuide/MiscellaneousTopics/chapter_951_section_4.html#//apple_ref/doc/uid/TP40005233-CH14-DontLinkElementID_42>
    88 
    89 3) run the shark program ("/usr/bin/shark") and wait until it's ready to
    90 receive signals that control its operation.
    91 
    92 This startup activity typically takes a few seconds; after it's been
    93 completed, subsequent use of CHUD:METER doesn't involve that overhead.
    94 (See the discussion of :RESET below.)
    95 
    96 After any startup activity is complete, CHUD:METER arranges to send
    97 a "start profiling" signal to the running shark program, executes
    98 the form, sends a "stop profiling" signal to the shark program, and
    99 reads its diagnostic output, looking for the name of the ".mshark"
    100 file it produces.  If it's able to find this filename, it arranges
    101 for "Shark.app" to open it
    102 
    103 Profiling "configurations".
    104 --------------------------
    105 
    106 By default, a shark profiling session will:
    107 a) use "time based" sampling, to periodically interrupt the lisp
    108    process and note the value of the program counter and at least
    109    a few levels of call history.
    110 b) do this sampling once every millisecond
    111 c) run for up to 30 seconds, unless told to stop earlier.
    112 
    113 This is known as "the default configuration"; it's possible to use
    114 items on the "Config" menu in the Shark application to create alternate
    115 configurations which provide different kinds of profiling parameters
    116 and to save these configurations in files for subsequent reuse.
    117 (The set of things that CHUD knows how to monitor is large and interesting.)
    118 
    119 You use alternate profiling configurations (created and "exported" via
    120 Shark.app) with CHUD:METER, but the interface is a little awkward.
    121 
    122 Reference
    123 ---------
    124 
    125 CHUD:*SHARK-CONFIG-FILE*   [Variable]
    126 
    127 When non-null, this should be the pathname of an alternate profiling
    128 configuration file created by the "Config Editor" in Shark.app.
    129 
    130 (CHUD:METER form &key (reset nil) (debug-output nil))  [Macro]
    131 
    132 Executes FORM (an arbitrary lisp form) and returns whatever result(s)
    133 it returns, with CHUD profiling enabled during the form's execution.
    134 Tries to determine the name of the session file (*.mshark) to which
    135 the shark program wrote profiling data and opens this file in the
    136 Shark application.
    137 
    138 Arguments:
    139 
    140 debug-output   - when non-nil, causes output generated by the shark program to
    141                  be echoed to *TERMINAL-IO*.  For debugging.
    142 reset          - when non-nil, terminates any running instance of the
    143                  shark program created by previous invocations of CHUD:METER
    144                  in this lisp session, generates a new .spatch file
    145                  (describing the names and addresses of lisp functions),
    146                  and starts a new instance of the shark program; if
    147                  CHUD:*SHARK-CONFIG-FILE* is non-NIL when this new instance
    148                  is started, that instance is told to use the specified
    149                  config file for profiling (in lieu of the default profiling
    150                  configuration.)
    151 
    152 Acknowledgments
    153 ---------------
    154 
    155 Both Dan Knapp and Hamilton Link have posted similar CHUD interfaces
    156 to openmcl-devel in the past; Hamilton's also reported bugs in the
    157 spatch mechanism to CHUD developers (and gotten those bugs fixed.)
     1See section 11.2 of the Clozure CL manual.
Note: See TracChangeset for help on using the changeset viewer.