Changes between Version 2 and Version 3 of WatchedObjects


Ignore:
Timestamp:
Oct 13, 2009, 3:33:57 AM (10 years ago)
Author:
rme
Comment:

substantial revision

Legend:

Unmodified
Added
Removed
Modified
  • WatchedObjects

    v2 v3  
    11= Overview =
    22
    3 As a debugging aid, many lisp objects can be ''watched'' so that a condition will
    4 be signaled when the object is about to be written to.
     3Clozure CL provides a way for lisp objects to be ''watched'' so that a
     4condition will be signaled when a thread attempts to write to the
     5watched object.  For a certain class of bugs (someone is changing this
     6value, but I don't know who), this can be extremely helpful.
     7
    58
    69= Usage =
    710
     11== WATCH ==
     12
    813'''WATCH &optional object [function]'''
    914
    1015The WATCH function arranges for the specified object to be monitored
    11 for writes.  This is done via the system's memory-management hardware,
    12 so reading the object takes place at full speed.
    13 
    14 When any write to the object is attempted, the condition
    15 WRITE-TO-WATCHED-OBJECT will be signaled.
    16 
    17 WATCH knows about a few complicated objects (hash tables, fancy arrays,
    18 CLOS standard instances), and gives them special treatment:
    19 
    20   hash tables: the underlying hash table vector will be watched
    21 
    22   arrays: the underlying data vector will be watched
    23 
    24   standard instances: the slot vector will be watched
    25 
    26 If you don't want WATCH to do this for you, call PRIMITIVE-WATCH instead.
    27 
    28 WATCH can monitor cons cells, but in order to watch a chain of cons
    29 cells, they have to be watched individually. Each cons cell will take
    30 up its own virtual memory page, so it's only really feasible to watch
    31 short lists.
    32 
    33 When called with no arguments, WATCH returns a list of objects
    34 currently being watched.  These objects will always be the underlying
    35 data vectors, and not complicated user-level objects like hash tables, etc.
    36 
    37 
    38 '''PRIMITIVE-WATCH object [function]'''
    39 
    40 Like WATCH, except it doesn't give ''object'' any
    41 special treatment;  it simply uses it as-is.
    42 
     16for writes.  This is accomplished by copying the object to its own set
     17of virtual memory pages, which are then write-protected.  This
     18protection is enforced by the computer's memory-management hardware;
     19the write-protection does not slow down reads at all.
     20
     21When any write to the object is attempted, a WRITE-TO-WATCHED-OBJECT
     22condition will be signaled.
     23
     24When called with no arguments, WATCH returns a freshly-consed list of
     25the objects currently being watched.
     26
     27=== Discussion ===
     28
     29WATCH can monitor any memory-allocated lisp object.
     30
     31In Clozure CL a memory-allocated object is either a cons cell or
     32a uvector.
     33
     34WATCH operates on cons cells, not lists.  In order to watch a chain of
     35cons cells, each cons cell must be watched individually.  Because each
     36watched cons cell takes up its own own virtual memory page (4 Kbytes),
     37it's only feasible to watch relatively short lists.
     38
     39If a memory-allocated object isn't a cons cell, then it is a
     40vector-like object called a uvector.
     41
     42A uvector is a memory-allocated lisp object whose first word is a
     43header that describes the object's type and the number of elements
     44that it contains.
     45
     46So, a hash table is a uvector, as is a string, a standard instance,
     47a double-float, a CL array or vector, and so forth.
     48
     49Some CL objects, like strings and other simple vectors, map in a
     50straightforward way onto the uvector representation.  Simply watching
     51the object directly is sufficient in these simple cases.
     52
     53{{{
     54? (defvar *s* "xxxxx")
     55*S*
     56? (watch *s*)
     57"xxxxx"
     58? (setf (char *s* 3) #\o)
     59> Error: Write to watched uvector "xxxxx" at index 3
     60>        Faulting instruction: (movl (% eax) (@ -5 (% r15) (% rcx)))
     61> While executing: SET-CHAR, in process listener(1).
     62> Type :POP to abort, :R for a list of available restarts.
     63> Type :? for other options.
     64}}}
     65
     66In the case of more complicated objects (e.g., a hash-table, a
     67standard-instance, a package, etc.), the elements of the uvector are
     68like slots in a structure.  It's necessary to know which one of those
     69"slots" contains the data that will be changed when the object is
     70written to.
     71
     72An example might make this clearer.  Suppose we have a
     73standard-instance.  It turns out that it is represented by uvector of
     74three elements.  One of those
     75elements is an object called a slot-vector.  To monitor writes to the
     76slots of a standard-instance, it is necessary to watch the
     77slot-vector, not the standard-instance itself.
     78
     79{{{
     80? (defclass foo ()
     81    (slot-a slot-b slot-c))
     82#<STANDARD-CLASS FOO>
     83? (defvar *a-foo* (make-instance 'foo))
     84*A-FOO*
     85? (watch *a-foo*)
     86NIL
     87;;; Note that this doesn't catch the write..
     88? (setf (slot-value *a-foo* 'slot-a) 'foo)
     89FOO
     90;;; note use of internal accessor to get at the slot-vector
     91? (watch (ccl::instance-slots *a-foo*))
     92NIL
     93? (setf (slot-value *a-foo* 'slot-a) 'foo)
     94> Error: Write to watched uvector #<SLOT-VECTOR #xD600D> at index 1
     95>        Faulting instruction: (movq (% rsi) (@ -5 (% r8) (% rdi)))
     96> While executing: %MAYBE-STD-SETF-SLOT-VALUE-USING-CLASS, in process listener(1).
     97> Type :POP to abort, :R for a list of available restarts.
     98> Type :? for other options.
     99}}}
     100
     101Note that even though the write was to slot-a, the uvector index was 1
     102(not 0).  This is because the first element of a slot-vector is a
     103pointer to the instance that owns the slots.
     104
     105{{{
     106#!comment
     107=== DWIM ===
     108
     109WATCH operates at a fairly low level; it is not possible to avoid
     110the details of the internal representation of objects.
     111
     112Nevertheless, as a convenience, WATCHing a standard-instance, a
     113hash-table, or a multi-dimensional or non-simple CL array will watch
     114the underlying slot-vector, hash-table-vector, and data-vector,
     115respectively.
     116}}}
     117
     118== UNWATCH ==
    43119
    44120'''UNWATCH object [function]'''
     
    49125object is returned.
    50126
    51 A note on thread-safety: avoid unwatching an object from a thread while
    52 some other thread might be signaling or handling a
    53 WRITE-TO-WATCHED-OBJECT condition.
    54 The lisp tries to detect when it would be unsafe to unwatch
    55 an object; UNWATCH will return NIL and not unwatch the object in such
    56 a case.  Nevertheless, there may be subtle race conditions lurking
    57 here.
    58 
     127== WRITE-TO-WATCHED-OBJECT ==
    59128
    60129'''WRITE-TO-WATCHED-OBJECT [condition]'''
    61130
    62131This condition is signaled when a watched object is written to.  There
    63 are three slots of interest: the "object" slot, the "containing-object"
    64 slot, and the "location" slot.
    65 
    66 The object slot is the actual object that was the target of a write operation.
    67 
    68 If the object slot contains something that is a part of some larger
    69 composite object, then the containing-object slot will contain
    70 that object.  For example, if object is a hash-table-vector, then the
    71 contents of the containing-object will be the hash table of which the
    72 hash-table-vector is a part.  If the contents of object slot
    73 aren't part of any larger object, then the containing-object and
    74 object slots will contain the same thing.
    75 
    76 The contents of the location slot differs according to what is in the
    77 object slot.  In the case of arrays, it will be the row-major index of
    78 the element that was about to be written.  For hash tables, location
    79 will contain the hash table key, or #<Unbound> if the key can't be
    80 determined (as might happen if you are adding a new key-value pair to
    81 the hash table).  For CLOS objects, location will contain the slot
    82 name.  If object is a cons cell, location will be either the actual
    83 car or cdr of the cons cell, depending on which half of the cons cell
    84 was written to.  If the object is some other uvector, then location
    85 will contain the uvector index of the element that was about to be
    86 written.  If location is NIL, the location could not be
    87 determined.
     132are three slots of interest: object, offset, and instruction.
     133
     134The object slot is the actual object that was written.
     135
     136The offset slot is the byte offset from the tagged object pointer to
     137the starting address of the write.
     138
     139The instruction slot contains the disassembled machine instruction
     140that attempted the write.
     141
     142The :report function for this condition will attempt to use the offset
     143to determine what uvector index (or which half, either the car or the
     144cdr, for a cons cell) was attempted to be written.  The disassembled
     145instruction will also be displayed, if possible.
     146
     147=== Restarts ===
     148Two restarts are provided: one will skip over the faulting write instruction,
     149and proceed;  the other offers to unwatch the object and continue.
     150
     151= Notes =
     152
     153Although some care has been taken to minimize potential problems
     154arising from watching and unwatching objects from multiple threads,
     155there may well be subtle race conditions present that could cause bad
     156behavior.
     157
     158For example, suppose that a thread attempts to write to a watched
     159object.  This causes the operating system to generate an exception.
     160The lisp kernel figures out what the exception is, and calls back into
     161lisp to signal the write-to-watched-object condition and perhaps
     162handle the error.
     163
     164Now, as soon lisp code starts running again (for the callback), it's
     165possible that some other thread could unwatch the very watched object
     166that caused the exception, perhaps before we even have a chance to
     167signal the condition, much less respond to it.
     168
     169Having the object unwatched out from underneath a handler may at least
     170confuse it, if not cause deeper trouble.  Use caution with unwatch.
     171
     172= Examples =
     173
     174Here are a couple more examples in addition to the above examples of watching a
     175string and a standard-instance.
     176
     177== Fancy arrays ==
     178
     179{{{
     180? (defvar *f* (make-array '(2 3) :element-type 'double-float))
     181*F*
     182;;; Find the underlying data vector
     183? (ccl::array-data-and-offset *f*)
     184#(0.0D0 0.0D0 0.0D0 0.0D0 0.0D0 0.0D0)
     1850
     186? (watch *)
     187#(0.0D0 0.0D0 0.0D0 0.0D0 0.0D0 0.0D0)
     188? (setf (aref *f* 1 2) pi)
     189> Error: Write to watched uvector #<VECTOR 6 type DOUBLE-FLOAT, simple> at index 5
     190>        Faulting instruction: (movq (% rax) (@ -5 (% r8) (% rdi)))
     191> While executing: ASET, in process listener(1).
     192> Type :POP to abort, :R for a list of available restarts.
     193> Type :? for other options.
     194}}}
     195
     196Note the the uvector index in the report is the row-major index of the
     197element that was written to.
     198
     199== Hash tables ==
     200
     201Hash tables are surprisingly complicated.  The representation of a hash
     202table includes an element called a hash-table-vector.  The keys and values
     203of the elements are stored pairwise in this vector.
     204
     205{{{
     206? (defvar *h* (make-hash-table))
     207#<HASH-TABLE :TEST EQL size 0/60 #x3000412BF96D>
     208? (setf (gethash 'noise *h*) 'feep)
     209FEEP
     210? (watch (ccl::nhash.vector *h*))
     211NIL
     212? (setf (gethash 'noise *h*) 'ding)
     213> Error: Write to watched uvector #<HASH-TABLE-VECTOR #xD500D> at index 35
     214>        Faulting instruction: (lock)
     215>          (cmpxchgq (% rsi) (@ (% r8) (% rdx)))
     216> While executing: %STORE-NODE-CONDITIONAL, in process listener(1).
     217> Type :POP to abort, :R for a list of available restarts.
     218> Type :? for other options.
     219;;; see what value is being replaced...
     2201 > (uvref (write-to-watched-object-object *break-condition*) 35)
     221FEEP
     222;;; backtrace shows useful context
     2231 > :b
     224*(1A109F8) : 0 (%STORE-NODE-CONDITIONAL ???) NIL
     225 (1A10A50) : 1 (LOCK-FREE-PUTHASH NOISE #<HASH-TABLE :TEST EQL size 1/60 #x30004117F4ED> DING) 653
     226 (1A10AC8) : 2 (CALL-CHECK-REGS PUTHASH NOISE #<HASH-TABLE :TEST EQL size 1/60 #x30004117F4ED> DING) 229
     227 (1A10B00) : 3 (TOPLEVEL-EVAL (SETF (GETHASH # *H*) 'DING) NIL) 709
     228 ...
     229}}}