Opened 4 years ago

Closed 4 years ago

#1411 closed defect (notabug)

GC bug when reading a file

Reported by: pebblexe Owned by:
Priority: normal Milestone:
Component: other Version: trunk
Keywords: Cc:

Description

I am using cl-csv to read in a large file, and for some reason there is a bug and it won't garbage collect.

(defparameter +test-big-file+

(asdf:system-relative-pathname :testpackage "big.csv"))

(defun count-big-file ()

(let ((cnt 0))

(time (cl-csv:read-csv +test-big-file+

:row-fn (lambda (r) (declare (ignore r))

(incf cnt))))

cnt))

Takes up a lot of ram, and (gc) won't clear it. I have to kill the slime instance and restart it to reclaim up my ram.

When I run the same code in SBCL it is properly garbage collected.

Change History (5)

comment:1 Changed 4 years ago by rme

What is the output of (lisp-implementation-version), please?

Did you get cl-csv via Quicklisp?

By the way, it would be better to use https://github.com/Clozure/ccl/issues now rather than Trac tickets.

comment:2 Changed 4 years ago by pebblexe

(lisp-implementation-version) is "Version 1.11-r16635 (LinuxX8664)"

I got cl-csv via quicklisp today. It's cl-csv-20170124.

Sorry, I'll use the github tracker for other bugs.

comment:3 Changed 4 years ago by rme

I would never say that there are no bugs in CCL, but I would be very surprised indeed if this is a GC bug.

There is almost certainly something that is holding references to your data somewhere.

When I run your test case, I don't think I see abnormal behavior.

(defparameter *csv-file* #p"/tmp/foo.csv")

(defun count-big-file ()
  (let ((cnt 0))
    (time (cl-csv:read-csv *csv-file*
			   :row-fn (lambda (r) (declare (ignore r))
					   (incf cnt))))
    cnt))


? (lisp-implementation-version)
"Version 1.11-r16807  (DarwinX8664)"
? (gc)
NIL
? (room)
Approximately 33,292,288 bytes of memory can be allocated 
before the next full GC is triggered. 

                   Total Size             Free                 Used
Lisp Heap:       55574528 (54272K)   33292288 (32512K)   22282240 (21760K)
Stacks:          11211368 (10949K)   11203112 (10941K)       8256 (8K)
Static:          19766272 (19303K)          0 (0K)       19766272 (19303K)
376747.000 MB reserved for heap expansion.
NIL
? (count-big-file)
(CL-CSV:READ-CSV *CSV-FILE* :ROW-FN (LAMBDA (R) (DECLARE (IGNORE R)) (INCF CNT)))
took 7,571,839 microseconds (7.571839 seconds) to run.
        16,941 microseconds (0.016941 seconds, 0.22%) of which was spent in GC.
During that period, and with 4 available CPU cores,
     7,320,830 microseconds (7.320830 seconds) were spent in user mode
       253,889 microseconds (0.253889 seconds) were spent in system mode
 512,002,784 bytes of memory allocated.
1000000
? (gc)
NIL
? (room)
Approximately 33,423,360 bytes of memory can be allocated 
before the next full GC is triggered. 

                   Total Size             Free                 Used
Lisp Heap:       55705600 (54400K)   33423360 (32640K)   22282240 (21760K)
Stacks:          11211368 (10949K)   11203112 (10941K)       8256 (8K)
Static:          19766272 (19303K)          0 (0K)       19766272 (19303K)
376746.870 MB reserved for heap expansion.
NIL

I'm running a 1.11 that has a few bug fixes that have been made since the release binaries were created, but I don't think any of them would have any bearing here.

comment:4 Changed 4 years ago by pebblexe

I wanted to create a small example program that has this issue but it seems I can't recreate it.

So I don't think this was a bug, my example program works fine.

I don't know how to close this, but this bug was my fault.

In the future I will use the github project issue tracker.

comment:5 Changed 4 years ago by rme

  • Resolution set to notabug
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.