"complete" callback of Database.each() leaking memory?

[thierer]

I've got a memory leak in the complete callback of Database.each() that I can't explain.

This is running node-sqlite3 2.2.3 and node 0.10.22 on Linux 3.12.13 x86_64.

The following stripped down example shows the behaviour:

var sqlite3 = require('sqlite3');

function doLeak(callback) {
  var leak = new Buffer(1000000);

  db.each('SELECT 1',
    function item(err, row) {
    },
    function complete(err, found) {
      callback(leak);
    }
  );
}

var db = new sqlite3.Database(':memory:', sqlite3.OPEN_READONLY);
db.once('open', function(err) {
  if (err) return console.error(err);

  var active = 0;

  (function callLeak() {
    active++;
    doLeak(function() {
      console.log(--active);
      callLeak();
    });
  })();
});

The Buffer allocation is just there to increase the effect. If I use heapdump to take snapshots, I can see the "complete" functions pile up in the "Containment" view of Devtools. (Note that it only seems to affect the completeand not the item callback).

I'm not sure if it's really a bug or just my lack of understanding javascript / node.js / v8 / whatever.

OTOH, I don't see any leaks if I replace the sqlite3 query and use the same call pattern with fs.readFile(), which I would consider a comparable situation?

var fs = require('fs');

function doLeak(callback) {
  var leak = new Buffer(1000000);

  fs.readFile('test.txt',
    function complete(err, data) {
      callback(leak);
    }
  );
}

var active = 0;

(function callLeak() {
  active++;
  doLeak(function() {
    console.log(--active);
    callLeak();
  });
})();

Any help would be appreciated.

[springmeyer]

thanks for the report and for providing a patch. I'm busy with other projects but I will try to test when I have time. One think to try: run the test with node --expose-gc and then call gc() to force garbage collection during your script. Then watch memory - this is to make sure you are not just seeing a lag in garbage collection and thinking its a leak.

[thierer]

I did that. Same result: Virtual memory going through the roof (without the fix).

[springmeyer]

thanks. another thing I'll check is if it is also present with node v0.11.x and #184. If you have time to test that too I'd be grateful