incubator-lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <mar...@rectangular.com>
Subject Re: [lucy-dev] Remaining CPAN module dependencies
Date Tue, 15 Mar 2011 22:13:24 GMT
On Sun, Mar 13, 2011 at 09:53:51PM -0500, Peter Karman wrote:
> > Unfortunately I can't think of a way to replace the Test::LeakTrace
> > dependency.  Peter, do you feel strongly enough about this test file to do the
> > research on whether we can ship it with Lucy?  My inclination would be to
> > remove it because unlike Parse::RecDescent and JSON::XS, Lucy can still build
> > and run without it -- but maybe we should investigate what it tells us first?
> 
> fine to remove it. I actually find it generating false positives, so not sure
> it's worth investigation since valgrind works well.

Since you'd gone to the trouble of committing that file, I went and looked
anyway, installing Test::LeakTrace and enabling the test.  It found a
refcounting error in RegexTokenizer's constructor, for the qr// scalar.  I'd
forgotten that lucy_Host_callback_host() increfs the SV it returns.

The resulting memory leak wasn't very large or very serious, but the fact that
Test::LeakTrace caught something Valgrind missed demonstrates its utility.
Valgrind has a limitation in the way it tracks memory: it works well with
malloc() and friends by overwriting them and installing its own tracking
version, but if you roll your own allocator -- like Perl does -- Valgrind
doesn't know about it.

I started a thread on PerlMonks to see whether by custom compiling Perl we
could coax the same functionality out of Valgrind that we can get out
Test::LeakTrace.  Unfortunately, it looks like that won't work:

    http://www.perlmonks.org/?node_id=893231

In any case, rather than use Test::LeakTrace in a single test which gets run
consistently, I think we would want to take the same approach we take with
Valgrind: run the *entire* test suite under Test::LeakTrace every once in a
while.  Since most of Lucy's binding code gets autogenerated and it doesn't
change that often, refcounting mistakes will happen less frequently than in
core.

For now, I'm going to go ahead and delete the file, but in the future, we
should see if we can work an approximation of this functionality in, perhaps
using Perl core tools such as the -DDEBUG_MEM_LOG and -DDEBUG_LEAKING_SCALARS.

Marvin Humphrey


Mime
View raw message