lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <mar...@rectangular.com>
Subject Re: [lucy-dev] Non-deterministic destruction in Perl 5.15
Date Wed, 09 Nov 2011 05:33:48 GMT
On Tue, Nov 08, 2011 at 07:53:03PM -0600, Peter Karman wrote:
> >     void
> >     DESTROY(self)
> >         lucy_IndexSearcher *self;
> >     PPCODE:
> >         if (PL_phase != PERL_PHASE_DESTRUCT) {
> >             lucy_IxSearcher_destroy(self);
> >         }

> Would it help if Perl's global destruction phase called DESTROY() with an
> argument in order to indicate that it was in that phase? Or is that
> PERL_PHASE_DESTRUCT var supposed to achieve that? Is that real code?

That's real code.  We autogenerate the method bindings, so the patch won't
look exactly like that, but PL_phase and PERL_PHASE_DESTRUCT are part of the
Perl C API in Perl 5.15.  That construct will at least compile; I hope it
solves our problems.

> I guess what I'm getting at is, who cares if the change to Lucy defeats the
> purpose of the Perl feature.

I agree -- it's not important.

> The change to Perl is supposed to address a problem that Lucy does not have.

Well, Lucy wouldn't play well in an embedded system because we leak VTables.
We could fix that, but it's not a priority.

> Does putting that PERL_PHASE_DESTRUCT check in there work around the issue
> with no ill side effects to Lucy? Could we just plow through the code, add
> that check, and call it a day?

I've opened an issue and uploaded patches at
<https://issues.apache.org/jira/browse/LUCY-187>.  As of now, I can get tests
passing on Perl 5.15, but test_valgrind still has lots of glitches.  I can't
yet tell whether we're seeing long-standing problems that have suddenly
been revealed or new defects showing up.

> > I wonder how many other systems like ours are out there that are vulnerable to
> > this flaw.  Not many CPAN distros are going to have test suites that validate
> > behavior under refcount leakage.
> 
> fwiw, SWISH::3 is largely C/XS and it is apparently passing its 5.15 tests right
> now according to cpantesters. I don't have the same VTable architecture in
> there, but as you'll remember Marvin, the object and memory management model was
> inspired by our conversations around how Lucy does it.

It's not surprising that SWISH::3 would pass its tests.  You haven't set up
tests to validate behavior under refcount leakage, have you?

What I'm suggesting if a user writes a program which leaks Lucy objects, and
if we disconnect that "tripwire" exception without implementing the
PERL_PHASE_DESTRUCT wrapping, that program may segfault during global
destruction in Perl 5.15.

Most small programs don't leak, because most small programs don't have
circular references.  Circular refs are pretty common in big, complex
programs, though.

If SWISH::3 -- or Perl/Tk, or whatever else -- has complex objects which count
on deterministic order of destruction, I believe that the potential for
segfaulting during global destruction exists in Perl 5.15.

Marvin Humphrey


Mime
View raw message