Mailing-List: contact lucy-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: lucy-dev@lucene.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <20090328225538.GB27686@rectangular.com>
References: <9ac0c6aa0903161602icd8d9d8jcbf34b682481297@mail.gmail.com>
	 <9ac0c6aa0903170250t31e418f9pe6df3d072d9e4738@mail.gmail.com>
	 <20090320003355.GA12256@rectangular.com>
	 <9ac0c6aa0903201330j44db7a16wd0ddbca73e053da7@mail.gmail.com>
	 <20090325001222.GA21515@rectangular.com>
	 <9ac0c6aa0903250508v45f42a46s3b25e0323d844bbb@mail.gmail.com>
	 <20090327050826.GA8409@rectangular.com>
	 <9ac0c6aa0903270521r7808a618v71e177a111553a96@mail.gmail.com>
	 <9ac0c6aa0903270523o4d2331eci4db8854e26e5579a@mail.gmail.com>
	 <20090328225538.GB27686@rectangular.com>
Date: Sun, 29 Mar 2009 06:01:16 -0400
Message-ID: <9ac0c6aa0903290301v352591c2t84815454d29df596@mail.gmail.com>
Subject: Re: Reference counting inside a GC host (was "real time updates")
From: Michael McCandless <lucene@mikemccandless.com>
To: lucy-dev@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Sat, Mar 28, 2009 at 6:55 PM, Marvin Humphrey <marvin@rectangular.com> w=
rote:
> On Fri, Mar 27, 2009 at 08:23:45AM -0400, Michael McCandless wrote:
>
>> since Lucy will use reference counting, how will you deal with cycles?
>
> I don't think we should do anything about them. =A0It will be up to the u=
ser to
> avoid them.
>
> There *are* limits to my ambitions for the Lucy object model. =A0:)

OK :)  Intentionally leaky abstraction.  FWIW Python took that same
approach for many years, before sprouting a cycle detector addition to
its GC.  But it's awkward, because sometimes things are collected
immediately (when they decref to 0), but at other times, only when the
cyclic collector sweeps.  And of course, because it's tracing, it
requires each class to expose a "visit all things I refer to" API.

You might consider adding the hooks for a tracing collector, but not
creating such a collector now.  IE have each Lucy obj expose a method
to visit the other objs it refers to.

>> since the host language is involved, a cycle could easily reach out to t=
he
>> host language and wrap back around into Lucy?
>
> Possible. =A0I doubt it will be much of a problem for programmers accusto=
med to
> working within a refcounted environment.

I think you'd be surprised at how easily cycles are accidentally created.

It's sort of like saying programmers are accustomed to writing high
performance code, but then an O(N^2) slips in somewhere and goes
undetected until N happens to get large deep in production.

> Programmers who normally work within a tracing GC environment may be more
> prone to create reference cycles, because they're not trained to avoid
> cyclical data structures and alarms might not go off in their heads when =
they
> see them.
>
> But what are our options? =A0Our class hierarchy is sophisticated and lar=
ge
> enough that we have to use either tracing GC or refcounting. =A0Writing o=
ur own
> tracing GC, now *that* would be ambitious. ;) =A0And though I haven't stu=
died
> cycle detection, I imagine it has to be both involved and costly.

Agreed.  I wonder if there's a decent tracing GC library somehow out
there, that only requires certain limited methods to be exposed for
your objects...

And note that I'm really playing devil's advocate here -- I don't have
good solutions to offer for these hard problems.

Mike