lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: [PMC] [DISCUSS] Lucy
Date Sat, 12 Jun 2010 20:47:14 GMT
On Sat, Jun 12, 2010 at 07:10:32AM -0400, Grant Ingersoll wrote:
> Since the last email discussing Lucy, Marvin has been working on it, AFAICT,
> which is a good thing.  

A year ago, the Lucy repository contained 6,275 lines of code.  Today, it
contains 46,827.

> there does not appear to be anyone else who has contributed to it at any
> level beyond the occasional email here and there.  

Not true.   Over the last year there have been steady patch contributions from
Peter Karman, and occasional contributions from Nathan Kurz.

> There also has yet to be a single release in its time here.  

If that's the new criteria, Lucy could make an initial release within 3
months.  Technically speaking, it could achieved within a couple weeks.

Lucy's high-level design is essentially done.  The hardest part was the
Clownfish object model, which took a long time to conceive and refine, and
which finally went into the repository last summer and fall.  The second
hardest part was the integration of mmap, for which the support classes went
in last winter.

Everything which remains has working prototype code which could be committed
at any moment.  All of this code is "new" in that it was written after the
Lucy project was launched.  With the exception of the highlighter, all code
was written entirely by myself, Peter and Nate -- so there are no IP clearance
issues to resolve.  Furthermore, the prototype code adheres to designs
hammered out on the Lucy and Lucene mailing lists.

It would be both possible and justifiable to accelerate the pace of commits so
that we can make an initial release within some arbitrarily compressed time
frame.  However, the pace has remained steady and deliberate for several

First, some of the code is being reworked as it gets contributed: interfaces
and documentation are reviewed, test coverage is expanded, gnarly sections of
code are refactored for clarity and so on.  We don't *have* to do that, but
it's being done with the community in mind, in that the primary goal is to
make the code as easy as possible for other people to understand, maintain and

Second, Lucy was proposed as a development project, not a software grant.  It
would be within the letter of the proposal to accelerate the pace of commits,
but if we go too fast, I worry that we start to wander outside the spirit.

Third, there have been other agenda items which have taken priority -- in
particular last year, the integration of mmap.  Memory-mapping sort caches was
a breakthrough which allowed us to implement real-time search, and in my view,
it was more important to work on that then it was to make an initial release.

Fourth, Lucy will inherit the KinoSearch user community at the instant that we
deprecate KinoSearch and declare Lucy its successor, and at that point our
user community will be big enough to satisfy Apache requirements.

Lastly, it seems to me that adding 40,000 lines of code over the course of a
year ought to be sufficient to demonstrate that a project is active.

> If this were a project coming from the Incubator to us via a "graduation
> vote", I would vote to not let it graduate.

I wouldn't either, but I don't believe that's relevant.  If it were an
Incubator project, we wouldn't be calling a graduation vote at this time.

> Finally, given that Lucy undoubtedly is a separate community (if it ever
> exists) with separate goals from Lucene and that it is considered ASF best
> practice for PMC's to not be "umbrella" projects, I think we should consider
> either Lucy going into the Incubator with the goal of growing it's own
> community and standing on it's own as a TLP in its own right (just as we
> recommended for CLucene recently) or going to Google Code or some other such
> hosting service where it will be free to make decisions on its future
> without the hinderance of a PMC that isn't aligned with its needs and
> objectives, as I believe is the current case with the Lucene PMC.  

The last time Lucy was discussed, in March of last year, the PMC issued a
6-month deadline to demonstrate activity.  We met that test easily.

If the new requirement is that Lucy spin off as a top-level project, tell us
how soon you want that to happen.  We will meet your requirements this time,


Marvin Humphrey

View raw message