lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucy Wiki] Update of "LucyIncubatorProposal" by MarvinHumphrey
Date Thu, 01 Jul 2010 01:41:05 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucy Wiki" for change notification.

The "LucyIncubatorProposal" page has been changed by MarvinHumphrey.
The comment on this change is: Flesh out "An Excessive Fascination...".
http://wiki.apache.org/lucy/LucyIncubatorProposal?action=diff&rev1=22&rev2=23

--------------------------------------------------

  ## page was renamed from GraduationPlan
  For reference, see the original template at [http://incubator.apache.org/guides/proposal.html]
+ 
  == Preface ==
- 
  Lucy is a sub-project which is being spun off from the Lucene TLP but is not yet ready for
graduation.  We propose to address certain needs of the project by assimilating the KinoSearch
code base, and enter the Incubator on a top-level-project track.
  
  == Abstract ==
@@ -17, +17 @@

  
  In the last year, the work has come to fruition: major technical milestones have been achieved
and Lucy's underpinnings have been completed.  Additionally, other developers from the KinoSearch
community have taken an interest in Lucy and have begun to ramp up their contributions.  The
next steps for Lucy were articulated by the Lucene PMC in a recent review: make releases,
acquire users, grow community.
  
- To implement the Lucene PMC's recommendations and get to a release as quickly as possible,
the Lucy community proposes to assimilate the KinoSearch code base, which has been retrofitted
to use Lucy's core.  Lucy still lacks a number of important indexing and search classes; we
wish to flesh these out via IP clearance work rather than software development. 
+ To implement the Lucene PMC's recommendations and get to a release as quickly as possible,
the Lucy community proposes to assimilate the KinoSearch code base, which has been retrofitted
to use Lucy's core.  Lucy still lacks a number of important indexing and search classes; we
wish to flesh these out via IP clearance work rather than software development.
  
  Since the Lucene PMC will not be responsible for Lucy much longer, it is more appropriate
for the software grant to take place within the context of the Incubator than the Lucene TLP.
 As none of our current members have Apache PMC experience, we also seek to take advantage
of the Incubator environment to prepare ourselves for responsible self-governance.
  
@@ -30, +30 @@

   * Expose a public C API.
  
  == Current Status ==
- 
  === Meritocracy ===
  Our initial committer list includes two individuals (Peter Karman and Nathan Kurz) who started
off as KinoSearch users, demonstrated merit through constructive forum participation, adept
negotiation, consensus building, and submission of high-quality contributions, and were invited
to become committers.  Peter now rolls most releases.
  
@@ -42, +41 @@

  Lucy's chief challenge is growing its community, which it hopes to achieve through efforts
in two areas: reaching a 1.0 release, and actively reaching out to its target audience, users
and developers in the dynamic language communities who want a fast, scalable full-text search
solution in their native language.
  
  === Core Developers ===
- 
   * Marvin Humphrey is the project founder of KinoSearch, and co-founded the existing Lucy
subproject.  He is presently employed by Eventful, Inc.
   * Peter Karman has contributed to several open source projects since 2001, including being
a committer at http://swish-e.org/ (a search engine), http://code.google.com/p/rose/ (an ORM)
and http://catalyst.perl.org/ (web framework).  He is employed by American Public Media.
   * Nathan Kurz has participated in numerous open source projects and has been a KinoSearch
committer since 2007.  He is currently Chief Flavor Engineer of Scream Sorbet, and writes
software in his copious free time.
@@ -55, +53 @@

  A third priority of ours is to be bound by existing Apache institutions, for the protection
of all our stakeholders.
  
  == Known Risks ==
- 
  === Orphaned products ===
  All initial committers have been associated with the project for several years across multiple
jobs.  However, at this time, the project would probably not survive the departure of Marvin
Humphrey, so there is a risk of being orphaned.  Marvin has no plans to leave, but we have
been actively working to disperse his knowledge of the code base and administrative responsibilities
in order to make him dispensable.  Having staggered badly after Dave Balmain's departure,
we are keenly aware of this vulnerability and highly motivated to eliminate it.
  
  === Inexperience with Open Source ===
- 
  The initial committers have all have significant experience with open source development,
and include one present Apache committer.  We recognize that we lack PMC experience and seek
to address that deficiency by going through the Incubator.  In retrospect, Marvin wishes that
Lucy had gone through the Incubator during its first inception.
  
  === Homogenous Developers ===
@@ -74, +70 @@

  
  Still, Lucy is a loose port and its core differs in fundamental ways from that of Lucene.
 The biggest difference is that for Lucy, "the OS is our JVM":  Lucene Searcher objects build
up optimized data structures at search-time in process RAM, while Lucy writes its data structures
to disk at index-time and reads them via memory-mapped IO at search-time.  This affords Lucy
several advantages, such as fast process launch, low process RAM requirements, and OO design
flexibility because Lucy's "cheap Searcher" objects are lightweight, thin wrappers around
the system IO cache.
  
- === A Excessive Fascination with the Apache Brand ===
+ === An Excessive Fascination with the Apache Brand ===
- Lucy's past sins:
+ Our desire to maintain Lucy's affiliation with Apache has much less to do with the brand
and much more to do with our conviction that developing the project The Apache Way under Apache
institutions is in Lucy's best interests.  However, we have to acknowledge that during its
time as a Lucene subproject, Lucy has not always fulfilled certain key requirements for an
Apache project.  In particular, it has failed to "release early, release often", and it has
made only limited progress in expanding its community.
  
+ We attribute some of our difficulties to the what may have been excess ambition in the original
Lucy plan, given the scope of the project and the size of the initial committer list:
-  * Failed to release early and often.
-  * Failed to build community.
  
- Proposal is intended to address those deficiencies.
+  . [http://www.apache.org/foundation/how-it-works.html#incubator]
+ 
+  . The basic requirements for  incubation are:
+   * a working codebase -- over the years and after several failures, the foundation came
to understand that without an initial working codebase, it is generally hard to bootstrap
a community.
+ 
+ By rebooting the project with a working codebase, we expect to avoid the trap that ensnared
Lucy's first incarnation: we will release early, release often, accumulate users, nurture
contributors, and grow our community.
  
  == Documentation ==
- 
   * Subversion repository: [http://www.rectangular.com/svn/kinosearch/]
   * Perl API documentation: [http://www.rectangular.com/kinosearch/docs/devel/]
   * Discussion list: [http://www.rectangular.com/mailman/listinfo/kinosearch/]
@@ -97, +96 @@

  == External Dependencies ==
  The Perl bindings for KinoSearch depend on a few CPAN modules which do not have Apache-compatible
licenses.  It will be possible to address all of these during incubation:
  
-  * JSON::XS can be eliminated by writing our own JSON parser. 
+  * JSON::XS can be eliminated by writing our own JSON parser.
   * Lingua::Stem::Snowball and Lingua::StopWords are both wrappers around Snowball, which
is itself BSD licensed.  We can eliminate them by writing our own wrappers and bundling the
Snowball files.
   * The build currently requires Parse::RecDescent.  However, the compilation stage which
requires Parse::RecDescent can be moved up into release preparation, so it can be eliminate
as an end-user dependency.
  
  The build also requires modules which are now bundled with core Perl as of 5.10.0 (Module::Build,
ExtUtils::CBuilder), but were not with Perl 5.8.x.  Guidance is sought as to whether they
are acceptable.
  
  == Required Resources ==
- 
  === Mailing lists ===
- 
   * lucy-dev
   * lucy-private (with moderated subscriptions)
   * lucy-commits
@@ -128, +125 @@

  
  == Sponsors ==
  === Champion ===
- 
  TBD
  
  === Nominated Mentors ===
- 
  TBD
  
  === Sponsoring Entity ===
- 
  Lucy is currently sponsored by Lucene as a sub-project. This proposal advocates changing
Lucy's relationship with Apache from developing all new code as a Lucene sub-project, to instead
assimilating existing code (KinoSearch) under the sponsorship of the Incubator.
  

Mime
View raw message