lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <>
Subject Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes
Date Thu, 04 Dec 2008 23:23:04 GMT

On Dec 4, 2008, at 2:21 PM, Jason Rutherglen wrote:

> To put things in perspective, I believe Microsoft (who could  
> potentially place a lot of resources towards Lucene) now uses Lucene  
> through Powerset? and I don't think those folks are contributing  
> back.  I know of several other companies who do the same, and many  
> potential contributions that are not submitted because people and  
> their companies do not see the benefit of going through the hoops  
> required to get patches committed.  A relatively simple patch such  
> as 1473 Serialization represents this well.

What do you suggest?  We didn't force anyone to use Lucene.  Heck,  
most of our users don't even ever participate on the mailing list.

We do provide a very clear, transparent path for making contributions  
and becoming a committer.  I don't know what else we can do, but we're  
totally open to suggestions on how to improve it.

FWIW, just b/c you think 1473 is trivial doesn't make it so.  You have  
a single use case and that's all you care about.  The community has  
dozens, if not hundreds of use cases, and your "trivial" patch may not  
be so trivial in that regards.  How would you feel if we "broke"  
something that you have relied on for years in the name of us moving  
faster?  I am willing to bet the large number of people here in Lucene  
appreciate our deliberations for the most part.  As for my opinion on  
1473, I personally think there are better ways of achieving what you  
are trying to do, as Robert and others have suggested and I don't  
think it is worth it to maintain serialization across versions as it  
is a too large of a burden, IMO.  But, heh, make an argument  
(preferably w/o the accusations) and convince me otherwise.

> For example if a company is developing custom search algorithms,  
> Lucene supports TF/IDF but not much else.  Custom search algorithms  
> require rewriting lots of Lucene code.  Companies who write new  
> search algorithms do not necessarily want to rewrite Lucene as well  
> to make it pluggable for new scoring as it is out of scope, they  
> will simply branch the code.  It does not help that the core APIs  
> underneath IndexReader are protected and package protected which  
> assumes a user that is not advanced.  It is repeated in the mailing  
> lists that new features will threaten the existing user base which  
> is based on opinion rather than fact.  More advanced users are  
> currently hindered by the conservatism of the project and so  
> naturally have stopped trying to submit changes that alter the core  
> non-public code.

So, your mad at us for others not contributing back their forks?  Even  
the ones we don't know about?  Simply put, I'm sorry we can't please  
you.  If you go read the archives, you will see plenty of times when  
even us committers have been frustrated from time to time by the  
process (just look at the JDK 1.5 debate, or the Interface/Abstract  
debate) but in the end, I feel Lucene is stronger for it.  Community  
over code, it's the Apache Way.  You are free to disagree.  In fact,  
you have several options available to you to show that disagreement:   
1. You can work to become a committer and change it from within.  The  
bar really isn't that high, 3 to 4 non-trivial patches and a  
willingness to work with others in a mostly pleasant way.  2.  You can  
make us aware of the patches and be persistent about seeing it through  
and we'll try to get to it.  Just look at CHANGES.txt and JIRA and you  
will see that this happens all the time and from a wide variety of  
contributors (including both you and John).  3.  You can fork the code  
and go do your thing and build your own community, etc.

Personally, I hope you choose 1 or 2, as we're all stronger together  
than we are apart.

> The rancor is from users would benefit from a faster pace and the  
> ability to be more creative inside the core Lucene system.  As the  
> internals change frequently and unnannounced the process of  
> developing core patches is difficult and frustrating.

I'm sorry that we can't work at a faster pace.  Suggestions on how to  
deal with the number of patches we have and still maintain quality and  
how to move forward w/o breaking old patches are much appreciated.

As for the internals changing, you have just hit the nail on the head  
as to why it is so important to maintain back-compat.

I simply don't get the unannounced part.  What isn't announced?  Geez,  
I've been a committer for a few years now, and I have yet to see  
another open source project that is as public as Lucene, for better or  
worse.  Look at the archives, we regularly even put our warts out for  
public consumption in an effort to improve ourselves.

Rather than continue hijacking this thread, why don't we either let it  
die and focus on serialization, or we go over to java-dev and you and  
John and the rest of us can create a concrete list of suggestions that  
we think could make Lucene better and we can all discuss them in a  
positive manner and see how we can go about addressing them.  I'd be  
more than happy to discuss there if you want.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message