lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eks dev <eks...@yahoo.co.uk>
Subject Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes
Date Thu, 04 Dec 2008 06:36:28 GMT
John, 
sorry I have to comment,  but I feel here some substantial missconceptions abot Open Source

1)
"e.g. >30 million documents indexed and searched in realtime., and I really had to do some
tweaking."
So what? What I or anyone else has to do with it? "some tweaking" is definitely better than
making everything from the scratch or going  to commercial vendors... no?


2) 
"what would make it better is some transparency on how patches/issues
are evaluated to be committed. At least seemed from the outside, it is
purely being decided on by the committers, and since my understanding
is that an open source project belongs to the public, the public user
base should have some say."

Transparency, Jira + this mailing list. Everybody is allowed to express an opinion,  *even
committers* , weather you like it or not is just another question. If you put up convincing
arguments, be assured even committers can change opinions.
Imo, it does not go much more transparent than that. 
Sure it belongs to public, you do not have to pay for it, read ASF Licence. If you have better
proposal on how to organize Open Source projects, speak-up.  I do not know how we could ever
avoid committers having final say on things without provoking haos? 

3) "Would groups/companies that pay for consulting service get their patches/requirements
committed with higher priority?"
Sure, of course, *even commmercial users are parts of the comunity* and we  schould be greatful
that they contribute and commit ther resouces so that others can benefit from it. Think again
about it, there is absolutly nothing bad behind it, no conspiracy.
Just one example on micro scale. I had an itch  and had to do some "tweaking", my customer(comercial)
had nothing against contributing back to Lucene, so I did it. I get my money and I give something
back to the comunity. End result, I am happy, Lucene gets better and everybody profits a bit
from it.  
Should I have problems with my consciones?  I do not think so.  

Conflict of interests, no, that is rather evolution. What do you think why commiters work
on Lucene, do you honestly beleive they have no families to feed and just sit and wait someone
feeds them with proposals for nice features?  Commiters as well as everybody else here have
their own, private agendas, goals, ideas, needs ... and all these things get somehow conflated
into Lucene. 
Back to my example, I was lucky that a few commiters shared my opinion about usfulness and
the priority of this patch, it could have been different. If all commiters were busy with
private agenda and had higher priorities at that moment, well, that would habe been bad luck
for me. No hard feelings even in that case, why should I expect someone puts my itch as their
priority.

Cheers, eks

  





 





________________________________
From: John Wang <john.wang@gmail.com>
To: java-dev@lucene.apache.org
Sent: Thursday, 4 December, 2008 6:36:28
Subject: Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher
classes

Grant:

        I am sorry that I disagree with some points:

1) "I think it's a sign that Lucene is pretty stable." - While lucene is a great project,
especially with 2.x releases, great improvements are made, but do we really have a clear picture
on how lucene is being used and deployed. While lucene works great running as a vanilla search
library, when pushed to limits, one needs to "hack" into lucene to make certain things work.
If 90% of the user base use it to build small indexes and using the vanilla api, and the other
10% is really stressing both on the scalability and api side and are running into issues,
would you still say: "running well for 90% of the users, therefore it is stable or extensible"?
I think it is unfair to the project itself to be measured by the vanilla use-case. I have
done couple of large deployments, e.g. >30 million documents indexed and searched in realtime.,
and I really had to do some tweaking.

2) "You want stuff committed, keep it up to date, make it manageable to review, document it,
respond to questions/concerns with answers as best you can. " - To some degree I would hope
it depends on what the issue is, e.g. enforcing such process on a one-line null check seems
to be an overkill. I agree with the process itself, what would make it better is some transparency
on how patches/issues are evaluated to be committed. At least seemed from the outside, it
is purely being decided on by the committers, and since my understanding is that an open source
project belongs to the public, the public user base should have some say.

3) which brings me to this point: "I personally, would love to work on Lucene all day every
day as I have a lot of things I'd love to engage the community on, but the fact is I'm not
paid to do that, so I give what I can when I can.  I know most of the other committers are
that way too." - Is this really true? Isn't a large part of the committer base also a part
of the for-profit, consulting business, e.g. Lucid? Would groups/companies that pay for consulting
service get their patches/requirements committed with higher priority? If so, seems to me
to be a conflict of interest there.

4) "Lather, rinse, repeat.   Next thing you know, you'll be on the receiving end as a committer."
- While I agree that being a committer is a great honor and many committers are awesome, but
assuming everyone would want to be a committer is a little presumptuous.

In conclusion, I hope I didn't unleash any wrath from the committers for expressing candor.

-John


On Wed, Dec 3, 2008 at 2:52 PM, Grant Ingersoll <gsingers@apache.org> wrote:


On Dec 3, 2008, at 2:27 PM, Jason Rutherglen (JIRA) wrote:




Hoss wrote: "sort of mythical "Lucene powerhouse"
Lucene seems to run itself quite differently than other open source Java projects.  Perhaps
it would be good to spell out the reasons for the reluctance to move ahead with features that
developers work on, that work, but do not go in.  The developer contributions seem to be quite
low right now, especially compared to neighbor projects such as Hadoop.  Is this because fewer
people are using Lucene?  Or is it due to the reluctance to work with the developer community?
 Unfortunately the perception in the eyes of some people who work on search related projects
it is the latter.



Or, could it be that Hadoop is relatively new and in vogue at the moment, very malleable and
buggy(?) and has a HUGE corporate sponsor who dedicates lots of resources to it on a full
time basis, whilst Lucene has been around in the ASF for 7+ years (and 12+ years total) and
has a really large install base and thus must move more deliberately and basically has 1 person
who gets to work on it full time while the rest of us pretty much volunteer?    That's not
an excuse, it's just the way it is.  I personally, would love to work on Lucene all day every
day as I have a lot of things I'd love to engage the community on, but the fact is I'm not
paid to do that, so I give what I can when I can.  I know most of the other committers are
that way too.

Thus, I don't think any one of us has a reluctance to move ahead with features or bug fixes.
  Looking at CHANGES.txt, I see a lot of contributors.  Looking at java-dev and JIRA, I see
lots of engagement with the community.  Is it near the historical high for traffic, no it's
not, but that isn't necessarily a bad thing.  I think it's a sign that Lucene is pretty stable.

What we do have a reluctance for are patches that don't have tests (i.e. this one), patches
that massively change Lucene APIs in non-trivial ways or break back compatibility or are not
kept up to date.  Are we perfect?  Of course not.  I, personally, would love for there to
be a way that helps us process a larger volume of patches (note, I didn't say commit a larger
volume).  Hadoop's automated patch tester would be a huge start in that, but at the end of
the day, Lucene still works the way all ASF projects do: via meritocracy and volunteerism.
    You want stuff committed, keep it up to date, make it manageable to review, document it,
respond to questions/concerns with answers as best you can.  To that end, a real simple question
can go a long way and getting something committed, and it simply is:  "Hey Lucener's,  what
else can I do to help you review and commit LUCENE-XXXX?"  Lather, rinse, repeat.   Next thing
you know, you'll be on the receiving end
 as a committer.

-Grant



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


      
Mime
View raw message