lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <>
Subject Re: Special Board Report for May 2011
Date Thu, 12 May 2011 20:09:04 GMT

On May 12, 2011, at 3:23 PM, Smiley, David W. wrote:

> Hi.
> I haven't commented yet but I've absolutely been following this soap opera.  I'm referring
to all related threads and JIRA discussions, not just this thread.
> I think the current state of logging only #lucene-dev is good.

Yeah, except no one is on it other than a few people even though many of them (committers
that is) are on #lucene

>  I go to #lucene-dev now. I think only IRC channel(s) that are Lucene/Solr internal development
in nature need to be logged -- and that's just #lucene-dev. So just because you have observed
many developers are on #lucene instead of #lucene-dev doesn't indicate a problem, so long
as no design decisions for Lucene/Solr take place on #lucene or #solr.  #lucene and #solr
is where users get to ask questions, much like how it is on the user mailing lists.  So *if*
(I don't know if it happens) internal Lucene / Solr design decisions are taking place on #lucene
or #solr then obviously that must stop. I'd rather these channels not get logged so that we
can have an expectation of a single place for these discussions on IRC and have that place
be clear of user support questions.
> RE refactoring / modularization, it's good to finally see a sense of agreement on how
to move forward.
> ~ David Smiley
> On May 12, 2011, at 2:36 PM, Grant Ingersoll wrote:
>> I'm not trying to be a pest here, but this conversation and this report is hugely
important to the future of this Lucene PMC.  So far, I see a sum total of 4 PMC members out
of 20+ total PMC members weighing in here, as well as a few other community members, on proactive
things to do to move forward.
>> Are you all really in lazy consensus with what has been said so far? (hint: lazy
consensus is not a good idea here, so, if you are in consensus, at least speak up and say
so)  Do you have other suggestions?  The Board Meeting is on the 19th and this report needs
to be filled at least 2 days prior to that.   Claims of thread fatigue, I am sure, are not
going to go over well with the Board, so I suggest all PMC Members (as well as others) take
some time to think about how to contribute to this report.
>> As it stands now, we have the following concrete suggestions:
>> 1. Log IRC -- from the looks of #lucene-dev, it appears that people have not migrated
to the new logged version.  To me, we really should just hook up the logger to #lucene and
forget #lucene-dev ever existed.  We should also put a note that the room is being logged.
 I am beginning to be of the mindset that any design/dev conversation that is not logged on
IRC is the equivalent of a private conversation.
>> 2. Rotate the Chair -- I would propose that this Report is my last official one and
that the next Board meeting contains a resolution changing the chair.
>> 3. Put in the automated patch checking system that Hadoop uses.  Volunteers?  Perhaps
we can knock this out at Lucene Revolution?
>> 4. Write up lessons learned by all on commit/revert and scratching/itches and make
sure newcomers and old timers alike understand how it works.
>> 5. I gather, via lazy consensus from the other thread, that we are in agreement on
refactoring and we have a way forward.
>> 6. Discourage private emails, phone calls, etc. as they relate to the project.  I
personally am starting to think that if there is wind of this happening more that it is not
at all unreasonable to remove commit bits.
>> -Grant
>> On May 7, 2011, at 6:47 AM, Michael McCandless wrote:
>>> +1
>>> Mike
>>> On Sat, May 7, 2011 at 6:41 AM, Simon Willnauer
>>> <> wrote:
>>>> On Sat, May 7, 2011 at 9:52 AM, Greg Stein <> wrote:
>>>>> I've seen several people note that "IRC is not logged". Fine. LOG IT.
>>>>> I see absolutely no reason for you guys not to set up logging for the
>>>>> channel that you use. We do this for Subversion development:
>>>>> If IRC is posing so much of a problem, then just log it. I saw a
>>>>> comment about civility on the channel. Well... if it is logged, then
>>>>> you may see that fixed. Discussions can then be referenced when it is
>>>>> brought to the dev list. And people can always refer back to the log
>>>>> to read about the nuances around some particular discussion.
>>>>> Seems to be a simple solution to me.
>>>> huge +1! IRC is so powerful we should not put it down just because we
>>>> don't log the channel.
>>>> We already have a logged channel #lucene-dev and its logged here
>>>> having tech discussion there should be ok I think and for major
>>>> decisions we can still send a mail to dev@l.a.o referencing the
>>>> discussion. I think we all have the discipline to do that right?
>>>> I am moving there now... we should also eventually add this channel to
>>>> the website and maybe mark #lucene as the user channel?
>>>> Simon
>>>>> Cheers,
>>>>> -g
>>>>> On Fri, May 06, 2011 at 10:17:07PM +0300, Shai Erera wrote:
>>>>>> bq. shall I say required reading?
>>>>>> You should ! If only so that people don't miss that great article
>>>>>> On IRC, I agree with Grant (and partly w/ Mike). IMO, we should scale
>>>>>> the amount of discussion on IRC. While there are several advantages
to IRC
>>>>>> (faster response time, easier to hash things out etc.), I think there
>>>>>> several drawbacks:
>>>>>> * As Grant mentioned, TimeZone -- IRC makes it hard for people to
>>>>>> discussions that happened while they were asleep
>>>>>> * IRC is not logged
>>>>>> * Even trying to follow discussions on IRC, the nature of the UI
>>>>>> makes it too hard. Many times I've seen two and more discussions
>>>>>> simultaneously, and the way the UI is constructed, they're all mixed
>>>>>> each other. This is not so with email threads.
>>>>>> * I myself have too many communication mediums I need to follow today:
>>>>>> job's email and messaging system, Gmail (Lucene and other mailing
lists, as
>>>>>> well as private stuff), phone, people stopping by for questions ..
IRC is a
>>>>>> very busy and demanding channel. You're kinda expected to respond
>>>>>> immediately (which is why, I think, it's easier to hash things out
-- the
>>>>>> response time is instantaneous). If you only want to follow, you
must stay
>>>>>> tuned to it. If I turn on "flash the taskbar for new messages", it
drives me
>>>>>> crazy. If I turn it off, I miss important discussions ... it's impossible
>>>>>> :).
>>>>>> With emails, I can prioritize things. At least, Gmail helps to some
>>>>>> That that we now receive all JIRA emails under one thread is a great
>>>>>> progress too.
>>>>>> With emails, I can always go back when I have time, and re-read the
>>>>>> discussion. I can respond to it 2 days after the last email, and
people will
>>>>>> immediately know what I respond about, because we can include quoted
>>>>>> And if people's memory is very bad, they can (at least in Gmail)
>>>>>> quickly previous messages. Hack ... I can do that 1 month after the
>>>>>> was sent, and most people will be able to quickly pick up from where
>>>>>> left. This is not so with IRC ...
>>>>>> * Getting in the middle of a discussion is practically impossible
on IRC. I
>>>>>> have nothing to read for reference (unless I had my IRC client open
and I
>>>>>> turned on the 'logging' feature).
>>>>>> * Is it really that easier to hash things out on IRC? I mean, the
>>>>>> time is great, so you get answers really quick. But then, there are
>>>>>> only a handful of participants in that discussion, which makes hashing
>>>>>> and agreeing much easier anyway. If the same group of people (usually
>>>>>> communicated in email, they'd hash things out in almost the same
>>>>>> After all, IRC mandates they are all awake at the same time, so they
>>>>>> also email each other in NRT :).
>>>>>> * Imagine this discussion happening on IRC. Most of us would have
been able
>>>>>> to pick only shards of it. At some point, maybe Grant or another
PMC member
>>>>>> would 'summarize' the discussion to the list. The summary could be
>>>>>> decided to not use IRC because email is better", followed by some
>>>>>> he's able to pull back from his memory and maybe IRC log. Would *you*
>>>>>> (people reading this growing-by-the-minute note) want to get a summary
>>>>>> that? Would you be satisfied?
>>>>>> I think that most of us wouldn't and all that would happen is that
>>>>>> email would start its own thread, repeating mostly what have been
said on
>>>>>> IRC, b/c people would want answers ...
>>>>>> I'm not against IRC, don't get me wrong. I think it's useful b/c
>>>>>> turnaround time is great. But we should not have so many discussions
>>>>>> as we do today. I don't know where to draw the line. I trust the
>>>>>> people of this community to know when it's better to discuss something
>>>>>> email. An example, if a new feature is being discussed, then it's
ok if two
>>>>>> people want to hash few things out quickly, before they send a detailed
>>>>>> organized proposal to the list -- the details to hash out are the
>>>>>> proposal's details. The rest should be followed on list, even if
it means
>>>>>> slightly slower response time.
>>>>>> Today's list and JIRA volume always look to me like the response
time is
>>>>>> instantaneous. We have very active people from around the globe,
so you have
>>>>>> a high chance receiving response in no time. In the worse case, it
will take
>>>>>> a couple of hours, but I don't remember when did that happen (which
is an
>>>>>> amazing thing !)
>>>>>> Cheers,
>>>>>> Shai
>>>>>> On Fri, May 6, 2011 at 8:35 PM, Grant Ingersoll <>
>>>>>>> More reading (shall I say required reading?).  Benson does a
good job of
>>>>>>> explaining some of the concepts around consensus and why we also
should be
>>>>>>> primarily using mailing lists:
>>>>>>> -Grant
>>>>>>> On May 5, 2011, at 10:10 AM, Grant Ingersoll wrote:
>>>>>>>> I'd like to throw out another idea:
>>>>>>>> I think we should standardize on rotating the PMC Chair every
year.  I
>>>>>>> think to date, there have been two Chairs:  Doug and me.  Back
when Doug
>>>>>>> left, no one wanted to do it (both Hoss and I said we would if
no one else
>>>>>>> wanted to) and so I took it on.  For the most part, it's a thankless
task of
>>>>>>> herding cats (albeit low volume, thankfully), despite the important
>>>>>>> name that marketing types love.  I would like us to share the
burden across
>>>>>>> the PMC by rotating it on an annual basis.  Many other ASF projects
>>>>>>> exactly this and I think it removes any political pressure. 
Have I sold it
>>>>>>> enough? ;-)  Besides, I just know others are dying to file board
reports on
>>>>>>> a quarterly basis!
>>>>>>>> More inline below...
>>>>>>>> On May 5, 2011, at 8:27 AM, Michael McCandless wrote:
>>>>>>>>> On Wed, May 4, 2011 at 6:40 PM, Grant Ingersoll <>
>>>>>>> wrote:
>>>>>>>>>> 2. I think we need to prioritize getting patch contributors
>>>>>>> feedback sooner.  I think some of this can be automated much
like what
>>>>>>> Hadoop has done.  This should help identify new committers sooner
>>>>>>> encourage them to keep contributing.
>>>>>>>>> Big +1.  We should be using automation everywhere we
>>>>>>>>> But, really, we (as all projects do) need more devs.
 Growing the
>>>>>>>>> community should be job #1 of all committers.
>>>>>>>> Agreed, but this dovetails w/ the use of IRC.  I realize
live collab is
>>>>>>> nice, but it discourages those who aren't "in the know" about
the channel
>>>>>>> being used from ever contributing.    Say, for instance, I'm
interested in
>>>>>>> DWPT (DocWriterPerThread), how am I supposed to know that at
8 am EDT on May
>>>>>>> 5th (made up example), three of the committers are going to be
talking about
>>>>>>> it on IRC?  If there is email about it, then I can participate.
 Nothing we
>>>>>>> do is so important that it can't wait a few hours or a day, besides
>>>>>>> fact, that email is damn near instantaneous these days anyway.
>>>>>>>> Also, keep in mind that until about a year ago, most everything
was done
>>>>>>> on the mailing list and I think we progressed just fine.  Since
then, dev@has almost completely dried up in terms of discussions (factoring out JIRA
>>>>>>> mails which have picked up -- which is good) and the large majority
>>>>>>> discussion takes place on IRC.  I agree, however, we should have
the IRC
>>>>>>> discussion on another thread.
>>>>>>>>>> So, what other ideas do people have?  I'll leave
this thread open for a
>>>>>>> week or so and then add what we think are good things to
The board meeting is on May 19th.  I plan on attending.
>>>>>>>>> How about also "PMC members will be more proactive in
tackling issues
>>>>>>>>> that erode the community?  I think this would start with
a thread on
>>>>>>>>> general@.  We need to get in the habit of discussing
even tiny
>>>>>>>>> elephants as soon as they appear, somehow.
>>>>>>>> Yeah, I agree.  The hard part for me, is I often feel like
people on the
>>>>>>> outside make big deals about this stuff and don't get that even
having the
>>>>>>> discussion is a very healthy sign.  Besides the fact, that no
one likes
>>>>>>> confrontation and uncomfortable topics.  We also, I think, are
all tired of
>>>>>>> endless debates that go on and on w/ no resolution.  It's one
of the big
>>>>>>> downsides (and, of course, upsides) to consensus based open source
>>>>>>> opposed to the dictatorial approach.
>>>>>>>>> Here's an example: "Is Lucid abusing their too-strong
influence over
>>>>>>>>> Lucene/Solr"?  It's a great question, and I personally
feel the answer
>>>>>>>>> today is "no", but nevertheless we should be able to
discuss it and
>>>>>>>>> similar could-be-controversial topics.
>>>>>>>> I hopefully would agree we are good stewards of the fact
that we employ a
>>>>>>> good number of committers (but not nearly all the active ones),
but I know
>>>>>>> some disagree.  I do, however, think that the recent spat shows
that we at
>>>>>>> Lucid are still free to speak our minds when it comes to open
source, as
>>>>>>> clearly not all Lucid employees agree on the issue and were pretty
>>>>>>> about it.  I firmly believe we baked this into the company from
Day 1 and I
>>>>>>> consider it one of our best strengths, but of course, most can't
see that
>>>>>>> from the outside.  Does that mean we are perfect?  Of course
not, but I
>>>>>>> think we try to follow the ASF guidelines and show up as individuals.
>>>>>>> also know we work pretty hard to mind the ASF TM policy, etc.
(just ask our
>>>>>>> marketing folks how much I remind them.)  I think we all realize
that there
>>>>>>> would be no such thing as Lucid if it weren't for the ASF and
>>>>>>> Lucene/Solr, so why would we want to hurt that?
>>>>>>>> The fact is, every single committer here and a good number
>>>>>>> contributors are paid to work on Lucene all day, (most) every
day or have
>>>>>>> some other financial stake (i.e. via a book, consulting biz,
etc.)  Any of
>>>>>>> us could be accused of only acting in our own financial interest.
 At the
>>>>>>> end of the day, I like to think that instead, the cool thing
is we all have
>>>>>>> a great opportunity to have our financial interests aligned with
a great
>>>>>>> project that we like to work on.
>>>>>>>> For the record, we have pretty diverse PMC and committer
base.  As I said
>>>>>>> in our Dec. 2010 Board Report, we are comprised of:
>>>>>>>> "[a] total to 17 PMC members from 12 different
>>>>>>>> companies, spanning the globe. The flagship Lucene/Solr
>>>>>>>> has 26 total committers from 20 different companies, again
>>>>>>>> spanning the globe."
>>>>>>>> The only one that has changed since then is Robert has joined
>>>>>>> Now, one can argue that some of those members from other companies
are not
>>>>>>> active, but that isn't Lucid's fault.  ASF development has always
been about
>>>>>>> those who do the work and we do a fair amount of that.  Those
who are not
>>>>>>> active, should, ideally, leave on their own by stating they wish
to go
>>>>>>> Emeritus.  Beyond that, we have a pretty standard policy that
>>>>>>> people are removed after 1 year of no activity.  That has been
the case
>>>>>>> since I joined Lucene way back when and I think makes sense.

Grant Ingersoll
Lucene & Solr User Conference
May 25-26, San Francisco

View raw message