incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <>
Subject Re: concerns about high overhead in Apache incubator releases
Date Mon, 28 Nov 2011 01:26:51 GMT
OK, I'm sorry, but leviadocs??!!

I'll buy you a beer for that AWESOME word :-) BTW, I agree with all
of your points below, dude. In fact, to add to them, I would suggest 
my approach is simply what i learned from Justin, Jukka, Joe S. and 
others -- teach the project how to build community, to learn how to identify 
"wants" from "requirements", to push back when necessary (as Joe 
S. mentioned), all the while shielding it from the wild west that's general@, 
and all the while trying to check items off the list towards graduation.

It's really more of a navigation process more than a learning process,
unfortunately in my experience. And no, the answer to me is not improving the docs.
They've been there forever. They've had tons of work on them from 
many many members. Plus, software engineers and open source 
geeks like us don't really do well reading documentation. 

The answer to me is figuring out how to distill experience into action, 
without handing someone all 7 years of the Defence Against the 
Dark Arts books and asking them to brush up on them before casting
any spells. 

I honestly find myself agreeing with some of the folks on the thread
that have suggested an IPMC reboot. I would first start by identifying
the roles or types of mentors that are present and required in the IPMC, 
perhaps as an initial list:

1. those that are good at release management
2. those that get the ALv2 license requirements
3. those that are good at encouraging the growth of community by 
adding new committers/PPMC members.
4. those that are good at filing board reports, or project reports.
5. those that are actually technical mentors, and who have an understanding
of the code going on in the projects.
6. those that have been at the ASF for a LONG time, e.g., the "old guard", that 
can provide context and JFDI attitude to resolve disputes. 

I'm sure there's more types of mentors than this, but I'm just reeling off 
some of the types I've seen. And usually the roles are disjoint and not 
provided by a single person.

Then, after identifying these role types, I'd get 3 mentors for each type, just 
so there are backups and folks who can do the work. So let's say I missed 5 
roles, so that would be ~11 roles, times 3, ~33 members of the IPMC. Then I'd
make sure that each IPMC mentor is on at least 3 Incubator projects, watching 
over them. We could initially start out with disjoint partitioning, but then ease up 
and allow the IPMC'ers to have overlapping projects they watch.

I'd then do away with the podling specific reports as has been suggested 
earlier either on this thread, or on one of the related ones, and then simply 
have an IPMC report to the effect of:

Project X hasn't made a release in > 9 months, mentor Y is prodding them on the dev list
Project Z is being attic'd
Project A has added a new PPMC member
Project B presented at ApacheCon NA on Topic D

I'd also watch out for IPMC members that are too overcommitted. I myself, am 
overcommitted, but I do try and make sure I at least contribute monthly to 
the several projects that I mentor by at least making contributions in one of 
those 6 mentor-type functions I listed above.

So, anyways, while I agree in principle that Apache usually benefits from 
small, incremental change, in revertible steps, what these threads have 
shown me over the past few weeks is that the experience within the 
Incubator for folks is largely too inconsistent, too unwieldy, and filled with 
overhead that is causing folks to have a difficult time bringing their software
to the ASF, and that perhaps grand change may be needed, beyond small
revertible steps. That's a big concern for me, and I hope it is for the rest of the ASF
members who want folks to keep bringing their projects here.


On Nov 27, 2011, at 4:46 PM, Chris Douglas wrote:

> Ross is 100% in identifying mentors as critical to a smooth release.
> More specifically, mentors familiar with what a project is likely to
> face in an Incubator vote.
> I'm sorry to say that I was an AWOL mentor for the first 5 RCs. I
> still wouldn't have anticipated the objections from the IPMC that- as
> Jun points out- were true of every release. By way of illustration,
> take the debate on source releases that spread outside of general@ and
> into other foundation lists. It took over three days to get a yes/no
> answer from *anyone*, and while hundreds of words on why the answer
> could be yes were written, the closest we got to a definitive answer
> on foundation policy was a link to something Roy said in 2009 on
> legal-discuss@. And none of that discussion is available to podlings!
> Even that didn't speak to our question. RC6 contained all the source
> and unit tests, but it also included artifacts of a successful build.
> The discussion was focused on minimum requirements, while RC6 was
> rejected (in part) for including too much.
> The incubator documentation on releases is over 10k words with at
> least 80 TODO items. So while I agree that mentors' familiarity with
> the process is critical to smooth releases, I reject the RTFM
> suggestion as trolling. Further, it's not policy when objections *not*
> in the documentation get added and cited ex post facto.
> In some of these threads, the Incubator is confused with an ASF
> project. This is incoherent given its size and composition. The
> Incubator is a curriculum, not a community. And if we're going to
> continue to use metaphors like "graduation" and "mentor", then the
> requirements for a release must 1) be stated crisply and succinctly 2)
> be separated from best practices, no matter how widely practiced and
> highly regarded some of those procedures may be.
> As examples from recent release votes: a particular sequence of
> transformations in subversion for composing a release is not a
> requirement. Small tarballs are not a requirement. Correctly composing
> the LICENSE, DISCLAIMER, and NOTICE files are requirements.
> If I've learned one thing from trying to advise on a release, it's
> that I know a lot less than I thought I did. I might be an acceptable
> teaching assistant, but of the 100+ IPMC members, there are only a
> handful of tenured members who can distinguish lore from canon. I (and
> others, no doubt) would happily furnish pints to IPMC members who can
> distill what already exists into a small set of rules, rather than
> augmenting the existing Leviadocs. -C
> On Sun, Nov 27, 2011 at 12:09 PM, Ross Gardler
> <> wrote:
>> I sympathize with you're comments, however, I do want to point out that the
>> problems are more to do with the Project committers and mentors than the
>> process (although documentation can always be improved).
>> As evidence I submit the Apache Rave poddling. This project made its first
>> release within a couple of months of entering the incubator and has made a
>> release every month since (I've not checked the dates, but I think this
>> statement is accurate).
>> Rave achieved this because Ate Douma (mentor) pointed to the appropriate
>> docs. Matt Franklin read and understood the docs and did a release. Ate
>> watched and advised throughout the process. The first trekker took a couple
>> of cycles to get right. All sidewinder releases have been "simple".
>> Please don't think I'm saying there is no value in your mail, there is. We
>> can certainly improve in the support we provide. To address your specific
>> points:
>> 1. Your mentors are the example, if they are not guiding you ask if anyone
>> here can help.
>> 2. Different views of different people is difficult to resolve (see Roberts
>> recent mail on the same topic). My advice is to understand the (admittedly
>> confusing) documentation. If that doesn't help ask on the appropriate list
>> (here if you don't know which list)
>> 3. Clone or best mentors - sorry nothing better to suggest here
>> 4. Get it right first time (mentors like Ate only let it go to a vote if it
>> is ready, so 72 hours is called once only). Also know the rules with
>> respect to release voting (see Joe's mail).
>> Finally, and most importantly, help us improve the process (as you are
>> doing with this mail). Given my responses above is there anything concrete
>> you suggest we do to improve things (patches to docs seem like an obvious
>> start - most of those docs are written by people who already do Apache
>> releases).
>> Ross
>> Sent from my mobile device, please forgive errors and brevity.
>> On Nov 27, 2011 7:13 PM, "Jun Rao" <> wrote:
>>> Dear Apache members,
>>> Over the past 2 months, the Kafka Apache incubator project has been trying
>>> to release its very first version in Apache. After 7 RCs, we are still not
>>> done. Part of this is because most of us are new to the Apache release
>>> process and are learning things along the way. However, I think Apache can
>>> do a better job in the incubating process to make releases much less
>>> painful. In the following, I will summarize some of problems that we
>>> had experienced. This is not an accusation, nor is it personal. I just hope
>>> that we can all learn from our experience so that Kafka and other incubator
>>> projects can release more smoothly in the future.
>>> 1. There is no good example to follow.
>>> As a new incubator project, the natural thing for us to do when it comes to
>>> releasing our code is to follow what other Apache projects do. However,
>>> more than once, the feedback that we got is that those are not good
>>> examples to follow. It seems that those "bad" examples are not isolated
>>> cases.
>>> 2. Different Apache members have different interpretations of the same
>>> rule.
>>> It seems that there is no consensus on some of the basic rules even among
>>> Apache members. For example, what constitutes a source distribution and
>>> what should be put in the NOTICE file? Since all it takes is one negative
>>> vote to block a release, this increases the turnover rate of RCs.
>>> 3. Not enough constructive and comprehensive suggestions.
>>> Some of the issues that are present in Kafka RC7 exist in RC1. Those issues
>>> could have been resolved much earlier had there been more constructive and
>>> comprehensive feedbacks from early on. Instead, often, the feedback just
>>> points out the violation of one or two issues that are enough to block a
>>> release. People like Ant Edler have made some constructive suggestions and
>>> we really appreciate that. We could use more suggestions like that.
>>> 4. Not enough flexibility in applying the rules.
>>> Some of the rules don't make common sense. For example, if we publish a new
>>> RC that simply fixes a few lines in NOTICE/LICENSE. We are still required
>>> to go through a full 3-day vote in Kafka and another full 3-day vote in
>>> Apache general. This, coupled with the high turnover rate of RCs, can delay
>>> the release for a significant long time. Both Chris Douglas and Ant Edler
>>> wanted to relax the rule slightly to help us speed things up. However, not
>>> every Apache member tolerates such flexibility. Again, all it takes is just
>>> one vote to kill a release.
>>> To summarize, our experience of releasing in Apache has not been very
>>> pleasant so far. I am not sure if our experience is the exception or the
>>> norm among incubator projects. In any case, I sincerely hope that at least
>>> some of those concerns can be addressed in Apache to make the release
>>> process more enjoyable, especially for new comers.
>>> Thanks,
>>> Jun
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message