Mailing-List: contact flume-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: flume-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of
 juhani_connolly@cyberagent.co.jp designates 210.134.177.71 as permitted
 sender)
Message-ID: <4F54224C.2070502@cyberagent.co.jp>
Date: Mon, 05 Mar 2012 11:17:48 +0900
From: Juhani Connolly <juhani_connolly@cyberagent.co.jp>
User-Agent: Mozilla/5.0 (X11; Linux i686;
 rv:10.0.1) Gecko/20120216 Thunderbird/10.0.1
MIME-Version: 1.0
To: flume-dev@incubator.apache.org
Subject: Re: Poor code reviews
References: 
 <CANpnisoaOAFNmQPkq9LZx9K8a9ZRDSgN_Y_0NwVUGH1j_SCTQg@mail.gmail.com>
 <CAHUddLMnS=E4ifiW_v63UGO9mMwxxhk6ijcgyyfnvRAkrs2bJw@mail.gmail.com>
 <CANpnisp3OCg0LBXPThKevFnnT=aZv397CTg5rf9L_Br5yrgT9Q@mail.gmail.com>
 <CAHUddLNCoL62TJSV+jiiz9bLStO8PYFCt8roGS1gSeDOY9BMXA@mail.gmail.com>
In-Reply-To: 
 <CAHUddLNCoL62TJSV+jiiz9bLStO8PYFCt8roGS1gSeDOY9BMXA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I'd just like to address the feature creep issue a bit myself.
>> Maybe I'm being overly cautious but I worry about getting back to a place
>> where the code base is unnecessarily complex and loaded with features
>> without paying attention to stabilization, performance, resource control,
>> and correctness.
> We are all cautious and vigilant about issues creeping in. That said,
> if you notice we miss out anything, help us by pointing it out.
>
>> I know features are exciting, but we're back on to worry
>> about tail and a Hive sink and that worries me. I'd like to see us
>> collectively prioritize a stable, fast, correct core first. That's my
>> opinion.
>>
> If the community feels that a Hive sink is necessary and provides a
> patch, so be it. I don't see anything wrong with it. That said, which
> part of the system do you think the community should be focused on
> right now? Make a case for it and open issues as you see fit.
>

When I first got involved in the project, and actually managed to 
familiarise myself with the codebase and outstanding issues somewhat, 
one of my greatest concerns was that it seemed like some of the core 
initial objectives for FLUME-728 seem to have fallen by the wayside in 
favor of feature creep. When I first found that the memory channel was 
in no way thread safe I was quite surprised and somewhat concerned.

Further, with the JDBC channel and memory channel we have a hard choice 
between a flimsy channel with the potential for high dataloss and a 
heavyweight one with only moderate throughput... The FileChannel issue 
has been more or less stationary. Where are we going with this? Do we 
not consider it particularly important, or is it just stationary because 
it is a hard problem? If the latter, hopefully we could kick off some 
discussion on how to deal with it.

Finally, it seems like every single issue is reported as major, when it 
really isn't the case for many of them.  Many issues also do not have a 
version number attached. It makes prioritizing anything to work on 
awkward, perhaps we should be taking more liberties with recategorizing 
the severity of issues? If others also feel this way I would like to 
sort through the current open major tickets and recategorize some so we 
can focus work on the core issues.