Mailing-List: contact dev-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAAOiHjFDOZE+r2zmn+YWF=bKc4JAocVKGcvaCpfTj0gXdfxLUw@mail.gmail.com>
References: 
 <CAAOiHjFDOZE+r2zmn+YWF=bKc4JAocVKGcvaCpfTj0gXdfxLUw@mail.gmail.com>
Date: Sun, 18 Oct 2015 16:44:20 -0700
Message-ID: 
 <CAJrw0ORny0S-z1QKRqoay7itNReZYJdQgoAMDpr7uzQHGG29Fg@mail.gmail.com>
Subject: Re: [DISCUSS] Design Documents
From: Jacques Nadeau <jacques@dremio.com>
To: dev@drill.apache.org
Content-Type: multipart/alternative; boundary=047d7b86ca78914a230522699bbf

--047d7b86ca78914a230522699bbf
Content-Type: text/plain; charset=UTF-8

Parth,

Thanks for bringing this up. We definitely need to do a better job of
discussing development decisions. I think whether this is done as a set of
descriptions and comments on JIRA or a formal doc on Google is less
important (and I wouldn't be inclined to enforce one over the other).

That being said, I think there is something else that limits the success of
such an effort. We first must ask: how do we get more people responding and
providing feedback among the things people have already posted. I know I've
experienced silence numerous times when asking for feedback and so have
others. Some recent examples I've seen in the community:

 - DRILL-3738 has received very little to no feedback despite providing an
initial design document
 - DRILL-3229 has one general response (ask for more detail) from you with
a follow-up from Steven but no additional feedback on the actual proposal

So I put it back to you and the general list, how do we get people to
provide more feedback on all contributions and proposals? I think it goes
beyond designs. More issues should be opened with better descriptions and
proposals around why one would do something. When the basic outline has
consensus and feedback, people can move to more thorough designs. Why
haven't we seen response on these issues?

I can't see a requirement of reviewed design docs being enforced until we
start to seeing people providing feedback on feature proposals and existing
(albeit thin) design documents. So +1 long term but -1 until people start
to respond and provide feedback on the outstanding items. Contributors need
to perceive value in presenting a design doc. Let's get the WIIFM right so
that developer incentives are aligned.

Jacques


--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Fri, Oct 16, 2015 at 10:21 AM, Parth Chandra <parthc@apache.org> wrote:

> Hi guys,
>
> Now that 1.2 is out I wanted to bring up the exciting topic of design
> documents for Drill. As the project gets more contributors, we definitely
> need to start documenting our designs and also allow for a more substantial
> review process. In particular, we need to make sure that there is
> sufficient time for comment as well as a time limit for comments so that
> developers are not left stranded. It is understood that committers should
> ensure they spend enough time in reviewing designs.
>
> I can see some substantial improvements in the works (some may even have
> pull requests for initial work) and I think that this is a good time to
> make sure that the design is done and understood by all before we get too
> far ahead with the implementation.
>
> [1] is an example from Spark, though that might be asking for a lot.
>
> [2] is an example from Drill - Hash Aggregation in Drill - This is an ideal
> design document. It could be improved even further perhaps by adding some
> implementation level details (for example parameters that could be used to
> tune Hash aggregation) that could aid QA/documentation.
>
> What do people think? Can we start enforcing the requirement to have
> reviewed design docs before submitting pull requests for *advanced*
> features?
>
> Parth
>
> [1] http://people.csail.mit.edu/matei/papers/2012/nsdi_spark.pdf
> [2]
> https://issues.apache.org/jira/secure/attachment/12622804/DrillAggrs.pdf
>

--047d7b86ca78914a230522699bbf--