drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Parth Chandra <par...@apache.org>
Subject Re: [DISCUSS] Design Documents
Date Mon, 19 Oct 2015 18:54:09 GMT
Agreed, that we need to have a better response from the dev community when
a proposal is put forward but the absence of a response does not imply that
we need not write any design down. The document has many consumers down the
road (new contributors, documentation, and even end users who want to
understand the internals). More immediately the design document will help
code reviews because the developer's intent is explicitly stated.  And, of
course, without the design document, there can never be any comment or
community participation. So even if only a few people comment, we are still
much better off.

On a more philosophical note, I believe that the ability to write a good
design document translates directly into the ability to write good code. If
you cannot express your design clearly, the design itself is probably not
clear, and the implementation will be even less so. (I'm not religious
about this, I mention it only to explain where I'm coming from).

I did propose that we need to have a design discussion limited by time.
Very often, a potential reviewer will put off a comment for later when
(s)he gets more time and then, of course, never gets around to it. Maybe
having a limit will get some people to respond in a more timely fashion.
This is the same as having a 'public comment' period; if no one responds,
then the design, as proposed, stands.

I have a few more observations, especially about using JIRAs.  I feel a
shared document is probably the best way to comment on designs - I find
JIRAs are hard to carry out a discussion in and looking for a specific
design some months down the road is a little bit like [1] .  Secondly,
there are JIRAs  which have a fair amount of detail, especially about the
implementation, but implementation details may not be sufficient as a
design document.  Finally, I sometimes see that the pull/review request
follows immediately after the creation of the JIRA and/or after a comment.
This really means there is no actual discussion before implementation and I
think this is best avoided.

These are general comments on what I feel is the right way forward. I would
rather not go into specific instances until we are all in general
agreement. I would also venture to suggest that any developer waiting for
comments should use the hangouts to ask committers and other members to
comment and drive the review forward.

Are there any other thoughts out there?


[1] http://www.goodreads.com/quotes/tag/bypass

On Sun, Oct 18, 2015 at 4:44 PM, Jacques Nadeau <jacques@dremio.com> wrote:

> Parth,
> Thanks for bringing this up. We definitely need to do a better job of
> discussing development decisions. I think whether this is done as a set of
> descriptions and comments on JIRA or a formal doc on Google is less
> important (and I wouldn't be inclined to enforce one over the other).
> That being said, I think there is something else that limits the success of
> such an effort. We first must ask: how do we get more people responding and
> providing feedback among the things people have already posted. I know I've
> experienced silence numerous times when asking for feedback and so have
> others. Some recent examples I've seen in the community:
>  - DRILL-3738 has received very little to no feedback despite providing an
> initial design document
>  - DRILL-3229 has one general response (ask for more detail) from you with
> a follow-up from Steven but no additional feedback on the actual proposal
> So I put it back to you and the general list, how do we get people to
> provide more feedback on all contributions and proposals? I think it goes
> beyond designs. More issues should be opened with better descriptions and
> proposals around why one would do something. When the basic outline has
> consensus and feedback, people can move to more thorough designs. Why
> haven't we seen response on these issues?
> I can't see a requirement of reviewed design docs being enforced until we
> start to seeing people providing feedback on feature proposals and existing
> (albeit thin) design documents. So +1 long term but -1 until people start
> to respond and provide feedback on the outstanding items. Contributors need
> to perceive value in presenting a design doc. Let's get the WIIFM right so
> that developer incentives are aligned.
> Jacques
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
> On Fri, Oct 16, 2015 at 10:21 AM, Parth Chandra <parthc@apache.org> wrote:
> > Hi guys,
> >
> > Now that 1.2 is out I wanted to bring up the exciting topic of design
> > documents for Drill. As the project gets more contributors, we definitely
> > need to start documenting our designs and also allow for a more
> substantial
> > review process. In particular, we need to make sure that there is
> > sufficient time for comment as well as a time limit for comments so that
> > developers are not left stranded. It is understood that committers should
> > ensure they spend enough time in reviewing designs.
> >
> > I can see some substantial improvements in the works (some may even have
> > pull requests for initial work) and I think that this is a good time to
> > make sure that the design is done and understood by all before we get too
> > far ahead with the implementation.
> >
> > [1] is an example from Spark, though that might be asking for a lot.
> >
> > [2] is an example from Drill - Hash Aggregation in Drill - This is an
> ideal
> > design document. It could be improved even further perhaps by adding some
> > implementation level details (for example parameters that could be used
> to
> > tune Hash aggregation) that could aid QA/documentation.
> >
> > What do people think? Can we start enforcing the requirement to have
> > reviewed design docs before submitting pull requests for *advanced*
> > features?
> >
> > Parth
> >
> > [1] http://people.csail.mit.edu/matei/papers/2012/nsdi_spark.pdf
> > [2]
> > https://issues.apache.org/jira/secure/attachment/12622804/DrillAggrs.pdf
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message