hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From larry mccay <lmc...@apache.org>
Subject Re: Guidance needed on HADOOP-13096 and HADOOP-13097
Date Fri, 06 May 2016 20:11:04 GMT
I agree with your rationale for not doing C now.
And those clean up tasks can more easily be discussed when separated from
this effort.


On Fri, May 6, 2016 at 3:11 PM, Allen Wittenauer <aw@apache.org> wrote:

>         After thinking about it, I think you are correct here: I’m more
> inclined to do D w/follow-up JIRAs to fix this up. The hadoop and hdfs
> script functionality is being tested, so it isn’t like HADOOP-12930 is
> going in with zero unit tests. Never mind that large chunks of hadoop-tools
> gets modified to use this “for reals” as well. The yarn and mapred tests
> don’t really bring _that_ much to the table.
>
>         I think the biggest worry about doing C inside the HADOOP-12930
> feature branch is that it seems like the wrong time/place to do it.  Making
> that big of a change to the build should probably be two separate,
> orthogonal JIRAs (one for YARN, one for MR) in their own right.  But I do
> think C is the correct, long-term path.  We should probably move hdfs and
> common scripts into separate dirs as well, honestly.
>
>         Thanks for the feedback!
>
>
> > On May 5, 2016, at 7:22 PM, Larry McCay <lmccay@hortonworks.com> wrote:
> >
> > I would vote for C or D with a filed JIRA to clean up the maven
> structure as a separate effort.
> > Before moving to D, could you describe any reason to not go with C?
> >
> > On May 4, 2016, at 9:51 PM, Allen Wittenauer <aw@apache.org> wrote:
> >
> >>
> >>      When the sub-projects re-merged, maven work was done, whatever,
> the shell scripts for MR and YARN were placed (effectively) outside of the
> normal maven hierarchy.  In order to add unit tests to the shell scripts
> for these sub-projects, it means effectively turning
> hadoop-yarn-project/hadoop-yarn and hadoop-mapreduce-project into “real”
> modules so that mvn test works as expected.   Doing so will likely have
> some surprising consequences, such as anyone who modifies java code and the
> shell code in a patch will trigger _all_ of the unit tests in yarn.
> >>
> >>      I think we have four options:
> >>
> >> a) Continue forward turning these into real modules with src
> directories, etc and we live with the consequences
> >>
> >> b) Move the related bits into an existing module, making them similar
> to HDFS, common, tools
> >>
> >> c) Move the related bits into a new module, using the layout that maven
> really really wants
> >>
> >> d) Skip the unit tests; we don’t have them now
> >>
> >>      This is clearly more work than what I really wanted to cover in
> this branch, but given that there was a specific request to add unit test
> code for this functionality, I’m sort of stuck here.
> >>
> >>      Thoughts?
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message