hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer ...@apache.org>
Subject Re: Guidance needed on HADOOP-13096 and HADOOP-13097
Date Fri, 06 May 2016 19:11:12 GMT
	After thinking about it, I think you are correct here: I’m more inclined to do D w/follow-up
JIRAs to fix this up. The hadoop and hdfs script functionality is being tested, so it isn’t
like HADOOP-12930 is going in with zero unit tests. Never mind that large chunks of hadoop-tools
gets modified to use this “for reals” as well. The yarn and mapred tests don’t really
bring _that_ much to the table.

	I think the biggest worry about doing C inside the HADOOP-12930 feature branch is that it
seems like the wrong time/place to do it.  Making that big of a change to the build should
probably be two separate, orthogonal JIRAs (one for YARN, one for MR) in their own right.
 But I do think C is the correct, long-term path.  We should probably move hdfs and common
scripts into separate dirs as well, honestly.

	Thanks for the feedback!


> On May 5, 2016, at 7:22 PM, Larry McCay <lmccay@hortonworks.com> wrote:
> 
> I would vote for C or D with a filed JIRA to clean up the maven structure as a separate
effort.
> Before moving to D, could you describe any reason to not go with C?
> 
> On May 4, 2016, at 9:51 PM, Allen Wittenauer <aw@apache.org> wrote:
> 
>> 
>> 	When the sub-projects re-merged, maven work was done, whatever, the shell scripts
for MR and YARN were placed (effectively) outside of the normal maven hierarchy.  In order
to add unit tests to the shell scripts for these sub-projects, it means effectively turning
hadoop-yarn-project/hadoop-yarn and hadoop-mapreduce-project into “real” modules so that
mvn test works as expected.   Doing so will likely have some surprising consequences, such
as anyone who modifies java code and the shell code in a patch will trigger _all_ of the unit
tests in yarn.
>> 
>> 	I think we have four options:
>> 
>> a) Continue forward turning these into real modules with src directories, etc and
we live with the consequences
>> 
>> b) Move the related bits into an existing module, making them similar to HDFS, common,
tools
>> 
>> c) Move the related bits into a new module, using the layout that maven really really
wants
>> 
>> d) Skip the unit tests; we don’t have them now
>> 
>> 	This is clearly more work than what I really wanted to cover in this branch, but
given that there was a specific request to add unit test code for this functionality, I’m
sort of stuck here.
>> 
>> 	Thoughts?
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> 
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message