hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Foley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8924) Hadoop Common creating package-info.java must not depend on sh, at least for Windows
Date Mon, 14 Jan 2013 21:28:14 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13553131#comment-13553131

Matt Foley commented on HADOOP-8924:

Hi Chris and Alejandro, it's fine with me to test on Mac, Win, and Ubuntu, for consistent
MD5, and I agree it isn't strictly necessary to have the same MD5 as saveVersion.sh.

But I'm concerned about:
One thing I've forgot to mention is that currently (and with this patch) MD5 are done only
for the sources in common and in yarn. And the VersionInfo from common is used in hdfs. IMO,
we should either have a global MD5 & VersionInfo for the whole project or one per module.
This is out of scope of this JIRA, just wanted to bring it up.

I didn't notice because I was focusing on the Hadoop-1 version where I'm more familiar with
the env.  In Hadoop-1 there is simply one checksum for the whole project.

Absent that, I think the checksum for hdfs in Hadoop-2 should be created by summing the MD5
for the hdfs sub-project sources, so each sub-project sums its own sources (as do common and
yarn).  The most important use of the checksum is to enforce the constraint that all the servers
talking to each other be running code compiled from the same source tree.  And that is clearly
important to HDFS, and won't be enforced with the current scheme.

Agree if this behavior was already in the Hadoop-2 code we don't _have_ to fix it here, but
if it would be a simple change, I would support fixing it here.  If not, please open a bug
for it.  Thanks.
> Hadoop Common creating package-info.java must not depend on sh, at least for Windows
> ------------------------------------------------------------------------------------
>                 Key: HADOOP-8924
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8924
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: build
>    Affects Versions: trunk-win
>            Reporter: Chris Nauroth
>            Assignee: Alejandro Abdelnur
>             Fix For: trunk-win
>         Attachments: HADOOP-8924.2.patch, HADOOP-8924.3.patch, HADOOP-8924.3.patch, HADOOP-8924.4.patch,
HADOOP-8924.5.patch, HADOOP-8924-branch-trunk-win.2.patch, HADOOP-8924-branch-trunk-win.3.patch,
HADOOP-8924-branch-trunk-win.4.patch, HADOOP-8924-branch-trunk-win.5.patch, HADOOP-8924-branch-trunk-win.patch,
> Currently, the build process relies on saveVersion.sh to generate package-info.java with
a version annotation.  The sh binary may not be available on all developers' machines (e.g.
Windows without Cygwin). This issue tracks removal of that dependency in Hadoop Common.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message