Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Tue, 10 May 2011 05:52:09 +0000 (UTC)
From: "Vinod Kumar Vavilapalli (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <1413575092.33967.1305006729285.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <51228582.33465.1304981703138.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-2480) MR-279: mr app should not
 depend on hard-coded version of shuffle
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031033#comment-13031033 ] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-2480:
----------------------------------------------------

bq. However a bigger issue is whether app (a user land module) should depend on a specific implementation shuffle (loaded into node manager). I think it should not. The MAPREDUCE_SERVICE_ID should probably resides in a MRServices class in mr common. I have the impression (based on previous conversations) that Chris was gonna refactor the NM service stuff later. It's probably OK as is now.

No, that isn't quite correct. An app really SHOULD depend on a specific implementation of shuffle. An app is tied to a specific version of shuffle be it the request format or the authentication data. Shuffle Service isn't really 'known' to the NM, it is an abstract service started by NM.

> MR-279: mr app should not depend on hard-coded version of shuffle
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-2480
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2480
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>             Fix For: 0.23.0
>
>         Attachments: mr-2480-shuffle-version-v1.patch
>
>
> The following commit introduced a dependency of shuffle with hard-coded version for mr app:
> {noformat}
> commit 6f69742140516be7493c9a9177b81d0516cc9539
> Author: Vinod Kumar Vavilapalli <vinodkv@apache.org>
> Date:   Wed May 4 06:53:52 2011 +0000
>     Adding user log handling for YARN. Making NM put the user-logs on DFS and providing log-dump tools. Contributed by Vinod Kumar Vavilapalli.
> {noformat}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira