Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7130210F4B for ; Tue, 2 Jul 2013 09:22:29 +0000 (UTC) Received: (qmail 17783 invoked by uid 500); 2 Jul 2013 09:22:24 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 17230 invoked by uid 500); 2 Jul 2013 09:22:17 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 17223 invoked by uid 99); 2 Jul 2013 09:22:16 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Jul 2013 09:22:16 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of samliuhadoop@gmail.com designates 74.125.83.51 as permitted sender) Received: from [74.125.83.51] (HELO mail-ee0-f51.google.com) (74.125.83.51) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Jul 2013 09:22:11 +0000 Received: by mail-ee0-f51.google.com with SMTP id e52so2612130eek.10 for ; Tue, 02 Jul 2013 02:21:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=cXO7aea5q20RioBh85RGMavBb1aGJ4cW0nm5Strh3hE=; b=YpRd7hkzd0jgyd40PqvDteQV+jdoIou/0H/8JOyDhX+K/oIWQpb4NmfQMaMeUk5+3V /j7MWLxTpqHfEF0w/vX/qvxo7cQZ0WqDlzL14uTBcH01jYQZuMCa/+DKa9tcENzm8+jW HCH2tR4nKFvlbuLTlVTd9kxt/j8h5fpVEo870cFdEFLCZPOV8SCKSVEOZ1E4ctc91FZ8 iuuomN2XkQAKTVVqoVqjiGl4jVxNiApvU7CplR+COWR4B3xaH8AezDCSRqd5twZElTOl l/PH+fV3AukX6WPt1w96EfX/zFXb1bfk+TopbY9ONhXMU7sel6TP2ZQMRyiResASR+3K grkQ== MIME-Version: 1.0 X-Received: by 10.14.38.14 with SMTP id z14mr25445881eea.49.1372756910825; Tue, 02 Jul 2013 02:21:50 -0700 (PDT) Received: by 10.14.223.137 with HTTP; Tue, 2 Jul 2013 02:21:50 -0700 (PDT) In-Reply-To: References: Date: Tue, 2 Jul 2013 17:21:50 +0800 Message-ID: Subject: Re: How to write/run MPI program on Yarn? From: sam liu To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=001a11c3d2ee0884d004e083e007 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c3d2ee0884d004e083e007 Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: quoted-printable Any one could help answer above questions? Thanks a lot! 2013/7/1 sam liu > Thanks Pramod and Clark! > > 1. What's the relationship of Hadoop 2.x branch and mpich2-yarn project? > 2. Does Hadoop 2.x branch plan to include MPI implementation? I mentioned > there is already a JIRA: > https://issues.apache.org/jira/browse/MAPREDUCE-2911 > > > 2013/7/1 Clark Yang (=D1=EE=D7=BF=DC=FD) > > Hi, sam >> >> Please try it following the README.md on >> https://github.com/clarkyzl/mpich2-yarn/. >> The design is simple ,make each nodemanger runs a "smpd" daemon and >> make application master run "mpiexec" to submit mpi jobs. The biggest >> problem here, I think, is the yarn scheduler do not support gang >> secheduling algorithm, and the scheduling is very very slow. >> >> Any ideas please feel free to share. >> >> Cheers, >> Zhuoluo (Clark) Yang >> >> >> 2013/6/30 Pramod N >> >>> Hi Sam, >>> This might interest you. >>> https://github.com/clarkyzl/mpich2-yarn >>> >>> >>> >>> Pramod N >>> Bruce Wayne of web >>> @machinelearner >>> >>> -- >>> >>> >>> On Sun, Jun 30, 2013 at 6:02 PM, sam liu wrote= : >>> >>>> Hi Experts, >>>> >>>> Does Hadoop 2.0.5-alpha supports MPI programs? >>>> >>>> - If yes, is there any example of writing a MPI program on Yarn? How t= o >>>> write the client side code to configure and submit the MPI job? >>>> - If no, which Hadoop version will support MPI programming? >>>> >>>> Thanks! >>>> >>> >>> >> > --001a11c3d2ee0884d004e083e007 Content-Type: text/html; charset=GB2312 Content-Transfer-Encoding: quoted-printable
Any one could help answer above questions? Thanks a lot!


2013/= 7/1 sam liu <samliuhadoop@gmail.com>
Thanks Pramod and Clar= k!

1. What's the relationship of Hadoop 2.x branch and mpi= ch2-yarn project? 
2. Does Hadoop 2.x branch plan to include MPI implementation? I mentioned t= here is already a JIRA: https://issues.apache.org/jira/browse/MAPRE= DUCE-2911


2013/7/= 1 Clark Yang (=D1=EE=D7=BF=DC=FD) <yangzhuoluo@gmail.com>

Hi, sam

   Please try it following the README= .md on https://github.com/clarkyzl/mpich2-yarn/.
  The design is simple ,make each nodemanger runs a "smpd" d= aemon and make application master run "mpiexec" to submit mpi job= s. The biggest problem here, I think, is the yarn scheduler do not support = gang secheduling algorithm, and the scheduling is very very slow.

  Any ideas please feel free to share.

Cheers,
Zhuoluo (Clark) Yang


2013/6/30 Pramod N <= ;npramod05@gmail.c= om>
Hi Sam,
This might interest you.
https://gith= ub.com/clarkyzl/mpich2-yarn



Pramod N
Bruce Wayne of web
@mac= hinelearner

--


On Sun, Jun 30, 2013 at 6:02 PM, sam liu= <samliuhadoop@gmail.com> wrote:
Hi Experts,

Does Hadoop 2.0.5-alpha= supports MPI programs?

- If yes, is there any example of writ= ing a MPI program on Yarn? How to write the client side code to configure a= nd submit the MPI job?
- If no, which Hadoop version will support MPI programming?
Thanks!




--001a11c3d2ee0884d004e083e007--