Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A46F0E050 for ; Wed, 29 May 2013 17:30:57 +0000 (UTC) Received: (qmail 72629 invoked by uid 500); 29 May 2013 17:30:52 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 72484 invoked by uid 500); 29 May 2013 17:30:52 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 72477 invoked by uid 99); 29 May 2013 17:30:52 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 May 2013 17:30:52 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rahul.rec.dgp@gmail.com designates 209.85.212.42 as permitted sender) Received: from [209.85.212.42] (HELO mail-vb0-f42.google.com) (209.85.212.42) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 May 2013 17:30:45 +0000 Received: by mail-vb0-f42.google.com with SMTP id w15so6158518vbf.29 for ; Wed, 29 May 2013 10:30:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=8bNyyLbThTmv9oZ2MHs4twXbV/hyvsC/TrgGPWGaz4E=; b=i90K9t8iqOD5dZ+ABKxdKW1MGDqOQ8o4Ujj1ft/7XppNxv/CnXG4YGwNqRirgeXuNy 803TZiiDd17A2d4if36FiJfRlyDzJtTC30Ls2KLt+A1r1N4nFWUyuHfLWUROo5k6dLAs YC6oerJQqjIrbOYMJS8PScQkVfY17lU5wax5SpwH8T326/PkkHsw9Y1pNjeLULV/NBH6 8+X88pEnGtQr2tthAHWPktr5+05hcAUHTCPYn/Ugjnlozu5WskEHzpIcu9B8Om/XlrfH 1klVaCX/meIupaZwYe69YXsRH0TZ74qPgU5MqLQCU1FtkWlYVjys0EfuOgEt56rZGLvz rWdA== X-Received: by 10.58.116.229 with SMTP id jz5mr2338293veb.14.1369848624567; Wed, 29 May 2013 10:30:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.59.6.68 with HTTP; Wed, 29 May 2013 10:30:04 -0700 (PDT) In-Reply-To: References: From: Rahul Bhattacharjee Date: Wed, 29 May 2013 23:00:04 +0530 Message-ID: Subject: Re: What else can be built on top of YARN. To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=047d7b5d5f66a9dd1004dddebca0 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5d5f66a9dd1004dddebca0 Content-Type: text/plain; charset=UTF-8 Thanks for the response Krishna. I was wondering if it were possible for using MR to solve you problem instead of building the whole stack on top of yarn. Most likely its not possible , thats why you are building it . I wanted to know why is that ? I am in just trying to find out the need or why we might need to write the application on yarn. Rahul On Wed, May 29, 2013 at 8:23 PM, Krishna Kishore Bonagiri < write2kishore@gmail.com> wrote: > Hi Rahul, > > I am porting a distributed application that runs on a fixed set of given > resources to YARN, with the aim of being able to run it on a dynamically > selected resources whichever are available at the time of running the > application. > > Thanks, > Kishore > > > On Wed, May 29, 2013 at 8:04 PM, Rahul Bhattacharjee < > rahul.rec.dgp@gmail.com> wrote: > >> Hi all, >> >> I was going through the motivation behind Yarn. Splitting the >> responsibility of JT is the major concern.Ultimately the base (Yarn) was >> built in a generic way for building other generic distributed applications >> too. >> >> I am not able to think of any other parallel processing use case that >> would be useful to built on top of YARN. I though of a lot of use cases >> that would be beneficial when run in parallel , but again ,we can do those >> using map only jobs in MR. >> >> Can someone tell me a scenario , where a application can utilize Yarn >> features or can be built on top of YARN and at the same time , it cannot be >> done efficiently using MRv2 jobs. >> >> thanks, >> Rahul >> >> >> > --047d7b5d5f66a9dd1004dddebca0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for the response Krishna.

I was wondering if it were possible for using MR to=C2=A0 solve you probl= em instead of building the whole stack on top of yarn.
Most likely its not possible , thats why you are building it .= I wanted to know why is that ?

I am in just trying to find out the need or why we might need to write the = application on yarn.

Rahul


On Wed, May 29, 2013 at 8:23 PM, Krishna= Kishore Bonagiri <write2kishore@gmail.com> wrote:
=
Hi Rahul,

=C2=A0 I am porting a distributed a= pplication that runs on a fixed set of given resources to YARN, with the ai= m of =C2=A0being able to run it on a dynamically selected resources whichev= er are available at the time of running the application.

Thanks,
Kishore


On Wed, May 29, 2013 at 8:04 PM, Rahul Bhattacharjee <rahul.rec.dgp@gmail.com> wrote:
Hi all,

I was going through the motivation behind Yarn. Splitting the responsibilit= y of JT is the major concern.Ultimately the base (Yarn) was built in a gene= ric way for building other generic distributed applications too.

= I am not able to think of any other parallel processing use case that would= be useful to built on top of YARN. I though of a lot of use cases that wou= ld be beneficial when run in parallel , but again ,we can do those using ma= p only jobs in MR.

= Can someone tell me a scenario , where a application can utilize Yarn featu= res or can be built on top of YARN and at the same time , it cannot be done= efficiently using MRv2 jobs.

= thanks,
Rahul




--047d7b5d5f66a9dd1004dddebca0--