Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7094D10676 for ; Tue, 24 Sep 2013 05:40:43 +0000 (UTC) Received: (qmail 6050 invoked by uid 500); 24 Sep 2013 05:40:31 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 5757 invoked by uid 500); 24 Sep 2013 05:40:28 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 5748 invoked by uid 99); 24 Sep 2013 05:40:27 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Sep 2013 05:40:27 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of nitin.motgi@gmail.com designates 209.85.216.173 as permitted sender) Received: from [209.85.216.173] (HELO mail-qc0-f173.google.com) (209.85.216.173) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Sep 2013 05:40:20 +0000 Received: by mail-qc0-f173.google.com with SMTP id c3so2761248qcv.4 for ; Mon, 23 Sep 2013 22:39:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:in-reply-to:references:from:to:cc :subject:content-type; bh=Or47z/pvb95Z7bOo1n8pGLvw/pJno3jhMo41/hb8k50=; b=iyKKs4wEW84JNtDpGY9nobfE0RgiaW+zViO4KjVn2RNBMxLnO2brrHncIBc9/Bhh9C qFZYTDZBgTKip9uz91RnOgU0xIVK8r0bCSVnMdPX6cs6t3nvefEnQmrEqupAqd+taKaN o0JF90/MDktnXxMZMnHLzpJORDOseaulLSqhagVgAi0Bv6yH5j6UDOhWyBY6xv4y031r zTpOM+bZzTvhKRhQ7Z+fLV+vFq8pxIPcKZgalbKML7mKosOlmvEweHObw94IfY9kLYfr 4pd7r95x4UxbtzfqjEc28MHsiEQ5I33TWsY1x86KX7NwZBnlYJw+z6uF6VBrqZllCl9e OIHw== X-Received: by 10.224.13.130 with SMTP id c2mr26958088qaa.69.1380001199270; Mon, 23 Sep 2013 22:39:59 -0700 (PDT) Received: from [127.0.0.1] (ec2-54-235-159-155.compute-1.amazonaws.com. [54.235.159.155]) by mx.google.com with ESMTPSA id b9sm51331678qad.1.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 23 Sep 2013 22:39:58 -0700 (PDT) MIME-Version: 1.0 X-Mailer: Nodemailer (0.5.0; +http://www.nodemailer.com/) Date: Mon, 23 Sep 2013 22:39:58 -0700 (PDT) Message-Id: <1380001198237.f544614d@Nodemailer> In-Reply-To: References: X-Orchestra-Oid: 79609E54-0F43-4DF4-8290-72D17868505B X-Orchestra-Sig: 4461aae63391997506534c901678509cf21a61e2 X-Orchestra-Thrid: TF13358A5-5035-4E32-9F57-5C10AC4D63D9_1446721911315670303 X-Orchestra-Thrid-Sig: b1e6319b48dda4e02a6d7b0fdfcd34576576edbd X-Orchestra-Account: 32e4213e26a9230502de341da398cfef4f09fc11 From: "Personal" To: user@hadoop.apache.org Cc: user@hadoop.apache.org Subject: Re: Help writing a YARN application Content-Type: multipart/alternative; boundary="----Nodemailer-0.5.0-?=_1-1380001198653" X-Virus-Checked: Checked by ClamAV on apache.org ------Nodemailer-0.5.0-?=_1-1380001198653 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Pradeep, You can also look at Weave - A simpler thread abstraction over YARN.=C2=A0 http://github.com/continuuity/weave Runs on multiple HDs. Nitin On Mon, Sep 23, 2013 at 10:20 PM, Pradeep Gollakota wrote: > Hi Arun, > Thanks so much for your reply. I looked at the app you linked, it's way > easier to understand. Unfortunately, I can't use YarnClient class. It > doesn't seem to be in the 2.0.0 branch. Our production environment is = using > cdh-4.x and it looks like they're tracking the 2.0.x branch. > However, I have looked at other implementations (notably Kitten and = Giraph) > for inspiration and able to get my containers successfully launching. > Thanks for the help! > - Pradeep > On Mon, Sep 23, 2013 at 4:46 PM, Arun C Murthy = wrote: >> I looked at your code, it's using really old apis/protocols which are >> significantly different now. See hadoop-2.1.0-beta release for latest >> apis/protocols. >> >> Also, you should really be using yarn client module rather than raw >> protocols. >> >> See https://github.com/hortonworks/simple-yarn-app for a simple example.= >> >> thanks, >> Arun >> >> On Sep 20, 2013, at 11:24 AM, Pradeep Gollakota >> wrote: >> >> Hi All, >> >> I've been trying to write a Yarn application and I'm completely lost. = I'm >> using Hadoop 2.0.0-cdh4.4.0 (Cloudera distribution). I've uploaded my >> sample code to github at https://github.com/pradeepg26/sample-yarn >> >> The problem is that my application master is exiting with a status of 1 >> (I'm expecting that since my code isn't complete yet). But I have no = logs >> that I can examine. So I'm not sure if the error I'm getting is the = error >> I'm expecting. I've attached the nodemanger and resourcemanager logs = for >> your reference as well. >> >> How can I get started on writing YARN applications beyond the initial >> tutorial=3F >> >> Thanks for any help/pointers! >> >> Pradeep >> >> >> >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or = entity >> to which it is addressed and may contain information that is = confidential, >> privileged and exempt from disclosure under applicable law. If the = reader >> of this message is not the intended recipient, you are hereby notified = that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender = immediately >> and delete it from your system. Thank You. ------Nodemailer-0.5.0-?=_1-1380001198653 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi Pradeep,

You can also look at Weave - A simpler thread abstraction over YARN.= =C2=A0

http://github.com/continuuity/weave

Runs on multiple HDs.

Nitin


On Mon, Sep 23, 2013 at 10:20 = PM, Pradeep Gollakota <pradeepg26@gmail.= com> wrote:

Hi Arun,

Thanks so much for your reply. I looked at the app you linked, it's = way easier to understand. Unfortunately, I can't use YarnClient class. It = doesn't seem to be in the 2.0.0 branch. Our production environment is using= cdh-4.x and it looks like they're tracking the 2.0.x branch.

However, I have looked at other implementations (notably Kitten and = Giraph) for inspiration and able to get my containers successfully = launching.

Thanks for the help!
- Pradeep


On Mon, Sep 23, 2013 at 4:46 PM, = Arun C Murthy <acm@hortonworks.com> = wrote:
I looked at your code, it's using = really old apis/protocols which are significantly different now. See = hadoop-2.1.0-beta release for latest apis/protocols.

Also, you should really be using yarn client module rather than raw = protocols.


thanks,
Arun

Hi All,=

<= /div>
I've been trying to write a Yarn application and I'm completely lost. I'm = using Hadoop 2.0.0-cdh4.4.0 (Cloudera distribution). I've uploaded my = sample code to github at=C2=A0https://github.= com/pradeepg26/sample-yarn

The problem = is that my application master is exiting with a status of 1 (I'm expecting = that since my code isn't complete yet). But I have no logs that I can = examine. So I'm not sure if the error I'm getting is the error I'm = expecting. I've attached the nodemanger and resourcemanager logs for your = reference as well.

How can I = get started on writing YARN applications beyond the initial = tutorial=3F

Thanks for = any help/pointers!

Pradeep
<yarn-yarn-nodemanager-pradeep-gollakota.vm.lithium.com.= log><yarn-yarn-resourcemanager-pradeep-gollakota.vm.= lithium.com.log>

--
Arun C. Murthy


CONFIDENTIALITY NOTICE
NOTICE: This message is= intended for the use of the individual or entity to which it is addressed = and may contain information that is confidential, privileged and exempt = from disclosure under applicable law. If the reader of this message is not = the intended recipient, you are hereby notified that any printing, copying,= dissemination, distribution, disclosure or forwarding of this = communication is strictly prohibited. If you have received this = communication in error, please contact the sender immediately and delete it= from your system. Thank You.


------Nodemailer-0.5.0-?=_1-1380001198653--