Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 05D6511133 for ; Mon, 21 Apr 2014 08:35:59 +0000 (UTC) Received: (qmail 71417 invoked by uid 500); 21 Apr 2014 08:35:51 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 71317 invoked by uid 500); 21 Apr 2014 08:35:51 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 71309 invoked by uid 99); 21 Apr 2014 08:35:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Apr 2014 08:35:50 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of girish_hilage@yahoo.com designates 98.138.91.189 as permitted sender) Received: from [98.138.91.189] (HELO nm29-vm4.bullet.mail.ne1.yahoo.com) (98.138.91.189) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Apr 2014 08:35:42 +0000 Received: from [98.138.101.129] by nm29.bullet.mail.ne1.yahoo.com with NNFMP; 21 Apr 2014 08:35:21 -0000 Received: from [98.138.89.232] by tm17.bullet.mail.ne1.yahoo.com with NNFMP; 21 Apr 2014 08:35:21 -0000 Received: from [127.0.0.1] by omp1047.mail.ne1.yahoo.com with NNFMP; 21 Apr 2014 08:35:21 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 662414.57666.bm@omp1047.mail.ne1.yahoo.com Received: (qmail 42544 invoked by uid 60001); 21 Apr 2014 08:35:21 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1398069321; bh=6Sxx6egyVTIJaD+wh18A2CaRKWVZOKeB/1jnO9sHF8k=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=eE0VFum+Krw0EfFpTTxdXP4NiBLDP1f+qulXFxBv5/szA3U4s5j0w5V2QQx5H5v4hd941Q8lpXCjX0Uq6fk5/Dc9/0gw4EEl9mWI25g2vNvvBtAL+bHMPA/pLmBEkkyUwG0MAvD0TxE6odrkttyT1l18nTg22Yuz3M+EtJ/96Io= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=g1af6bbIkWJ1W2Ckk0PjsPehI5m/FbkmVky70jXT4kPh4+dHjSYSzat45kX8UZiz+mqfSwziHkt3FQkTC1PKIYu/3FQv0nKzA+BiPTjsSpQZuPg3hLOE4vY6cefObVtmRxnbNJLGcEYHqyR0bCWfWpUBpbWhfF5lMUSC70GDn80=; X-YMail-OSG: gN1AuhYVM1m4Kb2XHEpx_kpJ7ZfrVXsxgRuPoSgS_TXI6j1 KjZ5_TpzacEwABW4pvF71cSWCeNY8AwC48L1V0szOACAFcTZWWtb7dIwjzOn unftwFQ6.SPQB3nzIrhcS4rsfxukH9kNfAwWtiimned.0fqRfHXuDpADtqDF G2TrM2bDwEn3GvEAKzkib4w.qK4TuPxfvW02sWUX_Pe0.FE9DHhGOWt5RCvW 7dCznDF6GLQYeu2Scf3S2TzQmce3n3StI9ig1gANj5cLiPQgL7G.jETXNyxa 3xC.7tcvHMl0EREFkK9VTBDPlePVLPOYxCcAgsivHtRt1Vo_4IhnPyg0bi55 EzoyrW8i.qYt0jpWNx7RRwMaKfNggqU8fl3sCVS_NmGzmIi3Yvk4TBR.zGS_ Rv84myNO3Vy2XPfMcwmSkN4csW70Yi1JAG4U3kRbUrTpSd19VcAoDB3CoW_1 K1CmJs7xB0vWRjN4HIF2qNYRbXXgpFEjG4R5P2yz0ixu4_Grwt2FA3Kode8E dOki7WcyawBw1tfSo63tDtcfkVcXs0xfg5LNMLbHxmO.t_6WbmxxPjMM- Received: from [182.72.249.26] by web122303.mail.ne1.yahoo.com via HTTP; Mon, 21 Apr 2014 01:35:21 PDT X-Rocket-MIMEInfo: 002.001,VGhhbmtzLgpJIHdpbGwgY2hlY2sgaWYgSSBjYW4gdXNlIFBsYXkuCsKgCi0gR2lyaXNoCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KRnJvbTogTW9oYW4gUmFkaGFrcmlzaG5hbiA8cmFkaGFrcmlzaG5hbi5tb2hhbkBnbWFpbC5jb20.ClRvOiB1c2VyQGhhZG9vcC5hcGFjaGUub3JnIApTZW50OiBTYXR1cmRheSwgQXByaWwgMTksIDIwMTQgNzoyNyBBTQpTdWJqZWN0OiBSZTogY2FsbGluZyBtYXByZWR1Y2UgZnJvbSB3ZWJzZXJ2aWNlCgoKClBsYXkgZnJhbWV3b3JrIGlzIHJlYWN0aXZlIGEBMAEBAQE- X-Mailer: YahooMailWebService/0.8.185.657 References: <1397847692.29705.YahooMailNeo@web122302.mail.ne1.yahoo.com> <1397848278.88952.YahooMailNeo@web122301.mail.ne1.yahoo.com> Message-ID: <1398069321.12322.YahooMailNeo@web122303.mail.ne1.yahoo.com> Date: Mon, 21 Apr 2014 01:35:21 -0700 (PDT) From: girish hilage Reply-To: girish hilage Subject: Re: calling mapreduce from webservice To: "user@hadoop.apache.org" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-1062763926-1563233386-1398069321=:12322" X-Virus-Checked: Checked by ClamAV on apache.org ---1062763926-1563233386-1398069321=:12322 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Thanks.=0AI will check if I can use Play.=0A=A0=0A- Girish=0A=0A=0A________= ________________________=0AFrom: Mohan Radhakrishnan =0ATo: user@hadoop.apache.org =0ASent: Saturday, April 19, 2014 7:= 27 AM=0ASubject: Re: calling mapreduce from webservice=0A=0A=0A=0APlay fram= ework is reactive and uses push channels. It may be useful here if the UI h= as to be asynchronous and reactive. =0A=0AMohan=0A=0A=0A=0AOn Sat, Apr 19, = 2014 at 4:37 AM, Shahab Yunus wrote:=0A=0AAs far a= s I know there is no API to kick of M/R jobs. There is for M/R v2, a REST A= PI to get status of jobs:=A0http://hadoop.apache.org/docs/r2.3.0/hadoop-yar= n/hadoop-yarn-site/MapredAppMasterRest.html#Mapreduce_Application_Master_In= fo_API =0A>=0A>=0A>I would say that you have invoke M/R jobs in your middle= tier or back-end, you have to implement a custom solution i.e. invoking th= e M/R jobs in standard way and then monitoring the status of the job and th= en update the UI asynchronously depending on which UI framework or web serv= ice implementation (e.g. WS-Addressing) you are using.=0A>=0A>=0A>Regards,= =0A>Shahab=0A>=0A>=0A>=0A>On Fri, Apr 18, 2014 at 3:11 PM, girish hilage wrote:=0A>=0A>Yes. I intend to run the jobs asynchr= onously and show the status of the user submitted job as "running/completed= " etc. and user will be able to submit new jobs simultaneously.=A0 I have n= ot checked PigLipStick though.=0A>>=0A>>Regards,=0A>>Girish=0A>>=0A>>=0A>>= =0A>>On Saturday, April 19, 2014 12:34 AM, Shahab Yunus wrote:=0A>>=0A>>Question: M/R jobs are supposed to run for a long tim= e. They are essentially batch processes. Do you plan to keep the Web UI blo= cked for that while? Or are you looking for asynchronous invocation of the = M/R job? Or are you thinking about building sort of an Admin UI (e.g. PigLi= pstick) What exactly is your requirement? =0A>>=0A>>=0A>>Regards,=0A>>Shaha= b=0A>>=0A>>=0A>>=0A>>On Fri, Apr 18, 2014 at 3:01 PM, girish hilage wrote:=0A>>=0A>>Hi,=0A>>>=0A>>>=A0=A0 This is just to ch= eck with you, if it is possible to call MR jobs from Java Webservices.=0A>>= >=A0=A0 If yes, then could you please help me by pointing to some resouces/= docs.=0A>>>=0A>>>=A0=A0 Actually, what I intend to do is create a Web UI wi= th some functionality which would call MR jobs and present the result to th= e user in browser.=0A>>>=0A>>>=0A>>>=0A>>>Regards,=0A>>>Girish=0A>>>=0A>>= =0A>>=0A>>=0A> ---1062763926-1563233386-1398069321=:12322 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks= .
I will check if I ca= n use Play.
 
- Girish

From: Mohan Radhakrishnan <radhakrishn= an.mohan@gmail.com>
To: user@hadoop.apache.org
Sent: Saturday, April 19, 2014 7:27 AM
Subject: Re: calling mapreduce from webservice
=

shahab.yunus@gmail.com> wrote:
As far as I know there is no API to kick of M/R jobs. There = is for M/R v2, a REST API to get status of jobs: http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-= site/MapredAppMasterRest.html#Mapreduce_Application_Master_Info_API=20

I would say that you have invoke M/R jobs in your middle tier or back-= end, you have to implement a custom solution i.e. invoking the M/R jobs in = standard way and then monitoring the status of the job and then update the = UI asynchronously depending on which UI framework or web service implementa= tion (e.g. WS-Addressing) you are using.

Regards,
Shahab


On Fri, Apr 18, 2014 at 3:11 PM, giri= sh hilage <girish_hilage@yahoo.com> wrote:
Yes. I intend to run the jobs = asynchronously and show the status of the user submitted job as "running/co= mpleted" etc. and user will be able to submit new jobs simultaneously. = ; I have not checked PigLipStick though.

R= egards,
Girish

On Saturday, April 19, 2014 12:34 AM, Sha= hab Yunus <shahab.y= unus@gmail.com> wrote:
Question: M/R jobs are supposed to run for a long time. They= are essentially batch processes. Do you plan to keep the Web UI blocked fo= r that while? Or are you looking for asynchronous invocation of the M/R job= ? Or are you thinking about building sort of an Admin UI (e.g. PigLipstick)= What exactly is your requirement?=20

Regards,
Shahab


On Fri, Apr 18, 2014 at 3:01 PM, girish hilage <girish_hilage@yahoo.co= m> wrote:
Hi,

   This is just to check with you, if it is possible to c= all MR jobs from Java Webservices.
   If yes, the= n could you please help me by pointing to some resouces/docs.

   Actually, what I intend to do is create a = Web UI with some functionality which would call MR jobs and present the res= ult to the user in browser.

Regards,
Girish
=







---1062763926-1563233386-1398069321=:12322--