hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahab Yunus <shahab.yu...@gmail.com>
Subject Re: calling mapreduce from webservice
Date Fri, 18 Apr 2014 23:07:26 GMT
As far as I know there is no API to kick of M/R jobs. There is for M/R v2,
a REST API to get status of jobs:
http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-site/MapredAppMasterRest.html#Mapreduce_Application_Master_Info_API

I would say that you have invoke M/R jobs in your middle tier or back-end,
you have to implement a custom solution i.e. invoking the M/R jobs in
standard way and then monitoring the status of the job and then update the
UI asynchronously depending on which UI framework or web service
implementation (e.g. WS-Addressing) you are using.

Regards,
Shahab


On Fri, Apr 18, 2014 at 3:11 PM, girish hilage <girish_hilage@yahoo.com>wrote:

> Yes. I intend to run the jobs asynchronously and show the status of the
> user submitted job as "running/completed" etc. and user will be able to
> submit new jobs simultaneously.  I have not checked PigLipStick though.
>
> Regards,
> Girish
>
>   On Saturday, April 19, 2014 12:34 AM, Shahab Yunus <
> shahab.yunus@gmail.com> wrote:
>  Question: M/R jobs are supposed to run for a long time. They are
> essentially batch processes. Do you plan to keep the Web UI blocked for
> that while? Or are you looking for asynchronous invocation of the M/R job?
> Or are you thinking about building sort of an Admin UI (e.g. PigLipstick)
> What exactly is your requirement?
>
> Regards,
> Shahab
>
>
> On Fri, Apr 18, 2014 at 3:01 PM, girish hilage <girish_hilage@yahoo.com>wrote:
>
> Hi,
>
>    This is just to check with you, if it is possible to call MR jobs from
> Java Webservices.
>    If yes, then could you please help me by pointing to some resouces/docs.
>
>    Actually, what I intend to do is create a Web UI with some
> functionality which would call MR jobs and present the result to the user
> in browser.
>
> Regards,
> Girish
>
>
>
>
>

Mime
View raw message