Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of pkalmegh@gmail.com designates
 209.85.161.176 as permitted sender)
MIME-Version: 1.0
From: Prajakta Kalmegh <pkalmegh@gmail.com>
Date: Fri, 6 Jul 2012 13:59:15 +0530
Message-ID: 
 <CAMBwUE7GxibCQ+AmysoHKKAPnOT7=x9tZR8n8VCWM1iQDoMwkQ@mail.gmail.com>
Subject: (Repost) Using REST to get ApplicationMaster info
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=e89a8f6430e07c67ea04c4251038

--e89a8f6430e07c67ea04c4251038
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Re-posting as I haven't got a solution yet. Sorry for spamming. I won't be
able to proceed in my code until I get a JSON response using AppMaster REST
URL. :(

Thanks,
Prajakta


On Wed, Jul 4, 2012 at 5:55 PM, Prajakta Kalmegh <pkalmegh@gmail.com> wrote=
:

> Hi Robert/Harsh
>
> Thanks for your reply.
>
> My RM is starting just fine. The problem is with the use of http://<proxy=
 httpddress:port>/proxy/{appid}/ws/v1/mapreduce
> to get the JSON response.
>
> As I said before, I had not configured the yarn.web-proxy.address propert=
y in yarn-site.xml. I assumed it will use the RM's yarn.resourcemanager.web=
app.address property value as default. However, it gives me a '404-Page not=
 found error'.  Today I tried specifying a value explicitly for the yarn.we=
b-proxy.address property.
>
> On running the wordcount example, it even gives a url <http://localhost:8=
090>/proxy/{appid}/> to track the App Mast info.    However, I am still not=
 able to get a json response.
>
> Also, I tried to get the data from historyserver instead of runtime using=
 the instructions given on page <http://hadoop.apache.org/common/docs/r2.0.=
0-alpha/hadoop-yarn/hadoop-yarn-site/HistoryServerRest.html>
>
> HistoryServer REST response does not give me jobids corresponding to an a=
pplication. It just lists all the jobs run until now. By the way, the docum=
entation does say
>
> ----------
>
> "Both of the following URI's give you the history server information, fro=
m an application id identified by the appid value.
>   * http://<history server http address:port>/ws/v1/history
>   * http://<history server http address:port>/ws/v1/history/info"
> ---------
>
> But there is no provision to specify the application id with these REST U=
RLs.
>
> Any idea how I can get the Application Master REST working and also linki=
ng jobids to application id using the HistoryServerREST API?
>
> Any help is appreciated. Thanks in advance.
> Regards,
> Prajakta
>
>
>
>
> On Fri, Jun 29, 2012 at 8:55 PM, Robert Evans <evans@yahoo-inc.com> wrote=
:
>
>> Please don't file that JIRA.  The proxy server is intended to front the
>> web server for all calls to the AM.  This is so you only have to go to a
>> single location to get to any AM's web service.  The proxy server is a
>> very simple proxy and just forwards the extra part of the path on to the
>> AM.
>>
>> If you are having issues with this please include the version you are
>> having problems with.  Also please look at the logs for the RM on startu=
p
>> to see if there is anything there indicating why it is not starting up.
>>
>> --Bobby Evans
>>
>> On 6/28/12 9:46 AM, "Harsh J" <harsh@cloudera.com> wrote:
>>
>> >As far as I can tell, the MR WebApp, as the name itself indicates on
>> >its doc page, starts only at the MR AM (which may be running at any
>> >NM), and it starts as an ephemeral port logged at in the AM logs
>> >usually as:
>> >
>> >INFO Web app /mapreduce started at [PORT]
>> >
>> >That it starts its own server with an ephemeral access point makes
>> >sense, since each job uses its own AM and having a common location may
>> >not work with the form of REST API documented at your link. Can you
>> >please file a JIRA to fix the doc and remove the proxy server refs,
>> >which are misleading?
>> >
>> >Do correct me if I'm wrong.
>> >
>> >On Thu, Jun 28, 2012 at 6:13 PM, Prajakta Kalmegh <pkalmegh@gmail.com>
>> >wrote:
>> >> Hi
>> >>
>> >> I am trying to get the ApplicationMaster info using the <http://<prox=
y
>> >>http
>> >> address:port>/proxy/{appid}/ws/v1/mapreduce/info> link as described o=
n
>> >>the <
>> >>
>> >>
>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yar=
n
>> >>-site/MapredAppMasterRest.html>
>> >> page.
>> >>
>> >> I am able to access and retrieve JSON response for other modules
>> >> (ResourceManager, NodeManager and HistoryServer). However, I am getti=
ng
>> >> 'Page not found' when I try to use my ResourceManager Http address to
>> >> access the ApplicationMaster info. I am using <
>> >> http://localhost:8088/proxy/{appid}/ws/v1/mapreduce/info> to retrieve
>> >>JSON
>> >> response.
>> >>
>> >> The instructions say "The application master should be accessed via t=
he
>> >> proxy. This proxy is configurable to run either on the resource manag=
er
>> >>or
>> >> on a separate host."
>> >>
>> >> My yarn-default.xml contains:
>> >>  <property>
>> >>    <description>The address for the web proxy as HOST:PORT, if this i=
s
>> >>not
>> >>     given then the proxy will run as part of the RM</description>
>> >>     <name>yarn.web-proxy.address</name>
>> >>     <value/>
>> >>  </property>
>> >>
>> >> and I did not set a value explicitly in yarn-site.xml.  Any idea how =
I
>> >>can
>> >> get this working? Thanks in advance.
>> >>
>> >> Regards,
>> >> Prajakta
>> >
>> >
>> >
>> >--
>> >Harsh J
>>
>>
>

--e89a8f6430e07c67ea04c4251038--