Mailing-List: contact users-help@apex.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@apex.apache.org
MIME-Version: 1.0
From: Sunil Parmar <sunilosunil@gmail.com>
Date: Tue, 26 Sep 2017 11:53:11 -0700
Message-ID: <CAHdkLAMXQOBqqC9_Ctmj3yN9qXzj2Y5jU-JTkTPM-E8+EWKG3w@mail.gmail.com>
Subject: how to find application master URL ( non proxy )
To: users@apex.apache.org
Content-Type: multipart/alternative; boundary="94eb2c1fc8a40bfb81055a1c31ec"
archived-at: Tue, 26 Sep 2017 18:53:36 -0000

--94eb2c1fc8a40bfb81055a1c31ec
Content-Type: text/plain; charset="UTF-8"

We're using plain vanilla version of apex in our environment ( no
datatorrent or dtgateway ). We're using STRAM API to access the application
details. We are able to find the STRAM endpoints from YARN application API.

http://<RM>:8088/ws/v1/cluster/apps?state=RUNNING

The tracking URL property points to STRAM service but it's exposed by YARN
web proxy. i.e.

http://<RM>:8088/proxy/application_1505865155823_0039/ws/v2/stram/
http://
<RM>:8088/proxy/application_1505865155823_0039/ws/v2/stram/logicalPlan
http://
<RM>:8088/proxy/application_1505865155823_0039/ws/v2/stram/physicalPlan
http://<RM>:8088/proxy/application_1505865155823_0039/ws/v2/stram/loggers

It works for monitoring but the limitation here though is that it only
allows users to submit "GET" type requests on STRAM api. So it doesn't
allow to run POST API  i.e. to change log level.

How to find direct application master tracking URL  ( no proxy ) ? Is there
a YARN api / STRAM api to do so ? So far I have found a brute force way to
scrap AM log to find it but I wanted to check with community experts if
there is a better way.

Thanks,
Sunil Parmar

--94eb2c1fc8a40bfb81055a1c31ec
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div>We&#39;re using plain vanilla version of apex in=
 our environment ( no datatorrent or dtgateway ). We&#39;re using STRAM API=
 to access the application details. We are able to find the STRAM endpoints=
 from YARN application API.<br><br>http://&lt;RM&gt;:8088/ws/v1/cluster/app=
s?state=3DRUNNING<br><br></div>The tracking URL property points to STRAM se=
rvice but it&#39;s exposed by YARN web proxy. i.e.</div><div><br></div><div=
><span style=3D"color:rgb(80,80,80);font-family:OpenSans,Helvetica,Arial,sa=
ns-serif;font-size:12px;font-style:normal;font-variant-ligatures:normal;fon=
t-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:l=
eft;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0=
px;background-color:rgb(250,250,250);text-decoration-style:initial;text-dec=
oration-color:initial;display:inline;float:none">http://&lt;RM&gt;:8088/pro=
xy/application_1505865155823_0039/ws/v2/stram/</span></div><div><span style=
=3D"color:rgb(80,80,80);font-family:OpenSans,Helvetica,Arial,sans-serif;fon=
t-size:12px;font-style:normal;font-variant-ligatures:normal;font-variant-ca=
ps:normal;font-weight:normal;letter-spacing:normal;text-align:left;text-ind=
ent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;backgroun=
d-color:rgb(250,250,250);text-decoration-style:initial;text-decoration-colo=
r:initial;display:inline;float:none">http://&lt;RM&gt;:8088/proxy/applicati=
on_1505865155823_0039/ws/v2/stram/logicalPlan</span><br><div><span style=3D=
"color:rgb(80,80,80);font-family:OpenSans,Helvetica,Arial,sans-serif;font-s=
ize:12px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:=
normal;font-weight:normal;letter-spacing:normal;text-align:left;text-indent=
:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-c=
olor:rgb(250,250,250);text-decoration-style:initial;text-decoration-color:i=
nitial;display:inline;float:none">http://&lt;RM&gt;:8088/proxy/application_=
1505865155823_0039/ws/v2/stram/physicalPlan</span></div><span style=3D"colo=
r:rgb(80,80,80);font-family:OpenSans,Helvetica,Arial,sans-serif;font-size:1=
2px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:norma=
l;font-weight:normal;letter-spacing:normal;text-align:left;text-indent:0px;=
text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:=
rgb(250,250,250);text-decoration-style:initial;text-decoration-color:initia=
l;display:inline;float:none"></span></div><span style=3D"color:rgb(80,80,80=
);font-family:OpenSans,Helvetica,Arial,sans-serif;font-size:12px;font-style=
:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:=
normal;letter-spacing:normal;text-align:left;text-indent:0px;text-transform=
:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(250,250,25=
0);text-decoration-style:initial;text-decoration-color:initial;display:inli=
ne;float:none">http://&lt;RM&gt;:8088/proxy/application_1505865155823_0039/=
ws/v2/stram/loggers</span><div><div><br><div>It works for monitoring but th=
e limitation here though is that it only allows users to submit &quot;GET&q=
uot; type requests on STRAM api. So it doesn&#39;t allow to run POST API=C2=
=A0 i.e. to change log level. <br></div><div><br></div><div>How to find dir=
ect application master tracking URL=C2=A0 ( no proxy ) ? Is there a YARN ap=
i / STRAM api to do so ? So far I have found a brute force way to scrap AM =
log to find it but I wanted to check with community experts if there is a b=
etter way.<br></div><div><br></div><div>Thanks,<br clear=3D"all"><div><div =
class=3D"gmail_signature">Sunil Parmar</div></div>
</div></div></div></div>

--94eb2c1fc8a40bfb81055a1c31ec--