hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Kelly <jonathaka...@gmail.com>
Subject Re: [Erorr:]vieiwng Web UI on EMR cluster
Date Tue, 13 Sep 2016 20:15:52 GMT
Yes, Spark on EMR runs on YARN, so there is only a Spark UI when a Spark
app is running. To expand on what Natu says, the best way to view the Spark
UI for both running and completed Spark apps is to start from the YARN
ResourceManager UI (port 8088) and to click the "Application Master" link
(for running apps) or "History" link (for completed apps).

~ Jonathan

On Tue, Sep 13, 2016 at 2:30 AM Natu Lauchande <nlauchande@gmail.com> wrote:

> Hi,
>
> I think the spark UI will be accessible whenever you launch a spark app in
> the cluster it should be the Application Tracker link.
>
>
> Regards,
> Natu
>
> On Tue, Sep 13, 2016 at 9:37 AM, Divya Gehlot <divya.htconex@gmail.com>
> wrote:
>
>> Hi ,
>> Thank you all..
>> Hurray ...I am able to view the hadoop web UI now  @ 8088 . even Spark
>> Hisroty server Web UI @ 18080
>> But unable to figure out the Spark UI web port ...
>> Tried with 4044,4040.. ..
>> getting below error
>> This site can’t be reached
>> How can I find out the Spark port ?
>>
>> Would really appreciate the help.
>>
>> Thanks,
>> Divya
>>
>>
>> On 13 September 2016 at 15:09, Divya Gehlot <divya.htconex@gmail.com>
>> wrote:
>>
>>> Hi,
>>> Thanks all for your prompt response.
>>> I followed the instruction in the docs EMR SSH tunnel
>>> <https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html>
>>> shared by Jonathan.
>>> I am on MAC and set up foxy proxy in my chrome browser
>>>
>>> Divyas-MacBook-Pro:.ssh divyag$ ssh  -N -D 8157
>>> hadoop@ec2-xx-xxx-xxx-xx.ap-southeast-1.compute.amazonaws.com
>>>
>>> channel 3: open failed: connect failed: Connection refused
>>>
>>> channel 3: open failed: connect failed: Connection refused
>>>
>>> channel 4: open failed: connect failed: Connection refused
>>>
>>> channel 3: open failed: connect failed: Connection refused
>>>
>>> channel 4: open failed: connect failed: Connection refused
>>>
>>> channel 3: open failed: connect failed: Connection refused
>>>
>>> channel 3: open failed: connect failed: Connection refused
>>>
>>> channel 4: open failed: connect failed: Connection refused
>>>
>>> channel 5: open failed: connect failed: Connection refused
>>>
>>> channel 22: open failed: connect failed: Connection refused
>>>
>>> channel 23: open failed: connect failed: Connection refused
>>>
>>> channel 22: open failed: connect failed: Connection refused
>>>
>>> channel 23: open failed: connect failed: Connection refused
>>>
>>> channel 22: open failed: connect failed: Connection refused
>>>
>>> channel 8: open failed: administratively prohibited: open failed
>>>
>>>
>>> What am I missing now ?
>>>
>>>
>>> Thanks,
>>>
>>> Divya
>>>
>>> On 13 September 2016 at 14:23, Jonathan Kelly <jonathakamzn@gmail.com>
>>> wrote:
>>>
>>>> I would not recommend opening port 50070 on your cluster, as that would
>>>> give the entire world access to your data on HDFS. Instead, you should
>>>> follow the instructions found here to create a secure tunnel to the
>>>> cluster, through which you can proxy requests to the UIs using a browser
>>>> plugin like FoxyProxy:
>>>> https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html
>>>>
>>>> ~ Jonathan
>>>>
>>>> On Mon, Sep 12, 2016 at 10:40 PM Mohammad Tariq <dontariq@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Divya,
>>>>>
>>>>> Do you you have inbounds enabled on port 50070 of your NN machine.
>>>>> Also, it's a good idea to have the public DNS in your /etc/hosts for
proper
>>>>> name resolution.
>>>>>
>>>>>
>>>>> [image: --]
>>>>>
>>>>> Tariq, Mohammad
>>>>> [image: https://]about.me/mti
>>>>>
>>>>> <https://about.me/mti?promo=email_sig&utm_source=email_sig&utm_medium=external_link&utm_campaign=chrome_ext>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [image: http://] <http://about.me/mti>
>>>>> Tariq, Mohammad
>>>>> about.me/mti
>>>>> [image: http://]
>>>>> <http://about.me/mti>
>>>>>
>>>>> On Tue, Sep 13, 2016 at 9:28 AM, Divya Gehlot <divya.htconex@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> Hi,
>>>>>> I am on EMR 4.7 with Spark 1.6.1   and Hadoop 2.7.2
>>>>>> When I am trying to view Any of the web UI of the cluster either
>>>>>> hadoop or Spark ,I am getting below error
>>>>>> "
>>>>>> This site can’t be reached
>>>>>>
>>>>>> "
>>>>>> Has anybody using EMR and able to view WebUI .
>>>>>> Could you please share the steps.
>>>>>>
>>>>>> Would really appreciate the help.
>>>>>>
>>>>>> Thanks,
>>>>>> Divya
>>>>>>
>>>>>
>>>>>
>>>
>>
>

Mime
View raw message