hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject Re: HWI use on AWS/EMR
Date Fri, 18 Jan 2013 17:07:48 GMT
ok so much discussion around this but why dont you open access to port 9999
for your specific machine (home/office) and access it normally without
setting up port forwarding.

AWS does give you ip level access control in the security groups


On Fri, Jan 18, 2013 at 10:35 PM, Ariel Marcus <ariel.marcus@openbi.com>wrote:

> This should be localhost instead of proxy:
>
> - In Connection/SSH/Tunnels I’ve added 9999 in the Source Port box, *
> localhost*:9999 in the Destination box. Not sure if this is correct.
>
> Best,
> Aril
>
> ---------------------------------
> Ariel Marcus, Consultant
> www.openbi.com | ariel.marcus@openbi.com
> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
> Cell: 314-827-4356
>
>
> On Fri, Jan 18, 2013 at 12:00 PM, Tony Burton <TBurton@sportingindex.com>wrote:
>
>> ** **
>>
>> Ok, thanks Ariel (and Dean in the next email!)****
>>
>> ** **
>>
>> I’ve set up PuTTY (v0.62) as follows:****
>>
>> - EMR instance hostname in the Session/Host Name box (port left at 22,
>> ok?)****
>>
>> - In Connection/SSH/Auth I’ve added my private key file, downloaded from
>> AWS ages for this account.****
>>
>> - In Connection/SSH/Tunnels I’ve added 9999 in the Source Port box,
>> proxy:9999 in the Destination box. Not sure if this is correct.****
>>
>> ** **
>>
>> Hit the Open button, I can connect. (And I can see my changes in
>> hive-default.xml, so I know I’m in the right place:)****
>>
>> Then pointing my browser at http://localhost:9999 I get the same timeout
>> again. Have I set up everything right?****
>>
>> ** **
>>
>> Also, following good advice I copied the config changes to hive-site.xml.
>> How do I stop hive so I can restart with ‘hive –service hwi’ so that it
>> picks up the new config?****
>>
>> ** **
>>
>> Tony****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> *From:* Ariel Marcus [mailto:ariel.marcus@openbi.com]
>> *Sent:* 18 January 2013 16:31
>>
>> *To:* user@hive.apache.org
>> *Subject:* Re: HWI use on AWS/EMR****
>>
>> ** **
>>
>> Hey Tony,****
>>
>> ** **
>>
>> Port forwarding is the same as what Dean referred to as ssh tunneling.***
>> *
>>
>> ** **
>>
>> Here is a website explaining how to set up port forwarding in putty (use
>> port 9999):****
>>
>> http://www.cs.uu.nl/technical/services/ssh/putty/puttyfw.html****
>>
>> ** **
>>
>> You can edit either hive-default or hive-site but the general practice is
>> to store changes from the default configuration in hive-site.****
>>
>> ** **
>>
>> Best,****
>>
>> Ariel****
>>
>> ** **
>>
>>
>> ****
>>
>> ---------------------------------****
>>
>> Ariel Marcus, Consultant****
>>
>> www.openbi.com | ariel.marcus@openbi.com****
>>
>> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
>> Cell: 314-827-4356****
>>
>> ** **
>>
>> On Fri, Jan 18, 2013 at 11:26 AM, Tony Burton <TBurton@sportingindex.com>
>> wrote:****
>>
>> Hi Ariel,****
>>
>>  ****
>>
>> Thanks for the speedy reply. We’ll be accessing the HWI from Windows
>> rather Linux desktops, so can you help me out with understanding how to
>> carry out the port forwarding from a Windows environment? Currently I use
>> WinSCP to initiate a connection to the EMR instance, then spawn a PuTTY
>> session.****
>>
>>  ****
>>
>> The changes to the hive config look identical to what I’ve changed in my
>> hive-default.xml. Does it make any difference which config file the
>> changesare applied to (hive-default.xml vs hive-site.xml)?****
>>
>>  ****
>>
>> Tony****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Ariel Marcus [mailto:ariel.marcus@openbi.com]
>> *Sent:* 18 January 2013 16:16****
>>
>>
>> *To:* user@hive.apache.org
>> *Subject:* Re: HWI use on AWS/EMR****
>>
>>  ****
>>
>> Hey Tony,****
>>
>>  ****
>>
>> I would recommend using ssh port forwarding instead of making your hwi
>> publicly available on the internet. When you connect to the master node
>> using ssh you should use a command like the following:****
>>
>>  ****
>>
>> ssh -i /path/to/keyfile.pem -L 9999:localhost:9999 -l hadoop
>> MASTER_HOSTNAME****
>>
>>  ****
>>
>> After you have connected, add the following to your
>> hive/conf/hive-site.xml file:****
>>
>>  ****
>>
>> <property>****
>>
>>   <name>hive.hwi.war.file</name>****
>>
>>   <value>lib/hive-hwi-0.8.1.war</value>****
>>
>>   <description>This is the WAR file with the jsp content for Hive Web
>> Interface</description>****
>>
>> </property>****
>>
>>  ****
>>
>> Run this command to start up hwi:****
>>
>>  ****
>>
>> hive --service hwi****
>>
>>  ****
>>
>> And finally point your browser to:****
>>
>>  ****
>>
>> localhost:9999****
>>
>>  ****
>>
>> That worked for me.****
>>
>>  ****
>>
>> Best,****
>>
>> Ariel****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>>
>> ****
>>
>> ---------------------------------****
>>
>> Ariel Marcus, Consultant****
>>
>> www.openbi.com | ariel.marcus@openbi.com****
>>
>> 150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
>> Cell: 314-827-4356****
>>
>>  ****
>>
>> On Fri, Jan 18, 2013 at 11:09 AM, Dean Wampler <
>> dean.wampler@thinkbiganalytics.com> wrote:****
>>
>> Oops, I overlooked that you have the public domain name in your message.
>> Can you surf to  http://ec2-54-247-61-206.eu-west-1.compute.amazonaws.com?  If not,
does HWI use port 80? Is whatever port it uses blocked by EC2?
>> ****
>>
>>  ****
>>
>> If it's blocked you can use ssh to tunnel the port through.****
>>
>>  ****
>>
>> dean****
>>
>>  ****
>>
>> On Fri, Jan 18, 2013 at 10:06 AM, Dean Wampler <
>> dean.wampler@thinkbiganalytics.com> wrote:****
>>
>> That's the internal hostname, not visible outside. Use the name like
>> ec2-NNN-NN-NN-NNN.compute-1.amazonaws.com. It's shown in the EMR console
>> and the elastic-mapreduce script you might have used to launch the cluster.
>> ****
>>
>>  ****
>>
>> If that doesn't work, verify that port 80 is not blocked by default.
>> That's certainly true for ports 9XXX used by the JobTracker, etc. ****
>>
>>  ****
>>
>> dean****
>>
>>  ****
>>
>> On Fri, Jan 18, 2013 at 9:54 AM, Tony Burton <TBurton@sportingindex.com>
>> wrote:****
>>
>> Hi,
>>
>> I'm trying to get HWI running and accessible from an Amazon Web Services
>> EMR instance. I've hit a blocker early on though, and the documentation is
>> less than illuminating. Can you share any experiences you have had?
>> Specifically, here's what I'm curious about.
>>
>> - Running on AWS. I've created a Hive job flow on AWS, edited
>> hive-default.xml (in /home/hadoop/.versions/hive-0.8.1/conf) and changed
>> the hive.hwi.war.file to lib/hive-hwi-0.8.1.war.
>> - HWI starts up fine, but when I try to connect with
>> http://ip-XX-AAA-BBB-CCC.eu-west-1.compute.internal:9999/hwi, I get a
>> timeout message, in Firefox it's "The connection has timed out - The server
>> at ec2-54-247-61-206.eu-west-1.compute.amazonaws.com is taking too long
>> to respond".
>>
>> Has anyone successfully connected to HWI running on an AWS EMR instance?
>> From the same browser I can connect to HWI on a local Ubuntu box.
>>
>> Thanks!
>>
>> Tony
>>
>>
>> **********************************************************************
>> Please consider the environment before printing this email or attachments
>>
>> This email and any attachments are confidential, protected by copyright
>> and may be legally privileged.  If you are not the intended recipient, then
>> the dissemination or copying of this email is prohibited. If you have
>> received this in error, please notify the sender by replying by email and
>> then delete the email completely from your system.  Neither Sporting Index
>> nor the sender accepts responsibility for any virus, or any other defect
>> which might affect any computer or IT system into which the email is
>> received and/or opened.  It is the responsibility of the recipient to scan
>> the email and no responsibility is accepted for any loss or damage arising
>> in any way from receipt or use of this email.  Sporting Index Ltd is a
>> company registered in England and Wales with company number 2636842, whose
>> registered office is at Gateway House, Milverton Street, London, SE11 4AP.
>>  Sporting Index Ltd is authorised and regulated by the UK Financial
>> Services Authority (reg. no. 150404) and Gambling Commission (reg. no.
>> 000-027343-R-308898-001).  Any financial promotion contained herein has
>> been issued
>> and approved by Sporting Index Ltd.
>>
>> Outbound email has been scanned for viruses and SPAM****
>>
>>
>>
>> ****
>>
>>  ****
>>
>> --
>> *Dean Wampler, Ph.D.*****
>>
>> thinkbiganalytics.com****
>>
>> +1-312-339-1330****
>>
>>  ****
>>
>>
>>
>> ****
>>
>>  ****
>>
>> --
>> *Dean Wampler, Ph.D.*****
>>
>> thinkbiganalytics.com****
>>
>> +1-312-339-1330****
>>
>>  ****
>>
>>  ****
>> ------------------------------
>>
>> This transmission is confidential and intended solely for the use of the
>> recipient named above. It may contain confidential, proprietary, or legally
>> privileged information. If you are not the intended recipient, you are
>> hereby notified that any unauthorized review, use, disclosure or
>> distribution is strictly prohibited. If you have received this transmission
>> in error, please contact the sender by reply e-mail and delete the original
>> transmission and all copies from your system.****
>>
>>  ****
>>
>>  ****
>> ------------------------------
>>
>> This transmission is confidential and intended solely for the use of the
>> recipient named above. It may contain confidential, proprietary, or legally
>> privileged information. If you are not the intended recipient, you are
>> hereby notified that any unauthorized review, use, disclosure or
>> distribution is strictly prohibited. If you have received this transmission
>> in error, please contact the sender by reply e-mail and delete the original
>> transmission and all copies from your system.****
>>
>>  ****
>>
>>
>>
>> *****************************************************************************
>> P *Please consider the environment before printing this email* ****
>>
>>
>> www.sportingindex.com
>>
>> Inbound email has been scanned for viruses & spam****
>>
>> ** **
>> ------------------------------
>>
>> This transmission is confidential and intended solely for the use of the
>> recipient named above. It may contain confidential, proprietary, or legally
>> privileged information. If you are not the intended recipient, you are
>> hereby notified that any unauthorized review, use, disclosure or
>> distribution is strictly prohibited. If you have received this transmission
>> in error, please contact the sender by reply e-mail and delete the original
>> transmission and all copies from your system.****
>>
>> ** **
>>
>> ** **
>> ------------------------------
>>
>> This transmission is confidential and intended solely for the use of the
>> recipient named above. It may contain confidential, proprietary, or legally
>> privileged information. If you are not the intended recipient, you are
>> hereby notified that any unauthorized review, use, disclosure or
>> distribution is strictly prohibited. If you have received this transmission
>> in error, please contact the sender by reply e-mail and delete the original
>> transmission and all copies from your system.****
>>
>> ------------------------------
>>
>> This transmission is confidential and intended solely for the use of the
>> recipient named above. It may contain confidential, proprietary, or legally
>> privileged information. If you are not the intended recipient, you are
>> hereby notified that any unauthorized review, use, disclosure or
>> distribution is strictly prohibited. If you have received this transmission
>> in error, please contact the sender by reply e-mail and delete the original
>> transmission and all copies from your system.
>>
>
>
> ------------------------------
>
> This transmission is confidential and intended solely for the use of the
> recipient named above. It may contain confidential, proprietary, or legally
> privileged information. If you are not the intended recipient, you are
> hereby notified that any unauthorized review, use, disclosure or
> distribution is strictly prohibited. If you have received this transmission
> in error, please contact the sender by reply e-mail and delete the original
> transmission and all copies from your system.
>



-- 
Nitin Pawar

Mime
View raw message