hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yabo-Arber Xu <arber.resea...@gmail.com>
Subject Re: Hbase on EC2 and issues with external access
Date Fri, 20 Mar 2009 19:46:09 GMT
I have upgraded it to 0.19.1, and apply the patch. A newbie question: do I
have to re-build hbase to make it work?

Best,
Arber

On Fri, Mar 20, 2009 at 12:20 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Err you could try to apply the patch on 0.18.1 first but the best
> would be to upgrade to 0.19.1
>
> J-D
>
> On Fri, Mar 20, 2009 at 3:10 PM, Yabo-Arber Xu <arber.research@gmail.com>
> wrote:
> > Hi J-D,
> >
> > Thank you very much. I shall try it. I am still using hbase-0.18.1. Do I
> > have upgrade it to 0.19.x and then apply the patch?
> >
> > Best,
> > Arber
> >
> > On Fri, Mar 20, 2009 at 7:51 AM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
> >
> >> Arber,
> >>
> >> See my patch in https://issues.apache.org/jira/browse/HBASE-1279
> >>
> >> Now setting hbase.regionserver should work.
> >>
> >> J-D
> >>
> >>
> >> On Fri, Mar 20, 2009 at 8:44 AM, Jean-Daniel Cryans <
> jdcryans@apache.org>
> >> wrote:
> >> > I'm currently looking at the code and I think I see something that may
> >> > be fixing that problem. We have a similar problem here so I'll check
> >> > if it fixes it.
> >> >
> >> > J-D
> >> >
> >> > On Fri, Mar 20, 2009 at 5:51 AM, Yabo-Arber Xu <
> arber.research@gmail.com>
> >> wrote:
> >> >> Hi J-D,
> >> >>
> >> >> I further found that the region server can actually be connected via
> Web
> >> UI
> >> >> from an client outside Amazon network. That further verify the view
> that
> >> >> Master/Region server both are working, and it's just that Master
> mapped
> >> >> region servers' external IP into internal IP automatically, and the
> >> clients
> >> >> fail to communicate with region server via the mapped internal IP.
> >> >>
> >> >> Do you see any possible solution for this in the near future?
> >> >>
> >> >> Best,
> >> >> Arber
> >> >>
> >> >> On Thu, Mar 19, 2009 at 4:48 PM, Yabo-Arber Xu <
> >> arber.research@gmail.com>wrote:
> >> >>
> >> >>> Thx for your explanation. I suspect the reason is that when the
> master
> >> >>> initiate the contact with region server, region server was
> recognized
> >> >>> through its internal address ( as both of them are on Amazon
> network),
> >> even
> >> >>> i explicitly put the external address for region server.
> >> >>>
> >> >>> So the consequence is is that the internal address shows in
> namenode,
> >> to
> >> >>> which external clients can not connect.
> >> >>>
> >> >>>
> >> >>> On Thu, Mar 19, 2009 at 4:33 PM, Jean-Daniel Cryans <
> >> jdcryans@apache.org>wrote:
> >> >>>
> >> >>>> Getting the "good" host name for a node is kind of a pain.
For the
> >> >>>> moment, the implemented solution is that the Master tells the
> region
> >> >>>> server to override it's known address with what the region
server
> was
> >> >>>> able to contact the master with. It was implemented like this
so
> that
> >> >>>> the region servers stop showing all up as "127.0.0.1" and hadoop
> was
> >> >>>> doing it the same way.
> >> >>>>
> >> >>>> Now, I know that they changed it on their side, so I'd really
> really
> >> >>>> like to know with which address the datanodes are showing up
in the
> >> >>>> namenode. Internal or external? If it's the external one, I'll
dive
> in
> >> >>>> their code.
> >> >>>>
> >> >>>> Thx,
> >> >>>>
> >> >>>> J-D
> >> >>>>
> >> >>>> On Thu, Mar 19, 2009 at 7:26 PM, Yabo-Arber Xu <
> >> arber.research@gmail.com>
> >> >>>> wrote:
> >> >>>> > Hi J-D,
> >> >>>> >
> >> >>>> > Thanks for your reply, and sorry for my late response
as I was
> >> engaged
> >> >>>> in
> >> >>>> > sth else in the past two days.
> >> >>>> >
> >> >>>> > Yes, I've unlocked the port, and i am actually able to
access
> from
> >> the
> >> >>>> web
> >> >>>> > UI with a client not running on EC2 to HBase at
> example.com:60010.
> >> It
> >> >>>> shows
> >> >>>> > all User Tables, but the Region Servers Address is the
EC2
> internal
> >> >>>> address:
> >> >>>> > domU-12-31-39-00-65-E5.compute-1.internal:60020.
> >> >>>> >
> >> >>>> > I guess the client fails because it can not connect region
> server,
> >> which
> >> >>>> > serves only for an internal IP. However, in hbase-site.xml,
I did
> >> >>>> configure
> >> >>>> > with region server explicitly in its external IP.
> >> >>>> >
> >> >>>> >  <property>^M
> >> >>>> >    <name>hbase.regionserver</name>
> >> >>>> >    <value>ec2-67-202-57-127.compute-1.amazonaws.com:60020
> </value>
> >> >>>> >    <description>The host and port a HBase region
server runs
> at.^M
> >> >>>> >    </description>
> >> >>>> >  </property>
> >> >>>> >
> >> >>>> > What could I do wrong?
> >> >>>> >
> >> >>>> > Thanks again,
> >> >>>> > Arber
> >> >>>> >
> >> >>>> >
> >> >>>> > On Tue, Mar 17, 2009 at 5:05 AM, Jean-Daniel Cryans <
> >> >>>> jdcryans@apache.org>wrote:
> >> >>>> >
> >> >>>> >> Arber,
> >> >>>> >>
> >> >>>> >> There are security policies with EC2, did you unblock
port 60000
> >> for
> >> >>>> >> your own IP address?
> >> >>>> >>
> >> >>>> >> J-D
> >> >>>> >>
> >> >>>> >> On Tue, Mar 17, 2009 at 4:07 AM, Yabo-Arber Xu <
> >> >>>> arber.research@gmail.com>
> >> >>>> >> wrote:
> >> >>>> >> > Hi all,
> >> >>>> >> >
> >> >>>> >> > I set up a small HBase cluster on EC2. It works
fine
> internally
> >> if
> >> >>>> all
> >> >>>> >> the
> >> >>>> >> > applications are within EC2. It, however, does
not work if i
> am
> >> >>>> trying
> >> >>>> >> run
> >> >>>> >> > shell or client program on a external host. I've
been googling
> >> for a
> >> >>>> >> while,
> >> >>>> >> > and found there was similar issues raised before:
> >> >>>> >> >
> >> >>>> >>
> >> >>>>
> >>
> http://www.nabble.com/Hbase-on-EC2-and-issues-with-Amazon-NAT-Internal-Addresses-td21621367.html
> >> >>>> >> ,
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> > But it seems no solutions so far. I wonder whether
anybody has
> >> made
> >> >>>> >> progress
> >> >>>> >> > on this issue. Any thoughts would be greatly
appreciated!
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> > Attached with my hbase-site.xml. I am running
the
> master/region
> >> >>>> server
> >> >>>> >> all
> >> >>>> >> > on one instance for testing.
> >> >>>> >> >
> >> >>>> >> > <configuration>
> >> >>>> >> >  <property>
> >> >>>> >> >    <name>hbase.master</name>
> >> >>>> >> >
> >>  <value>*domU-12-31-39-00-E0-96.compute-1.internal*:60000</value>
> >> >>>> //It
> >> >>>> >> > does not work even if i changed this the Amazon
public IP
> >> >>>> >> >    <description>The host and port that
the HBase master runs
> at.
> >> >>>> >> >    </description>
> >> >>>> >> >  </property>
> >> >>>> >> >
> >> >>>> >> >  <property>
> >> >>>> >> >    <name>hbase.rootdir</name>
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >>
> >> >>>>
> >>
> <value>hdfs://domU-12-31-39-00-E0-96.compute-1.internal:54310/hbase</value>
> >> >>>> >> >    <description>The directory shared by
region servers.
> >> >>>> >> >    </description>
> >> >>>> >> >  </property>
> >> >>>> >> > </configuration>
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> > Thanks for your attention,
> >> >>>> >> > Arber
> >> >>>> >> >
> >> >>>> >>
> >> >>>> >
> >> >>>>
> >> >>>
> >> >>
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message