hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: Hbase on EC2 and issues with external access
Date Fri, 20 Mar 2009 19:47:44 GMT
Yes if you patch src/ you need to rebuild.

this should be as simple as:
export JAVA_HOME=...
ant

and you are ready to go.

On Fri, Mar 20, 2009 at 12:46 PM, Yabo-Arber Xu <arber.research@gmail.com>wrote:

> I have upgraded it to 0.19.1, and apply the patch. A newbie question: do I
> have to re-build hbase to make it work?
>
> Best,
> Arber
>
> On Fri, Mar 20, 2009 at 12:20 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
>
> > Err you could try to apply the patch on 0.18.1 first but the best
> > would be to upgrade to 0.19.1
> >
> > J-D
> >
> > On Fri, Mar 20, 2009 at 3:10 PM, Yabo-Arber Xu <arber.research@gmail.com
> >
> > wrote:
> > > Hi J-D,
> > >
> > > Thank you very much. I shall try it. I am still using hbase-0.18.1. Do
> I
> > > have upgrade it to 0.19.x and then apply the patch?
> > >
> > > Best,
> > > Arber
> > >
> > > On Fri, Mar 20, 2009 at 7:51 AM, Jean-Daniel Cryans <
> jdcryans@apache.org
> > >wrote:
> > >
> > >> Arber,
> > >>
> > >> See my patch in https://issues.apache.org/jira/browse/HBASE-1279
> > >>
> > >> Now setting hbase.regionserver should work.
> > >>
> > >> J-D
> > >>
> > >>
> > >> On Fri, Mar 20, 2009 at 8:44 AM, Jean-Daniel Cryans <
> > jdcryans@apache.org>
> > >> wrote:
> > >> > I'm currently looking at the code and I think I see something that
> may
> > >> > be fixing that problem. We have a similar problem here so I'll check
> > >> > if it fixes it.
> > >> >
> > >> > J-D
> > >> >
> > >> > On Fri, Mar 20, 2009 at 5:51 AM, Yabo-Arber Xu <
> > arber.research@gmail.com>
> > >> wrote:
> > >> >> Hi J-D,
> > >> >>
> > >> >> I further found that the region server can actually be connected
> via
> > Web
> > >> UI
> > >> >> from an client outside Amazon network. That further verify the
view
> > that
> > >> >> Master/Region server both are working, and it's just that Master
> > mapped
> > >> >> region servers' external IP into internal IP automatically, and
the
> > >> clients
> > >> >> fail to communicate with region server via the mapped internal
IP.
> > >> >>
> > >> >> Do you see any possible solution for this in the near future?
> > >> >>
> > >> >> Best,
> > >> >> Arber
> > >> >>
> > >> >> On Thu, Mar 19, 2009 at 4:48 PM, Yabo-Arber Xu <
> > >> arber.research@gmail.com>wrote:
> > >> >>
> > >> >>> Thx for your explanation. I suspect the reason is that when
the
> > master
> > >> >>> initiate the contact with region server, region server was
> > recognized
> > >> >>> through its internal address ( as both of them are on Amazon
> > network),
> > >> even
> > >> >>> i explicitly put the external address for region server.
> > >> >>>
> > >> >>> So the consequence is is that the internal address shows in
> > namenode,
> > >> to
> > >> >>> which external clients can not connect.
> > >> >>>
> > >> >>>
> > >> >>> On Thu, Mar 19, 2009 at 4:33 PM, Jean-Daniel Cryans <
> > >> jdcryans@apache.org>wrote:
> > >> >>>
> > >> >>>> Getting the "good" host name for a node is kind of a pain.
For
> the
> > >> >>>> moment, the implemented solution is that the Master tells
the
> > region
> > >> >>>> server to override it's known address with what the region
server
> > was
> > >> >>>> able to contact the master with. It was implemented like
this so
> > that
> > >> >>>> the region servers stop showing all up as "127.0.0.1"
and hadoop
> > was
> > >> >>>> doing it the same way.
> > >> >>>>
> > >> >>>> Now, I know that they changed it on their side, so I'd
really
> > really
> > >> >>>> like to know with which address the datanodes are showing
up in
> the
> > >> >>>> namenode. Internal or external? If it's the external one,
I'll
> dive
> > in
> > >> >>>> their code.
> > >> >>>>
> > >> >>>> Thx,
> > >> >>>>
> > >> >>>> J-D
> > >> >>>>
> > >> >>>> On Thu, Mar 19, 2009 at 7:26 PM, Yabo-Arber Xu <
> > >> arber.research@gmail.com>
> > >> >>>> wrote:
> > >> >>>> > Hi J-D,
> > >> >>>> >
> > >> >>>> > Thanks for your reply, and sorry for my late response
as I was
> > >> engaged
> > >> >>>> in
> > >> >>>> > sth else in the past two days.
> > >> >>>> >
> > >> >>>> > Yes, I've unlocked the port, and i am actually able
to access
> > from
> > >> the
> > >> >>>> web
> > >> >>>> > UI with a client not running on EC2 to HBase at
> > example.com:60010.
> > >> It
> > >> >>>> shows
> > >> >>>> > all User Tables, but the Region Servers Address is
the EC2
> > internal
> > >> >>>> address:
> > >> >>>> > domU-12-31-39-00-65-E5.compute-1.internal:60020.
> > >> >>>> >
> > >> >>>> > I guess the client fails because it can not connect
region
> > server,
> > >> which
> > >> >>>> > serves only for an internal IP. However, in hbase-site.xml,
I
> did
> > >> >>>> configure
> > >> >>>> > with region server explicitly in its external IP.
> > >> >>>> >
> > >> >>>> >  <property>^M
> > >> >>>> >    <name>hbase.regionserver</name>
> > >> >>>> >    <value>ec2-67-202-57-127.compute-1.amazonaws.com:60020
> > </value>
> > >> >>>> >    <description>The host and port a HBase region
server runs
> > at.^M
> > >> >>>> >    </description>
> > >> >>>> >  </property>
> > >> >>>> >
> > >> >>>> > What could I do wrong?
> > >> >>>> >
> > >> >>>> > Thanks again,
> > >> >>>> > Arber
> > >> >>>> >
> > >> >>>> >
> > >> >>>> > On Tue, Mar 17, 2009 at 5:05 AM, Jean-Daniel Cryans
<
> > >> >>>> jdcryans@apache.org>wrote:
> > >> >>>> >
> > >> >>>> >> Arber,
> > >> >>>> >>
> > >> >>>> >> There are security policies with EC2, did you
unblock port
> 60000
> > >> for
> > >> >>>> >> your own IP address?
> > >> >>>> >>
> > >> >>>> >> J-D
> > >> >>>> >>
> > >> >>>> >> On Tue, Mar 17, 2009 at 4:07 AM, Yabo-Arber Xu
<
> > >> >>>> arber.research@gmail.com>
> > >> >>>> >> wrote:
> > >> >>>> >> > Hi all,
> > >> >>>> >> >
> > >> >>>> >> > I set up a small HBase cluster on EC2. It
works fine
> > internally
> > >> if
> > >> >>>> all
> > >> >>>> >> the
> > >> >>>> >> > applications are within EC2. It, however,
does not work if i
> > am
> > >> >>>> trying
> > >> >>>> >> run
> > >> >>>> >> > shell or client program on a external host.
I've been
> googling
> > >> for a
> > >> >>>> >> while,
> > >> >>>> >> > and found there was similar issues raised
before:
> > >> >>>> >> >
> > >> >>>> >>
> > >> >>>>
> > >>
> >
> http://www.nabble.com/Hbase-on-EC2-and-issues-with-Amazon-NAT-Internal-Addresses-td21621367.html
> > >> >>>> >> ,
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >> > But it seems no solutions so far. I wonder
whether anybody
> has
> > >> made
> > >> >>>> >> progress
> > >> >>>> >> > on this issue. Any thoughts would be greatly
appreciated!
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >> > Attached with my hbase-site.xml. I am running
the
> > master/region
> > >> >>>> server
> > >> >>>> >> all
> > >> >>>> >> > on one instance for testing.
> > >> >>>> >> >
> > >> >>>> >> > <configuration>
> > >> >>>> >> >  <property>
> > >> >>>> >> >    <name>hbase.master</name>
> > >> >>>> >> >
> > >>  <value>*domU-12-31-39-00-E0-96.compute-1.internal*:60000</value>
> > >> >>>> //It
> > >> >>>> >> > does not work even if i changed this the
Amazon public IP
> > >> >>>> >> >    <description>The host and port
that the HBase master runs
> > at.
> > >> >>>> >> >    </description>
> > >> >>>> >> >  </property>
> > >> >>>> >> >
> > >> >>>> >> >  <property>
> > >> >>>> >> >    <name>hbase.rootdir</name>
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >>
> > >> >>>>
> > >>
> >
> <value>hdfs://domU-12-31-39-00-E0-96.compute-1.internal:54310/hbase</value>
> > >> >>>> >> >    <description>The directory shared
by region servers.
> > >> >>>> >> >    </description>
> > >> >>>> >> >  </property>
> > >> >>>> >> > </configuration>
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >> > Thanks for your attention,
> > >> >>>> >> > Arber
> > >> >>>> >> >
> > >> >>>> >>
> > >> >>>> >
> > >> >>>>
> > >> >>>
> > >> >>
> > >> >
> > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message