hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Something Something <mailinglist...@gmail.com>
Subject Re: Starting HBase in fully distributed mode...
Date Wed, 09 Dec 2009 07:08:39 GMT
Didn't get a chance to go thru Andrew's script, but I tested the theory of
using fully qualified internal domain names (such as,
domU-12-31-38-00-89-AA.compute-1.internal) and that didn't work!  Zookeeper
is still having problems.

Sounds like others have used Andrew's script successfully.  The only
difference seems to be that it starts a *dedicated* ZooKeeper quorum.
Should have listened to Mark when he suggested that 4 days ago :)

Anyway, I will try Andrew's script tomorrow.


On Tue, Dec 8, 2009 at 2:36 PM, Andrew Purtell <apurtell@apache.org> wrote:

> > If HBase has scripts that start ZK for you it should be sure to
> > specify fully qualified names in the ZK configuration.
>
> The scripts build the ZK config dynamically and use the fully qualified
> (internal) domain names.
>
>  - Andy
>
>
>
>
> ________________________________
> From: Patrick Hunt <phunt@apache.org>
> To: hbase-user@hadoop.apache.org
> Cc: Jean-Daniel Cryans <jdcryans@apache.org>
> Sent: Tue, December 8, 2009 3:31:53 PM
> Subject: Re: Starting HBase in fully distributed mode...
>
> I consulted with a ZK user who runs exclusively on EC2. He mentioned
> that one potential problem ZK users may be seeing is the following:
>
> EC2 instances can be on different subdomains, even in the same
> geographical region.
>
> As a result if you configure ZK without fully qualified names the quorum
> may fail as DHCP is typically configured to not setup name resolution to
> search for hostnames across sub-domains.
>
> Verify that you are using fully qualified names in your ZK configuration
> file. (an alternative is to configure DHCP to resolve across the
> sub-domain, but this is not as simple as modifying resolve.conf afaik)
>
> If HBase has scripts that start ZK for you it should be sure to specify
> fully qualified names in the ZK configuration.
>
> Patrick
>
> Vaibhav Puranik wrote:
> > Something,
> >
> > You can now boot your instance with an EBS  volume. This is different
> from
> > attaching an EBS volume after booting up an instance.
> > This will simplify your setup. Furthermore, this functionality also
> offers a
> > new command to stop an instance which is different from 'Terminate'. Thus
> > you can stop your instance but never loose data.
> >
> > Regards,
> > Vaibhav Puranik
> > Gumgum
> >
> > On Tue, Dec 8, 2009 at 11:11 AM, Something Something <
> > mailinglists19@gmail.com> wrote:
> >
> >> Awesome.  Does it allow me to create HDFS on *my* EBS volumes?  If not,
> I
> >> probably can change that later.  Without EBS volumes all my data would
> be
> >> lost after I terminate my instances, right?
> >>
> >> On Tue, Dec 8, 2009 at 10:00 AM, Andrew Purtell <apurtell@apache.org>
> >> wrote:
> >>
> >>> Use the HBase EC2 scripts to launch and manage a cluster up in EC2. The
> >>> scripts launch the instances themselves, and take care of details such
> >>> as setting up security groups, allocating and starting a dedicated ZK
> >>> quorum ensemble, adding ZK and master private DNS names to the config
> >>> and distributing it to the slaves, etc.
> >>>
> >>> See subtasks on HBASE-1961 for further developments planned.
> >>>
> >>>   - Andy
> >>>
> >>>
> >>>
> >>>
> >>> ________________________________
> >>> From: Something Something <mailinglists19@gmail.com>
> >>> To: hbase-user@hadoop.apache.org
> >>> Sent: Tue, December 8, 2009 12:40:17 PM
> >>> Subject: Re: Starting HBase in fully distributed mode...
> >>>
> >>> Cool.  Didn't know there were scripts in the trunk for HBase.  Will
> >>> definitely try those.
> >>>
> >>> Should I first run the scripts for Hadoop (from
> <Hadoop>/src/contrib/ec2)
> >>> and then run scripts for HBase (from <HBase>/src/contrib/ec2)?  Or
> would
> >>> the
> >>> HBase scripts install/format HDFS automatically for me so that I don't
> >> need
> >>> to install Hadoop separately?
> >>>
> >>> In other words, what should be my approach:
> >>>
> >>> 1)  Launch 3 instances on EC2.  Run HBase scripts against them.
> >>>
> >>> OR
> >>>
> >>> 2)  Launch 3 instances on EC2. Install Hadoop.  Format HDFS.  Start
> HDFS.
> >>> Run HBase scripts against these instances?
> >>>
> >>> Please let me know.  Thanks.
> >>>
> >>>
> >>> On Tue, Dec 8, 2009 at 8:50 AM, Andrew Purtell <apurtell@apache.org>
> >>> wrote:
> >>>
> >>>>>> Good advice. This is what our EC2 scripts -- in src/contrib/ec2/
--
> >>>>>> do, and so I can say it works.
> >>>>> I don't think there are scripts for HBase on EC2, are there?
> >>>> We (HBase) have EC2 scripts -- src/contrib/ec2/, in SVN, in trunk and
> >> on
> >>>> the 0.20 branch.
> >>>>
> >>>>   - Andy
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message