hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Region server goes away
Date Thu, 15 Apr 2010 06:18:41 GMT
On Wed, Apr 14, 2010 at 9:20 PM, Geoff Hendrey <ghendrey@decarta.com> wrote:

> Thanks for your help. See answers below.
>
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
> Sent: Wednesday, April 14, 2010 8:45 PM
> To: hbase-user@hadoop.apache.org
> Cc: Paul Mahon; Bill Brune; Shaheen Bahauddin; Rohit Nigam
> Subject: Re: Region server goes away
>
> On Wed, Apr 14, 2010 at 8:27 PM, Geoff Hendrey <ghendrey@decarta.com>
> wrote:
> > Hi,
> >
> > I have posted previously about issues I was having with HDFS when I
> > was running HBase and HDFS on the same box both pseudoclustered. Now I
> > have two very capable servers. I've setup HDFS with a datanode on each
> box.
> > I've setup the namenode on one box, and the zookeeper and HDFS master
> > on the other box. Both boxes are region servers. I am using hadoop
> > 20.2 and hbase 20.3.
>
> What do you have for replication?  If two datanodes, you've set it to two
> rather than default 3?
> Geoff: I didn't change the default, so it was 3. I will change it to 2
> moving forward. Actually, for now I am going to make it 1. For initial test
> runs I don't see why I need replication at all.
>
>
> >
> > I have set dfs.datanode.socket.write.timeout to 0 in hbase-site.xml.
> >
> This is probably not necessary.
>
>
> > I am running a mapreduce job with about 200 concurrent reducers, each
> > of which writes into HBase, with 32,000 row flush buffers.
>

 Do you really mean 200 concurrent reducers?? That is to say 100 reducers
per box? I would recommend that only if you have a 100+ core machine... not
likely.

FYI typical values for reduce slots on dual quad core Nehalem with
hyperthreading (ie 16 logical cores) are in the range of 8-10, not 100!

-Todd


-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message