hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dejan Menges <dejan.men...@gmail.com>
Subject Re: Right value for hbase.rpc.timeout
Date Mon, 04 May 2015 08:31:40 GMT
Hi Ted,

Max filesize for region is set to 75G in our case. Regarding split policy
we use most likely ConstantSizeRegionSplitPolicy
<http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.html>
(it's
0.98.0 with bunch of patches and that should be default one).

Also, regarding link you sent me in 98.3 - I can not find anywhere what's
default value for hbase.regionserver.lease.period? Is this parameter still
called like this?

On Thu, Apr 30, 2015 at 11:27 PM Ted Yu <yuzhihong@gmail.com> wrote:

> Please take a look at 98.3 under
> http://hbase.apache.org/book.html#trouble.client
>
> BTW what's the value for hbase.hregion.max.filesize ?
> Which split policy do you use ?
>
> Cheers
>
> On Thu, Apr 30, 2015 at 6:59 AM, Dejan Menges <dejan.menges@gmail.com>
> wrote:
>
> > Basically how I came to this question - this happened super rarely, and
> we
> > narrowed it down to hotspotting. Map was timing out on three regions
> which
> > were 4-5 times bigger then other regions for the same table, and region
> > split fixed this.
> >
> > However, was just thinking about if there are maybe some recommendations
> or
> > something about this, as it's also super hard to reproduce again same
> > situation to retest it.
> >
> > On Thu, Apr 30, 2015 at 3:56 PM Michael Segel <michael_segel@hotmail.com
> >
> > wrote:
> >
> > > There is no single ‘right’ value.
> > >
> > > As you pointed out… some of your Mapper.map() iterations are taking
> > longer
> > > than 60 seconds.
> > >
> > > The first thing is to determine why that happens.  (It could be normal,
> > or
> > > it could be bad code on your developers part. We don’t know.)
> > >
> > > The other thing is that if you determine that your code is perfect and
> it
> > > does what you want it to do… and its a major part of your use case… you
> > > then increase your timeouts to 120 seconds.
> > >
> > > The reason why its a tough issue is that we don’t know what hardware
> you
> > > are using. How many nodes… code quality.. etc … too many factors.
> > >
> > >
> > > > On Apr 30, 2015, at 6:51 AM, Dejan Menges <dejan.menges@gmail.com>
> > > wrote:
> > > >
> > > > Hi,
> > > >
> > > > What's the best practice to calculate this value for your cluster, if
> > > there
> > > > is some?
> > > >
> > > > In some situations we saw that some maps are taking more than default
> > 60
> > > > seconds which was failing specific map job (as if it failed once, it
> > > failed
> > > > also every other time by number of configured retries).
> > > >
> > > > I would like to tune RPC parameters a bit, but googling and looking
> > into
> > > > HBase Book doesn't tell me how to calculate right values, and what
> else
> > > to
> > > > take a look beside hbase.rpc.timeout.
> > > >
> > > > Thanks a lot,
> > > > Dejan
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message