hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajeshkumar J <rajeshkumarit8...@gmail.com>
Subject Re: RPC calls in hbase
Date Thu, 05 Jan 2017 14:09:09 GMT
Adding this
Here Clauster have three machines each have 125 GB as RAM in which 70 GB is
free, One Machine acts as HMaster and also Region Server. Another machine
acts as Secondary Hmaster and also Region Server. Third Machine is a
dedicated Hregion Server. We have about 7000 Regions.
Maybe is this the problem?? Containing more regions

On Thu, Jan 5, 2017 at 3:27 AM, Stack <stack@duboce.net> wrote:

> (Thanks for jumping in Billy)
>
> Smaller Scans? (This is good on Scans in 1.1.+ which you seem to be on:
> https://blogs.apache.org/hbase/entry/scan_improvements_in_hbase_1)
>
> Otherwise look at other factors. Are the servers loaded? GC on
> client/server or resources being taken up by adjacent processes; etc.
>
> St.Ack
>
>
>
> On Wed, Jan 4, 2017 at 11:00 AM, Billy Watson <williamrwatson@gmail.com>
> wrote:
>
> > Disclaimer: this is what I think I know and I'm not an HBase
> contributor...
> >
> > Anytime we get scanner timeouts like that, we first expand the timeout
> > itself like you're doing to fix the short-term problem. But then we look
> at
> > our rowkey design and what we're doing. Timeouts like that are usually
> > symptomatic of bad rowkey utilization, too few servers or too many
> regions
> > per server (and therefore too little RAM to serve the regions).
> >
> > Make sure you're using good rowkey design and actually taking advantage
> of
> > your rowkey. HBase wasn't really intended to do random scans of columns
> > without rowkey filters which is why I think that timeout is set so low by
> > default.
> >
> > This page helps:
> >
> > http://hbase.apache.org/0.94/book/rowkey.design.html
> >
> > William Watson
> > Lead Software Engineer
> >
> > On Wed, Jan 4, 2017 at 1:41 AM, Rajeshkumar J <
> rajeshkumarit8292@gmail.com
> > >
> > wrote:
> >
> > > Hi Stack,
> > >
> > >    Yes it worked. I had a serious doubt regarding lease does not exist
> > > exception. We have 30 tables which have 45 million records each. We are
> > > scanning the table initially process is successful but after it gets
> > failed
> > > with lease does not exist exception. By changing hbase.rpc.timeout and
> > > hbase.client.scanner.timeout.period it got succeeded. It runs for
> > sometime
> > > but again it fails. Then again increasing the value it gets succeeded.
> > > Initially I have default values for these two property 60000 and then
> > > changed to 70000 and then changed to 80000.
> > >
> > >   Can you tell me how to resolve this?
> > >
> > > On Wed, Jan 4, 2017 at 11:42 AM, Stack <stack@duboce.net> wrote:
> > >
> > > > Yeah. Try it.
> > > > S
> > > >
> > > > On Tue, Jan 3, 2017 at 9:18 PM, Rajeshkumar J <
> > > rajeshkumarit8292@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > So I need to add property in log4j.properties as below
> > > > >
> > > > > log4j.logger.org.apache.hadoop.hbase.ipc.RpcServer = TRACE
> > > > >
> > > > > On Wed, Jan 4, 2017 at 10:23 AM, Stack <stack@duboce.net> wrote:
> > > > >
> > > > > > See http://hbase.apache.org/book.html#log4j
> > > > > >
> > > > > > Or to set it temporarily via UI, see
> > > > > > http://hbase.apache.org/book.html#trouble.log.levels
> > > > > >
> > > > > > St.Ack
> > > > > >
> > > > > > On Tue, Jan 3, 2017 at 6:58 PM, Rajeshkumar J <
> > > > > rajeshkumarit8292@gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Do we have to include this property in hbase-site.xml?
> > > > > > >
> > > > > > > On Wed, Jan 4, 2017 at 2:12 AM, Stack <stack@duboce.net>
> wrote:
> > > > > > >
> > > > > > > > On Mon, Jan 2, 2017 at 10:32 PM, Rajeshkumar J <
> > > > > > > > rajeshkumarit8292@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Do all rpc calls are logged in hbase region server
log?
> Also
> > I
> > > > need
> > > > > > to
> > > > > > > > find
> > > > > > > > > the time taken for each scan calls is this possible
> > > > > > > > >
> > > > > > > > > If you enable trace level logging for the rpcserver
class,
> > > > > > > > org.apache.hadoop.hbase.ipc.RpcServer, then you can
see a
> > bunch
> > > of
> > > > > > > detail
> > > > > > > > on each rpc invocation (including timings, size, etc):
> > > > > > > >
> > > > > > > > St.Ack
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message