hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Li <fancye...@gmail.com>
Subject Re: speed control on the server side
Date Wed, 18 Jun 2014 10:54:23 GMT
I mean client slow itself down. e.g.
my client code(one of many threads)
    while(true){
         // process data and generate data need to insert to hbase
         List<Put> puts=...;
         table.put(puts);
    }

the client (I mean threads created by hbase api, not by codes) is
slowed down by retries. But if table.put(puts) don't block. my codes
will send requests. And the server is too busy and gc stop the world
and zookeeper don't receive heartbeat and regards it as dead

On Wed, Jun 18, 2014 at 6:17 PM, Nicolas Liochon <nkeywal@gmail.com> wrote:
> What do you mean by down? Does it crash?
>
> The server does not block on 0.96, it immediately sends back an exception.
> (See HBASE-9467)
> The client is implicitly slowed down by the retries, w/o blocking on the
> server. It's managed by the hbase client itself, and it's transparent for
> the client application. You've got some logs (info level), to tell you that
> something is happening, but that's it.
>
> You can configure the client to be less more or less verbose
> (hbase.client.start.log.errors.counter) or more or less pushy on the server
> (for example hbase.client.max.perserver.tasks).
>
> Cheers,
>
> Nicolas
>
>
> On Wed, Jun 18, 2014 at 11:59 AM, Li Li <fancyerii@gmail.com> wrote:
>
>> hi all,
>>     the hbase client send too much requests and the some region server
>> down.
>>     1. region server down because of gc pause
>>         I can see it from log:
>>        [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM
>> or host machine (eg GC): pause of approximately 3056ms
>>          I can adjust zookeeper config
>>     2. as a client, how can I know server is busy?
>>          I am using hbase 0.96 with hadoop1. I can see log such as:
>> org.apache.hadoop.hbase.client.AsyncProcess #178, table=vc2.url_db,
>> attempt=13/35 SUCCEEDED on mphbase1,60020,1403084938641, tracking
>> started Wed Jun 18 17:31:10 CST 2014
>>          I googled and found
>>
>> http://apache-hbase.679495.n3.nabble.com/RegionTooBusyException-Above-memstore-limit-td4056339.html
>>          it says in hbase 0.94, if the server is busy, it will block
>> the client request. That's what I want, if the client send too much
>> request than server can handle, it will be blocked. But in 0.96, it
>> seems not.
>>          I did not find any exception in client. So I can't slow the clien
>> down.
>>

Mime
View raw message