hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tatsuya Kawano <tatsuy...@snowcocoa.info>
Subject Re: TableIndexed extreme performance degredation?
Date Fri, 16 Oct 2009 04:25:59 GMT
Kevin,

On Fri, Oct 16, 2009 at 9:50 AM, Tatsuya Kawano wrote:
> Maybe you can workaround this by adding more RPC handlers (increase
> the value of "hbase.regionserver.handler.count" in hbase-site.xml)
> I'm going to run the same test again with 70 RPC handlers, and let you
> know the result.

I ran the test with the proposed workaround (100 RPC handlers as Andy
suggested) and it worked fine. I was able to insert 40 million records
without deadlock (by the way, I'm running 3 region servers.)

You have 12 mappers running at the same time (4 tasktrackers x 3 map
slots each), so having 100 RPC handlers on each indexed region server
will be safe enough to avoid the deadlock.


Andy,

Thanks for adding the troubleshooting tip and filing the issue. Those
tips are very helpful.


Ryan,

On Fri, Oct 16, 2009 at 9:55 AM, Ryan Rawson wrote:
> In general for at-scale I am not sure I can recommend the indexed
> features. You might get better miliage with hand maintaining the
> indexes yourself.

Thanks for your advice. I agree with you. Server managed secondary
index is convenient, but making the client application to maintain
secondary indices should scale better. Also, different applications
will need different secondary indices, so it's more like application's
responsibly to maintain these indices.


Thanks,

-- 
Tatsuya Kawano (Mr.)
Tokyo, Japan

Mime
View raw message