hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shaosu Liu <Shaosu....@turn.com>
Subject concurrent scan optimization
Date Thu, 24 Apr 2014 19:02:30 GMT

I want to large amount of concurrent scan (10k), each fetching 200 - 500 keys, each key is
around 500 - 1k bytes.

I have 5 regions servers using offheap cache, data is fully cached and all machines are on
the same rack with gigabit connection. So GC or disk should not slow down me much. 

Scan caching is set to 10000, each region server has 200 handlers.

Currently it takes around 30 seconds to process 10k scan concurrently. If I use asynchbase,
it takes around 18seconds. It should be able to possible do this around 1 - 2 seconds. 

Could anybody shed light on what I am doing wrong here?

~Shaosu Liu

View raw message