hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Owhadi <eric.owh...@esgyn.com>
Subject trying HBase-9272 style parallel scanner for Trafodion-1421
Date Thu, 15 Oct 2015 14:58:23 GMT
Hi Hbasers,



I am experimenting with HBase-9272 (a parallel unordered scanner) to
provide Trafodion a primitive that can be used when we need to scan regions
as fast as possible and don’t care about order.

However, while HBase-9272 has been optimized to round robin on region
servers, the primitive I am trying to implement is supposed to “most often”
perform parallel scan from a single region server, where our ESP (Executor
Server Process) is located -> so it should be able to use short circuit
HConnection.



Today, trafodion can perform this parallel scanning task by launching
multiple ESP per region server. However, this is higher resource cost than
would be thread level parallelism, as ESP are processes.



After having adapted HBase-9272 to work on HBase scanner 1.0 and the
interference of the replica feature (I actually ported back ClientScanner
of Hbase 98 to use HBase-9272 with Hbase 1.0), I got things working but I
am getting unexpected results:



Running on a single node HBase (the development environment), when I use
the ESP parallelism I am getting the expected gain in performance, but when
using the HBase-9272 I am not seeing any benefit compared to single regular
scan.



So I am wondering if I am not bottlenecking on a shared connection when
using HBase-9272 that would not let concurrency happen when running against
the same region server? Or what else could that be?

In my testing, I use a full scan on a table with 10 regions, and limit the
degree of parallelism to 2, just to be able to compare between ESP
parallelism and thread level parallelism.



Thanks in advance for the help,

Eric Owhadi

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message