hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@apache.org>
Subject Re: trying HBase-9272 style parallel scanner for Trafodion-1421
Date Thu, 29 Oct 2015 05:09:14 GMT
Hi Eric,

Did you ever get to the bottom of this? Maybe you can pastebin some
jstack's while your scanners are running with the different modes?

-n

On Thu, Oct 15, 2015 at 7:58 AM, Eric Owhadi <eric.owhadi@esgyn.com> wrote:
> Hi Hbasers,
>
>
>
> I am experimenting with HBase-9272 (a parallel unordered scanner) to
> provide Trafodion a primitive that can be used when we need to scan regions
> as fast as possible and don’t care about order.
>
> However, while HBase-9272 has been optimized to round robin on region
> servers, the primitive I am trying to implement is supposed to “most often”
> perform parallel scan from a single region server, where our ESP (Executor
> Server Process) is located -> so it should be able to use short circuit
> HConnection.
>
>
>
> Today, trafodion can perform this parallel scanning task by launching
> multiple ESP per region server. However, this is higher resource cost than
> would be thread level parallelism, as ESP are processes.
>
>
>
> After having adapted HBase-9272 to work on HBase scanner 1.0 and the
> interference of the replica feature (I actually ported back ClientScanner
> of Hbase 98 to use HBase-9272 with Hbase 1.0), I got things working but I
> am getting unexpected results:
>
>
>
> Running on a single node HBase (the development environment), when I use
> the ESP parallelism I am getting the expected gain in performance, but when
> using the HBase-9272 I am not seeing any benefit compared to single regular
> scan.
>
>
>
> So I am wondering if I am not bottlenecking on a shared connection when
> using HBase-9272 that would not let concurrency happen when running against
> the same region server? Or what else could that be?
>
> In my testing, I use a full scan on a table with 10 regions, and limit the
> degree of parallelism to 2, just to be able to compare between ESP
> parallelism and thread level parallelism.
>
>
>
> Thanks in advance for the help,
>
> Eric Owhadi

Mime
View raw message