hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randy Fox <r...@connexity.com>
Subject Re: Parallelizing TableSnapshotScanner
Date Thu, 20 Oct 2016 15:41:03 GMT
I can work on that.  Since it is reading the restore data from a snapshot it is definitely
not changing.  I wanted to make sure there was not some inherent thread model under the scanner
that prohibited this from working.




On 10/20/16, 8:30 AM, "Ted Yu" <yuzhihong@gmail.com> wrote:

>Can you come up with unit test that shows what you observed ?
>You may have seen the javadoc for ClientSideRegionScanner :
>
> * A client scanner for a region opened for read-only on the client side.
>Assumes region data
>
> * is not changing.
>
>Cheers
>
>On Thu, Oct 20, 2016 at 8:26 AM, Randy Fox <rfox@connexity.com> wrote:
>
>> Hi,
>>
>> I have attempted to parallelize the TableSnapshotScanner by taking the
>> ClientSideRegionScanner’s it generates and sending them each to their own
>> thread to read the results.  I am finding that I do not get all the rows
>> per region when I do this.  It seems the more parallelism the less data I
>> get back.  Any ideas on why these scanners cannot be accessed in parallel?
>>
>> Thanks in advance,
>>
>> Randy
>>
Mime
View raw message