accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4643) Allow iterators to interrupt themselves
Date Fri, 09 Jun 2017 19:31:18 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16044884#comment-16044884
] 

Keith Turner commented on ACCUMULO-4643:
----------------------------------------

I created a [test suite|https://github.com/keith-turner/yield-test] and ran the following
test.

Test 1 : 128 tserver threads, 127 non-yielding long running scans, 1 thread doing 50 batch
scans of 5 random ranges.  Below are the times in ms for the batch scans.  There were 5,000,000
key/values.  Each long runnning scan was reading all.  The total time for all long running
scans was : 104 seconds

{noformat}
min: 30.0
max: 2395.0
mean: 882.3599999999999
standard deviation: 653.56063254159
{noformat}

Test 2 : 128 tserver threads, 127 yielding long running scans, 1 thread doing 50 batch scans
of 5 random ranges.  The long running scans yielded every 10,000.  Below are the times in
ms for the batch scans.  Total time for all long running scans was : 215 seconds

{noformat}
min: 428.0
max: 827.0
mean: 626.2800000000002
standard deviation: 104.54157263100298
{noformat}

Test 3 : 16 tserver threads, 20 long running scans, 1 thread doing 50 batch scans of 5 random
ranges.  Below are the times in ms for the batch scans.  There were 31,750,000 key/values.
 Each long runnning scan was reading all.  The total time for all long running scans was :
174 seconds

{noformat}
min: 120.0
max: 548.0
mean: 210.54000000000002
standard deviation: 70.64345948031524
{noformat}



> Allow iterators to interrupt themselves
> ---------------------------------------
>
>                 Key: ACCUMULO-4643
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4643
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>    Affects Versions: 1.8.1, 2.0.0
>            Reporter: Ivan Bella
>            Assignee: Ivan Bella
>              Labels: features
>             Fix For: 2.0.0
>
>          Time Spent: 16h 50m
>  Remaining Estimate: 0h
>
> The idea here is to allow an iterator stack to send back a special key or throw a special
exception which will allow the tablet server to tear down the scan to be rebuilt later.  This
is to handle the case where an iterator is doing a lot of work without returning results to
avoid starving out other scans.
> There are two thoughts on how to do this:
> 1) A special "interrupt" key is returned from the getTopKey call that is detected in
the Tablet.nextBatch call, is not added to the results, but is used to add an unfinished range
and results in the remaining ranges to be deemed unfinished.
> 2) An special exception is thrown from the next or seek call that included the key of
the current position, and the same actions are taken as in 1).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message