accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (ACCUMULO-4643) Allow iterators to interrupt themselves
Date Mon, 05 Jun 2017 16:43:04 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037191#comment-16037191
] 

Keith Turner edited comment on ACCUMULO-4643 at 6/5/17 4:42 PM:
----------------------------------------------------------------

Based on comments [~phrocker] made on the PR, I think it would be nice to create a simple
test that contrast the responsiveness and throughput of this approach with lots of threads.
 Unless someone else wants to take a crack at it, I am thinking of creating a little test
that does the following.

 * Write lots of data to Accumulo.
 * Create X long running scans that filter alot and return little.
 * Create a few clients that constantly do random lookups.  Measure the time of these.

The same test would be run with the following.

  * X + Y where Y >= 1 threads in tserver and a non-yielding filter.
  * X/Y  where Y > 1 threads and a yielding filter.  



was (Author: kturner):
Based on comments [~phrocker] made on the PR, I think it would be nice to create a simple
test that contrast the responsiveness and throughput of this approach with lots of threads.
 Unless someone else wants to take a crack at it, I am thinking of creating a little test
that does the following.

 * Write lots of data to Accumulo.
 * Create X long running scans that filter alot and return little.
 * Create a few clients that constantly do random lookups.  Measure the time of these.

The same test would be run with the following.

  * X threads in tserver and a non-yielding filter.
  * X/Y  where Y > 1 threads and a yielding filter.  


> Allow iterators to interrupt themselves
> ---------------------------------------
>
>                 Key: ACCUMULO-4643
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4643
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>    Affects Versions: 1.8.1, 2.0.0
>            Reporter: Ivan Bella
>            Assignee: Ivan Bella
>              Labels: features
>             Fix For: 1.8.2, 2.0.0
>
>          Time Spent: 15h 20m
>  Remaining Estimate: 0h
>
> The idea here is to allow an iterator stack to send back a special key or throw a special
exception which will allow the tablet server to tear down the scan to be rebuilt later.  This
is to handle the case where an iterator is doing a lot of work without returning results to
avoid starving out other scans.
> There are two thoughts on how to do this:
> 1) A special "interrupt" key is returned from the getTopKey call that is detected in
the Tablet.nextBatch call, is not added to the results, but is used to add an unfinished range
and results in the remaining ranges to be deemed unfinished.
> 2) An special exception is thrown from the next or seek call that included the key of
the current position, and the same actions are taken as in 1).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message