accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3931) Repeated invocations of BatchScanner iterator().hasNext() causes MAC to become unresponsive
Date Wed, 29 Jul 2015 17:29:05 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646461#comment-14646461
] 

Keith Turner commented on ACCUMULO-3931:
----------------------------------------

Like Josh said, I would not assume recreating the batch scanner is expensive.  The most expensive
part may be creating the thread pool, but like Josh said that probably fast compared to RPCs.
  If you find otherwise, please let us know.

For better or worse, some expensive resources are static in Accumuo.  The tablet location
cache and tserver connection pool are both static.  Unrelated, but [~dlmarion] has found the
timeout on tserver connection pool was too aggressive.  I think he made a fix for that, but
I can not remember.   Anyway these static resources are not impacted when you recreate a batch
scanner.

> Repeated invocations of BatchScanner iterator().hasNext() causes MAC to become unresponsive
> -------------------------------------------------------------------------------------------
>
>                 Key: ACCUMULO-3931
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3931
>             Project: Accumulo
>          Issue Type: Bug
>          Components: mini
>    Affects Versions: 1.6.3, 1.7.0
>            Reporter: Russ Weeks
>            Assignee: Eric Newton
>         Attachments: tablet_server_0.txt, tablet_server_1.txt, test_runner_stack.txt
>
>
> Steps to reproduce:
> # Instantiate a MiniAccumuloCluster with at least two tablet servers
> # Create a table; add splits to the table.
> # Add some mutations to the table distributed across the splits; flush the mutations.
> # Create a BatchScanner across the full range of the table.
> # Assert that the batch scanner has at least one KV pair by calling {{scanner.iterator().hasNext()}}
> # Repeat.
> It doesn't seem to matter if you close the scanner and create a new one in between calls
to hasNext, or if you re-seek the same scanner, or if the scanner is created in a static context
and re-used by multiple tests or created by each test. Eventually you will see that the {{TabletServerBatchReaderIterator}}
gets stuck polling its resultsQueue, preventing further tests from running.
> This happens roughyl 20% of the time in 1.6 when I run {{mvn clean test -Dtest=org.apache.accumulo.minicluster.MultipleHasNextTest
--projects minicluster}}, maybe less often in 1.7, and 100% of the time when I try to use
the MAC in my company's product build environment, which uses gradle.
> (I'll update with a link to a failing unit test as soon as I get an issue ID)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message