accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (ACCUMULO-4063) TServer should verify its lock is held before answering any RPC
Date Tue, 19 Jan 2016 17:12:39 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107019#comment-15107019
] 

Eric Newton edited comment on ACCUMULO-4063 at 1/19/16 5:12 PM:
----------------------------------------------------------------

Any chance you have a log message?

I'm trying to discern if the tserver is detecting a master with an out-of-date or unknown
lock, or if the tserver is detecting a missing, loss or failure to get its own lock.

These are very different scenarios, with different fixes.  The title of this JIRA ticket implies
that the tserver is detecting its own missing lock, but I have not ever seen this in practice,
except during failures. But, I don't use Ambari, so it might happen all the time.



was (Author: ecn):
Any chance you have a log message?

I'm trying to discern if the tserver is detecting a master with an out-of-date or unknown
lock, or if the tserver is detecting a missing, loss or failure to get it's own lock.

These are very different scenarios, with different fixes.  The title of this JIRA ticket implies
that the tserver is detecting its own missing lock, but I have not ever seen this in practice,
except during failures. But, I don't use Ambari, so it might happen all the time.


> TServer should verify its lock is held before answering any RPC
> ---------------------------------------------------------------
>
>                 Key: ACCUMULO-4063
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4063
>             Project: Accumulo
>          Issue Type: Bug
>          Components: rpc, tserver
>    Affects Versions: 1.7.0
>            Reporter: Josh Elser
>
> Noticing the following:
> {noformat}
> 2015-11-19 22:42:47,130 [thrift.ProcessFunction] ERROR: Internal error processing flush
> org.apache.thrift.TException: Lock not acquired
>         at org.apache.accumulo.server.rpc.RpcWrapper$1.invoke(RpcWrapper.java:51)
>         at com.sun.proxy.$Proxy21.flush(Unknown Source)
>         at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$flush.getResult(TabletClientService.java:2671)
>         at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$flush.getResult(TabletClientService.java:2657)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>         at org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:63)
>         at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:516)
>         at org.apache.accumulo.server.rpc.CustomNonBlockingServer$1.run(CustomNonBlockingServer.java:78)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
>         at java.lang.Thread.run(Thread.java:745)
> 2015-11-19 22:42:47,402 [tserver.TabletServer$ThriftClientHandler] ERROR: org.apache.accumulo.tserver.session.MultiScanSession
cannot be cast to org.apache.accumulo.tserver.session.ScanSession
> java.lang.ClassCastException: org.apache.accumulo.tserver.session.MultiScanSession cannot
be cast to org.apache.accumulo.tserver.session.ScanSession
>         at org.apache.accumulo.tserver.TabletServer$ThriftClientHandler.closeScan(TabletServer.java:574)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:497)
>         at org.apache.accumulo.core.trace.wrappers.RpcServerInvocationHandler.invoke(RpcServerInvocationHandler.java:46)
>         at org.apache.accumulo.server.rpc.RpcWrapper$1.invoke(RpcWrapper.java:47)
>         at com.sun.proxy.$Proxy21.closeScan(Unknown Source)
>         at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$closeScan.getResult(TabletClientService.java:2309)
>         at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$closeScan.getResult(TabletClientService.java:2295)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>         at org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:63)
>         at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:516)
>         at org.apache.accumulo.server.rpc.CustomNonBlockingServer$1.run(CustomNonBlockingServer.java:78)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
>         at java.lang.Thread.run(Thread.java:745)
> {noformat}
> a {{flush}} rpc came in and was denied, but then a {{closeScan}} came in and threw an
unrelated error (ignore the classcast for now).
> Are there _any_ RPCs that a tabletserver should answer before it holds its lock?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message