hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6227) Short circuit read failed due to ClosedChannelException
Date Fri, 11 Apr 2014 17:58:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966878#comment-13966878

Colin Patrick McCabe commented on HDFS-6227:

bq. I quickly checked the code, in ShortCircuitCache#unref, we close the replica when the
ref count is 0, but I did not find the corresponding code to remove the replica object. Thus
is it possible that the cause of the issue is a closed ShortCircuitReplica object was still
retrieved from the ShortCircuitCache and used for reading? Colin Patrick McCabe, could you
provide some input?

That should not be possible.  See this code in {{ShortCircuitCache#unref}}:

      int newRefCount = --replica.refCount;
      if (newRefCount == 0) {
        // Close replica, since there are no remaining references to it.
            "Replica " + replica + " reached a refCount of 0 without " +
            "being purged");

Notice that we would throw a precondition exception if the replica hadn't been purged.  ("Purged"
means that it has been removed from the cache and will not be handed out to new readers.)
 There is no other path to calling {{ShortCircuitReplica#close}}.

Can you say a little bit more about the platform and version that you saw this on?  Can you
reproduce it?  Also, are there any other messages in the log?

> Short circuit read failed due to ClosedChannelException
> -------------------------------------------------------
>                 Key: HDFS-6227
>                 URL: https://issues.apache.org/jira/browse/HDFS-6227
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.4.0
>            Reporter: Jing Zhao
> While running tests in a single node cluster, where short circuit read is enabled and
multiple threads may read the same file concurrently, one of the read got ClosedChannelException
and failed. Full exception trace see comment.

This message was sent by Atlassian JIRA

View raw message