hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4530) return buffer into direct bufferPool in BlockReaderLocal as possible
Date Tue, 19 Mar 2013 20:47:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606775#comment-13606775
] 

Ted Yu commented on HDFS-4530:
------------------------------

In BlockReaderLocal ctor:
{code}
    slowReadBuff = bufferPool.getBuffer(bytesPerChecksum * chunksPerChecksumRead);
    checksumBuff = bufferPool.getBuffer(checksumSize * chunksPerChecksumRead);
    // Initially the buffers have nothing to read.
    slowReadBuff.flip();
    checksumBuff.flip();
{code}
It is assumed that bufferPool.getBuffer() would always return ByteBuffer successfully. I think
this can be improved so that we respect -XX:MaxDirectMemorySize setting.
                
> return buffer into direct bufferPool in BlockReaderLocal as possible
> --------------------------------------------------------------------
>
>                 Key: HDFS-4530
>                 URL: https://issues.apache.org/jira/browse/HDFS-4530
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 3.0.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HDFS-4530.txt, HDFS-4530-v2.txt, HDFS-4530-v3.txt
>
>
> {code}
>   public synchronized void close() throws IOException {
>     dataIn.close();
>     if (checksumIn != null) {
>       checksumIn.close();
>     }
>     if (slowReadBuff != null) {
>       bufferPool.returnBuffer(slowReadBuff);
>       slowReadBuff = null;
>     }
>     if (checksumBuff != null) {
>       bufferPool.returnBuffer(checksumBuff);
>       checksumBuff = null;
>     }
>     startOffset = -1;
>     checksum = null;
>   }
> {code}
> If there's an IOException occurred in dataIn.close(), then the slowReadBuff&checksumBuff
could not be returned anymore.  let's make a trivial change to reduce this risk.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message