hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3067) DFSInputStream 'pread' does not close its sockets
Date Mon, 24 Mar 2008 21:31:25 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Raghu Angadi updated HADOOP-3067:

    Attachment: HADOOP-3067.patch

Attached patch fixes one more bug that contributed to this problem :

{{BlockReader.checksumOk()}} is part of data read protocol and it should be inside BlockReader
class. Earlier it was invoked by {{FSInputStream.read()}} only because BlockReader did not
have access to the socket. Now the socket is stored. No API is changed. Since checksumOk()
was invoked only by regular read and not by the position read used by unit test. This combined
with the fact that the socket was not closed made the DataNodes wait on the socket.

> DFSInputStream 'pread' does not close its sockets
> -------------------------------------------------
>                 Key: HADOOP-3067
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3067
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>             Fix For: 0.17.0
>         Attachments: HADOOP-3067.patch, HADOOP-3067.patch
> {{DFSInputStream.read(int, buffer)}} does not close the sockets it opens. Main reason
this problem did not show up till now is that pread interface is not used much.
> TestCrcCorruption failure first reported in HADOOP-2902 is caused by this bug. Hadoop
0.17 uses more file descriptors for each thread waiting on socket io. Since client does not
close sockets, it leaves a lot of DataNode threads waiting in the unit test.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message