hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2346) DataNode should have timeout on socket writes.
Date Sat, 16 Feb 2008 01:48:07 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Raghu Angadi updated HADOOP-2346:

    Attachment: HADOOP-2346.patch

Update patch that I think is complete is attached.

This uses a pool of Selectors. This pool grows and shrinks with the number of threads blocked
on these sockets. No extra thread is used. Selector is closed if it stays inactive for 30
sec (next time some thread blocks or wakes up). This should not have any perf penalty.

Earlier I was thinking of using just one selector all threads waiting on read and one for
all threads waiting on write.. I don't think that complexity is required.

These streams can be used with other channels like pipes and UDP sockets. The unit test uses

I don't think this patch needs to wait till performance evaluation. 

Unit test: a new test is added to test timeout and read and write. Note that pretty much every
DFS unit test test these streams.

> DataNode should have timeout on socket writes.
> ----------------------------------------------
>                 Key: HADOOP-2346
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2346
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.15.1
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>             Fix For: 0.17.0
>         Attachments: HADOOP-2346.patch, HADOOP-2346.patch, HADOOP-2346.patch, HADOOP-2346.patch,
HADOOP-2346.patch, HADOOP-2346.patch
> If a client opens a file and stops reading in the middle, DataNode thread writing the
data could be stuck forever. For DataNode sockets we set read timeout but not write timeout.
I think we should add a write(data, timeout) method in IOUtils that assumes it the underlying
FileChannel is non-blocking.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message