hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1787) "Not enough xcievers" error should propagate to client
Date Fri, 13 May 2011 00:28:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032768#comment-13032768
] 

Aaron T. Myers commented on HDFS-1787:
--------------------------------------

Patch looks pretty good, Jon. A few comments:

# Looks to me like {{AtomicInteger maxTransfersErrors}} in {{BlockReader.java}} is unused.
# Error message "...exceeding max transfers..." in {{BlockReader.java}} should only be indented
4 spaces.
# In {{DFSInputStream.java}}, I think you should log the "Failed to connect..." error message
even if a {{MaxTransfersExceededException}} is encountered.
# In {{DFSOutputStream.java}}, is it really necessary that {{maxTransferErrors}} be an {{AtomicInteger}}
instead of an {{int}}? Since this is only used for testing, it doesn't seem like a big deal
to me if we miss a few increments, as you do in {{DFSInputStream.java}}.
# I don't understand the "{{if (errorIndex >= 0)}} check in {{DFSOutputStream.java}}. How
is this related?
# In {{DFSOutputStream.java}}, space between "{{>=}}" and "{{0}}".
# In {{DataXceiver.java}}, please only indent two spaces at "{{throw new MaxTransfersExceededException}}",
and four spaces for the line continuations.
# In {{DataXceiver.java}}, please indent two sapces at "{{sendErrorResponse}}", and put a
space before "{{t.getMessage()}}" on the same line.
# In {{DataXceiver.java}}, indent four spaces at "{{new DataOutputStream(...)}}".
# In {{TestDFSClientErrorMessages.java}} - Hadoop uses commons logging, not mortbay for logging
output.
In {{TestDFSClientErrorMessages.java}} - there's a bunch of commented-out code and TODOs.
Do you intend to implement these as part of this JIRA? Or is this a note for future work?

> "Not enough xcievers" error should propagate to client
> ------------------------------------------------------
>
>                 Key: HDFS-1787
>                 URL: https://issues.apache.org/jira/browse/HDFS-1787
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Jonathan Hsieh
>              Labels: newbie
>             Fix For: 0.23.0
>
>         Attachments: hdfs-1787.patch
>
>
> We find that users often run into the default transceiver limits in the DN. Putting aside
the inherent issues with xceiver threads, it would be nice if the "xceiver limit exceeded"
error propagated to the client. Currently, clients simply see an EOFException which is hard
to interpret, and have to go slogging through DN logs to find the underlying issue.
> The data transfer protocol should be extended to either have a special error code for
"not enough xceivers" or should have some error code for generic errors with which a string
can be attached and propagated.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message