hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11574) Uber-JIRA: improve Hadoop network resilience & diagnostics
Date Wed, 18 Feb 2015 10:20:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325677#comment-14325677
] 

Steve Loughran commented on HADOOP-11574:
-----------------------------------------

If you look at the open JIRA list, especially related to net and ipc, there's a vast collection
of minor JIRAs, related to
# diagnostics and failure handling (exception swallowing, error text)
# reacting to IPv6 addresses. Even without IPv6 support, the code should not be surprised
to see them, and fail meaningfully.
# messy teardown/cleanup, including some potential blocks & deadlocks.

None of these is significant (e.g multi-NIC support), but together they'd make for a network
client more resilient to config problems & slightly easier to debug when things are playing
up. I think its the fact that they are so minor that nobody ever sits down to fix them. Together
they'd be good.

This JIRA can simply act as a place to aggregate/link those outstanding issues under the common
theme of resilience and diagnostics. things like performance & checksums would be separate
bits of work altogether

> Uber-JIRA: improve Hadoop network resilience & diagnostics
> ----------------------------------------------------------
>
>                 Key: HADOOP-11574
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11574
>             Project: Hadoop Common
>          Issue Type: Task
>          Components: net
>    Affects Versions: 2.6.0
>            Reporter: Steve Loughran
>
> Improve Hadoop's resilience to bad network conditions/problems, including
> * improving recognition of problem states
> * improving diagnostics
> * better handling of IPv6 addresses, even if the protocol is unsupported.
> * better behaviour client-side when there are connectivity problems. (i.e while some
errors you can spin on, DNS failures are not on the list)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message