hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6639) Process hangs in LocatedFileStatusFetcher if FileSystem.get throws
Date Wed, 20 Apr 2016 22:53:25 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15250889#comment-15250889
] 

Ryan Blue commented on MAPREDUCE-6639:
--------------------------------------

[~rkanter], can you take a look at this?

> Process hangs in LocatedFileStatusFetcher if FileSystem.get throws
> ------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6639
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6639
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 2.7.2
>            Reporter: Ryan Blue
>            Assignee: Ryan Blue
>         Attachments: MAPREDUCE-6639.2.patch
>
>
> ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread functions,
[{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
doesn't catch exceptions (the callbacks do). When an exception is thrown, the thread exists
and doesn't signal the error to the calling thread, which continues waiting to be signaled.
This can happen when a FS implementation cannot be found.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message