hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bertrand Dechoux (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10436) ToolRunner is not thread-safe
Date Mon, 07 Apr 2014 21:14:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962263#comment-13962263

Bertrand Dechoux commented on HADOOP-10436:

Actually, the issue is even wider. All tools relying on OptionBuilder (and most do) are not
thread safe...

Fixing ToolRunner is not the solution to the real problem because ToolRunner is often use
in practice to run tools...

> ToolRunner is not thread-safe
> -----------------------------
>                 Key: HADOOP-10436
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10436
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>            Reporter: Ajay Chitre
>            Assignee: Tsuyoshi OZAWA
>         Attachments: HADOOP-10436.1.patch
> ToolRunner class is not thread-safe because it uses GenericOptionsParser.  The constructor
of GenericOptionsParser uses 'OptionBuilder' which is a singleton class that uses instance
variables.  In other words, OptionBuilder is NOT thread safe.  As a result, when multiple
Hadoop jobs are triggered simultaneously using ToolRunner they end up stepping on each other.
> The easiest way to fix it is by making 'buildGeneralOptions' synchronized in GenericOptionsParser.
> private static synchronized Options buildGeneralOptions(Options opts) {
> If this seems like the correct way of fixing this, either we can provide a patch or someone
can quickly fix it.  Thanks.
> Ajay Chitre
> achitre@cisco.com
> Virendra Singh
> virsingh@cisco.com

This message was sent by Atlassian JIRA

View raw message