hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Something Something <mailinglist...@gmail.com>
Subject Re: Is Hadoop's TooRunner thread-safe?
Date Wed, 19 Mar 2014 17:09:54 GMT
Any thoughts on this?  Confirm or Deny it's an issue.. may be?


On Mon, Mar 17, 2014 at 11:43 AM, Something Something <
mailinglists19@gmail.com> wrote:

> I would like to trigger a few Hadoop jobs simultaneously.  I've created a
> pool of threads using Executors.newFixedThreadPool.  Idea is that if the
> pool size is 2, my code will trigger 2 Hadoop jobs at the same exact time
> using 'ToolRunner.run'.  In my testing, I noticed that these 2 threads
> keep stepping on each other.
>
> When I looked under the hood, I noticed that ToolRunner creates
> GenericOptionsParser which in turn calls a static method
> 'buildGeneralOptions'.  This method uses 'OptionBuilder.withArgName'
> which uses an instance variable called, 'argName'.  This doesn't look
> thread safe to me and I believe is the root cause of issues I am running
> into.
>
> Any thoughts?
>

Mime
View raw message