Return-Path: 
 <user-return-3623-apmail-hadoop-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 7F560E1A0
	for <apmail-hadoop-user-archive@minotaur.apache.org>;
 Fri, 14 Dec 2012 05:58:03 +0000 (UTC)
Received: (qmail 42205 invoked by uid 500); 14 Dec 2012 05:57:58 -0000
Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org
Received: (qmail 42019 invoked by uid 500); 14 Dec 2012 05:57:58 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 41991 invoked by uid 99); 14 Dec 2012 05:57:57 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Dec 2012 05:57:57 +0000
X-ASF-Spam-Status: No, hits=1.7 required=5.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of manoj444@gmail.com designates
 209.85.210.176 as permitted sender)
Received: from [209.85.210.176] (HELO mail-ia0-f176.google.com)
 (209.85.210.176)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Dec 2012 05:57:49 +0000
Received: by mail-ia0-f176.google.com with SMTP id y26so2755868iab.35
        for <user@hadoop.apache.org>; Thu, 13 Dec 2012 21:57:28 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        bh=+qJTC3FyJWscXQTV/2rKAcmgmVvqRjHMu4vY0qFKHJA=;
        b=mppa6O5H0YWfhpeAGh0ajGhW2zI3R7jHglJiGHtLHNY4SBXLSHfVTMtncEE+/GyAwF
         GFZuyJoGnW/TQzw4UPIcYLuF6rXrbZ2D4qwfQvfbnc1/hmvUSWFupB3oUiG6ueDJSDW6
         zCPlVjJ4tWk8UKqvWzv/f46EnJqxMosURPv6SoRE0ymzDQT5J6DELnZsEUhPM9gP3cHu
         87MCTP1UVDKPfpnisYvEuIyhD4YTgFo/qXILD84suJJEcKyS+MrrHmhV/Dmb/sd866re
         6hVpxeRA8viGkKvsWPBktYCPZ6EqI8UsRV+0V4jgFdJyYIHXfyRIQhLODBQI4JcoWHhC
         btGw==
Received: by 10.50.152.240 with SMTP id vb16mr615124igb.46.1355464648527; Thu,
 13 Dec 2012 21:57:28 -0800 (PST)
MIME-Version: 1.0
Received: by 10.50.181.232 with HTTP; Thu, 13 Dec 2012 21:57:08 -0800 (PST)
In-Reply-To: <085401cdd9b4$f48a7df0$dd9f79d0$@yahoo.com>
References: <085401cdd9b4$f48a7df0$dd9f79d0$@yahoo.com>
From: Manoj Babu <manoj444@gmail.com>
Date: Fri, 14 Dec 2012 11:27:08 +0530
Message-ID: 
 <CACvfG=rcj3brA1Ei5_-nZcV6snEzFn5Rho-jUq21fR_rC+Udrw@mail.gmail.com>
Subject: Re: How to submit Tool jobs programatically in parallel?
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=e89a8f3ba01be18e5c04d0c9b418
X-Virus-Checked: Checked by ClamAV on apache.org

--e89a8f3ba01be18e5c04d0c9b418
Content-Type: text/plain; charset=ISO-8859-1

David,

You try like below instead of runJob() you can try submitJob().

JobClient jc = new JobClient(job);
jc.submitJob(job);



Cheers!
Manoj.



On Fri, Dec 14, 2012 at 10:09 AM, David Parks <davidparks21@yahoo.com>wrote:

> I'm submitting unrelated jobs programmatically (using AWS EMR) so they run
> in parallel.
>
> I'd like to run an s3distcp job in parallel as well, but the interface to
> that job is a Tool, e.g. ToolRunner.run(...).
>
> ToolRunner blocks until the job completes though, so presumably I'd need to
> create a thread pool to run these jobs in parallel.
>
> But creating multiple threads to submit concurrent jobs via ToolRunner,
> blocking on the jobs completion, just feels improper. Is there an
> alternative?
>
>

--e89a8f3ba01be18e5c04d0c9b418
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

David,<div><br></div><div><div>You try like below=A0instead=A0of runJob() y=
ou can try submitJob().</div><div><br></div><div>JobClient jc =3D new JobCl=
ient(job);</div><div>jc.submitJob(job);</div></div><div><br></div><div><br>=
</div>

<div class=3D"gmail_extra"><br clear=3D"all"><div>Cheers!<div>Manoj.</div><=
/div><br>
<br><br><div class=3D"gmail_quote">On Fri, Dec 14, 2012 at 10:09 AM, David =
Parks <span dir=3D"ltr">&lt;<a href=3D"mailto:davidparks21@yahoo.com" targe=
t=3D"_blank">davidparks21@yahoo.com</a>&gt;</span> wrote:<br><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;p=
adding-left:1ex">

I&#39;m submitting unrelated jobs programmatically (using AWS EMR) so they =
run<br>
in parallel.<br>
<br>
I&#39;d like to run an s3distcp job in parallel as well, but the interface =
to<br>
that job is a Tool, e.g. ToolRunner.run(...).<br>
<br>
ToolRunner blocks until the job completes though, so presumably I&#39;d nee=
d to<br>
create a thread pool to run these jobs in parallel.<br>
<br>
But creating multiple threads to submit concurrent jobs via ToolRunner,<br>
blocking on the jobs completion, just feels improper. Is there an<br>
alternative?<br>
<br>
</blockquote></div><br></div>

--e89a8f3ba01be18e5c04d0c9b418--