Return-Path: 
 <core-dev-return-47997-apmail-hadoop-core-dev-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org
Received: (qmail 30381 invoked from network); 5 Sep 2008 06:48:07 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 5 Sep 2008 06:48:07 -0000
Received: (qmail 77848 invoked by uid 500); 5 Sep 2008 06:48:04 -0000
Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org
Received: (qmail 77551 invoked by uid 500); 5 Sep 2008 06:48:02 -0000
Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:core-dev-help@hadoop.apache.org>
List-Unsubscribe: <mailto:core-dev-unsubscribe@hadoop.apache.org>
List-Post: <mailto:core-dev@hadoop.apache.org>
List-Id: <core-dev.hadoop.apache.org>
Reply-To: core-dev@hadoop.apache.org
Delivered-To: mailing list core-dev@hadoop.apache.org
Received: (qmail 77540 invoked by uid 99); 5 Sep 2008 06:48:02 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Sep 2008 23:48:02 -0700
X-ASF-Spam-Status: No, hits=-2000.0 required=10.0
	tests=ALL_TRUSTED
X-Spam-Check-By: apache.org
Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Sep 2008 06:47:13 +0000
Received: from brutus (localhost [127.0.0.1])
	by brutus.apache.org (Postfix) with ESMTP id 3DF38234C1CE
	for <core-dev@hadoop.apache.org>; Thu,  4 Sep 2008 23:47:44 -0700 (PDT)
Message-ID: <1914447473.1220597264239.JavaMail.jira@brutus>
Date: Thu, 4 Sep 2008 23:47:44 -0700 (PDT)
From: "Sharad Agarwal (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Updated: (HADOOP-3570) Including user specified jar files in
 the client side classpath path in Hadoop 0.17 streaming
In-Reply-To: <1307039378.1213638344969.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Virus-Checked: Checked by ClamAV on apache.org


     [ https://issues.apache.org/jira/browse/HADOOP-3570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sharad Agarwal updated HADOOP-3570:
-----------------------------------

    Attachment: 3570_v2.patch

Setting the classpath in JobClient.configureCommandLineOptions. I think it would be the better place than GenericOptionsParser. The classloader is set in the thread context as well as in the conf. 
Also, modified an existing test case to test this.

> Including user specified jar files in the client side classpath path in Hadoop 0.17 streaming
> ---------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3570
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3570
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/streaming
>    Affects Versions: 0.17.0
>            Reporter: Suhas Gogate
>            Assignee: Mahadev konar
>             Fix For: 0.19.0
>
>         Attachments: 3570_v1.patch, 3570_v2.patch
>
>
> Hadoop 0.17 streaming allows specifying multiple user specific jar files on the command line along with the hadoop-streaming.jar. These user specific jar files are not automatically  added on the client side class path when hadoop runs the streaming job
> For example, in the following streaming job <user-app.jar> is not automatically included in client side path before executing the streaming job
>  
> hadoop  jar -libjars <user-app.jar> $HADOOP_HOME/hadoop-streaming.jar \
>         -input </input-data-path> \
>         -output </output data path> \
>         -inputformat  <Input-format> \
>         -outputformat <OutputFormat> \
>         -mapper /bin/cat -reducer /bin/cat \
>         -additionalconfspec  <xxxx> \
>         -jobconf mapred.reduce.tasks=3\
> Workaround:
> Current work around is to set these user specific jar files in
> HADOOP_CLASSPATH environment variable on the client side before running above example.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.