Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=darose.net; h=message-id:date
	:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=darose.net; b=
	Kg94wKpiBnKqLFautEbxxJZnYPqv3CUvyWKhoYDBWJtbvnam65TUp2CHLUBr1uIV
	ONRE4Nloq+S+y8HgaiZzL5TQ1eqbIrgdvG5ANCV86CuysGOK/3qVfmH6wZkrLFrM
	7jb7M4C6dDZGSUKEuwFo9RMuHStrvSFQH6A8hywJNH0=
Message-ID: <4C9CC3E7.1040001@darose.net>
Date: Fri, 24 Sep 2010 11:29:43 -0400
From: David Rosenstrauch <darose@darose.net>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
 rv:1.9.2.9) Gecko/20100916 Lanikai/3.1.4
MIME-Version: 1.0
To: mapreduce-user@hadoop.apache.org
Subject: Re: JobClient using deprecated JobConf
References: <4C99CE4F.9040305@web.de>	<C8BFCDAE.1E5DD%amarsri@yahoo-inc.com>
	<AANLkTimtR5TX8eayXarQWSzw=nKO4eGo2UjxiEMUCosN@mail.gmail.com>
	<4C9B0EC2.50906@web.de>
	<AANLkTi=U-ZUB6mkP+RsG7r-dzU41foBBjg41E-WxQ4xV@mail.gmail.com>
	<4C9B8CB9.7050008@web.de>
 <AANLkTik0MSg0xJXjSvDnzXBpGG96eCrXAPrPk3QgG_hz@mail.gmail.com>
 <4C9CBFDB.5060208@web.de>
In-Reply-To: <4C9CBFDB.5060208@web.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 09/24/2010 11:12 AM, Martin Becker wrote:
> Hi James,
>
> I am trying to avoid to call any command line command. I want to submit
> a job from within a java application. If possible without packing any
> jar file at all. But I guess that will be necessary to allow Hadoop to
> load the specific classes. The tutorial definitely does not contain any
> explicit java code how to do this. Sorry, for not stating my problem
> clearly:
>
> Right now I want to use Eclipse to submit my job by doing using the "Run
> as..." dialog. Later I want to embed that part in a java application
> submitting configured jobs to a remote Hadoop system/cluster.
>
> Regards,
> Martin

This is very do-able.  (I do this now.)

Here is a skeleton for how it can be done:

public class JobSubmitter implements Tool {
	public static void main(String[] args) throws Exception {
		ToolRunner.run(new Configuration(), new JobSubmitter(), args);
	}

	public JobSubmitter() {
		<your code here>
	}

	public Configuration getConf() {
		return appConf;
	}

	public void setConf(Configuration conf) {
		this.appConf = conf;
	}

	public int run(String[] args) throws Exception {
		Job job = new Job(appConf);
		Configuration jobConf = job.getConfiguration();
		jobConf.set(<your code here>);
		<your code here>
		job.submit();
	}
}


re: "without packing any jar file at all":

If you use Tool/ToolRunner (as we are doing above), that lets your 
Hadoop app automatically handle some key command line args.  One them 
that you will use here is the -libjars argument.  If you use -libjars 
and specify a list of jars that contain your code, then ToolRunner will 
automatically take those jars and put them in the Distributed Cache on 
each task node, where they will get added to the classpath of every 
map/reduce task.

HTH,

DR