hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rab ra <rab...@gmail.com>
Subject Launching Hadoop map reduce job from a servlet
Date Fri, 16 Jan 2015 17:53:09 GMT

I have a servlet program deployed in jetty server listening to the port
8080. As soon as a request arrives from a client, it parses the request,
and instantiate MR program that is to be launched in Hadoop cluster. Here,
I cannot launch the hadoop job using hadoop command as 'hadoop jar <jar
file> .....". From servlet code, I instantiate the MR main program that
implements Tools and contains Mapper and Reducer classes in it.

My issue is that though the job is launched, it always uses Local
JobRunner. I do have hadoop installed with all the configuration files
contains right information in it. For instance, in my mapred-site.xml, i
have setup 'yarn' as my mapreduce framework.

With the current configuration setup, I was able to submit jobs to yarn
through hadoop command. But i want to achieve this through 'java' command.

1. How can I do it? If there is any pointer/link, please share it.
2. I tried to setup all the configuration inside the code something like as

But somehow, it seems that these information is not cascading to job
despite creating job instance with the above configuration. So, I am
struggling to make hadoop configuration to java application.

I would be grateful to you for any help to fix this issue


View raw message