hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andre Kelpe <ake...@concurrentinc.com>
Subject Re: mr1 and mr2
Date Wed, 14 May 2014 10:31:16 GMT
May I recommend using Cascading instead of using MR directly? Cascading
supports Hadoop 1.x and Hadoop 2.x based distros and you don't have to
wrestle with these things all the time: http://www.cascading.org/ It's OSS,
ASL v2 licensed and all the good stuff.

- André


On Sun, May 11, 2014 at 1:52 AM, Tony Dean <Tony.Dean@sas.com> wrote:

>  Hi,
>
>
>
> I am trying to write a Java application that works with either MR1 and
> MR2.  At the present I have MR2 (YARN) implementation deployed and
> running.  I am using mapred API.  I believe that I read mapred and
> mapreduce APIs are compatible so either should work.  The only thing that
> is different is the configuration properties that need to be specified
> depending on whether the back-end is MR1 or MR2. BTW: I’m using CDH 4.6
> (Hadoop 2.0).
>
>
>
> My problem is that I can’t seem to submit a job to the cluster.  It always
> runs locally.  I setup JobConf with appropriate properties and submit the
> jobs using JobClient.  The properties that I set on JobConf are as follows:
>
>
>
> mapreduce.jobtracker.address=host:port (I know this is for MR1, but I’m
> trying everything)
>
> mapreduce.framework.name=yarn
>
> yarn.resourcemanager.address=host:port
>
> yarn.resourcemanager.host=host:port
>
>
>
> The last 2 are the same but I read 2 different ways to set it in different
> conflicting documentations.
>
>
>
> Anyway, can someone explain how to get this seemingly simple deployment to
> work?  What am I missing?
>
>
>
> Thanks!!!
>



-- 
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com

Mime
View raw message