hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prajakta Kalmegh <pkalm...@gmail.com>
Subject Re: Hadoop-Git-Eclipse
Date Fri, 15 Jun 2012 07:00:37 GMT
Hi Harsh

You were right - it was executed via the LocalJobRunner. What settings do I
need to change to ensure it goes through RM/NM? I did specify to use the
yarn framework in the mapred-site.xml. The eclipse console even shows to
check http://localhost:8080 to track job progress.

I will again cross-check all the parameters today with the ones specified
in the urls you gave me. By the way, I found this link <
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
>
which gives a list of deprecated properties. I have, in fact, specified a
lot of old properties in my core-site.xml, hdfs-site.xml, mapred-site.xml
files. Will that cause a problem?

Regards,
Prajakta



On Wed, Jun 13, 2012 at 8:18 PM, Harsh J <harsh@cloudera.com> wrote:

> Good to know your progress Prajakta!
>
> Did your submission surely go via the RM/NM or did it execute via the
> LocalJobRunner (logs show this classname)?
>
> You would ideally want to set the config "mapreduce.framework.name" to
> value "yarn" (in either config object before you use it, or in local
> mapred-site.xml), for it to use the YARN framework. A set of general
> configs for YARN deployment may be found at http://bit.ly/M2Eobz or at
> http://bit.ly/LW3Var.
>
> Does this help?
>
> On Wed, Jun 13, 2012 at 6:39 PM, Prajakta Kalmegh <pkalmegh@gmail.com>
> wrote:
> > Hi Harsh
> >
> > Appreciate the response. I was able to configure and implement basic
> JUnits
> > within eclipse and get some code running. Still getting familiar with the
> > new YARN and federation architecture.
> >
> > I was, however, not able to check the MR jobs submitted within eclipse
> for
> > a sample WordCount program on the
> > http://localhost:8088/<http://localhost:8080/>page. I am starting my
> > namenode/datanode/resourcemanager/nodemanager/historyserver as instructed
> > on the wiki page. And then executing JUnit tests from eclipse.
> >
> > I believe a single MR job will be submitted as a single application in
> the
> > new framework, right? The eclipse console shows a successful execution
> (the
> > details are pretty neat). However, the webpage shows 'No applications
> > submitted'. Do I have to tweak with any config properties to get this
> done?
> >
> > Please let me know.
> >
> > Regards,
> > Prajakta
> >
> >
> >
> >
> >
> > On Wed, Jun 13, 2012 at 6:09 PM, Harsh J <harsh@cloudera.com> wrote:
> >
> >> Hi Prajakta,
> >>
> >> I have Eclipse setup with M2E plugins. And once thats done, I merely
> >> clone a repo and import projects in via M2E's "Import existing maven
> >> projects" feature. This seems to work just fine for apache/hadoop's
> >> trunk.
> >>
> >> On Thu, Jun 7, 2012 at 5:18 PM, Prajakta Kalmegh <prkalmeg@in.ibm.com>
> >> wrote:
> >> > Hi
> >> >
> >> > I have done MapReduce programming using Eclipse before but now I need
> to
> >> > learn the Hadoop code internals for one of my projects.
> >> >
> >> > I have forked Hadoop from github (
> >> https://github.com/apache/hadoop-common
> >> > ) and need to configure it to work with Eclipse. All the links I could
> >> > find list steps for earlier versions of Hadoop. I am right now
> following
> >> > instructions given in these links:
> >> > - http://wiki.apache.org/hadoop/GitAndHadoop
> >> > - http://wiki.apache.org/hadoop/EclipseEnvironment
> >> > - http://wiki.apache.org/hadoop/HowToContribute
> >> >
> >> > Can someone please give me a link to the steps to be followed for
> getting
> >> > Hadoop (latest from trunk) started in Eclipse? I need to be able to
> >> commit
> >> > changes to my forked repository on github.
> >> >
> >> > Thanks in advance.
> >> > Regards,
> >> > Prajakta
> >>
> >>
> >>
> >> --
> >> Harsh J
> >>
>
>
>
> --
> Harsh J
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message