hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: Problem while running eclipse-files for Next Gen Mapreduce branch
Date Fri, 08 Jul 2011 16:00:22 GMT
Hey Josh,

I just updated now, and did not realize that it had been fixed.  Everything mostly looks good.

Then name of the tar ball is now -all.tar.gz not -bin.tar.gz

It also doesn't talk about how you need to install the MAPRED, COMMON, and HDFS tar balls
as well to get things to work, although it is implied in the exports.

The biggest issue I have is that the default configs are not good enough to actually make
map/reduce work with the Resource Manager.  I realize that a lot of this is likely going to
be fixed before mrv2 is released, but as it stands you can only run in local mode with the
INSTALL and README files which fall back on the MRV1 code to do it.  If that is the intent
of the documentation then it is fine, but it would be nice some where, even if it is just
on twiki, to indicate what configs are required to setup a single node cluster.

If you want the configs I have been using I am happy to send it to you.

--Bobby Evans


On 7/8/11 9:58 AM, "Josh Wills" <jwills@cloudera.com> wrote:

Hey Bobby-

Vinod and I did a cleanup pass over INSTALL and yarn/README in
https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a look and
check if there's anything else we need to add/update.

Josh

On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <evans@yahoo-inc.com> wrote:

> I mapreduce/INSTALL also has some important information in it, and be aware
> that you do not have to install the avro plugin any more.  Maven can
> download it and install it automatically now, but the README was never
> updated.  Also be sure to install protocol buffers.  The build will fail
> without it.
>
> --Bobby
>
> On 7/8/11 9:04 AM, "Josh Wills" <jwills@cloudera.com> wrote:
>
> You want to generate them using mvn instead.  See the mapreduce/yarn/README
> file for how to do it.
>
> On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <devaraj.k@huawei.com> wrote:
>
> > Hi,
> >
> >
> >
> >   I am getting this below errors when I try to generate eclipse files
> using
> > eclipse-files target. Can anybody help me?
> >
> >
> >
> >
> >
> > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
> >
> > ivy-download:
> >      [get] Getting:
> > <http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> >      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
> >      [get] Not modified - so not downloaded
> >
> > ivy-init-dirs:
> >
> > ivy-probe-antlib:
> >
> > ivy-init-antlib:
> >
> > ivy-init:
> > [ivy:configure] :: Ivy non official version -  ::
> > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> > [ivy:configure] :: loading settings :: file =
> > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
> >
> > ivy-resolve-common:
> > [ivy:resolve]
> > [ivy:resolve] :: problems summary ::
> > [ivy:resolve] :::: WARNINGS
> > [ivy:resolve]                 module not found:
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> > [ivy:resolve]         ==== apache-snapshot: tried
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
> > [ivy:resolve]         ==== maven2: tried
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > SHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> > HOT/yarn-server-common-1.0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > SHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> > HOT/yarn-server-common-1.0-SNAPSHOT.jar
> > [ivy:resolve]                 module not found:
> > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT
> > [ivy:resolve]         ==== apache-snapshot: tried
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> >
> >
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> > .0-SNAPSHOT.pom>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >
> >
> op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
> > 0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> >
> >
> org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce
> > -client-core.jar:
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> >
> >
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> > .0-SNAPSHOT.jar>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> >
> >
> op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
> > 0-SNAPSHOT.jar
> > [ivy:resolve]         ==== maven2: tried
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor
> > e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom>
> >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core
> > /1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> >
> >
> org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce
> > -client-core.jar:
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor
> > e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar>
> >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core
> > /1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar
> > [ivy:resolve]                 module not found:
> > org.apache.hadoop#yarn-common;1.0-SNAPSHOT
> > [ivy:resolve]         ==== apache-snapshot: tried
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> > org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar:
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar
> > [ivy:resolve]         ==== maven2: tried
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya
> > rn-common-1.0-SNAPSHOT.pom>
> >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar
> > n-common-1.0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> > org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar:
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya
> > rn-common-1.0-SNAPSHOT.jar>
> >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar
> > n-common-1.0-SNAPSHOT.jar
> > [ivy:resolve]
> > ::::::::::::::::::::::::::::::::::::::::::::::
> >
> > [ivy:resolve]                 ::          UNRESOLVED DEPENDENCIES
> > ::
> >
> > [ivy:resolve]
> > ::::::::::::::::::::::::::::::::::::::::::::::
> >
> > [ivy:resolve]                 ::
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT: not found
> > [ivy:resolve]                 ::
> > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT: not found
> > [ivy:resolve]                 ::
> > org.apache.hadoop#yarn-common;1.0-SNAPSHOT:
> > not found
> > [ivy:resolve]
> > ::::::::::::::::::::::::::::::::::::::::::::::
> >
> > [ivy:resolve]
> > [ivy:resolve]
> > [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> >
> >
> >
> >
> > Devaraj K
> >
> >
> >
> ----------------------------------------------------------------------------
> > ---------------------------------------------------------
> > This e-mail and its attachments contain confidential information from
> > HUAWEI, which
> > is intended only for the person or entity whose address is listed above.
> > Any
> > use of the
> > information contained herein in any way (including, but not limited to,
> > total or partial
> > disclosure, reproduction, or dissemination) by persons other than the
> > intended
> > recipient(s) is prohibited. If you receive this e-mail in error, please
> > notify the sender by
> > phone or email immediately and delete it!ss
> >
> >
> >
> >
> >
> >
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message