hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haoyuan Li <haoyuan...@gmail.com>
Subject Re: Problem while running eclipse-files for Next Gen Mapreduce branch
Date Sat, 09 Jul 2011 00:04:46 GMT
Thank you Josh. It will be great if you could send me your configs for
pseudo-distributed mode.

Best,

Haoyuan

On Sat, Jul 9, 2011 at 12:25 AM, Josh Wills <jwills@cloudera.com> wrote:

> Haoyuan,
>
> Did you specify HADOOP_CONF_DIR, YARN_CONF_DIR, etc. as per Step 8 of the
> INSTALL doc? I've been running my mrv2 cluster in pseudo-distributed mode
> on
> my linux box- happy to send you my configs if you'd like to play. There are
> also a couple of changes you need to make to the yarn config files to get
> real MR running, as Bobby alluded to in his email.
>
> To run the example in the INSTALL file, you'll need to patch in this
> change:
> https://issues.apache.org/jira/browse/MAPREDUCE-2644
>
> Josh
>
> On Fri, Jul 8, 2011 at 8:07 AM, Haoyuan Li <haoyuan.li@gmail.com> wrote:
>
> > Hi Josh,
> >
> > Which configuration file could tell Yarn the location of HDFS? This could
> > be
> > found on
> >
> >
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
> >
> > Thank you.
> >
> > Best,
> >
> > Haoyuan
> >
> > On Fri, Jul 8, 2011 at 10:58 PM, Josh Wills <jwills@cloudera.com> wrote:
> >
> > > Hey Bobby-
> > >
> > > Vinod and I did a cleanup pass over INSTALL and yarn/README in
> > > https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a
> look
> > > and
> > > check if there's anything else we need to add/update.
> > >
> > > Josh
> > >
> > > On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <evans@yahoo-inc.com>
> > wrote:
> > >
> > > > I mapreduce/INSTALL also has some important information in it, and be
> > > aware
> > > > that you do not have to install the avro plugin any more.  Maven can
> > > > download it and install it automatically now, but the README was
> never
> > > > updated.  Also be sure to install protocol buffers.  The build will
> > fail
> > > > without it.
> > > >
> > > > --Bobby
> > > >
> > > > On 7/8/11 9:04 AM, "Josh Wills" <jwills@cloudera.com> wrote:
> > > >
> > > > You want to generate them using mvn instead.  See the
> > > mapreduce/yarn/README
> > > > file for how to do it.
> > > >
> > > > On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <devaraj.k@huawei.com>
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > >
> > > > >
> > > > >   I am getting this below errors when I try to generate eclipse
> files
> > > > using
> > > > > eclipse-files target. Can anybody help me?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
> > > > >
> > > > > ivy-download:
> > > > >      [get] Getting:
> > > > > <
> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> > > > >
> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> > > > >      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
> > > > >      [get] Not modified - so not downloaded
> > > > >
> > > > > ivy-init-dirs:
> > > > >
> > > > > ivy-probe-antlib:
> > > > >
> > > > > ivy-init-antlib:
> > > > >
> > > > > ivy-init:
> > > > > [ivy:configure] :: Ivy non official version -  ::
> > > > > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> > > > > [ivy:configure] :: loading settings :: file =
> > > > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
> > > > >
> > > > > ivy-resolve-common:
> > > > > [ivy:resolve]
> > > > > [ivy:resolve] :: problems summary ::
> > > > > [ivy:resolve] :::: WARNINGS
> > > > > [ivy:resolve]                 module not found:
> > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> > > > > [ivy:resolve]         ==== apache-snapshot: tried
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > > >
> > >
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > > >
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> > > > > [ivy:resolve]           -- artifact
> > > > >
> > >
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > > >
> > >
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > > >
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
> > > > > [ivy:resolve]         ==== maven2: tried
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > > > > SHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> > > > >
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> > > > > HOT/yarn-server-common-1.0-SNAPSHOT.pom
> > > > > [ivy:resolve]           -- artifact
> > > > >
> > >
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > > > > SHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> > > > >
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> > > > > HOT/yarn-server-common-1.0-SNAPSHOT.jar
> > > > > [ivy:resolve]                 module not found:
> > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT
> > > > > [ivy:resolve]         ==== apache-snapshot: tried
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > > >
> > > > >
> > > >
> > >
> >
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> > > > > .0-SNAPSHOT.pom>
> > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > > >
> > > > >
> > > >
> > >
> >
> op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
> > > > > 0-SNAPSHOT.pom
> > > > > [ivy:resolve]           -- artifact
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce
> > > > > -client-core.jar:
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > > >
> > > > >
> > > >
> > >
> >
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> > > > > .0-SNAPSHOT.jar>
> > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > > >
> > > > >
> > > >
> > >
> >
> op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
> > > > > 0-SNAPSHOT.jar
> > > > > [ivy:resolve]         ==== maven2: tried
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor
> > > > > e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom>
> > > > >
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core
> > > > > /1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom
> > > > > [ivy:resolve]           -- artifact
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce
> > > > > -client-core.jar:
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor
> > > > > e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar>
> > > > >
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core
> > > > > /1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar
> > > > > [ivy:resolve]                 module not found:
> > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT
> > > > > [ivy:resolve]         ==== apache-snapshot: tried
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > > > oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom>
> > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > > > op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom
> > > > > [ivy:resolve]           -- artifact
> > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar:
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > > > oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar>
> > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > > > op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar
> > > > > [ivy:resolve]         ==== maven2: tried
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya
> > > > > rn-common-1.0-SNAPSHOT.pom>
> > > > >
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar
> > > > > n-common-1.0-SNAPSHOT.pom
> > > > > [ivy:resolve]           -- artifact
> > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar:
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya
> > > > > rn-common-1.0-SNAPSHOT.jar>
> > > > >
> > > > >
> > > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar
> > > > > n-common-1.0-SNAPSHOT.jar
> > > > > [ivy:resolve]
> > > > > ::::::::::::::::::::::::::::::::::::::::::::::
> > > > >
> > > > > [ivy:resolve]                 ::          UNRESOLVED DEPENDENCIES
> > > > > ::
> > > > >
> > > > > [ivy:resolve]
> > > > > ::::::::::::::::::::::::::::::::::::::::::::::
> > > > >
> > > > > [ivy:resolve]                 ::
> > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT: not found
> > > > > [ivy:resolve]                 ::
> > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT: not
> > found
> > > > > [ivy:resolve]                 ::
> > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT:
> > > > > not found
> > > > > [ivy:resolve]
> > > > > ::::::::::::::::::::::::::::::::::::::::::::::
> > > > >
> > > > > [ivy:resolve]
> > > > > [ivy:resolve]
> > > > > [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE
> DETAILS
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Devaraj K
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> ----------------------------------------------------------------------------
> > > > > ---------------------------------------------------------
> > > > > This e-mail and its attachments contain confidential information
> from
> > > > > HUAWEI, which
> > > > > is intended only for the person or entity whose address is listed
> > > above.
> > > > > Any
> > > > > use of the
> > > > > information contained herein in any way (including, but not limited
> > to,
> > > > > total or partial
> > > > > disclosure, reproduction, or dissemination) by persons other than
> the
> > > > > intended
> > > > > recipient(s) is prohibited. If you receive this e-mail in error,
> > please
> > > > > notify the sender by
> > > > > phone or email immediately and delete it!ss
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message