Return-Path: X-Original-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A358668BE for ; Fri, 8 Jul 2011 16:26:31 +0000 (UTC) Received: (qmail 12411 invoked by uid 500); 8 Jul 2011 16:26:31 -0000 Delivered-To: apmail-hadoop-mapreduce-dev-archive@hadoop.apache.org Received: (qmail 12260 invoked by uid 500); 8 Jul 2011 16:26:30 -0000 Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-dev@hadoop.apache.org Delivered-To: mailing list mapreduce-dev@hadoop.apache.org Received: (qmail 12248 invoked by uid 99); 8 Jul 2011 16:26:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 Jul 2011 16:26:30 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jwills@cloudera.com designates 209.85.220.176 as permitted sender) Received: from [209.85.220.176] (HELO mail-vx0-f176.google.com) (209.85.220.176) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 Jul 2011 16:26:26 +0000 Received: by vxd2 with SMTP id 2so2416278vxd.35 for ; Fri, 08 Jul 2011 09:26:04 -0700 (PDT) Received: by 10.52.106.232 with SMTP id gx8mr2915762vdb.236.1310142364094; Fri, 08 Jul 2011 09:26:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.169.3 with HTTP; Fri, 8 Jul 2011 09:25:44 -0700 (PDT) In-Reply-To: References: From: Josh Wills Date: Fri, 8 Jul 2011 09:25:44 -0700 Message-ID: Subject: Re: Problem while running eclipse-files for Next Gen Mapreduce branch To: mapreduce-dev@hadoop.apache.org Cc: "devaraj.k@huawei.com" Content-Type: multipart/alternative; boundary=bcaec547ca333791d004a7914a6c --bcaec547ca333791d004a7914a6c Content-Type: text/plain; charset=ISO-8859-1 Haoyuan, Did you specify HADOOP_CONF_DIR, YARN_CONF_DIR, etc. as per Step 8 of the INSTALL doc? I've been running my mrv2 cluster in pseudo-distributed mode on my linux box- happy to send you my configs if you'd like to play. There are also a couple of changes you need to make to the yarn config files to get real MR running, as Bobby alluded to in his email. To run the example in the INSTALL file, you'll need to patch in this change: https://issues.apache.org/jira/browse/MAPREDUCE-2644 Josh On Fri, Jul 8, 2011 at 8:07 AM, Haoyuan Li wrote: > Hi Josh, > > Which configuration file could tell Yarn the location of HDFS? This could > be > found on > > http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL > > Thank you. > > Best, > > Haoyuan > > On Fri, Jul 8, 2011 at 10:58 PM, Josh Wills wrote: > > > Hey Bobby- > > > > Vinod and I did a cleanup pass over INSTALL and yarn/README in > > https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a look > > and > > check if there's anything else we need to add/update. > > > > Josh > > > > On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans > wrote: > > > > > I mapreduce/INSTALL also has some important information in it, and be > > aware > > > that you do not have to install the avro plugin any more. Maven can > > > download it and install it automatically now, but the README was never > > > updated. Also be sure to install protocol buffers. The build will > fail > > > without it. > > > > > > --Bobby > > > > > > On 7/8/11 9:04 AM, "Josh Wills" wrote: > > > > > > You want to generate them using mvn instead. See the > > mapreduce/yarn/README > > > file for how to do it. > > > > > > On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K > wrote: > > > > > > > Hi, > > > > > > > > > > > > > > > > I am getting this below errors when I try to generate eclipse files > > > using > > > > eclipse-files target. Can anybody help me? > > > > > > > > > > > > > > > > > > > > > > > > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml > > > > > > > > ivy-download: > > > > [get] Getting: > > > > < > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar> > > > > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar > > > > [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar > > > > [get] Not modified - so not downloaded > > > > > > > > ivy-init-dirs: > > > > > > > > ivy-probe-antlib: > > > > > > > > ivy-init-antlib: > > > > > > > > ivy-init: > > > > [ivy:configure] :: Ivy non official version - :: > > > > http://ant.apache.org/ivy/ :: > > > > [ivy:configure] :: loading settings :: file = > > > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml > > > > > > > > ivy-resolve-common: > > > > [ivy:resolve] > > > > [ivy:resolve] :: problems summary :: > > > > [ivy:resolve] :::: WARNINGS > > > > [ivy:resolve] module not found: > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT > > > > [ivy:resolve] ==== apache-snapshot: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > > > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > > > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar > > > > [ivy:resolve] ==== maven2: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP > > > > SHOT/yarn-server-common-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS > > > > HOT/yarn-server-common-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP > > > > SHOT/yarn-server-common-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS > > > > HOT/yarn-server-common-1.0-SNAPSHOT.jar > > > > [ivy:resolve] module not found: > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT > > > > [ivy:resolve] ==== apache-snapshot: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > > > > > > > > > > > oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1 > > > > .0-SNAPSHOT.pom> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > > > > > > > > > > > op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1. > > > > 0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > > > > > > > > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce > > > > -client-core.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > > > > > > > > > > > oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1 > > > > .0-SNAPSHOT.jar> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > > > > > > > > > > > op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1. > > > > 0-SNAPSHOT.jar > > > > [ivy:resolve] ==== maven2: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor > > > > e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core > > > > /1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > > > > > > > > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce > > > > -client-core.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor > > > > e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core > > > > /1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar > > > > [ivy:resolve] module not found: > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT > > > > [ivy:resolve] ==== apache-snapshot: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar > > > > [ivy:resolve] ==== maven2: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya > > > > rn-common-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar > > > > n-common-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya > > > > rn-common-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar > > > > n-common-1.0-SNAPSHOT.jar > > > > [ivy:resolve] > > > > :::::::::::::::::::::::::::::::::::::::::::::: > > > > > > > > [ivy:resolve] :: UNRESOLVED DEPENDENCIES > > > > :: > > > > > > > > [ivy:resolve] > > > > :::::::::::::::::::::::::::::::::::::::::::::: > > > > > > > > [ivy:resolve] :: > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT: not found > > > > [ivy:resolve] :: > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT: not > found > > > > [ivy:resolve] :: > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT: > > > > not found > > > > [ivy:resolve] > > > > :::::::::::::::::::::::::::::::::::::::::::::: > > > > > > > > [ivy:resolve] > > > > [ivy:resolve] > > > > [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS > > > > > > > > > > > > > > > > > > > > Devaraj K > > > > > > > > > > > > > > > > > > ---------------------------------------------------------------------------- > > > > --------------------------------------------------------- > > > > This e-mail and its attachments contain confidential information from > > > > HUAWEI, which > > > > is intended only for the person or entity whose address is listed > > above. > > > > Any > > > > use of the > > > > information contained herein in any way (including, but not limited > to, > > > > total or partial > > > > disclosure, reproduction, or dissemination) by persons other than the > > > > intended > > > > recipient(s) is prohibited. If you receive this e-mail in error, > please > > > > notify the sender by > > > > phone or email immediately and delete it!ss > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --bcaec547ca333791d004a7914a6c--