From hdfs-dev-return-3346-apmail-hadoop-hdfs-dev-archive=hadoop.apache.org@hadoop.apache.org Fri Jun 17 02:06:44 2011 Return-Path: X-Original-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EDFB94080 for ; Fri, 17 Jun 2011 02:06:43 +0000 (UTC) Received: (qmail 25689 invoked by uid 500); 17 Jun 2011 02:06:43 -0000 Delivered-To: apmail-hadoop-hdfs-dev-archive@hadoop.apache.org Received: (qmail 25624 invoked by uid 500); 17 Jun 2011 02:06:43 -0000 Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-dev@hadoop.apache.org Delivered-To: mailing list hdfs-dev@hadoop.apache.org Received: (qmail 25616 invoked by uid 99); 17 Jun 2011 02:06:43 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Jun 2011 02:06:43 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of aoriani@gmail.com designates 209.85.214.48 as permitted sender) Received: from [209.85.214.48] (HELO mail-bw0-f48.google.com) (209.85.214.48) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Jun 2011 02:06:37 +0000 Received: by bwz8 with SMTP id 8so466195bwz.35 for ; Thu, 16 Jun 2011 19:06:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=pOJDFMQ3ltblS4SfUUBb5gk6ZQaXsm6wIspNYIsNZqc=; b=Zhkgj12a9o08zdf/ywDIU3/AZsYmlLMufXMTzIdg+NmCyRxOOqO2ZFVGDS+W6jRn87 RfJzcC48qPUKb/fT93si9ICsAvyTFb3sjH+VvLFsu4vOuSX3AU47H6rKE75gkxGQND5c SN7yHgb25rCPLj7xwZFxYZ6sa4Kr50ZFNZ0vU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=cthi1atItUPXAdNE6VcWs8h+b+XQpi+ACmJmagWdoAhncuZaSI8BhWOOBVOuR64kHl 93sbhkfVOS5sOwArI9kc83XIB4hFBkhAXbRAhUxdkv7w/9I9eskmMhEI8HONjKZLKsKq BoYo3+3w1EvFybHAMGwGKod5ist7yjXOSaCs8= Received: by 10.204.38.88 with SMTP id a24mr1151876bke.130.1308276377365; Thu, 16 Jun 2011 19:06:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.121.73 with HTTP; Thu, 16 Jun 2011 19:05:57 -0700 (PDT) In-Reply-To: <4DFA9CA0.1020800@mustardgrain.com> References: <4DFA9CA0.1020800@mustardgrain.com> From: =?ISO-8859-1?Q?Andr=E9_Oriani?= Date: Thu, 16 Jun 2011 23:05:57 -0300 Message-ID: Subject: Re: How to run HDFS from source? To: hdfs-dev@hadoop.apache.org Content-Type: multipart/alternative; boundary=0003255546f2bdac8604a5ded44f X-Virus-Checked: Checked by ClamAV on apache.org --0003255546f2bdac8604a5ded44f Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I cannot say for the current code, since it look like people had undone the project split. But based on Eli`s repo I could, with the code of June 12th, do the follow to run HDFS in pseudo distributed mode -Set your JAVA_HOME on hadoop-common/conf/hadoop-env.shto match your system. -Applied diff --git a/bin/hadoop b/bin/hadoop index 88c49eb..84992e5 100755 --- a/bin/hadoop +++ b/bin/hadoop @@ -21,7 +21,7 @@ bin=3D`which $0` bin=3D`dirname ${bin}` bin=3D`cd "$bin"; pwd` -. "$bin"/../libexec/hadoop-config.sh +. "$bin"/hadoop-config.sh function print_usage(){ echo "Usage: hadoop [--config confdir] COMMAND" - export some environment variables: export HADOOP_COMMON_HOME=3D export HADOOP_HDFS_HOME=3D -Configured core-site.xml and hdfs-site.xml as usual Then $HADOOP_HDFS_HOME/bin/hdfs namenode should work , as well the command for datanodes. Personally, I would expect to run from source, because sometimes is good to test running the actual thing. Regards, Andr=E9 Oriani On Thu, Jun 16, 2011 at 21:15, Kirk True wrote: > Should running ./bin/hdfs from the source root work? > > I get these errors: > > [kirk@bubbas apache]$ ./bin/hdfs namenode > ./bin/hdfs: line 154: cygpath: command not found > ./bin/hdfs: line 177: exec: : not found > > I can hack around the latter by setting the JAVA env var up first: > > export JAVA=3D$JAVA_HOME/bin/java > > Still, I get this: > > [kirk@bubbas apache]$ ./bin/hdfs namenode > ./bin/hdfs: line 154: cygpath: command not found > Exception in thread "main" java.lang.**NoClassDefFoundError: > org/apache/hadoop/hdfs/server/**namenode/NameNode > Caused by: java.lang.**ClassNotFoundException: > org.apache.hadoop.hdfs.server.**namenode.NameNode > at java.net.URLClassLoader$1.run(**URLClassLoader.java:202) > at java.security.**AccessController.doPrivileged(**Native Method) > at java.net.URLClassLoader.**findClass(URLClassLoader.java:**190) > at java.lang.ClassLoader.**loadClass(ClassLoader.java:**307) > at sun.misc.Launcher$**AppClassLoader.loadClass(** > Launcher.java:301) > at java.lang.ClassLoader.**loadClass(ClassLoader.java:**248) > Could not find the main class: org.apache.hadoop.hdfs.server.**namenod= e.NameNode. > Program will exit. > > This is after running a build (`ant compile`) on the source. > > I can dig into it, but I'm wondering if running from source/stand-alone > like this is even expected to work. > > -- Kirk > > > On 06/07/2011 01:40 PM, Andr=E9 Oriani wrote: > >> Thanks a lot Eli, it really helped a lot. I think I got the general ide= a >> of >> the scripts. >> >> Thanks, >> Andr=E9 >> >> On Tue, Jun 7, 2011 at 16:51, Eli Collins wrote: >> >> Hey Andre, >>> >>> You can run an hdfs build out of common/hdfs trees checked out from >>> svn or git. Here are some scripts that make this easier: >>> >>> https://github.com/elicollins/**hadoop-dev >>> >>> Thanks, >>> Eli >>> >>> On Tue, Jun 7, 2011 at 11:56 AM, Andr=E9 Oriani >>> > >>> wrote: >>> >>>> Hi, >>>> >>>> >>>> I have clone the repo for hadoop-common and hadoop-hdfs and built it >>>> >>> using >>> >>>> "ant mvn-install" . Now I would like to be able run HDFS in >>>> pseudo-distributed mode to test some modifications of mine. One year >>>> ago >>>> >>> I >>> >>>> could do it but now I had no luck. The scripts are failing, complaini= ng >>>> about not found files and stuff. >>>> >>>> Has anyone succeed recently in doing something similar to what I am >>>> intending to do . Or do I need to generate a tarball and install it ? >>>> >>>> >>>> Thanks and Regards, >>>> Andr=E9 >>>> >>>> --0003255546f2bdac8604a5ded44f--