Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of dlieu.7@gmail.com designates
 74.125.82.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=OtxlGSF2B3AQ+aGvPEF0hmsFpLhHRG0Ofc/VHMLG/4NSSPBb16JAK4EvRzKJBMU3tE
         sgz77DZc/B+O/+H+8OgTBAGsgzITpvPdgVjj/AO3XL3sE1nPCfnaVaW1v0lzXDqj0sa1
         hXB0zcMjs/B/qpA+ly2YLq8e5+Ee0DHMupLeg=
MIME-Version: 1.0
In-Reply-To: <AANLkTikoS=bS+AadGKpFCMegARDnh=rQ4aPiBR35_5RZ@mail.gmail.com>
References: <AANLkTikoS=bS+AadGKpFCMegARDnh=rQ4aPiBR35_5RZ@mail.gmail.com>
Date: Sun, 5 Dec 2010 15:28:40 -0800
Message-ID: <AANLkTikq+JQnEQ2FkAvmM6djOGusWBE=ebpDOdrbk_CZ@mail.gmail.com>
Subject: Re: Command line integration question
From: Dmitriy Lyubimov <dlieu.7@gmail.com>
To: user <user@hbase.apache.org>, dev@mahout.apache.org
Content-Type: multipart/alternative; boundary=0016e6d9a266b5e4f20496b221ee

--0016e6d9a266b5e4f20496b221ee
Content-Type: text/plain; charset=ISO-8859-1

PS. also if I needed to play with various MR settings, such as child
processes arguments, could i pass that on to Configuration object thru a
command line? Or i would have to add a definition for a custom job setting
for every instance where i'd want to supply a custom MR setting?

Thank you in advance.
-Dmitriy


On Sun, Dec 5, 2010 at 3:17 PM, Dmitriy Lyubimov <dlieu.7@gmail.com> wrote:

> Dear all,
>
> I am testing the command line integration for the SSVD patch in hadoop mode
> and running into some difficulties.
> Even that I defined $HADDOP_HOME and $HADOOP_CONF_DIR, apparently dfs
> configuration is not being picked up.
>
> I do run on CDH3b3, however, all hadoop configuration is 100% compatible
> with 0.20. I am using AbstractJob.getConf() to acquire initial properties
> but it looks like fs.default.name is not being set still. And i tried to
> locate theh code that loads that hadoop conf but wasn't immediately able to
> find it. Could you please help me what i need to do to retrieve initial
> hadoop configuration correctly? I am missing something very simple here.
>
> Thank you in advance.
> -Dmitriy
>
> bin/mahout ssvd -i /mahout/ssvdtest/A -o /mahout/ssvd-out/1 -k 100 -p 100
> -r 200
> Running on hadoop, using HADOOP_HOME=/home/dmitriy/tools/hadoop
> HADOOP_CONF_DIR=/home/dmitriy/tools/hadoop/conf
> 10/12/05 15:09:55 INFO common.AbstractJob: Command line arguments:
> {--blockHeight=200, --computeU=true, --computeV=true, --endPhase=2147483647,
> --input=/mahout/ssvdtest/A, --minSplitSize=-1, --output=/mahout/ssvd-out/1,
> --oversampling=100, --rank=100, --reduceTasks=1, --startPhase=0,
> --tempDir=temp}
> Exception in thread "main" java.lang.NullPointerException
>         at
> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:118)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:110)
>         at
> org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver.run(SSVDSolver.java:177)
>         at
> org.apache.mahout.math.hadoop.stochasticsvd.SSVDCli.run(SSVDCli.java:75)
>         at
> org.apache.mahout.math.hadoop.stochasticsvd.SSVDCli.main(SSVDCli.java:108)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>         at
> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>         at
> org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:182)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>
>

--0016e6d9a266b5e4f20496b221ee--