hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Lyubimov <dlie...@gmail.com>
Subject Re: Command line integration question
Date Sun, 05 Dec 2010 23:28:40 GMT
PS. also if I needed to play with various MR settings, such as child
processes arguments, could i pass that on to Configuration object thru a
command line? Or i would have to add a definition for a custom job setting
for every instance where i'd want to supply a custom MR setting?

Thank you in advance.
-Dmitriy



On Sun, Dec 5, 2010 at 3:17 PM, Dmitriy Lyubimov <dlieu.7@gmail.com> wrote:

> Dear all,
>
> I am testing the command line integration for the SSVD patch in hadoop mode
> and running into some difficulties.
> Even that I defined $HADDOP_HOME and $HADOOP_CONF_DIR, apparently dfs
> configuration is not being picked up.
>
> I do run on CDH3b3, however, all hadoop configuration is 100% compatible
> with 0.20. I am using AbstractJob.getConf() to acquire initial properties
> but it looks like fs.default.name is not being set still. And i tried to
> locate theh code that loads that hadoop conf but wasn't immediately able to
> find it. Could you please help me what i need to do to retrieve initial
> hadoop configuration correctly? I am missing something very simple here.
>
> Thank you in advance.
> -Dmitriy
>
> bin/mahout ssvd -i /mahout/ssvdtest/A -o /mahout/ssvd-out/1 -k 100 -p 100
> -r 200
> Running on hadoop, using HADOOP_HOME=/home/dmitriy/tools/hadoop
> HADOOP_CONF_DIR=/home/dmitriy/tools/hadoop/conf
> 10/12/05 15:09:55 INFO common.AbstractJob: Command line arguments:
> {--blockHeight=200, --computeU=true, --computeV=true, --endPhase=2147483647,
> --input=/mahout/ssvdtest/A, --minSplitSize=-1, --output=/mahout/ssvd-out/1,
> --oversampling=100, --rank=100, --reduceTasks=1, --startPhase=0,
> --tempDir=temp}
> Exception in thread "main" java.lang.NullPointerException
>         at
> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:118)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:110)
>         at
> org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver.run(SSVDSolver.java:177)
>         at
> org.apache.mahout.math.hadoop.stochasticsvd.SSVDCli.run(SSVDCli.java:75)
>         at
> org.apache.mahout.math.hadoop.stochasticsvd.SSVDCli.main(SSVDCli.java:108)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>         at
> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>         at
> org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:182)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message