incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: [Q] Error in executing an example (Matrix Addition)
Date Wed, 10 Sep 2008 03:00:58 GMT
Good plan.

BTW, If we sets the entries of the matrix using matrix.set() method
every time in a loop as current example, it'll be slow. I think we
should think about the batch operations for set(Vector v), and
re-thinking an examples.

-Ed

On Wed, Sep 10, 2008 at 9:04 AM, joosun yoon <ologist0@gmail.com> wrote:
> if edward has be come back at next week, we will make pseudo-distributed
> operation system and test it.
>
>
> 2008/9/8, Edward J. Yoon <edwardyoon@apache.org>:
>>
>> This issue was addressed by HAMA-53, And I just committed this. Please
>> re-try on your cluster. : )
>>
>> Thanks, Edward
>>
>>
>> On Mon, Sep 8, 2008 at 6:17 AM, Edward J. Yoon <edwardyoon@apache.org>
>> wrote:
>> > Hmm, it was bug. Thanks for your report.
>> >
>> > On Mon, Sep 8, 2008 at 4:51 PM, Kyungho Jeon <khjeon@dcslab.snu.ac.kr>
>> wrote:
>> >> A NullPointerException occurred in my system (Pseudo-distributed
>> operation),
>> >> too.
>> >>
>> >> I will try standalone mode.
>> >>
>> >> Thank you for an answer.
>> >>
>> >> --
>> >> kyungho jeon
>> >>
>> >> On Mon, Sep 8, 2008 at 4:40 PM, Edward J. Yoon <edwardyoon@apache.org
>> >wrote:
>> >>
>> >>> Are you ready to run the Hama on the Hadoop + Hbase cluster? Then,
>> >>> This is quick guide for you :
>> >>>
>> >>> 1. [hama-trunk]# ant clean package
>> >>>    Then, You should add hama-x.x.x.jar to the $HADOOP_HOME/lib and
>> >>> copy these changes across your cluster.
>> >>>
>> >>> 2. [hama-trunk]# cp ./build/hama-0.1.0-dev.jar {HADOOP_HOME}/lib
>> >>>    You also need to restart your hadoop cluster.
>> >>>
>> >>> 3. [hama-trunk]# cp ./build/hama-0.1.0-dev-examples.jar {HADOOP_HOME}
>> >>> 4. [hadoop-0.17.1]# ./bin/hadoop jar hama-0.1.0-dev-examples.jar
>> addition 2
>> >>> 2
>> >>> ...
>> >>>
>> >>> BTW, It seems work just fine on Eclipse (local mode), but I recieved
>> >>> an NullPointerException when I run it on 2-node cluster. I'll fix it.
>> >>>
>> >>> Thanks, Edward
>> >>>
>> >>> On Mon, Sep 8, 2008 at 1:27 PM, Kyungho Jeon <khjeon@dcslab.snu.ac.kr>
>> >>> wrote:
>> >>> > I installed hadoop, hbase, and hama in pseudo-distributed operation
>> mode,
>> >>> >
>> >>> > but I can not run the example code. How can I fix it?
>> >>> >
>> >>> > (Sorry for the messy logs...)
>> >>> >
>> >>> > They are installed on following directories:
>> >>> >
>> >>> > /home/khjeon/mapred/hadoop
>> >>> > /home/khjeon/mapred/hbase
>> >>> > /home/khjeon/mapred/hama-trunk
>> >>> >
>> >>> > In addition, my hadoop-env.sh file contains following lines
>> >>> > export HBASE_HOME=/home/khjeon/mapred/hbase
>> >>> > export HAMA_HOME=/home/khjeon/mapred/hama-trunk
>> >>> >
>> >>> > export
>> >>> >
>> >>>
>> HADOOP_CLASSPATH=${HBASE_HOME}:${HBASE_HOME}/hbase-0.2.0.jar:${HBASE_HOME}/conf
>> >>> > export
>> >>> >
>> >>>
>> HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:${HAMA_HOME}:${HAMA_HOME}/build:${HAMA_HOME}/build/hama-0.1.0-dev.jar:${HAMA_HOME}/conf
>> >>> >
>> >>> > I got errors like these:
>> >>> >
>> >>> > ../hadoop/bin/hadoop jar build/hama-0.1.0-dev-examples.jar addition
3
>> 3
>> >>> >
>> >>> > 08/09/08 13:05:52 WARN fs.FileSystem: "localhost:9000" is a
>> deprecated
>> >>> > filesystem name. Use "hdfs://localhost:9000/" instead.
>> >>> > 08/09/08 13:05:53 INFO hama.AbstractMatrix: Initializing the matrix
>> >>> storage.
>> >>> > 08/09/08 13:06:03 INFO hama.AbstractMatrix: Create the 3 * 3 random
>> >>> matrix :
>> >>> > randjowrz
>> >>> > 08/09/08 13:06:03 INFO hama.AbstractMatrix: Initializing the matrix
>> >>> storage.
>> >>> > 08/09/08 13:06:13 INFO hama.AbstractMatrix: Create the 3 * 3 random
>> >>> matrix :
>> >>> > randswrhz
>> >>> > 08/09/08 13:06:13 INFO hama.AbstractMatrix: Initializing the matrix
>> >>> storage.
>> >>> > 08/09/08 13:06:23 WARN fs.FileSystem: "localhost:9000" is a
>> deprecated
>> >>> > filesystem name. Use "hdfs://localhost:9000/" instead.
>> >>> > 08/09/08 13:06:23 WARN mapred.JobClient: No job jar file set. 
User
>> >>> classes
>> >>> > may not be found. See JobConf(Class) or JobConf#setJar(String).
>> >>> > 08/09/08 13:06:23 INFO mapred.JobClient: Running job:
>> >>> job_200809081230_0008
>> >>> > 08/09/08 13:06:24 INFO mapred.JobClient:  map 0% reduce 0%
>> >>> > 08/09/08 13:06:29 INFO mapred.JobClient: Task Id :
>> >>> > task_200809081230_0008_m_000000_0, Status : FAILED
>> >>> > java.lang.RuntimeException: java.lang.RuntimeException:
>> >>> > java.lang.ClassNotFoundException: org.apache.hama.DenseVector
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:633)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:369)
>> >>> >        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2122)
>> >>> > Caused by: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.DenseVector
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:601)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
>> >>> >        ... 4 more
>> >>> > Caused by: java.lang.ClassNotFoundException:
>> org.apache.hama.DenseVector
>> >>> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> >>> >        at java.security.AccessController.doPrivileged(Native Method)
>> >>> >        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> >>> >        at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> >>> >        at
>> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> >>> >        at java.lang.Class.forName0(Native Method)
>> >>> >        at java.lang.Class.forName(Class.java:242)
>> >>> >        at
>> >>> >
>> >>>
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
>> >>> >        ... 5 more
>> >>> >
>> >>> > 08/09/08 13:06:33 INFO mapred.JobClient: Task Id :
>> >>> > task_200809081230_0008_r_000000_0, Status : FAILED
>> >>> > java.lang.RuntimeException: java.lang.RuntimeException:
>> >>> > java.lang.ClassNotFoundException:
>> org.apache.hama.algebra.AdditionReduce
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:633)
>> >>> >        at
>> >>> > org.apache.hadoop.mapred.JobConf.getReducerClass(JobConf.java:747)
>> >>> >        at
>> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2122)
>> >>> > Caused by: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.algebra.AdditionReduce
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:601)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
>> >>> >        ... 3 more
>> >>> > Caused by: java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.algebra.AdditionReduce
>> >>> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> >>> >        at java.security.AccessController.doPrivileged(Native Method)
>> >>> >        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> >>> >        at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> >>> >        at
>> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> >>> >        at java.lang.Class.forName0(Native Method)
>> >>> >        at java.lang.Class.forName(Class.java:242)
>> >>> >        at
>> >>> >
>> >>>
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
>> >>> >        ... 4 more
>> >>> >
>> >>> > 08/09/08 13:06:34 INFO mapred.JobClient: Task Id :
>> >>> > task_200809081230_0008_m_000000_1, Status : FAILED
>> >>> > java.lang.RuntimeException: java.lang.RuntimeException:
>> >>> > java.lang.ClassNotFoundException: org.apache.hama.DenseVector
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:633)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:369)
>> >>> >        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2122)
>> >>> > Caused by: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.DenseVector
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:601)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
>> >>> >        ... 4 more
>> >>> > Caused by: java.lang.ClassNotFoundException:
>> org.apache.hama.DenseVector
>> >>> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> >>> >        at java.security.AccessController.doPrivileged(Native Method)
>> >>> >        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> >>> >        at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> >>> >        at
>> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> >>> >        at java.lang.Class.forName0(Native Method)
>> >>> >        at java.lang.Class.forName(Class.java:242)
>> >>> >        at
>> >>> >
>> >>>
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
>> >>> >        ... 5 more
>> >>> >
>> >>> > 08/09/08 13:06:37 INFO mapred.JobClient: Task Id :
>> >>> > task_200809081230_0008_r_000000_1, Status : FAILED
>> >>> > java.lang.RuntimeException: java.lang.RuntimeException:
>> >>> > java.lang.ClassNotFoundException:
>> org.apache.hama.algebra.AdditionReduce
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:633)
>> >>> >        at
>> >>> > org.apache.hadoop.mapred.JobConf.getReducerClass(JobConf.java:747)
>> >>> >        at
>> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2122)
>> >>> > Caused by: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.algebra.AdditionReduce
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:601)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
>> >>> >        ... 3 more
>> >>> > Caused by: java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.algebra.AdditionReduce
>> >>> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> >>> >        at java.security.AccessController.doPrivileged(Native Method)
>> >>> >        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> >>> >        at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> >>> >        at
>> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> >>> >        at java.lang.Class.forName0(Native Method)
>> >>> >        at java.lang.Class.forName(Class.java:242)
>> >>> >        at
>> >>> >
>> >>>
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
>> >>> >        ... 4 more
>> >>> >
>> >>> > 08/09/08 13:06:39 INFO mapred.JobClient: Task Id :
>> >>> > task_200809081230_0008_m_000000_2, Status : FAILED
>> >>> > java.lang.RuntimeException: java.lang.RuntimeException:
>> >>> > java.lang.ClassNotFoundException: org.apache.hama.DenseVector
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:633)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:552)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:369)
>> >>> >        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2122)
>> >>> > Caused by: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.DenseVector
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:601)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
>> >>> >        ... 4 more
>> >>> > Caused by: java.lang.ClassNotFoundException:
>> org.apache.hama.DenseVector
>> >>> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> >>> >        at java.security.AccessController.doPrivileged(Native Method)
>> >>> >        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> >>> >        at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> >>> >        at
>> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> >>> >        at java.lang.Class.forName0(Native Method)
>> >>> >        at java.lang.Class.forName(Class.java:242)
>> >>> >        at
>> >>> >
>> >>>
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
>> >>> >        ... 5 more
>> >>> >
>> >>> > 08/09/08 13:06:43 INFO mapred.JobClient: Task Id :
>> >>> > task_200809081230_0008_r_000000_2, Status : FAILED
>> >>> > java.lang.RuntimeException: java.lang.RuntimeException:
>> >>> > java.lang.ClassNotFoundException:
>> org.apache.hama.algebra.AdditionReduce
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:633)
>> >>> >        at
>> >>> > org.apache.hadoop.mapred.JobConf.getReducerClass(JobConf.java:747)
>> >>> >        at
>> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
>> >>> >        at
>> >>> >
>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2122)
>> >>> > Caused by: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.algebra.AdditionReduce
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:601)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
>> >>> >        ... 3 more
>> >>> > Caused by: java.lang.ClassNotFoundException:
>> >>> > org.apache.hama.algebra.AdditionReduce
>> >>> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> >>> >        at java.security.AccessController.doPrivileged(Native Method)
>> >>> >        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> >>> >        at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>> >>> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> >>> >        at
>> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> >>> >        at java.lang.Class.forName0(Native Method)
>> >>> >        at java.lang.Class.forName(Class.java:242)
>> >>> >        at
>> >>> >
>> >>>
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
>> >>> >        at
>> >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
>> >>> >        ... 4 more
>> >>> >
>> >>> > 08/09/08 13:06:44 INFO mapred.JobClient:  map 100% reduce 100%
>> >>> > java.io.IOException: Job failed!
>> >>> >        at
>> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1062)
>> >>> >        at org.apache.hama.DenseMatrix.add(DenseMatrix.java:146)
>> >>> >        at
>> >>> > org.apache.hama.examples.MatrixAddition.main(MatrixAddition.java:44)
>> >>> >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>> >        at
>> >>> >
>> >>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >>> >        at
>> >>> >
>> >>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >>> >        at java.lang.reflect.Method.invoke(Method.java:597)
>> >>> >        at
>> >>> >
>> >>>
>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>> >>> >        at
>> >>> > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>> >>> >        at
>> >>> > org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:29)
>> >>> >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>> >        at
>> >>> >
>> >>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >>> >        at
>> >>> >
>> >>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >>> >        at java.lang.reflect.Method.invoke(Method.java:597)
>> >>> >        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
>> >>> >        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)
>> >>> >        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> >>> >        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> >>> >        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220)
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Best regards, Edward J. Yoon
>> >>> edwardyoon@apache.org
>> >>> http://blog.udanax.org
>> >>>
>> >>
>> >
>> >
>> >
>> > --
>> > Best regards, Edward J. Yoon
>> > edwardyoon@apache.org
>> > http://blog.udanax.org
>> >
>>
>>
>>
>>
>> --
>>
>> Best regards, Edward J. Yoon
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
>
>
> --
> Yoon Joo Sun
> Java Developer Engineer
> Blog - http://www.ologist.co.kr
>



-- 
Best regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org

Mime
View raw message