incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Date Mon, 08 Dec 2008 07:43:13 GMT
This Issue addressed by HAMA-127.

And, "Task Id : attempt_200812081428_0020_m_000004_1, Status : FAILED"
seems a problem of Hbase/Hadoop.

On Mon, Dec 8, 2008 at 4:00 PM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> In random_mapred(), IntWritable(m) should be IntWritable(m-1) since
> start from 0.
>
> On Mon, Dec 8, 2008 at 3:08 PM, Edward J. Yoon <edwardyoon@apache.org> wrote:
>> Somtimes, job done successfully. BTW, Reduce input groups=110. I guess
>> this is a number of BlockIDs. Should it be a 10 * 10 = 100?
>>
>> ----
>> 08/12/08 14:46:52 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked matrix
>> ...
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Local bytes written=2662743172
>> 08/12/08 14:52:27 INFO mapred.JobClient:   Job Counters
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Launched reduce tasks=2
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Launched map tasks=13
>> 08/12/08 14:52:27 INFO mapred.JobClient:   Map-Reduce Framework
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce input groups=110
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Combine output records=0
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Map input records=7556
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce output records=0
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Map output bytes=1020493900
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Map input bytes=0
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Combine input records=0
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Map output records=75560
>> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce input records=75560
>>
>> On Mon, Dec 8, 2008 at 2:31 PM, Samuel Guo <guosijie@gmail.com> wrote:
>>> It is strange.
>>> Let's review the code again to find what's wrong.
>>>
>>> On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>>>
>>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>>> Size : 4,000
>>>> block_num : 10
>>>>
>>>> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
>>>> ...
>>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>>>
>>>> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
>>>> matrix
>>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
>>>> parsing the arguments. Applications should implement Tool for the
>>>> same.
>>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
>>>> option -libjars
>>>> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set.  User
>>>> classes may not be found. See JobConf(Class) or
>>>> JobConf#setJar(String).
>>>> 08/12/08 13:23:57 INFO mapred.JobClient: Running job: job_200812051425_0027
>>>> 08/12/08 13:23:58 INFO mapred.JobClient:  map 0% reduce 0%
>>>> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
>>>> attempt_200812051425_0027_m_000001_0, Status : FAILED
>>>> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>>>>        at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>>>>        at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>>>>        at
>>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>>>>        at
>>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>>>>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>>>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>>>        at
>>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>>>
>>>>
>>>>
>>>> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <guosijie@gmail.com> wrote:
>>>> > what is the matrix's size?
>>>> > what is the block's size?
>>>> > can the matrix's size be interge-divided by block's size?
>>>> > if not, the last sub-vector's index range will be out of bound.
>>>> >
>>>> > so, the get() method in AbstractVector will get *Null* when calling
>>>> > *this.entries.get(index)*. The code is as below.
>>>> >
>>>> > public double get(int index) throws NullPointerException {
>>>> >    return this.entries.get(index).getValue();
>>>> > }
>>>> >
>>>> > It is a bug. A Jira may be filed to fix two files of the code.
>>>> >
>>>> > first:
>>>> > Method *setBlockPosition* in DenseMatrix class;
>>>> > second:
>>>> > Class *BlockingMapper* in BlockingMapRed class;
>>>> >
>>>> > I am sorry. :-(
>>>> > I have found the potential bug in DenseMatrix when I write the
>>>> > BlockingMapRed class. But I forget to fix it and use the same way to
>>>> > caculate the block ranges during mapping.
>>>> >
>>>> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <edwardyoon@apache.org
>>>> >wrote:
>>>> >
>>>> >> Can someone figure out this problem?
>>>> >>
>>>> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked
>>>> >> matrix
>>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser
for
>>>> >> parsing the arguments. Applications should implement Tool for the
>>>> >> same.
>>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for
the
>>>> >> option -libjars
>>>> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set.  User
>>>> >> classes may not be found. See JobConf(Class) or
>>>> >> JobConf#setJar(String).
>>>> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job:
>>>> job_200812041310_0026
>>>> >> 08/12/04 15:58:39 INFO mapred.JobClient:  map 0% reduce 0%
>>>> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
>>>> >> attempt_200812041310_0026_m_000002_0, Status : FAILED
>>>> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>>>> >>        at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>>>> >>        at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>>>> >>        at
>>>> >>
>>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>>>> >>        at
>>>> >>
>>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>>>> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>>> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>>> >>        at
>>>> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>>> >> --
>>>> >> Best Regards, Edward J. Yoon @ NHN, corp.
>>>> >> edwardyoon@apache.org
>>>> >> http://blog.udanax.org
>>>> >>
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards, Edward J. Yoon @ NHN, corp.
>>>> edwardyoon@apache.org
>>>> http://blog.udanax.org
>>>>
>>>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>



-- 
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org

Mime
View raw message