hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Date Mon, 08 Dec 2008 07:00:51 GMT
In random_mapred(), IntWritable(m) should be IntWritable(m-1) since
start from 0.

On Mon, Dec 8, 2008 at 3:08 PM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> Somtimes, job done successfully. BTW, Reduce input groups=110. I guess
> this is a number of BlockIDs. Should it be a 10 * 10 = 100?
>
> ----
> 08/12/08 14:46:52 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked matrix
> ...
> 08/12/08 14:52:27 INFO mapred.JobClient:     Local bytes written=2662743172
> 08/12/08 14:52:27 INFO mapred.JobClient:   Job Counters
> 08/12/08 14:52:27 INFO mapred.JobClient:     Launched reduce tasks=2
> 08/12/08 14:52:27 INFO mapred.JobClient:     Launched map tasks=13
> 08/12/08 14:52:27 INFO mapred.JobClient:   Map-Reduce Framework
> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce input groups=110
> 08/12/08 14:52:27 INFO mapred.JobClient:     Combine output records=0
> 08/12/08 14:52:27 INFO mapred.JobClient:     Map input records=7556
> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce output records=0
> 08/12/08 14:52:27 INFO mapred.JobClient:     Map output bytes=1020493900
> 08/12/08 14:52:27 INFO mapred.JobClient:     Map input bytes=0
> 08/12/08 14:52:27 INFO mapred.JobClient:     Combine input records=0
> 08/12/08 14:52:27 INFO mapred.JobClient:     Map output records=75560
> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce input records=75560
>
> On Mon, Dec 8, 2008 at 2:31 PM, Samuel Guo <guosijie@gmail.com> wrote:
>> It is strange.
>> Let's review the code again to find what's wrong.
>>
>> On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>>
>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>> Size : 4,000
>>> block_num : 10
>>>
>>> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
>>> ...
>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
>>>
>>> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
>>> matrix
>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
>>> parsing the arguments. Applications should implement Tool for the
>>> same.
>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
>>> option -libjars
>>> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set.  User
>>> classes may not be found. See JobConf(Class) or
>>> JobConf#setJar(String).
>>> 08/12/08 13:23:57 INFO mapred.JobClient: Running job: job_200812051425_0027
>>> 08/12/08 13:23:58 INFO mapred.JobClient:  map 0% reduce 0%
>>> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
>>> attempt_200812051425_0027_m_000001_0, Status : FAILED
>>> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>>>        at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>>>        at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>>>        at
>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>>>        at
>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>>>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>>        at
>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>>
>>>
>>>
>>> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <guosijie@gmail.com> wrote:
>>> > what is the matrix's size?
>>> > what is the block's size?
>>> > can the matrix's size be interge-divided by block's size?
>>> > if not, the last sub-vector's index range will be out of bound.
>>> >
>>> > so, the get() method in AbstractVector will get *Null* when calling
>>> > *this.entries.get(index)*. The code is as below.
>>> >
>>> > public double get(int index) throws NullPointerException {
>>> >    return this.entries.get(index).getValue();
>>> > }
>>> >
>>> > It is a bug. A Jira may be filed to fix two files of the code.
>>> >
>>> > first:
>>> > Method *setBlockPosition* in DenseMatrix class;
>>> > second:
>>> > Class *BlockingMapper* in BlockingMapRed class;
>>> >
>>> > I am sorry. :-(
>>> > I have found the potential bug in DenseMatrix when I write the
>>> > BlockingMapRed class. But I forget to fix it and use the same way to
>>> > caculate the block ranges during mapping.
>>> >
>>> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <edwardyoon@apache.org
>>> >wrote:
>>> >
>>> >> Can someone figure out this problem?
>>> >>
>>> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked
>>> >> matrix
>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
>>> >> parsing the arguments. Applications should implement Tool for the
>>> >> same.
>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
>>> >> option -libjars
>>> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set.  User
>>> >> classes may not be found. See JobConf(Class) or
>>> >> JobConf#setJar(String).
>>> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job:
>>> job_200812041310_0026
>>> >> 08/12/04 15:58:39 INFO mapred.JobClient:  map 0% reduce 0%
>>> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
>>> >> attempt_200812041310_0026_m_000002_0, Status : FAILED
>>> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
>>> >>        at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
>>> >>        at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
>>> >>        at
>>> >>
>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
>>> >>        at
>>> >>
>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
>>> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>> >>        at
>>> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>> >> --
>>> >> Best Regards, Edward J. Yoon @ NHN, corp.
>>> >> edwardyoon@apache.org
>>> >> http://blog.udanax.org
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon @ NHN, corp.
>>> edwardyoon@apache.org
>>> http://blog.udanax.org
>>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>



-- 
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org

Mime
View raw message