incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Samuel Guo" <guosi...@gmail.com>
Subject Re: NullPointerException at AbstractVector.get(AbstractVector.java:44)
Date Tue, 09 Dec 2008 05:53:54 GMT
Yes. I think it should be the number of BlockIDs. It is strange.

And the input records = 7556. I think it is the number of rows. Because we
block it 10 blocks, so we will got 10 times Map Input Records in Map Output/
Reduce Input stage. so Map output records = 75560 = 7556 * 10.

But why we have 7556 rows? I guess the size of the matrix in your test is
4000 or 1000.

On Mon, Dec 8, 2008 at 2:08 PM, Edward J. Yoon <edwardyoon@apache.org>wrote:

> Somtimes, job done successfully. BTW, Reduce input groups=110. I guess
> this is a number of BlockIDs. Should it be a 10 * 10 = 100?
>
> ----
> 08/12/08 14:46:52 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
> matrix
> ...
> 08/12/08 14:52:27 INFO mapred.JobClient:     Local bytes written=2662743172
> 08/12/08 14:52:27 INFO mapred.JobClient:   Job Counters
> 08/12/08 14:52:27 INFO mapred.JobClient:     Launched reduce tasks=2
> 08/12/08 14:52:27 INFO mapred.JobClient:     Launched map tasks=13
> 08/12/08 14:52:27 INFO mapred.JobClient:   Map-Reduce Framework
> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce input groups=110
> 08/12/08 14:52:27 INFO mapred.JobClient:     Combine output records=0
> 08/12/08 14:52:27 INFO mapred.JobClient:     Map input records=7556
> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce output records=0
> 08/12/08 14:52:27 INFO mapred.JobClient:     Map output bytes=1020493900
> 08/12/08 14:52:27 INFO mapred.JobClient:     Map input bytes=0
> 08/12/08 14:52:27 INFO mapred.JobClient:     Combine input records=0
> 08/12/08 14:52:27 INFO mapred.JobClient:     Map output records=75560
> 08/12/08 14:52:27 INFO mapred.JobClient:     Reduce input records=75560
>
> On Mon, Dec 8, 2008 at 2:31 PM, Samuel Guo <guosijie@gmail.com> wrote:
> > It is strange.
> > Let's review the code again to find what's wrong.
> >
> > On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon <edwardyoon@apache.org
> >wrote:
> >
> >> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
> >> Size : 4,000
> >> block_num : 10
> >>
> >> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB
> >> ...
> >> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB
> >>
> >> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked
> >> matrix
> >> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for
> >> parsing the arguments. Applications should implement Tool for the
> >> same.
> >> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the
> >> option -libjars
> >> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set.  User
> >> classes may not be found. See JobConf(Class) or
> >> JobConf#setJar(String).
> >> 08/12/08 13:23:57 INFO mapred.JobClient: Running job:
> job_200812051425_0027
> >> 08/12/08 13:23:58 INFO mapred.JobClient:  map 0% reduce 0%
> >> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id :
> >> attempt_200812051425_0027_m_000001_0, Status : FAILED
> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
> >>        at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
> >>        at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
> >>        at
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
> >>        at
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >>        at
> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >>
> >>
> >>
> >> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo <guosijie@gmail.com> wrote:
> >> > what is the matrix's size?
> >> > what is the block's size?
> >> > can the matrix's size be interge-divided by block's size?
> >> > if not, the last sub-vector's index range will be out of bound.
> >> >
> >> > so, the get() method in AbstractVector will get *Null* when calling
> >> > *this.entries.get(index)*. The code is as below.
> >> >
> >> > public double get(int index) throws NullPointerException {
> >> >    return this.entries.get(index).getValue();
> >> > }
> >> >
> >> > It is a bug. A Jira may be filed to fix two files of the code.
> >> >
> >> > first:
> >> > Method *setBlockPosition* in DenseMatrix class;
> >> > second:
> >> > Class *BlockingMapper* in BlockingMapRed class;
> >> >
> >> > I am sorry. :-(
> >> > I have found the potential bug in DenseMatrix when I write the
> >> > BlockingMapRed class. But I forget to fix it and use the same way to
> >> > caculate the block ranges during mapping.
> >> >
> >> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon <edwardyoon@apache.org
> >> >wrote:
> >> >
> >> >> Can someone figure out this problem?
> >> >>
> >> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20
> blocked
> >> >> matrix
> >> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for
> >> >> parsing the arguments. Applications should implement Tool for the
> >> >> same.
> >> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the
> >> >> option -libjars
> >> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set.  User
> >> >> classes may not be found. See JobConf(Class) or
> >> >> JobConf#setJar(String).
> >> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job:
> >> job_200812041310_0026
> >> >> 08/12/04 15:58:39 INFO mapred.JobClient:  map 0% reduce 0%
> >> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id :
> >> >> attempt_200812041310_0026_m_000002_0, Status : FAILED
> >> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException
> >> >>        at org.apache.hama.AbstractVector.get(AbstractVector.java:44)
> >> >>        at org.apache.hama.DenseVector.subVector(DenseVector.java:151)
> >> >>        at
> >> >>
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112)
> >> >>        at
> >> >>
> >>
> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98)
> >> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> >> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >> >>        at
> >> >>
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >> >> --
> >> >> Best Regards, Edward J. Yoon @ NHN, corp.
> >> >> edwardyoon@apache.org
> >> >> http://blog.udanax.org
> >> >>
> >> >
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon @ NHN, corp.
> >> edwardyoon@apache.org
> >> http://blog.udanax.org
> >>
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message