Return-Path: Delivered-To: apmail-incubator-hama-dev-archive@locus.apache.org Received: (qmail 53188 invoked from network); 8 Dec 2008 07:43:39 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 8 Dec 2008 07:43:39 -0000 Received: (qmail 95985 invoked by uid 500); 8 Dec 2008 07:43:51 -0000 Delivered-To: apmail-incubator-hama-dev-archive@incubator.apache.org Received: (qmail 95975 invoked by uid 500); 8 Dec 2008 07:43:51 -0000 Mailing-List: contact hama-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hama-dev@incubator.apache.org Delivered-To: mailing list hama-dev@incubator.apache.org Received: (qmail 95964 invoked by uid 99); 8 Dec 2008 07:43:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Dec 2008 23:43:51 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.142.184] (HELO ti-out-0910.google.com) (209.85.142.184) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Dec 2008 07:43:37 +0000 Received: by ti-out-0910.google.com with SMTP id w7so793360tib.6 for ; Sun, 07 Dec 2008 23:43:13 -0800 (PST) Received: by 10.110.43.16 with SMTP id q16mr4180004tiq.3.1228722193646; Sun, 07 Dec 2008 23:43:13 -0800 (PST) Received: by 10.110.49.18 with HTTP; Sun, 7 Dec 2008 23:43:13 -0800 (PST) Message-ID: Date: Mon, 8 Dec 2008 16:43:13 +0900 From: "Edward J. Yoon" Sender: edward@udanax.org To: hama-dev@incubator.apache.org Subject: Re: NullPointerException at AbstractVector.get(AbstractVector.java:44) In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <25aacb800812040630h15d50eecx48fe9eee430646b0@mail.gmail.com> <25aacb800812072131t79fdf0d2k1fea6c1b9bfbbe94@mail.gmail.com> X-Google-Sender-Auth: 12f9354be86c47b1 X-Virus-Checked: Checked by ClamAV on apache.org This Issue addressed by HAMA-127. And, "Task Id : attempt_200812081428_0020_m_000004_1, Status : FAILED" seems a problem of Hbase/Hadoop. On Mon, Dec 8, 2008 at 4:00 PM, Edward J. Yoon wrote: > In random_mapred(), IntWritable(m) should be IntWritable(m-1) since > start from 0. > > On Mon, Dec 8, 2008 at 3:08 PM, Edward J. Yoon wrote: >> Somtimes, job done successfully. BTW, Reduce input groups=110. I guess >> this is a number of BlockIDs. Should it be a 10 * 10 = 100? >> >> ---- >> 08/12/08 14:46:52 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked matrix >> ... >> 08/12/08 14:52:27 INFO mapred.JobClient: Local bytes written=2662743172 >> 08/12/08 14:52:27 INFO mapred.JobClient: Job Counters >> 08/12/08 14:52:27 INFO mapred.JobClient: Launched reduce tasks=2 >> 08/12/08 14:52:27 INFO mapred.JobClient: Launched map tasks=13 >> 08/12/08 14:52:27 INFO mapred.JobClient: Map-Reduce Framework >> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce input groups=110 >> 08/12/08 14:52:27 INFO mapred.JobClient: Combine output records=0 >> 08/12/08 14:52:27 INFO mapred.JobClient: Map input records=7556 >> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce output records=0 >> 08/12/08 14:52:27 INFO mapred.JobClient: Map output bytes=1020493900 >> 08/12/08 14:52:27 INFO mapred.JobClient: Map input bytes=0 >> 08/12/08 14:52:27 INFO mapred.JobClient: Combine input records=0 >> 08/12/08 14:52:27 INFO mapred.JobClient: Map output records=75560 >> 08/12/08 14:52:27 INFO mapred.JobClient: Reduce input records=75560 >> >> On Mon, Dec 8, 2008 at 2:31 PM, Samuel Guo wrote: >>> It is strange. >>> Let's review the code again to find what's wrong. >>> >>> On Mon, Dec 8, 2008 at 12:26 PM, Edward J. Yoon wrote: >>> >>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB >>>> Size : 4,000 >>>> block_num : 10 >>>> >>>> [d8g053:/root/hama-trunk]# bin/hama examples random 4000 4000 fB >>>> ... >>>> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 10 fA fB >>>> >>>> 08/12/08 13:23:57 INFO hama.AbstractMatrix: Convert to 10 * 10 blocked >>>> matrix >>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use GenericOptionsParser for >>>> parsing the arguments. Applications should implement Tool for the >>>> same. >>>> 08/12/08 13:23:57 WARN mapred.JobClient: Use genericOptions for the >>>> option -libjars >>>> 08/12/08 13:23:57 WARN mapred.JobClient: No job jar file set. User >>>> classes may not be found. See JobConf(Class) or >>>> JobConf#setJar(String). >>>> 08/12/08 13:23:57 INFO mapred.JobClient: Running job: job_200812051425_0027 >>>> 08/12/08 13:23:58 INFO mapred.JobClient: map 0% reduce 0% >>>> 08/12/08 13:24:29 INFO mapred.JobClient: Task Id : >>>> attempt_200812051425_0027_m_000001_0, Status : FAILED >>>> java.lang.NullPointerException: v(1) : java.lang.NullPointerException >>>> at org.apache.hama.AbstractVector.get(AbstractVector.java:44) >>>> at org.apache.hama.DenseVector.subVector(DenseVector.java:151) >>>> at >>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112) >>>> at >>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98) >>>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) >>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) >>>> at >>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) >>>> >>>> >>>> >>>> On Thu, Dec 4, 2008 at 11:30 PM, Samuel Guo wrote: >>>> > what is the matrix's size? >>>> > what is the block's size? >>>> > can the matrix's size be interge-divided by block's size? >>>> > if not, the last sub-vector's index range will be out of bound. >>>> > >>>> > so, the get() method in AbstractVector will get *Null* when calling >>>> > *this.entries.get(index)*. The code is as below. >>>> > >>>> > public double get(int index) throws NullPointerException { >>>> > return this.entries.get(index).getValue(); >>>> > } >>>> > >>>> > It is a bug. A Jira may be filed to fix two files of the code. >>>> > >>>> > first: >>>> > Method *setBlockPosition* in DenseMatrix class; >>>> > second: >>>> > Class *BlockingMapper* in BlockingMapRed class; >>>> > >>>> > I am sorry. :-( >>>> > I have found the potential bug in DenseMatrix when I write the >>>> > BlockingMapRed class. But I forget to fix it and use the same way to >>>> > caculate the block ranges during mapping. >>>> > >>>> > On Thu, Dec 4, 2008 at 3:09 PM, Edward J. Yoon >>> >wrote: >>>> > >>>> >> Can someone figure out this problem? >>>> >> >>>> >> 08/12/04 15:58:37 INFO hama.AbstractMatrix: Convert to 20 * 20 blocked >>>> >> matrix >>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use GenericOptionsParser for >>>> >> parsing the arguments. Applications should implement Tool for the >>>> >> same. >>>> >> 08/12/04 15:58:37 WARN mapred.JobClient: Use genericOptions for the >>>> >> option -libjars >>>> >> 08/12/04 15:58:38 WARN mapred.JobClient: No job jar file set. User >>>> >> classes may not be found. See JobConf(Class) or >>>> >> JobConf#setJar(String). >>>> >> 08/12/04 15:58:38 INFO mapred.JobClient: Running job: >>>> job_200812041310_0026 >>>> >> 08/12/04 15:58:39 INFO mapred.JobClient: map 0% reduce 0% >>>> >> 08/12/04 15:59:29 INFO mapred.JobClient: Task Id : >>>> >> attempt_200812041310_0026_m_000002_0, Status : FAILED >>>> >> java.lang.NullPointerException: v(1) : java.lang.NullPointerException >>>> >> at org.apache.hama.AbstractVector.get(AbstractVector.java:44) >>>> >> at org.apache.hama.DenseVector.subVector(DenseVector.java:151) >>>> >> at >>>> >> >>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:112) >>>> >> at >>>> >> >>>> org.apache.hama.mapred.BlockingMapRed$BlockingMapper.map(BlockingMapRed.java:98) >>>> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) >>>> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) >>>> >> at >>>> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) >>>> >> -- >>>> >> Best Regards, Edward J. Yoon @ NHN, corp. >>>> >> edwardyoon@apache.org >>>> >> http://blog.udanax.org >>>> >> >>>> > >>>> >>>> >>>> >>>> -- >>>> Best Regards, Edward J. Yoon @ NHN, corp. >>>> edwardyoon@apache.org >>>> http://blog.udanax.org >>>> >>> >> >> >> >> -- >> Best Regards, Edward J. Yoon @ NHN, corp. >> edwardyoon@apache.org >> http://blog.udanax.org >> > > > > -- > Best Regards, Edward J. Yoon @ NHN, corp. > edwardyoon@apache.org > http://blog.udanax.org > -- Best Regards, Edward J. Yoon @ NHN, corp. edwardyoon@apache.org http://blog.udanax.org