incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Samuel Guo" <guosi...@gmail.com>
Subject Re: UnknownScannerException
Date Tue, 04 Nov 2008 05:43:46 GMT
it seems that a 200 * 200 matrix is not very huge matrix.
maybe the reason is not about matrix spliting/moving.

On Tue, Nov 4, 2008 at 12:04 PM, Edward J. Yoon <edwardyoon@apache.org>wrote:

> This is my full log. (5 node test cluster)
>
> ----
> hadoop@vldb ~/hadoop $ bin/hadoop jar hama-0.1.0-dev-examples.jar mult
> -m 50 -r 20 200 200
> 08/11/04 12:57:28 INFO hama.AbstractMatrix: Initializing the matrix
> storage.
> 08/11/04 12:57:39 INFO hama.AbstractMatrix: Create Matrix
> DenseMatrix_randnxkdh
> 08/11/04 12:57:43 INFO hama.AbstractMatrix: Create the 200 * 200
> random matrix : DenseMatrix_randnxkdh
> 08/11/04 12:57:43 INFO hama.AbstractMatrix: Initializing the matrix
> storage.
> 08/11/04 12:57:48 INFO hama.AbstractMatrix: Create Matrix
> DenseMatrix_randgufzg
> 08/11/04 12:57:49 INFO hama.AbstractMatrix: Create the 200 * 200
> random matrix : DenseMatrix_randgufzg
> 08/11/04 12:57:49 INFO hama.AbstractMatrix: Initializing the matrix
> storage.
> 08/11/04 12:57:54 INFO hama.AbstractMatrix: Create Matrix
> DenseMatrix_randetuuu
> 08/11/04 12:57:54 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the
> same.
> 08/11/04 12:57:54 WARN mapred.JobClient: No job jar file set.  User
> classes may not be found. See JobConf(Class) or
> JobConf#setJar(String).
> 08/11/04 12:57:55 INFO mapred.JobClient: Running job: job_200811041234_0002
> 08/11/04 12:57:56 INFO mapred.JobClient:  map 0% reduce 0%
> 08/11/04 12:58:03 INFO mapred.JobClient:  map 1% reduce 0%
> 08/11/04 12:58:10 INFO mapred.JobClient:  map 3% reduce 0%
> 08/11/04 12:58:12 INFO mapred.JobClient:  map 7% reduce 0%
> 08/11/04 12:58:14 INFO mapred.JobClient:  map 10% reduce 0%
> 08/11/04 12:58:19 INFO mapred.JobClient:  map 12% reduce 0%
> 08/11/04 12:58:21 INFO mapred.JobClient:  map 12% reduce 1%
> 08/11/04 12:58:25 INFO mapred.JobClient:  map 15% reduce 1%
> 08/11/04 12:58:26 INFO mapred.JobClient:  map 17% reduce 2%
> 08/11/04 12:58:28 INFO mapred.JobClient:  map 19% reduce 2%
> 08/11/04 12:58:33 INFO mapred.JobClient:  map 21% reduce 2%
> 08/11/04 12:58:36 INFO mapred.JobClient:  map 21% reduce 3%
> 08/11/04 12:58:38 INFO mapred.JobClient:  map 23% reduce 3%
> 08/11/04 12:58:40 INFO mapred.JobClient:  map 28% reduce 3%
> 08/11/04 12:58:41 INFO mapred.JobClient:  map 32% reduce 4%
> 08/11/04 12:58:46 INFO mapred.JobClient:  map 32% reduce 5%
> 08/11/04 12:58:49 INFO mapred.JobClient:  map 34% reduce 5%
> 08/11/04 12:58:51 INFO mapred.JobClient:  map 36% reduce 6%
> 08/11/04 12:58:53 INFO mapred.JobClient:  map 38% reduce 6%
> 08/11/04 12:58:54 INFO mapred.JobClient:  map 42% reduce 6%
> 08/11/04 12:58:55 INFO mapred.JobClient:  map 42% reduce 7%
> 08/11/04 12:59:03 INFO mapred.JobClient:  map 42% reduce 8%
> 08/11/04 12:59:05 INFO mapred.JobClient:  map 46% reduce 8%
> 08/11/04 12:59:06 INFO mapred.JobClient:  map 46% reduce 9%
> 08/11/04 12:59:07 INFO mapred.JobClient:  map 48% reduce 9%
> 08/11/04 12:59:08 INFO mapred.JobClient:  map 54% reduce 9%
> 08/11/04 12:59:12 INFO mapred.JobClient:  map 58% reduce 9%
> 08/11/04 12:59:16 INFO mapred.JobClient:  map 58% reduce 10%
> 08/11/04 12:59:18 INFO mapred.JobClient:  map 58% reduce 11%
> 08/11/04 12:59:21 INFO mapred.JobClient:  map 63% reduce 12%
> 08/11/04 12:59:23 INFO mapred.JobClient:  map 65% reduce 12%
> 08/11/04 12:59:26 INFO mapred.JobClient:  map 65% reduce 13%
> 08/11/04 12:59:27 INFO mapred.JobClient:  map 69% reduce 13%
> 08/11/04 12:59:28 INFO mapred.JobClient:  map 73% reduce 13%
> 08/11/04 12:59:31 INFO mapred.JobClient:  map 75% reduce 13%
> 08/11/04 12:59:33 INFO mapred.JobClient:  map 77% reduce 14%
> 08/11/04 12:59:34 INFO mapred.JobClient:  map 79% reduce 14%
> 08/11/04 12:59:35 INFO mapred.JobClient:  map 81% reduce 15%
> 08/11/04 12:59:38 INFO mapred.JobClient:  map 81% reduce 16%
> 08/11/04 12:59:42 INFO mapred.JobClient:  map 81% reduce 17%
> 08/11/04 12:59:43 INFO mapred.JobClient:  map 85% reduce 17%
> 08/11/04 12:59:44 INFO mapred.JobClient:  map 91% reduce 17%
> 08/11/04 12:59:45 INFO mapred.JobClient:  map 93% reduce 18%
> 08/11/04 12:59:54 INFO mapred.JobClient:  map 95% reduce 19%
> 08/11/04 12:59:57 INFO mapred.JobClient:  map 95% reduce 20%
> 08/11/04 13:00:00 INFO mapred.JobClient:  map 95% reduce 21%
> 08/11/04 13:00:59 INFO mapred.JobClient: Task Id :
> attempt_200811041234_0002_m_000000_0, Status : FAILED
> org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Name:
> -4870261170067331403
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
>         at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
>
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>        at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>        at
> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
>         at
> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
>        at
> org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:174)
>        at
> org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:56)
>        at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>        at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>
> 08/11/04 13:01:22 INFO mapred.JobClient:  map 97% reduce 21%
> 08/11/04 13:01:37 INFO mapred.JobClient:  map 97% reduce 22%
> 08/11/04 13:02:00 INFO mapred.JobClient:  map 100% reduce 22%
> 08/11/04 13:02:08 INFO mapred.JobClient:  map 100% reduce 28%
> 08/11/04 13:02:09 INFO mapred.JobClient:  map 100% reduce 32%
> 08/11/04 13:02:10 INFO mapred.JobClient:  map 100% reduce 33%
> 08/11/04 13:02:11 INFO mapred.JobClient:  map 100% reduce 37%
> 08/11/04 13:02:13 INFO mapred.JobClient:  map 100% reduce 45%
> 08/11/04 13:02:14 INFO mapred.JobClient:  map 100% reduce 47%
> 08/11/04 13:02:16 INFO mapred.JobClient:  map 100% reduce 50%
> 08/11/04 13:02:17 INFO mapred.JobClient:  map 100% reduce 63%
> 08/11/04 13:02:21 INFO mapred.JobClient:  map 100% reduce 65%
> 08/11/04 13:02:23 INFO mapred.JobClient:  map 100% reduce 66%
> 08/11/04 13:02:26 INFO mapred.JobClient:  map 100% reduce 67%
> 08/11/04 13:02:28 INFO mapred.JobClient:  map 100% reduce 68%
> 08/11/04 13:02:31 INFO mapred.JobClient:  map 100% reduce 69%
> 08/11/04 13:02:36 INFO mapred.JobClient:  map 100% reduce 70%
> 08/11/04 13:02:38 INFO mapred.JobClient:  map 100% reduce 71%
> 08/11/04 13:02:42 INFO mapred.JobClient:  map 100% reduce 72%
> 08/11/04 13:02:46 INFO mapred.JobClient:  map 100% reduce 74%
> 08/11/04 13:02:47 INFO mapred.JobClient:  map 100% reduce 75%
> 08/11/04 13:02:48 INFO mapred.JobClient:  map 100% reduce 79%
> 08/11/04 13:02:51 INFO mapred.JobClient:  map 100% reduce 86%
> 08/11/04 13:02:52 INFO mapred.JobClient:  map 100% reduce 89%
> 08/11/04 13:02:55 INFO mapred.JobClient:  map 100% reduce 93%
> 08/11/04 13:02:56 INFO mapred.JobClient:  map 100% reduce 97%
> 08/11/04 13:02:57 INFO mapred.JobClient:  map 100% reduce 98%
> 08/11/04 13:02:59 INFO mapred.JobClient: Job complete:
> job_200811041234_0002
> 08/11/04 13:02:59 INFO mapred.JobClient: Counters: 13
> 08/11/04 13:02:59 INFO mapred.JobClient:   File Systems
> 08/11/04 13:02:59 INFO mapred.JobClient:     Local bytes read=1006933217
> 08/11/04 13:02:59 INFO mapred.JobClient:     Local bytes written=1730888348
> 08/11/04 13:02:59 INFO mapred.JobClient:   Job Counters
> 08/11/04 13:02:59 INFO mapred.JobClient:     Launched reduce tasks=28
> 08/11/04 13:02:59 INFO mapred.JobClient:     Launched map tasks=58
> 08/11/04 13:02:59 INFO mapred.JobClient:   Map-Reduce Framework
> 08/11/04 13:02:59 INFO mapred.JobClient:     Reduce input groups=200
> 08/11/04 13:02:59 INFO mapred.JobClient:     Combine output records=0
> 08/11/04 13:02:59 INFO mapred.JobClient:     Map input records=547
> 08/11/04 13:02:59 INFO mapred.JobClient:     Reduce output records=200
> 08/11/04 13:02:59 INFO mapred.JobClient:     Map output bytes=711260600
> 08/11/04 13:02:59 INFO mapred.JobClient:     Map input bytes=0
> 08/11/04 13:02:59 INFO mapred.JobClient:     Combine input records=0
> 08/11/04 13:02:59 INFO mapred.JobClient:     Map output records=109400
> 08/11/04 13:02:59 INFO mapred.JobClient:     Reduce input records=109400
> 08/11/04 13:03:05 INFO client.HBaseAdmin: Disabled DenseMatrix_randnxkdh
> 08/11/04 13:03:05 INFO client.HBaseAdmin: Deleted DenseMatrix_randnxkdh
> 08/11/04 13:03:10 INFO client.HBaseAdmin: Disabled DenseMatrix_randgufzg
> 08/11/04 13:03:10 INFO client.HBaseAdmin: Deleted DenseMatrix_randgufzg
> 08/11/04 13:03:16 INFO client.HBaseAdmin: Disabled DenseMatrix_randetuuu
> 08/11/04 13:03:17 INFO client.HBaseAdmin: Deleted DenseMatrix_randetuuu
>
>
> On Tue, Nov 4, 2008 at 12:33 PM, Samuel Guo <guosijie@gmail.com> wrote:
> > what size of matrix you used to run matrix-matrix mult?
> >
> > On Tue, Nov 4, 2008 at 11:21 AM, Edward J. Yoon <edwardyoon@apache.org
> >wrote:
> >
> >> Hmm... Yes, probably.
> >> Current mult algorithm needs one full scan by each rows (n). If it
> >> replaced to 2D blocking algorithm, it'll be solved.
> >>
> >> /Ed
> >>
> >> On Tue, Nov 4, 2008 at 12:07 PM, Samuel Guo <guosijie@gmail.com> wrote:
> >> > On Tue, Nov 4, 2008 at 10:30 AM, Edward J. Yoon <
> edwardyoon@apache.org
> >> >wrote:
> >> >
> >> >> While matrix-matrix mult, I received a message as below. Does anyone
> >> >> know about UnknownScannerException?
> >> >>
> >> >> ----
> >> >> 08/11/04 11:24:57 INFO mapred.JobClient:  map 90% reduce 28%
> >> >> 08/11/04 11:24:59 INFO mapred.JobClient:  map 90% reduce 30%
> >> >> 08/11/04 11:26:49 INFO mapred.JobClient: Task Id :
> >> >> attempt_200811041040_0003_m_000002_0, Status : FAILED
> >> >> org.apache.hadoop.hbase.UnknownScannerException:
> >> >> org.apache.hadoop.hbase.UnknownScannerException: Name:
> >> >> -1151932123796109789
> >> >>        at
> >> >>
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
> >> >>        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
> >> >>        at
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> >>        at java.lang.reflect.Method.invoke(Method.java:597)
> >> >>        at
> >> >> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
> >> >>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> >> >>
> >> >>        at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >> >> Method)
> >> >>        at
> >> >>
> >>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> >> >>        at
> >> >>
> >>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >> >>        at
> >> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >> >>        at
> >> >>
> >>
> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
> >> >>        at
> >> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
> >> >
> >> >
> >> > I doubt that there may be some region splitting/moving during
> scanning,
> >> so
> >> > we get the RemoteException while getRegionServerWithRetires.
> >> >
> >> >
> >> >>
> >> >>        at
> >> >>
> >>
> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
> >> >>        at
> >> >>
> >>
> org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:174)
> >> >>        at
> >> >>
> >>
> org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:56)
> >> >>        at
> >> >>
> >>
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
> >> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
> >> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >> >>        at
> >> >>
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >> >>
> >> >> 08/11/04 11:27:11 INFO mapred.JobClient:  map 100% reduce 30%
> >> >> 08/11/04 11:27:17 INFO mapred.JobClient:  map 100% reduce 64%
> >> >> 08/11/04 11:27:23 INFO mapred.JobClient: Job complete:
> >> >> job_200811041040_0003
> >> >> 08/11/04 11:27:23 INFO mapred.JobClient: Counters: 13
> >> >> 08/11/04 11:27:23 INFO mapred.JobClient:   File Systems
> >> >> 08/11/04 11:27:23 INFO mapred.JobClient:     Local bytes
> read=32057196
> >> >> 08/11/04 11:27:23 INFO mapred.JobClient:     Local bytes
> >> written=64599524
> >> >>
> >> >> --
> >> >> Best regards, Edward J. Yoon @ NHN, corp.
> >> >> edwardyoon@apache.org
> >> >> http://blog.udanax.org
> >> >>
> >> >
> >>
> >>
> >>
> >> --
> >> Best regards, Edward J. Yoon @ NHN, corp.
> >> edwardyoon@apache.org
> >> http://blog.udanax.org
> >>
> >
>
>
>
> --
> Best regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message