hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: UnknownScannerException
Date Tue, 04 Nov 2008 05:31:40 GMT
Hmm... Current mult operation algorithm seems not good.

On Tue, Nov 4, 2008 at 1:04 PM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> This is my full log. (5 node test cluster)
>
> ----
> hadoop@vldb ~/hadoop $ bin/hadoop jar hama-0.1.0-dev-examples.jar mult
> -m 50 -r 20 200 200
> 08/11/04 12:57:28 INFO hama.AbstractMatrix: Initializing the matrix storage.
> 08/11/04 12:57:39 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randnxkdh
> 08/11/04 12:57:43 INFO hama.AbstractMatrix: Create the 200 * 200
> random matrix : DenseMatrix_randnxkdh
> 08/11/04 12:57:43 INFO hama.AbstractMatrix: Initializing the matrix storage.
> 08/11/04 12:57:48 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randgufzg
> 08/11/04 12:57:49 INFO hama.AbstractMatrix: Create the 200 * 200
> random matrix : DenseMatrix_randgufzg
> 08/11/04 12:57:49 INFO hama.AbstractMatrix: Initializing the matrix storage.
> 08/11/04 12:57:54 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randetuuu
> 08/11/04 12:57:54 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the
> same.
> 08/11/04 12:57:54 WARN mapred.JobClient: No job jar file set.  User
> classes may not be found. See JobConf(Class) or
> JobConf#setJar(String).
> 08/11/04 12:57:55 INFO mapred.JobClient: Running job: job_200811041234_0002
> 08/11/04 12:57:56 INFO mapred.JobClient:  map 0% reduce 0%
> 08/11/04 12:58:03 INFO mapred.JobClient:  map 1% reduce 0%
> 08/11/04 12:58:10 INFO mapred.JobClient:  map 3% reduce 0%
> 08/11/04 12:58:12 INFO mapred.JobClient:  map 7% reduce 0%
> 08/11/04 12:58:14 INFO mapred.JobClient:  map 10% reduce 0%
> 08/11/04 12:58:19 INFO mapred.JobClient:  map 12% reduce 0%
> 08/11/04 12:58:21 INFO mapred.JobClient:  map 12% reduce 1%
> 08/11/04 12:58:25 INFO mapred.JobClient:  map 15% reduce 1%
> 08/11/04 12:58:26 INFO mapred.JobClient:  map 17% reduce 2%
> 08/11/04 12:58:28 INFO mapred.JobClient:  map 19% reduce 2%
> 08/11/04 12:58:33 INFO mapred.JobClient:  map 21% reduce 2%
> 08/11/04 12:58:36 INFO mapred.JobClient:  map 21% reduce 3%
> 08/11/04 12:58:38 INFO mapred.JobClient:  map 23% reduce 3%
> 08/11/04 12:58:40 INFO mapred.JobClient:  map 28% reduce 3%
> 08/11/04 12:58:41 INFO mapred.JobClient:  map 32% reduce 4%
> 08/11/04 12:58:46 INFO mapred.JobClient:  map 32% reduce 5%
> 08/11/04 12:58:49 INFO mapred.JobClient:  map 34% reduce 5%
> 08/11/04 12:58:51 INFO mapred.JobClient:  map 36% reduce 6%
> 08/11/04 12:58:53 INFO mapred.JobClient:  map 38% reduce 6%
> 08/11/04 12:58:54 INFO mapred.JobClient:  map 42% reduce 6%
> 08/11/04 12:58:55 INFO mapred.JobClient:  map 42% reduce 7%
> 08/11/04 12:59:03 INFO mapred.JobClient:  map 42% reduce 8%
> 08/11/04 12:59:05 INFO mapred.JobClient:  map 46% reduce 8%
> 08/11/04 12:59:06 INFO mapred.JobClient:  map 46% reduce 9%
> 08/11/04 12:59:07 INFO mapred.JobClient:  map 48% reduce 9%
> 08/11/04 12:59:08 INFO mapred.JobClient:  map 54% reduce 9%
> 08/11/04 12:59:12 INFO mapred.JobClient:  map 58% reduce 9%
> 08/11/04 12:59:16 INFO mapred.JobClient:  map 58% reduce 10%
> 08/11/04 12:59:18 INFO mapred.JobClient:  map 58% reduce 11%
> 08/11/04 12:59:21 INFO mapred.JobClient:  map 63% reduce 12%
> 08/11/04 12:59:23 INFO mapred.JobClient:  map 65% reduce 12%
> 08/11/04 12:59:26 INFO mapred.JobClient:  map 65% reduce 13%
> 08/11/04 12:59:27 INFO mapred.JobClient:  map 69% reduce 13%
> 08/11/04 12:59:28 INFO mapred.JobClient:  map 73% reduce 13%
> 08/11/04 12:59:31 INFO mapred.JobClient:  map 75% reduce 13%
> 08/11/04 12:59:33 INFO mapred.JobClient:  map 77% reduce 14%
> 08/11/04 12:59:34 INFO mapred.JobClient:  map 79% reduce 14%
> 08/11/04 12:59:35 INFO mapred.JobClient:  map 81% reduce 15%
> 08/11/04 12:59:38 INFO mapred.JobClient:  map 81% reduce 16%
> 08/11/04 12:59:42 INFO mapred.JobClient:  map 81% reduce 17%
> 08/11/04 12:59:43 INFO mapred.JobClient:  map 85% reduce 17%
> 08/11/04 12:59:44 INFO mapred.JobClient:  map 91% reduce 17%
> 08/11/04 12:59:45 INFO mapred.JobClient:  map 93% reduce 18%
> 08/11/04 12:59:54 INFO mapred.JobClient:  map 95% reduce 19%
> 08/11/04 12:59:57 INFO mapred.JobClient:  map 95% reduce 20%
> 08/11/04 13:00:00 INFO mapred.JobClient:  map 95% reduce 21%
> 08/11/04 13:00:59 INFO mapred.JobClient: Task Id :
> attempt_200811041234_0002_m_000000_0, Status : FAILED
> org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Name:
> -4870261170067331403
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
>        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
>
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>        at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
>        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
>        at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
>        at org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:174)
>        at org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:56)
>        at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>        at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>
> 08/11/04 13:01:22 INFO mapred.JobClient:  map 97% reduce 21%
> 08/11/04 13:01:37 INFO mapred.JobClient:  map 97% reduce 22%
> 08/11/04 13:02:00 INFO mapred.JobClient:  map 100% reduce 22%
> 08/11/04 13:02:08 INFO mapred.JobClient:  map 100% reduce 28%
> 08/11/04 13:02:09 INFO mapred.JobClient:  map 100% reduce 32%
> 08/11/04 13:02:10 INFO mapred.JobClient:  map 100% reduce 33%
> 08/11/04 13:02:11 INFO mapred.JobClient:  map 100% reduce 37%
> 08/11/04 13:02:13 INFO mapred.JobClient:  map 100% reduce 45%
> 08/11/04 13:02:14 INFO mapred.JobClient:  map 100% reduce 47%
> 08/11/04 13:02:16 INFO mapred.JobClient:  map 100% reduce 50%
> 08/11/04 13:02:17 INFO mapred.JobClient:  map 100% reduce 63%
> 08/11/04 13:02:21 INFO mapred.JobClient:  map 100% reduce 65%
> 08/11/04 13:02:23 INFO mapred.JobClient:  map 100% reduce 66%
> 08/11/04 13:02:26 INFO mapred.JobClient:  map 100% reduce 67%
> 08/11/04 13:02:28 INFO mapred.JobClient:  map 100% reduce 68%
> 08/11/04 13:02:31 INFO mapred.JobClient:  map 100% reduce 69%
> 08/11/04 13:02:36 INFO mapred.JobClient:  map 100% reduce 70%
> 08/11/04 13:02:38 INFO mapred.JobClient:  map 100% reduce 71%
> 08/11/04 13:02:42 INFO mapred.JobClient:  map 100% reduce 72%
> 08/11/04 13:02:46 INFO mapred.JobClient:  map 100% reduce 74%
> 08/11/04 13:02:47 INFO mapred.JobClient:  map 100% reduce 75%
> 08/11/04 13:02:48 INFO mapred.JobClient:  map 100% reduce 79%
> 08/11/04 13:02:51 INFO mapred.JobClient:  map 100% reduce 86%
> 08/11/04 13:02:52 INFO mapred.JobClient:  map 100% reduce 89%
> 08/11/04 13:02:55 INFO mapred.JobClient:  map 100% reduce 93%
> 08/11/04 13:02:56 INFO mapred.JobClient:  map 100% reduce 97%
> 08/11/04 13:02:57 INFO mapred.JobClient:  map 100% reduce 98%
> 08/11/04 13:02:59 INFO mapred.JobClient: Job complete: job_200811041234_0002
> 08/11/04 13:02:59 INFO mapred.JobClient: Counters: 13
> 08/11/04 13:02:59 INFO mapred.JobClient:   File Systems
> 08/11/04 13:02:59 INFO mapred.JobClient:     Local bytes read=1006933217
> 08/11/04 13:02:59 INFO mapred.JobClient:     Local bytes written=1730888348
> 08/11/04 13:02:59 INFO mapred.JobClient:   Job Counters
> 08/11/04 13:02:59 INFO mapred.JobClient:     Launched reduce tasks=28
> 08/11/04 13:02:59 INFO mapred.JobClient:     Launched map tasks=58
> 08/11/04 13:02:59 INFO mapred.JobClient:   Map-Reduce Framework
> 08/11/04 13:02:59 INFO mapred.JobClient:     Reduce input groups=200
> 08/11/04 13:02:59 INFO mapred.JobClient:     Combine output records=0
> 08/11/04 13:02:59 INFO mapred.JobClient:     Map input records=547
> 08/11/04 13:02:59 INFO mapred.JobClient:     Reduce output records=200
> 08/11/04 13:02:59 INFO mapred.JobClient:     Map output bytes=711260600
> 08/11/04 13:02:59 INFO mapred.JobClient:     Map input bytes=0
> 08/11/04 13:02:59 INFO mapred.JobClient:     Combine input records=0
> 08/11/04 13:02:59 INFO mapred.JobClient:     Map output records=109400
> 08/11/04 13:02:59 INFO mapred.JobClient:     Reduce input records=109400
> 08/11/04 13:03:05 INFO client.HBaseAdmin: Disabled DenseMatrix_randnxkdh
> 08/11/04 13:03:05 INFO client.HBaseAdmin: Deleted DenseMatrix_randnxkdh
> 08/11/04 13:03:10 INFO client.HBaseAdmin: Disabled DenseMatrix_randgufzg
> 08/11/04 13:03:10 INFO client.HBaseAdmin: Deleted DenseMatrix_randgufzg
> 08/11/04 13:03:16 INFO client.HBaseAdmin: Disabled DenseMatrix_randetuuu
> 08/11/04 13:03:17 INFO client.HBaseAdmin: Deleted DenseMatrix_randetuuu
>
>
> On Tue, Nov 4, 2008 at 12:33 PM, Samuel Guo <guosijie@gmail.com> wrote:
>> what size of matrix you used to run matrix-matrix mult?
>>
>> On Tue, Nov 4, 2008 at 11:21 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>>
>>> Hmm... Yes, probably.
>>> Current mult algorithm needs one full scan by each rows (n). If it
>>> replaced to 2D blocking algorithm, it'll be solved.
>>>
>>> /Ed
>>>
>>> On Tue, Nov 4, 2008 at 12:07 PM, Samuel Guo <guosijie@gmail.com> wrote:
>>> > On Tue, Nov 4, 2008 at 10:30 AM, Edward J. Yoon <edwardyoon@apache.org
>>> >wrote:
>>> >
>>> >> While matrix-matrix mult, I received a message as below. Does anyone
>>> >> know about UnknownScannerException?
>>> >>
>>> >> ----
>>> >> 08/11/04 11:24:57 INFO mapred.JobClient:  map 90% reduce 28%
>>> >> 08/11/04 11:24:59 INFO mapred.JobClient:  map 90% reduce 30%
>>> >> 08/11/04 11:26:49 INFO mapred.JobClient: Task Id :
>>> >> attempt_200811041040_0003_m_000002_0, Status : FAILED
>>> >> org.apache.hadoop.hbase.UnknownScannerException:
>>> >> org.apache.hadoop.hbase.UnknownScannerException: Name:
>>> >> -1151932123796109789
>>> >>        at
>>> >>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
>>> >>        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
>>> >>        at
>>> >>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> >>        at java.lang.reflect.Method.invoke(Method.java:597)
>>> >>        at
>>> >> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
>>> >>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
>>> >>
>>> >>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>> >> Method)
>>> >>        at
>>> >>
>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>>> >>        at
>>> >>
>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>>> >>        at
>>> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>>> >>        at
>>> >>
>>> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
>>> >>        at
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
>>> >
>>> >
>>> > I doubt that there may be some region splitting/moving during scanning,
>>> so
>>> > we get the RemoteException while getRegionServerWithRetires.
>>> >
>>> >
>>> >>
>>> >>        at
>>> >>
>>> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
>>> >>        at
>>> >>
>>> org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:174)
>>> >>        at
>>> >>
>>> org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:56)
>>> >>        at
>>> >>
>>> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
>>> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
>>> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>>> >>        at
>>> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>>> >>
>>> >> 08/11/04 11:27:11 INFO mapred.JobClient:  map 100% reduce 30%
>>> >> 08/11/04 11:27:17 INFO mapred.JobClient:  map 100% reduce 64%
>>> >> 08/11/04 11:27:23 INFO mapred.JobClient: Job complete:
>>> >> job_200811041040_0003
>>> >> 08/11/04 11:27:23 INFO mapred.JobClient: Counters: 13
>>> >> 08/11/04 11:27:23 INFO mapred.JobClient:   File Systems
>>> >> 08/11/04 11:27:23 INFO mapred.JobClient:     Local bytes read=32057196
>>> >> 08/11/04 11:27:23 INFO mapred.JobClient:     Local bytes
>>> written=64599524
>>> >>
>>> >> --
>>> >> Best regards, Edward J. Yoon @ NHN, corp.
>>> >> edwardyoon@apache.org
>>> >> http://blog.udanax.org
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> Best regards, Edward J. Yoon @ NHN, corp.
>>> edwardyoon@apache.org
>>> http://blog.udanax.org
>>>
>>
>
>
>
> --
> Best regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>



-- 
Best regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org

Mime
View raw message