incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: UnknownScannerException
Date Tue, 04 Nov 2008 04:04:15 GMT
This is my full log. (5 node test cluster)

----
hadoop@vldb ~/hadoop $ bin/hadoop jar hama-0.1.0-dev-examples.jar mult
-m 50 -r 20 200 200
08/11/04 12:57:28 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/04 12:57:39 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randnxkdh
08/11/04 12:57:43 INFO hama.AbstractMatrix: Create the 200 * 200
random matrix : DenseMatrix_randnxkdh
08/11/04 12:57:43 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/04 12:57:48 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randgufzg
08/11/04 12:57:49 INFO hama.AbstractMatrix: Create the 200 * 200
random matrix : DenseMatrix_randgufzg
08/11/04 12:57:49 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/04 12:57:54 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randetuuu
08/11/04 12:57:54 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the
same.
08/11/04 12:57:54 WARN mapred.JobClient: No job jar file set.  User
classes may not be found. See JobConf(Class) or
JobConf#setJar(String).
08/11/04 12:57:55 INFO mapred.JobClient: Running job: job_200811041234_0002
08/11/04 12:57:56 INFO mapred.JobClient:  map 0% reduce 0%
08/11/04 12:58:03 INFO mapred.JobClient:  map 1% reduce 0%
08/11/04 12:58:10 INFO mapred.JobClient:  map 3% reduce 0%
08/11/04 12:58:12 INFO mapred.JobClient:  map 7% reduce 0%
08/11/04 12:58:14 INFO mapred.JobClient:  map 10% reduce 0%
08/11/04 12:58:19 INFO mapred.JobClient:  map 12% reduce 0%
08/11/04 12:58:21 INFO mapred.JobClient:  map 12% reduce 1%
08/11/04 12:58:25 INFO mapred.JobClient:  map 15% reduce 1%
08/11/04 12:58:26 INFO mapred.JobClient:  map 17% reduce 2%
08/11/04 12:58:28 INFO mapred.JobClient:  map 19% reduce 2%
08/11/04 12:58:33 INFO mapred.JobClient:  map 21% reduce 2%
08/11/04 12:58:36 INFO mapred.JobClient:  map 21% reduce 3%
08/11/04 12:58:38 INFO mapred.JobClient:  map 23% reduce 3%
08/11/04 12:58:40 INFO mapred.JobClient:  map 28% reduce 3%
08/11/04 12:58:41 INFO mapred.JobClient:  map 32% reduce 4%
08/11/04 12:58:46 INFO mapred.JobClient:  map 32% reduce 5%
08/11/04 12:58:49 INFO mapred.JobClient:  map 34% reduce 5%
08/11/04 12:58:51 INFO mapred.JobClient:  map 36% reduce 6%
08/11/04 12:58:53 INFO mapred.JobClient:  map 38% reduce 6%
08/11/04 12:58:54 INFO mapred.JobClient:  map 42% reduce 6%
08/11/04 12:58:55 INFO mapred.JobClient:  map 42% reduce 7%
08/11/04 12:59:03 INFO mapred.JobClient:  map 42% reduce 8%
08/11/04 12:59:05 INFO mapred.JobClient:  map 46% reduce 8%
08/11/04 12:59:06 INFO mapred.JobClient:  map 46% reduce 9%
08/11/04 12:59:07 INFO mapred.JobClient:  map 48% reduce 9%
08/11/04 12:59:08 INFO mapred.JobClient:  map 54% reduce 9%
08/11/04 12:59:12 INFO mapred.JobClient:  map 58% reduce 9%
08/11/04 12:59:16 INFO mapred.JobClient:  map 58% reduce 10%
08/11/04 12:59:18 INFO mapred.JobClient:  map 58% reduce 11%
08/11/04 12:59:21 INFO mapred.JobClient:  map 63% reduce 12%
08/11/04 12:59:23 INFO mapred.JobClient:  map 65% reduce 12%
08/11/04 12:59:26 INFO mapred.JobClient:  map 65% reduce 13%
08/11/04 12:59:27 INFO mapred.JobClient:  map 69% reduce 13%
08/11/04 12:59:28 INFO mapred.JobClient:  map 73% reduce 13%
08/11/04 12:59:31 INFO mapred.JobClient:  map 75% reduce 13%
08/11/04 12:59:33 INFO mapred.JobClient:  map 77% reduce 14%
08/11/04 12:59:34 INFO mapred.JobClient:  map 79% reduce 14%
08/11/04 12:59:35 INFO mapred.JobClient:  map 81% reduce 15%
08/11/04 12:59:38 INFO mapred.JobClient:  map 81% reduce 16%
08/11/04 12:59:42 INFO mapred.JobClient:  map 81% reduce 17%
08/11/04 12:59:43 INFO mapred.JobClient:  map 85% reduce 17%
08/11/04 12:59:44 INFO mapred.JobClient:  map 91% reduce 17%
08/11/04 12:59:45 INFO mapred.JobClient:  map 93% reduce 18%
08/11/04 12:59:54 INFO mapred.JobClient:  map 95% reduce 19%
08/11/04 12:59:57 INFO mapred.JobClient:  map 95% reduce 20%
08/11/04 13:00:00 INFO mapred.JobClient:  map 95% reduce 21%
08/11/04 13:00:59 INFO mapred.JobClient: Task Id :
attempt_200811041234_0002_m_000000_0, Status : FAILED
org.apache.hadoop.hbase.UnknownScannerException:
org.apache.hadoop.hbase.UnknownScannerException: Name:
-4870261170067331403
        at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
        at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
        at org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:174)
        at org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:56)
        at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
        at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)

08/11/04 13:01:22 INFO mapred.JobClient:  map 97% reduce 21%
08/11/04 13:01:37 INFO mapred.JobClient:  map 97% reduce 22%
08/11/04 13:02:00 INFO mapred.JobClient:  map 100% reduce 22%
08/11/04 13:02:08 INFO mapred.JobClient:  map 100% reduce 28%
08/11/04 13:02:09 INFO mapred.JobClient:  map 100% reduce 32%
08/11/04 13:02:10 INFO mapred.JobClient:  map 100% reduce 33%
08/11/04 13:02:11 INFO mapred.JobClient:  map 100% reduce 37%
08/11/04 13:02:13 INFO mapred.JobClient:  map 100% reduce 45%
08/11/04 13:02:14 INFO mapred.JobClient:  map 100% reduce 47%
08/11/04 13:02:16 INFO mapred.JobClient:  map 100% reduce 50%
08/11/04 13:02:17 INFO mapred.JobClient:  map 100% reduce 63%
08/11/04 13:02:21 INFO mapred.JobClient:  map 100% reduce 65%
08/11/04 13:02:23 INFO mapred.JobClient:  map 100% reduce 66%
08/11/04 13:02:26 INFO mapred.JobClient:  map 100% reduce 67%
08/11/04 13:02:28 INFO mapred.JobClient:  map 100% reduce 68%
08/11/04 13:02:31 INFO mapred.JobClient:  map 100% reduce 69%
08/11/04 13:02:36 INFO mapred.JobClient:  map 100% reduce 70%
08/11/04 13:02:38 INFO mapred.JobClient:  map 100% reduce 71%
08/11/04 13:02:42 INFO mapred.JobClient:  map 100% reduce 72%
08/11/04 13:02:46 INFO mapred.JobClient:  map 100% reduce 74%
08/11/04 13:02:47 INFO mapred.JobClient:  map 100% reduce 75%
08/11/04 13:02:48 INFO mapred.JobClient:  map 100% reduce 79%
08/11/04 13:02:51 INFO mapred.JobClient:  map 100% reduce 86%
08/11/04 13:02:52 INFO mapred.JobClient:  map 100% reduce 89%
08/11/04 13:02:55 INFO mapred.JobClient:  map 100% reduce 93%
08/11/04 13:02:56 INFO mapred.JobClient:  map 100% reduce 97%
08/11/04 13:02:57 INFO mapred.JobClient:  map 100% reduce 98%
08/11/04 13:02:59 INFO mapred.JobClient: Job complete: job_200811041234_0002
08/11/04 13:02:59 INFO mapred.JobClient: Counters: 13
08/11/04 13:02:59 INFO mapred.JobClient:   File Systems
08/11/04 13:02:59 INFO mapred.JobClient:     Local bytes read=1006933217
08/11/04 13:02:59 INFO mapred.JobClient:     Local bytes written=1730888348
08/11/04 13:02:59 INFO mapred.JobClient:   Job Counters
08/11/04 13:02:59 INFO mapred.JobClient:     Launched reduce tasks=28
08/11/04 13:02:59 INFO mapred.JobClient:     Launched map tasks=58
08/11/04 13:02:59 INFO mapred.JobClient:   Map-Reduce Framework
08/11/04 13:02:59 INFO mapred.JobClient:     Reduce input groups=200
08/11/04 13:02:59 INFO mapred.JobClient:     Combine output records=0
08/11/04 13:02:59 INFO mapred.JobClient:     Map input records=547
08/11/04 13:02:59 INFO mapred.JobClient:     Reduce output records=200
08/11/04 13:02:59 INFO mapred.JobClient:     Map output bytes=711260600
08/11/04 13:02:59 INFO mapred.JobClient:     Map input bytes=0
08/11/04 13:02:59 INFO mapred.JobClient:     Combine input records=0
08/11/04 13:02:59 INFO mapred.JobClient:     Map output records=109400
08/11/04 13:02:59 INFO mapred.JobClient:     Reduce input records=109400
08/11/04 13:03:05 INFO client.HBaseAdmin: Disabled DenseMatrix_randnxkdh
08/11/04 13:03:05 INFO client.HBaseAdmin: Deleted DenseMatrix_randnxkdh
08/11/04 13:03:10 INFO client.HBaseAdmin: Disabled DenseMatrix_randgufzg
08/11/04 13:03:10 INFO client.HBaseAdmin: Deleted DenseMatrix_randgufzg
08/11/04 13:03:16 INFO client.HBaseAdmin: Disabled DenseMatrix_randetuuu
08/11/04 13:03:17 INFO client.HBaseAdmin: Deleted DenseMatrix_randetuuu


On Tue, Nov 4, 2008 at 12:33 PM, Samuel Guo <guosijie@gmail.com> wrote:
> what size of matrix you used to run matrix-matrix mult?
>
> On Tue, Nov 4, 2008 at 11:21 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>
>> Hmm... Yes, probably.
>> Current mult algorithm needs one full scan by each rows (n). If it
>> replaced to 2D blocking algorithm, it'll be solved.
>>
>> /Ed
>>
>> On Tue, Nov 4, 2008 at 12:07 PM, Samuel Guo <guosijie@gmail.com> wrote:
>> > On Tue, Nov 4, 2008 at 10:30 AM, Edward J. Yoon <edwardyoon@apache.org
>> >wrote:
>> >
>> >> While matrix-matrix mult, I received a message as below. Does anyone
>> >> know about UnknownScannerException?
>> >>
>> >> ----
>> >> 08/11/04 11:24:57 INFO mapred.JobClient:  map 90% reduce 28%
>> >> 08/11/04 11:24:59 INFO mapred.JobClient:  map 90% reduce 30%
>> >> 08/11/04 11:26:49 INFO mapred.JobClient: Task Id :
>> >> attempt_200811041040_0003_m_000002_0, Status : FAILED
>> >> org.apache.hadoop.hbase.UnknownScannerException:
>> >> org.apache.hadoop.hbase.UnknownScannerException: Name:
>> >> -1151932123796109789
>> >>        at
>> >>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
>> >>        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
>> >>        at
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >>        at java.lang.reflect.Method.invoke(Method.java:597)
>> >>        at
>> >> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
>> >>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
>> >>
>> >>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> >> Method)
>> >>        at
>> >>
>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>> >>        at
>> >>
>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>> >>        at
>> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>> >>        at
>> >>
>> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
>> >>        at
>> >>
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
>> >
>> >
>> > I doubt that there may be some region splitting/moving during scanning,
>> so
>> > we get the RemoteException while getRegionServerWithRetires.
>> >
>> >
>> >>
>> >>        at
>> >>
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
>> >>        at
>> >>
>> org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:174)
>> >>        at
>> >>
>> org.apache.hama.mapred.MatrixInputFormatBase$TableRecordReader.next(MatrixInputFormatBase.java:56)
>> >>        at
>> >>
>> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
>> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
>> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
>> >>        at
>> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
>> >>
>> >> 08/11/04 11:27:11 INFO mapred.JobClient:  map 100% reduce 30%
>> >> 08/11/04 11:27:17 INFO mapred.JobClient:  map 100% reduce 64%
>> >> 08/11/04 11:27:23 INFO mapred.JobClient: Job complete:
>> >> job_200811041040_0003
>> >> 08/11/04 11:27:23 INFO mapred.JobClient: Counters: 13
>> >> 08/11/04 11:27:23 INFO mapred.JobClient:   File Systems
>> >> 08/11/04 11:27:23 INFO mapred.JobClient:     Local bytes read=32057196
>> >> 08/11/04 11:27:23 INFO mapred.JobClient:     Local bytes
>> written=64599524
>> >>
>> >> --
>> >> Best regards, Edward J. Yoon @ NHN, corp.
>> >> edwardyoon@apache.org
>> >> http://blog.udanax.org
>> >>
>> >
>>
>>
>>
>> --
>> Best regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>



-- 
Best regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org

Mime
View raw message