hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takuya Ueshin (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5663) MultithreadedTableMapper doesn't work.
Date Sat, 31 Mar 2012 14:36:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243172#comment-13243172
] 

Takuya Ueshin commented on HBASE-5663:
--------------------------------------

No, any MapReduce Job using current MultithreadedTableMapper would not work.

When the Mapper starts, it creates some threads and 'original' Mapper object which should
be called by the thread and MapperContext object used by the original mapper.
But because the constructor call of MapperContext through reflection is wrong, it throws the
NoSuchMethodException.

As I reported in an issue HBASE-5636, trunk version's test case TestMulitthreadedTableMapper
has a bug, so please test by new version test attached in it.
                
> MultithreadedTableMapper doesn't work.
> --------------------------------------
>
>                 Key: HBASE-5663
>                 URL: https://issues.apache.org/jira/browse/HBASE-5663
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>    Affects Versions: 0.94.0
>            Reporter: Takuya Ueshin
>         Attachments: HBASE-5663.patch
>
>
> MapReduce job using MultithreadedTableMapper goes down throwing the following Exception:
> {noformat}
> java.io.IOException: java.lang.NoSuchMethodException: org.apache.hadoop.mapreduce.Mapper$Context.<init>(org.apache.hadoop.conf.Configuration,
org.apache.hadoop.mapred.TaskAttemptID, org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader,
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter, org.apache.hadoop.hbase.mapreduce.TableOutputCommitter,
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter, org.apache.hadoop.hbase.mapreduce.TableSplit)
> 	at org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.<init>(MultithreadedTableMapper.java:260)
> 	at org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper.run(MultithreadedTableMapper.java:133)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.mapreduce.Mapper$Context.<init>(org.apache.hadoop.conf.Configuration,
org.apache.hadoop.mapred.TaskAttemptID, org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader,
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter, org.apache.hadoop.hbase.mapreduce.TableOutputCommitter,
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter, org.apache.hadoop.hbase.mapreduce.TableSplit)
> 	at java.lang.Class.getConstructor0(Class.java:2706)
> 	at java.lang.Class.getConstructor(Class.java:1657)
> 	at org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.<init>(MultithreadedTableMapper.java:241)
> 	... 8 more
> {noformat}
> This occured when the tasks are creating MapRunner threads.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message