hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14557) MapReduce WALPlayer issue with NoTagsKeyValue
Date Wed, 07 Oct 2015 10:41:26 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946656#comment-14946656
] 

Anoop Sam John commented on HBASE-14557:
----------------------------------------

It wont be good to add any new state variable to KeyValue and increase its heap size. We will
keep this type in Memstore also.

We can do one thing may be
{code}
KeyValue kv = KeyValueUtil.ensureKeyValue(cell);
            if (WALEdit.isMetaEditFamily(kv)) continue;
            context.write(new ImmutableBytesWritable(CellUtil.cloneRow(kv)), kv);
{code}
before we write to context, we can make sure the type is KeyValue only. (exact class)..  ensureKeyValue
just checks type.NoTagsKeyValue is an extension of KeyValue only.  When NoTagsKeyValue comes,
we can just create a new KeyValue object from it using its buffer, offset and length.

> MapReduce WALPlayer issue with NoTagsKeyValue
> ---------------------------------------------
>
>                 Key: HBASE-14557
>                 URL: https://issues.apache.org/jira/browse/HBASE-14557
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>            Reporter: Jerry He
>
> Running MapReduce WALPlayer to convert WAL info HFiles:
> {noformat}
> 15/10/05 20:28:08 INFO mapred.JobClient: Task Id : attempt_201508031611_0029_m_000000_0,
Status : FAILED
> java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.hbase.KeyValue,
recieved org.apache.hadoop.hbase.NoTagsKeyValue
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:997)
>         at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
>         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
>         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
>         at org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:111)
>         at org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:96)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(AccessController.java:369)
>         at javax.security.auth.Subject.doAs(Subject.java:572)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message