hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clint Morgan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1858) Master can't split logs created by THBase
Date Thu, 24 Sep 2009 21:02:16 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Clint Morgan updated HBASE-1858:

    Attachment: 1858.patch

This patch clean up the transaction WALing, and makes it work. 

- Introduce config property "hbase.regionserver.hlog.keyclass" which is used to instantiate

- When we commit a transaction, we let the puts/deletes go into the normal WAL. This way,
they are handled normally during recovery. The only time we need to do special WAL recovery
in the transactional layer is when we find a transaction that started, but does not have an
commit or abort message. In this case, its status should be in the "global" trx log.

- Clean up the use of the "global" transaction log. This holds the state of a transaction
while the transaction is still in-process. This state can be forgotten after a successful
commit/abort. This state is only used when we recover from the WAL and don't know what actually
happened to the transaction.

- Ports HbaseBackTransactionalLogger to the new API.

- fixes a bug: when multiple puts in the same transaction to the same cell, make the last
put should be used for a trx-local get.

I tested the WAL recovery and it works for me.

> Master can't split logs created by THBase
> -----------------------------------------
>                 Key: HBASE-1858
>                 URL: https://issues.apache.org/jira/browse/HBASE-1858
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.20.0
>            Reporter: Clint Morgan
>            Assignee: Clint Morgan
>             Fix For: 0.20.1, 0.21.0
>         Attachments: 1858.patch
> When master tries to split logs created by THbase, it fails because it tries to read
in the wrong key type. (THBase subclasses HLogKey to add fields to the key).
> 2009-09-16 09:03:01,943 WARN org.apache.hadoop.hbase.regionserver.HLog:
> Exception processing
> hdfs://domU-12-31-39-07-CC-A2.compute-1.internal:9000/hbase/.logs/domU-12-31-39-07-CC-A2.compute-1.internal,60020,1253103101743/hlog.dat.1253103102168
> -- continuing. Possible DATA LOSS!
> java.io.IOException: wrong key class:
> org.apache.hadoop.hbase.regionserver.HLogKey is not class
> org.apache.hadoop.hbase.regionserver.transactional.THLogKey
>        at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1824)
>        at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876)
>        at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:880)
>        at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802)
>        at
> org.apache.hadoop.hbase.master.ProcessServerShutdown.process(ProcessServerShutdown.java:274)
>        at
> org.apache.hadoop.hbase.master.HMaster.processToDoQueue(HMaster.java:492)
>        at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:426)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message