hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11595) WAL files with encryption not flushed properly
Date Sat, 26 Jul 2014 17:46:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075441#comment-14075441

Andrew Purtell commented on HBASE-11595:

My attempt to reproduce this issue:
# Set up Hadoop 2.4.1 namenode, secondarynamenode, and datanode on a dev box.
# Set up HBase 0.98.5-SNAPSHOT hosted zk, master, and regionserver also on this dev box. Set
dfs.replication and hbase.regionserver.hlog.tolerable.lowreplication to 1. Set up a keystore
and enabled  WAL encryption.
# Created a test table.
# Used YCSB to write 1000 rows to the test table. No flushes observed.
# Used the shell to count the number of records in the test table. Count = 1000 rows
# kill -9 the regionserver process.
# Started a new regionserver process. Observed log splitting and replay in the regionserver
log, no errors.
# Used the shell to count the number of records in the test table. Count = 1000 rows

> WAL files with encryption not flushed properly
> ----------------------------------------------
>                 Key: HBASE-11595
>                 URL: https://issues.apache.org/jira/browse/HBASE-11595
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.3
>            Reporter: Andrew Purtell
>            Priority: Critical
>             Fix For: 0.99.0, 0.98.5, 2.0.0
> Reported using HBase 0.98.3 and HDFS 2.4.1
> All data before failure has not yet been flushed so only exists in the WAL files. During
distributed splitting, the WAL has either not been written out and synced in the same way
as an unencrypted WAL or is unreadable:
> {noformat}
> 2014-07-26 19:29:16,160 ERROR [RS_LOG_REPLAY_OPS-host1:60020-0] codec.BaseDecoder: Partial
cell read caused by EOF: java.io.IOException: Premature EOF from inputStream
> {noformat}
> This file is still moved to oldWALs even though splitting failed. 
> Setting 'hbase.regionserver.wal.encryption' to false allows data recovery.

This message was sent by Atlassian JIRA

View raw message