hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13096) NPE from SecureWALCellCodec$EncryptedKvEncoder#write when using WAL encryption and Phoenix secondary indexes
Date Thu, 24 Nov 2016 16:22:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15693668#comment-15693668

Andrew Purtell commented on HBASE-13096:

This is not a bug unfortunately. The secure codec MUST be used if you want the cells in the
WAL to be encrypted. The reason the secure writer doesn't consider any other codec option
is because no other option makes sense. Otherwise why would you want to use the secure writer,
if WAL entries are not encrypted? So the answer here is Phoenix secondary indexes are not
compatible with WAL encryption, and won't be, unless Phoenix provides support for the secure

We can still make a change to the secure writer for it to honor configuration that says to
use a different codec, but if that codec doesn't implement encryption (like the current Phoenix
index codec) then you are writing data in the clear to HDFS and you are inherently compromised.

> NPE from SecureWALCellCodec$EncryptedKvEncoder#write when using WAL encryption and Phoenix
secondary indexes
> ------------------------------------------------------------------------------------------------------------
>                 Key: HBASE-13096
>                 URL: https://issues.apache.org/jira/browse/HBASE-13096
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.6
>            Reporter: Andrew Purtell
>              Labels: phoenix
> On user@phoenix Dhavi Rami reported:
> {quote}
> I tried using phoenix in hBase with Transparent Encryption of Data At Rest enabled (
AES encryption) 
> Works fine for a table with primary key column.
> But it doesn't work if I create Secondary index on that tables.I tried to dig deep into
the problem and found WAL file encryption throws exception when I have Global Secondary Index
created on my mutable table.
> Following is the error I was getting on one of the region server.
> {noformat}
> 2015-02-20 10:44:48,768 ERROR org.apache.hadoop.hbase.regionserver.wal.FSHLog: UNEXPECTED
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.util.Bytes.toInt(Bytes.java:767)
>         at org.apache.hadoop.hbase.util.Bytes.toInt(Bytes.java:754)
>         at org.apache.hadoop.hbase.KeyValue.getKeyLength(KeyValue.java:1253)
>         at org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec$EncryptedKvEncoder.write(SecureWALCellCodec.java:194)
>         at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:117)
>         at org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1137)
>         at java.lang.Thread.run(Thread.java:745)
> 2015-02-20 10:44:48,776 INFO org.apache.hadoop.hbase.regionserver.wal.FSHLog: regionserver60020-WAL.AsyncWriter
> {noformat}
> I had to disable WAL encryption, and it started working fine with secondary Index. So
Hfile encryption works with secondary index but WAL encryption doesn't work.
> {quote}
> Parking this here for later investigation. For now I'm going to assume this is something
in SecureWALCellCodec that needs looking at, but if it turns out to be a Phoenix indexer issue
I will move this JIRA there.

This message was sent by Atlassian JIRA

View raw message