hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17012) Handle Offheap cells in CompressedKvEncoder
Date Wed, 16 Nov 2016 11:06:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15670165#comment-15670165
] 

ramkrishna.s.vasudevan commented on HBASE-17012:
------------------------------------------------

I think EncryptedKVEncoder is ok as such. Because even if we 
{code}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
      OutputStream cout = encryptor.createEncryptionStream(baos);
{code}
wrap the BAOS with ByteBufferWriterOuputStream, still what we write is to the cout (that is
the encrypted stream).
{code}
StreamUtils.writeRawVInt32(cout, KeyValueUtil.keyLength(cell));
      StreamUtils.writeRawVInt32(cout, cell.getValueLength());
      // To support tags
      StreamUtils.writeRawVInt32(cout, tlen);

      // Write row, qualifier, and family
      StreamUtils.writeRawVInt32(cout, cell.getRowLength());
      cout.write(cell.getRowArray(), cell.getRowOffset(), cell.getRowLength());
      StreamUtils.writeRawVInt32(cout, cell.getFamilyLength());
      cout.write(cell.getFamilyArray(), cell.getFamilyOffset(), cell.getFamilyLength());

{code}
So am not sure how to write the ByteBuffer content directly to cout. But I could actually
use the hbase's BAOS and try writing to it and then do a getBuffer() and use that buffer to
write to cout.
But finally we need that baos to actually write it to the actual outputstream 'out'.
So am not sure if this will be same as writing individual byte[] using cout.

> Handle Offheap cells in CompressedKvEncoder
> -------------------------------------------
>
>                 Key: HBASE-17012
>                 URL: https://issues.apache.org/jira/browse/HBASE-17012
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>    Affects Versions: 2.0.0
>            Reporter: Anoop Sam John
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17012_1.patch
>
>
> When we deal with off heap cells we will end up copying Cell components on heap
> {code}
> public void write(Cell cell) throws IOException {
> .................
>       write(cell.getRowArray(), cell.getRowOffset(), cell.getRowLength(), compression.rowDict);
>       write(cell.getFamilyArray(), cell.getFamilyOffset(), cell.getFamilyLength(),
>           compression.familyDict);
>       write(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength(),
>           compression.qualifierDict);
> ......
>       out.write(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength());
> ...
> {code}
> We need to avoid this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message