hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15709) Handle large edits for asynchronous WAL
Date Mon, 24 Oct 2016 05:34:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15601002#comment-15601002
] 

Duo Zhang commented on HBASE-15709:
-----------------------------------

{quote}
By the time we get into flush0, all bytes have been checksummed?
{quote}
No, the checksum is computed in the flushBuffer method. In flush0 we may split the buf to
multiple chunks if it exceeded the max packet size limit. The align is done at the end of
flush0, where we create a new ByteBuf and copy the unaligned data into it. Yes, not in this
patch, right after the last line of the diff of FanOutOneBlockAsyncDFSOutput.java...



> Handle large edits for asynchronous WAL
> ---------------------------------------
>
>                 Key: HBASE-15709
>                 URL: https://issues.apache.org/jira/browse/HBASE-15709
>             Project: HBase
>          Issue Type: Sub-task
>          Components: io, wal
>    Affects Versions: 2.0.0
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15709.patch
>
>
> First, FanOutOneBlockAsyncDFSOutput can not work if the buffered data is larger than
PacketReceiver.MAX_PACKET_SIZE(16MB).
> Second, since we only allow one block here, we need to make sure we do not exceed the
block size after writing a large chunk of data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message