hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15709) Handle large edits for asynchronous WAL
Date Sat, 22 Oct 2016 05:56:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15597245#comment-15597245
] 

Duo Zhang commented on HBASE-15709:
-----------------------------------

{quote}
Block size seems fine as limit. Block would run on till opportunity to roll so could be bigger
than 256M, right?
{quote}

I'm not sure if there are some checks of the block size at datanode or namenode. And if there
is no check for now, they maybe added later, we should not rely on an unstable feature...

Anyway, let me try to add the support for chunking and finish this issue. At least this will
move the limit from 16M to 256M...

Thanks.

> Handle large edits for asynchronous WAL
> ---------------------------------------
>
>                 Key: HBASE-15709
>                 URL: https://issues.apache.org/jira/browse/HBASE-15709
>             Project: HBase
>          Issue Type: Sub-task
>          Components: io, wal
>            Reporter: Duo Zhang
>            Priority: Critical
>
> First, FanOutOneBlockAsyncDFSOutput can not work if the buffered data is larger than
PacketReceiver.MAX_PACKET_SIZE(16MB).
> Second, since we only allow one block here, we need to make sure we do not exceed the
block size after writing a large chunk of data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message