hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15709) Handle large edits for asynchronous WAL
Date Sun, 23 Oct 2016 01:28:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15598815#comment-15598815
] 

Duo Zhang commented on HBASE-15709:
-----------------------------------

Oh I missed one word, it should be 'prevent writing a block larger than the specific block
size'...

And I mean stop the feature being implemented in HDFS. If they add the check at NN or DN,
then our AsyncFSWAL will be broken if we have an edit that larger than the specific block
size...

Or that, we could set a very large block size when creating the file as we have a separated
log roll size?

> Handle large edits for asynchronous WAL
> ---------------------------------------
>
>                 Key: HBASE-15709
>                 URL: https://issues.apache.org/jira/browse/HBASE-15709
>             Project: HBase
>          Issue Type: Sub-task
>          Components: io, wal
>            Reporter: Duo Zhang
>            Priority: Critical
>
> First, FanOutOneBlockAsyncDFSOutput can not work if the buffered data is larger than
PacketReceiver.MAX_PACKET_SIZE(16MB).
> Second, since we only allow one block here, we need to make sure we do not exceed the
block size after writing a large chunk of data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message