hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15536) Make AsyncFSWAL as our default WAL
Date Sun, 02 Oct 2016 14:13:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15540424#comment-15540424
] 

Duo Zhang commented on HBASE-15536:
-----------------------------------

Seems there is no key value size limitation at server side currently. And the max key-value
size is only used to limit one key-value, which means you can add 100 cells into one put object
and each cell is 1MB, then the size of the put is 100MB and breaks our {{AsyncFSWAL}}...

We can introduce a new config named 'max.wal.edit.size' which is used to limit the size of
one atomic operation and set the default value less than 16MB? This could make it possible
to make AsyncFSWAL as our default WAL.

BTW, I think we could add the 'max.wal.edit.size' as a new stop condition when grab row locks
in the multi operation as multi does not guarantee cross row atomic. Then AsyncFSWAL could
also work with batch operation larger than 16MB unless the size of one row is larger than
16MB or MultiRowMutationEndpoint is used.

What do you think? [~stack] [~anoopsamjohn]. Too many configs maybe... Thanks.

> Make AsyncFSWAL as our default WAL
> ----------------------------------
>
>                 Key: HBASE-15536
>                 URL: https://issues.apache.org/jira/browse/HBASE-15536
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, HBASE-15536-v3.patch,
HBASE-15536-v4.patch, HBASE-15536-v5.patch, HBASE-15536.patch
>
>
> As it should be predicated on passing basic cluster ITBLL



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message