hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Helmling (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-4143) HTable.doPut(List) should check the writebuffer length every so often
Date Fri, 29 Jul 2011 00:12:09 GMT

     [ https://issues.apache.org/jira/browse/HBASE-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gary Helmling updated HBASE-4143:
---------------------------------

    Attachment: HBASE-4143_update.patch

Update to internal check so that we're only checking against the write buffer and so we continue
checking if the first check is false (previously "n" would not get reset and subsequent checks
would fail if the first write buffer check was negative).

> HTable.doPut(List) should check the writebuffer length every so often
> ---------------------------------------------------------------------
>
>                 Key: HBASE-4143
>                 URL: https://issues.apache.org/jira/browse/HBASE-4143
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Doug Meil
>            Assignee: Doug Meil
>            Priority: Minor
>         Attachments: HBASE-4143_update.patch, client_HBASE_4143.patch
>
>
> This came up on a dist-list conversation between Andy P., Ted Yu, and myself.  Andy noted
that extremely large lists passed into put(List) can cause issues.  Ted suggested that having
doPut check the write-buffer length every so often (5-10 records?) so the flush doesn't happen
only at the end, and I think that's good idea.
>  public void put(final List<Put> puts) throws IOException {
>     doPut(puts);
>   }
>   private void doPut(final List<Put> puts) throws IOException {
>     for (Put put : puts) {
>       validatePut(put);
>       writeBuffer.add(put);
>       currentWriteBufferSize += put.heapSize();
>     }
>     if (autoFlush || currentWriteBufferSize > writeBufferSize) {
>       flushCommits();
>     }
>   }
> Once this change is made, remove the comment in HBASE-4142 about large lists being a
performance problem.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message