hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15506) FSDataOutputStream.write() allocates new byte buffer on each operation
Date Tue, 22 Mar 2016 23:33:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207524#comment-15207524

Vladimir Rodionov commented on HBASE-15506:

HDFS-7276 provides buffer reuse for DFSOutputStream.
The following configuration properties are added. 

- dfs.client.write.byte-array-manager.enabled: 
for enabling/disabling byte array manager. Default is false. 

- dfs.client.write.byte-array-manager.count-threshold: 
The count threshold for each array length so that a manager is created only after the allocation
count exceeds the threshold. In other words, the particular array length is not managed until
the allocation count exceeds the threshold. Default is 128. 

- dfs.client.write.byte-array-manager.count-limit: 
The maximum number of arrays allowed for each array length. Default is 2048. 

- dfs.client.write.byte-array-manager.count-reset-time-period-ms: 
The time period in milliseconds that the allocation count for each array length is reset to
zero if there is no increment. Default is 10,000ms, i.e. 10 seconds.

Setting *dfs.client.write.byte-array-manager.enabled* to *true* enables buffer management
and resolves the issue.


> FSDataOutputStream.write() allocates new byte buffer on each operation
> ----------------------------------------------------------------------
>                 Key: HBASE-15506
>                 URL: https://issues.apache.org/jira/browse/HBASE-15506
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
> Deep inside stack trace in DFSOutputStream.createPacket.
> This should be opened in HDFS. This JIRA is to track HDFS work.

This message was sent by Atlassian JIRA

View raw message