hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-16454) Compactor's shipping logic decision should be based on the current store's block size
Date Mon, 22 Aug 2016 11:54:21 GMT

     [ https://issues.apache.org/jira/browse/HBASE-16454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ramkrishna.s.vasudevan updated HBASE-16454:
-------------------------------------------
    Attachment: HBASE-16454.patch

Patch that changes the way the compactor does the shipping. It is now dependent on the number
of files involved in compaction and the actual block size configured.

> Compactor's shipping logic decision should be based on the current store's block size
> -------------------------------------------------------------------------------------
>
>                 Key: HBASE-16454
>                 URL: https://issues.apache.org/jira/browse/HBASE-16454
>             Project: HBase
>          Issue Type: Bug
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Minor
>             Fix For: 2.0.0
>
>         Attachments: HBASE-16454.patch
>
>
> Currently we do call KeyValueScanner.shipped() if the size of the KVs compacted reaches
certain threshold
> {code}
>      if (kvs != null && bytesWrittenProgressForShippedCall > shippedCallSizeLimit)
{
>             kvs.shipped();
>             bytesWrittenProgressForShippedCall = 0;
> }
> {code}
> Where the shippedCallSizeLimit is
> {code}
>     long shippedCallSizeLimit = (long) minFilesToCompact * HConstants.DEFAULT_BLOCKSIZE;
> {code}
> Instead it is better if it is based on the actual configured block size for that store.
It can still have the multiplying factore of 'minFilesToCompact ' but having default store
size would mean that if the store has smaller block sizes configured we hold on to more blocks
and those will not get evicted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message