hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliott Clark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
Date Fri, 17 Jul 2015 17:44:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14631655#comment-14631655
] 

Elliott Clark commented on HBASE-14098:
---------------------------------------

I tested this out yesterday on a cluster with 1.1.
When compacting 100+ GB storfiles I can reliably make the linux kernel pause trying to thrash
and find pages to clear out.
With just dropping the write's out of cache things are better and there are fewer stalls.
With dropping the caches behind writes and reads from a compaction there are no noticeable
stalls ( ones that atop sees that take > 1 second ).

So I think that we should provide the ability to drop caches behind reads and writes on large
compactions ( those that would run on the large compaction threads ). However because this
could adversely affect people with less data per machine I think that we should have this
turned off by default.

Thoughts?

> Allow dropping caches behind compactions
> ----------------------------------------
>
>                 Key: HBASE-14098
>                 URL: https://issues.apache.org/jira/browse/HBASE-14098
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction, hadoop2, HFile
>    Affects Versions: 2.0.0, 1.3.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>             Fix For: 2.0.0, 1.3.0
>
>         Attachments: HBASE-14098-v1.patch, HBASE-14098.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message