hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HBASE-5476) Merge flush/merging compaction; save on i/o by merging an existing file with content memstore
Date Sat, 11 Apr 2015 01:15:12 GMT

     [ https://issues.apache.org/jira/browse/HBASE-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell resolved HBASE-5476.
-----------------------------------
    Resolution: Incomplete

> Merge flush/merging compaction; save on i/o by merging an existing file with content
memstore
> ---------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5476
>                 URL: https://issues.apache.org/jira/browse/HBASE-5476
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: stack
>
> Compactions are slow.  To improve, we can work on  making compactions faster (TODO) or
make it so compactions have less work to do.  This issue is about the later; doing something
the bigtable paper talks off where the content of memory is merged with existing content in
the filesystem (I'm sure we've discussed this a bunch in the past but can't find an explicit
issue for it).
> We save on the reading of the size of a memstore from the filesystem if we merge flush/merge
compact.
> BT seems to include memstore content in minor compactions.  We should look at doing that
(would snapshot at start of compaction and then would integrate the snapshot into resultant
compacted file).  Alternatively we could flush into a small file that is already in the filesystem
(this could be tougher given that we have this flush-compaction separation at the moment...
how would we make it so the file we're merging into is not picked up for a normal compaction).
> Doing the merge compaction/flush merge would slow down flushes.  It could back up memory
into the global barrier such that we stop taking on writes altogether.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message