hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sayyid Amin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-3067) in_memory column family records should be available in memory past memstore flush if total memory to hbase > memory to memstore
Date Wed, 27 Jul 2011 08:42:09 GMT

     [ https://issues.apache.org/jira/browse/HBASE-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sayyid Amin updated HBASE-3067:
-------------------------------

    Summary: in_memory column family records should be available in memory past memstore flush
if total memory to hbase > memory to memstore  (was: in_memory column family records should
be available in memory past memstore flush if total memory to habse > memory to memstore)

> in_memory column family records should be available in memory past memstore flush if
total memory to hbase > memory to memstore
> -------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-3067
>                 URL: https://issues.apache.org/jira/browse/HBASE-3067
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.89.20100621
>            Reporter: Abhijit Pol
>
> On Fri, Oct 1, 2010 at 8:31 AM, Stack <stack@duboce.net> wrote:
> Blockcache is made of blocks pulled from HDFS.  It'd be a little awkward inserting hot
records into the block cache w/o going via HDFS.
> But, yes, you have a good point that flush is disruptive of hot records.
> In the past we talked of a keyvalue cache on top of the block cache but it fell out of
favor because block cache seemed to be good enough but sounds like we need to revive it or
do some fancy dancing if column family is marked in-memory, we keep around the snapshot of
memstore until we know the block cache has been populated?
> Any other suggestions?
> On Fri, Oct 1, 2010 at 1:26 AM, Abhijit Pol <apol@rocketfuel.com> wrote:
> > we are trying to read efficiently a hot column family (in_memory=true, blockcaching=true)
that get writes at say 500 qps and reads at 10,000 qps.
> > - as long as writes are in memstore we get them from memstore and its fast
> > - if we have read it once it will be at least in block cache (gets priority due
to in_memory=true) and subsequent reads are faster
> > - however memstore flush puts records on disk which demands for disk IO to get them
back in block cache
> >
> > is there a way for memstore flush to go to blockcache?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message