hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Nguyen <anthony.an.ngu...@gmail.com>
Subject Re: Prefetching Indexes of HFiles
Date Fri, 25 Sep 2015 14:39:12 GMT
Thanks! If I understand it correctly, we need to at least open every HFile
once to load root indexes into the cache, right?

Another question - I went through Nick's BlockCache 101 and was intrigued
by BucketCache / CombinedBlockCache. If I used BucketCache in file mode
with SSDs, I see that I can persist data blocks across cluster lifecycles
and therefore have a warmed cluster. Would it be possible for me to modify
it such that I could also store the bloom filters / index blocks which are
currently only stored in the LruBlockCache into the BucketCache?

Thanks again.

On Thu, Sep 24, 2015 at 5:13 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> I think the option you mentioned is for data blocks.
>
> As for index blocks, please refer to HFile V2 design doc.
> Also see this:
> https://issues.apache.org/jira/browse/HBASE
>
> -3857?focusedCommentId=13031489&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13031489
>
> Cheers
>
> On Thu, Sep 24, 2015 at 1:16 PM, Anthony Nguyen <
> anthony.an.nguyen@gmail.com
> > wrote:
>
> > Hi all,
> >
> > Is there a better way to warm HFile indexes other than scanning through
> my
> > datasets? I did see PREFETCH_BLOCKS_ON_OPEN, but the warning that it "is
> > not a good idea if the data to be preloaded will not fit into the
> > blockcache" makes me wary. Why would this be a bad idea?
> >
> > Thanks!
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message