hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: Beware of PREFIX_TREE block encoding
Date Sun, 20 Oct 2013 03:50:44 GMT
StoreScanner gives 3x degradation on sequential scans with PREFIX_TREE
enabled. Block size 64K. StoreScanner has a lot of overhead including but
not limited to KeyValueHeap and ScanQueryMatcher.


On Sat, Oct 19, 2013 at 8:33 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Interesting.
>
> From this graph (HBASE-4676), PREFIX_TREE block encoding is not sensitive
> to variation in block size:
>
> https://issues.apache.org/jira/secure/attachment/12500778/SeeksPerSec%20by%20blockSize.png
>
> What was your block size ?
> If you don't bypass StoreScanner/KeyValueHeap, would that make a difference
> ?
>
> Thanks
>
>
> On Sat, Oct 19, 2013 at 7:34 PM, Vladimir Rodionov
> <vrodionov@carrieriq.com>wrote:
>
> > What I wanted to say by this? HBase still does not have block encoding
> > which is optimal for both scan and seek (re-seek).
> > I do not think these goals are mutually exclusive.
> >
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: vrodionov@carrieriq.com
> >
> > ________________________________________
> > From: Vladimir Rodionov [vladrodionov@gmail.com]
> > Sent: Saturday, October 19, 2013 7:32 PM
> > To: dev@hbase.apache.org
> > Subject: Beware of PREFIX_TREE block encoding
> >
> > The scan performance is bad. 10 x slower on my tests than for blocks with
> > NONE encoding. I scan data directly from block cache through
> > StoreFileScanner (bypassing all StoreScanner/KeyValueHeap stuff). It
> should
> > be clearly stated  that this encoding degrades overall performance
> > significantly in favor of data size reduction and is suitable only for
> Gets
> > - not for Scans.
> >
> > Best regards,
> > -Vladimir Rodionov
> >
> > -
> >
> > Confidentiality Notice:  The information contained in this message,
> > including any attachments hereto, may be confidential and is intended to
> be
> > read only by the individual or entity to whom this message is addressed.
> If
> > the reader of this message is not the intended recipient or an agent or
> > designee of the intended recipient, please note that any review, use,
> > disclosure or distribution of this message or its attachments, in any
> form,
> > is strictly prohibited.  If you have received this message in error,
> please
> > immediately notify the sender and/or Notifications@carrieriq.com and
> > delete or destroy any copy of this message and its attachments.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message