hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AnilKumar B <akumarb2...@gmail.com>
Subject Re: How and where exactly LSM trees are used in HBase?
Date Mon, 09 Dec 2013 15:57:31 GMT
Thanks Ted & Mikael.

Thanks & Regards,
B Anil Kumar.


On Mon, Dec 9, 2013 at 6:38 PM, Mikael Sitruk <mikael.sitruk@gmail.com>wrote:

> LSM tree are the basis for reducing random I/O which is a huge performance
> factor with big data system. A good overview can be found in HBase in
> action book, from Lars George.
> The basic idea is that you have an in memory structure for the latest
> changes and a structure stored on files, The files content is always
> ordered by key, and each row the file is jus the row_key, Column family
> identifier, column name, timestamp and the value (+ a marker).
> When the memory is full, the memory structure is flushed to disk, when
> there are a certain amount of files on filesystem the files are merged to
> bigger ones, since the files are ordered the merge is very fast, (like
> merge in mergesort algo)
>
>
> On Sun, Dec 8, 2013 at 8:42 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Searching for 'lsm tree hbase' would give you several articles.
> >
> > I am in China - the search results are mostly in Chinese.
> >
> > You should be able to read this:
> >
> >
> http://stackoverflow.com/questions/13762992/log-structured-merge-tree-in-hbase
> >
> > Cheers
> >
> >
> > On Wed, Dec 4, 2013 at 6:49 PM, AnilKumar B <akumarb2010@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > We are trying to understand how and where exactly LSM tress are used in
> > > HBase. Currently as per our understanding, while flushing memstore to
> > Store
> > > files and while HFile compaction it is used. And sits on top of HFiles
> at
> > > memstore level.
> > >
> > > Is this understanding correct. Can you please give more insight on
> this?
> > > How exactly is the merging done?
> > >
> > > Thanks & Regards,
> > > B Anil Kumar.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message