Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of mikael.sitruk@gmail.com
 designates 209.85.160.49 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALte62yCA-SSo=-uQf+fRnYO--1r4LgLrYB7HVPLyz8zAgfiOA@mail.gmail.com>
References: 
 <CAHdEuM3ZVs7GYaR1Z2qqGJoO8RWeozA37op6a6TrrNCHxQ51aA@mail.gmail.com>
	<CALte62yCA-SSo=-uQf+fRnYO--1r4LgLrYB7HVPLyz8zAgfiOA@mail.gmail.com>
Date: Mon, 9 Dec 2013 15:08:03 +0200
Message-ID: 
 <CAPpvZ9B3HBmSDzMw21g8FNxOc1g22oNPazucKn0Yu5TL1usAtw@mail.gmail.com>
Subject: Re: How and where exactly LSM trees are used in HBase?
From: Mikael Sitruk <mikael.sitruk@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=e89a8ff1caf2a4e5ff04ed19af37

--e89a8ff1caf2a4e5ff04ed19af37
Content-Type: text/plain; charset=UTF-8

LSM tree are the basis for reducing random I/O which is a huge performance
factor with big data system. A good overview can be found in HBase in
action book, from Lars George.
The basic idea is that you have an in memory structure for the latest
changes and a structure stored on files, The files content is always
ordered by key, and each row the file is jus the row_key, Column family
identifier, column name, timestamp and the value (+ a marker).
When the memory is full, the memory structure is flushed to disk, when
there are a certain amount of files on filesystem the files are merged to
bigger ones, since the files are ordered the merge is very fast, (like
merge in mergesort algo)


On Sun, Dec 8, 2013 at 8:42 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> Searching for 'lsm tree hbase' would give you several articles.
>
> I am in China - the search results are mostly in Chinese.
>
> You should be able to read this:
>
> http://stackoverflow.com/questions/13762992/log-structured-merge-tree-in-hbase
>
> Cheers
>
>
> On Wed, Dec 4, 2013 at 6:49 PM, AnilKumar B <akumarb2010@gmail.com> wrote:
>
> > Hi,
> >
> > We are trying to understand how and where exactly LSM tress are used in
> > HBase. Currently as per our understanding, while flushing memstore to
> Store
> > files and while HFile compaction it is used. And sits on top of HFiles at
> > memstore level.
> >
> > Is this understanding correct. Can you please give more insight on this?
> > How exactly is the merging done?
> >
> > Thanks & Regards,
> > B Anil Kumar.
> >
>

--e89a8ff1caf2a4e5ff04ed19af37--