hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shushant Arora <shushantaror...@gmail.com>
Subject Re: hbase architecture doubts
Date Mon, 02 May 2016 08:25:55 GMT
Thanks!

Few doubts;

1.LSM tree comprises two tree-like
<https://en.wikipedia.org/wiki/Tree_(data_structure)> structures, called C0 and
C1 and If the insertion causes the C0 component to exceed a certain size
threshold, a contiguous segment of entries is removed from C0 and merged
into C1 on disk

But in Hbase when C0 which is memstore I guess? is exceeded the threshold
size its dumped on to HDFS as HFIle(c1 I guess?) - and does compaction is
the process which here means as merging of C0 and C1 ?

2.Moves current, active Map aside as a snapshot (while a write lock is held
for a short period of time), and then creates a new CSLS instances.

In background, the snapshot is then dumped to disk. We get an Iterator on
CSLS. We write a block at a time. When we exceed configured block size, we
start a new one.

-- Does write lock is held till the time complete CSLS is dumpled on
disk.And read is allowed using snapshot.

Thanks!



On Mon, May 2, 2016 at 11:39 AM, Stack <stack@duboce.net> wrote:

> On Sun, May 1, 2016 at 3:36 AM, Shushant Arora <shushantarora09@gmail.com>
> wrote:
>
> > 1.Does Hbase uses ConcurrentskipListMap(CSLM) to store data in memstore?
> >
> > Yes (We use a CSLS but this is implemented over a CSLM).
>
>
> > 2.When mwmstore is flushed to HDFS- does it dump the memstore
> > Concurrentskiplist as Hfile2? Then How does it calculates blocks out of
> > CSLM and dmp them in HDFS.
> >
> >
> Moves current, active Map aside as a snapshot (while a write lock is held
> for a short period of time), and then creates a new CSLS instances.
>
> In background, the snapshot is then dumped to disk. We get an Iterator on
> CSLS. We write a block at a time. When we exceed configured block size, we
> start a new one.
>
>
> > 3.After dumping the inmemory CSLM of memstore to HFILe does memstore
> > content is discarded
>
>
> Yes
>
>
>
> > and if while dumping memstore any read request comes
> > will it be responded by copy of memstore or discard of memstore will be
> > blocked until read request is completed?
> >
> > We will respond using the snapshot until it has been successfully dumped.
> Once dumped, we'll respond using the hfile.
>
> No blocking (other than for the short period during which the snapshot is
> made and the file is swapped into the read path).
>
>
>
> > 4.When a read request comes does it look in inmemory CSLM and then in
> > HFile?
>
>
> Generally, yes.
>
>
>
> > And what is LogStructuredMerge tree and its usage in Hbase.
> >
> >
> Suggest you read up on LSM Trees (
> https://en.wikipedia.org/wiki/Log-structured_merge-tree) and if you still
> can't see the LSM tree in the HBase forest, ask specific questions and
> we'll help you out.
>
> St.Ack
>
>
>
>
> > Thanks!
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message