Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAAAYAnNQ3ZWvLowFE=k8+BZWdCpoBUrTFzs+Cqg6VuiH4HvOpQ@mail.gmail.com>
References: 
 <CAAha9a2sRngY-i9vQUzE_4sQ3cNRK1e4Erxcj=LM3LmmEkhhxg@mail.gmail.com>
 <CALte62yLbrdtLH23+C4mQ7QW7Ot5HpCzoaFaEVp6wEit0zek+g@mail.gmail.com>
 <CAAAYAnNQ3ZWvLowFE=k8+BZWdCpoBUrTFzs+Cqg6VuiH4HvOpQ@mail.gmail.com>
From: Jonathan Hsieh <jon@cloudera.com>
Date: Thu, 21 May 2015 11:20:34 -0700
Message-ID: 
 <CAAha9a0rPkV47K_fk-DYCY6zbRYhdk2N_9jXrPOG5GA_dEUyQA@mail.gmail.com>
Subject: Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=089e013d1e0eb5b6e305169b9ae1

--089e013d1e0eb5b6e305169b9ae1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Not that I know of.

Here's a quick high level comparison based on my read of the ozone doc are
api differences:

- ozone is described an s3 clone while hbase mob is similar to a rdbms's
blob feature.
- ozone names - 3-64 bytes, hbase mob names - hbase qual limit -(64k i
think)
- ozone object sizes - 100k's to 100M's;  hbase mob - 10k's-10'sM (tested
at 50M)
- ozone - hash partitioning, hbase mob - range partitioning

It isn't clear to me how they handle updates and deletes, hdfs snapshots
and other hdfs backup mechanisms.

Jon.

On Wed, May 20, 2015 at 8:10 PM, =E5=BC=A0=E9=93=8E <palomino219@gmail.com>=
 wrote:

> Is there any comparison between HBASE-11339 and HDFS-7240?
> Is their 'Object' a super set of our 'Medium Object'?
>
> 2015-05-21 10:38 GMT+08:00 Ted Yu <yuzhihong@gmail.com>:
>
> > This is a useful feature, Jon.
> >
> > I went over the mega-patch and left some comments on review board.
> >
> > I noticed that hbck was not included in the patch. Neither did I find a
> > sub-task of HBASE-11339 that covers hbck.
> >
> > Do you or Jingcheng plan to add MOB-aware capability for hbck ?
> >
> > Cheers
> >
> > On Wed, May 20, 2015 at 9:21 AM, Jonathan Hsieh <jon@cloudera.com>
> wrote:
> >
> > > Hi folks,
> > >
> > > The Medium Object (MOB) Storage feature (HBASE-11339[1]) is modified
> I/O
> > > and compaction path that allows individual moderately sized values
> > > (10k-10MB) to be stored so that write amplification is reduced when
> > > compared to the normal I/O path.   At a high level, it provides
> alternate
> > > flush and compaction mechanisms that segregates large cells into a
> > separate
> > > area where they are not subject to potentially frequent compaction an=
d
> > > splits that can be encountered in the normal I/O path. A more detaile=
d
> > > design doc can be found on the hbase-11339 jira.
> > >
> > > Jingcheng Du has been working on the mob feature for a while and Anoo=
p,
> > Ram
> > > and I have been shepherding him through the design revisions and
> > > implementation of the feature in the hbase-11339 branch.[2]
> > >
> > > The branch we are proposing to merge into master is compatible with
> > HBase's
> > > core functionality including snapshots, replication, shell support,
> > behaves
> > > well with table alters, bulk loads and does not require external MR
> > > processes. It has been documented, and subject to many integration te=
st
> > > runs  (ITBLL, ITAcidGuarantees, ITIngest) including fault injection.
> > > Performance testing of the feature shows what can be a 2x-3x throughp=
ut
> > > improvement for workloads that contain mobs. These results can be see=
n
> on
> > > the hbase 2.0 panel discussion slides from hbasecon (once published).
> > >
> > > Recently there have been some hfile encryption related shortcomings
> that
> > we
> > > could address in branch or in master.
> > >
> > > Earlier iterations of the feature has been tested in production by
> users
> > > that Jingcheng has been responsible for.  A version has also been
> > deployed
> > > at users I have been responsible for.  Some of the folks from Huawei
> > > (ashutosh) have also been submitting the recent encryption bug report=
s
> > > against the hbase-11339 branch so there is some evidence of usage by
> > them.
> > >
> > > The four of us  (Jingcheng, Ram, Anoop and I) are satisfied with the
> > > feature and feel it is a good time to call a merge vote.  Ive posted =
a
> > > megapatch version for folks who want to peruse the code. [3]
> > >
> > > What do you all think?
> > >
> > > Thanks,
> > > Jingcheng, Jon, Ram, and Anoop.
> > >
> > > [1] https://issues.apache.org/jira/browse/HBASE-11339
> > > [2] https://github.com/apache/hbase/tree/hbase-11339
> > > [3] https://reviews.apache.org/r/34475/
> > > --
> > > // Jonathan Hsieh (shay)
> > > // HBase Tech Lead, Software Engineer, Cloudera
> > > // jon@cloudera.com // @jmhsieh
> > >
> >
>


--=20
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

--089e013d1e0eb5b6e305169b9ae1--