Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAKKbPbL8SL8iLE2ueD+kKwStCOV2XEh3f=d8t2OnmMFP8L5_pw@mail.gmail.com>
References: 
 <CAAAYAnNPaD3eHHYBVAtZ9cc3GNqJ4iCMW1qc458Zm7c1r58obg@mail.gmail.com>
	<CAGHyZ6K81JWut-C1c0e-kF9CM7W9p3vi0cT4O=FAyv+L1qYVfw@mail.gmail.com>
	<CAAAYAnOv2prLc2hqNGBqEoog0ebDbYK_0kWYkK58kmcifh_UnQ@mail.gmail.com>
	<CAKKbPbL8SL8iLE2ueD+kKwStCOV2XEh3f=d8t2OnmMFP8L5_pw@mail.gmail.com>
Date: Thu, 28 Apr 2016 20:36:16 -0700
Message-ID: 
 <CALte62xFV34YKpUGuFNLvguNnMPHbwDBmccpDRx6hktcwf6ZDw@mail.gmail.com>
Subject: Re: [DISCUSS] Make AsyncFSWAL the default WAL in 2.0
From: Ted Yu <yuzhihong@gmail.com>
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=94eb2c06a1086d0ea90531975835

--94eb2c06a1086d0ea90531975835
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

+1 to what Heng said.

On Thu, Apr 28, 2016 at 8:34 PM, Heng Chen <heng.chen.1986@gmail.com> wrote=
:

> The performance is quite great,  but i think maybe we should collect some
> experience on real production cluster before we make it as default.
>
> 2016-04-29 11:30 GMT+08:00 =E5=BC=A0=E9=93=8E <palomino219@gmail.com>:
>
> > Inline comments.
> > Thanks,
> >
> > 2016-04-29 10:57 GMT+08:00 Sean Busbey <busbey@cloudera.com>:
> >
> > > I am nervous about having default out-of-the-box new HBase users
> reliant
> > on
> > > a bespoke HDFS client, especially given Hadoop's compatibility
> > > promises and history. Answers for these questions would make me more
> > > confident:
> > >
> > > 1) Where are we on getting the client-side changes to HDFS pushed bac=
k
> > > upstream?
> > >
> > No progress yet... Here I want to tell a good story that HBase is alrea=
dy
> > use it as default :)
> >
> > >
> > > 2) How well do we detect when our FS is not HDFS and what does
> > > fallback look like?
> > >
> > Just wrap FSDataOutputStream to make it act like an asynchronous
> > output(call hflush in a separated thread). The performance is not good =
I
> > think.
> >
> > >
> > > 3) Will this mean altering the versions of Hadoop we label as
> > > supported for HBase 2.y+?
> > >
> > I have tested with hadoop versions from 2.4.x to 2.7.x, so I don't thin=
k
> we
> > need to change the supported versions?
> >
> > >
> > > 4) How are we going to ensure our client remains compatible with newe=
r
> > > Hadoop releases?
> > >
> > We can not ensure, HDFS always breaks HBase at a new release...
> > I need to test AsyncFSWAL on every new 2.x release and make it compatib=
le
> > with that version. And back to #1, I think we should make sure that the
> > AsyncFSOutput will be in HDFS-3.0. And in HBase-3.0, we can introduce a
> new
> > 'AsyncFSWAL' that use the AsyncFSOutput in HDFS.
> >
> > >
> > > On Thu, Apr 28, 2016 at 9:42 PM, Duo Zhang <zhangduo@apache.org>
> wrote:
> > > > Six month after I filed HBASE-14790...
> > > >
> > > > Now the AsyncFSWAL is ready. The WALPE result shows that it is
> > > *1.4x~3.7x*
> > > > faster than FSHLog. The ITBLL result turns out that it is *not bad*
> > than
> > > > FSHLog(the master branch is not that stable itself...).
> > > >
> > > > More details can be found on HBASE-15536.
> > > >
> > > > So here we propose to change the default WAL from FSHLog to
> AsyncFSWAL.
> > > > Suggestions are welcomed.
> > > >
> > > > Thanks.
> > >
> > >
> > >
> > > --
> > > busbey
> > >
> >
>

--94eb2c06a1086d0ea90531975835--