Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CAMUu0w_bWLBSbo6BjrZGQ8KKunYLEj4b_qqVGtODXiafYGW9Eg@mail.gmail.com>
References: <CABOhbr7ZLiPcWW7b8yVyB15R7iF11-s2hnt1iK-xfhPtGUJkeA@mail.gmail.com>
 <CADcMMgFNupeLGnC=D=Xqk_KHdHq57tb8hwoUfuZ8rDKDG2K6kA@mail.gmail.com>
 <CADcMMgE_oG7Q3NHM4NQbBL=TrXwCQ69uz1r4BusWi6r7VdZTkQ@mail.gmail.com>
 <CAMUu0w-9n-djpedrMYH9nD+fvt+cfhEbvz3hJEw1bdd8XfVoyw@mail.gmail.com>
 <CABOhbr6vWfPUo5XXoFrfUs4hPt9bFTs_7GEHBm9mJyrmWDpSCA@mail.gmail.com>
 <CAMUu0w9-m3k=cdSJwSpyQ5ztGz5PfUGaea2Mi_T0L0Bd76CrXA@mail.gmail.com>
 <CAAjhxrpEwWNFhsJzpXSPrYU_ALQRkfyPAv4dwr_irOqw9ci5kw@mail.gmail.com> <CAMUu0w_bWLBSbo6BjrZGQ8KKunYLEj4b_qqVGtODXiafYGW9Eg@mail.gmail.com>
From: Apekshit Sharma <appy@cloudera.com>
Date: Fri, 25 Aug 2017 14:46:41 -0700
Message-ID: <CAAjhxrrcBWcc-u=UaU4+0k7XjnoQn0bFABCRm2BRMTjRJN5GDA@mail.gmail.com>
Subject: Re: [DISCUSS] Merging Hybrid Logical Clocks with Master
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary="001a11435efc36121805579ae3cf"
archived-at: Fri, 25 Aug 2017 21:47:27 -0000

--001a11435efc36121805579ae3cf
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

bq.  ...but if we are recommending HLC for user tables, we have to also do
end-to-end testing to demonstrate that perf is not affected by HLC.
Agree. I think the only way of doing it right now is running ycsb
with/without HLC.
If it highlights an issue, it means there is one (there can't be false
-ves). But if it doesn't, it doesn't mean there isn't one (there can be
false +ve).
But i think it's a reasonable way of testing.
Any other ideas?


On Fri, Aug 25, 2017 at 2:36 PM, Enis S=C3=B6ztutar <enis.soz@gmail.com> wr=
ote:

> HLC will never be able to match the throughput for the system clock of
> course. I was expecting the perf to be in the 2-3x slower range, not more
> than an order of magnitude difference. The current perf is definitely
> acceptable for meta updates, but if we are recommending HLC for user
> tables, we have to also do end-to-end testing to demonstrate that perf is
> not affected by HLC.
>
> Enis
>
> On Fri, Aug 25, 2017 at 2:00 PM, Apekshit Sharma <appy@cloudera.com>
> wrote:
>
> >  It's true that hlc clock is very slower than system, but is it really =
a
> > bottleneck? I mean 7mil ops/sec with 16 threads doesn't sound bottlenec=
k
> > because:
> > -  It's like 70 times of any RS' throughput (assuming 100k write
> > throughput)
> > - It's certainly enough for meta operations.
> > What say?
> >
> > But i agree that we should try to make it better and file a jira for
> > followup.
> >
> >
> > On Fri, Aug 25, 2017 at 11:53 AM, Enis S=C3=B6ztutar <enis.soz@gmail.co=
m>
> > wrote:
> >
> > > Indeed it seems like a perf improvement in the code HLC clock is need=
ed
> > > before the merge. Do you mind creating a sub ticket for this.
> > >
> > > Enis
> > >
> > > On Fri, Aug 25, 2017 at 10:54 AM, Amit Patel <amit.patel@cloudera.com=
>
> > > wrote:
> > >
> > > > Just as an update with regard to performance: getting the time from
> or
> > > > updating the hybrid logical clock is quite expensive (see clocks
> > > > microbenchmark
> > > > <https://issues.apache.org/jira/browse/HBASE-18542?
> > > > focusedCommentId=3D16141932&page=3Dcom.atlassian.jira.
> > > > plugin.system.issuetabpanels:comment-tabpanel#comment-16141932>)
> > > > to the extent that it would be a blocker for this work. There is
> > however
> > > > certainly room for improvement given that the current implementatio=
n
> > uses
> > > > synchronized blocks for both Clock#now and Clock#update.
> > > >
> > > > On Wed, Aug 23, 2017 at 6:18 PM, Enis S=C3=B6ztutar <enis@apache.or=
g>
> > wrote:
> > > >
> > > > > Thanks Amit for continuing on the work.
> > > > >
> > > > > I am biased, but of course would like to see this merged back to
> > master
> > > > > (and possibly make it to 2.0) once the issues are ironed out. We
> can
> > > > > eliminate a very large class of problems with a mathematically
> sound
> > > > > strategy.
> > > > >
> > > > > Enis
> > > > >
> > > > > On Tue, Aug 22, 2017 at 1:28 PM, Stack <stack@duboce.net> wrote:
> > > > >
> > > > > > Look at the pretty HLC timestamps in meta after a run:
> > > > > >
> > > > > >  hbase:meta
> > > > > >           column=3Dtable:state, timestamp=3D1576463246879096833=
,
> > > > > value=3D\x08\x00
> > > > > >  hbase:namespace
> > > > > >            column=3Dtable:state, timestamp=3D157646036991056281=
7,
> > > > > > value=3D\x08\x00
> > > > > >  hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec
> b5.
> > > > > >           column=3Dinfo:regioninfo, timestamp=3D157646259330482=
1761,
> > > > > > value=3D{ENCODED =3D> 6d365281d5ca8ee0bc0fa12cdf33ecb5, NAME =
=3D>
> > > > > > 'hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec
> > b5.',
> > > > > > STARTKEY =3D> '', ENDKEY =3D> ''}
> > > > > >  hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec
> b5.
> > > > > >           column=3Dinfo:seqnumDuringOpen,
> > > timestamp=3D1576462593304821761,
> > > > > > value=3D\x00\x00\x00\x00\x00\x00\x00\x15
> > > > > >  hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec
> b5.
> > > > > >           column=3Dinfo:server, timestamp=3D1576462593304821761=
,
> value=3D
> > > > > > ve0536.halxg.cloudera.com:16020
> > > > > >  hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec
> b5.
> > > > > >           column=3Dinfo:serverstartcode,
> > timestamp=3D1576462593304821761,
> > > > > > value=3D1503431013602
> > > > > >  hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec
> b5.
> > > > > >           column=3Dinfo:sn, timestamp=3D1576462593050017795, va=
lue=3D
> > > > > > ve0536.halxg.cloudera.com,16020,1503431013602
> > > > > >  hbase:namespace,,1503429763610.6d365281d5ca8ee0bc0fa12cdf33ec
> b5.
> > > > > >           column=3Dinfo:state, timestamp=3D1576462593304821761,
> > > value=3DOPEN
> > > > > >
> > > > > > St.Ack
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Aug 22, 2017 at 1:26 PM, Stack <stack@duboce.net> wrote=
:
> > > > > >
> > > > > > > Nice work Amit. Idea of merging sounds good to me. Our metada=
ta
> > > > system
> > > > > > > will run w/ a new rigor as HLC allows us leave behind a whole
> > class
> > > > of
> > > > > > > issues.
> > > > > > >
> > > > > > > I ran the HLC branch on small cluster for a while w/ chaos. T=
he
> > > ITBLL
> > > > > job
> > > > > > > passed so basically works I'd say. +1 from me on merge.
> > > > > > >
> > > > > > > St.Ack
> > > > > > >
> > > > > > > On Mon, Aug 21, 2017 at 1:29 PM, Amit Patel <
> > > amit.patel@cloudera.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi everyone,
> > > > > > >>
> > > > > > >> I=E2=80=99d like to begin the discussion of merging the curr=
ent HLC
> work
> > > > with
> > > > > > >> master. Note that the current work is not ready to merge yet
> > since
> > > > > there
> > > > > > >> are a few remaining issues (see below). For reference,
> > HBASE-14070
> > > > > > >> <https://issues.apache.org/jira/browse/HBASE-14070> proposes
> > > using
> > > > > > hybrid
> > > > > > >> logical clocks, which combine both logical and physical cloc=
ks
> > to
> > > > > > capture
> > > > > > >> not only causality relationship between events, but also
> > physical
> > > > time
> > > > > > at
> > > > > > >> which events occur. HLC can be enabled on a per-table basis
> and
> > at
> > > > the
> > > > > > >> moment just the meta table has HLC enabled by default (the
> other
> > > > > tables
> > > > > > >> default to use the system clock). An initial design document
> > > created
> > > > > by
> > > > > > >> Enis Soztutar can be found here
> > > > > > >> <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LD
> > > > > > >> T4e_bXy8P9h6kWC05Bhw/edit#>.
> > > > > > >> The more recent status document created by Sai Teja Ranuva c=
an
> > be
> > > > > found
> > > > > > >> here
> > > > > > >> <https://docs.google.com/document/d/1n32DUKoL3LSoKQ1_NuF8TbQ
> > > > > > >> kZiZCwFhbHZ7JXRjg3oA/edit#heading=3Dh.881hwl2sdbvx>.
> > > > > > >>
> > > > > > >>
> > > > > > >> Changes
> > > > > > >>
> > > > > > >> The new classes that would be introduced include: a Clock
> > > interface
> > > > > with
> > > > > > >> three clock implementations mentioned above and enums for th=
e
> > > clock
> > > > > and
> > > > > > >> timestamp type.  The three clock implementations are hybrid
> > > logical,
> > > > > > >> system
> > > > > > >> monotonic, and system clocks and each generate 64-bit
> > timestamps.
> > > > > > >> Timestamps from the system monotonic and system clock
> represent
> > > > > physical
> > > > > > >> time and are directly backwards compatible.
> > > > > > >>
> > > > > > >> Users can set the clock type of a table during table
> creation. A
> > > > > table=E2=80=99s
> > > > > > >> clock type cannot be changed after the table has been create=
d.
> > For
> > > > > > tables
> > > > > > >> whose clock type is HLC, users would not be permitted to set
> > their
> > > > own
> > > > > > >> timestamps for mutations. Certain events such as region
> > > assignment,
> > > > > > >> unassignment, and recovery update the clocks upon receiving
> > > > timestamps
> > > > > > >> externally.
> > > > > > >>
> > > > > > >> Benefits
> > > > > > >>
> > > > > > >>
> > > > > > >>    -
> > > > > > >>
> > > > > > >>    Hybrid logical clocks (and even system monotonic clocks)
> can
> > > > solve
> > > > > a
> > > > > > >>    variety of long standing issues related to time highlight=
ed
> > > here
> > > > > > >>    <https://docs.google.com/document/d/
> 1LL2GAodiYi0waBz5ODGL4L
> > > > > > >> DT4e_bXy8P9h6kWC05Bhw/edit#heading=3Dh.msabldfs8oly>.
> > > > > > >>
> > > > > > >>    -
> > > > > > >>
> > > > > > >>    Future work with hybrid logical clocks would include
> enabling
> > > > them
> > > > > > with
> > > > > > >>    user tables. Ideally we would want HLC to be used
> everywhere.
> > > > > Further
> > > > > > >> work
> > > > > > >>    such as global point-in-time snapshots can leverage HLC
> > > > > > >>
> > > > > > >>
> > > > > > >> Remaining issues
> > > > > > >>
> > > > > > >> There are still a few remaining issues to close out before
> > > actually
> > > > > > >> merging, but I wanted to at least start the discussion.
> > Currently
> > > I
> > > > am
> > > > > > >> working on fixing remaining tests that are either failing or
> > > timing
> > > > > out
> > > > > > in
> > > > > > >> the public branch <https://builds.apache.org/
> > job/HBASE-14070.HLC/
> > > >
> > > > as
> > > > > > >> well
> > > > > > >> as doing local performance tests, and cluster tests. Some of
> the
> > > > > > remaining
> > > > > > >> issues we are tracking are:
> > > > > > >>
> > > > > > >>
> > > > > > >>    -
> > > > > > >>
> > > > > > >>    HBASE-18542 (Perf numbers)
> > > > > > >>    <https://issues.apache.org/jira/browse/HBASE-18542>
> > > > > > >>    -
> > > > > > >>
> > > > > > >>    HBASE-18508 (Fixing unit tests)
> > > > > > >>    <https://issues.apache.org/jira/browse/HBASE-18508>
> > > > > > >>    -
> > > > > > >>
> > > > > > >>    HBASE-18432 (Clock skew)
> > > > > > >>    <https://issues.apache.org/jira/browse/HBASE-18432>
> > > > > > >>
> > > > > > >> Some brainstorming/discussions we also are tracking:
> > > > > > >>
> > > > > > >>    -
> > > > > > >>
> > > > > > >>    (HBASE-18643) Adding transaction id to Result
> > > > > > >>    <https://issues.apache.org/jira/browse/HBASE-18643>
> > > > > > >>    -
> > > > > > >>
> > > > > > >>    (HBASE-18642) Deprecate setting of timestamp in client fo=
r
> > HLC
> > > > > > >>    <https://issues.apache.org/jira/browse/HBASE-18642>
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > -- Appy
> >
>


--=20

-- Appy

--001a11435efc36121805579ae3cf--