hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Patel <amit.pa...@cloudera.com>
Subject Re: Plan for Hybrid Logical Clocks
Date Thu, 06 Jul 2017 21:10:16 GMT
Hi everyone,

As an update, the core HLC work has been pushed out as an initial commit to
a public branch called HBASE-14070.HLC
<https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC>.
Details regarding the core HLC commit can be viewed at HBASE-18305
<https://issues.apache.org/jira/browse/HBASE-18305>. The next step will be
another commit on that branch that would undo using the master's timestamp
for meta updates. The progress for this task will be tracked in HBASE-18328
<https://issues.apache.org/jira/browse/HBASE-18328>.

On Fri, Jun 23, 2017 at 10:38 AM, Enis Söztutar <enis.soz@gmail.com> wrote:

> The plan looks good.
>
> >However, if it is done before HBase 2.0.0 beta and if it is proven to be
> solid, then we can discuss bringing it back to 2.0.0.
> It will be pretty good to have at least the base in 2.0, so that we can
> later enable HLC for user tables in later releases. Of course, it depends
> sufficient testing.
>
> Enis
>
> On Fri, Jun 23, 2017 at 10:03 AM, Amit Patel <amit.patel@cloudera.com>
> wrote:
>
> > For now, yes, that is the plan. However, if it is done before HBase 2.0.0
> > beta and if it is proven to be solid, then we can discuss bringing it
> back
> > to 2.0.0.
> >
> > On Fri, Jun 23, 2017 at 6:12 AM, Sean Busbey <busbey@apache.org> wrote:
> >
> > > It'd be great to see this land. You only mention the master branch as
> > > a merge target; is the assumption that this would only be a HBase 3.0+
> > > addition?
> > >
> > > On Thu, Jun 22, 2017 at 7:34 PM, Amit Patel <amit.patel@cloudera.com>
> > > wrote:
> > > > Hi everyone,
> > > >
> > > >
> > > >
> > > > I'm Amit and I've been picking up on the past work on Hybrid Logical
> > > Clocks
> > > > (HBASE-14070 <https://issues.apache.org/jira/browse/HBASE-14070>)
> that
> > > was
> > > > done by Sai Teja Ranuva last summer. The most recent status of HLC on
> > > HBase
> > > > can be found here
> > > > <https://docs.google.com/document/d/1n32DUKoL3LSoKQ1_
> > > NuF8TbQkZiZCwFhbHZ7JXRjg3oA/edit#>
> > > > and a prior document by Enis Soztutar with discussion can be found
> here
> > > > <https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_
> > > bXy8P9h6kWC05Bhw/edit#>.
> > > > I think the effort is at a point where it makes sense to create a
> > public
> > > > branch.
> > > >
> > > >
> > > >
> > > > Currently the plan is to create a public branch and incrementally add
> > on
> > > > functionality and tests. Initially, core HLC would be introduced as a
> > > > commit. From there, HLC would then be enabled for just the meta table
> > > > (updating the clock on events like region open/close, recovery,
> > > > replication, etc). Once we are fully confident that HLC at least
> works
> > on
> > > > the meta table then I anticipate it would be appropriate to merge
> with
> > > the
> > > > master branch.
> > > >
> > > >
> > > >
> > > > Follow up commits would extend the effort by:
> > > >
> > > >
> > > >
> > > >    -
> > > >
> > > >    Addition of a protobuf message called NodeTime that holds
> timestamp
> > > >    corresponding to a send event between nodes.
> > > >    -
> > > >
> > > >    Addition of NodeTime as a field to messages sent between nodes for
> > > >    messages like requests and responses for region open/close.
> > > >    -
> > > >
> > > >    Addition of integration tests that verify HLC works correctly with
> > > >    recovery, replication, region open, and region close.
> > > >    -
> > > >
> > > >    Later enabling HLC on user tables (main barrier are tests that
> rely
> > on
> > > >    client-side setting of timestamps for table mutations which would
> no
> > > longer
> > > >    be allowed on an HLC table).
> > > >
> > > >
> > > >
> > > > Work that I have implemented under the outstanding work
> > > > <https://docs.google.com/document/d/1n32DUKoL3LSoKQ1_
> > > NuF8TbQkZiZCwFhbHZ7JXRjg3oA/edit#heading=h.8x5d8iakmo8b>
> > > > as a part of getting HLC working for user tables includes:
> > > >
> > > >
> > > >
> > > >    -
> > > >
> > > >    Mapping of time ranges from client GETs
> > > >    -
> > > >
> > > >    Disallowing clients to set timestamps for HLC and System Monotonic
> > > tables
> > > >
> > > >
> > > >
> > > > Feel free to chime in with any comments, suggestions, or other input.
> > > >
> > > > --
> > > > Amit
> > >
> >
> >
> >
> > --
> > Amit
> >
>



-- 
Amit

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message