hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: [DISCUSSION] Removing the bypass semantic from the Coprocessor APIs
Date Tue, 10 Oct 2017 18:51:45 GMT
Agreed with the metrics. This is tied to the bypass consideration in my
opinion. Updating core metrics would be part of substituting core code via
bypass. Without bypass, it doesn't make sense for coprocessors to touch
core metrics, because they can only add functionality, so can export their
own metrics.

On Tue, Oct 10, 2017 at 11:49 AM, Stack <stack@duboce.net> wrote:

> On Tue, Oct 10, 2017 at 11:10 AM, Andrew Purtell <apurtell@apache.org>
> wrote:
> > The coprocessor API provides an environment method, bypass(), that when
> > called from a preXXX hook will cause the core code to skip all remaining
> > processing. This capability was introduced on HBASE-3348. Since this
> time I
> > think we are more enlightened about the complications of this feature.
> (Or,
> > anyway, speaking for myself:)
> >
> > Not all hooks provide the bypass semantic. Where this is the case the
> > javadoc for the hook says so, but it can be missed. If you call bypass()
> in
> > a hook where it is not supported it is a no-op. This can lead to a poor
> > developer experience.
> >
> > Where bypass is supported what is being bypassed is all of the core code
> > implementing the remainder of the operation. In order to understand what
> > calling bypass() will skip, a coprocessor implementer should read and
> > understand all of the remaining code and its nuances. Although I think
> this
> > is good practice for coprocessor developers in general, it demands a
> lot. I
> > think it would provide a much better developer experience if we didn't
> > allow bypass, even though it means - in theory - a coprocessor would be a
> > lot more limited in some ways than before. What is skipped is extremely
> > version dependent. That core code will vary, perhaps significantly, even
> > between point releases. We do not provide the promise of consistent
> > behavior even between point releases for the bypass semantic. To achieve
> > that we could not change any code between hook points. Therefore the
> > coprocessor implementer becomes an HBase core developer in practice as
> soon
> > as they rely on bypass(). Every release of HBase may break the assumption
> > that the replacement for the bypassed code takes care of all necessary
> > skipped concerns. Because those concerns can change at any point, such an
> > assumption is never safe.
> >
> > I say "in theory" because I would be surprised if anyone is relying on
> the
> > bypass for the above reason. I seem to recall that Phoenix might use it
> in
> > one place to promote a normal mutation into an atomic operation, by
> > substituting one for the other, but if so that objective could be
> > reimplemented using their new locking manager.
> >
> >
> Thanks Andrew for starting the discussion.
> Up in JIRA we've also talked of a 'wonky' case where CPs need write-access
> to the internal Metrics system so a CP on by-pass is able to increment
> standard counters. For example, a CP might bypass a Get operation instead
> returning a Cell from a CP-managed cache. In this latter case, it might
> want to increment the Get metrics counters so the by-pass shows in the
> general Get counts.
> This is problematic but there is at least an instance happening downstream.
> We'd like to keep internal metrics internal. CPs should be able to publish
> their own metrics.
> Please speak up if you depend on by-pass.
> S
> > --
> > Best regards,
> > Andrew
> >

Best regards,

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message