hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From qiang tian <tian...@gmail.com>
Subject Re: On coprocessor API evolution
Date Sat, 17 May 2014 10:44:47 GMT
My small 2 cents...:-)

Hook/coprocessor is useful mechanism to interacting with a system for
things that cannot be done via API.  For end user, the tradeoff  factors
like performance, security, reliability etc can be control by upper layer'
policy.
e.g. In RDBMS, the end user has limited usage case for triggers, which
eliminates the security factor at all, and the performance tradeoff is
given to end user to decide. so from evolution's perspective,
hook/coprocessor for end user could be controlled by query engine layer
like Phoenix.

For internal user, hook better not be used widely unless it is a MUST or
strong flexibility/plugability is required.  e.g. things can be part of the
core better not use it.

thanks.



On Sat, May 17, 2014 at 4:04 PM, Michael Segel <michael_segel@hotmail.com>wrote:

> Andrew,
>
> Is ‘magical fairy dust’ a reference to some new synthetic drug you take at
> raves?
> But lets get back to reality.
>
>
> Lets try this again; simply put… the coprocessor runs on the same JVM as
> the RS, therefore you have an unacceptable level of risk.
> That inherent risk means that you cannot run HBase with end-user
> coprocessors enabled when you want to have a stable and somewhat secure
> environment.
>
> The simple truth is that you need to decouple the end-user code
> (coprocessor) from the RS.
> Its not a difficult concept to understand, and while reasonable, it would
> mean a major rewrite and work done on co-processors.
>
> Will de-coupling the user-space from the RS remove all risk? No.  And no,
> I’m not suggesting that.
> But its a critical piece to the puzzle.
>
> Its not just security, but also reliability.
>
>
> On May 17, 2014, at 4:43 AM, Andrew Purtell <apurtell@apache.org> wrote:
>
> > Michael,
> >
> > As you know, we have implemented security features with coprocessors
> > precisely because they can be interposed on internal actions to make
> > authoritative decisions in-process. Coprocessors are a way to have
> > composable internal extensions. They don't have and probably never will
> > have magic fairy security dust. We do trust the security coprocessor code
> > because it was developed by the project. That is not the same thing as
> > saying you can have 'security' and execute arbitrary user code in-process
> > as a coprocessor. Just want to clear that up for you.
> >
> >> will want to allow system coprocessors but then write a coprocessor that
> > reject user coprocessors.
> >
> > That's a reasonable point.
> >
> >
> >
> >
> > On Sat, May 17, 2014 at 12:13 AM, Michael Segel
> > <michael_segel@hotmail.com>wrote:
> >
> >> Until you move the coprocessor out of the RS space and into its own
> >> sandbox… saying security and coprocessor in the same sentence is a joke.
> >> Oh wait… you were serious… :-(
> >>
> >> I’d say there’s a significant rethink on coprocessors that’s required.
> >>
> >> Anyone running a secure (kerberos) cluster, will want to allow system
> >> coprocessors but then write a coprocessor that reject user coprocessors.
> >>
> >> Just putting it out there…
> >>
> >> On May 15, 2014, at 2:13 AM, Andrew Purtell <apurtell@apache.org>
> wrote:
> >>
> >>> Because coprocessor APIs are so tightly bound with internals, if we
> apply
> >>> suggested rules like as mentioned on HBASE-11054:
> >>>
> >>>     I'd say policy should be no changes to method apis across minor
> >>> versions
> >>>
> >>> This will lock coprocessor based components to the limitations of the
> API
> >>> as we encounter them. Core code does not suffer this limitation, we are
> >>> otherwise free to refactor and change internal methods. For example, if
> >> we
> >>> apply this policy to the 0.98 branch, then we will have to abandon
> >> further
> >>> security feature development there and move to trunk only. This is
> >> because
> >>> we already are aware that coprocessor APIs as they stand are
> insufficient
> >>> still.
> >>>
> >>> Coprocessor APIs are a special class of internal method. We have had a
> >>> tension between allowing freedom of movement for developing them out
> and
> >>> providing some measure of stability for implementors for a while.
> >>>
> >>> It is my belief that the way forward is something like HBASE-11125.
> >> Perhaps
> >>> we can take this discussion to that JIRA and have this long overdue
> >>> conversation.
> >>>
> >>> Regarding security features specifically, I would also like to call
> your
> >>> attention to HBASE-11127. I think security has been an optional feature
> >>> long enough, it is becoming a core requirement for the project, so
> should
> >>> be moved into core. Sure, we can therefore sidestep any issues with
> >>> coprocessor API sufficiency for hosting security features. However, in
> my
> >>> opinion we should pursue both HBASE-11125 and HBASE-11127; the first to
> >>> provide the relative stability long asked for by coprocessor API users,
> >> the
> >>> latter to cleanly solve emerging issues with concurrency and
> versioning.
> >>>
> >>>
> >>> --
> >>> Best regards,
> >>>
> >>>  - Andy
> >>>
> >>> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >>> (via Tom White)
> >>
> >>
> >
> >
> > --
> > Best regards,
> >
> >   - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message