hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: On coprocessor API evolution
Date Wed, 21 May 2014 17:50:03 GMT
On Wed, May 21, 2014 at 5:16 AM, Michael Segel <michael_segel@hotmail.com>wrote:

> And they accuse me of raising a straw man. ;-)

I wasn't arguing for or against coprocessors being out of process. I think
both sides of this "argument" are in fact in agreement: if you think of
coprocessors as a place to run *user* code, they need to run out of
process, or otherwise be sandboxed (eg by having a stripped down DSL, as
many SQL DBs do with procedural SQL variants). Another option might be to
use embeddable Javascript, which is fairly doable in the context of the JVM.

But today, coprocessors are not a place to run arbitrary user code. I doubt
you can find examples where we suggest non-sophisticated users to drop in
arbitrary code into their clusters and expect full stability. The analogy
I've always used is Linux kernel modules: you can extend the kernel in all
sorts of fun ways, but you have to trust whoever gives you the code to run,
and a bad module can kill your whole system. Similar to kernel modules, I
expect only sanctioned "vendors" (eg other open source projects like
Phoenix) or highly sophisticated users to ever drop in a CP.

> Todd, really? A parent/child relationship can be secured… how depends on
> how you communicate.
> You could always encrypt the data… in the messaging… ;-)

I think you are confusing confidentiality and security, and not sure you
read the article I linked to. Shared memory does not imply better
performance for all applications, exactly because of the issue I linked to.
If you plan to map the shared memory, and you want to treat that memory
like some kind of structure (eg in which there are length prefixes,
pointers, etc), you must either trust your peer or you must copy the data
before validating it. So, it's no magic bullet for faster sharing of data
between a CP and HBase.

> On May 19, 2014, at 11:37 PM, Todd Lipcon <todd@cloudera.com> wrote:
> > On Mon, May 19, 2014 at 12:05 PM, Vladimir Rodionov
> > <vladrodionov@gmail.com>wrote:
> >
> >> Michael S.
> >>>> To the best of my knowledge,  MapR’s M7 doesn’t have coprocessors.
> I’ll
> >> wager that when they do, it will work and not have these issues. I
> believe
> >> that they are writing their stuff in C/C++, if so, then they’d have an
> >> advantage of using shared memory.  Apache would have write C/C++ code
> and
> >> wrap it in JNI… which you may not want to do…<<
> >>
> >> MapR M7 does not support coprocessors and custom Filters as well. I
> >> consider this to be a serious limitation of the product.
> >> Shared memory communication can be done in Java w/o single line of C/C++
> >> code, Michael by means of memory-mapped files.
> >>
> >> -Vladimir
> >>
> >>
> > And even in native code, shared memory for communication between
> untrusted
> > peers can be pretty tricky to do securely (read
> > http://lwn.net/Articles/593918/ for details)
> >
> > -Todd
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera

Todd Lipcon
Software Engineer, Cloudera

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message