hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steen Manniche <boxun...@gmail.com>
Subject Re: On signalling clients from coprocessors
Date Tue, 31 Jan 2017 12:00:16 GMT
Hi Gary,

I did some further testing of this in our docker setup, and to my
surprise, I'm getting a
where I expected my custom exception (subclass of
CoprocessorException) to appear. For illustration, I have the
following in my coprocessor:

if(failureKeys.contains(sourceColumn)) {
    String err = String.format("The key to write an entry in %s was
empty! It came from %s={%s:%s}", destinationTable, new
String(sourceKey), "e", sourceColumn);
    throw new MissingIndexKeyException(err);

Unfortunately I get a long timeout followed by the
RetriesExhaustedWithDetailsException. If I do a try...catch in the
client sending this put, I would expect that it didn't just continue
retrying the Put? Is there something I need to signal to the
regionserver in order for it to stop the processing of the Put and
just return to the client?

Thanks again for your time.

Best regards,

On Sun, Jan 29, 2017 at 4:42 PM, Steen Manniche <boxunbox@gmail.com> wrote:
> Hi Gary,
> thanks for taking the time to answer this. Indeed, from my tests, it seems
> that throwing a CoprocessorException type is carried through to the client
> and will not cause the HBase RegionServer to crash in the process. This is
> great news, and will allow us to implement the application logic in a
> coprocessor instead of outside the HBase service.
> I can only wonder why this isn't documented better both in the API docs
> (which says nothing at all) as well as in the HBase book - not one mention
> of the semantics of the CoprocessorException type. Would it be good to open
> an issue on Jira regarding this? Or am I just overlooking where the
> CoprocessorException is described?
> On Fri, Jan 27, 2017 at 6:31 PM, Gary Helmling <ghelmling@gmail.com> wrote:
>> Did you try throwing CoprocessorException or making your custom exception
>> a
>> subclass of it?  These should be carried through to the client.
>> Yes, for exceptions outside of this hierarchy, there is no way to know if
>> the exception is recoverable or not, so the safe route is chosen and
>> either
>> the regionserver will abort or the coprocessor will be unloaded, depending
>> on configuration.
>> On Fri, Jan 27, 2017 at 12:55 AM Steen Manniche <boxunbox@gmail.com>
>> wrote:
>> > We have been using coprocessors to create secondary indexes as well as
>> > doing some sanity checks on incoming data.
>> >
>> > Especially in the last case, we planned to use the prePut event to
>> > reject
>> > Puts to hbase that did not pass our sanity checks for incoming data. I
>> > had
>> > expected being able to signal the client ingesting data into hbase via
>> > throwing a custom exception, but learned that the regionserver the
>> > coprocessor is installed on basically only have two options in the face
>> > of
>> > a coprocessor throwing an exception: unloading the coprocessor or
>> > shutting
>> > down.
>> >
>> > This behaviour surprises me since coprocessors almost by definition
>> > contain
>> > application logic, and exceptions are a standard way of signalling in
>> > application logic. Are there any plans to make the signal handling work
>> > better wrt. clients in coprocessors? Or have I simply misunderstood the
>> > intention of coprocessors here?
>> >
>> > Thanks in advance and best regards,
>> > Steen Manniche
>> >

View raw message