hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5269) Bubble exceptions and errors all the way up the calls, including to clients.
Date Fri, 10 Mar 2017 17:54:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905481#comment-15905481
] 

Haibo Chen commented on YARN-5269:
----------------------------------

Having putEntities call putEntitiesAsync and writer.flush() will make sure data is flushed
to HBase when putEntities() returns. It however, may have issues for TimelineClients under
certain circumstances (if these circumstances are possible). The scenario I am think of is,
before a client sends some entities through the synchronous putEntities() call, there are
already some data in the BufferredMutator buffer, so when writer.flush() is called, I'd assume
that the client's data as well as the data that is already in the buffer will be flushed to
HBase together. If somehow HBase backend returns an error because of the data that is already
buffer (i.e., not because of this specific client's data), how are we going to handle this?
Does HBase allow us to tell if the error is caused by client's data or not? One reason I can
think of why clients want to use the synchronous putEntities() call is for critical data.
If clients retry if they see an error/exception is returned just to ensure data is persisted
in HBase, all of a sudden multiple copies of the same data in HBase. My apologies for my imaginary
cases if they are unrealistic.

It seems to me that with all data going though BufferedMutator, putEntitiesAsync + writer.flush()
is probably the best we can provide for putEntitties(). Regardless of whether SpoolingBufferedMutator
can do better or not, I think we should add some javadoc to describe what we can provide.



> Bubble exceptions and errors all the way up the calls, including to clients.
> ----------------------------------------------------------------------------
>
>                 Key: YARN-5269
>                 URL: https://issues.apache.org/jira/browse/YARN-5269
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Joep Rottinghuis
>            Assignee: Haibo Chen
>              Labels: YARN-5355, yarn-5355-merge-blocker
>
> Currently we ignore (swallow) exception from the HBase side in many cases (reads and
writes).
> Also, on the client side, neither TimelineClient#putEntities (the v2 flavor) nor the
#putEntitiesAsync method return any value.
> For the second drop we may want to consider how we properly bubble up exceptions throughout
the write and reader call paths and if we want to return a response in putEntities and some
future kind of result for putEntitiesAsync.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message