hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <enis....@gmail.com>
Subject Re: ThrottlingException should be handled by client? (was Re: [VOTE] First release candidate for HBase 1.1.0 (RC0) is available.)
Date Tue, 12 May 2015 01:12:21 GMT
We are dealing with throttling / backpressure for mem space and
backpressure transparently in the client which is similar to throttling via
quotas. That is why I was thinking that we should be handling them the same
in the client layer and not in the application layer.

When memstore is full, and cannot flush, we throw RegionTooBusyException
which then causes sleep + retry. Similarly, the
ExponentialClientBackoffPolicy calculates how long to wait etc.

It is a matter of deciding between whether the throttling should be exposed
to the application via fast fail, or we should handle the sleeping in the
client which I expect is the common case. What is the application to do
when it gets the throttle exception? It seems like it should sleep + retry.
For example an MR job can only do that. Ideally we should auto handle this
in the client layer, but give the option to inspect and listen to the
throttle exceptions and fail fast if it needs to.

Enis

On Mon, May 11, 2015 at 4:54 PM, Jesse Yates <jesse.k.yates@gmail.com>
wrote:

> It seems like it should be handled by the user - it implies that the space
> for which the user has been allocated has been maxed out. There doesn't
> seem to be anything HBase (server or client) can do at this point to manage
> the failure, unless we also add support for 'out of space management'.
>
> On Mon, May 11, 2015 at 4:49 PM Matteo Bertozzi <theo.bertozzi@gmail.com>
> wrote:
>
> > handled by the client in what way? what is your suggestion?
> thread.sleep()?
> > the choice to raise the exception up to the client was to have them deal
> > with it in a app-specific way.
> >
> >
> > Matteo
> >
> >
> > On Mon, May 11, 2015 at 4:43 PM, Nick Dimiduk <ndimiduk@apache.org>
> wrote:
> >
> > > On Mon, May 11, 2015 at 12:16 PM, Enis Söztutar <enis@apache.org>
> wrote:
> > >
> > > > >
> > > > >  - a resolution to the RegionScanner interface change, if deemed
> > > > necessary
> > > > >
> > > >
> > > > Sorry, I confused RegionScanner with ResultScanner. RegionScanner is
> > not
> > > a
> > > > Public class, only for co-processors. I think we do not need a fix
> > here.
> > > >
> > > > On a totally unrelated note, I was going over the ThrottlingException
> > for
> > > > HBASE-13661, and noticed that it extends DoNotRetryIOException.
> Looking
> > > at
> > > > the unit tests as well, it means that if the throttle is exceeded, it
> > > > bubbles up to the client level as a RetriedExhaustedException, and
> the
> > > > application side has to do explicit handling because of this. I was
> > > > intuitively expecting the hbase-client level to handle the throttling
> > > > instead of raising this as an application-level exception. Is this
> the
> > > > expected behavior? The exception contains enough details on the
> > > throttling
> > > > that it seems it can do the wait, but seems strange to delegate that
> to
> > > the
> > > > application instead of handling it at the retry layer. Did we chose
> > this
> > > > because of fast fail semantics? Sorry I missed the reviews.
> > > >
> > > > This semantics is important for the RC I think, since it is the first
> > > time
> > > > we are introducing it. Just wanted to confirm that it is an explicit
> > > > decision.
> > > >
> > >
> > > I'd like to raise the visibility of this question as its answer may
> > impact
> > > the timeliness of the next RC. From the ThrottlingException class
> > comment,
> > > I see
> > >
> > >  * TODO: At some point this will be handled on the client side to
> prevent
> > >  * operation to go on the server if the waitInterval is grater than the
> > one
> > > got
> > >  * as result of this exception.
> > >
> > > Please advise.
> > >
> > > Thanks,
> > > Nick
> > >
> > > >  - corrected docs and site build
> > > > >
> > > > > Since this RC has already been open for 12 days and that RC would
> > > contain
> > > > > an extremely limited set of changes above RC0, I would like to run
> it
> > > > > through an abbreviated voting window -- say 48 hours.
> > > > >
> > > > > On Wed, Apr 29, 2015 at 10:35 PM, Nick Dimiduk <
> ndimiduk@apache.org>
> > > > > wrote:
> > > > >
> > > > > > I'm happy to announce the first release candidate of HBase 1.1.0
> > > > > > (HBase-1.1.0RC0) is available for download at
> > > > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.0.1RC2/
> > > > > >
> > > > > > Maven artifacts are also available in the staging repository
> > > > > >
> > > https://repository.apache.org/content/repositories/orgapachehbase-1076
> > > > > >
> > > > > > Artifacts are signed with my code signing subkey
> > 0xAD9039071C3489BD,
> > > > > > available in the Apache keys directory
> > > > > > https://people.apache.org/keys/committer/ndimiduk.asc and in
> > > > > > http://people.apache.org/~ndimiduk/KEY
> > > > > >
> > > > > > There's also a signed tag for this release at
> > > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=2c102dbe56116ca342abd08e906d70d900048a55
> > > > > >
> > > > > > HBase 1.1.0 is the first minor release in the HBase 1.x line,
> > > > continuing
> > > > > > on the theme of bringing a stable, reliable database to the
> Hadoop
> > > and
> > > > > > NoSQL communities. This release includes nearly 200 resolved
> issues
> > > > above
> > > > > > the 1.0.x series to date. Notable features include:
> > > > > >
> > > > > >  - Async RPC client (HBASE-12684)
> > > > > >  - Simple RPC throttling (HBASE-11598)
> > > > > >  - Improved compaction controls (HBASE-8329, HBASE-12859)
> > > > > >  - New extension interfaces for coprocessor users, better
> > supporting
> > > > > > projects like Phoenix (HBASE-12972, HBASE-12975)
> > > > > >  - Per-column family flush (HBASE-10201)
> > > > > >  - WAL on SSD (HBASE-12848)
> > > > > >  - BlockCache in Memcached (HBASE-13170)
> > > > > >  - Tons of region replica enhancements around META, WAL, and
bulk
> > > > loading
> > > > > > (HBASE-11574, HBASE-11568, HBASE-11571, HBASE-11567)
> > > > > >
> > > > > > The full list of issues can be found at
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12329043
> > > > > >
> > > > > > Please try out this candidate and vote +/-1 by midnight Pacific
> > time
> > > on
> > > > > > 2015-05-06 as to whether we should release these bits as HBase
> > 1.1.0.
> > > > > >
> > > > > > Thanks,
> > > > > > Nick
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message