hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Baranau <alex.barano...@gmail.com>
Subject Re: Retry HTable.put() on client-side to handle temp connectivity problem
Date Mon, 27 Jun 2011 20:33:28 GMT
Yes, that is what intended, I think. To make the whole picture clear, here's
the context:

* there's a Flume's HBase sink (read: HBase client) which writes data from
Flume "pipe" (read: some event-based messages source) to HTable;
* when HBase is down for some time (with default HBase configuration on
Flume's sink side) HTable.put throws exception and client exits (it usually
takes ~10 min to fail);
* Flume is smart enough to accumulate data to be written reliably if sink
behaves badly (not writing for some time, pauses, etc.), so it would be
great if the sink tries to write data until HBase is up again, BUT:
* but here, as we have complete "failure" of sink process (thread needs to
be restarted) the data never reaches HTable even after HBase cluster is
brought up again.

So you suggest instead of this extra construction around HTable.put to use
configuration properties "hbase.client.pause" and
"hbase.client.retries.number"? I.e. make retries attempts to be (reasonably)
close to "perform forever". Is that what you meant?

Thank you,
Alex Baranau
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase

On Mon, Jun 27, 2011 at 11:16 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> This would retry indefinitely, right ?
> Normally maximum retry duration would govern how long the retry is
> attempted.
>
> On Mon, Jun 27, 2011 at 1:08 PM, Alex Baranau <alex.baranov.v@gmail.com
> >wrote:
>
> > Hello,
> >
> > Just wanted to confirm that I'm doing things in a proper way here. How
> > about
> > this code to handle the temp cluster connectivity problems (or cluster
> down
> > time) on client-side?
> >
> > +    // HTable.put() will fail with exception if connection to cluster is
> > temporarily broken or
> > +    // cluster is temporarily down. To be sure data is written we retry
> > writing.
> > +    boolean dataWritten = false;
> > +    do {
> > +      try {
> > +        table.put(p);
> > +        dataWritten = true;
> > +      } catch (IOException ioe) { // indicates cluster connectivity
> > problem
> > (also thrown when cluster is down)
> > +        LOG.error("Writing data to HBase failed, will try again in " +
> > RETRY_INTERVAL_ON_WRITE_FAIL + " sec", ioe);
> > +        Thread.currentThread().wait(RETRY_INTERVAL_ON_WRITE_FAIL *
> 1000);
> > +      }
> > +    } while (!dataWritten);
> >
> > Thank you in advance,
> > Alex Baranau
> > ----
> > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop -
> HBase
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message