hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: HLog's AsyncHLogWriter aborted abruptly
Date Tue, 22 Sep 2015 19:56:35 GMT
On Tue, Sep 22, 2015 at 12:16 PM, Shrijeet <shrijeet@pinterest.com.invalid>
wrote:

> Thanks Stack. Few comments.
>
>
> > Have you put much work into it? If you went to HBASE-10156, write
> > throughput is better again and you'd be on same system as is in current
> > hbase-1.x so would be easier to help you.
> >
>
> Good to know, 1.x move is planned soonish so will wait for it.
>
> <snip>
>
> > > Since AsyncWriter got aborted due to an unhanded exception the
> > AsyncSyncer
> > > etc. will not be notified, thus deadlocking the server. I am not
> familiar
> > > with HLog's life cycle, how does it handles errors in its worker
> threads?
> >
> >
> > My guess is that they are not supposed to have exited... the exception is
> > unexpected. They are not supposed to die in normal operation.
> >
>
> Thats true - in case of a checked exception (IOE in this case) - HLog's
> async mechanism marks an append as failed - goes ahead with sync,  moves on
> & never dies. And I think what you are saying is it might be okay to do
> same for this specific (AIOBException raised by HDFS) error too. Fixing
> HDFS in not an option right now - so I might have to put a band aid in
> HLog. Let me know if I misunderstood.
>

AIOBE out of HDFS is not expected (smile). Yeah, catch and carry on.
Catching all Throwables might make for new headache types.



>
> <snip>
>
>  In the later versions of the WAL subsystem, there is
> > careful handling to make sure we don't do things like fail an append but
> > successfully sync.
> >
>
> By later version you mean 1.x and beyond?
>


Yeah and beyond. See HBASE-14317

St.Ack



>
> Thanks again!
>
>
> > St.Ack
> >
> >
> >
> >
> > >
> > > // 4. update 'lastWrittenTxid' and notify AsyncSyncer to do 'sync'
> > > asyncSyncer.setWrittenTxid(this.lastWrittenTxid);
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message