hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hansi Klose" <hansi.kl...@web.de>
Subject Aw: Re: taking snapshot's creates to many TCP CLOSE_WAIT handles on the hbase master server
Date Tue, 22 Apr 2014 10:18:19 GMT
Hi Ted,

I inserted the output at pastebin

http://pastebin.com/n3mMPxBA

At the moment the hbase master process holds 10716 handles.
We stopped making snapshots last week.
After 4 days the count is still the same.

Regards Hansi

> Gesendet: Donnerstag, 17. April 2014 um 19:09 Uhr
> Von: "Ted Yu" <yuzhihong@gmail.com>
> An: "user@hbase.apache.org" <user@hbase.apache.org>
> Betreff: Re: taking snapshot's creates to many TCP CLOSE_WAIT handles on the hbase master
server
>
> Can you take jstack of master process and pastebin it ?
> 
> Thanks
> 
> 
> On Thu, Apr 17, 2014 at 6:51 AM, Hansi Klose <hansi.klose@web.de> wrote:
> 
> > Hi,
> >
> > we use a script to take on a regular basis snapshot's and delete old one's.
> >
> > We recognizes that the web interface of the hbase master was not working
> > any more becaues of too many open files.
> >
> > The master reaches his number of open file limit of 32768
> >
> > When I run lsof I saw that there where a lot of TCP CLOSE_WAIT handles open
> > with the regionserver as target.
> >
> > On the regionserver there is just one connection to the hbase master.
> >
> > I can see that the count of the CLOSE_WAIT handles grow each time
> > i take a snapshot. When i delete on nothing changes.
> > Each time i take a snapshot  there are 20 - 30 new CLOSE_WAIT handles.
> >
> > Why does the master do not close the handles? Is there a parameter
> > with a timeout we can use?
> >
> > We use hbase 0.94.2-cdh4.2.0.
> >
> > Regards Hansi
> >
> 

Mime
View raw message