hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: max number of open files when testing HBase on the Jenkins server
Date Sun, 27 Nov 2011 17:29:28 GMT
We can observe the difference between Jenkins build setup and PreCommit
build setup here:
https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK/2487/console

Linux hemera 2.6.32-32-server #62-Ubuntu SMP Wed Apr 20 22:07:43 UTC
2011 x86_64 GNU/Linux
[HBase-TRUNK] $ /bin/bash -xe /tmp/hudson4606140560635170338.sh
+ ulimit -a

...

open files                      (-n) 40000

We have many issues to tackle, build failure being one of them.

Trying to reduce number of descriptors to 32k could be a goal for the
future. For the time being, its priority is not high.

Cheers

On Sat, Nov 26, 2011 at 5:01 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:

> I agree. The tests should not need more than 32k open files.
>
>
>
> ----- Original Message -----
> From: Todd Lipcon <todd@cloudera.com>
> To: dev@hbase.apache.org
> Cc:
> Sent: Saturday, November 26, 2011 10:42 AM
> Subject: Re: max number of open files when testing HBase on the Jenkins
> server
>
> I disagree that we should raise the limit. If we're using up 32k
> descriptors in unit tests, it's probably a real leak we should be
> concerned about. Maybe we can add a temporary patch upstream which
> does:
> catch (IOException ioe) {
>   if (ioe.getMessage().contains("Too many open files")) {
>     Runtime.getRuntime().exec("lsof -u " + System.getProperty("user.name
> "));
>   }
> }
> or something like that?
>
> -Todd
>
> On Thu, Nov 24, 2011 at 6:27 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > Mikhail:
> > If you look at
> > https://builds.apache.org/job/PreCommit-HBASE-Build/364/console, you
> would
> > see:
> >
> > max memory size         (kbytes, -m) unlimited
> > open files                      (-n) 32768
> >
> > It is not clear why 32768 was not enough for TestAdmin.
> >
> > Normally we don't regard test failure resulted from 'Too many open files'
> > as real test failure.
> >
> > I agree the effective limit should be raised.
> >
> > Cheers
> >
> > On Thu, Nov 24, 2011 at 5:03 PM, Mikhail Bautin <
> > bautin.mailing.lists@gmail.com> wrote:
> >
> >> Hello,
> >>
> >> I saw this error when testing my patch on Jenkins:
> >>
> >> Caused by: java.io.IOException: Too many open files
> >>        at sun.nio.ch.IOUtil.initPipe(Native Method)
> >>        at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:49)
> >>        at
> >>
> sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18)
> >>        at java.nio.channels.Selector.open(Selector.java:209)
> >>        at
> >>
> org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:42)
> >>        at sun.reflect.GeneratedConstructorAccessor40.newInstance(Unknown
> >> Source)
> >>        at
> >>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >>        at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >>        at java.lang.Class.newInstance0(Class.java:355)
> >>        at java.lang.Class.newInstance(Class.java:308)
> >>        at
> >> org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1737)
> >>        ... 55 more
> >>
> >> (Full log:
> >>
> https://builds.apache.org/job/PreCommit-HBASE-Build/364//testReport/org.apache.hadoop.hbase.client/TestAdmin/testCheckHBaseAvailableClosesConnection/
> >> )
> >>
> >> I am not sure who maintains the Jenkins server, but it would be nice to
> >> increase the maximum number of open files for HBase unit tests.
> >>
> >> Thanks,
> >> --Mikhail
> >>
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message