kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lk gen <lkge...@gmail.com>
Subject Re: problems in Kafka unit testing trunk
Date Wed, 28 Nov 2018 16:28:17 GMT
ulimit shows
$ ulimit -n
1024

is it too small for Kafka ?




On Wed, Nov 28, 2018 at 6:18 AM Dhruvil Shah <dhruvil@confluent.io> wrote:

> The unit test itself does not seem to use too many files. What is the
> output for `ulimit -n` on your system? Running `lsof` might also be helpful
> to determine how many open files you have while Kafka is not running.
>
> - Dhruvil
>
> On Tue, Nov 27, 2018 at 9:20 AM lk gen <lkgen11@gmail.com> wrote:
>
> > When running ./gradlew test
> > on a centos machine with gradle and java set
> > In the trunk version from today
> >
> > There are errors about too many files open of the form
> > "
> > kafka.admin.DeleteTopicTest > testDeletingPartiallyDeletedTopic FAILED
> >     org.apache.kafka.common.KafkaException: java.io.IOException: Too many
> > open files
> >         at
> > org.apache.kafka.common.network.Selector.<init>(Selector.java:160)
> >         at
> > org.apache.kafka.common.network.Selector.<init>(Selector.java:212)
> >         at
> > org.apache.kafka.common.network.Selector.<init>(Selector.java:225)
> >         at
> >
> >
> kafka.coordinator.transaction.TransactionMarkerChannelManager$.apply(TransactionMarkerChannelManager.scala:66)
> >         at
> >
> >
> kafka.coordinator.transaction.TransactionCoordinator$.apply(TransactionCoordinator.scala:62)
> >         at kafka.server.KafkaServer.startup(KafkaServer.scala:279)
> >         at kafka.utils.TestUtils$.createServer(TestUtils.scala:135)
> >         at
> >
> >
> kafka.admin.DeleteTopicTest.$anonfun$createTestTopicAndCluster$2(DeleteTopicTest.scala:372)
> >         at
> >
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233)
> >         at scala.collection.Iterator.foreach(Iterator.scala:937)
> >         at scala.collection.Iterator.foreach$(Iterator.scala:937)
> >         at scala.collection.AbstractIterator.foreach(Iterator.scala:1425)
> >         at scala.collection.IterableLike.foreach(IterableLike.scala:70)
> >         at scala.collection.IterableLike.foreach$(IterableLike.scala:69)
> >         at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> >         at
> scala.collection.TraversableLike.map(TraversableLike.scala:233)
> >         at
> scala.collection.TraversableLike.map$(TraversableLike.scala:226)
> >         at
> scala.collection.AbstractTraversable.map(Traversable.scala:104)
> >         at
> >
> >
> kafka.admin.DeleteTopicTest.createTestTopicAndCluster(DeleteTopicTest.scala:372)
> >         at
> >
> >
> kafka.admin.DeleteTopicTest.createTestTopicAndCluster(DeleteTopicTest.scala:366)
> >         at
> >
> >
> kafka.admin.DeleteTopicTest.testDeletingPartiallyDeletedTopic(DeleteTopicTest.scala:418)
> >
> >         Caused by:
> >         java.io.IOException: Too many open files
> >             at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
> >             at
> > sun.nio.ch.EPollArrayWrapper.<init>(EPollArrayWrapper.java:130)
> >             at
> > sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:69)
> >             at
> > sun.nio.ch
> > .EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
> >             at java.nio.channels.Selector.open(Selector.java:227)
> >             at
> > org.apache.kafka.common.network.Selector.<init>(Selector.java:158)
> >             ... 20 more
> >
> > "
> >
> > Is the environment I am using for gradle test is invalid ? are there
> > special settings required ?
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message