drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5994) Cannot start web server on a machine with more than 200 cores
Date Wed, 10 Jan 2018 21:34:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16321148#comment-16321148
] 

ASF GitHub Bot commented on DRILL-5994:
---------------------------------------

Github user vrozov commented on the issue:

    https://github.com/apache/drill/pull/1069
  
    @MitchelLabonte @arina-ielchiieva I don't think that Drill needs that many threads/acceptors
to handle HTTP(s) requests as it is not a real web (REST API) server. For proper resource
utilization, it will be better to limit the number of acceptors to a small value (let's say
2 or 4 by default) instead of the current default that uses a number of available processors
and can be huge on machines with lots of cores (32 or more). @paul-rogers  What is your take
on this?  


> Cannot start web server on a machine with more than 200 cores
> -------------------------------------------------------------
>
>                 Key: DRILL-5994
>                 URL: https://issues.apache.org/jira/browse/DRILL-5994
>             Project: Apache Drill
>          Issue Type: Improvement
>    Affects Versions: 1.11.0
>            Reporter: Mitchel Labonte
>            Priority: Minor
>              Labels: doc-impacting, ready-to-commit
>             Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you get the following
stack trace:
> {noformat}
> Exception in thread "main" org.apache.drill.exec.exception.DrillStartupException: Failure
during initial startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in ThreadPool: max=200
< needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is created
with the default constructor, which initializes a QueuedThreadPool with a default maxThreads
value of 200, and there is no way to configure this value.
> *For documentation*
> New config option - drill.exec.web_server.thread_pool_max.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message