drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5994) Cannot start web server on a machine with more than 200 cores
Date Fri, 02 Mar 2018 23:30:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384310#comment-16384310
] 

ASF GitHub Bot commented on DRILL-5994:
---------------------------------------

Github user ilooner commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1148#discussion_r171988996
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/WebServer.java
---
    @@ -154,35 +152,31 @@ public void start() throws Exception {
     
         final boolean authEnabled = config.getBoolean(ExecConstants.USER_AUTHENTICATION_ENABLED);
     
    -    port = config.getInt(ExecConstants.HTTP_PORT);
    -    boolean portHunt = config.getBoolean(ExecConstants.HTTP_PORT_HUNT);
    -    int retry = 0;
    -
    -    for (; retry < PORT_HUNT_TRIES; retry++) {
    -      embeddedJetty = new Server(new QueuedThreadPool(config.getInt(ExecConstants.WEB_SERVER_THREAD_POOL_MAX)));
    -      embeddedJetty.setHandler(createServletContextHandler(authEnabled));
    -      embeddedJetty.addConnector(createConnector(port));
    -
    +    int port = config.getInt(ExecConstants.HTTP_PORT);
    +    final boolean portHunt = config.getBoolean(ExecConstants.HTTP_PORT_HUNT);
    +    final int acceptors = config.getInt(ExecConstants.HTTP_JETTY_SERVER_ACCEPTORS);
    +    final int selectors = config.getInt(ExecConstants.HTTP_JETTY_SERVER_SELECTORS);
    +    final QueuedThreadPool threadPool = new QueuedThreadPool(2, 2, 60000);
    +    embeddedJetty = new Server(threadPool);
    +    embeddedJetty.setHandler(createServletContextHandler(authEnabled));
    +    ServerConnector connector = createConnector(port, acceptors, selectors);
    +    threadPool.setMaxThreads(1 + connector.getAcceptors() + connector.getSelectorManager().getSelectorCount());
    +    embeddedJetty.addConnector(connector);
    +    for (int retry = 0; retry < PORT_HUNT_TRIES; retry++) {
    +      connector.setPort(port);
           try {
             embeddedJetty.start();
    +        return;
           } catch (BindException e) {
             if (portHunt) {
    -          int nextPort = port + 1;
    -          logger.info("Failed to start on port {}, trying port {}", port, nextPort);
    -          port = nextPort;
    -          embeddedJetty.stop();
    +          logger.info("Failed to start on port {}, trying port {}", port, ++port, e);
    --- End diff --
    
    I see thx


> Cannot start web server on a machine with more than 200 cores
> -------------------------------------------------------------
>
>                 Key: DRILL-5994
>                 URL: https://issues.apache.org/jira/browse/DRILL-5994
>             Project: Apache Drill
>          Issue Type: Improvement
>    Affects Versions: 1.11.0
>            Reporter: Mitchel Labonte
>            Assignee: Mitchel Labonte
>            Priority: Minor
>              Labels: doc-impacting, ready-to-commit
>             Fix For: 1.13.0
>
>
> If the WebServer is launched on a machine that has more than 200 cores, you get the following
stack trace:
> {noformat}
> Exception in thread "main" org.apache.drill.exec.exception.DrillStartupException: Failure
during initial startup of Drillbit:
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:313)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:289)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> Caused by: java.lang.IllegalStateException: Insufficient max threads in ThreadPool: max=200
< needed=206
> at org.eclipse.jetty.server.Server.doStart(Server.java:321)
> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at org.eclipse.drill.exec.server.rest.WebServer.start(WebServer.java:197)
> at org.eclipse.drill.exec.server.Drillbit.run(Drillbit.java:140)
> at org.eclipse.drill.exec.server.Drillbit.start(Drillbit.java:309)
> ... 2 more
> {noformat}
> The cause of this is that in the WebServer start method, a Server instance is created
with the default constructor, which initializes a QueuedThreadPool with a default maxThreads
value of 200, and there is no way to configure this value.
> *For documentation*
> New config option - drill.exec.web_server.thread_pool_max.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message