drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Westin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-2698) Add allocation policy for (direct memory) limits per-query-per-node
Date Mon, 06 Apr 2015 18:53:12 GMT
Chris Westin created DRILL-2698:
-----------------------------------

             Summary: Add allocation policy for (direct memory) limits per-query-per-node
                 Key: DRILL-2698
                 URL: https://issues.apache.org/jira/browse/DRILL-2698
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 0.8.0
            Reporter: Chris Westin
            Assignee: Chris Westin
             Fix For: 1.0.0


We want the allocator to support a policy that limits memory allocation per-query-per-node.
Current policy is to divide the available direct memory by the number of fragments running
on a node, and limit each fragment to that. This new policy would count fragments that belong
to the same query and are running on the same node together. So, if query A has 3 fragments
running on a node, and query B has 5 fragments running on the same node, and there are no
other queries with fragments on that node, then there are two queries running on the node,
and their allocation limit is the available direct memory divided by two. (Under the original
policy, which divides the available memory by the number of fragments, A would be allowed
3/8 of the memory, and B would be allowed 5/8 of the memory.)

Note that this policy should be a user-settable boot-time option; the original policy would
still also be available as an option.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message