drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5066) External sort attempts to retry sv2 memory alloc, even if can never succeed
Date Wed, 23 Nov 2016 23:51:59 GMT
Paul Rogers created DRILL-5066:
----------------------------------

             Summary: External sort attempts to retry sv2 memory alloc, even if can never
succeed
                 Key: DRILL-5066
                 URL: https://issues.apache.org/jira/browse/DRILL-5066
             Project: Apache Drill
          Issue Type: Bug
    Affects Versions: 1.8.0
            Reporter: Paul Rogers
            Assignee: Paul Rogers
            Priority: Minor


The external sort contains rather complex code to allocate an sv2 in the method {{newSV2()}}.
The code tries to allocate an sv2. If the allocation fails, the code attempts to spill (which
is fine) and try again. If things still fail, the code waits 1 sec. and tries again. It will
continue to wait up to a minute, doubling the wait time each cycle.

Presumably, this is so that some other part of Drill will release memory. But, because of
the way the allocator currently works, the allocation is limited by the limit set on the external
sort's own allocator. This limit won't change by waiting.

The loop only makes sense if the memory allocation failed because the external sort's allocator
is not above its limit, but the parent can't provide memory.

In practice, this scenario should not occur once the external sort is resource managed, so
the retry code can simply be removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message