drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (DRILL-5026) ExternalSortBatch uses two memory allocators; one will do
Date Fri, 17 Feb 2017 22:23:44 GMT

     [ https://issues.apache.org/jira/browse/DRILL-5026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Paul Rogers resolved DRILL-5026.
--------------------------------
    Resolution: Fixed

> ExternalSortBatch uses two memory allocators; one will do
> ---------------------------------------------------------
>
>                 Key: DRILL-5026
>                 URL: https://issues.apache.org/jira/browse/DRILL-5026
>             Project: Apache Drill
>          Issue Type: Sub-task
>    Affects Versions: 1.8.0
>            Reporter: Paul Rogers
>            Assignee: Paul Rogers
>            Priority: Minor
>
> The {{ExternalSortBatch}} (ESB) operator performs a sort while spilling to disk to stay
within a defined memory budget.
> ESB uses two memory allocators:
> * The operator allocator for most operations,
> * The copier allocator (child of the operator allocator) for merge operations.
> The copier allocator is used only when merging prior to spill.
> However, since the ESB must manage within a single memory budget, the code is made more
complex by using two allocators, and keeping track of the memory used in both.
> Better to use a single allocator. When checking for the need to spill, the code already
checks if enough memory is available for an in-memory sort, if needed. Similarly, the code
should also check if sufficient memory is available for the copy step when needed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message