drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-1601) Ensure a minimum allocation for variable length vectors and improve error messages
Date Fri, 07 Nov 2014 15:17:33 GMT

     [ https://issues.apache.org/jira/browse/DRILL-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Aman Sinha updated DRILL-1601:
    Assignee: DrillCommitter  (was: Aman Sinha)

> Ensure a minimum allocation for variable length vectors and improve error messages
> ----------------------------------------------------------------------------------
>                 Key: DRILL-1601
>                 URL: https://issues.apache.org/jira/browse/DRILL-1601
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>    Affects Versions: 0.6.0
>            Reporter: Aman Sinha
>            Assignee: DrillCommitter
>             Fix For: 0.7.0
>         Attachments: 0001-DRILL-1601-Have-a-minimum-allocation-for-variable-le.patch
> The allocateNew() and allocateNewSafe() methods for value vectors compute the estimated
allocation size based on previous requests.   There could be situations where the allocator
determines a very small allocation size based on previous pattern (e.g a bunch of null values)
and then encounters a large enough record size.  In such cases, we see the following types
of exception: 
>   The Partitioner sometimes encounters the following exception:
> java.io.IOException: Record {} is too big to fit into the allocated memory of ValueVector.
> This issue is mainly applicable for variable length vectors since it uses bytes for the
estimation whereas the fixed length vectors use the number of values for estimation. 
> A second part of this JIRA is to improve error messages in the Partitioner and UnlimitedRawBatchBuffer.


This message was sent by Atlassian JIRA

View raw message