drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-1601) Ensure a minimum allocation for variable length vectors and improve error messages
Date Wed, 29 Oct 2014 15:49:34 GMT
Aman Sinha created DRILL-1601:
---------------------------------

             Summary: Ensure a minimum allocation for variable length vectors and improve
error messages
                 Key: DRILL-1601
                 URL: https://issues.apache.org/jira/browse/DRILL-1601
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 0.6.0
            Reporter: Aman Sinha
            Assignee: Aman Sinha


The allocateNew() and allocateNewSafe() methods for value vectors compute the estimated allocation
size based on previous requests.   There could be situations where the allocator determines
a very small allocation size based on previous pattern (e.g a bunch of null values) and then
encounters a large enough record size.  In such cases, we see the following types of exception:

  The Partitioner sometimes encounters the following exception:
java.io.IOException: Record {} is too big to fit into the allocated memory of ValueVector.

This issue is mainly applicable for variable length vectors since it uses bytes for the estimation
whereas the fixed length vectors use the number of values for estimation. 

A second part of this JIRA is to improve error messages in the Partitioner and UnlimitedRawBatchBuffer.

        



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message