drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-2088) BitVector.Mutator.setValueCount() fails with invalid buffer access
Date Thu, 29 Jan 2015 00:11:36 GMT

     [ https://issues.apache.org/jira/browse/DRILL-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Venki Korukanti updated DRILL-2088:
-----------------------------------
    Attachment: 0001-DRILL-2088-ReAlloc-in-FixedWidthVector.setValueCount.patch

Attaching patch to fix the following issues:
1. In BitVector, find the valueCapacity from the buffer size (similar to other fixed width
vectors), not based on the allocation size requested. Currently we store the valueCapacity
in a member variable which is not transferred, causing failures when setValueCount is called
on vector that got the buffer transfer.

2. Change the FixedLengthVector.java template to generate setValueCount that reallocs when
requested valueCount falls beyond the available buffer capacity. This is a problem for nullable
vectors (which internally use fixed width vectors). 

Patch has tests for both changes.

> BitVector.Mutator.setValueCount() fails with invalid buffer access
> ------------------------------------------------------------------
>
>                 Key: DRILL-2088
>                 URL: https://issues.apache.org/jira/browse/DRILL-2088
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Venki Korukanti
>            Assignee: Venki Korukanti
>         Attachments: 0001-DRILL-2088-ReAlloc-in-FixedWidthVector.setValueCount.patch
>
>
> Run the following physical plan:
> {code}
> {
>     head:{
>         type:"APACHE_DRILL_PHYSICAL",
>         version:"1",
>         generator:{
>             type:"manual"
>         }
>     },
>     graph:[
>         {
>             @id:1,
>             pop:"mock-scan",
>             url: "http://apache.org",
>             entries:[
>                 {records: 100, types: [
>                   {name: "blue", type: "BIT", mode: "REQUIRED"},
>                   {name: "red", type: "BIGINT", mode: "REQUIRED"},
>                   {name: "green", type: "INT", mode: "OPTIONAL"}
>                 ]}
>             ]
>         },
>         {
>             @id:2,
>             child: 1,
>             pop:"selection-vector-remover"
>         },
>         {
>             @id: 3,
>             child: 2,
>             pop: "screen"
>         }
>     ]
> }
> {code}
> It fails with:
> {code}
> Failure while running fragment., index: 0, length: 8192 (expected: range(0, 1024))
> {code}
> Couple of issues:
> 1. We don't transfer valueCapacity as part of TransferPair.transfer(). This causes the
vector that got the buffers to have 0 as valueCapacity. During the realloc of vector that
got the buffer, we rely on valueCapacity to decide whether to realloc or not.
> 2. When BitVector.Mutator.setValueCount() is called we try to reallocate the buffer as
the valueCapacity is zero. As part of realloc, we allocate a buffer of size 1028bytes and
try to copy the existing buffer. Problem is existing buffer has size more than 1028bytes and
we fail in copying. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message