drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Barclay (Drill) (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (DRILL-3659) UnionAllRecordBatch infers wrongly from next() IterOutcome values
Date Mon, 28 Sep 2015 23:50:04 GMT

     [ https://issues.apache.org/jira/browse/DRILL-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Barclay (Drill) reassigned DRILL-3659:
---------------------------------------------

    Assignee: Daniel Barclay (Drill)  (was: Sean Hsuan-Yi Chu)

> UnionAllRecordBatch infers wrongly from next() IterOutcome values
> -----------------------------------------------------------------
>
>                 Key: DRILL-3659
>                 URL: https://issues.apache.org/jira/browse/DRILL-3659
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Relational Operators
>            Reporter: Daniel Barclay (Drill)
>            Assignee: Daniel Barclay (Drill)
>             Fix For: 1.4.0
>
>
> When UnionAllRecordBatch uses IterOutcome values returned from the next() method of upstream
batches, it seems to be using those values wrongly (making incorrect inferences about what
they mean).
> In particular, some switch statements seem to check for NONE vs. OK_NEW_SCHEMA in order
to determine whether there are any rows (instead of explicitly checking the number of rows).
 However, OK_NEW_SCHEMA can be returned even when there are zero rows.
> The apparent latent bug in the union code blocks the fix for DRILL-2288 (having ScanBatch
return OK_NEW_SCHEMA for a zero-rows case in which is was wrongly (per the IterOutcome protocol)
returning NONE without first returning OK_NEW_SCHEMA).
> &nbsp;
> For details of IterOutcome values, see the Javadoc documentation of RecordBatch.IterOutcome
(after DRILL-3641 is merged; until then, see https://github.com/apache/drill/pull/113).
> For an environment/code state that exposes the UnionAllRecordBatch problems, see https://github.com/dsbos/incubator-drill/tree/bugs/WORK_2288_etc,
which includes:
> - a test that exposes the DRILL-2288 problem;
> - an enhanced IteratorValidatorBatchIterator, which now detects IterOutcome value sequence
violations; and
> - a fixed (though not-yet-cleaned) version of ScanBatch that fixes the DRILL-2288 problem
and thereby exposes the UnionAllRecordBatch problem (several test methods in each of TestUnionAll
and TestUnionDistinct fail).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message