drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Barclay (Drill) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3659) UnionAllRecordBatch infers wrongly from next() IterOutcome values
Date Mon, 17 Aug 2015 17:55:04 GMT
Daniel Barclay (Drill) created DRILL-3659:

             Summary: UnionAllRecordBatch infers wrongly from next() IterOutcome values
                 Key: DRILL-3659
                 URL: https://issues.apache.org/jira/browse/DRILL-3659
             Project: Apache Drill
          Issue Type: Bug
            Reporter: Daniel Barclay (Drill)

When UnionAllRecordBatch uses IterOutcome values returned from the next() method of upstream
batches, it seems to be using those values wrongly (making incorrect inferences about what
they mean).

In particular, some switch statements seem to check for NONE vs. OK_NEW_SCHEMA in order to
determine whether there are any rows (instead of explicitly checking the number of rows).
 However, OK_NEW_SCHEMA can be returned even when there are zero rows.

The apparent latent bug in the union code blocks the fix for DRILL-2288 (having ScanBatch
return OK_NEW_SCHEMA for a zero-rows case in which is was wrongly (per the IterOutcome protocol)
returning NONE without first returning OK_NEW_SCHEMA).

For details of IterOutcome values, see the Javadoc documentation of RecordBatch.IterOutcome
(after DRILL-3641 is merged; until then, see https://github.com/apache/drill/pull/113).

For an environment/code state that exposes the UnionAllRecordBatch problems, see https://github.com/dsbos/incubator-drill/tree/bugs/WORK_2288_etc,
which includes:
- a test that exposes the DRILL-2288 problem;
- an enhanced IteratorValidatorBatchIterator, which now detects IterOutcome value sequence
violations; and
- a fixed (though not-yet-cleaned) version of ScanBatch that fixes the DRILL-2288 problem
and thereby exposes the UnionAllRecordBatch problem (several test methods in each of TestUnionAll
and TestUnionDistinct fail).

This message was sent by Atlassian JIRA

View raw message