drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dsbos <...@git.apache.org>
Subject [GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...
Date Tue, 03 Nov 2015 21:23:11 GMT
Github user dsbos commented on a diff in the pull request:

    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
    @@ -93,13 +93,15 @@ public void ensureAtLeastOneField(ComplexWriter writer) {
         if (!atLeastOneWrite) {
           // if we had no columns, create one empty one so we can return some data for count
           SchemaPath sp = columns.get(0);
    -      PathSegment root = sp.getRootSegment();
    +      PathSegment fieldPath = sp.getRootSegment();
           BaseWriter.MapWriter fieldWriter = writer.rootAsMap();
    -      while (root.getChild() != null && !root.getChild().isArray()) {
    -        fieldWriter = fieldWriter.map(root.getNameSegment().getPath());
    -        root = root.getChild();
    +      while (fieldPath.getChild() != null && ! fieldPath.getChild().isArray())
    +        fieldWriter = fieldWriter.map(fieldPath.getNameSegment().getPath());
    +        fieldPath = fieldPath.getChild();
    +      }
    +      if (fieldWriter.isEmptyMap()) {
    --- End diff --
    Recall that ensureAtLeastOneField used to create that dummy NullableIntVector field/column
(so that the schema didn't end up with zero fields/columns) _regardless_ of whether it needed
to (i.e., even if the schema already had a column or columns from a previous JsonRecordReader
for the same ScanBatch).  
    Under the failure conditions, that blind vector setting frequently replaced a NullableVarCharVector
with that NullableIntVector, causing either explicit "schema change not supported" user-targeted
errors or internal errors about the mismatch between the NullableIntVector and the expectation
of a NullableVarCharVector.
    The additional check prevents ensureAtLeastOneField from overriding a column in an existing
non-empty schema and causing those schema changes.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message