orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ORC-536) "expectedEntries should be > 0" error when bloom filters enabled but rowIndexStride set to 0
Date Tue, 23 Jul 2019 21:49:00 GMT
Jason Dere created ORC-536:
------------------------------

             Summary: "expectedEntries should be > 0" error when bloom filters enabled
but rowIndexStride set to 0
                 Key: ORC-536
                 URL: https://issues.apache.org/jira/browse/ORC-536
             Project: ORC
          Issue Type: Bug
          Components: Java
            Reporter: Jason Dere


{noformat}
java.lang.IllegalArgumentException: expectedEntries should be > 0
    at shadeorcorg.apache.orc.util.BloomFilter.checkArgument(BloomFilter.java:54)
    at shadeorcorg.apache.orc.util.BloomFilter.<init>(BloomFilter.java:63)
    at shadeorcorg.apache.orc.util.BloomFilterUtf8.<init>(BloomFilterUtf8.java:30)
    at shadeorcorg.apache.orc.impl.writer.TreeWriterBase.<init>(TreeWriterBase.java:116)
    at shadeorcorg.apache.orc.impl.writer.StringBaseTreeWriter.<init>(StringBaseTreeWriter.java:64)
    at shadeorcorg.apache.orc.impl.writer.StringTreeWriter.<init>(StringTreeWriter.java:33)
    at shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:129)
    at shadeorcorg.apache.orc.impl.writer.StructTreeWriter.<init>(StructTreeWriter.java:41)
    at shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:155)
    at shadeorcorg.apache.orc.impl.writer.StructTreeWriter.<init>(StructTreeWriter.java:41)
    at shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:155)
    at shadeorcorg.apache.orc.impl.WriterImpl.<init>(WriterImpl.java:180)
    at shadehive.org.apache.hadoop.hive.ql.io.orc.WriterImpl.<init>(WriterImpl.java:94)
    at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcFile.createWriter(OrcFile.java:378)
    at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.initWriter(OrcRecordUpdater.java:603)
    at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.addSimpleEvent(OrcRecordUpdater.java:424)
    at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.addSplitUpdateEvent(OrcRecordUpdater.java:433)
    at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.insert(OrcRecordUpdater.java:485)
    at org.apache.hive.streaming.AbstractRecordWriter.write(AbstractRecordWriter.java:439)
    at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.write(HiveStreamingConnection.java:794)
{noformat}

Pretty easy to repro with TestWriterImpl:
{code}
  public void testNoBFIfNoIndex() throws Exception {
    // overriding the flag should result in a successful write (no exception)
    conf.set(OrcConf.OVERWRITE_OUTPUT_FILE.getAttribute(), "true");
    // Enable bloomfilter, but disable index
    conf.set(OrcConf.ROW_INDEX_STRIDE.getAttribute(), "0");
    conf.set(OrcConf.BLOOM_FILTER_COLUMNS.getAttribute(), "*");
    Writer w = OrcFile.createWriter(testFilePath, OrcFile.writerOptions(conf).setSchema(schema));
    w.close();
  }
{code}

cc [~prasanth_j]



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message