drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "abdelhakim deneche" <adene...@gmail.com>
Subject Re: Review Request 32273: DRILL-2408: Invalid (0 length) parquet file created by CTAS
Date Mon, 27 Apr 2015 01:57:22 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32273/
-----------------------------------------------------------

(Updated April 27, 2015, 1:57 a.m.)


Review request for drill, Aman Sinha and Jason Altekruse.


Changes
-------

- changed ParquetRecordWriter to create the parquet files as soon as one record is added to
the writer
- moved corresponding test to TestParquetWriterEmptyFiles
- fixed BaseTestQuery to update storage plugin in all drillbits


Bugs: DRILL-2408
    https://issues.apache.org/jira/browse/DRILL-2408


Repository: drill-git


Description
-------

I updated ParquetRecordWriter to delete the last file created if it's empty (no records written
to it). I also added two unit tests one that checks the default case where we try to create
a table using a query that returns 0 rows, the second case can happen if the ParquetRecordWriter
flushes it's content just after writing the last record, it will then create a new empty file.


Diffs (updated)
-----

  exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetRecordWriter.java
3e35721 
  exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java b02051b 
  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestParquetWriter.java
89837e7 
  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestParquetWriterEmptyFiles.java
PRE-CREATION 

Diff: https://reviews.apache.org/r/32273/diff/


Testing
-------

- Added 2 unit tests to TestParquetWriter
- Unit tests are passing


Thanks,

abdelhakim deneche


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message