arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Grove (Jira)" <j...@apache.org>
Subject [jira] [Created] (ARROW-6697) [Rust] [DataFusion] Validate that all parquet partitions have the same schema
Date Wed, 25 Sep 2019 14:11:00 GMT
Andy Grove created ARROW-6697:
---------------------------------

             Summary: [Rust] [DataFusion] Validate that all parquet partitions have the same
schema
                 Key: ARROW-6697
                 URL: https://issues.apache.org/jira/browse/ARROW-6697
             Project: Apache Arrow
          Issue Type: New Feature
          Components: Rust, Rust - DataFusion
            Reporter: Andy Grove
             Fix For: 1.0.0


When reading a partitioned Parquet file in DataFusion, the schema is read from the first partition
and it is assumed that all other partitions have the same schema.

It would be better to actually validate that all of the partitions have the same schema since
there is no support for schema merging yet.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message