arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe L. Korn (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ARROW-357) Default Parquet chunk_size of 64k is too small
Date Tue, 01 Nov 2016 12:45:58 GMT

    [ https://issues.apache.org/jira/browse/ARROW-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625339#comment-15625339
] 

Uwe L. Korn commented on ARROW-357:
-----------------------------------

PR: https://github.com/apache/arrow/pull/192

> Default Parquet chunk_size of 64k is too small
> ----------------------------------------------
>
>                 Key: ARROW-357
>                 URL: https://issues.apache.org/jira/browse/ARROW-357
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.1.0
>            Reporter: Uwe L. Korn
>            Assignee: Uwe L. Korn
>
> The current default size of RowGroups when writing Parquet files is 64k rows. This produces
a lot of RowGroups given large files which are then really ineffective to read.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message