crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Created] (CRUNCH-331) Change default settings for CombineFileInputFormat
Date Thu, 23 Jan 2014 20:53:38 GMT
Josh Wills created CRUNCH-331:

             Summary: Change default settings for CombineFileInputFormat
                 Key: CRUNCH-331
             Project: Crunch
          Issue Type: Bug
          Components: IO
    Affects Versions: 0.8.2, 0.9.0
            Reporter: Josh Wills

Currently, we default to enabling the CombineFileInputFormat settings for any extensions of
FileSourceImpl b/c it tends to improve performance for common file formats like text, sequence
files, and Avro files. However, this default has caused problems for formats like Parquet
and for custom file formats that have complex split logic.

This JIRA is to track modifying the default combine file settings in at least some contexts,
such as with From.formattedFile for custom input formats.

This message was sent by Atlassian JIRA

View raw message