crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-228) FileTargetImpl cuts off extensions of output files
Date Fri, 28 Jun 2013 23:32:20 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Wills updated CRUNCH-228:
------------------------------

    Attachment: CRUNCH-228.patch

Hey Dave-- I think it's fine to edit the test and not the trevni output stuff, based on my
read of how this stuff works. Here's a patch that does that, but I'm agnostic w/respect to
which one you commit. +1.

                
> FileTargetImpl cuts off extensions of output files
> --------------------------------------------------
>
>                 Key: CRUNCH-228
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-228
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Dave Beech
>         Attachments: CRUNCH-228.patch, CRUNCH-228.patch
>
>
> Compressed files written by mapreduce often have extensions, e.g. '.deflate', '.gz' or
'.snappy'. Crunch currently cuts off these extensions during the move of output files to their
final destination, which is fine in some circumstances but causes problems in others. 
> For example, running 'hadoop fs -text myfile.deflate' will show the decompressed text
on screen but running 'hadoop fs -text myfile' on a deflate-compressed file with no extension
prints unreadable compressed data instead. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message