crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-311) Expose injection of FileNamingScheme into AvroPathPerKeyTarget
Date Thu, 12 Dec 2013 04:07:07 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846050#comment-13846050
] 

Micah Whitacre commented on CRUNCH-311:
---------------------------------------

That's what I get for putting out the patch prior to writing the tests and seeing that it
doesn't work.  :(

I didn't know we were using this functionality until the end of today but essentially we are
going to be dropping files multiple times into a directory and were using the FileNamingScheme
to prefix the files with a timestamp marker of when they were generated.  

I don't think this should hold up the release.  I think we should be able to work around this
by changing the prefix to be a directory and using a regex in the paths instead.

> Expose injection of FileNamingScheme into AvroPathPerKeyTarget
> --------------------------------------------------------------
>
>                 Key: CRUNCH-311
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-311
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.9.0
>            Reporter: Micah Whitacre
>            Assignee: Micah Whitacre
>             Fix For: 0.9.0
>
>         Attachments: CRUNCH-311.patch
>
>
> Similar to AvroFileTarget, injection of custom FileNamingScheme's should be supported
on the brand new AvroPathPerKeyTarget.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message