crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-219) Support multiple paths in Avro source
Date Mon, 17 Jun 2013 14:21:21 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13685602#comment-13685602
] 

Josh Wills commented on CRUNCH-219:
-----------------------------------

There are some dependencies in MaterializableIterable (which calls the getPath method on FileSourceImpl)
that would need to change, which would propagate downstream to the MapsideJoinStrategy, BloomFilterJoinStrategy,
and Sort classes. We could change MaterializableIterable to use a getPaths() method and mark
the old getPath() method as deprecated and then update the downstream methods to use it. I
think that's all very do-able.
                
> Support multiple paths in Avro source
> -------------------------------------
>
>                 Key: CRUNCH-219
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-219
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Tom White
>            Assignee: Josh Wills
>         Attachments: CRUNCH-219.patch
>
>
> It would be useful to be able to specify multiple paths (which may be files, or directories,
or a combination of both) to read from in a source.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message