avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Lewi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-570) python implementation of mapreduce connector
Date Thu, 18 Aug 2011 05:59:27 GMT

    [ https://issues.apache.org/jira/browse/AVRO-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086830#comment-13086830
] 

Jeremy Lewi commented on AVRO-570:
----------------------------------

I've found a possible work around for the maven issue. I can adjust the shade plugin so that
it builds a jar avro-tools-1.6.0-SNAPSHOT-withdeps.jar which is equivalent to the current
output of the shade plugin (i.e the ./tools/target/avro-tools-1.6.0-SNAPSHOT.jar). 

avro-tools-1.6.0-SNAPSHOT-withdeps.jar would contain unpacked versions of all dependency jars.

avro-tools-1.6.0-SNAPSHOT.jar would only contain the code for avro-tools (i.e no dependencies)


avro-tools-1.6.0-SNAPSHOT-job.jar would be a jar suitable for hadoop.

So the only change to existing code would be the jar generated by the shade plugin would be
renamed to avro-tools-1.6.0-SNAPSHOT-withdeps.jar. Not sure what this would break if anything.





> python implementation of mapreduce connector
> --------------------------------------------
>
>                 Key: AVRO-570
>                 URL: https://issues.apache.org/jira/browse/AVRO-570
>             Project: Avro
>          Issue Type: New Feature
>          Components: python
>            Reporter: Doug Cutting
>            Assignee: Jeremy Lewi
>            Priority: Critical
>         Attachments: AVRO-570.patch
>
>
> AVRO-512 defines protocols for implementing mapreduce tasks.  It would be good to have
a Python implementation of this.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message