incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-60) Splitting the core crunch module
Date Tue, 18 Sep 2012 06:20:07 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-60?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457634#comment-13457634
] 

Vinod Kumar Vavilapalli commented on CRUNCH-60:
-----------------------------------------------

bq. Almost everything ended up in the base package, I don't think there's a way to split it
into subpackages without introducing dependency cycles. 
I've observed this too. There are lots of circular dependencies between the API classes themselves,
we need to untangle these.
                
> Splitting the core crunch module
> --------------------------------
>
>                 Key: CRUNCH-60
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-60
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli
>         Attachments: CRUNCH-60-Create-crunch-api-package-mafr-v1.patch, refactor-0.png,
refactor-1.png
>
>
> It looks like the api is interspersed with the implementation details and libraries/utils
a bit. How about:
>  - An api module which only has the APIs that users need to code against
>   -- Most of org.apache.crunch
>   --  org.apache.crunch.types.*
>  - A common/lib module
>   -- package org.apache.crunch.fn
>   -- some stuff like MapFn, FilterFn from org.apache.crunch package
>   -- All of org.apache.crunch.lib.* that is not included in the other modules above and
below
>   -- org.apache.crunch.util
>   -- org.apache.crunch.tool
>  - A crunch-impl module where the rest of it resides.
>   -- All of *impl* packages
>   -- org.apache.crunch.hadoop.mapreduce.lib.jobcontrol
>   -- org.apache.crunch.hadoop.mapreduce.lib.output
>   -- org.apache.crunch.materialize?
> Also move org.apache.crunch.test to src/test/java.
> Need help on placing org.apache.crunch.io.* correctly.
> Note that despite all this, if necessary, we can choose to have a single artifact (jars
etc) to avoid users the onus of importing multiple modules.
> Thoughts?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message