crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Friedrich (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-60) Splitting the core crunch module
Date Wed, 12 Sep 2012 16:58:07 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthias Friedrich updated CRUNCH-60:
-------------------------------------

    Attachment: refactor-1.png
                refactor-0.png

The patch helped a little to simplify dependencies of the crunch module. The attached files
shows the situation before and after the patch. Red lines are dependencies to cut (according
to the tool), number annotations indicate the number of references (import statements).
                
> Splitting the core crunch module
> --------------------------------
>
>                 Key: CRUNCH-60
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-60
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli
>         Attachments: CRUNCH-60-Create-crunch-api-package-mafr-v1.patch, refactor-0.png,
refactor-1.png
>
>
> It looks like the api is interspersed with the implementation details and libraries/utils
a bit. How about:
>  - An api module which only has the APIs that users need to code against
>   -- Most of org.apache.crunch
>   --  org.apache.crunch.types.*
>  - A common/lib module
>   -- package org.apache.crunch.fn
>   -- some stuff like MapFn, FilterFn from org.apache.crunch package
>   -- All of org.apache.crunch.lib.* that is not included in the other modules above and
below
>   -- org.apache.crunch.util
>   -- org.apache.crunch.tool
>  - A crunch-impl module where the rest of it resides.
>   -- All of *impl* packages
>   -- org.apache.crunch.hadoop.mapreduce.lib.jobcontrol
>   -- org.apache.crunch.hadoop.mapreduce.lib.output
>   -- org.apache.crunch.materialize?
> Also move org.apache.crunch.test to src/test/java.
> Need help on placing org.apache.crunch.io.* correctly.
> Note that despite all this, if necessary, we can choose to have a single artifact (jars
etc) to avoid users the onus of importing multiple modules.
> Thoughts?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message