incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-30) Add cross join to Scrunch PTable and PCollection
Date Sun, 05 Aug 2012 23:58:02 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428938#comment-13428938
] 

Josh Wills commented on CRUNCH-30:
----------------------------------

Thanks for this-- three things I noticed when I ran it locally:

1) Whitespace issues in the CrossJoinTest-- some lines have tabs for some reason.
2) The CrossJoinTest isn't really an integration test-- it runs entirely in-memory, and so
it should live in src/test/scala instead of src/it/scala.
3) I think that the toMap call in the materializeToMap function would trigger a MapReduce
pipeline to run by pulling the data from the materialized instance when it is called-- inserting
a .view call in there, or delegating to the materialize() method in PTable that does that
already, is a better choice.
                
> Add cross join to Scrunch PTable and PCollection
> ------------------------------------------------
>
>                 Key: CRUNCH-30
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-30
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Scrunch
>    Affects Versions: 0.3.0
>            Reporter: Brian Martin
>             Fix For: 0.3.0
>
>         Attachments: 0001-CRUNCH-30-Add-cross-join-to-Scrunch-PTable-and-PColl.patch
>
>   Original Estimate: 0h
>  Remaining Estimate: 0h
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message