hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-953) Enable merge join in pig to work with loaders and store functions which can internally index sorted data
Date Thu, 29 Oct 2009 17:29:59 GMT

     [ https://issues.apache.org/jira/browse/PIG-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pradeep Kamath updated PIG-953:
-------------------------------

       Resolution: Fixed
    Fix Version/s: 0.6.0
     Hadoop Flags: [Reviewed]
           Status: Resolved  (was: Patch Available)

I ran the test-patch process and junit tests on my local machine since the hudson queue was
backed up. Here are results - I have explained the reason for the javac warnings and release
audit warnings in my previous comment. 
{noformat}
test-patch results
====================
....
    [exec] -1 overall.
     [exec]
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]
     [exec]     +1 tests included.  The patch appears to include 6 new or modified tests.
     [exec]
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]
     [exec]     -1 javac.  The applied patch generated 200 javac compiler warnings (more than
the trunk's current 197 warnings).
     [exec]
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]
     [exec]     -1 release audit.  The applied patch generated 298 release audit warnings
(more than the trunk's current 291 warnings).
     [exec]
    
core unit test results
======================
...
    [junit] Running org.apache.pig.test.TestUnion
    [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 44.03 sec

test-contrib:

BUILD SUCCESSFUL
{noformat}

Patch committed to trunk

> Enable merge join in pig to work with loaders and store functions which can internally
index sorted data 
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-953
>                 URL: https://issues.apache.org/jira/browse/PIG-953
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.3.0
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>             Fix For: 0.6.0
>
>         Attachments: PIG-953-2.patch, PIG-953-3.patch, PIG-953-4.patch, PIG-953-5.patch,
PIG-953-6.patch, PIG-953-7.patch, PIG-953-8.patch, PIG-953-9.patch, PIG-953.patch
>
>
> Currently merge join implementation in pig includes construction of an index on sorted
data and use of that index to seek into the "right input" to efficiently perform the join
operation. Some loaders (notably the zebra loader) internally implement an index on sorted
data and can perform this seek efficiently using their index. So the use of the index needs
to be abstracted in such a way that when the loader supports indexing, pig uses it (indirectly
through the loader) and does not construct an index. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message