hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-960) Using Hadoop's optimized LineRecordReader for reading Tuples in PigStorage
Date Tue, 29 Sep 2009 03:19:16 GMT

    [ https://issues.apache.org/jira/browse/PIG-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760484#action_12760484
] 

Hadoop QA commented on PIG-960:
-------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12420748/pig_rlr.patch
  against trunk revision 819691.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 6 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    -1 javac.  The applied patch generated 406 javac compiler warnings (more than the trunk's
current 403 warnings).

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/48/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/48/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/48/console

This message is automatically generated.

> Using Hadoop's optimized LineRecordReader for reading Tuples in PigStorage 
> ---------------------------------------------------------------------------
>
>                 Key: PIG-960
>                 URL: https://issues.apache.org/jira/browse/PIG-960
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>            Reporter: Ankit Modi
>         Attachments: pig_rlr.patch
>
>
> PigStorage's reading of Tuples ( lines ) can be optimized using Hadoop's {{LineRecordReader}}.
> This can help in following areas
> - Improving performance reading of Tuples (lines) in {{PigStorage}}
> - Any future improvements in line reading done in Hadoop's {{LineRecordReader}} is automatically
carried over to Pig
> Issues that are handled by this patch
> - BZip uses internal buffers and positioning for determining the number of bytes read.
Hence buffering done by {{LineRecordReader}} has to be turned off
> - Current implementation of {{LocalSeekableInputStream}} does not implement {{available}}
method. This method has to be implemented.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message